Is it possible to use reinforcement learning to train AGVs?

[vothan_salomao]

[ FlexSim 23.2.0 ]

Is it possible to use reinforcement learning to train AGVs? I was thinking of using the Advanced AGV template in a network with multiple AGVs circulating, implementing the label of the Tokens for the current CP (Control Point), the last CP, and the AGV's destination in a global table, and creating a Parameter Table with this information for each AGV. This data would be used as observations for machine learning, and the actions would involve modifying the destinations of the AGVs or assigning each AGV to a specific destination. The goal would be to maximize the number of pallet outputs throughout the model, and I believe this approach could help optimize routes and reduce potential deadlocks.



I have read the available documentation on reinforcement learning

[kavika_faleumu]

Hey @Vothan Salomão, I haven't built any models like that, but by the way you've described the problem I think it's solvable. Is there a specific aspect of your issue you're trying to solve or figure out?

[support5CRPZ · Accepted solution]

@Vothan Salomão

I take my client's case study as the example of using AI:RL looking for an optimized AGV route without dynamic traffic. It's not my project but the customer's project by themselves.

Yes. It's flexsim's python AI:RL feature.

So I think you can solve your issues like maxmize outputs through the model using AI:RL.

[jason_lightfootVL7B4]

Hi @Vothan Salomão, was David Seo's answer helpful? If so, please click the "Accept" button at the bottom of their answer. Or if you still have questions, add a comment and we'll continue the conversation.

If we haven't heard back from you within 3 business days we'll auto-accept an answer, but you can always comment back to reopen your question.

[vothan_salomao]

ReinforcementLearning_AGVs2.fsm

I would like to try building a model using reinforcement learning with AGVs, but I'm facing some difficulties.

First, it's related to the Process Flow and the Global Table I've constructed. I'm having trouble exactly capturing the CP and Last CP of the AGVs in the dynamic table I built throughout the route using the Process Flow and the CP and Last CP labels.

Another question concerns the AGVs' Destinations. In the global table, when referencing the Destination Label, the AGV doesn't actually update with the true destination.

The idea behind my model is to train an agent to make the best allocation decisions for the AGVs for Pallets periodically in the model, thus updating the Destination Label for empty AGVs or those on their way to pick up an item.

The agent's observations would be the columns CP, Last CP, and Destination. The actions would involve modifying the Destinations of AGVs that are not carrying items to pick up Pallets periodically. The reward function would aim to maximize the number of Pallets transported over time.

How could I update my Global Table more accurately by obtaining this information from each AGV and not from the Process Flow?

[vothan_salomao]

Hello @David Seo

Could you provide me with an example or guidance for my implementation? What parameters do you consider important for observations, actions, and rewards?

This is my model with the Global Table that I intend to use as an observation.

ReinforcementLearning_AGVs2.fsm

[vothan_salomao]

Hello @Jason Lightfoot I elaborated my questions in a better way after the initial inquiry, thank you

[jason_lightfootVL7B4]

We've explained how to update the table in this post. Do you still need help with this question?

[vothan_salomao]

Hello @Jason Lightfoot , I managed to create a dynamic table with the AGVs collecting information in a Global Table with columns for the next CP, the previous CP, the destination, and the distances of the AGVs. However, I am having trouble creating a Parameters Table with this information for the reinforcement learning observation. I will create another post to clarify my question further because I think there is no way to pass the values from the Global Table to the Parameters Table.