topic Re: Reinforcement learning in FlexSim Forum

Reinforcement learning

Clara1 — Tue, 30 Jan 2024 10:34:40 GMT

[ FlexSim 24.0.1 ]

I am trying to simulate model with AI using reinforcement learning. I am following your tutorial of flexsim 2022 (https://docs.flexsim.com/en/22.1/ModelLogic/ReinforcementLearning/Training/Training.html) . But I am facing issue in python It cant able to fetch observation data. It did not update further value.

ChangeoverTimesRL.fsm

Version 24.0.2

Re: Reinforcement learning

nilP8GEZ — Tue, 30 Jan 2024 13:04:21 GMT

Hello,

I've reviewed your model and it appears that it should be working perfectly. However, it seems there might be a bug causing the issue you're experiencing.

I'm sending you a modified model that resolves this problem. I believe this is a better approach and I highly recommend it. The problem lies in the Decision Events within the Reinforcement Learning tab. We’re going to delete them all (as they seem to be failing) and we’ll call the RL from another place.

The solution involves using ProcessFlow to create a token each time an event triggers, and then calling the RL from a customCode. The default code for this is located in control. After this, we remove the token.

I hope this helps!

changeovertimesrl.fsm

Re: Reinforcement learning

Clara1 — Wed, 31 Jan 2024 09:19:38 GMT

Thank you so much!!

Re: Reinforcement learning

Clara1 — Thu, 01 Feb 2024 06:35:17 GMT

Hello, even tough your answer solved the issue, i have kept going with the training part and i'm facing another problem now.

It seems that, because we changed the way observation and action space are defined, it's not able to inherit from the gymnasium class correctly. (Error image attached).

Thanks in advance!

Re: Reinforcement learning

nilP8GEZ — Thu, 01 Feb 2024 06:40:57 GMT

Hello Clara,

We will try to look at it as soon as possible.

However, since it’s not the same problem as before, I strongly recommend that you create a new question, so it will have more visibility for all users

Re: Reinforcement learning

Clara1 — Thu, 01 Feb 2024 06:54:15 GMT

Ok, thanks for you time!