topic Re: Reinforcement Learning's versioning problem in FlexSim Forum

Reinforcement Learning's versioning problem

a9080109 — Tue, 08 Aug 2023 08:30:46 GMT

[ FlexSim 22.0.16 ]

When I was running the new py environment, I found that the training could not be executed smoothly. I don’t know which link is the problem. I bought a new computer and built a new vscode, but it couldn’t be executed smoothly.

PS C:\Users\R2504_111\Documents\FlexSim 2022 Projects\RL> & C:/Users/R2504_111/AppData/Local/Programs/Python/Python311/python.exe "c:/Users/R2504_111/Documents/FlexSim 2022 Projects/RL/flexsim_training.py"

Initializing FlexSim environment...

Traceback (most recent call last):

File "C:\Users\R2504_111\AppData\Local\Programs\Python\Python311\Lib\site-packages\stable_baselines3\common\env_checker.py", line 402, in check_env

env.reset(seed=0)

TypeError: FlexSimEnv.reset() got an unexpected keyword argument 'seed'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):

File "c:\Users\R2504_111\Documents\FlexSim 2022 Projects\RL\flexsim_training.py", line 53, in <module>

main()

File "c:\Users\R2504_111\Documents\FlexSim 2022 Projects\RL\flexsim_training.py", line 19, in main

check_env(env) # Check that an environment follows Gym API.

^^^^^^^^^^^^^^

File "C:\Users\R2504_111\AppData\Local\Programs\Python\Python311\Lib\site-packages\stable_baselines3\common\env_checker.py", line 404, in check_env

raise TypeError("The reset() method must accept a `seed` parameter") from e

TypeError: The reset() method must accept a `seed` parameter

Re: Reinforcement Learning's versioning problem

joerg_vogel_HsH — Tue, 08 Aug 2023 08:59:31 GMT

You are sure, you did all needed upgrades to use Python 3.11 on FlexSim 22.0 yourself?

Re: Reinforcement Learning's versioning problem

a9080109 — Tue, 08 Aug 2023 10:17:45 GMT

Can you show me all the necessary upgrades?

Re: Reinforcement Learning's versioning problem

julie_weller — Tue, 08 Aug 2023 15:10:35 GMT

Hey @mark zhen! This looks like more of a python question than a FlexSim question. You can try searching other websites for answers or posting there. Let us know if you have a question related to FlexSim!

Re: Reinforcement Learning's versioning problem

philboboADSK — Tue, 08 Aug 2023 16:36:04 GMT

The provided python scripts are example code to demonstrate how to communicate with FlexSim from a reinforcement learning algorithm. If they don't work perfectly with a particular version of python or another language, library, package, or implementation, then you should customize them according to the needs of your project. As explained in the first paragraph of Getting Started with OpenAI Gym (flexsim.com).

It looks like the version of stable_baselines you are using requires a "seed" parameter on the environment's reset() function. In flexsim_env.py, try adding a seed parameter to that function. Something like this:

    def reset(self, seed=None):
        self.seedNum = seed
        self._reset_flexsim()
        state, reward, done = self._get_observation()
        return state

Alter the example python script to work if it isn't working. That's why the script is provided.

Re: Reinforcement Learning's versioning problem

JordanLJohnson — Tue, 08 Aug 2023 17:28:22 GMT

Is the issue resolved by commenting out the check_env() line? I think I have seen that come up before.

Re: Reinforcement Learning's versioning problem

a9080109 — Wed, 09 Aug 2023 00:36:54 GMT

I have no way to solve the problem I am encountering now

Re: Reinforcement Learning's versioning problem

joerg_vogel_HsH — Wed, 09 Aug 2023 10:11:46 GMT

@mark zhen , you checked if a path description allows characters of more bytes length. Maybe there are only characters allowed of one byte length like an ASCII set. You are dealing with a programming language coming from an origin completely different to a windows system

https://docs.python.org/3/tutorial/interpreter.html.

Re: Reinforcement Learning's versioning problem

julie_weller — Mon, 14 Aug 2023 13:43:45 GMT

Hi @mark zhen , was one of Phil BoBo's or Jordan Johnson's answers helpful? If so, please click the "Accept" button at the bottom of the one that best answers your question. Or if you still have questions, add a comment and we'll continue the conversation.

If we haven't heard back from you within 3 business days we'll auto-accept an answer, but you can always comment back to reopen your question.

Re: Reinforcement Learning's versioning problem

philboboADSK — Mon, 14 Aug 2023 14:00:03 GMT

Your comment doesn't describe the problem you are encountering now, just a screenshot of a code snippet.

Re: Reinforcement Learning's versioning problem

a9080109 — Thu, 18 Jul 2024 12:27:41 GMT

@Phil BoBo @Jordan Johnson @Joerg Vogel @Kavika F @Felix Möhlmann

I'm having the same problem, but there is still no good way to solve it, I'll provide my code and related versions, I hope it can help me solve the problem, thank youflexsim_training.py flexsim_training.py flexsim_env.py

Re: Reinforcement Learning's versioning problem

JordanLJohnson — Fri, 26 Jul 2024 20:34:39 GMT

Hi @mark zhen , see the files in the following answer:
https://answers.flexsim.com/questions/167537/reinforcement-learning-more-decision-events-settin.html

I posted an rl-sample.zip file in that answer. I made many changes to the default files to get it to work. You should be able to work from those with more success.