Reinforcement Learning's versioning problem

[a9080109]

[ FlexSim 22.0.16 ]

When I was running the new py environment, I found that the training could not be executed smoothly. I don’t know which link is the problem. I bought a new computer and built a new vscode, but it couldn’t be executed smoothly.

PS C:\Users\R2504_111\Documents\FlexSim 2022 Projects\RL> & C:/Users/R2504_111/AppData/Local/Programs/Python/Python311/python.exe "c:/Users/R2504_111/Documents/FlexSim 2022 Projects/RL/flexsim_training.py"

Initializing FlexSim environment...

Traceback (most recent call last):

File "C:\Users\R2504_111\AppData\Local\Programs\Python\Python311\Lib\site-packages\stable_baselines3\common\env_checker.py", line 402, in check_env

env.reset(seed=0)

TypeError: FlexSimEnv.reset() got an unexpected keyword argument 'seed'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):

File "c:\Users\R2504_111\Documents\FlexSim 2022 Projects\RL\flexsim_training.py", line 53, in <module>

main()

File "c:\Users\R2504_111\Documents\FlexSim 2022 Projects\RL\flexsim_training.py", line 19, in main

check_env(env) # Check that an environment follows Gym API.

^^^^^^^^^^^^^^

File "C:\Users\R2504_111\AppData\Local\Programs\Python\Python311\Lib\site-packages\stable_baselines3\common\env_checker.py", line 404, in check_env

raise TypeError("The reset() method must accept a `seed` parameter") from e

TypeError: The reset() method must accept a `seed` parameter

[joerg_vogel_HsH]

You are sure, you did all needed upgrades to use Python 3.11 on FlexSim 22.0 yourself?

[a9080109]

Can you show me all the necessary upgrades?

[julie_weller]

Hey @mark zhen! This looks like more of a python question than a FlexSim question. You can try searching other websites for answers or posting there. Let us know if you have a question related to FlexSim!

[philboboADSK · Accepted solution]

The provided python scripts are example code to demonstrate how to communicate with FlexSim from a reinforcement learning algorithm. If they don't work perfectly with a particular version of python or another language, library, package, or implementation, then you should customize them according to the needs of your project. As explained in the first paragraph of Getting Started with OpenAI Gym (flexsim.com).

It looks like the version of stable_baselines you are using requires a "seed" parameter on the environment's reset() function. In flexsim_env.py, try adding a seed parameter to that function. Something like this:

    def reset(self, seed=None):
        self.seedNum = seed
        self._reset_flexsim()
        state, reward, done = self._get_observation()
        return state

Alter the example python script to work if it isn't working. That's why the script is provided.

[JordanLJohnson]

Is the issue resolved by commenting out the check_env() line? I think I have seen that come up before.

[a9080109]

I have no way to solve the problem I am encountering now

[joerg_vogel_HsH]

@mark zhen , you checked if a path description allows characters of more bytes length. Maybe there are only characters allowed of one byte length like an ASCII set. You are dealing with a programming language coming from an origin completely different to a windows system

https://docs.python.org/3/tutorial/interpreter.html.

[julie_weller]

Hi @mark zhen , was one of Phil BoBo's or Jordan Johnson's answers helpful? If so, please click the "Accept" button at the bottom of the one that best answers your question. Or if you still have questions, add a comment and we'll continue the conversation.

If we haven't heard back from you within 3 business days we'll auto-accept an answer, but you can always comment back to reopen your question.

[philboboADSK]

Your comment doesn't describe the problem you are encountering now, just a screenshot of a code snippet.

[a9080109]

@Phil BoBo @Jordan Johnson @Joerg Vogel @Kavika F @Felix Möhlmann

I'm having the same problem, but there is still no good way to solve it, I'll provide my code and related versions, I hope it can help me solve the problem, thank youflexsim_training.pyflexsim_training.pyflexsim_env.py

[JordanLJohnson]

Hi @mark zhen , see the files in the following answer:
https://answers.flexsim.com/questions/167537/reinforcement-learning-more-decision-events-settin.html

I posted an rl-sample.zip file in that answer. I made many changes to the default files to get it to work. You should be able to work from those with more success.