topic Re: Error in initializing the environment using the flexsim_env.py file in FlexSim Forum

Error in initializing the environment using the flexsim_env.py file

lorenzo_pio_g — Sun, 16 Oct 2022 10:28:15 GMT

[ FlexSim 22.2.2 ]

Hi. I have developed a model in flexsim in which I need to optimize the idle rate of a processor by applying RL to a robot. It has to pick up the boxes from the right queue and decide on which combiner to leave it, based on the number of boxes present on the pallet placed on the top combiner (Combiner1). I used a process flow to manage the movement of the robots and defined all the parameters of the RLToolbox. However, initializing the environment via the flexsim_env.py file I get the following messages on the terminal:

The model in flexsim opens correctly; however, it appears that parameters regarding observations and actions are not being read correctly. At the end of loading, in fact, the terminal should return me the message "Waiting for input to close flexsim". Can you tell me what this may be due to? Is there some error in the initialization of the parameters?

progettoTesi.fsm

Re: Error in initializing the environment using the flexsim_env.py file

kavika_faleumu — Thu, 20 Oct 2022 22:11:15 GMT

Hey @Lorenzo_pio G, when I opened your model, it said that it couldn't set your parameters to 0. I checked them and they had a lower bound of 1. If you change their lower bounds to 0, the program should be able to set them to 0 and allow you to run the simulation. Let me know if that helps.

Re: Error in initializing the environment using the flexsim_env.py file

lorenzo_pio_g — Fri, 21 Oct 2022 08:59:11 GMT

Yes, I have already changed those parameters, but it still doesn't work.

Re: Error in initializing the environment using the flexsim_env.py file

moehlmann_fe — Fri, 21 Oct 2022 10:58:56 GMT

It seems using the "OnEntry" event of a process flow activity as the decision trigger doesn't work properly. If you use the "OnLoad" event of the robot instead it works. (Sorry, that I didn't check this before saying you could use the PF activity. I didn't expect there to be a (likely) bug)

You should also adjust the code that sets the observation parameters. You are currently not dividing the idle time by the model time, so the observation wouldn't be the rate but the total time in the state. Also, since you are using an integer parameter, the calculated value should be rounded before applying it to the parameter.

if(Model.time == 0)
{
    Model.parameters["IdleRate"].value = 0;
}
else
{
    Model.parameters["IdleRate"].value = Math.round(Model.find("Sigillatura").as(Object).stats.state().getTotalTimeAt("idle")/Model.time*100, 0)/;
}

Re: Error in initializing the environment using the flexsim_env.py file

lorenzo_pio_g — Fri, 21 Oct 2022 11:17:28 GMT

So do I need to change the Decision Events of the RL Tool? Should I delete the Event1 event and sample the Robot to which I need to add the OnLoad trigger?

Re: Error in initializing the environment using the flexsim_env.py file

lorenzo_pio_g — Fri, 21 Oct 2022 11:23:22 GMT

What is the purpose of the "/" character in line 7? Because it gives me an error the moment I enter the ";".

Re: Error in initializing the environment using the flexsim_env.py file

moehlmann_fe — Fri, 21 Oct 2022 11:40:48 GMT

Yes, use the sampling tool to select the new event from the robot.

I don't know how the "/" at the end of the line got there. There shouldn't be one.

Re: Error in initializing the environment using the flexsim_env.py file

lorenzo_pio_g — Fri, 21 Oct 2022 11:45:08 GMT

So can I also eliminate the "Make Decision" and "Breathe" blocks, since I will eliminate Event1?

Next, should the OnLoad trigger be applied only to the robot I want to make the decision? Also, do I leave the OnLoad trigger space blank or do I need to write a condition?

Re: Error in initializing the environment using the flexsim_env.py file

lorenzo_pio_g — Fri, 21 Oct 2022 12:00:53 GMT

However, it keeps giving me the same message over and over again.

At least now the value of the state is increased

Re: Error in initializing the environment using the flexsim_env.py file

lorenzo_pio_g — Fri, 21 Oct 2022 12:21:20 GMT

Using the flexsim_training.py file instead, it now succeeds in initializing the environment. However, despite the fact that I set a value of 100000 timesteps, it does not terminate the training, but stops at 2 iterations, without giving me a termination message. So, as with the flexsim_env-py file, it only manages to do 2 status updates.

Re: Error in initializing the environment using the flexsim_env.py file

moehlmann_fe — Fri, 21 Oct 2022 12:48:22 GMT

- You can eliminate the "Make Decision" activity, but I'd keep the "Breathe".

- You don't have to actually add an On Load trigger to the robot, the event will fire regardless.

- Are you sure that the training stopped. If I run it, it takes some time, but further iteration messages do appear.

Re: Error in initializing the environment using the flexsim_env.py file

lorenzo_pio_g — Fri, 21 Oct 2022 12:56:41 GMT

If I run it, it stops at this message:

The rollout/ section is missing.

Re: Error in initializing the environment using the flexsim_env.py file

lorenzo_pio_g — Fri, 21 Oct 2022 12:59:12 GMT

This is my project, maybe I made some mistakes.

progettoTesi.fsm

Re: Error in initializing the environment using the flexsim_env.py file

lorenzo_pio_g — Fri, 21 Oct 2022 14:28:35 GMT

I changed the value of the reward code, marking done=1 every 1000 seconds of simulation. Now everything works correctly.

Re: Error in initializing the environment using the flexsim_env.py file

lorenzo_pio_g — Sun, 23 Oct 2022 19:39:12 GMT

@Felix Möhlmann In analyzing the data from the different trainings I ran, I noticed that the value of the reward is not incremented in the various steps of the simulation. Perhaps this is due to the fact that I rounded up the idleRate value? Or is it due to other settings, such as the reward allocation formula?

First, I changed the observations. I used the number of boxes on the combiner1 pallet as the only observation parameter. I did this because, using a multidiscrete parameter as the observation space, the flexsim_interference.py file was not working. However, as mentioned, the reward does not increase.

That's the code that I used to set observations:

int itemNum = 0;

// Check if there is a pallet on the combiner (a subnode of the combiner)

if(Model.find("Combiner1").subnodes.length > 0)

{

/* Read the number of subnodes (items) of the pallet

"first" is the same as writing "subnodes[1]";

the first subnode of the combiner, so the pallet */

itemNum = Model.find("Combiner1").first.subnodes.length;

}

Model.parameters["numItem"].value = itemNum;

That's the code that sets rewards:

double reward = Model.time/(Model.find("Sigillatura").as(Object).stats.state().getTotalTimeAt("idle")+1);

int done = (Model.time>1000);

return [reward, done];

Re: Error in initializing the environment using the flexsim_env.py file

moehlmann_fe — Mon, 24 Oct 2022 06:24:41 GMT

If the timings are still set like in the latest model you uploaded, then this is to be expected. Currently the limiting factor of the throughput is the movetime of the robots. Which combiner Robot1 unloads to doesn't actually matter for the average throughput.