topic Re: Define the state of reinforcement learning in FlexSim Forum

Define the state of reinforcement learning

a9080109 — Mon, 04 Sep 2023 07:23:20 GMT

[ FlexSim 22.0.16 ]

I think I'm almost done, the state of my model now,

I want to define it as the number of deferred order but I'm a bit confused on how to do it?

And I define my actions I take six different actions

The part about the reward may be to minimize the tardness or to calculate the average of the overall tardness (but I don't know how to calculate the average in flexsim)

As for the label part, I have defined four labels in the source

ArrivalTime is the arrival time of the goods

date is the delivery time

total arrival total arrival time

mark the order in which goods enter

I want to calculate the average tardness in the global table of flexsim. How should I do it?

@Kavika F @Felix Möhlmann @Jason Lightfoot

rule0905_autosave.fsm

Re: Define the state of reinforcement learning

moehlmann_fe — Wed, 06 Sep 2023 17:18:23 GMT

You could just sum up the tardiness of each entering item in a label on the sink. Then get the average by dividing that value by the input stat.

rule0905-autosave.fsm

Re: Define the state of reinforcement learning

jason_lightfoot_adsk — Wed, 06 Sep 2023 17:23:31 GMT

The rolling average for the label on the sink would be:

((N-1)*avgTardiness+item.tardiness)/N

Using the global table you can use :

Table.query("SELECT AVG(tardiness) FROM [entry time]")[1][1]

..if the tardiness field contains the lateness of each item (not the rolling average).

Re: Define the state of reinforcement learning

a9080109 — Wed, 06 Sep 2023 18:30:13 GMT

I have another question to ask about my state. I am currently trying to calculate how many orders I have in total that are delayed, but I feel that my approach may not be right.

Re: Define the state of reinforcement learning

jason_lightfoot_adsk — Wed, 06 Sep 2023 22:14:13 GMT

That can just be a counter on a label that you increment with item.tardiness>0.

Please do the all tutorials if you haven't already or consult your academic institution's training material.

Re: Define the state of reinforcement learning

a9080109 — Thu, 07 Sep 2023 05:40:08 GMT

No, I've done the calculation, but I want to treat it as my state, and my model has this done, but there may be some definitions or details that I haven't dealt with.

Re: Define the state of reinforcement learning

jason_lightfoot_adsk — Thu, 07 Sep 2023 10:44:33 GMT

These sounds like comments, not questions.

Re: Define the state of reinforcement learning

a9080109 — Thu, 07 Sep 2023 13:48:15 GMT

My problem is that I'm not sure if my status is set correctly.

(My ideal state is to delay the order)

Re: Define the state of reinforcement learning

moehlmann_fe — Fri, 08 Sep 2023 06:20:23 GMT

What do you mean by 'state'? The observations for the RL algorithm?

We can't tell you if your model is set up correctly without seeing the current version.

Re: Define the state of reinforcement learning

a9080109 — Fri, 08 Sep 2023 09:48:15 GMT

rule0905-autosave_autosave.fsm

Yes, in the literature I read on rl, the three elements of state action reward are mentioned. In my understanding, state may be On obeservation?

Re: Define the state of reinforcement learning

jason_lightfoot_adsk — Mon, 11 Sep 2023 11:24:52 GMT

New post here.

Hi @mark zhen , was one of Jason Lightfoot's or Felix Möhlmann's answers helpful? If so, please click the "Accept" button at the bottom of the one that best answers your question. Or if you still have questions, add a comment and we'll continue the conversation.

If we haven't heard back from you within 3 business days we'll auto-accept an answer, but you can always comment back to reopen your question.

Re: Define the state of reinforcement learning

a9080109 — Thu, 14 Sep 2023 15:57:10 GMT

I do not understand what you mean!!