Run FlexSim in Parallel from Python

[joao_victor_sda]

[ FlexSim 22.2.1 ]

Hi,

I am controlling FlexSim from a python script (via FlexSimPy) for optimization, which works well when running it in serial. I.e.., I can change the parameters, random seed (via parameters table), run, and get the performance measure.

However, I need to run FlexSim in parallel (i.e., run multiple experiments simultaneously - as the Experimenter does). How can I do this?

Thank you very much.

Following is my python scrip to optimize a FlexSim Model (in at serial way). I already use traditional parallelization libraries to parallelize the FlexSim execution, but it falls.

###Import libraries###
import sys
import pandas as pd
import openpyxl
import os
import numpy as np
from EGO_NEW_rev01 import EGO_New
from AMSO_Rev01 import AMSO
from scipy.stats import norm
from scipy.optimize import minimize
from math import exp
import matplotlib.pyplot as plt
from matplotlib import colors
from smt.sampling_methods import LHS
from smt.applications import EGO
from smt.applications.mixed_integer import MixedIntegerSurrogateModel
import warnings
warnings.filterwarnings("ignore")
from smt.applications.mixed_integer import (
    FLOAT,
    ORD,
    INT,
    ENUM,
    MixedIntegerSamplingMethod,
    cast_to_mixed_integer, unfold_with_enum_mask
)
#Import FlexSimPy Module
sys.path.insert(0, path1)
import FlexSimPy as fp

  
#Import parameters from Excel
solution_space = pd.read_excel("Dashboard.xlsx",sheet_name="Painel",usecols="B:E",keep_default_na=False,header=11)
general_parameters = pd.read_excel("Dashboard.xlsx",sheet_name="Painel",usecols="G:H",keep_default_na=False,header=11)
files_path = pd.read_excel("Dashboard.xlsx",sheet_name="Painel",usecols="j:k",keep_default_na=False,header=11) 
general_parameters=general_parameters.set_index('Parameters')
files_path=files_path.set_index('Description')

#cleaning data
remove = solution_space.loc[(solution_space['Variables']=='')]
solution_space = solution_space.drop(remove.index)
#General Parameters
path1 = files_path.loc['FlexSimPy']['Path']
path2 = files_path.loc['FlexSim.exe']['Path']
replications = general_parameters.loc['Replications']['Value']
Response_variable=general_parameters.loc['Response Variable']['Value']
Stop_time= general_parameters.loc['Stop Time']['Value']
criterion=general_parameters.loc['EGO Criteria']['Value'] #'EI' or 'SBO' or 'LCB'
DOE_given = general_parameters.loc['DOE Given']['Value']
model_name= general_parameters.loc['Model Name']['Value']
ndoe = general_parameters.loc['Inicial Points']['Value']
n_iter = general_parameters.loc['Iterations']['Value']
Direction = general_parameters.loc['Direction']['Value']
#Variables Information
xlimits=solution_space[['Min','Max']].to_numpy()
xtypes = solution_space['Type'].tolist()
variaveis = solution_space['Variables'].tolist()

#The optimization direction
if Direction == "Max":
    factor = -1
else:
    factor = 1

print("start")
#Lauch FlexSim
scriptPath = os.path.dirname(os.path.realpath(__file__))
programDir = path2
controller = fp.launch(evaluationLicense=False,showGUI=False,programDir=programDir)
controller.open(scriptPath+"\\"+model_name)

def fun(x):
   
    n,dim = np.atleast_2d(x).shape
    y=[]
    w=[]
    for i in range(n):
        xi = x[i,:]
        for r in range(replications):
            w.append(simulate(xi, r, controller, variaveis, Stop_time, Response_variable))
            
        y.append(np.mean(w))
        
    print(np.atleast_2d(y).T)
    return np.atleast_2d(y).T



def simulate(x,r,controller,variaveis,Stop_time,Response_variable):

    seed=r+1
    print(x)
    #mudar seed
    instancia = controller
    instancia.setParameter("Replication",seed)

    for j in range(dim):
        valor = int(x)
        nome = variaveis
        instancia.setParameter(nome,valor)
            
    instancia.reset()
        
    instancia.runToTime(Stop_time)
        
    result = instancia.getPerformanceMeasure(Response_variable)
    
    print(result*factor)
    return result*factor
 

#number of points in the initial DOE
dim=len(variaveis)

#Create a inicial DOE (required in optimization algorithm)
path = scriptPath + "\\DOE.xlsx"
if DOE_given == 'False':
    sampling = MixedIntegerSamplingMethod(xtypes, xlimits, LHS, criterion="ese")
    xdoe = sampling(ndoe)
    X = pd.DataFrame(xdoe)
    ydoe=fun(xdoe)
    Y = pd.DataFrame(ydoe)
    
    with pd.ExcelWriter(path,mode="a", if_sheet_exists="replace",engine="openpyxl") as writer:
        X.to_excel(writer, sheet_name="X",index=False) 
        Y.to_excel(writer, sheet_name="Y",index=False)

else:
    X = pd.read_excel(path,sheet_name="X")
    Y= pd.read_excel(path,sheet_name="Y")
    xdoe= X.to_numpy()
    ydoe = Y.to_numpy()


print("X: ",xdoe," Y: ",ydoe)

#Optimization call
ego = AMSO(n_iter=n_iter, criterion=criterion, xdoe=xdoe,ydoe=ydoe, xlimits=xlimits,xtypes=xtypes)

x_opt, y_opt, ind_best, x_data, y_data, R_square, pred, std = ego.optimize(fun=fun)
y_opt=y_opt*factor

print('Xopt for Model ', x_opt, y_opt, ' obtained using EGO criterion = ', criterion )

controller.close(scriptPath+"\\ABDI_1.fsm")

with pd.ExcelWriter(path, mode="a", if_sheet_exists="replace",engine="openpyxl") as writer:
    pd.DataFrame(x_data).to_excel(writer, sheet_name="x_data",index=False) 
    pd.DataFrame(y_data).to_excel(writer, sheet_name="y_data",index=False)
    pd.DataFrame(R_square).to_excel(writer, sheet_name="R2",index=False)
    pd.DataFrame(pred).to_excel(writer, sheet_name="pred",index=False)
    pd.DataFrame(std).to_excel(writer, sheet_name="std",index=False)
    pd.DataFrame(x_opt).to_excel(writer, sheet_name="Result1",index=False,startcol=1,startrow=1)
    pd.DataFrame(y_opt).to_excel(writer, sheet_name="Result2",index=False,startcol=1,startrow=3)

[JordanLJohnson · Accepted solution]

The FlexSimPy module only allows one instance of FlexSim at a time, and it executes commands synchronously. To run multiple instances of FlexSim at a time, you'd need to use a "controller" script of Python launch several processes of Python, each controlling an instance of FlexSim. The controller would then use some form of inter-process communication (sockets, pipes, etc.) to send tasks to the FlexSim instances and return results.

[andrew_o2]

Hi @João Victor SDA, was Jordan Johnson's answer helpful? If so, please click the "Accept" button at the bottom of their answer. Or if you still have questions, add a comment and we'll continue the conversation.

If we haven't heard back from you within 3 business days we'll auto-accept an answer, but you can always unaccept and comment back to reopen your question.

[joao_victor_sda]

Hello Jordan, Thanks for your reply!

Following your think line, I put the simulate logic in a separate script, then call it from the main script in parallel way.

Best regards,

[anthony_johnsonT83CM]

This is actually a feature I'd like to add to the FlexSimPy module, namely launching FlexSim out-of-process, so that you can launch multiple FlexSims, and run multiple experiments simultaneously.