Plugin Development: Ways to add custom and external modules

[chr33z]

Hey!

I develop Scripts and Plugins for several years now and used several different approaches to include external python modules for plugins. All with their own advantages and drawbacks.

The general problem is, that every python file (.py) in the ScriptPlugins directory is automatically interpreted as a script plugin. A naive approach to adding external modules or splitting up you code in different files as plain scripts does not work.

There are several different ways I tried to work around that over the years:

• Put all plugin modules in a directory outside of ScriptPlugins
  • Pro: Easy to do with a custom build pipeline
  • Con: Not very elegant, may break when users load plugins from a directory specified by environment variables
• Remove file extensions from module files and use some python magic to load them anyway
  • Pro: Files can be places inside the plugin directory
  • Con: Complex build pipeline necessary, testing is slower because files have to be processed and loaded differently when running in VRED (you want the files to be .py in your dev environment but without .py in VRED)
• Compile python files to .pyc
  • Pro: Works with custom modules
  • Con: Does not work (out-of-the-box) when you want to ship other python packages
• Export everything to a custom module and load it from somewhere else
  • Really depends on the customer infrastructure
• Put all code in one file
  • Good look maintaining your 10000 lines of code plugin

I havent found the holy grail yet. What approaches do you prefer? Is there a "best practice" that we can agree on?

Suggestions for improvement:

• Use an "ignore" file where developers can define glob patterns to exclude .py files from being interpreted as Script Plugins.
• Suggest a default Plugin directory structure where custom and externa libraries have fixed folders

Best,

Christopher

[Christian_Garimberti]

Hi @chr33z , good analysis, i completely agree with you!

Right now i manage "complex" script plugins  with an "outside" folder and for me is the best way i found.

Best

Chris

[chr33z]

I added "Compiling python script to .pyc". This works for custom modules, but is harder when you want to ship additional python packages.

[clemens_schlicker_audi]

Hi and thanks for the exchange here!

---

1. Put all plugin modules in a directory outside of ScriptPlugins
2. Remove file extensions from module files and use some python magic to load them anyway
3. Compile python files to .pyc
4. Export everything to a custom module and load it from somewhere else
5. Put all code in one file

---

I've used 1., 3, 4 and 5 in the past. I can still recommend the first approach, but have an improved version (see below).

User environment variable VRED2025_3_SCRIPT_PLUGINS specifies a directory like C:\VRED-Plugins\VREDPro-17.3\ScriptPlugins as an additional search path for ScriptPlugins. This is where the main entry point of the Plugin has to be placed. I'll call it MyScriptPlugin.py here.

C:\VRED-Plugins\VREDPro-17.3\ScriptPlugins\MyScriptPlugin.py

In the MyScriptPlugin.py, I typically refer to (at least) two other directories: site-packages & scriptsDirectory.

The folder site-packages, is for all python packages, similar to what you can find in typical python installation directories, e.g. C:\Program Files\Autodesk\VREDPro-17.3\lib\python\Lib\site-packages

Those packages are shared among all plugins and live in the same global namespace.

The second folder scriptsDirectory is also added to path and refers to the custom modules and packages that are specifically used for this plugin. It is structured in python packages with (more or less) one package per plugin. I consider the top-level-packages as namespaces and only use relative imports to prevent cyclic dependencies and issues with python modules with similar names in other plugins.

...
# assumption: _hidden_ python modules (directories or py-files) are located next to this file
scriptsDirectory = os.path.dirname(__file__)
if not scriptsDirectory in sys.path:
    sys.path.append(scriptsDirectory)

# assumption: _hidden_ directory 'site-packages' is located next to this file
sitePackagesDirectory = os.path.join(os.path.dirname(__file__), "site-packages")
if not sitePackagesDirectory in sys.path:
    sys.path.append(sitePackagesDirectory)
...

The best solution for me is to just hide the top-level folders in the ScriptPlugins search path containing *.py files 🙂

With Installers like InnoSetup, you can specify certain attributes that directories should have when being installed to target machines.

Example:

.\ScriptPlugins\MyScriptPlugin.py <- entry point & defines plugin name in VRED
.\ScriptPlugins\AnyOtherPlugin.py

.\site-packages\ <- common site-packages, added to sys.path
.\ScriptPlugins\site-packages\ <- could also be here (but needs to be hidden)
.\ScriptPlugins\ScriptsDirectory\ <- scriptsDirectory, hidden, added to sys.path

.\ScriptPlugins\ScriptsDirectory\myscriptplugin\ <- use import myscriptpluginto in 'MyScriptPlugin.py'
.\ScriptPlugins\ScriptsDirectory\anyotherplugin\ <- does no harm

.\ScriptPlugins\ScriptsDirectory\myscriptplugin\__init__.py <- will be executed once (on first import)
.\ScriptPlugins\ScriptsDirectory\myscriptplugin\package\module.py <- custom modules with relative imports

In this layout, only the ScriptsDirectory itself (and possibly site-packages) needs to be hidden, VRED won't walk through the folders to locate *.py files.

[clemens_schlicker_audi]

In this other thread ScriptPlugin Questions,  I mentioned how I typically refer to those scrips.

<quote>

To access variables that are defined in the scope of your script plugin, you've got to inject some variables/attributes to modules. 

In the script plugin:

import sys
import os

pluginPath = os.path.dirname(__file__)
if not pluginPath in sys.path:
    sys.path.append(pluginPath)

# assumption: your module is called 'mymodulename' 
# prerequisite: 'mymodulename' is located next to the script plugin python file
# prerequisite: 'mymodulename' is a hidden directory or a hidden python file
import mymodulename

class MyPluginClass():
    pass

myPluginClass = MyPluginClass()

mymodulename.myPluginClass = myPluginClass

In Vred (e.g. terminal or variant set scripts)

import mymodulename

print(mymodulename.myPluginClass)
# terminal output: <VREDModule56.MyPluginClass object at 0x00000175B2455E50>

Keep in mind that the life-cycle of these attributes have to be considered since they might live longer than the VRED scene contents that they refer to. You can connect to vrMessageService or vrFileIOService signals to tidy up, reset or recreate your objects.

Also, any modules that you import are only imported onceduring the lifespan of a VRED runtime. Consequently, don't rely on just an import statement and global statements in the modules, but instead implement start and cleanup functions for your modules and call them from VRED (hotkeys, Qt GUI, singals, VarSets, ...).

</quote>

[chr33z]

@clemens_schlicker_audi Nice!

I'll have to try if a hidden directory actually is ignored. I like the idea! It's like an "ignore" pattern to ignore python files from loading. In theory it should be possible to hide a folder inside your plugin folder:

• ScriptPlugins
  • MyAwesomePlugin
    • modules [hidden]
    • MyAwesomePlugin.py
    • ...

At the moment I use a directory located outside of ScriptPlugins for additional plugin modules. But it don't like the fact that the files are spread over different folders and file hiearchies.

Edit: But imo a more cleaner approach would actually be a ignore-file because hidden folders are - well - hidden and obscured which makes it harder to understand the structure of a plugin

[clemens_schlicker_audi · Accepted solution]

If you main focus is to have just one plugin folder ".\ScriptPlugins\MyAwesomePlugin", here's another example folder structure as bullet list:

• .\ScriptPlugins
  • .\MyAwesomePlugin
    • .\data
      • ... [documentation, assets and stuff]
    • .\scripts
      • .\myawesomeplugin [hidden]
        • .\mypackageA
          • module1.py
          • module2.py
        • __init__.py
      • .\site-packages [hidden]
      • MyAwesomePlugin.py <- Entry point with VREDPluginWidget access
    • .\uninstaller
      • unins000.exe
    • app-version.txt
    • release-notes.md
    • ...
  • .\OtherAwesomePlugin
    • OtherAwesomePlugin.py
  • MonolithicPlugin.py
  • ...

In VRED:

import myawesomeplugin
myawesomeplugin.initialize() # from myawesomeplugin/__init__.py

from myawesomeplugin.mypackageA.module1 import MyClass
myInstance = MyClass()
myInstance.doSomething()

Hidden folders do work (at least since 2023.2) and continue to work in VRED 2025.3 🙂

[chr33z]

Hidden folders make it instantly more fun to develop plugins 😄