Python script module: import numpy not working...

[Anonymous]

Hi,

I have a python script where I want to import functions from numpy to use in fusion360.

I have a Mac with Python 2.7 which has the numpy module installed in /usr/local/lib/python2.7/site-packages. And from my Mac console python command line, I can import numpy successfully,

$ python
Python 2.7.10 (default, Sep 14 2015, 02:26:06) 
[GCC 4.2.1 Compatible Apple LLVM 6.1.0 (clang-602.0.53)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy as np
>>> a = np.array([1, 2, 3])
>>> print type(a)
<type 'numpy.ndarray'>
>>> 

I've read over this previous forum and it helped me to get going, but now I'm getting an import failure when I try to import numpy.

Here's my test script I am loading into Fusion360:

import adsk.core, adsk.fusion, traceback

##  1) did not work
#import sys
#sys.path.append("/usr/local/lib/python2.7/site-packages")
#import numpy as np

##  2) did not work
#import imp
#f, filename, desc = imp.find_module('numpy', ['/usr/local/lib/python2.7/site-packages'])
#np = imp.load_module('numpy', f, filename, desc)

##  3) almost works, but python crashes importing numpy
import sys
sys.path[0:0] = '/usr/local/lib/python2.7/site-packages'
import numpy as np

In Fusion360, I have clicked "Scripts and Add-ins" and selected my script and clicked "Edit". So now I am able to run and debug in the Spyder Python GUI.

When I run this test script in the Fusion360 Spyder GUI, I get this error message:

Welcome to Autodesk Fusion360 console!
Sample script:
	import adsk.core
	app = adsk.core.Application.get()
	help(app)

>>> Traceback (most recent call last):
  File "/Users/ali/Library/Application Support/Autodesk/Autodesk Fusion 360/API/Scripts/test_numpy_load/test_numpy_load.py", line 16, in <module>
    import numpy as np
  File "/usr/local/lib/python2.7/site-packages/numpy/__init__.py", line 180, in <module>
    from . import add_newdocs
  File "/usr/local/lib/python2.7/site-packages/numpy/add_newdocs.py", line 13, in <module>
    from numpy.lib import add_newdoc
  File "/usr/local/lib/python2.7/site-packages/numpy/lib/__init__.py", line 8, in <module>
    from .type_check import *
  File "/usr/local/lib/python2.7/site-packages/numpy/lib/type_check.py", line 11, in <module>
    import numpy.core.numeric as _nx
  File "/usr/local/lib/python2.7/site-packages/numpy/core/__init__.py", line 14, in <module>
    from . import multiarray
ImportError: dlopen(/usr/local/lib/python2.7/site-packages/numpy/core/multiarray.so, 2): Symbol not found: _PyBuffer_Type
  Referenced from: /usr/local/lib/python2.7/site-packages/numpy/core/multiarray.so
  Expected in: flat namespace
 in /usr/local/lib/python2.7/site-packages/numpy/core/multiarray.so

>>> 

Any pointers would be helpful and appreciated!

Thanks,

Sean

[Anonymous]

And another piece of info, I believe my numpy is 64bit from this on Mac python console,

$ python
Python 2.7.10 (default, Sep 14 2015, 02:26:06) 
[GCC 4.2.1 Compatible Apple LLVM 6.1.0 (clang-602.0.53)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy.distutils.system_info as sysinfo
>>> sysinfo.platform_bits
64
>>> 

[liujac]

Hi,

Fusion 360 is using Python 3.3.5 (64-bit). You installed numpy for Python 2.7, that is not compatible with Python 3.3.5. You have to install numpy for Python 3.3.5 ( 64-bit ) and import the numpy. The following example should work:

        import sys
        sys.path.append('/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages')
        import numpy as np
        a = np.array([1, 2, 3])
        print(type(a))

Regards,

Jack

[Anonymous]

Just some more details as I debug...

After many mis-steps and a broken Mac default Python install, I finally got 3.3.5 installed on my Mac using pyenv, with numpy installed using easy_install,

$ pwd
/Users/ali/.pyenv/versions/3.3.5/lib/python3.3/site-packages
$ ls -ld numpy*
drwxr-xr-x  27 root  staff  918 Oct 19 11:59 numpy/
drwxr-xr-x   9 root  staff  306 Oct 19 11:59 numpy-1.11.2.dist-info/

So now I can import this module on the Mac command-line Python,

$ python3.3 --version
Python 3.3.5
$ cat mac_test_numpy_load.py 
import sys
# put at end
#sys.path.append('/Users/ali/.pyenv/versions/3.3.5/lib/python3.3/site-packages')
# put at start
sys.path.insert(0,"/Users/ali/.pyenv/versions/3.3.5/lib/python3.3/site-packages")
import numpy as np
print (np.__path__)
$ python3.3 mac_test_numpy_load.py 
['/Users/ali/.pyenv/versions/3.3.5/lib/python3.3/site-packages/numpy']

But if I run this same script (which also loadsadsk.core, adsk.fusion, traceback), I get the same error response looking for multiarray,

Welcome to Autodesk Fusion360 console!
Sample script:
	import adsk.core
	app = adsk.core.Application.get()
	help(app)

>>> Traceback (most recent call last):
  File "/Users/ali/Library/Application Support/Autodesk/Autodesk Fusion 360/API/Scripts/test_numpy_load/test_numpy_load.py", line 7, in <module>
    import numpy as np
  File "/Users/ali/.pyenv/versions/3.3.5/lib/python3.3/site-packages/numpy/__init__.py", line 142, in <module>
    from . import add_newdocs
  File "/Users/ali/.pyenv/versions/3.3.5/lib/python3.3/site-packages/numpy/add_newdocs.py", line 13, in <module>
    from numpy.lib import add_newdoc
  File "/Users/ali/.pyenv/versions/3.3.5/lib/python3.3/site-packages/numpy/lib/__init__.py", line 8, in <module>
    from .type_check import *
  File "/Users/ali/.pyenv/versions/3.3.5/lib/python3.3/site-packages/numpy/lib/type_check.py", line 11, in <module>
    import numpy.core.numeric as _nx
  File "/Users/ali/.pyenv/versions/3.3.5/lib/python3.3/site-packages/numpy/core/__init__.py", line 14, in <module>
    from . import multiarray
ImportError: dlopen(/Users/ali/.pyenv/versions/3.3.5/lib/python3.3/site-packages/numpy/core/multiarray.so, 2): Symbol not found: _PyBool_Type
  Referenced from: /Users/ali/.pyenv/versions/3.3.5/lib/python3.3/site-packages/numpy/core/multiarray.so
  Expected in: /Users/ali/Library/Application Support/Autodesk/webdeploy/production/8beef08b232ffde48d2a02741cfc403a7f8753bf/Autodesk Fusion 360.app/Contents/MacOS/Autodesk Fusion 360
 in /Users/ali/.pyenv/versions/3.3.5/lib/python3.3/site-packages/numpy/core/multiarray.so

Next step is to remove other numpy installs in,

sudo rm -rf /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python

[Anonymous]

No luck so far - any more help would be appreciated!

$ which python3.3
/usr/local/bin/python3.3
$ find /System/Library/Frameworks/Python.framework/ -name numpy
$ find /Library/Frameworks/Python.framework/ -name numpy
$ find ~/Library/Frameworks -name numpy

[liujac]

I did not use pyenv. Here are the steps I installed Python 3.3.5 and numpy:

1. Download Python 3.3.5, and install to your Mac. 

2. Download get-pip.py file to your Mac. Enter "python3 /users/liujac/downloads/get-pip.py" in Terminal to install pip.

3. Enter "python3 -m pip install numpy" in Terminal to install numpy.

With the steps above, the numpy is installed to "/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages"

$ find /Library/Frameworks -name numpy
/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages/numpy
/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages/numpy/core/include/numpy

Run the script below in Fusion 360, it works.

import adsk.core, adsk.fusion, adsk.cam, traceback

def run(context):
    ui = None
    try:
        app = adsk.core.Application.get()
        ui  = app.userInterface
        
        import sys
        sys.path.append('/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages')
        import numpy as np
        a = np.array([1, 2, 3])
        print(type(a))
        print(np.__path__)
        
        ui.messageBox('Done')

    except:
        if ui:
            ui.messageBox('Failed:\n{}'.format(traceback.format_exc()))

Jack

[Anonymous]

Hi Jack,

Thanks that's great advice, and very detailed.

Initially that was my thinking as well, but the Python 3.3.5 Mac installer had crashed for me. So I was unable to install 3.3.5 that way. Maybe it's my Mac running 10.11.6 that is causing the problem?

I also tried Homebrew to install 3.3.5 but it doesn't support 'versions' anymore, so if I brew install python3, I get 3.5 or something latest. I can't pick specifically the 3.3.5 release.

I found I could install the 3.3.5 release with pyenv. It places the full Python install directory structure into my $HOME directory, and I got the numpy module installed in there. And as I mentioned previously, I can run python3.3 from the Mac cmd-line, add the path with sys.path.append, and import numpy no problem. And printing np.__path__ shows I'm truly loading that module.

As a last attempt, I made a sym-link in /Library so now I have the same directory structure as in your script,

$ pwd
/Library/Frameworks/Python.framework/Versions
$ ls -l
total 8
lrwxr-xr-x  1 root  wheel  32 Oct 20 10:44 3.3 -> /Users/ali/.pyenv/versions/3.3.5

$ cd /Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages
$ pwd
/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages
$ ls
README                       numpy/                       pip-8.1.2.dist-info/         setuptools-28.6.1.dist-info/
__pycache__/                 numpy-1.11.2.dist-info/      pkg_resources/               wheel/
easy_install.py              pip/                         setuptools/                  wheel-0.29.0.dist-info/

$ pwd -P
/Users/ali/.pyenv/versions/3.3.5/lib/python3.3/site-packages

I modified your script just slightly to try both of these scenarios, but both produces the same failure for me,

        sys.path.insert(0,'/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages')
        #sys.path.append('/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages')

Below is the failure when I have it as shown above with sys.path.insert()

I feel I'm really close to figuring this out but missing some fundamental piece of information...

Anything else I should check?

Thanks!

Sean

[liujac]

l am using osx 10.10.5. I will a try on osx 10.11.6. Looks like there is compatibility issue btween the numpy and the python shipped with Fusion 360 on osx 10.11.6.

Jack

[liujac · Accepted solution]

Hi Sean,

I reproduced your problem on Mac 10.11.6. Use cmd "$ /Users/jackliu/.pyenv/versions/3.3.5/bin/python -m pip install numpy" to install numpy, it downloads the sources code, and builds the binaries with the gcc on the Mac. Unfortunately, the gcc version on Mac 10.11.6 is different with the gcc version used to build Python 3.3.5 packed in Python official installer, seems there is compatibility issue between the two gcc versions, that causes some binaries of the numpy built on this Mac are not compatible with the Python 3.3.5 packed in Python official installer. Fusion 360 just packed the Python 3.3.5 installed from Python official installer.

Enter "gcc -v" in Terminal on my Mac 10.11.6:

$gcc -v
Configured with: --prefix=/Library/Developer/CommandLineTools/usr --with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 7.3.0 (clang-703.0.31)
Target: x86_64-apple-darwin15.6.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin

GCC version of Python 3.3.5 packed with Fusion 360.

$ /Users/jackliu/Library/Application\ Support/Autodesk/webdeploy/shared/PYTHON/3.3.5/MAC64c/Python.framework/Versions/3.3/bin/python 
Python 3.3.5 (v3.3.5:62cf4e77f785, Mar 9 2014, 01:12:57) 
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin

To resolve this problem, you need to use "pip install" with option "--only-binary" to install numpy, that will not build the source code just download the binaries. For example:

$ /Users/jackliu/.pyenv/versions/3.3.5/bin/python -m pip install --only-binary :all: numpy
Collecting numpy
Using cached numpy-1.11.0-cp33-cp33m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl
Installing collected packages: numpy
Successfully installed numpy-1.11.0

Regards,
Jack

[Anonymous]

Hi Jack,

Thanks for figuring that out - I'm now able to import numpy into F360!

Turns out I actually had to pip uninstall numpy first and then install with your options to get the right version. But it's working!

Thanks again!

Sean

[Anonymous]

Hello nice work on this work perfectly as this error was disturbing me greatly thank you for the help.

[jc8000]

for windows if anyone is curious:

sys.path.append('C:\\Users\\name\\AppData\\Local\\Programs\\Python\\PythonXX\\Lib\site-packages')

just replace 'name' with your username and 'PythonXX' with the version of python the package you'd like to use is installed in. 

not sure if this will work with older or newer versions of python than what fusion is currently using. 

[hyki]

I found a simpler solution that you don't need to look for python folders or version. Put the below script before importing numpy:

import sys
import subprocess
subprocess.check_call([sys.executable, '-m', 'pip', 'install', 'numpy'])

You can remove the script after running once. The script will not give you an error but will slow down the script a bit.