# segfault for python importing vtk but not for vtkpython
**URL:** https://discourse.vtk.org/t/segfault-for-python-importing-vtk-but-not-for-vtkpython/126
**Category:** Support
**Created:** [January 24, 2019, 7:37pm UTC](https://discourse.vtk.org/t/segfault-for-python-importing-vtk-but-not-for-vtkpython/126 "2019-01-24T19:37:35Z")
**Posts on this page:** 12
**Page:** 1
### Author:  [@danlipsa](https://discourse.vtk.org/u/danlipsa)
#### Post date: [January 24, 2019, 7:37pm UTC](https://discourse.vtk.org/t/segfault-for-python-importing-vtk-but-not-for-vtkpython/126/1 "2019-01-24T19:37:35Z")
Hi all,
I am building VTK with the latest conda-build v3 on mac. vtkpython seems to work fine, however importing vtk from python gives me a segfault. Any ideas? (See the attached information)
Python 3.7.1 | packaged by conda-forge | (default, Nov 13 2018, 10:30:07)
[Clang 4.0.1 (tags/RELEASE\_401/final)] :: Anaconda, Inc. on darwin
Type “help”, “copyright”, “credits” or “license” for more information.
\>\>\> import vtk
Segmentation fault: 11
The last 5 stack frames are:
- frame #0: 0x0000000111486f07 libpython3.7m.dylib`PyModule_Create2 + 7 frame #1: 0x00000001113f77a8 libvtkRenderingMatplotlibPython37D-8.2.1.dylib`real\_initvtkRenderingMatplotlibPython + 24
frame #2: 0x000000010019e19d python`_PyImport_LoadDynamicModuleWithSpec + 557 frame #3: 0x000000010019d5ac python`\_imp\_create\_dynamic + 220
frame #4: 0x000000010002e1a3 python`_PyMethodDef_RawFastCallDict + 259 frame #5: 0x000000010002f871 python`PyCFunction\_Call + 65
It seems that the latest python from conda is build statically, which means there are going to be two version for the C++ library. Could this be a reason?
otool -L /Users/danlipsa/anaconda3/envs/test-vtk/bin/python
/Users/danlipsa/anaconda3/envs/test-vtk/bin/python:
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1197.1.1)
An older python from conda links with the following libraries (and that works fine for importing vtk)
(nightly) [~]$ otool -L ~/anaconda3/envs/nightly/bin/python
/Users/danlipsa/anaconda3/envs/nightly/bin/python:
/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 120.0.0)
/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 1153.18.0)
@rpath/libpython3.6m.dylib (compatibility version 3.6.0, current version 3.6.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1213.0.0)
(nightly) [~]$ otool -L ~/anaconda3/envs/nightly/bin/python
/Users/danlipsa/anaconda3/envs/nightly/bin/python:
/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 120.0.0)
/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 1153.18.0)
@rpath/libpython3.6m.dylib (compatibility version 3.6.0, current version 3.6.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1213.0.0)
---
### Author:  [@dgobbi](https://discourse.vtk.org/u/dgobbi)
#### Post date: [January 24, 2019, 11:13pm UTC](https://discourse.vtk.org/t/segfault-for-python-importing-vtk-but-not-for-vtkpython/126/2 "2019-01-24T23:13:17Z")
Try importing smaller chunks. The fact that it failed on RenderingMatplotlib suggest that it managed to load some of the other modules:
```
import vtk.vtkmodules.vtkCommonCore
etc.
```
This might also have something to do with my new code for handling run-time dependencies between the python modules. For example, if you try importing vtkComonDataModel, the new code will try to import vtkCommonCore for you, and perhaps this works differently for python vs. vtkpython. The old (VTK 8.2) code relied on the system’s dynamic linker to handle these dependencies via the libvtkCommonCore36D.dylib etc. libraries which are no longer present in VTK.
---
### Author:  [@danlipsa](https://discourse.vtk.org/u/danlipsa)
#### Post date: [January 25, 2019, 6:56pm UTC](https://discourse.vtk.org/t/segfault-for-python-importing-vtk-but-not-for-vtkpython/126/3 "2019-01-25T18:56:54Z")
Hi David,
Thanks for your answer. Strangely it fails in the same place even if I load what you suggested.
(lldb) r
Process 94986 launched: ‘/Users/danlipsa/anaconda3/envs/test/bin/python’ (x86\_64)
Python 3.7.2 (default, Dec 29 2018, 00:00:04)
[Clang 4.0.1 (tags/RELEASE\_401/final)] :: Anaconda, Inc. on darwin
Type “help”, “copyright”, “credits” or “license” for more information.
> > > import vtk.vtkmodule.vtkCommonCore
Process 94986 stopped
- thread #1, queue = ‘com.apple.main-thread’, stop reason = EXC\_BAD\_ACCESS (code=1, address=0x10)
libpython3.7m.dylib`PyModule\_Create2:
→ 0x10f1af0f7 \<+7\>: movq 0x10(%rax), %rax
```
0x10f1af0fb <+11>: cmpq $0x0, 0x28(%rax)
0x10f1af100 <+16>: je 0x10f1af107 ; <+23>
0x10f1af102 <+18>: jmp 0x10f1af120 ; _PyModule_CreateInitialized
```
Target 0: (python) stopped.
(lldb) up
frame #1: 0x000000010f11f7a8 libvtkRenderingMatplotlibPython37D-8.2.1.dylib`real\_initvtkRenderingMatplotlibPython + 24
libvtkRenderingMatplotlibPython37D-8.2.1.dylib`real\_initvtkRenderingMatplotlibPython:
```
0x10f11f7a8 <+24>: movq %rax, %r14
0x10f11f7ab <+27>: movq %r14, %rdi
0x10f11f7ae <+30>: callq 0x10f11f8be ; symbol stub for: PyModule_GetDict
0x10f11f7b3 <+35>: movq %rax, %rbx
```
(lldb) bt
- thread #1, queue = ‘com.apple.main-thread’, stop reason = EXC\_BAD\_ACCESS (code=1, address=0x10)
I should have mentioned, I am building an older VTK, before the new module system
SHA
63f048ad7a9bd7915639a4d19d0e7e70487a2333
from Dec 10. This VTK worked fine with older conda packages, but they introduced a new way of compiling packages with an embedded compiler and this is causing these issues.
I chased a few mismatched libc++ (linked either from conda or from the system libraries). Are there any python bidings related libraries/headers I should worry about.
Thanks,
Dan
---
### Author:  [@dgobbi](https://discourse.vtk.org/u/dgobbi)
#### Post date: [January 25, 2019, 7:03pm UTC](https://discourse.vtk.org/t/segfault-for-python-importing-vtk-but-not-for-vtkpython/126/4 "2019-01-25T19:03:00Z")
Dan, I made a horrid typo. Try this instead:
```
import vtkmodules.vtkCommonCore
```
I typed `import vtk.vtkmodules.xxx` which of course is going to `import vtk` before doing anything else.
---
### Author:  [@danlipsa](https://discourse.vtk.org/u/danlipsa)
#### Post date: [January 25, 2019, 8:50pm UTC](https://discourse.vtk.org/t/segfault-for-python-importing-vtk-but-not-for-vtkpython/126/5 "2019-01-25T20:50:56Z")
David,
Yes, that works! Sorry for not catching this typo! I am not used to including only individual modules.
This is what I get:
(test) [~/anaconda3/conda-bld/vtk-cdat\_1548382855937/work (context2d-contour-label-improvements %=)]$ python
Python 3.7.2 (default, Dec 29 2018, 00:00:04)
[Clang 4.0.1 (tags/RELEASE\_401/final)] :: Anaconda, Inc. on darwin
Type “help”, “copyright”, “credits” or “license” for more information.
> > > import vtkmodules.vtkCommonCore
> > > import vtkmodules.vtkCommonDataModel
> > > import vtkmodules.vtkCommonTransforms
> > > import vtkmodules.vtkImagingCore
> > > import vtkmodules.vtkPython
Traceback (most recent call last):
File “”, line 1, in
ModuleNotFoundError: No module named ‘vtkmodules.vtkPython’
> > > import vtkmodules.vtkRenderingCore
> > > import vtkmodules.vtkWrappingPythonCore
Traceback (most recent call last):
File “”, line 1, in
ModuleNotFoundError: No module named ‘vtkmodules.vtkWrappingPythonCore’
> > > import vtkmodules.vtkRenderingFreeType
> > > import vtkmodules.vtkPythonInterpreter
Traceback (most recent call last):
File “”, line 1, in
ModuleNotFoundError: No module named ‘vtkmodules.vtkPythonInterpreter’
> > > import vtkmodules.vtkRenderingMatplotlib
Segmentation fault: 11
(test) [~/anaconda3/conda-bld/
---
### Author:  [@dgobbi](https://discourse.vtk.org/u/dgobbi)
#### Post date: [January 25, 2019, 9:43pm UTC](https://discourse.vtk.org/t/segfault-for-python-importing-vtk-but-not-for-vtkpython/126/6 "2019-01-25T21:43:07Z")
All the “ModuleNotFound” errors are expected, since those aren’t wrapped modules.
So that leaves vtkRenderingMatplotlib, which is special because it links to libvtkPythonInterpreter, and is the only python-wrapped module to do so.
I suspect that you are correct that static linking is the culprit. Really vtkPythonInterpreter is the guts of the vtkpython executable, so when you load vtkRenderingMatplotlib into regular statically-linked ‘python’ you end up with two copies of libpython: the one that python links to, and the one that vtkRenderingMatplotlib.so links to. When you use vtkpython there’s no problem, because both vtkpython and vtkRenderingMatplotlib.so get libpython via libvtkPythonInterpreter.dylib, i.e. only one copy of libpython is used.
The easiest fix is to disable vtkRenderingMatplotlib. I think the only other option is to remove its dependence on vtkPythonInterpreter.
---
### Author:  [@danlipsa](https://discourse.vtk.org/u/danlipsa)
#### Post date: [January 28, 2019, 10:42pm UTC](https://discourse.vtk.org/t/segfault-for-python-importing-vtk-but-not-for-vtkpython/126/7 "2019-01-28T22:42:30Z")
Since yesterday, I dug some more and found a few more details. I initially thought that just using vtkpython instead of python would solve our problem.
All tests pass fine, but we just realized that ipython would not work using vtkpython instead of python?
Do you know why that would be the case? I thought python and vtkpython are very similar: just some paths added to find VTK.
I am getting:
Python 3.6.7 | packaged by conda-forge | (default, Nov 20 2018, 18:39:03)
Type ‘copyright’, ‘credits’ or ‘license’ for more information
IPython 7.2.0 – An enhanced Interactive Python. Type ‘?’ for help.
In [1]: import sys
File “”, line 1
```
import
^
```
SyntaxError: invalid syntax
In [2]: ---------------------------------------------------------------------------
NameError Traceback (most recent call last)
in
----\> 1 sys
NameError: name ‘sys’ is not defined
In [3]:
> All the “ModuleNotFound” errors are expected, since those aren’t wrapped modules.
> So that leaves vtkRenderingMatplotlib, which is special because it links to libvtkPythonInterpreter, and is the only python-wrapped module to do so.
> I suspect that you are correct that static linking is the culprit. Really vtkPythonInterpreter is the guts of the vtkpython executable, so when you load vtkRenderingMatplotlib into regular statically-linked ‘python’ you end up with two copies of libpython: the one that python links to, and the one that vtkRenderingMatplotlib.so links to. When you use vtkpython there’s no problem, because both vtkpython and vtkRenderingMatplotlib.so get libpython via libvtkPythonInterpreter.dylib, i.e. only one copy of libpython is used.
Yes, that seems to be the case. Interestingly, python2 links dynamically with libpython.
The same happens on Linux (dynamic for python2 and static for python3), however VTK mathplotlib works fine on linux python3.
For python2 it works on both linux and mac.
> The easiest fix is to disable vtkRenderingMatplotlib.
Unfortunately we would loose math text which we would like to keep.
> I think the only other option is to remove its dependence on vtkPythonInterpreter.
Not sure how would this work. Can you describe this in more detail?
Thank you very much for your insights!
Dan
---
### Author:  [@dgobbi](https://discourse.vtk.org/u/dgobbi)
#### Post date: [January 28, 2019, 11:08pm UTC](https://discourse.vtk.org/t/segfault-for-python-importing-vtk-but-not-for-vtkpython/126/8 "2019-01-28T23:08:05Z")
I’m actually not sure how vtkPythonInterpreter could be removed from vtkRenderingMatplotlib. Allison Vacanti might know of a way, but in any case it wouldn’t be easy and might not be possible.
A third option is to link weakly to libpython, which basically means that all Python API symbols would end up linking to the running python interpreter. This isn’t easy either, though (if it was easy we already would have done it, see [https://gitlab.kitware.com/vtk/vtk/issues/17214](https://gitlab.kitware.com/vtk/vtk/issues/17214)).
---
### Author:  [@dgobbi](https://discourse.vtk.org/u/dgobbi)
#### Post date: [January 28, 2019, 11:44pm UTC](https://discourse.vtk.org/t/segfault-for-python-importing-vtk-but-not-for-vtkpython/126/9 "2019-01-28T23:44:25Z")
Actually, VTK might already have some of the magic needed for weak linking. For example, the `module.cmake` for Utilities/PythonInterpreter has the `OPTIONAL_PYTHON_LINK` option, but the `module.cmake` for Rendering/Matplotlib does not. Perhaps when `OPTIONAL_PYTHON_LINK` was added to VTK, the Matplotlib module was simply overlooked? The relevant MR is here:
[https://gitlab.kitware.com/vtk/vtk/merge\_requests/1713](https://gitlab.kitware.com/vtk/vtk/merge_requests/1713)
---
### Author:  [@danlipsa](https://discourse.vtk.org/u/danlipsa)
#### Post date: [January 29, 2019, 1:38pm UTC](https://discourse.vtk.org/t/segfault-for-python-importing-vtk-but-not-for-vtkpython/126/10 "2019-01-29T13:38:57Z")
Thanks for the great pointers David. We’ll look into it.
Dan
---
### Author:  [@danlipsa](https://discourse.vtk.org/u/danlipsa)
#### Post date: [February 8, 2019, 8:42pm UTC](https://discourse.vtk.org/t/segfault-for-python-importing-vtk-but-not-for-vtkpython/126/11 "2019-02-08T20:42:45Z")
Hi David,
We were able to temporarily fix our problem by using OPTIONAL\_PYTHON\_LINK and removing libpython from the link command line for
vtkRenderingMatplotlib, vtkRenderingMatplotlibPython, vtkPythonInterpreter
We are going to look into adding this to VTK master. For now, I built those libraries by hand. Also, likely, there is another step needed ( = loading libpython if none is provided) to make things work from both C++ and python. With the current solution they only work for python - which is what we care about in our project.
We’ll post an update once we have a fix for VTK.
Thanks,
Dan
---
### Author:  [@danlipsa](https://discourse.vtk.org/u/danlipsa)
#### Post date: [April 29, 2019, 2:16pm UTC](https://discourse.vtk.org/t/segfault-for-python-importing-vtk-but-not-for-vtkpython/126/12 "2019-04-29T14:16:32Z")
Hi all,
We have an update on VTK\_PYTHON\_OPTIONAL\_LINK.
To summarize the issue. Recently conda linked python3 statically, so all python symbols are included in the executable instead of being brought in by libpython. This created a problem with VTK used from python, because VTK links with libpython (it uses matplotlib for math text). So, you had python code brought in by the pyhton executable and by libpyton which resulted in a segfault for tests that used python and math text.
In the new module system, with Ben’s work, there is an option called VTK\_PYTHON\_OPTIONAL\_LINK that is disabled by default. Patching VTK to enable this option and using cmake \>= 3.13 would enable this option and compile VTK so that it works correctly with a python statically linked with libpython.
The reason why this option is still disable is that it does not work correctly with ParaView libraries because (we believe) more cleanup is needed there.
However, I used the patch VTK in our project using conda and statically linked python3 it works correctly.
Dan