# More pythonic interface / closer to numpy
**URL:** https://discourse.vtk.org/t/more-pythonic-interface-closer-to-numpy/7905
**Category:** Development
**Tags:** python, proposal
**Created:** [February 23, 2022, 5:15pm UTC](https://discourse.vtk.org/t/more-pythonic-interface-closer-to-numpy/7905 "2022-02-23T17:15:30Z")
**Posts on this page:** 13
**Page:** 1
### Author:  [@berk.geveci](https://discourse.vtk.org/u/berk.geveci)
#### Post date: [February 23, 2022, 5:15pm UTC](https://discourse.vtk.org/t/more-pythonic-interface-closer-to-numpy/7905/1 "2022-02-23T17:15:30Z")
Hi everyone,
I would like to propose some improvements to VTK both at C++ and Python level with the following goals:
1. Convenient interface between the VTK data model and numpy, pandas etc. (wrapper changes -done-, python modules)
2. More pythonic access to VTK objects through things like properties. (wrapper changes)
3. A few convenience methods to make it easier to use VTK filters without hooking up a pipeline (what I call the _imperative_ mode). (C++ changes mostly)
4. A high level interface to rendering based on representations and views similar to that of ParaView (C++ changes)
I have a basic prototype that does some of 1-3 [here](https://gitlab.kitware.com/berkgeveci/pythonic-vtk). Note that I prototyped 2 and 3 in python even though much would be done at C++ / wrapper layer in the final implementation. I would love to hear feedback and ideas about all of these items and volunteers for the work would be great too :-). In this proposal, I would like to concentrate on item 1. I will create separate proposals for 2-4 in the future.
There is already a module that provides numpy friendly dataset classes:
```python
from vtkmodules.numpy_interface import dataset_adapter as dsa
from vtkmodules.vtkImagingCore import vtkRTAnalyticSource
rt = vtkRTAnalyticSource()
rt.Update()
image = dsa.WrapDataObject(rt.GetOutput())
rtdata = image.PointData['RTData'] # rtdata is a numpy array compatible class
import vtkmodules.numpy_interface.algorithms as algs
rtmin = algs.min(rtdata) # wrappers around numpy algorithms
rtmax = algs.max(rtdata)
rtnorm = (rtdata - rtmin) / (rtmax - rtmin)
image.PointData.append(rtnorm, 'RTData - normalized')
```
This module also handles composite data and provides the same interface:
```python
from vtkmodules.vtkCommonDataModel import vtkMultiBlockDataSet
mb = vtkMultiBlockDataSet()
mb.SetBlock(0, image.VTKObject)
mb.SetBlock(1, image.VTKObject)
cds = dsa.WrapDataObject(mb)
rtdatac = cds.PointData['RTData'] # VTKCompositeDataArray which manages multiple arrays under the covers
rtmin = algs.min(rtdatac)
```
This interface goes beyond simply providing numpy access. It supports:
- Single datasets,
- Composite datasets and arrays,
- Algorithms that works over all of the data model and in distributed parallel.
These are used by ParaView’s python algorithms and filters.
IMO this is a fairly good interface and is pretty rich in functionality. However, it is not easy to discover, is not well integrated and could be more pythonic in the API.
In order to achieve these goals, David Gobbi and I made some changes to VTK wrapping. These enable one to override VTK wrapped classes with Python subclasses like this:
```python
from vtkmodules.vtkCommonCore import vtkPoints
@vtkPoints.override
class vtkPointsCustom(vtkPoints):
pass
p = vtkPoints() # p is actually a vtkPointsCustom instance
```
With this functionality in place (already merged to master, see [this MR](https://gitlab.kitware.com/vtk/vtk/-/merge_requests/8886)), we can take the existing functionality further:
```python
from vtkmodules.vtkImagingCore import vtkRTAnalyticSource
import datamodel
rt = vtkRTAnalyticSource()
rt.Update()
image = rt.GetOutput() # image is actually datamode.Image which has numpy accessors
rtdata2 = image.point_data["RTData"] * 2
image.point_data["rtdata2"] = rtdata2
```
We can also do things like this:
```python
image = datamodel.ImageData(dimensions=(10, 10, 10), spacing=(0.1, 0.1, 0.1), origin=(2,2,2))
cs = numpy.linspace(2, 2 + 9 * 0.1, 10)
x, y, z = numpy.meshgrimage(cs, cs, cs)
values = 30.0*numpy.sin(x*3)*numpy.cos(y*3)*numpy.cos(z)
image.point_data['scalar'] = values.flatten(order='F')
```
Note that ideally, the constructor would be `vtkImageData` not `ImageData`. This needs changes to the wrappers, which we could do in the future.
The prototype that I provided supports `vtkImageData`, `vtkPartitionedDataSet` (including iterating over partitions) and arrays/attributes. Extending it to all of VTK’s data model is fairly easy since the `dataset_adapter` module handles most of the complications.
So what do you think?
---
### Author:  [@will.schroeder](https://discourse.vtk.org/u/will.schroeder)
#### Post date: [February 23, 2022, 5:33pm UTC](https://discourse.vtk.org/t/more-pythonic-interface-closer-to-numpy/7905/2 "2022-02-23T17:33:25Z")
This looks really cool. Maybe I missed it, but what does an example VTK python script using imperative mode look like?
---
### Author:  [@Sebastien\_Jourdain](https://discourse.vtk.org/u/Sebastien_Jourdain)
#### Post date: [February 23, 2022, 5:40pm UTC](https://discourse.vtk.org/t/more-pythonic-interface-closer-to-numpy/7905/3 "2022-02-23T17:40:02Z")
Having example on how to define points and cells for various dataset type would be great.
All the field API seems great and I love it.
---
### Author:  [@amaclean](https://discourse.vtk.org/u/amaclean)
#### Post date: [February 23, 2022, 9:39pm UTC](https://discourse.vtk.org/t/more-pythonic-interface-closer-to-numpy/7905/4 "2022-02-23T21:39:07Z")
What a great idea.
We do actually use it in some of the vtk-examples e.g. [CurvaturesAdjustEdges](https://kitware.github.io/vtk-examples/site/Cxx/PolyData/CurvaturesAdjustEdges/), [VTKWithNumpy](https://kitware.github.io/vtk-examples/site/Python/Utilities/VTKWithNumpy/)
The only reason that I know of the interface is through these articles starting with: [Improved VTK – numpy integration](https://www.kitware.com/improved-vtk-numpy-integration/), there are five posts in the series.
I would be glad to help. If anyone wants to rough out examples I can easily enter them into the `vtk-examples`.
---
### Author:  [@mwestphal](https://discourse.vtk.org/u/mwestphal)
#### Post date: [February 24, 2022, 9:06am UTC](https://discourse.vtk.org/t/more-pythonic-interface-closer-to-numpy/7905/5 "2022-02-24T09:06:14Z")
I like the override mechanism.
Regarding numpyfication, let’s keep in ming that numpy is an optional dependency of VTK.
---
### Author:  [@Timothee\_Chabat](https://discourse.vtk.org/u/Timothee_Chabat)
#### Post date: [February 24, 2022, 9:23am UTC](https://discourse.vtk.org/t/more-pythonic-interface-closer-to-numpy/7905/6 "2022-02-24T09:23:05Z")
> [@berk.geveci](#):
>
> - A few convenience methods to make it easier to use VTK filters without hooking up a pipeline (what I call the _imperative_ mode). (C++ changes mostly)
> - A high level interface to rendering based on representations and views similar to that of ParaView (C++ changes)
I’m not much of a python guy but I’m really thrilled by these 2 changes. However I hope that this representation layer will not bloat the rendering part of VTK too much. This is only my personal opinion but :
- representation in ParaView are cool but lacks of clarity. For exemple the geometry representation takes care of way too many different representations and use cases.
- I find diving into the rendering code of VTK very daunting every time. This might be out of scope of this proposition but the whole rendering layer could be redesigned and simplified so it is easier to use for both developers and users.
---
### Author:  [@ken-martin](https://discourse.vtk.org/u/ken-martin)
#### Post date: [February 24, 2022, 3:13pm UTC](https://discourse.vtk.org/t/more-pythonic-interface-closer-to-numpy/7905/7 "2022-02-24T15:13:56Z")
I know (3) was one of the goals of the pipeline re architecture ages ago. While no one packaged it up, the idea was that with the new APIs you could in vtkAlgorithm define a function that calls the needed requestFoo methods under the hood and present the user with an immediate functional API like
filter-\>GenerateData(inputDatas, outputDatas);
with the user not required to know about or create any pipelines. I think this is still possible today without a lot of code. So the more full example would be something like:
```auto
wave = vtkWavelet()
wave.GenerateData(waveData);
iso = vtkIsosurace();
iso.SetIsoValues(2,3,4);
iso.GenerateData(waveData, isoData);
writer = vtkOBJWriter();
writer.SetFileName("foo.obj");
writer.WriteData(isoData);
```
---
### Author:  [@berk.geveci](https://discourse.vtk.org/u/berk.geveci)
#### Post date: [February 24, 2022, 6:52pm UTC](https://discourse.vtk.org/t/more-pythonic-interface-closer-to-numpy/7905/8 "2022-02-24T18:52:20Z")
I am going to have another proposal for (3) but since we started discussing it here is what I prototyped:
```python
# Create an image and contour
id = ImageData(spacing=(0.1, 0.1, 0.1), origin=(2,2,2))
id.dimensions = (10, 10, 10)
cs = numpy.linspace(2, 2 + 9 * 0.1, 10)
x, y, z = numpy.meshgrid(cs, cs, cs)
values = 30.0*numpy.sin(x*3)*numpy.cos(y*3)*numpy.cos(z)
id.point_data['scalar'] = values.flatten(order='F')
ct = Contour(contour_values=(-10, 0, 10), scalar_name='scalar').execute(id)
```
So essentially as simple as adding some kind of execute() method that takes input(s), request parameters (such as extent optionally) and returns output(s). Being able to define properties in the constructor makes it possible to have one liners where the filter is temporary. This is just a prototype where I wanted to explore the concept. I didn’t spend much time worrying about the API.
---
### Author:  [@berk.geveci](https://discourse.vtk.org/u/berk.geveci)
#### Post date: [February 24, 2022, 6:54pm UTC](https://discourse.vtk.org/t/more-pythonic-interface-closer-to-numpy/7905/9 "2022-02-24T18:54:16Z")
> Regarding numpyfication, let’s keep in ming that numpy is an optional dependency of VTK.
Yes totally. The new module would depend on numpy like the dataset\_adapter module and would return an error if numpy is not available. Having said that I expect that we will add more and more functionality in python that depend on core python science modules like numpy, scipy and pandas. There is not a whole lot you can do in python without those unless you are willing to write python loops that are very slow.
---
### Author:  [@cory.quammen](https://discourse.vtk.org/u/cory.quammen)
#### Post date: [February 25, 2022, 1:47pm UTC](https://discourse.vtk.org/t/more-pythonic-interface-closer-to-numpy/7905/10 "2022-02-25T13:47:39Z")
With regards to Pythonic API, we need to engage people in the Python community to see what interface is Pythonic to them. In my mind, the API would avoid carrying over artifacts from VTK’s execution model, e.g., the explicit `execute()` function in your example.
---
### Author:  [@berk.geveci](https://discourse.vtk.org/u/berk.geveci)
#### Post date: [February 25, 2022, 3:10pm UTC](https://discourse.vtk.org/t/more-pythonic-interface-closer-to-numpy/7905/11 "2022-02-25T15:10:54Z")
Yeah I agree. That was something that I did to quickly show something. Do note that I am actually creating a filter in that example and calling execute() on it. I am strongly against a purely functional API as it would not jive with VTK’s spirit.
I am hoping that we have enough python folks here to get some feedback (at least once we start that topic separately 🙂 ). If not, I am open to suggestions on where to reach out.
---
### Author:  [@banesullivan](https://discourse.vtk.org/u/banesullivan)
#### Post date: [February 27, 2022, 9:36pm UTC](https://discourse.vtk.org/t/more-pythonic-interface-closer-to-numpy/7905/12 "2022-02-27T21:36:36Z")
Your 4 listed goals are exactly PyVista’s offerings. [https://docs.pyvista.org/](https://docs.pyvista.org/)
---
### Author:  [@banesullivan](https://discourse.vtk.org/u/banesullivan)
#### Post date: [February 28, 2022, 6:32am UTC](https://discourse.vtk.org/t/more-pythonic-interface-closer-to-numpy/7905/13 "2022-02-28T06:32:39Z")
Here is the proposed example, except in working PyVista code:
```python
import pyvista as pv
import numpy as np
# Create an image and contour
image = pv.UniformGrid(dims=(10, 10, 10),
spacing=(0.1, 0.1, 0.1),
origin=(2,2,2)
)
# Add some data computed with NumPy
cs = np.linspace(2, 2 + 9 * 0.1, 10)
x, y, z = np.meshgrid(cs, cs, cs)
values = 30.0 * np.sin(x * 3) * np.cos(y * 3) * np.cos(z)
image.point_data['scalar'] = values.flatten(order='F')
# Apply a Contour filter
ct = image.contour(isosurfaces=(-10, 0, 10), scalars='scalar')
# Plot it, simply
ct.plot()
```
This handles data generation, filtering, and plotting.
Furthermore, it looks quite nice in Jupyter with rich outputs:
And interactive plotting (options for either server rendering or client rendering).
* * *
For completeness, a `numpy.meshgrid` is better represented as a `vtkStructuredGrid` as in many cases, a `meshgrid` can be curvilinear. Here is that example with a structured grid:
```auto
import pyvista as pv
import numpy as np
# Add some data computed with NumPy
cs = np.linspace(2, 2 + 9 * 0.1, 10)
x, y, z = np.meshgrid(cs, cs, cs)
values = 30.0 * np.sin(x * 3) * np.cos(y * 3) * np.cos(z)
# Use vtkStructuredGrid instead
mesh = pv.StructuredGrid(x,y,z)
mesh.point_data['scalar'] = values.flatten(order='F')
# Apply a Contour filter
ct = mesh.contour(isosurfaces=(-10, 0, 10), scalars='scalar')
# Plot it, simply
ct.plot()
```