# The world coordinate is changed when the shown image is rotated.

**URL:** https://discourse.vtk.org/t/the-world-coordinate-is-changed-when-the-shown-image-is-rotated/5879
**Category:** Support
**Created:** [May 28, 2021, 12:06pm UTC](https://discourse.vtk.org/t/the-world-coordinate-is-changed-when-the-shown-image-is-rotated/5879 "2021-05-28T12:06:18Z")
**Posts on this page:** 14
**Page:** 1

<div class="post-metadata">

### Author: ![zhang-qiang-github](https://discourse.vtk.org/user_avatar/discourse.vtk.org/zhang-qiang-github/32/2519_2.png) [@zhang-qiang-github](https://discourse.vtk.org/u/zhang-qiang-github)
#### Post date: [May 28, 2021, 12:06pm UTC](https://discourse.vtk.org/t/the-world-coordinate-is-changed-when-the-shown-image-is-rotated/5879/1 "2021-05-28T12:06:18Z")

</div>

I want to know how to calculate the image pixel coordinate fro world coordinate. The code of my experiment is:

```auto
import vtk
import numpy as np
from vtk.util.numpy_support import numpy_to_vtk
def numpyToVTK(data, multi_component=False, type='float'):
    if type == 'float':
        data_type = vtk.VTK_FLOAT
    elif type == 'char':
        data_type = vtk.VTK_UNSIGNED_CHAR
    else:
        raise RuntimeError('unknown type')
    if multi_component == False:
        if len(data.shape) == 2:
            data = data[:, :, np.newaxis]
        flat_data_array = data.transpose(2,1,0).flatten()
        vtk_data = numpy_to_vtk(num_array=flat_data_array, deep=True, array_type=data_type)
        shape = data.shape
    else:
        assert len(data.shape) == 3, 'only test for 2D RGB'
        flat_data_array = data.transpose(1, 0, 2)
        flat_data_array = np.reshape(flat_data_array, newshape=[-1, data.shape[2]])
        vtk_data = numpy_to_vtk(num_array=flat_data_array, deep=True, array_type=data_type)
        shape = [data.shape[0], data.shape[1], 1]
    img = vtk.vtkImageData()
    img.GetPointData().SetScalars(vtk_data)
    img.SetDimensions(shape[0], shape[1], shape[2])
    return img
global sphereActor, textActor
sphereActor = None
textActor = None
def mouseMoveEvent(iren, event):
    x, y = iren.GetEventPosition()

    picker = vtk.vtkWorldPointPicker()
    picker.Pick(x, y, 0, render)
    worldPoint = picker.GetPickPosition()

    sphere = vtk.vtkSphereSource()
    sphere.SetCenter(worldPoint[0], worldPoint[1], worldPoint[2])
    sphere.SetRadius(2)
    sphere.Update()
    sphereMapper = vtk.vtkPolyDataMapper()
    sphereMapper.SetInputData(sphere.GetOutput())
    global sphereActor, textActor
    if sphereActor != None:
        render.RemoveActor(sphereActor)
    sphereActor = vtk.vtkActor()
    sphereActor.SetMapper(sphereMapper)
    sphereActor.GetProperty().SetColor(255, 0, 0)
    render.AddActor(sphereActor)
    render.Render()

    if textActor != None:
        render.RemoveActor(textActor)
    textActor = vtk.vtkTextActor()
    textActor.SetInput('world coordinate: (%.2f, %.2f, %.2f)'%(worldPoint[0], worldPoint[1], worldPoint[2]))
    textActor.GetTextProperty().SetColor(1, 0, 0)
    textActor.GetTextProperty().SetFontSize(15)
    render.AddActor(textActor)

    print('屏幕坐标：', (x, y), '世界坐标: ', worldPoint)

img = np.zeros(shape=[128, 128])
for i in range(128):
    for j in range(128):
        img[i, j] = i+j

vtkImg = numpyToVTK(img)
imgActor = vtk.vtkImageActor()
imgActor.SetInputData(vtkImg)

render = vtk.vtkRenderer()
render.AddActor(imgActor)
# render.Render()

renWin = vtk.vtkRenderWindow()
renWin.AddRenderer(render)
renWin.Render()
iren = vtk.vtkRenderWindowInteractor()
iren.SetRenderWindow(renWin)
iren.SetInteractorStyle(vtk.vtkInteractorStyleTrackballCamera())
iren.Initialize()
iren.AddObserver('MouseMoveEvent', mouseMoveEvent)
iren.Start()

```

The used vtk is 9.0.1.

1. My first question is: why the red sphere don’t moved along with my mouse? The red sphere will update only when the mouse is stop. Does the render() cost much time?

2. My second quesion is: the display image is converted from numpy, and the origin should be (0, 0, 0), the direction matrix should be: x(1, 0, 0), y(0, 1, 0), z(0, 0, 1). I think the z value of the world point in the display image should be 0. When the image is not rotated, the world point is actually (x, y, 0)

Any suggestion is appreciated!

---

<div class="post-metadata">

### Author: ![marcomusy](https://discourse.vtk.org/user_avatar/discourse.vtk.org/marcomusy/32/95_2.png) [@marcomusy](https://discourse.vtk.org/u/marcomusy)
#### Post date: [May 29, 2021, 12:33am UTC](https://discourse.vtk.org/t/the-world-coordinate-is-changed-when-the-shown-image-is-rotated/5879/2 "2021-05-29T00:33:36Z")

</div>

1. I guess you dont need to recreate the sphere at each call, this can explain why it’s slow
2. no idea why, but indeed it is non zero… maybe a precision issue?

You can play around with this, moving the mouse drags the sphere in real time with no much delay:

```python
import numpy as np
from vedo import *

img = np.zeros((128,128))
for i in range(128):
    for j in range(128):
        img[j,i] = i+j

def func(evt):
    if evt.picked3d is None:
        return
    nr = np.floor(np.array(evt.picked3d)).astype(int)
    txt.text(f"屏幕坐: {evt.picked2d}\n世界坐: {evt.picked3d}\n{nr[:2]}")
    sph.pos(evt.picked3d)
    plt.render()

plt = Plotter(axes=1)
pic = Picture(img)
sph = Sphere(r=2).pickable(False)
txt = Text2D(font="LogoType")
plt.addCallback("MouseMoveEvent", func)
plt.show(pic, sph, txt)

```

 ![Screenshot from 2021-05-29 02-22-26](https://discourse.vtk.org/uploads/default/original/2X/4/4efdc7519a3344e8a736272324e80dfdf7a58eb1.png)

---

<div class="post-metadata">

### Author: ![zhang-qiang-github](https://discourse.vtk.org/user_avatar/discourse.vtk.org/zhang-qiang-github/32/2519_2.png) [@zhang-qiang-github](https://discourse.vtk.org/u/zhang-qiang-github)
#### Post date: [May 29, 2021, 1:22am UTC](https://discourse.vtk.org/t/the-world-coordinate-is-changed-when-the-shown-image-is-rotated/5879/3 "2021-05-29T01:22:01Z")

</div>

@marcomusy Thank you for your kindly reply.

What’s your vedo version? It report some bug for my current version (vedo 2020.4.2)

1. `Picture` function requre a image with 3 component.

```auto
Traceback (most recent call last):
  File "C:/Users/MLoong/Desktop/test/test.py", line 18, in <module>
    pic = Picture(img)
  File "D:\Anaconda3\envs\general\lib\site-packages\vedo\picture.py", line 36, in __init__
    nchan = obj.shape[2] # get number of channels in inputimage (L/LA/RGB/RGBA)
IndexError: tuple index out of range

```

I modify the code as:

```auto
img = np.zeros((128,128,1))
for i in range(128):
    for j in range(128):
        img[j,i,0] = i+j

```

1. bug for `Text2D`:

```auto
Traceback (most recent call last):
  File "C:/Users/MLoong/Desktop/test/test.py", line 20, in <module>
    txt = Text2D(font="LogoType")
TypeError: Text2D() missing 1 required positional argument: 'txt'

```

I modify the code as: `txt = Text2D('txt', font="LogoType")`.

1. bug for `func` as: `TypeError: func() takes 1 positional argument but 2 were given`. I change the definition from `def func(evt)` to `def func(evt, p)`.

2. bug for `evt.picked3d` as following:

```auto
Traceback (most recent call last):
  File "C:/Users/MLoong/Desktop/test/test.py", line 10, in func
    if evt.picked3d is None:
AttributeError: 'vtkmodules.vtkRenderingUI.vtkWin32RenderWindowInte' object has no attribute 'picked3d'

```

and I don’t know how to fix this bug.

Moreover, why the z value of a world point is not zero? In my thought, the conversion between pixel index (i, j, z) and world point (x, y, z) should be:

```auto
worldPoint (x,y,z) = i*spacingX*directionX + j*spacingY*directionY + k*spacingZ*directionZ + originPoint

```

Now, the direction should be:

```auto
directionX = [1, 0, 0]
directionY = [0, 1, 0]
directionZ = [0, 0, 1]

```

and `originPoint=[0, 0, 0]`, `spacingX=1`, `spacingY=1`, `spacingZ=1`. Thus, the conversion should be:

```auto
x = i
y = j
z = k

```

Since the image in this code is converted from numpy, `k` should be 0.

I am confuzed that the world point is (20.43, 29.55, -0.18), why the z value is not 0.

![image](https://discourse.vtk.org/uploads/default/original/2X/c/c4038e72fc3bbcef32ce969cf0728a1bdac1f985.png)

And I don’t think it is a precision issue. It is a huge difference between 0 and -0.18.

On the other hand, when the image is not rotated, the z value is 0. While the image is rotated, the z value is not 0. But, I think the world point should be not related with the image rotation.

---

<div class="post-metadata">

### Author: ![marcomusy](https://discourse.vtk.org/user_avatar/discourse.vtk.org/marcomusy/32/95_2.png) [@marcomusy](https://discourse.vtk.org/u/marcomusy)
#### Post date: [May 29, 2021, 9:42am UTC](https://discourse.vtk.org/t/the-world-coordinate-is-changed-when-the-shown-image-is-rotated/5879/4 "2021-05-29T09:42:26Z")

</div>

Just update with `pip install vedo -U`. If that doesn’t do the job please open an issue on github.

If it’s world coordinates you dont need to make conversions apart from a simple scaling  
(check out the example).

I don’t know about the z not being zero maybe someone here has a better answer.

---

<div class="post-metadata">

### Author: ![zhang-qiang-github](https://discourse.vtk.org/user_avatar/discourse.vtk.org/zhang-qiang-github/32/2519_2.png) [@zhang-qiang-github](https://discourse.vtk.org/u/zhang-qiang-github)
#### Post date: [May 29, 2021, 2:47pm UTC](https://discourse.vtk.org/t/the-world-coordinate-is-changed-when-the-shown-image-is-rotated/5879/5 "2021-05-29T14:47:25Z")

</div>

@marcomusy Do you know how to acquire the image intensity according to the world point?

---

<div class="post-metadata">

### Author: ![dgobbi](https://discourse.vtk.org/user_avatar/discourse.vtk.org/dgobbi/32/18_2.png) [@dgobbi](https://discourse.vtk.org/u/dgobbi)
#### Post date: [June 1, 2021, 2:28pm UTC](https://discourse.vtk.org/t/the-world-coordinate-is-changed-when-the-shown-image-is-rotated/5879/6 "2021-06-01T14:28:14Z")

</div>

I recommend using [vtkImageInterapolator](https://vtk.org/doc/nightly/html/classvtkImageInterpolator.html) (also see base class [vtkAbstractImageInterpolator](https://vtk.org/doc/nightly/html/classvtkAbstractImageInterpolator.html)) to get the image intensity at a point.

```python
interp = vtk.vtkImageInterpolator()
interp.Initialize(img)
interp.SetInterpolationModeToLinear()
# some images are multi-component
value = [0.0] * interp.GetNumberOfComponents()
# the xs, ys, zs here are in continuous structured coordinates
interp.InterpolateIJK((xs, ys, zs), value)
print(value)

```

To convert from image data physical coords to structured coords, you can use  
`img.TransformPhysicalPointToContinuousIndex()`.

If you use [vtkCellPicker](https://vtk.org/doc/nightly/html/classvtkCellPicker.html) for picking, then you can use `picker.GetMapperPosition()` to get the pick position in image physical coords. Otherwise, you have to do the world\_coords to image\_physical\_coords transformation yourself (the transformation is identity unless you have repositioned the vtkImageActor).

Also note that the vtkCellPicker will be more precise than vtkWorldPointPicker, because the vtkWorldPointPicker is limited by the precision of the GPU depth buffer.

---

<div class="post-metadata">

### Author: ![zhang-qiang-github](https://discourse.vtk.org/user_avatar/discourse.vtk.org/zhang-qiang-github/32/2519_2.png) [@zhang-qiang-github](https://discourse.vtk.org/u/zhang-qiang-github)
#### Post date: [June 1, 2021, 3:13pm UTC](https://discourse.vtk.org/t/the-world-coordinate-is-changed-when-the-shown-image-is-rotated/5879/7 "2021-06-01T15:13:16Z")

</div>

Thank you for your kindly reply, but I still have two questions:

1. Why the picker point is outside of image bounds? In the above figure, I have picked a world point (20.43, 29.55, -0.18). But the image bounds is: `(0, 127, 0, 127, 0, 0)`.

2. In this code, the `TransformPhysicalPointToContinuousIndex` retuan the same value for voxel coordinate with world coordinate. The `TransformPhysicalPointToContinuousIndex` return (20.43,29.55,-0.18) for the voxel coordinate, but the size of image is: (128,128,1). Why the z-value is -0.18?

---

<div class="post-metadata">

### Author: ![dgobbi](https://discourse.vtk.org/user_avatar/discourse.vtk.org/dgobbi/32/18_2.png) [@dgobbi](https://discourse.vtk.org/u/dgobbi)
#### Post date: [June 1, 2021, 3:35pm UTC](https://discourse.vtk.org/t/the-world-coordinate-is-changed-when-the-shown-image-is-rotated/5879/8 "2021-06-01T15:35:52Z")

</div>

The z-value of 0.18 is probably due to the limited precision of the depth buffer, which is used by vtkWorldPointPicker. That is why I suggested trying vtkCellPicker to see if it gives a more precise result.

I’m sure you already know why TransformPhysicalPointToContinuousIndex() returns the same value. It is because the spacing is [1,1,1], the origin is [0,0,0] and the direction is the identity matrix. Under these circumstances, do you expect TransformPhysicalPointToContinuousIndex() to give a different answer?

---

<div class="post-metadata">

### Author: ![marcomusy](https://discourse.vtk.org/user_avatar/discourse.vtk.org/marcomusy/32/95_2.png) [@marcomusy](https://discourse.vtk.org/u/marcomusy)
#### Post date: [June 1, 2021, 10:21pm UTC](https://discourse.vtk.org/t/the-world-coordinate-is-changed-when-the-shown-image-is-rotated/5879/9 "2021-06-01T22:21:21Z")

</div>

maybe the `vtkImageActorPointPlacer` is also a relevant class?  
[https://vtk.org/doc/nightly/html/classvtkImageActorPointPlacer.html#details](https://vtk.org/doc/nightly/html/classvtkImageActorPointPlacer.html#details)

---

<div class="post-metadata">

### Author: ![zhang-qiang-github](https://discourse.vtk.org/user_avatar/discourse.vtk.org/zhang-qiang-github/32/2519_2.png) [@zhang-qiang-github](https://discourse.vtk.org/u/zhang-qiang-github)
#### Post date: [June 2, 2021, 12:19am UTC](https://discourse.vtk.org/t/the-world-coordinate-is-changed-when-the-shown-image-is-rotated/5879/10 "2021-06-02T00:19:04Z")

</div>

I have tried the `vtkCellPicker` and `GetMapperPosition`:

```auto
    # picker = vtk.vtkWorldPointPicker()
    picker = vtk.vtkCellPicker()
    picker.Pick(x, y, 0, render)
    # worldPoint = picker.GetPickPosition()
    worldPoint = picker.GetMapperPosition()

```

And the result is:

![image](https://discourse.vtk.org/uploads/default/original/2X/8/8f22d2c49a358eb5933cc0cfac6a6caee54e7e6b.png)

The obtained world point is still wrong.

Moreover, even the z-value is 0.18, if I use `TransformPhysicalPointToContinuousIndex` to obtain the voxel coordinate (i, j, k), is the cooridiante i/j correct?

---

<div class="post-metadata">

### Author: ![dgobbi](https://discourse.vtk.org/user_avatar/discourse.vtk.org/dgobbi/32/18_2.png) [@dgobbi](https://discourse.vtk.org/u/dgobbi)
#### Post date: [June 2, 2021, 1:26am UTC](https://discourse.vtk.org/t/the-world-coordinate-is-changed-when-the-shown-image-is-rotated/5879/11 "2021-06-02T01:26:54Z")

</div>

That result does not make sense to me. Can you show me the original (x,y) screen position, the vtkWorldPointPicker.GetPickPosition(), the vtkCellPicker.GetPickPosition(), and vtkCellPicker.GetMapperPosition() all together so that I can compare them?

As I stated above, for your situation, TransformPhysicalPointToContinuousIndex() will just be the identity transformation. It does not do any clamping or anything like that, it just transforms the point based on the spacing, origin, and direction.

---

<div class="post-metadata">

### Author: ![zhang-qiang-github](https://discourse.vtk.org/user_avatar/discourse.vtk.org/zhang-qiang-github/32/2519_2.png) [@zhang-qiang-github](https://discourse.vtk.org/u/zhang-qiang-github)
#### Post date: [June 2, 2021, 11:54am UTC](https://discourse.vtk.org/t/the-world-coordinate-is-changed-when-the-shown-image-is-rotated/5879/12 "2021-06-02T11:54:59Z")

</div>

@dgobbi I modify the `mouseMoveEvent` as:

```auto

    x, y = iren.GetEventPosition()

    worldPicker = vtk.vtkWorldPointPicker()
    worldPicker.Pick(x, y, 0, render)
    worldPickPoint = worldPicker.GetPickPosition()

    cellPicker = vtk.vtkCellPicker()
    cellPicker.Pick(x, y, 0, render)
    cellPickPoint = cellPicker.GetPickPosition()
    cellMapperPickPoint = cellPicker.GetMapperPosition()

    ...

    textActor = vtk.vtkTextActor()
    textActor.SetInput('screen pos: (%.1f, %.1f)\n'
                       'vtkWorldPointPicker position: (%.3f, %.3f, %.3f)\n'
                       'vtkCellPicker GetPickPosition: (%.3f, %.3f, %.3f)\n'
                       'vtkCellPicker GetMapperPosition: (%.3f, %.3f, %.3f)'%(x, y, worldPickPoint[0], worldPickPoint[1], worldPickPoint[2], cellPickPoint[0], cellPickPoint[1], cellPickPoint[2],
                                                                              cellMapperPickPoint[0], cellMapperPickPoint[1], cellMapperPickPoint[2]))

```

And the shown result is:

 ![image](https://discourse.vtk.org/uploads/default/original/2X/1/15e3e99f175b52302b47f395c4e9d5fb2778447a.png)

All picked point has non-zero z-value.

---

<div class="post-metadata">

### Author: ![dgobbi](https://discourse.vtk.org/user_avatar/discourse.vtk.org/dgobbi/32/18_2.png) [@dgobbi](https://discourse.vtk.org/u/dgobbi)
#### Post date: [June 2, 2021, 2:17pm UTC](https://discourse.vtk.org/t/the-world-coordinate-is-changed-when-the-shown-image-is-rotated/5879/13 "2021-06-02T14:17:29Z")

</div>

Good. For the CellPicker, the World and Mapper positions are exactly the same, which is what I expected for your data.

The pick is probably hitting the sphere instead of hitting the plane. Try setting PickableOff() on the sphere, and also on the text.

```auto
sphereActor.PickableOff()
textActor.PickableOff()

```

The WorldPointPicker position isn’t far different (we don’t expect the WorldPointPicker and the CellPicker to return exactly the same result, since they use different methods).

You should not expect the Z to be _exactly_ zero because there might be roundoff error in the pick calculations. The tolerance of vtkCellPicker is plus or minus 0.025 by default, but you can call picker-\>SetTolerance(…) to change it. The tolerance of vtkWorldPointPicker depends on the rendered pixel size and the depth buffer precision.

---

<div class="post-metadata">

### Author: ![zhang-qiang-github](https://discourse.vtk.org/user_avatar/discourse.vtk.org/zhang-qiang-github/32/2519_2.png) [@zhang-qiang-github](https://discourse.vtk.org/u/zhang-qiang-github)
#### Post date: [June 2, 2021, 4:06pm UTC](https://discourse.vtk.org/t/the-world-coordinate-is-changed-when-the-shown-image-is-rotated/5879/14 "2021-06-02T16:06:37Z")

</div>

After setting `sphereActor.PickableOff()`, the z-value of vtkCellPicker is about zero.

 ![image](https://discourse.vtk.org/uploads/default/original/2X/e/e8f9835937b85d81099919e9c63144f57724142d.jpeg)

It shows that vtkCellPicker is more accurate than vtkWorldPicker.

@dgobbi Really thank you for your kindly reply.  
@marcomusy Maybe `vtkCellPicker` is also accurate than `vtkPropPicker`.
