How to get polyline from drawn 2D image?

I have a grayscale image, for example, using paint brush to draw a circle(or polygon).
How can I get the polydata(points and lines, which forms a circle/polygon shape) from this picture?

You can try vtkMarchingSquares, it can generate polylines that separate the light parts of the image from the dark parts of the image. And if you put the output of vtkMarchingSquares into vtkContourTriangulator, it will generate polygons from the polylines. See the example here.

But if you are asking if VTK is a good general-purpose tool for vectorization, the answer is no, it is not.

thanks for answering, can you suggest some tools? I tried paraview, not sure how to achieve that.

I agree that for 2D image vectorization you can find better tools than VTK. However, if you want to implement free-form 3D image segmentation then it is better “vectorize” (extract isosurface) of the entire volume - and in this VTK is quite good. You paint with the brush tool in a 3D labelmap then extract surface using marching cubes (or optimized variants, such as flying edges) for 3D display. You may use vtkContourTriangulator to retrieve slice intersections (but usually it is faster and more robust to just retrieve edges).

If you are asking about vectorization because you want to export to the archaic DICOM RTSTRUCT format then the short answer is just don’t do it, but use DICOM Segmentation Object instead.

If you work in medical image segmentation then you you can use/customize/extent/build your segmentation application on the open-source 3D Slicer platform, which a comprehensive set of segmentation tools (paint, erase, draw, scissors, level tracing, fill between slices, grow from seeds, threshold, margin, hollow, smoothing, scissors, islands, logical operators, draw tube, engrave, fast marching, flood filling, local threshold, mask volume, split volume, surface cut, watershed, solidify, NVidia AI-assisted segmentation, and more). See more information here.