We worked a lot on implementing robust curve cutting in 3D Slicer and after some experimentation and VTK fixes we finally got it right. It is very robust and accurate and supports both straight and crinkle cuts:
@lassoan
The steps are so complicated, can you please show an excerpt where input is Mesh as curve, and another mesh as input too ? I would like to select the faces that the curve intersected.
The code is very short and a full, working example for cutting a mesh with a curve. I would recommend to copy the code into your project as is, and try to simplify it yourself.
auto InputModelToWorldTransform = vtkSmartPointer::New();
auto InputModelToWorldTransformFilter = vtkSmartPointer::New();
InputModelToWorldTransformFilter->SetInputConnection(CleanFilter->GetOutputPort());
InputModelToWorldTransformFilter->SetTransform(InputModelToWorldTransform);
auto SelectionFilter = vtkSmartPointer::New();
SelectionFilter->SetInputConnection(InputModelToWorldTransformFilter->GetOutputPort());
SelectionFilter->SetEdgeSearchModeToDijkstra();
// Further connections depend on cut mode (straight or whole cells)
auto ClipFilter = vtkSmartPointer::New();
ClipFilter->SetInputData(mandiblePolyData);
auto ConnectivityFilter = vtkSmartPointer::New();
ConnectivityFilter->SetInputConnection(ClipFilter->GetOutputPort());
auto OutputWorldToModelTransform = vtkSmartPointer::New();
auto OutputWorldToModelTransformFilter = vtkSmartPointer::New();
OutputWorldToModelTransformFilter->SetTransform(OutputWorldToModelTransform);
OutputWorldToModelTransformFilter->SetInputConnection(SelectionFilter->GetUnselectedOutputPort());
OutputWorldToModelTransformFilter->Update();
auto outputOutsideMesh = vtkSmartPointer::New();
outputOutsideMesh->DeepCopy(OutputWorldToModelTransformFilter->GetOutput());
3D Slicer allows performing any operations on arbitrarily transformed inputs. If the operation has multiple inputs then you need to convert all inputs to world coordinates, perform the operation, and then transform the result back to the object’s local coordinate system. If your objects are already all in world coordinate system then you can skip all the spatial transformations.
Input mesh is an object that stores a vtkPolyData containing a surface mesh.
Input curve is an object that stores a vtkPolyData containing a curve (a single line cell with many points).
You can get edge points and both sides of the cut surface of the surface using vtkSelectPolyData.
If your original curve input points are far from the mesh then you may want to constrain the curve to the mesh (using projection, optionally combined with Dijkstra path search on the surface). This projection is a quite complex task, we implemented a custom VTK filter for it: vtkProjectMarkupsCurvePointsFilter.
I would recommend to try this first using the 3D Slicer GUI. If it works well there then you know that the problem is in how you extracted the code from Slicer and moved it into your project. If you find that something does not work in Slicer either then you can report it in the Slicer forum.