Which method in VTK could I detect and repaire the mesh overlap?

Which method in VTK could I use to detect and repaire the mesh overlap?

Thanks in advance

In general, repairing meshes is very hard and it often requires tedious manual/semiautomatic processing. You are much better off if you can avoid them in the first place.

How do you generate the invalid mesh? Do you get self-intersecting mesh because you extrude a mesh with surface normals? Or do you get overlapping meshes because you append meshes instead of combining them with Boolean mesh operations?

Ok. I want to know is it also hard to detect meshes and is there any method to detect modes?

The mesh is a STL model which was outputted by 3D modeling software like UG.

Sorry, I don’t understand what you mean by detecting meshes and modes or what is UG. Maybe you could use different words or add screenshots or draw a sketch to make things more clear.

I‘m sorry to bring your confusion.
For me, I got the STL triangle mesh, and I will converse the STL to CLI. But the STL mesh sometimes has some errors, like bad edges、holes、Bad triangle(including bad orientation、intersections、Overlap), which make the conversion errors. So I need to find the errors of STL first.
UG is a 3D modeling software. Like the following picture.

The best is if you use a modeling software that does not generate invalid meshes. If you must use UG then probably your best bet is to report the issue to the developers and hope they will fix it.