Crash in ToLowercase()

Support
1

I am using a VTK 9.0.1 that i just built.
I only try to run some examples for now. Exemples that only use vtkMath work with no issues.
But when trying some examples that do visual things, i end up getting a crash. With the first ‘Cylinder Example’ for example.

The end of the call stack is
-std::back_insert_iterator std::transform<__gnu_cxx::__normal_iterator<char const*, std::string>, std::back_insert_oterator>, int ()(int)>(__gnu_cxx::__normal_iterator<char const, std::string>, __gnu_cxx::__normal_iterator<char const*, std::string>, std::back_insert_iterator, int (*)(int))
-vtkNamedColorsDataStore::ToLowercase(vtkStdString const&)

The passed strings are simple, and the code of toLowercase() seems valid to me, so i don’t really understand what is hapenning.
Valgrind says “Invalid read of size 1” as last message

I hoped that it could be a bug in the STL of the first GCC i used (4.8), so i rebuilt VTK with GCC 9.3, but i am still getting the same issue.

I am not really sure what’s wrong at this point. Has anyone else seen this error?

2

OK, after some blind tests, i think what is hapenning is because of passing std::string by reference between user code and the library. VTK was built on machine A and the example on machine B with different build toolsets. Runing the example works on A with VTK built on A, and works on B with VTK built on B, but not if i mix A & B binaries.

While debugging, i can see that, when entering in vtkNamedColors::setColor(name, rgba), the debug value of name is only visible when not mixing libraries. And that’s also when the first valgrind warning is triggered in the non working configuration.

I knew passing std::string by reference between DLLs can fail like this on windows if the Dll and the executable are not built with the same visual C++ version. I didn’t know the problem could also be there on Linux

I can’t be the only one to have come up with this issue?

3

I’ve never seen this issue, but here are a couple tools that might help.

The “objdump” program can be used to find out what version of gcc was used to compile a file (can be used on executables, static libraries, dynamic libraries, and .o files):

objdump -s --section .comment vtkObject.cxx.o

vtkObject.cxx.o:     file format elf64-x86-64

Contents of section .comment:
 0000 00474343 3a202855 62756e74 7520392e  .GCC: (Ubuntu 9.
 0010 332e302d 31377562 756e7475 317e3230  3.0-17ubuntu1~20
 0020 2e303429 20392e33 2e3000             .04) 9.3.0.

and the ldd command can be used to find out what libraries your program is dynamically linking to. With ldd and objdump, you might be able to discover discrepancies that point to the cause of the crash.

Edit: one thing to look for with ldd is whether your program is dynamically linked to libc++ instead of libstdc++. The latter is generally the default on linux.

4

When you say “different toolsets” between machines A & B, what do you mean? Different versions of GCC?

vtkNamedColors::SetColor() takes a vtkStdString parameter. Presumably building VTK with a different compiler means vtkStdString on machine A is a different object type from that on machine B.

I think only const char* should ever appear in VTK’s public API for methods containing text parameters.

5

One machine is centos6 with gcc 4.8/GLIBC 2.12. Another is centos7 with gcc 9.3/GLIBC 2.17. Another is Ubuntu18 with gcc7.5/glibc 2.27. I did not look what libstdc++ versions they use yet.

An example debugged on the ubuntu runs with a VTK built on that ubuntu. but not a VTK built on the CENTOS machines.
I also tested on the centos7 machine, which can run examples with a VTK built on this machine, but the VTK built on the ubuntu machine does a SEGV (same error location)

std::string, behind vtkStdString, being a template type, the template code of std::string that gets integrated in the library is the one of the compiler used to build the library (visible with the source path when debugging inside std::transform caled by Tolowercase()).
I don’t know to what extent g++ headers try to keep template types binary compatible between versions, but i know problems can occur on windows with different Visual C++. A public API of something i work on, that used std::string, has had to be changed so that the type string that is exchanged between the library & user code is a char*, or an abstract string type whose implementation is in the library only.

For my personal situation, i think i will just build a VTK for each build toolset i want to use.

6

That’s the correct approach.

That’s actually your only option. COM was designed to permit passing objects (interfaces actually) across library boundaries. Doing it in C++ is nothing but a hack in my opinion.

Using text as a key in the colour map is another problem. It should be an enumeration or integer key mapped to names.

7

I know this topic is over a year old now but I am having a similar issue with ToLowercase() and didn’t want to create a new topic for it.

In my case I have a similar crash at std::transform in ToLowercase. However, unlike the situation described above where the issue seems to have been caused by different toolsets, in my case I have built the VTK libraries and the example with Visual Studio 2022 on the same machine.

I have built the libraries with debug information and stepping through I can see that when the vtkStdString is first created and assigned to it is accessible, and the value set can be seen in the debugger. Once it is passed through to vtkNamedColors::GetColor3d the value assigned to the string is no longer accessible.

I’m assuming there is some mismatch in compiler settings between building the libraries and then the example but I cannot see anything obvious in the project settings. And seems odd to me when both were built on the same machine, one after the other, using CMAKE to create the project.

Any thoughts would be much appreciated.

8

Using the same machine does not guarantee ABI compatibility. You can have many toolsets installed in VS2022, so you need to make sure you use the same or compatible ones. You also need to share C and C++ compiler, flags, standard, and extensions between all the projects that you want to use together. See for example how external projects are configured in 3D Slicer, passing a number of options to build dependencies:

Also, it may worth rebuilding everything from scratch, because sometimes - especially when you change build options - some libraries in your projects might not be fully updated for some reason (e.g., due to incorrectly written CMake files). Rebuilding everything may make the problem go away and you might not be able to reproduce it ever.

9

I wonder if it might be worthwhile to write a different implementation for ToLowercase. The function pointer cast looks a bit fishy. And it seems it should have a check to make ensure the input is pure ASCII, in order to avoid undefined behavior.

  vtkStdString ToLowercase(const vtkStdString& str)
  {                 
    vtkStdString s;
    std::transform(str.begin(), str.end(), std::back_inserter(s), (int (*)(int))std::tolower);
    return s;
  }

@amaclean any thoughts?

10

Since vtkStdString, like std::string, is expected to be utf8 encoded bytes in VTK, having a toLowercase() method makes no sense at all.

11

Thanks @dgobbi, I’ll look into that. It is very old code! All those ints!

Probably something like:

std::tolower(c, std::locale())

wil do the trick.

I think std::locale() should be sufficient.

12

No, fair point, I think that was my frustration coming out! I double checked the toolsets and ensured those matched OK.

I will double check the compiler flags etc. I did try and compare them yesterday but I have probably missed something and this is leading to the issue. Thanks for the example, I will check that out.

I tried this multiple times, and built with the debug info to allow me to step through and all times the same issue existed. I’ll have a look at the project settings again and hopefully I have just missed something staring me in the face.

Thanks for your suggestions, appreciate you taking the time to reply. I put in a quick fix by updating the call to take char* so that I could check it was working otherwise, which it was thankfully.

1 Like
13

Please see: https://gitlab.kitware.com/vtk/vtk/-/merge_requests/9375

14

How does this solve anything?

The problem is that the vtkStdString instance created in the client application is not the same object type as the vtkStdString expected inside the VTK library so it crashes. std:string is not portable across library boundaries so the compiler behaviour is undefined.

15

It just fixes some very old code.