How to solve encoding problems on non-utf8 encoded systems

Make SystemTools::FileExists support more languages (#19215) · Issues · VTK / VTK · GitLab (
I submitted an issue last week, and then I learned that when reading a file kwsys determines the file’s existence.kwsys requires the string encoding to be utf-8.So I solved the problem by converting the string encoding to utf8.
But now I’ve found that CGNS determines if a file exists via the _access function in io.h, encoded in MSVC as the local encoding gb18030, which leads to the fact that I can’t get both determinations to pass on this computer at the same time.
How to solve the encoding problem.

That works for your build, but we should fix it for all builds. While we can definitely fix the vendored copy, we’ll need to get CGNS upstream to also patch it for external CGNS usages to use Windows APIs directly. We can patch ParaView’s superbuild too.

Cc: @toddy @MicK7 @mwestphal

That is a terrible solution. Your VTK build has now broken the UTF-8 support internally.

@ben.boeckel It looks like _access should be patched to _wacess in combination with the use of MultiByteToWideChar

@realwhzc, can you please try this? See Wrapping/Tools/vtkParseSystem.c’s system_win32_stat for how _wstat is used to see if the change works for you.

Okay, I’ll try.

In cgio. c,I copied the function static wchar_t* system_utf8_to_wide(const char* str) to this file,and added a line wchar_t* wname = system_utf8_to_wide(filename); to the function cgio_check_file.

Then replace if (ACCESS (filename, 0) || cgns_stat (filename, &st) || S_IFREG != (st.st_mode & S_IFREG)) by if (_waccess(wname, 0) || _wstat64(wname, &st)).

After that I replaced all the _fopen, _open in the read related section of CGNS with _wfopen, _wopen, involving files ADF_internals.c, ADF_interface.c.

Now the problem is solved.

I think it’s totally a CGNS issue. However, when reading the openfoam file, an exception seems to occur as well. Although no error occurs, there is no time step in the read result, indicating that the foam file is not being read properly either.

Thanks for testing. Let’s get these diffs into CGNS itself. We can cherry-pick into VTK and apply it to ParaView’s superbuild too.