VTK 9.1 Release Cycle

Hi all,

It’s about time for a VTK release. The CI dashboards have been stabilizing fairly well and it seems like a good time to start cutting a release.

Tentative dates:

  • Sep 27: branch and create rc1
  • Oct 4: rc2 and bugfix focusing (CI should be solid by this point)
  • Oct 11: rc3 if needed (final if possible); revert becomes more likely for issues that arise
  • Oct 18: rc4 if needed (final if possible); platform/compiler fixes only by now
  • Oct 25: rc5 if needed (final if possible)
  • Nov 1: final release (if all else fails)

First up, the next release is planned for about 6 months from now (late March/early April) with a similar schedule (see below). So if you miss this release, the next one shouldn’t be too far away. With CI and the robot able to help out more and the release process being more structured, there isn’t as much manual work involved anymore.

Second, there are many features and changes which are not mentioned in release notes. It’d be nice to get these in before we collate them (those added after the branching will need backported to add to both master and release). I would recommend perusing the Documentation/release/dev directory to make sure that any contributions are mentioned that should be in the release notes. Release note MRs should be prioritized.

Third, it is up to code authors to ping reviewers for MRs. We cannot commit to reviewing everything just because it was submitted, sorry. Please tag MRs with the 9.1 milestone to at least get some metadata wrangling involved.

Fourth, in order to keep the release under control, as we get closer to the final release, features are likely to be reverted rather than waiting for a fix (waiting for fixes that never happened is why 8.2.1 never got released). If a problem is there on Monday and you can get to it on Thursday, it is likely to be reverted until it can be fixed. In order to bring the change back, running git revert -m1 $merge_commit will undo a merge and can be added to the MR that fixes it up. Alternatively, the branch can just be redone as if from scratch; up to you.

Note that Python 3.10 is currently slated for Nov 2, but we cannot wait for it because it could slip as well. We’ll make wheels when it comes out and the builds work, but it’s not likely to happen for the release itself. A 9.1.1 can be made to add support as well.

Thanks,

–Ben

9 Likes

My dashboards are in horrible shape though.

I really have the impression people only pay attention to gitlab CI. :frowning:

Anyway, I’ve taken a look at some of the issues and one worth discussing:

/Users/builder/external/VTK/Common/Misc/vtkExprTkFunctionParser.cxx:306:3: error: thread-local storage is not supported for the current target
  thread_local auto rng = std::default_random_engine(std::random_device{}());
  ^

So it seems the C++11 thread_local keyword is only supported by Xcode 8 and later, which requires macOS 10.11.5 to run. My oldest buildbot runs macOS 10.10.x. I don’t believe there is any other bot running any even-older macOS. Newer macOS often drop support for older Mac models, but it so happens that all Macs that can run 10.10 can upgrade to 10.11.

So maybe we should just increase VTK’s minimum Xcode to 8 now? vtkCompilerChecks.cmake currently checks for Xcode >= 5, so it would be quite a jump. Or maybe there is an alternative to thread_local?

Thanks,

Sean

I’m not against bumping VTK’s minimum Xcode to 8, but we might not have to.

@spyridon97 Is there an alternative you can use here? Do we need to random name, or could we just ensure a unique name another way, e.g., by appending a number to the name and incrementing that number until a unique name is found, or something like that?

Note that the preprocessor macro VTK_HAS_THREADLOCAL can be consulted to determine if thread_local is available.

We need the random name to ensure (probabilistically speaking) that the variables which need to be fixed because they are not sanitized, will have unique names.

The proposed solution that you suggested creates problems when the substitution of the not sanitized variables occurs.

Example:
Dataset array 1: _test
Dataset array 2: test0

Expression: _test + test0

Calculator: add _test
Original variable = _test, Valid variable = test0 (this will pass because test0 has not been added yet)
Add test0 variable to the exprtk’s variable list (success)

Calculator: add test0
Original variable = test0, Valid Variable = test0
Add test0 variable to the exprtk’s variable list (failure, because the variable already exists)

We could pass original variables and valid variables and check for such cases in the generation of unique names, but we also would have to remove and re-add variables to the exprtk variable list.

Randomness is the only way that I can think of without making the code extremely convoluted.

Thanks for the explanation. So the real question is, can the thread_local be removed or worked around?

I am looking into that

Actually removing the `thread_local’, would not create any problems, since we don’t mind if the random generator produces the same results for different threads. I had just copied it from How do I create a random alpha-numeric string in C++? - Stack Overflow.

I’ve collated the release notes in this MR if anyone would like to review it before I branch later today.

VTK 9.1.0.rc1 has been tagged; artifacts will be generated by the CI pipeline and uploaded today (or Monday if there’s a long lag).

1 Like

Is there any update on a release for Python 3.10?

Or how to build from scratch?

3.10 for VTK proper is blocked on this issue, so it can’t be tested (IIRC).

You can build it yourself (it probably works) using the wheel build instructions.

Hmm, in the thread you point to, though, you suggest to make wheels and David suggests that should be possible. I assume this does not mean wheels for PyPi but to build yourself?

Yes, you can build wheels however you want with those instructions. I’m more hesitant about what we push to PyPI since that is more “official” and without our test suite actually checking 3.10, I don’t want to push it up saying we support it when it actually doesn’t.

maybe you could make available an unofficial non-tested release to alleviate people’s headaches and mainy duplications of failing efforts? It does not have to be perfect or final, just work for the case it work fr some for their purposes?

Yes, you can build wheels however you want with those instructions. I’m more hesitant about what we push to PyPI since that is more “official” and without our test suite actually checking 3.10, I don’t want to push it up saying we support it when it actually doesn’t.

We’re considering doing just that at GitHub - pyvista/pyvista-wheels: VTK wheels for PyVista and then uploading them as release artifacts.