After a VTK commit is made, CDash sends out emails to the committer if any build or test failures occur in nightly builds. This was enabled presumably to help point out errors that may have been introduce, but typically the emails include unrelated persistent failures on the dashboards and cause concern to new contributors that they may have broken something.
Should we disable these emails?
It is rarely usefull and when it is, gets discarded as the usual “spam from the nightly”. I’d say to disable it, someone would still need to monitor the nightly dashboard to see new failures.
Ideally, the nightly should be part of the CI or at least triggered on merge, not during the night.
Ideally, the nightly should be part of the CI or at least triggered on merge, not during the night.
Latest master builds are triggered by changes to master. External sites (e.g., RogueResearch machines) run nightly.
I don’t like the idea of disabling them. It would be preferable to make them more useful.
Already I have the suspicion that people just trash them, which is unfortunate.
Every once in a while I fix all the warnings reported by my bots (or git blame and ping people), but then next I look there are a bunch of new warnings introduced. 
It feels to me like people only pay attention to gitlab and not CDash.
CDash has the option “Email redundant failures”. Is that disabled for VTK? If not, disabling it should help quite a bit.
Sean
My proposal was poorly worded. Instead of disabling emails altogether, I meant, should we disable automatically sending contributors email even though they haven’t opted in to receiving emails from CDash? In this day and age, it seems problematic to send emails that are not specifically requested.
To opt out of receiving these emails, you need to create an account on CDash with the email address you used in your commit message and change your preferences. That isn’t at all obvious.
So the proposal is actually to turn off automatically sending emails for failures.
That would help for many of the build warnings and failures, and should be available for the VTK project (I’m not an admin on CDash for VTK). There is still the problem that contributors haven’t specifically opted in to receiving emails CDash, redundant or no.
This might be wishful thinking but is it not possible for CDash to identify the erroneous file(s) and only email developers whose change sets include that file/those files?
Definitely not. People are looking at the CDash of their own MR, they are not looking at the whole CDash 24hours after merging to check if they broke something, and imo, they should not be expected to.
CDash has the option “Email redundant failures”. Is that disabled for VTK? If not, disabling it should help quite a bit.
Many failures are inconsistent, that is why these emails are mostly spam
So the proposal is actually to turn off automatically sending emails for failures.
I definitely agree, however that means, in essense, that developpers are not responsible to check for dashboard outside of the one being run for their own MR.
External Site : Nightly dashboard should be checked by their maintainers which can then message contributors if a contributor broke something.
The alternative is to integrate the nightly dashboard into the MR dashboard are at least to trigger them on merge.
Ideally, but it’s hard enough for a human to track down which changes may have contributed to a particular test failure.
In principle, developers should check the nightlies the next day for new failures, but in practice that usually doesn’t happen (even among core developers). I think whether or not email is sent, especially email with errors unrelated to what a developer has contributed, doesn’t encourage this.
Not practical.
Mathieu, I should have been clearer, when I said I felt people ignored cdash, I indeed meant nightly submissions (like mine).
I disagree that developers shouldn’t be expected to check the nightly dashboards. As you know, VTK supports many platforms and many build option permutations, and the builds and tests performed via gitlab MRs are only a subset.
What’s the point of community submitted builds if the developers just ignore them?
How would you propose a nightly dashboard maintainer know who to email? It would seem such a maintainer would need to use git blame to figure it out. If only that could be automated. Oh wait, cdash can automatically email people whose commits cause issues! 
Years ago there were many builds submitted by the community, but now, sadly, there are very few.
Cory, if you’re not a cdash admin for vtk, who is? We use cdash where I work too, and the number of annoying emails dropped dramatically once we disabled the “Email redundant failures” option. Would really help to know the current setting for the purposes of this discussion.
Cheers,
Sean
That box is currently unchecked.
I disagree that developers shouldn’t be expected to check the nightly dashboards. As you know, VTK supports many platforms and many build option permutations, and the builds and tests performed via gitlab MRs are only a subset.
I’m afraid this process of checking nightly build after merging something as not documented at all:
https://gitlab.kitware.com/vtk/vtk/-/blob/master/Documentation/dev/git/develop.md
What’s the point of community submitted builds if the developers just ignore them ?
What is important is that configuration that are officially supported are tested and that there is a clear process that help VTK dev take these tests into account.
How would you propose a nightly dashboard maintainer know who to email? It would seem such a maintainer would need to use
git blameto figure it out. If only that could be automated. Oh wait, cdash can automatically email people whose commits cause issues!
With the number of merge a day in VTK project, sending a mail to all developpers that contributed this specific day is basically spam. A human would be able to identify the right developper to email.
