Coverage is still messed up

Development
1

Foks,

Coverage is running but not reporting file coverage.

Bill

2

Looking at the buildbot logs, I see this:

Error(s) while accumulating results:
  Looks like there are more lines in the file: /home/kitware/dashboards/buildbot/vtk_nightly-master-ike-linux-shared-release_coverage_mpi_nightly_python2/source/Common/Core/vtkSimpleCriticalSection.cxx
  Looks like there are more lines in the file: /home/kitware/dashboards/buildbot/vtk_nightly-master-ike-linux-shared-release_coverage_mpi_nightly_python2/source/Common/Core/vtkTimeStamp.cxx

which might be throwing a wrench into the file-level coverage data. Anyone know what this means? @brad.king?

There are 48 coverage.xml files uploaded however, so I assume the file-level data is in there somewhere. Maybe CDash has a bug?

3

I talked with Zack here and he’s going to look at it.

4

He says the CDash side is fixed. Not sure what’s up with ike itself right now. I’ve started a coverage build manually to see if it just needs to run again.

Edit to add: Seems to work now: https://open.cdash.org/viewCoverage.php?buildid=5846221

5

Hey @ben.boeckel @lorensen, do you remember if you guys figured out the cause of the Looks like there are more lines in the file ... messages?

I’m asking because I’m in the process of switching the CI build of our application over from gcc+gcov to clang+llvm-cov, and messages like these have started popping up (and the coverage reported is a bit inconsistent with those we got from gcc+gcov), and this thread is what turned up when googling :slight_smile: (very few hits on that type of error).

6

It’s strange, I seem to have gotten rid of those particular messages now. But I don’t know what I did to do so :confused:

7

I figured this out on SMTK. VTK will get the same attention when I get to it. The problem is that gcov, by default, writes out coverage files in the form objbase##includedbase, so foo.o that has coverage for inc.h will be written to a file whose basename is foo.o##inc.h. If an object file includes two headers with the same basename, gcov will reuse the same file. When CTest comes around and gets the information, it finds a mismatch between one of them. The fix is to pass -x (--hash-filenames) to gcov to make it do objbase##hash(include) as the filename to avoid conflicts. CTest really should have this in the default set of flags; please feel free to submit a CMake MR to add it (I don’t know when I’ll get the chance to do it).

8

Ah, thanks for the detailed explanation @ben.boeckel. I’m not sure if that’s the exact issue we ran into, but it sounds like it could be related…

For us, we had working coverage using gcc+gcov, and the problems started to appear when we switched from gcc+gcov to clang+llvm-cov gcov.

I have since switched our build back to gcc+gcov, since I didn’t want to lose our coverage reporting.

I had a look, and it seems llvm-cov gcov, unlike gcov, does not have support for -x:

estan@edison:~$ gcov --help | grep -- -x
  -x, --hash-filenames            Hash long pathnames
estan@edison:~$ llvm-cov-8 gcov --help | grep -- -x
estan@edison:~$

So if the problem we ran into when switching to llvm-cov gcov instead of gcov is the one you mentioned, perhaps we are out of luck until llvm-cov gcov gains support for this flag.

In any case, thanks for the explanation. I’ll have a closer look next time I try to switch the build over to clang+llvm-cov gcov. The reason we want to switch is that I want to enable some of the sanitizers offered by clang.

EDIT: Looks like more recent versions of llvm-cov supports -x (https://llvm.org/docs/CommandGuide/llvm-cov.html). We were using llvm-cov-8 from Ubuntu 18.04.

9

I’d suggest keeping the coverage build separate. Coverage needs a Debug build, but I find sanitizers to be better when tested as normal builds (RelWithDebInfo) to try and catch/expose optimizer-related problems too.

10

Ah yes, good advise, will probably set up a separate build for that when I get around to it then.