# CI/CD infrastructure on gitlab

**URL:** https://discourse.vtk.org/t/ci-cd-infrastructure-on-gitlab/6142
**Category:** Support
**Created:** [July 6, 2021, 4:41pm UTC](https://discourse.vtk.org/t/ci-cd-infrastructure-on-gitlab/6142 "2021-07-06T16:41:32Z")
**Posts on this page:** 7
**Page:** 1

<div class="post-metadata">

### Author: ![kayarre](https://discourse.vtk.org/user_avatar/discourse.vtk.org/kayarre/32/808_2.png) [@kayarre](https://discourse.vtk.org/u/kayarre)
#### Post date: [July 6, 2021, 4:41pm UTC](https://discourse.vtk.org/t/ci-cd-infrastructure-on-gitlab/6142/1 "2021-07-06T16:41:32Z")

</div>

Hi,  
TL;DR: Can someone suggest a medium complex CI/CD example project that builds on Linux, OSX, and windows? I have been trying to follow the VTK implementation but it got dense fast for me. (maybe a paired down example if it exists, osx, windows, and linux targeting x64?)

One issue on windows I have found so far is that when I use [download\_python.cmake](https://gitlab.kitware.com/vtk/vtk/-/blob/master/.gitlab/ci/download_python.cmake) and pip install numpy it provides a 64/32 bit mismatched installation.

More context:  
We are internally using gitlab along with the CI/CD tools. We generally use Docker for most things (with great success), however I have a need to do build on the windows directly. I started following the steps used in VTK, and it works to some extent, but the numpy/python mismatch has me stumped.

If there is another approach or a simplified example out there that someone would suggest I would be greatly appreciative.

---

<div class="post-metadata">

### Author: ![ben.boeckel](https://discourse.vtk.org/letter_avatar_proxy/v4/letter/b/ea5d25/32.png) [@ben.boeckel](https://discourse.vtk.org/u/ben.boeckel)
#### Post date: [July 6, 2021, 4:57pm UTC](https://discourse.vtk.org/t/ci-cd-infrastructure-on-gitlab/6142/2 "2021-07-06T16:57:18Z")

</div>

VTK’s CI/CD extracts some information from the `CMAKE_CONFIGURATION` environment variable. For Windows specifically, the `VCVARS*` variables are also important. Our setup assumes that the runner sets `VS160COMNTOOLS` to where its installed Visual Studio lives. This is then used by `.gitlab/ci/vcvarsall.ps1` to set up the Visual Studio toolchain environment.

Your situation seems like you have a 32bit environment loaded rather than the 64bit env.

Unfortunately, all of our projects are sharing the same infrastructure, so they’re all going to have this level of complexity.

---

<div class="post-metadata">

### Author: ![kayarre](https://discourse.vtk.org/user_avatar/discourse.vtk.org/kayarre/32/808_2.png) [@kayarre](https://discourse.vtk.org/u/kayarre)
#### Post date: [July 6, 2021, 7:53pm UTC](https://discourse.vtk.org/t/ci-cd-infrastructure-on-gitlab/6142/3 "2021-07-06T19:53:20Z")

</div>

Thank you so much @ben.boeckel. I was hopeful, but that matches up with the similar setups across the related projects i looked into.

Yes it seems super weird because I am downloading the same python file `python-win64-3.8.6.tar.xz` and using `python -m pip install`, but pip installs `numpy-1.21.0-cp38-cp38-win32.whl` which I would not have expected and gives an error when importing it, and trying to install the x64 version from download gives an error.

I think you are saying that `VCVARS*` and `VS160COMNTOOLS` variables would affect the how python and pip execute? I would have thought an x64 python executable would run in x64 mode all the time. Is there something in the environment (not properly set) that would be causing the mismatch? I will keep searching. Thank you so much.

Thank you for the hint, the environment appears to be the culprit.

---

<div class="post-metadata">

### Author: ![kayarre](https://discourse.vtk.org/user_avatar/discourse.vtk.org/kayarre/32/808_2.png) [@kayarre](https://discourse.vtk.org/u/kayarre)
#### Post date: [July 6, 2021, 7:55pm UTC](https://discourse.vtk.org/t/ci-cd-infrastructure-on-gitlab/6142/4 "2021-07-06T19:55:15Z")

</div>

Where do those python packages come from? the ones hosted at [https://www.paraview.org/files/dependencies/](https://www.paraview.org/files/dependencies/)

---

<div class="post-metadata">

### Author: ![ben.boeckel](https://discourse.vtk.org/letter_avatar_proxy/v4/letter/b/ea5d25/32.png) [@ben.boeckel](https://discourse.vtk.org/u/ben.boeckel)
#### Post date: [July 6, 2021, 9:23pm UTC](https://discourse.vtk.org/t/ci-cd-infrastructure-on-gitlab/6142/5 "2021-07-06T21:23:49Z")

</div>

> [@kayarre](#):
>
> I think you are saying that `VCVARS*` and `VS160COMNTOOLS` variables would affect the how python and pip execute?

I think the loaded environment might have something to do with it. I really don’t know; Python has so many weird behaviors that I can’t keep track of all of them, sorry.

> [@kayarre](#):
>
> I would have thought an x64 python executable would run in x64 mode all the time

I would too. _shrug_

> [@kayarre](#):
>
> Where do those python packages come from?

They are made via the `.gitlab/ci/wheels/prep_python_windows.ps1` script in VTK’s repository.

---

<div class="post-metadata">

### Author: ![kayarre](https://discourse.vtk.org/user_avatar/discourse.vtk.org/kayarre/32/808_2.png) [@kayarre](https://discourse.vtk.org/u/kayarre)
#### Post date: [July 6, 2021, 9:33pm UTC](https://discourse.vtk.org/t/ci-cd-infrastructure-on-gitlab/6142/6 "2021-07-06T21:33:03Z")

</div>

Thank you. It was definitely something funky with how python interacts with the visual studio variables. the scripts I am using aren’t smart enough to know if x86 or x64 is loaded, only that visual studio variables are loaded.

I definitely didn’t expect that to be the issue. making sure the vcvars loaded x64 the thing works. super funky business.

---

<div class="post-metadata">

### Author: ![ben.boeckel](https://discourse.vtk.org/letter_avatar_proxy/v4/letter/b/ea5d25/32.png) [@ben.boeckel](https://discourse.vtk.org/u/ben.boeckel)
#### Post date: [July 6, 2021, 9:49pm UTC](https://discourse.vtk.org/t/ci-cd-infrastructure-on-gitlab/6142/7 "2021-07-06T21:49:03Z")

</div>

I think `pip` looks to see what (if any) VS toolchain is available if it needs to end up compiling C code and somewhere in there forgets to check whether it can actually _use_ such code. I would suggest filing an issue upstream.
