Feel free to experiment with VTKEx with the corresponding site: VTKEx Site. It is a mirror of Bill’s VTK Examples site with a lot of updates to the scripting and instructions.
I have rewritten the scripts and instructions so that (I hope!), the only changes needed to generate new web pages if we move to a new repository are in:
mkdocs.yml : Modify site names, username etc.
src/SyncSiteWithRepo.sh : Run with the new repository name and the local VTK folder
src/Admin/ScrapeRepo.py : Line 757 in the function: make_instruction_pages(…) would need to be modified for GitLab.
We have to move to a new site as I only have administrator access to the original VTKExamples site. So I can update the master but that is all, since I have no password access. Another reason for moving the site is that the tarballs for the C++ examples are written to a different site to which I have no access.
So this updated mirror VTKEx will do the job.
If you checkout VTKEx and want to try building a site, I would recommend uncommenting lines 72/73 in src/SyncSiteWithRepo.sh. This will mean that the repository is not updated. Thus you can go to the site folder and test it out on your local machine.
One thing we should consider in relation to the chapter files (e.g. src/Cxx.md ) is the removal of the Classes Demonstrated column as this data is in the coverage files. This column existed before I wrote the coverage scripts and was used to point out the classes used. Something to think about.
Just a passing thought, the VTK web pages could link to src/Coverage/CxxVTKClassesUsed.md and src/Coverage/PythonVTKClassesUsed.md, these markdown files tie together the VTK classes with the examples. So people using the VTK web pages, aside from having access to the tests, will have a link to examples.
Migration of the site to a new repository “should” be straightforward. My personal preference is to use Kitware GitLab.
Once migration happens:
Appoint several administrators (I’ll talk about their role later). In the “Year of the Rona” this is even more important. Any one of us could get the Rona and be out of action for a while, or, even worse, kark it.
Verify the site can be built. You can check the links in the local site folder before pushing. You will need to edit:
mkdocs.yml : Modify site names, username etc.
src/SyncSiteWithRepo.sh : Run with the new repository name and the local VTK folder.
src/Admin/ScrapeRepo.py : Line 757 in the function: make_instruction_pages(…) would need to be modified for GitLab.
src/Instructions : Some of the text will need to be updated. In these files note the use of special strings such as __REPOSITORY__, these are replaced by the appropriate string when src/admin/ScrapeRepo.py is run. See make_instruction_pages(...) in this script.
Once this is done the site will be operational.
Note: The site does use git-lfs, the repository is found here. I have a lot of trouble with it, once again this should be moved to a new site. I think I am the only person now who can update this site. Don’t forget to edit .lfsconfig once this is done.
Administrators
Based on experience, I recommend administrators check out the master. Their role is mainly to:
Vet new submissions making sure the guidelines are followed. The developer instructions must be pretty good as there is usually not much to do.
Merge and …
Update web pages after new submissions by running src/Admin/ScrapeRepo.py.
Some future things to do here:
It would be nice to have regular testing of the examples.
Python examples are not tested yet. I’m not sure how to do this and keep a “normal” looking Python file. It is something I have been wanting to do for a long time. Ideas or work on this is welcome.
Developers
Follow instructions here (they may need updating for a new site). Also read the Guidelines.
I think the key things are:
Decide what folder to put your code into, corresponding to the language, e.g src/Cxx/Graphs. Do not make sub-folders of these existing folders as they will not be scraped.
Update the corresponding language chapter file e.g. src/Cxx.md. You can add new headings in this file or add to the existing ones. Be very careful editing this file - follow the layout!!!
Remember to add data files into the appropriate folder, if needed, then update/touch CMakeLists.txt and add a test image after testing.
If you decide that an example is no longer useful or obsolete, you can move it into the corresponding Deprecated folder e.g. src/Cxx/Deprecated/. Remember to update the corresponding language chapter file e.g. src/Cxx.md and possibly CMakeLists.txt.
Remember that the left-hand index on the web pages reflects the global structure of the site and the right-hand index reflects the indexing in the page you have open, e.g here the indexing on the right-hand side reflects the headings in src/Cxx.md.
Today I plan on perusing VTK Examples and look for opportunities to make them better with new content. I’m going to make a list of potential new examples to create in the Google doc listed below. Feel free to add suggestions to the running list. At some point later we’ll prioritize and identify volunteers to code these up
We just closed down the VTK Hackathon. We made some good progress on the VTK Examples. If anyone is interested in seeing the notes they are here. Thanks to all who attended and/or helped out, particularly Andrew who stayed up late, and Seb and David who got up really early.
I checked out the site, was able to build and run ctest OK. I also moved across the files from the Hackathon.
@mwestphal would you be able to set up the website? I think you will only need to modify the three files mentioned above. I’ll be able to then fix any missing links.
Thanks Andrew
According to the discourse poll here : About the VTK examples
We want to keep the website on github, but I do not know how it is setup nor how to update it.
My take would be the following :
Mirror the kitware gitlab in the kitware github
Generate the website and push it into a gh-pages branch on github
@mwestphal Having it on two stites will be a pain. At present, in the old setup, when scraping the repository, the scripts generate the whole static site in the site folder, and, in the GitHub setup, it automatically picks up the site if it is in the site folder of the master. So there is no need for gh_pages. You do need to set the page source as master on the site - that’s the only trick. So I guess the scrape repo script could push just the site folder to the master on GitHub. We should then set .gitignore to ignore the src folder and force a push to the GitHub site. Rewriting the code for setting links to two different sites will be tricky but possible.
I would think having it all on GitLab would be much simpler and easier to maintain.
@mwestphal Maybe someone in Kitware has an answer. I agree in that I have felt that having the site folder and now the tarballs folder in the master is a bad idea.
@mwestphal What about this: " GitLab always deploys your website from a very specific folder called public in your repository. When you create a new project in GitLab, a repository becomes available automatically". We could rename site to repository. See https://docs.gitlab.com/ee/user/project/pages/
Sorry, I should have been more clear. Gitlab pages are not activated on gitlab.kitware.com and will no be activated in the future for security reasons.
Ok so maybe we need to extend the script to force push site and possibly the tarfiles folder to GitHub after the site is built. Not sure how to do this but someone may know.
@mwestphal I think it will be possible to have a GitHub web site where we would update the GitHub master from the GitLab site. So the GitHub master will have a site folder, images folder, a Tarballs folder and a Coverage folder. So we will not need Coverage and Tarballs in the GitLabSite.
I’ll work on updating the scripts to do this. It might take a week or so.
@will.schroeder, @mwestphal I have made good progress. The web site is now independent of the source repository. The script in the checked out repository copies across essential files and then calls other programs/scripts to build the site. You then manually do a git add etc. and push onto the GitLab site. So the web site just contains the html files, images and tarballs. Thus everything is self contained on the web site. There is a bit of testing/fixing links to do in the next few days. I’m currently cleaning up the documentation.
