What is your CI Toolchain?

[CaseyLamers1]

Continuous Integration generally requires more "tools" than just the Development Environment.

Code Changes

In my case I keep my source code in a Distribute Version Control System. We use Git to interface with our source code which is kept in a remote repository (repo). We employ the Gitflow Workflow.

Build Server Monitors Repo

The "Master" branch of the repo is monitored on a remote Build Server by a software product called Jenkins. Changes in the Master branch trigger a Jenkins Project to run.

Jenkins Runs the Active Project

A Jenkins project contains a reference to a Build File. This file describes the steps to be taken and their dependencies to execute Testing and Building and Packaging of the build files.

In our build file we first call the VI Package Manager from JKI to apply a VIPC configuration to the project to ensure that all of the correct modules are installed.

In our build file we generally invoke LabVIEW to run Unit Tests. We use the VI Tester tool from JKI to unit test our VIs prior to building. If a test fails then the subsequent steps of the build do not occur and an email is sent with a summary of the failure.

Following successful execution of the tests LabVIEW is called as a Windows Service from Jenkins and a Build Spec from a project is built.

Any number of these steps of applying VIPC configurations, running specific tests, and building can be executed and the specifics are defined in the Build File.

The final step in our current process is to move all of the build files to a folder and zip that folder. Jenkins then maes that folder available as a build "Artifact" which can be downloaded.

Progress of the Project is Tracked

On my roadmap is tieing the test results back into NI Requirements Gateway to keep track of project status with regards to code coverage of requirements, test coverage of code and impact analysis of code change.

I would like to get to the point of a code change triggering impact analysis. Then what gets built is only items that were impacted by the code change.

Testing of Built Code

Functional testing of the code occurs. In our case we use the software in house. We have a development copy of the control hardware. Software testing is performed by the software engineers first. Nuclear Technicians are then asked to try the new software on the development hardware. Finally the tested code is deployed to the production machine.

Deployment of the New Code

Finally there is deployment of the build. In our case we are deploying a control system for a custom scientific instrument (particle accelerator). We connect to a small number of Compact RIO (cRIO) controllers via a web interface (with secure user name and password) and transfer the new files to the control system. We do not yet use tools like Deploy from Wirebird Labs for push code changes to an install base.

So, what steps do you follow?

[joerg.hampel]

I've spent quite some time during the last months trying to get a Continuous Integration system up and running. I've written about it in the LabVIEW Architects Forum and also on lavag.org (twice, actually). Like you, I'm using git (hosted on gitlab.com), and also a Jenkins CI server, and a lot of things already work quite well.

Providing the releases

Here's a link in our toolchain that differs a bit from yours: We use Dokuwiki for all of our support and documentation needs. We also try to handle customer project communication and documentation with our Dokuwiki, so all of our customers get access to their very own namespace within the Dokuwiki (with user credentials and access control).

Before Gitlab, I manually uploaded new software releases to our FTP server and manually created the download links on a Dokuwiki page in the customer's namespace. With the API of Gitlab available, I created a Dokuwiki plugin that connects to the Gitlab API, queries all available Tags for a given project, and checks for each tag if there are files in a given directory on our FTP server available for download. The plugin then generates the list of tags and the corresponding download links.

So, whenever I publish a new tag to a project's gitlab repository, this "release" automatically appears on the Dokuwiki Release page of the project.

Roadmap

There are still a lot of little things left to solve, eg how to package the build results (in my case depending on wether the build was generating a windows application, a real-time application or a source distribution) and distribute them to our FTP server. Perhaps you could share some details on how you use Jenkins to make the Artifact available?

Another issue is wether / how to keep version information inside VIs. I used to have a string constant on the block diagram of a project's main VI, which held a manually maintained version history, but maintaining multiple version histories (git and block diagram) seems wrong to me. How do other people handle this?

Also, where I'm still struggling is how to automatically build source distributions for multiple LabVIEW versions if the project itself includes source distributions without block diagram. Not that it's not possible from a technical point of view - I rather can't find a simple, stable and reusable concept for that.

[ralph3ay]

Just stumbled on this thread while googling stuff and felt like helping you guys simplify your life a bit. I remember we used to struggle with CI/CD too with Jenkins giving us nightmares when trying set the whole thing up. Finally, I did some research on CI and after trying about a dozen tools gave a shot to Buddy ( https://buddy.works )

It does all those things that you mentioned but in one coherent tool. You just hook it with your Git repo, make a push to branch, and Buddy automatically builds, tests and deploys your app to the selected server. The main difference with your setup is that it uses isolated Docker containers for builds which are basically your working environment with preinstalled tools & dependencies. It makes the whole things fast as hell & eliminates the risk of configuration-specific errors. I don't see a dedicated image for LabVIEW in the Docker Hub, but I reckon you can easily configure it yourself with a Dockerfile like the guy here: https://hub.docker.com/r/brookst/bl4s-viewer/~/dockerfile/

It may sound complicated at first but Buddy makes the whole process a walk in the park. You don't have to know sh*t about Docker to create a delivery pipeline, and the whole configuration process takes like 15 minutes (I'm looking at you, Jenkins). There's also YAML configuration if leaving the command line makes you uncomfortable 🙂

The only drawback is that it doesn't support Windows workers but as far as I know they're already working on them.

Go check it out, it's solid gold.

[chinghwayu]

---

@ralph3ay wrote:

It may sound complicated at first but Buddy makes the whole process a walk in the park. You don't have to know sh*t about Docker to create a delivery pipeline, and the whole configuration process takes like 15 minutes (I'm looking at you, Jenkins). There's also YAML configuration if leaving the command line makes you uncomfortable 🙂

The only drawback is that it doesn't support Windows workers but as far as I know they're already working on them.

Go check it out, it's solid gold.

---

This quite the sales talk going in overdrive...