Unit Testing Group

cancel
Showing results for 
Search instead for 
Did you mean: 

Building Good Habits

So I have been trying to really get the habit of unit testing going.

The easiest way I have found so far is simple. Whenever I go for the run button on a subVI because I have filled it in to test it, instead create a unit test with UTF. Once I have run it and happy there is a single button to import the values and now my test is recorded for all time to be re-run as needed. It needs to ultimately progress further but I have found it a good way to build the habit.

What is interesting is for once LabVIEW could be on the forefront of unit testing! It reminded me of a podcast I recently listened to about an idea called approval tests. Each time the test fails the user can import the new results as correct on inspection, it's not quite the same but actually closer to what I described earlier than normal unit testing! The podcast is http://www.thisagilelife.com/46/ if your interested.

I'm curious, how did other people start getting it into their workflow?

Cheers,

James

James Mc
========
CLA and cRIO Fanatic
My writings on LabVIEW Development are at devs.wiresmithtech.com
Message 1 of 24
(14,995 Views)

A good next step is to automate your unit test run on a continuous integration server.

What we do in our labs is we have a Jenkins Server that is polling our source code control server. We have a job set up to run all unit tests on a project at check in.

The nice thing about this is that instead of relying on making running the unit tests a habit, you can just automate that part away. Also this frees up your machine to work on other tasks while your tests run on the server (not to mention in any team, there's always at least one person who forgets to manually run the tests...).

There are a few items in setting up Jenkins and LabVIEW that aren't totally obvious but if you have questions feel free to ask.

Scott

0 Kudos
Message 2 of 24
(8,013 Views)

We have a build server we run as well (TeamCity). Much like Jenkins the build is triggered by source control check-in. We have a generic module we call GBuild that is called to build the specified project, update the build version to the build server definition, run any unit tests and execute any present VI Analyzer configuration. Messages are passed back to TeamCity so that the entire build log is captured and browsed (eg. the history of unit test results), just like any other build type such as .NET.

Because of the lengthy time to actually do all this we don't always trigger a build on every check-in. We prefer regular check-in but scheduled builds for LabVIEW projects.

We don't use UTF much at this stage - we found the lack of object-oriented support / mocking frustrating since we always had to write a bunch of custom tests for everything which also mitigated code coverage. We use VI Tester now that it supports LabVIEW 2012 onwards. Prior we had to use our own framework GUnit which was structured virtually the same. Maybe we'll come back to UTF some day and automate some of the frustration out.

0 Kudos
Message 3 of 24
(8,013 Views)

What are the limitations with UTF with respect to classes that you are running into tyk007?

We have many tests in UTF with LV 2012 which are testing class method VIs. We aren't doing dynamic dispatch in our unit tests however so maybe that is where the limitation is?

I'm genuinely curious as to what you have found as it may influence how I plan for future projects using the CI/UTF setup I have now.

Scott

0 Kudos
Message 4 of 24
(8,013 Views)

Good question, I'd like some thoughts on this. The limitations we ran into were (no particular order) are from the 2012 version:

  • We use mocking extensively to test our modules (basically constructor dependency injection) - so we are using dyanmic dispatch extensively. We ended up having to generate a large number of setup methods for tests as part of the "arrange" activity as well as the lvtest files. This was because the UTF focuses on the ''act' and 'assert' phases and provides most IDE benefit in these areas. With VI Tester the AAA format was part of the unit test method.
  • We wanted to create our own types of assertions (for example - string contains a particular regex) We found no easy way to do this in UTF - maybe we didn't look hard enough?
  • We found generating the tests time consuming using the IDE. We didn't put any effort into 'automating' the framework though (either editing the files manually or generataing them automatically).
  • Our unit test suite for one small project took almost an hour to run (around 400 tests). We found this to be out of sync with our TDD practices (and also quite frustrating). Moving to VI Tester has reduced the same test time down to less than 20 seconds. I have seen similar comments on the forums (both 'dark' sides).
  • We had to manually move lvtest files into a seperate set of folders so that they would not be attached to a  build (we use packed project libraries extensively) - the default location is with the source. There was no apparent easy way to do this for multiple files since the path's seem to be relative in the lvtest file.
  • Source control on the lvtest files was difficult. You could examine the contents as a text file but it was still manual if I wanted to determine what unit tests had changed between different revisions.
  • It was difficult to tell what each test in a lvtest file actually did. We follow best practices around this (each test case challenges only one Unit of Work). We'd often have several cases in a single lvtest file which made maintainability hard. Maybe there is a way? With VI Tester each unit test method was named to indicate the method under test, the state being tested and the expected result - this gave developer visibility over what the test was actually doing since there are seperate explcitly named files. Down-side - more files.

Tony

EDIT: I should probably mention that there is nothing wrong with UTF per se - its great that such a framework exists and, if the developer is not familiar with unit testing, I recommend it as a good starting point. Our needs just seemed to out-grow it. Its possible that there are strategies that help improvethe areas we outlined.

Message 5 of 24
(8,013 Views)

tyk007 wrote:

EDIT: I should probably mention that there is nothing wrong with UTF per se - its great that such a framework exists and, if the developer is not familiar with unit testing, I recommend it as a good starting point. Our needs just seemed to out-grow it. Its possible that there are strategies that help improvethe areas we outlined.

Tony,

I agree with your comments. I find myself using more JKI VI Tester for my projects. However, when the project is done for a regulated industry, then I have to use Unit Test Framework and at that point a lot of wrapper VIs have to done for the dynamic dispatch methods.

I started this community, because I believe there are a lot more people doing Unit Testing than one might think and if we start these discussions we stand a chance to getting better tools.

Thanks for joining the conversation,

Fab

For an opportunity to learn from experienced developers / entrepeneurs (Steve, Joerg, and Brian amongst them):
Check out DSH Pragmatic Software Development Workshop!

DQMH Lead Architect * DQMH Trusted Advisor * Certified LabVIEW Architect * Certified LabVIEW Embedded Developer * Certified Professional Instructor * LabVIEW Champion * Code Janitor

Have you been nice to future you?
0 Kudos
Message 6 of 24
(8,013 Views)

scrook wrote:

A good next step is to automate your unit test run on a continuous integration server.

...

There are a few items in setting up Jenkins and LabVIEW that aren't totally obvious but if you have questions feel free to ask.

Scott

Scott,

Would you mind starting a new discussion/document on this group pointing people where to get started with setting up Jenkins and LabVIEW. I understand you have a full time job and this might not be possible right away, but I believe that there would be a lot of people who would benefit.

Thanks,

Fab

For an opportunity to learn from experienced developers / entrepeneurs (Steve, Joerg, and Brian amongst them):
Check out DSH Pragmatic Software Development Workshop!

DQMH Lead Architect * DQMH Trusted Advisor * Certified LabVIEW Architect * Certified LabVIEW Embedded Developer * Certified Professional Instructor * LabVIEW Champion * Code Janitor

Have you been nice to future you?
Message 7 of 24
(8,013 Views)

tyk007 wrote:

We have a build server we run as well (TeamCity). Much like Jenkins the build is triggered by source control check-in. We have a generic module we call GBuild that is called to build the specified project, update the build version to the build server definition, run any unit tests and execute any present VI Analyzer configuration. Messages are passed back to TeamCity so that the entire build log is captured and browsed (eg. the history of unit test results), just like any other build type such as .NET.

Tony,

Same request as for Scott, would you mind starting a discussion or document pointing us on how to get started with TeamCity?

Thanks,

Fab

For an opportunity to learn from experienced developers / entrepeneurs (Steve, Joerg, and Brian amongst them):
Check out DSH Pragmatic Software Development Workshop!

DQMH Lead Architect * DQMH Trusted Advisor * Certified LabVIEW Architect * Certified LabVIEW Embedded Developer * Certified Professional Instructor * LabVIEW Champion * Code Janitor

Have you been nice to future you?
Message 8 of 24
(8,013 Views)

Great community, great conversation. Fab: Would you mind setting up a poll to find out which testing solutions people are choosing to adopt in their LabVIEW development? I'm interested to know just how many are not using UTF.

Thoric (CLA, CLED, CTD and LabVIEW Champion)


Message 9 of 24
(8,013 Views)

Thoric wrote:

Great community, great conversation. Fab: Would you mind setting up a poll to find out which testing solutions people are choosing to adopt in their LabVIEW development? I'm interested to know just how many are not using UTF.

Done! I added UTF, JKI VI Tester, TestStand and Other:

https://decibel.ni.com/content/polls/19201

For an opportunity to learn from experienced developers / entrepeneurs (Steve, Joerg, and Brian amongst them):
Check out DSH Pragmatic Software Development Workshop!

DQMH Lead Architect * DQMH Trusted Advisor * Certified LabVIEW Architect * Certified LabVIEW Embedded Developer * Certified Professional Instructor * LabVIEW Champion * Code Janitor

Have you been nice to future you?
0 Kudos
Message 10 of 24
(8,013 Views)