Using Subversion (SVN) with LabVIEW

[Elijah_K]

As anyone who has asked me about source code control knows, I've become a very big fan of Subversion; specifically, TortoiseSVN.  I originally started using it because a large number of NI customers I was speaking with had made the switch, due in no small part to the price tag (it's free).  But it's not just the price that makes it so appealing - TortoiseSVN is very easy and simple to use, which makes it something that makes you more efficient, rather than weigh your development process down.

If you're not using source code control at all, you're playing with fire.  And since Subversion is free, you now have no excuse.  Still not convinced?  Has any of the following ever happened to you:

• You delete or change some code as part of an experiment or perhaps debugging, and then save it.  If you want to undo these changes, you have to redo it yourself.
• You want to experiment with a new implementation or function, so you save the old VI or project to 'My Project OLD,' to make sure you have your backup handy. (Now you have a bunch of other copies of your application all over your laptop and your network drive).
• You want someone else to look at your code or even modify it. (again, you put a copy on the network folder or a jump drive... but when you make a change, you have to go do it all again!)
• You open an older copy of your application or a copy saved in a different location, causing a link conflict (lets face it, fixing this is not fun)
• You introduced a bug into your code a few days ago and want to revert to an older version to figure out what changed (too bad)
• An angered or jealous C developer steals your laptop and destroys it, taking with it the only copy of your code. (that's what you get for under-bidding him by 3X)

Any of these could've been avoided or made easier if you'd been using source code control.   So go download Subversion!  Go here.

UPDATE (7/29/13)

There is now a free plugin from Viewpoint Systems that sets up integration with Tortoise for you (including diff and merge) and overlays the icons in the Project Explorer - the best part is that it's totally FREE.  I strongly recommend that anyone using SVN+LabVIEW install this tool.  The latest version can be found here: Subversion Version Control for LabVIEW. Please make sure you already have Tortioise installed - if you don't, make sure to restart your computer after installing Tortoise.  You'll also need to restart LabVIEW after installing this plugin.  Enjoy!

Tip 1

If you're going to use Subversion with LabVIEW, you need to know a few things.  Subversion stores some hidden folders and files in your directory structure, which can cause some problems when mass compiling or using auto-populating folders.  Not to worry!  Open your labview.ini file and enter 'skipSVNFolders=true' to resolve this issue (only works in 2009, will be on by default in 2010).

Tip 2

If you're using the professional version of LabVIEW, you can tell Subversion (or any source code control tool) to invoke the command-line diff and merge executables.  To set this up, right click in Windows Explorer and select TortoiseSVN > Settings.  Select Advanced and enter the following for a .vi file type (this can also be used for a .ctl)  "C:\Program Files\National Instruments\Shared\LabVIEW Compare\LVCompare.exe" %mine %base -nobdcosm -nobdpos.  Do the same for merge, but enter "C:\Program Files\National Instruments\Shared\LabVIEW Merge\LVMerge.exe" %base %mine %theirs %merged.  The command line parameters are explained in the LabVIEW help, click here for LVCompare.exe.  More info on command-line differencing.

Getting Started with Subversion

If you've never used source code control and want to give it a try, check out this hands-on exercise and example program I published here: http://decibel.ni.com/content/docs/DOC-10122

As for experienced source code control readers and users, what packages are you using, and do you have any tips for use with LabVIEW?  Is anyone using Git or Mercurial? (two other popular open source SCC providers)

[Todd_Lesher]

Can you elaborate in a way that people in this group can read, as opposed to the restricted CLA group?

elset191 Aug 2, 2013 11:47 AM (in response to Elijah_K)

Here's a relevant discussion on the CLA group https://decibel.ni.com/content/docs/DOC-27011

David Staab Aug 2, 2013 11:51 AM (in response to Daklu)

Daklu wrote:

Can you elaborate?  I haven't heard anything about Hg throwing in the towel.                   

Jack already did here: https://decibel.ni.com/content/docs/DOC-27011#comment-29651

[Daklu]

Jack already did here: https://decibel.ni.com/content/docs/DOC-27011#comment-29651

I had read that, but I don't interpret it as "Hg capitulated."  It's one cloud-based provider deciding to drop support for Hg.  Hg itself doesn't appear to be going away any time soon and I personally don't find "Git is more popular" a compelling reason to choose Git over Hg.  IMO the "correct" decision for any particular person or organization depends more on how well each solution solves their business requirements rather than their respective technological merits.

[David_Staab]

Time to get that CLA certificate!

[David_Staab]

Daklu wrote:

<a bunch of stuff>

Okay, whatever. Nancy asked my opinion, I gave it. Doesn't bother me if y'all use something else.

[Todd_Lesher]

Sure, that's the plan - when I have time. Finally took the CLD a couple months ago (used AF for it).

I prefer Hg. I like how easy it is to use a USB drive as the remote repo so I can sync machines that can never see the network.

I understand and appreciate the usefulness of a closed group. But at least on LAVA, people who don't have as much experience can still see what the experts are talking about. Nothing in the CLA Summit videos seemed to be THAT advanced - unless you want to avoid spending time answering questions that people should already know. And what good are architects if no laborers can understand them?

Just saying it's bad form to answer these public questions with private answers.

[David_Staab]

Todd_Lesher wrote:

I understand and appreciate the usefulness of a closed group...Just saying it's bad form to answer these public questions with private answers.                   

I don't know what to tell you. NI wants to add value to the CLA certification, so they lock down the forum. I agree that equally valuable content is available for free on LAVA, but that conversation took place on the NI forum. That's just where it happened. If you want access to NI-marshalled content, you'll have to play by NI's rules (or get them to change the rules). Conversely, if you don't want to have to "pay" for access to content, you're only going to be able to see/join the discussions that happen on LAVA.

Aside: I think it's completely insane that you passed the CLD using AF. Props for the effort, but it seems completely unncessary.

[Todd_Lesher]

David Staab wrote:
                   
 I don't know what to tell you. NI wants to add value to the CLA certification, so they lock down the forum. I agree that equally valuable content is available for free on LAVA, but that conversation took place on the NI forum. That's just where it happened. If you want access to NI-marshalled content, you'll have to play by NI's rules (or get them to change the rules). Conversely, if you don't want to have to "pay" for access to content, you're only going to be able to see/join the discussions that happen on LAVA.

I wasn't browsing a restricted group, nor was I whining about not having access. I was browsing a public board and clicked on what might or might not have been a useful link. When I click on links that turn out to be a restricted forum, my first thought is not, "Oh, I should go certify." My first thought is, "Why are people linking to restricted boards on these public posts?"

David Staab wrote:

                       

Aside: I think it's completely insane that you passed the CLD using AF. Props for the effort, but it seems completely unncessary.

It made sense at the time. And I didn't want to burn four hours doing something easy that I've already done, and in a way that I don't normally do it.

[Daklu]

Okay, whatever. Nancy asked my opinion, I gave it. Doesn't bother me if y'all use something else.

                   

I suppose I wasn't clear.  I'm not advocating Hg over Git or Git over Hg.  In fact I have Hg, Git, SVN, and Perforce all installed on my system.  As a consultant I have to be prepared to use whatever my customer uses.

What I'm saying is I haven't seen anything that leads me to believe the Hg team capitulated.  If they have in fact done so, then I agree that would be sufficient reason to go with Git.

Tood_Lesher wrote:

Just saying it's bad form to answer these public questions with private answers.

Posting a link is better than not posting at all, and I would be reluctant to publicly quote what somebody said in a private forum without asking permission.

[EricLM]

Daklu wrote:

Here's hoping Viewpoint ports their tool to support Git/Hg. 

We have discussed this and it is a possibility. If there are people interested in making this happen sooner rather than later, I could use some help. I am familiar with Hg and Git, but I do not know the APIs at all. What I am looking for is volunteers who could write a LV wrapper for Hg/Git APIs that I could then incorporate into a project provider. If anyone is interested, please send me a PM so we can discuss further.

[EricLM]

GabiTillmann wrote:

                       

I am still using TSVN on two people project with a cloud based server.

The only xml-style file that Tortoise tries to merge is the lvproj itself (I am the one that does the LVOOP ) .

That lvproj-merge usually goes completly havoc. You better call your partner and ask whats new or stay back and try to fit your modifications in the project tree manually.

                   

This is a good use case for the needs-lock property. I do this with all LV XML file types in any project I work with, so developers on our team only change these files one at a time. Yes this still causes problems with branch merges. I will reiterate something else that has already been said. It would be great to see a diff/merge tool from NI that handles ALL LV file types.

[Elijah_K]

Agreed.  We're very aware of the need to offer more sophisticated diff'ing and merging options for classes, projects, etc...  the reason it may take a while is that the most practical approach to the problem is to re-evaluate some of the ways in which the information is stored (while ensuring backwards compatibility).  For the foreseable future, I'm afraid Eric's right - you'll want to impose locks on these types of files for team based environments.

[danny_t]

Hi,

There already exists a Mercurial API  written by Ton Plomp  http://lavag.org/files/file/162-mercurial-api/ (GNU Public)

I have this installed and use it along with http://lavag.org/files/file/206-mercurial-provider/ (BSD Most Common) also by Ton.

These tools are well written tools by somebody who knows their stuff.

There might be some mileage in a combined solution

Danny

[Elijah_K]

That's great news!  Would love to see this plugin in the LabVIEW Tools Network

[chinghwayu]

Check Ton's bitbucket to connect to Mercurial repositories from the LabVIEW project at:

https://bitbucket.org/tcplomp/mercurial-labview-project-integration

[LabBEAN]

It is true that DVCS allows the developer to commit changes without a network link to the repo.  DVCS also inherently backs up the source on each developer’s machine since everyone has the repo.  That said, the main advantage I see is the change in thinking that in turn allows a more natural workflow.  Save and commit locally after each code change (e.g. edited type def that affected 5 VI conpanes) and push to an off-site, private repo at the end of the day.  Whenever you’re ready to share your tested, stable code, push to the global repo.

In DVCS (e.g. Hg), changesets represent any like group of edits.  For example, “new UI feature”, “new hardware driver feature”, and “fixed bug in new UI feature” might be 3 separate commits.  Centralized VCS (e.g. SVN) users may be familiar with holding back questionable code or code with broken run arrows from the team repo.  Instead of the three commits listed above, probably better to wait and commit once with a comment that says “new stable UI feature and new possibly buggy hardware feature”.  Unfortunately, if we discover later that hardware feature was implemented incorrectly, there’s no way to revert that changeset if we never committed it. So, with SVN there’s risk in sharing globally frequently and infrequently.

We ported from SVN to Hg (and RhodeCode) over a year ago.  Pre-Hg, before leaving the office it was general practice to send code up with a “daily commit” comment to kindly warn other developers of the backup that just took place.  In Hg, we can commit locally at the end of the day and easily push changesets to a private Dropbox folder or thumb drive for off-site backup instead of pushing to the main global repo.

Lastly, I’ll note that using Hg not only allows committing changes while using your development laptop in an area with no internet, but you can also develop on lab/manufacturing machines that are connected to the field hardware and will never have an internet/repo connection.  Simply push changesets to your thumb drive and sync back when finished.

Hg has allowed me at least to develop code more naturally.

[inspireTech]

Is the TSVN update supporting TortoiseSVN 1.8 available?

Thanks,

Ron

[EricLM]

Thanks for your patience everyone. A version supporting TortoiseSVN 1.8 is now available for download from our site at http://www.viewpointusa.com/product/ni-labview-toolkits/tsvn-toolkit/.

[Patur]

Brilliant! Thank you for the good work Eric

Cheers,

Patur

[KuGr]

So I have been trying to configure svnserve (started as a service) to use either NTLM or GSSAPI via SASL for authentication in a Windows environment.  Does anyone have any knowledge on what my svnserver.conf and svn.conf should look like?  Online documentation is pretty sketchy and often are Linux focused.  Not to mention that I am not an IT guy.

I know that in svnserver.conf I need:

     use-sasl: true

I am not sure I need to uncomment min and max encryption.

I then need a svn.conf that has:

     mech_list: ntlml (or gssapi)   <--- can both be there?

If I have ntlm then

     ntlm_server: <<<here i don't know what it need>>>

     ntlm_v2: yes  <---do I need this?  i read that it is client side, but where should that go?

If I have gssapi then it looks like I need:

     keytab: <<<which is some path?>>>

gssapi fails with 'Unable to find a callback: 2"

ntlm gives me a login in prompt but does not log me in.

Thanks for any guidance.

[cirrusio]

EricLM wrote:

                       

Thanks for your patience everyone. A version supporting TortoiseSVN 1.8 is now available for download from our site at http://www.viewpointusa.com/product/ni-labview-toolkits/tsvn-toolkit/.

                   

Looks like it is gone?  Only thing I see here is a "Speak to a Sales Engineer".  Is this going to be a pay-for toolkit?

[EricLM]

Sorry about that. There was a problem with our CMS for the website. The page is back up.

Still free

[warren_scott]

Do any of these add-ons support the idea of "blame" -- where I can pick a object in the block diagram (or FP) and ask the revision control system when was it added/modified and who did it?

[Ajay_MV]

How do you link JIRA with SVN?

[Ajay_MV]

Soupy & others, I have a question here.  Is it recommended or appreciated if we do the branching and merging very frequently?  Right now, I'm working in a bug fix for a project where my team recommended to branch and merge for each bug we are about to fix.  i.e., we have about 100 bugs to be fixed in couple of months.  For fixing each bug (say 1st bug), we should branch the trunk version, fix the bug and commit the fixes to the branch.  Later, a project coordinator will merge that branch (1st bug fix) with trunk.  This will go for each bug.  I personally feel it's consuming lot of time than fixing the bug in itself.  What do you think?

Thanks

Ajay

[EricLM]

Ajayvignesh_MV wrote:

                       

How do you link JIRA with SVN?

                   

Have a look at this Atlassian help page: https://confluence.atlassian.com/display/JIRA/Integrating+JIRA+with+Subversion. In short, you need a plugin called Fisheye.

[EricLM]

Ajayvignesh_MV wrote:

                       

Soupy & others, I have a question here.  Is it recommended or appreciated if we do the branching and merging very frequently?  Right now, I'm working in a bug fix for a project where my team recommended to branch and merge for each bug we are about to fix.  i.e., we have about 100 bugs to be fixed in couple of months.  For fixing each bug (say 1st bug), we should branch the trunk version, fix the bug and commit the fixes to the branch.  Later, a project coordinator will merge that branch (1st bug fix) with trunk.  This will go for each bug.  I personally feel it's consuming lot of time than fixing the bug in itself.  What do you think?

Thanks

Ajay

                   

Ajay,

This is a common practice in other languages and likely follows a workflow called gitflow (http://nvie.com/posts/a-successful-git-branching-model/). While this may not work for some team, it will for others. It completely depends on your team's established workflow. It sounds like you do have a project coordinator for planning and executing the merges, which is very helpful with making this workflow successful. It may feel like it takes more time, but there are benefits to it and you will get more efficient at it as you go.

If you do use this workflow, make sure to follow a couple best practices.

• Keep commits small
• Update/merge frequently (small merges are easier to manage than large ones)
• Assign a project manager to manage the merges back to the trunk and create releases (which it sounds like you do)

Eric

[justin.goeres]

Ajayvignesh_MV wrote:

                       

fixing each bug (say 1st bug), we should branch the trunk version, fix the bug and commit the fixes to the branch.  Later, a project coordinator will merge that branch (1st bug fix) with trunk.  This will go for each bug.  I personally feel it's consuming lot of time than fixing the bug in itself.  What do you think?

                   

Eric's response above was quite good, I just have a minor point to add to it.

It depends a lot on how well encapsulated your code is. If the bugs you're fixing are in large VIs where a single VI will contain multiple fixes (or parts of multiple fixes), your project coordinator is going to waste a lot of time because merging large VIs in LabVIEW is just really hard.

On the other hand, if your VIs/subVIs are small and decoupled from each other so the bugs are localized to small, independent areas of code, the branch & merge model can work.

Additionally, if you're going to the trouble of having a specific project coordinator whose job is to manage the branches, it would be an extremely good idea to create unit tests to validate the bugs and their fixes.

[Daklu]

...my team recommended to branch and merge for each bug we are about to fix...

  I personally feel it's consuming lot of time than fixing the bug in itself.  What do you think?

                   

There are too many unknown variables for me to offer an opinion as to whether this is a good plan in your environment.  I can imagine valid business reasons for adopting that kind of strategy for a Labview code base, so it's not necessarily a bad idea.

For example, I developed an scc policy for one of my clients that uses Perforce.  The policy dictates that each product's mainline (trunk) can always compile and run.  This allows nightly builds and unit testing to be automated.  We strongly discourage people from checking code into the mainline directly.  Instead, they should create a branch, make and validate their changes, then promote their changes to the mainline.

The thing to avoid with Labview code is any scenario where you have to merge files.  (.vi, .lvclass, .lvlib, .lvproj, etc.)  These files *can* be merged, but it is difficult and error prone.  Branching and merging at the component level is safe as long as you don't have multiple developers making changes that affect the same file.

Will your team run into problems with the branch and merge strategy?  *shrug*  I dunno.  Is your application designed to have independent components with well-defined interfaces, or is it a single monolithic vi?  If it's broken out into components, it's pretty straightforward to have mulitple developers working on independent branches without causing problems.  If it's monolithic, your safest course of action is to have everyone work on the same branch and use file locking.

[D*]

Agreed.  Your point "The thing to avoid with Labview code is any scenario where you have to merge files..." is good one. 

(".lvlib, .lvproj" are just .xml, so these aren't too bad)

LVMerge is okay for small stuff, but for a big project I found myself holding it's hand, and losing a big chunk of time convincing it that nothing really changed between two versions. If you know which version to keep it's also not so bad, but again that doesn't scale up to a large project with multiple developers.

[Daklu]

(".lvlib, .lvproj" are just .xml, so these aren't too bad)

lvlib, .lvclass, and .lvproj files are xml and can be merged successfully for certain kinds of changes.  However, all of them can contain binary streams or lengthy config strings that are very difficult (or impossible) to merge.

Oftentimes the only binary stream in .lvlibs is for the icon, so that can be a pretty easy one to work around.  .lvclass files store mutation history and default values as a binary stream.  Projects that contain an FPGA target have several long config strings and bits of information embedded as a binary stream.  (I don't know where LV puts FPGA config strings when the resource is in a library.  In the lvlib file or lvproj file?)

[aravindgowda]

yes....what is your requirement

[Oli_Wachno]

I have uninstalled the Viewpoint Systems toolkit, because it unfortunately has tremendously increased large project load times on several machines here. (I'm talking about several minutes!)

But still working SVN! Never again without.

Although I have to say: the more complex / the more layered your libraries get, the more you have to use the command line tool: the GUI does not all the options SVN has in store. But as just mentioned: this is true for advanced use cases.

No excuse to use no SCC!

[carlos_camargo]

It would be good if this toolkit were integrated into LV and improved by the NI engineers to address some of the flaws which currently exist in the tool.  I also agree that one should not require the JKI tool to add this on as it should be natively built-in.  This Toolkit also doesn't support Linux which is another reason it needs to be natively integrated.

It is amazing that NI hasn't yet bothered to integrate support for such a widely used and favored version control tool.