VI Analyzer Ignore Certain VIs Completely

[Taggart]

Searched briefly but couldn't find any ideas about this.

I know we have the ability #via_ignore comments to ignore specific tests for a specific VI, but I am looking at a different use case.

Here's the use case, I use DQMH. When you create a new DQMH module there is a lot of plumbing code that comes with it. It's standard stuff. Very rarely do I have to open or edit it. Much of it is scripting generated. It often fails tests but I don't care. In addition to failing, it takes up test time, which slows my feedback loop. I would a way to signal to VI analyzer to skip these files. I know I can use the VIAN API to limit the files it checks, but I was thinking there had to be a better way. 

Implementation Ideas - I had 2 main ideas

- Regex matching on VI names - with the regex pulled out of a text file somewhere ala gitignore. Many of the DQMH generated VIs have standard names, so that is easy. When generating events, they don't but you could easily add a prefix/suffix or something that the regex would pick up.

- Using the Tag API to tag the VI. I like this because the scripting can just apply the tag or apply it to the template the scripter uses. Downside is: kind of hidden from the user and perhaps if I decide to make some edits to this VI  I may want VIAN to stop ignoring it and it's not immediately obvious how to do that.

Note:

I picked the Execution and Performance Label because it didn't seem to fit any of the labels easily. If this is the wrong label and you are an admin, please relabel it.

[Taggart]

I see this being useful for other frameworks like Actor Framework. 

I also see it being useful for OOP in general. You could just skip checking all your simple accessors, which probably shouldn't be generating too many failures, but skipping them would definitely cut down on your test time. For that case a regex would work nicely (provided you are careful with how you name your other VIs) although tags would also work.

[felipefoz]

Everything to improve performance of the VI Analyzer Tests is a good thing, save us all some time, and that is one thing.

VI Analyzer API (toolkit) also is not available for Community Edition or Base edition I think, which supports more the idea.

Off topic: is there an idea to provide the toolkit as part o LV Installation already?

[Kevin.Shirey]

Sam It’s a decent idea but perhaps belongs in the api as an ignore to load. I built a filtered parser with our own tag in the vi description that ignores vis on the via api. Using the api is super easy until you get into passwords and ways to load via lvproj, classes and the like. I added another load filter if it’s within an ignore folder in the lvproj. 

[Taggart]

That is a good way to do it Kevin.

I guess it would still slow you down because you'd have to load everything to read the tags (or is there some way to read tags without loading a VI?), but much faster than running all the tests on all of them.

[joerg.hampel]

It's a funny coincidence - I brought up this topic only yesterday at our weekly DQMH Consortium meeting. We discussed way of improving working with VI Analyzer on DQMH-based projects, and I will post to the DQMH forum later this weekend with some details.

[AristosQueue (NI)]

I flagged Darren about this idea. He's not in R&D anymore but still works on VI Analyzer from time to time and is still the best resource for what is/isn't possible with VI Analyzer.

I added my personal kudos to the idea.

[AristosQueue (NI)]

Taggart: I prefer your second option (if only one gets implemented), but why use the Tag API? Just put a comment on the block diagram to "ignore all tests", the same way we put a comment on the diagram to ignore specific tests. That is both easy to script and easy to remove if someone decides they want VIA to target the VI again. And it is obvious to people looking at the diagram why the diagram isn't being checked.

[sth]

The comment on the block diagram means that the BD still has to be loaded.  The tag idea was that it could be queried in the file without loading either the FP or block diagram to be much faster.  The same with the REGEXP solution that blocks the filename before loading.

[joerg.hampel]

I think there are separate use cases for those two options.

1. When running VI Analyzer interactively (UI) and creating/using a configuration file, you can obviously manually remove any files you don't want tested. It would be helpful to have some way to either automate this, or have some  way to provide a blacklist of files not to test.

For DQMH, we are looking into creating a tool that will take an existing VI Analyzer configuration file (with all the objects to test already added) and strip certain or all DQMH framework VIs from it automatically.

2. When running VI Analyzer automatically via its API (on a build server, for example), it would be similarly handy to have a way to skip certain VIs independently of their actual content (*).

Or put the other way around: I'd like the tool on the build server to decide which VIs not to test, not the developer (who might or might not forget to put the bookmark, or remove it where he or she shouldn't).

The way we (HSE) currently do this is we create a VI Analyzer configuration file which defines the tests but not the objects to test. The VIs are then added to the task by our CI tools during runtime (either the whole project or only those VIs that changed in SCC since the last run/release).

3. For DQMH, we might add bookmarks in the future to skip certain tests where we deliberately can't or won't fix the "problem".

4. And finally, Sam, there is a way using the Tags API to identify any DQMH framework VI. The consortium is working on making this available, please bear with us.

(*) It seems that using the Tags API to parse the project is wayyyy faster than opening a reference to each VI. Can anybody confirm this? Also, is there a way to get to the bookmarks within a VI without opening a reference to it (maybe some other API)?

[Taggart]

To aristos - perhaps just add to the existing vía ignore, a vía ignore all bookmark. Still has loading issues, but simple.