Keep It Standard Stupid!

[swatts]

Evening All.

If you went to Austin for the CLA summit I trust you had fun and returned home safe, sound and brainy.

If you picked up the tone of my previous blog posts correctly you'll appreciate that I don't really like acronyms. I apologise for subverting one of the better ones for the title!

One of the keys to simplicity in my opinion is to standardise, by this I mean employing techniques that are accepted as the norm. The theory behind this is pretty well researched.

In the Design of Everyday Things by Don Norman talks about

New situations become more manageable when existing pattern knowledge can be applied to understanding how things work. Consistency is the key to helping users recognize and apply patterns.

It saves a lot of extra brain-work and the skills are more transferable.

The additional benefits - If there is a vibrant user-base you know that it will be supported, added to and will have an active eco-system. Within LabVIEW you know it's base purpose will be worked on and improved. If you are employing some peculiar edge-case you are risking no improvement and quite possibly deprecation.

Some Examples

Databases for Storing Data

There's a load of ways to store your results, but databases are the only one that's worth mastering. There's a great deal of support online and MySQL and SQLite have thriving communities with huge amounts of tools and help available. Type MySQL into Google and you'll get over 100 million hits, I think that's enough material for anyone.

SQL for Talking to Databases

It's all very well using a LabVIEW toolkit to plug data directly from cluster into a table, but if you want a skill you can build on have a bash at SQL. It will end up as one of the most useful tools in your toolkit. Again a search on Youtube for SQL Tutorial resulted in over 500k hits.

UDP for Communications

Inter-system communications is a rich area for toolkits, add on technologies, APIs etc etc. The biggest successes I have had are home-grown low-level protocols just chatting out of UDP (either broadcast, direct or a combination of both). It's robust, simple, low level and there are tools out there to sniff it. People get obsessed with security, but all of our systems are on secure networks and security can get added if you need it. Where-as it can make debugging and programming difficult when you don't. Google 58 Million hits.

TCP for Bulk Transfers of Data

One of the issues with UDP is that it is so basic, there's no handshaking and it can't handle large lumps of data. If you need this capability then you can use the TCP API. It's only marginally more difficult to use and we use UDP for inter-system messaging and TCP for transferring data.

OPC for Scada

In Industry pretty much everyone uses OPC for parking centralised data in distributed systems, it will be beneficial for us just to do the same. A write tag to OPC vi is not such a difficult abstraction to get my head around that I need to use a Network Shared Variable. 33 million hits on Google for OPC.

ODF for Documents

One day I'll find time to finish my ODF toolkit and when I have done that we should use ODF for all documents, spreadsheets and reporting.

Event Structures for Events

Events are really handy for passing data about, they have a great attribute in that they are strongly typed and yet still I have reservations (Chris Roebuck gives an extremely sound argument for the counter view and writes some lovely code based on this). For me Event Structures are for events, any other use is against the standard and therefore an additional burden.

Queues for Queuing

Back in the days of yore (BQ - Before Queues) and before Mr Mercer provided the wonderful queue API in LabVIEW we rolled our own. As we know we're now in the AQ period of history.

That was circa 2000 and we still have something similar in todays code.

This is how much we love Queues!

Globals for Global Data

As I've discussed many times Global data need's viewing with suspicion, however if you need it you'll find the most efficient and easy to understand method is just to use a global.

Feedback Nodes for Local Permanent Data Storage

I know they're a bit peculiar, named wrong and have a weird interface!

but compared to the previous technique of using shift registers in a while-loop (i.e. an accidental side-effect) I think it has a much clearer purpose.

So my friends the take out from this article is that it is better to use some technique or tool that is standard and popular even if it is not the "best".

Lots of Love

Steve

[crossrulz]

You might have just opened up the flood gates for the arguments between the use of User Events and then the Shift Register vs Feedback Node.

As long as you are consistant within the organization, I really do not care when it comes to the FBN vs Shift Register.

[swatts]

I think the point I'm trying to make is that it's actually meant to do the job intended, rather than have a while loop running once, which is a design nonsense.

As with all these discussions there's layers of benefits, so I agree with you, if you have a standard within your company that leverages one of these techniques then the benefits of that would far outweigh my issues with it.

The only thing I would add in defence of my point of view is that I wouldn't expect/demand that NI improve the data throughput of User Events because it's not the basic use case, whereas queues need to shift data, so I would expect this to be an area of support.

[joerg.hampel]

I love that:

BQ - before Mr. Mercer

AQ - well... Aristos, right?

[swatts]

Joerg you beauty! I was sooooooo pleased with that joke and thought I might be the only person in the world to get it!!

[crossrulz]

swatts wrote:

                       

I was sooooooo pleased with that joke and thought I might be the only person in the world to get it!!

                   

Trust me, you weren't.  I almost busted a gut on that one.

[Thoric]

I don't (yet) understand your reservations with Event structures for purposes such as messaging. You state that they should only be used for Events, but what do you mean by "Events"? Please elucidate.

[swatts]

You can create user events and essentially use them to pass data into VIs, I guess it's using the queuing mechanism of the Event Structure but with the advantage that the strong typing you get with Events. The examples I've seen have resulted in really neat code that works very well, which is great.

My reservation is that it's not really what the structure was designed to do.

So I'm not against having an event based system and passing data with the event, I am against just using the event structure as a data transport mechanism. (Sorry, it's not something I'm really solid on. I just have a mild sense of unease)

[CRoebuck]

Steve,

Great post. I agree with the flood gates opening but why not. It's good to have opinions.

Many thanks for the kind words. I'm very familiar with your reservations regarding events and as always I respect that but I'd love to try and change your mind (maybe over a beer at NI Week)

I really really like the Phrase "Globals for Global Data".......get's the point across in four words. Everyone has access to the data (both read and write) and that should be cause for concern for anyone serious about writing robust code or ensuring a good developer experience.

I'm a big fan of the feedback node too, I feel it aids readability.

Second, Third and Fourth your comments on SQL. One of the better decisions in my career was to pursue Microsoft Certification.

Great stuff , Cheers Steve !

[CRoebuck]

I remember your String Array based system. I loved "Popping" items. Sounds much more fun that Dequeing and Enqueing.

At the very least we should petition AQ to change the names !

[swatts]

I'd be much more solid in my convictions if your code wasn't so damn nice looking!!!

[FabiolaDelaCueva]

Steve,

Great article as always.

One word of caution regarding TCP, you say:

"

TCP for Bulk Transfers of Data

One of the issues with UDP is that it is so basic, there's no handshaking and it can't handle large lumps of data. If you need this capability then you can use the TCP API. It's only marginally more difficult to use and we use UDP for inter-system messaging and TCP for transferring data.

"

One of the biggest misconceptions about TCP as it is implemented in LabVIEW is to assume that it ensures the application level handshaking. It doesn't, it only does it at the OS level, meaning that the OS can acknowledge having received a package before the package has been read in LabVIEW. However, if one wants to ensure that the packages are not lost at the application level, we need to implement that handshaking ourselves. This results in less efficient throughput, and I have seen advanced developers end up creating the communication using UDP and adding the handshaking on top. More at:

http://www.ni.com/white-paper/2710/en/

http://www.ni.com/example/28346/en/

Regarding events, what can I say, I work with Chris Roebuck and you already know we love events. I think the part that we are missing here is the definition of an event. For example, LabVIEW does not offer a way to write an interrupt based code that would interrupt the code when there is a serial package received via the serial port. So, in that case, we can create a user event (which should be named dynamic event or custom event, but I digress), that will get fired when a new package arrives. Then the event structure can handle this event and have in its data node the data received via the serial port.

Is in that vein that we use events in the DQMH, it is one module telling another module that something happened and adding some extra information via the payload if needed.

Regarding the comment: "The only thing I would add in defence of my point of view is that I wouldn't expect/demand that NI improve the data throughput of User Events because it's not the basic use case, whereas queues need to shift data, so I would expect this to be an area of support."

Actually, there was a presentation at NI Week (I think 2012 or 2013) that showed how at the end if one benchmarks the same application implemented with Queues or with Events, the same throughput is achieved with the added advantage that if you create a Queue and enqueue elements, you are forced to make sure there is a dequeuer, with events, one can fire events all day long and if nobody is registered to receive them, no queue is created in the back end and no potential for memory leaks.

I second Chris' comment that we should probably have this discussion over your beers (I can have my ice tea) at NI Week or before, if I get myself to the UK before then

Thanks again for the food for thought and for opening the flood gates

Fab

[swatts]

CRoebuck wrote:

                       

I remember your String Array based system. I loved "Popping" items. Sounds much more fun that Dequeing and Enqueing.

At the very least we should petition AQ to change the names !

                   

I guess that came from my machine code days (yes I'm THAT old!)

[swatts]

I like thinking of them as "Interrupts", I think that may be the line I think it works at.

As you can see I've not thought too deeply about it.

[Thoric]

swatts wrote:

                       

You can create user events and essentially use them to pass data into VIs, I guess it's using the queuing mechanism of the Event Structure but with the advantage that the strong typing you get with Events. The examples I've seen have resulted in really neat code that works very well, which is great.

My reservation is that it's not really what the structure was designed to do.

So I'm not against having an event based system and passing data with the event, I am against just using the event structure as a data transport mechanism. (Sorry, it's not something I'm really solid on. I just have a mild sense of unease)

                   

We could probably chew the fat on this for quite a while. I'm interested in hearing more about your reservations. No, strictly speaking the event structure wasn't provided for passing data between components, it was to allow us to react to system level activities, and also create our own user level events that inform of activities.

And... All events have a payload, often just a few bytes of data associated directly and specifically to the event. For example "Drag Starting" has "Drag Mode" information - just four bytes. Small payloads are inherently part of the concept - it would struggle to be useful without them.

Creating custom events to extend the usefuless of event reporting, with or without small payloads, isn't a variation from the design intent that I would consider inappropriate. Specific events for specific activites with specific (small) payloads.

But... Where I think I might agree with you is when someone uses an event structure where a Queue would be more appropriate. Such as streaming of regular data, or irregular large payloads. Or sending a generic message with an overridable payload - that's not specific/strict. I'm currently extending a framework to accommodate command pattern through events for component-level dynamic calling, and that has me feeling slightly uneasy. But nonetheless it works, and it works efficiently.

[swatts]

A lot of the motivation behind some of my posts is trying to articulate a sense of unease about something or other. I'm lucky I have a responsive set of folks to help clarify some of this.

[Thoric]

Not as lucky as we are to have you Steve! 😄

[PhillipBrooks]

You forgot regular expressions! I struggle every time I use them, but many LabVIEW functions include regex inputs.

They are common in most programming languages, and you can do some incredible things with them if your regex-fu is strong.

http://forums.ni.com/t5/BreakPoint/Regular-Expressions-Board/td-p/1187799

[swatts]

Very true,

I also forgot to mention RT Linux and I'm very excited about that. 

[Daklu]

I struggle to use them too, mostly because I don't use them very often.  Several years ago I purchased a copy of RegexBuddy and it has saved me tens of hours of time.

[swatts]

It's very timely as I think it will very useful for a project I'm about to tackle, that regexbuddy looks damned handy

[GregPayne]

I use http://regexr.com/ whenever I need a regex. Works every time and has a really good cheat sheet

[crelf]

https://regex101.com/ is the bestest.