Network Published Shared Variable Array Handling

[NI-MarkN]

Currently, for an array variable, changing an element requires the developer to read the array, update the element and write the array out. Whilst this is fine for simple cases with not sophisticated data flows, it becomes very cumbersome for cases when parallel processing is introduced. For instance, multiple loops updating an array of status flags.

At present, if two loops (which may be in different Vis) both execute a read-update-write operation on a global shared variable, data can be lost if the operations happen at similar times. Consider the following example: 

Looking at the image above, let's imagine "array1" consists of two elements with values F and T respectively. Then the operation would happen as follows:

Step Number	Action taken	Array values
1.	Read "array1"	F and T
2.	Read "array1"	F and T
3.	Update "array1" element 0 to T	T and T
4.	Update "array1" element 1 to F	F and F
5.	Write to "array1"	T and T
6.	Write to "array1"	F and F

Hence all the data written by “Loop A” is lost. Putting user-defined locking using shared variables around the operations does not seem to work, presumably due to update rate of the locking variables.

What would be helpful to overcome these race conditions is to replicate the functionality present in other languages to do operations like a[3]=4 or printf(“%lf”,a[4]). In these cases, an atomic operation is performed to get or set the value in the defined memory location. The addition of this functionality to shared variables would be extremely powerful.

An example of such functionality could look something like this:

[RavensFan]

Not needed.

All you need to do is encapsulate the updating of an array element of a shared variable within a single, non-reentrant subVI.

[Intaris]

Coordinating things like this is why we are paid as programmers.

Atomic operations on network shared variables don't really work because there's quite a lot of work being done in the background.

[NI-MarkN]

Thanks for the comments. RavensFan, did you mean using a simple FGV architecture?

[Intaris]

Wouldn't a managed access model like this dramatically slow down the throughput of the network shared variable?  Atomic operations on distributed systems would require some kind of acknowledged lock which would introduce a lot of latency surely.  I am, however, a complete noob on network shared variables so I may be way off.

[RavensFan]

Originally I was going to write a simple functional global variable.  But an FGV implies that the data is stored within the FGV itself.  It's what I would suggest if you were trying to update an array that has a scope within your application.  But since you were commenting on network shared variables, that means the data has to be stored within the shared variable service rather than within the application.  So the FGV concept would lose its storage, and becomes simply a non-reentrant subVI that reads the array from the shared variable, replaces the array element, then writes it back to the shared variable.  It encapsulates it in a locking mechanism which is not difficult to implement.

But if that still doesn't work for you, then apparently the shared variable engine would not be processing the read and write requests in the same order that they are received.

Is there another computer or application that is also trying to write to the same shared variables?  If not, then I would use an FGV and just have an update of shared variable occuring as an action within it so that other PC's or applications can read the updated data that is stored within the application that is storing and permitted to update the shared variable.

[SteenSchmidt]

@Intaris: Latency and Shared Variables? Shared variables feed on latency, so don't fret giving them some more of that SVs are not built for streaming or high-throughput, they are built for about 1 Hz update rate.

No atomic encapsulation can be built around a Network Published Shared Variable, as you cannot control who else writes to the variable at any given time. The only way to ever get around this would be to introduce a read-modify-write API method, but that would probably not be desirable for the network variants of the SVs. It would have to entail a locking mechanism on the SVE that hosts the SV. The few times I get so low on caffeine that I use a Network Published SV I always make them one-way point-to-point, and then use another transport for the data inside each application at either end. I never distribute SVs throughout my applications. I can't spare the time for removing them again later when they fail on me .

Oh, and NI recommend you use the SV API for "scalability" instead of the SV nodes.

/Steen

[NI-MarkN]

Variables are accessed by separate machines and FGV in this case would not be a viable solution. Essentially what I was thinking of was an update operation that would have a single lock, where instead of updating the whole array you could only update a single value within it. Thanks for the comments by the way!!

[AristosQueue (NI)]

Just create a subVI. A regular, plain, nothing special subVI.

Select the variable read, the Replace Array Subset and the variable write and do "Create SubVI". That's it. Done.

[SteenSchmidt]

@AQ: This is a network enabled Shared Variable. The subVI won't lock another target out from racing your read-modify-write. That can only be done on the SVE.

A famous quote from Einstein: "Make things as simple as possible, but not simpler."

/Steen

[AristosQueue (NI)]

"Essentially what I was thinking of was an update operation that would have a single lock, where instead of updating the whole array you could only update a single value within it."

A lock that only locks a single element. It can be done. It has some drawbacks:

1. When you wrote an array to the shared variable, you'd have to allocate all those locks (or, at least, you'd have to allocate an array where those locks could live... the actual locks themselves can wait until you need to lock an element). So you'd have increased memory overhead.
2. You would need to have any code that wrote the entire array acquire all of the individual locks. This would make writing the array slower.

I think there's also something I remember reading a while back about the problems of acquiring a lock on a remote machine, doing the work locally, and then releasing the lock. There was some sort of complication involving network reliability and not knowing when a connection dropped whether or not to release the lock. I think. Would have to dig into that. That would apply to locking the whole array or to locking a single element, but I think it got more complicated when there were multiple locks involved (i.e., when writing the array had to acquire all the element locks).