NI needs to teach more actor-oriented programming principles

[Elijah_K]

Perhaps it is simply a semantic issue, as we've just recently introduced the notion of an 'Actor' into the G programmer's lexicon, but actor-like components have been a core aspect of LabVIEW for decades.  A consumer loop, or anything that effectively performs an action based upon an instruction sent from an a-sync process is an Actor - it literally performs an action based upon a request.  To this end, I would maintain that we've been teaching actor oriented design as long as we've been teaching QMHs.

Just my 2 cents,

Eli

[David_Staab]

Eli, your 2 cents are dangerously incorrect: A QMH is not an actor. It depends critically on the order in which messages are sent to it; an actor does not. That difference is what makes an actor robust, debuggable, and reusable. Noting that, the challenge is that NI has not been teaching actor oriiented design ever, and y'all have in fact been promoting practices antithetical to actor oriented design.

[Daklu]

Elijah_K wrote:

A consumer loop, or anything that effectively performs an action based upon an instruction sent from an a-sync process is an Actor - it literally performs an action based upon a request.

I disagree.  An "actor" in AOD is implemented so that it adheres to certain principles.  One of those principles is "an actor can receive any message at any time."  As David pointed out, QMH/QSM implementations very frequently violate that rule and require specific sequences of messages in order to work properly.  That sequential requirement is the direct cause of what AQ called, "the two-communications-channels-deliver-out-of-sequence-messages-leading-to-undefined-state bug."  (I just call it the QSM problem, but that works too.)

That dependency on specific sequences is also what has led to requests for the addition of a batch enqueue feature.  I've opposed those ideas when I've seen them because if you need a batch enqueue in a message sender, it indicates something is dangerously wrong with your receiver.

QMH/QSMs and AOD share some of the same ideas. This isn't unexpected, since they both attempt to address concurrent programming. But, just as is it incorrect to call a VW bus a Porche simply because they both have four wheels and an engine, it is incorrect to call a QMH/QSM an actor simply because they both perform actions based on input from an asynch process.

Elijah_K wrote:

I would maintain that we've been teaching actor oriented design as long as we've been teaching QMHs.

Admittedly, I haven't gone through all of NI's training material, so I can't directly dispute the claim.  However, other than a few documents Stephen has published I have seen *no* evidence that NI has been teaching AOD.  Not on the forums, not in the examples, not in the community, and not in any of the training material I have seen.  

Stephen identified four categories of bugs frequently encountered in concurrent applications:

1. Mutual request deadlock

2. Missed stop signal

3. Infinite echo chamber

4. Undefined state due to out of sequence messages

These are modelling errors, not implementation errors.  All of these potential bugs can be managed effectively if you follow the ideas AOD encourages.  Before people can follow those ideas, they have to know and understand them.  It's this fundamental understanding of the actor model of concurrency that is missing, not only from the broader user community, but (as suggested by your comment) from NI's corporate knowledge base as well.  As much as I stand on my soapbox and preach the principles, I don't have the influence to effectively counter NI's messaging.  If it is important for developers to understand AOD--and I strongly believe it is since it is the simplest model shown to be effective at managing large-scale concurrency--then NI needs to understand the model and figure out how to communicate it to users.

(FWIW, even though I have a different philosophy than Stephen regarding APIs and have concerns about building an ecosystem based on the AF, I am not at all opposed to the AF.  Really.  I do believe encouraging people to use the AF without first laying the foundation for AOD is ultimately counterproductive, both in terms of the customer's goals and NI's goals.)

[Elijah_K]

David - I have to disagree. Of course an actor depends upon message order! A message invokes a command - the order messages are received dictates the order commands are processed. How is that any different from a QMH?

[Elijah_K]

Daklu - in my humble opinion, your differeniation is based on implementation details that are not intrensic to Actor Framework or QMH.  A QMH may require a series of messagers to work, or as I see more commonly implemented, it may just receive a command and then perform the appropriate action - this is extremely similar to the Actor Framework.  These commands may come in an arbitrary order, but both will handle them in the order in which they are dequeued.

My original point was simply that we have been teaching the importance of creating async processes that are capable of sending and receiving messages for many years - this has come in form of producer / consumer loops and queued message handlers.  These concepts are fundamental building blocks towards mastering the principles found in the design of the Actor-based architecture.

[David_Staab]

Elijah_K wrote:

David - I have to disagree. Of course an actor depends upon message order! A message invokes a command - the order messages are received dictates the order commands are processed. How is that any different from a QMH?

No, the sender of a message makes a request; the actor retains the right to decide whether to act on that request now, later, or not at all depending on its current behavioral state. It may process the message after another has come in, when it's ready to process that message. Or it may not; the caller has no authority to dictate in what order messages are processed. Here's my explanation of this in terms of AF messaging. Here's wikipedia's breakdown of the fundamental concepts of the actor model of computation.

[David_Staab]

Elijah_K wrote:

My original point was simply that we have been teaching the importance of creating async processes that are capable of sending and receiving messages for many years - this has come in form of producer / consumer loops and queued message handlers.  These concepts are fundamental building blocks towards mastering the principles found in the design of the Actor-based architecture.

But they aren't. The QMH is an anti-pattern for concurrent computation. For example, I'm writing an app now that includes an instrument that implements a state machine. Control over the instrument is achieved by multiple means: in one deployment it's a GUI, in another it's a CLI, in yet another it's a TCP interface (as a web service). If the instrument receives a "run" message while in a state that doesn't transit to the "running" state, what should it do? A QMH must process the message and may end up throwing an error or getting into a bad state. As such, each of the three components that may control this QMH have to assume responsibility for managing/tracking the QMH's behavioral state and sending messages to it in correct sequence. An actor, however, can choose to cache the message and process it when it's in a good state to do so, or it can ignore the message (optionally replying to the sender with a NACK message). The difference between them lies in the intent of their designs, not in the implementation of their code. (I can write an actor using AMC instead of AF if I want to, and I've certainly seen people trying to write QMHs with AF like they do with AMC.)

[KALee]

To add another resource to this discussion.... Here is a discussion between Erik Meijer and Carl Hewitt, a key player in the development of the Actor Model.  It contains similar information as the Wikipedia entry, but the format allows fror some interesting inights and clarifications.

[Daklu]

Elijah_K wrote:

Daklu - in my humble opinion, your differeniation is based on implementation details that are not intrensic to Actor Framework or QMH. 

That's exactly right.  You can use QMH-style code to implement actors (I do it all the time), and you can use the AF to write non-AOD code.  The QMH and the AF are implementations.  AOD doesn't imply or impose any implementation.  If the implementation has the characteristics of an actor, it *is* an actor.  To repeat what I said in post #16,

The only important difference between an actor implementation and a QSM implementation is that the actor follows the guidelines and a QSM does not.

Elijah_K wrote:

My original point was simply that we have been teaching the importance of creating async processes that are capable of sending and receiving messages for many years - this has come in form of producer / consumer loops and queued message handlers. These concepts are fundamental building blocks towards mastering the principles found in the design of the Actor-based architecture.

Yes, NI has been teaching sending/receiving messages for years, but it hasn't been teaching them in a way that allows people to be able to create reliable concurrent applications.  AOD is much more than just sending and receiving messages.  It's a completely different mindset than QMH programming.  It's hard to explain exactly how it is different or why it is better.  It may be one of those things you have to experience to understand.  (It's similar to the difficulty in explaining how OOP is different to someone who has only done procedural programming.)

Elijah_K wrote:

David - I have to disagree. Of course an actor depends upon message order! A message invokes a command - the order messages are received dictates the order commands are processed. How is that any different from a QMH?

David Staab wrote:
No, the sender of a message makes a request; the actor retains the right to decide whether to act on that request now, later, or not at all depending on its current behavioral state.

To add a bit to what David said, the question of issuing a command versus sending a request, is extremely important to understanding AOD.  The difference is in who is responsible for deciding whether the receiver should act on the message.  In the typical QMH model, message handlers are implemented as commands.  The receiver immediately executes the command as soon as it is received.  The assumption is the sender will only send the messsage at times the receiver is able to handle it correctly.

This makes the sender responsible for figuring out if the receiver is in a state where it can handle the message prior to sending it.  I call this "sender side filtering."  Any non-trivial implementation that relies on sender side filtering is flawed.  It has a built-in race condition.  It requires the sender to know what state the receiver will be in when it eventually processes the message.  In simple cases it's possible for the sender to know that information; in the general case it is (for practical purposes) unknowable.

Deciding whether or not a message should be acted on is always the responsibility of the receiver.  ("Receiver side filtering.")  The point of calling it a request is to reinforce the proper assignment of responsibility.  Not all receivers need filtering, and sometimes it makes sense to implement a filter on the sender that duplicates the receiver's filter (i.e. if you're trying to reduce the load on the messaging system,) but you cannot rely only on a sender sider filter and expect your program to behave reliably.  That is one of the fatal flaws of QMH implementations.

[KuGr]

So, in all of this, do we, the collective WE, the LabVIEW community that sees the benefits of AOD, have a list of what 'actor-oriented programming principles' are?  I see fragments in many threads here on the AF group and in LAVA.  In Daklu's last message: "Deciding whether or not a message should be acted on is always the responsibility of the receiver."  Should this be a principle?  Probably.  What about what Hewitt says?  Some perhaps, some is very theorretical.

We can wait for NI to start teaching them, but not everybody takes their classes.  Personally, I have never taken the Core classes so I have not been jaded by NI's approaches to concurrent programming.  I have seen the results of code written in LabVIEW by those that have taken the courses and it was not good.  My point being that the teaching NI might do may not be the result you want.  So, lets get started ourselves.

In the end, it has to be readable.  I am not a CompSci guy.  I have just written a lot of code in my career (only a small amount is LabVIEW).  I don't have the time to digest tech papers.  Do I want bullet points?  YES, but obviously some explanation is needed.

Maybe one of you is planning on this as a presentation at NIWeek.  Great.  Maybe vet your ideas here.  I still promise to come to your presentation even if I know how it ends.