“asynchronous is best” they say. OK, but do you also need to maintain state on data you’ve requested & waiting for in addition to the internal state?

[DoctorAutomatic]

I “get” the advantages of asynchronous communication between modules (request, while also registered for the relevant broadcast instead of a request vi that doesn’t return until there’s a reply with data on the output terminals), I guess. 

but I’d be lying if I said it’s that simple and I haven’t ever had a real brain puzzle on my mind. The problem, seems to me (and best I can articulate), is that the caller/requester has to keep state information on what has been requested and what data we’re waiting to receive, in addition to the usual internal state data of the module. Probably also need some timeout logic, for when things take awhile or other modules are having problems. It all makes my head hurt just thinking about any simple, non-trivial, non-PITA to code, or code that might end being a problem area itself in the future. So how do you handle doing things mostly asynchronous between modules? What do you do about new inputs/requests that arrive in the caller module that may or may not have any need or interaction with data the module is waiting for? State is one of the worst things to maintain when things are complicated, adding additional states to monitor is something I try to avoid if possible. A lot of things I have to code involve processes that shouldn’t or can’t start until a bunch of prerequisites/conditions are satisfied, and often there’s no order in which they are set and it’s possible for their state to change from a go/no-go state whenever. Also a lot of states where various groups of commands should absolutely not be allowed. In other words, already plenty of things to keep track of without keeping track of whether I’m waiting for a response to an asynchronous request, if I’ve been waiting for too long (timeout), if i should make a new request, etc. I have a feeling there must be logical way to deal with asynchronous messaging that isn’t coding hell, but it’s not readily obvious to me. 

thoughts?

[Ozfarmboy]

May I suggest you please give us a specific real-world example to help explain your problem?

[Darren]

I regularly use Request and Wait for Reply events with DQMH. Other frameworks have been known to label async messaging as anathema, but it has yet to cause me any real headaches with DQMH.

If your caller needs a response back before it can reasonably proceed or change states, then use a Request and Wait for Reply. 

[drjdpowell]

Both Async and Sync are valuable.  Historically, we've had a lot of Sync-only technologies (like Action Engines, for example), which led to some trouble as people try to use them to do things that really should be asynchronous.  Then when getting their hands on some async messaging, it is common to go overboard in the other direction, trying to do, all-asynchronously, actions that really should not be asynchronous.

I talked about how the Continuous Measurement and Logging example (that ships with LabVIEW) makes exactly this kind of mistake:

What would a subVI do? 

[joerg.hampel]

---

@DoctorAutomatic wrote:

A lot of things I have to code involve processes that shouldn’t or can’t start until a bunch of prerequisites/conditions are satisfied, and often there’s no order in which they are set and it’s possible for their state to change from a go/no-go state whenever. Also a lot of states where various groups of commands should absolutely not be allowed. 

---

It sounds like part of what your describing would be covered by a state machine if we (HSE) were to work on it.

In our experience, with our typical designs, I think many of those things you describe would be broadcasts, which we register for in our State Machine module, and which would trigger transitions. The state machine then decides what to do with it.

I agree with Darren and James. Both sync and async have their place, and rightfully so.

[Taggart]

---

@DoctorAutomatic wrote:

I “get” the advantages of asynchronous communication between modules (request, while also registered for the relevant broadcast instead of a request vi that doesn’t return until there’s a reply with data on the output terminals), I guess. 

but I’d be lying if I said it’s that simple and I haven’t ever had a real brain puzzle on my mind. The problem, seems to me (and best I can articulate), is that the caller/requester has to keep state information on what has been requested and what data we’re waiting to receive, in addition to the usual internal state data of the module. Probably also need some timeout logic, for when things take awhile or other modules are having problems. It all makes my head hurt just thinking about any simple, non-trivial, non-PITA to code, or code that might end being a problem area itself in the future. So how do you handle doing things mostly asynchronous between modules? What do you do about new inputs/requests that arrive in the caller module that may or may not have any need or interaction with data the module is waiting for? State is one of the worst things to maintain when things are complicated, adding additional states to monitor is something I try to avoid if possible. A lot of things I have to code involve processes that shouldn’t or can’t start until a bunch of prerequisites/conditions are satisfied, and often there’s no order in which they are set and it’s possible for their state to change from a go/no-go state whenever. Also a lot of states where various groups of commands should absolutely not be allowed. In other words, already plenty of things to keep track of without keeping track of whether I’m waiting for a response to an asynchronous request, if I’ve been waiting for too long (timeout), if i should make a new request, etc. I have a feeling there must be logical way to deal with asynchronous messaging that isn’t coding hell, but it’s not readily obvious to me. 

thoughts?

---

Based on your project description, why are you trying to do things asynchronously at all? And if you don't need to do things asynchronously, then why DQMH? DQMH is a great hammer but not everything is a nail.

[Taggart]

---

@Darren wrote:

I regularly use Request and Wait for Reply events with DQMH. Other frameworks have been known to label async messaging as anathema, but it has yet to cause me any real headaches with DQMH.

If your caller needs a response back before it can reasonably proceed or change states, then use a Request and Wait for Reply. 

---

I agree with Darren, but I would add:

"If every message is a Request and Wait for Reply, then you aren't taking advantage of things being asynchronous and you probably shouldn't be using DQMH because it is adding complication for no real benefit."

[Dhakkan]

+1 on Ozfarmboy's request.

Much of what I'm stating below is my understanding gathered from various discussions in the Actor Framework forum. There's a wealth of knowledge there provided by the AF gurus and practitioners.

In communication, favor 'push' over 'pull' where possible (link). The former encourages event-based approach to design thinking, whereas the formerlatter leans towards request-response style thinking.

Start design with not maintaining state. If the modules (actors) are implemented to be responsive, as they should be, then request-events can be replied to almost immediately, within fractions of a second.

If some state info is needed, use boolean flags or enums as guard conditions. The placement of guard conditions would either be in the sender module or the receiver module, depending on the delegation of responsibility desired. For example, if you have a data-logging module that must not log data when testing is 'paused', you could choose to provide a pause/resume message to this module, which alters an internal 'paused?' flag that controls writing to log. Contrast this with the data-source module needing to keep track of when to invoke the data-logger's 'write-to-log' message.

Decompose, if possible, the more complex modules into simpler modules. The complex aggregate ends up becoming a message filter for the simpler ones. (Think tree-like messaging structure.) I see this being akin to hierarchical state machines.

The need for request and wait-because-the-lives-of-the-application-and-its-stakeholders-depend-on-the-response becomes localized, and possibly implemented on a real-time OS at that point.

[Peter_B]

---

If your caller needs a response back before it can reasonably proceed or change states, then use a Request and Wait for Reply. 

---

I agree with Darren, but I would add:

"If every message is a Request and Wait for Reply, then you aren't taking advantage of things being asynchronous and you probably shouldn't be using DQMH because it is adding complication for no real benefit."

---

Wow, that is insightful !   Since recently adopting DQMH, I naively thought it was reasonable to use DQMH for every medium to large sized LabVIEW project I was to undertake from here-on-in (yes I fell for the "everything's now looks like a nail" mindset).   I didn't even consider to avoid DQMH if the project had no async calls in it.  🙄

I assume the correct and popular alternative (non a-)synchronous options (for example) are a simple state machine or the default QMH template in LabVIEW.

Given DQMH is one of the two most popular LabVIEW frameworks, surely that doesn't imply everyone using DQMH needs async calls.  I wonder if others fell for the same trap I did. And if so, shouldn't there be a warning plastered somewhere in the DQMH help manual for new adopters ?  I could only find the following "The DQMH provides the greatest benefits in applications employing multiple modules running in parallel that communicate with each other.",  which seems desirable at an abstract level (i.e. who doesn't want to learn and use a versatile framework) - but clearly it should be avoided for non async projects to avoid pointless complications.  Is the following warning too direct:

"If using DQMH, you WILL adopt pointless complexity if your project does not employ multiple parallel modules talking to each other, instead consider using the basic QMH" ?

[Darren]

---

@Peter_B wrote:

"If using DQMH, you WILL adopt pointless complexity if your project does not employ multiple parallel modules talking to each other, instead consider using the basic QMH" ?

I don't think it's that simple. (And for users of your DQMH module, there's no "pointless complexity"... they're just calling VIs in the public API and registering for broadcasts... the internals of the module are opaque to callers of the module.)

Here's how I look at it. Let's say you have a serial instrument. You would write a standard-looking instrument driver for that instrument... VI wrappers for all the commands you might send, and responses you might receive. Each VI has a VISA in/out terminal pair. That's not a DQMH module.

But then for almost every serial instrument I've worked with, I end up writing an instrument manager, and that manager is a DQMH module. Even if every single command to the instrument is command+response (i.e. synchronous operations), I'll still use a DQMH module. Why?

Because async operations are not the only benefit of DQMH. You get:

1. A tester VI to be able to test your code in isolation.

2. Scripting tools to take care of framework-level scaffolding code so you never have to touch it.

3. Code that starts when you tell it to start and stops when you tell it to stop.

4. Basic debugging tools like showing/hiding panels, showing diagrams, broadcasting errors, etc.

5. The ability to spawn multiple instances of the (cloneable) module.

...

On top of those innate benefits of DQMH, the use of a DQMH module to manage the instrument gives me the ability to add async functionality if necessary. A couple of real-world examples that I've encountered:

1. Helper loop to process streaming data from the instrument.

2. Managing the connection to the instrument, and trying reconnects if the reference goes bad... Reconnects that can be seamless to the end user (since the DQMH module is managing the connection).

3. Registering for broadcasts from other code... for example, a lower-level E-stop module that could broadcast out to all listeners to shut down instruments in an emergency situation.

These are just some of the reasons that, in my opinion, there's nothing wrong with creating a DQMH module with a public API that you envision is going to be composed entirely of Request and Wait for Reply VIs.