Actor Framework Discussions
Data Transport Options
My team and I are having a debate about how to send data to other actors and I'm curious what others here think.
Here's the (generalized) scenario:
- There's one actor (or subtree of actors - it's not really the point of this debate) that acquires data from different sources and aggregates it (Data Aggregator actor) into a current value table (CVT).
- Other actors in the application are interested in this aggregated data for logging, display, etc., but at different rates. For example, a logging actor might want to log data at 100Hz whereas one view actor might want to display the data at 60Hz and another view might end up displaying it at 24Hz.
- The rates of data consumption are user-configurable so the software should be able to handle different values (at run-time) than the examples I gave.
- The data consuming actors may be multiple AF tree nodes away from the Data Aggregator actor.
- Data can be lossy (hence the CVT).
- The Data Aggregator updates (much) faster than any consumers of the CVT data.
How should the Data Aggregator actor distribute the CVT data to these other actors? Here are two options we're considering:
- The Data Aggregator learns of all the data consumption rates, finds the lowest common multiple and sends the CVT data out via a standard AF message at that rate. Each actor interested in that data would be responsible for determining if, when the data is received, whether it's actually time to log it, display it, etc.
- The Data Aggregator, on launch, initializes the CVT and then creates a DVR for it. The DVR is stored in the Data Aggregator's private data (so the CVT can be updated) AND also constructs a different class where the DVR is the private data of that class (CVT class). Aside from the construct method, the API for the CVT class only allows reads of the CVT from the DVR so that users of the class aren't be able to update the CVT themselves. The Data Aggregator actor would send this CVT class via AF message to the consuming actors ONCE. From then on the CVT consumers would rely on their own internal methods to control the timing of reading the data.
Option 1 doesn't involve any data transport mechanism other than AF messages and the direction of messages to consuming actors can be made easier via tools like the (shameless self plug incoming) Zyah AF Forwarding Utility, but involves many more messages and copies of data.
Option 2 eliminates most of the messages to the data consumers, but at the expense of "shortcutting" the AF message tree. It also (arguably) simplifies the handling of data on the consumer side - actors only use the class to read the CVT when they want to use the data rather than having to deal with extra incoming messages.
How would others resolve this issue? Perhaps there's a 3rd option that's even better?
Data doesn't need to be lossy, only logging and display. That would mean additional logic in any view or logging actors to discard unnecessary data, but everything's on the table.
Keep your life simple. Option 2.
Just have your aggregator throw all the data into a DVR and each process pulls out what it needs when it needs it.
@drjdpowell wrote:
This reminded me of this suggestion I made: https://forums.ni.com/t5/Actor-Framework-Discussions/Suggestion-a-Queue-Empty-overridable-method/td-...
I hadn't see that before but I like the idea.
@Taggart wrote:
Keep your life simple. Option 2.
@drjdpowell wrote:
Note that if you share a DVR you give up the ability to host your UI actor on a different computer to your data producer. You also get the potential for race conditions between the DVR poll and incoming messages. None of this may matter but it is a concern and I'd say this is actually the more complex option, even if it seems simple.
We're not concerned with the ability to host the UI actor on a different computer in this case, but it's funny - you guys have stumbled into the same debate our team was having - is the DVR actually simpler?
About the race condition, I don't quite see where it comes into play. If you're reading from the DVR in the payload method of your incoming messages, you'll always be acting on the most recent data. On the other hand, using the message approach we'd end up updating the same data internally in the view/logging actors and then I can see a race condition with regards to whether or not the incoming messages are acting upon the very latest data.
As a modification to option 1, you could have the subscribe method start a time-delayed send message to self (Aggregator Actor) with the subscriber's enqueuer stored in the message data.
Then you don't need factors/multiples/coercion. Just when the Aggregator receives the self-sent message, take the current values and the bundled (in message data) enqueuer and send the update message.
