Actor Framework Discussions

cancel
Showing results for 
Search instead for 
Did you mean: 

Actor Framework Inheritance and Execution Systems

Hello all,

I'm running a lot of actors (300+). Some of them are UI, some are daq, some are computation.

Running on a fairly beefy machine (32 cores, 64GB ram, 64 bit LabVIEW)

After much debugging it looks like we're starting to run into problems with thread contention. I'd like to be able to change the execution system of different actors, but they all inherit from the same base actor and LV won't let me change some and not others. I've tried programmatically setting it as well but it just generates a "1000" error.

Any ideas?

0 Kudos
Message 1 of 16
(9,768 Views)

First idea that comes to mind:

Edit Launch Actor so that instead of always calling "Actor.vi", you pass in a VI reference for it to call. Create multiple VIs that call Actor.vi, one in each of several different execution systems. Since AF uses "same as caller" that should mean each gets launched in the appropriate execution system.

Message 2 of 16
(5,291 Views)

That's doable

Thanks AQ

0 Kudos
Message 3 of 16
(5,291 Views)

A) I'd like to congratulate you on making a completely novel feature request for the Actor Framework.

B) I'd like to hear more about your scalability issues. Up to what point was the AF behaving well? 100 actors? 200? Are most of your actors asleep waiting for messages at any given time or do you have many actors that are acting simultaneously? How many developers are on your team and how are they handling the architecture?

While I've heard many tales of AF success, most have been in the 10 to 20 actor range. You're the next order of magnitude up. If you have stories (and time/permission to share them) I would be interested, and I think many other readers on this forum would be too.

Message 4 of 16
(5,291 Views)

Proud to be part of the process AQ.

Not sure if you remember, but this is the same software we showed you the class diagram for at NI Week that your team snagged a picture of(I'm at Prolucid). Once the first major production delivery of the software completes we're likely to do a case-study on it which usually covers NI technologies used. In this case I'll put in to talk about the use of AF in a second study for the project. Assuming we get permission to talk about the customer/project I'd be more than happy to talk details about how it all went.

That said, as a preview: I can't imagine doing a large project anymore without using AF. This particular project is one of Prolucid's largest LabVIEW developments, involves over 1000MB/s being processed/streamed to disk, a network UI with 20+ subpanels, and the vast majority of actors are firing/receiving messages at about 10 msg/s.  We've invested a fair amount to build up tooling around AF, but the basics remain the same.

0 Kudos
Message 5 of 16
(5,291 Views)

I use a different system than the AF, but it uses similar dynamic actor creation, and no messing with execution systems.  I have a “Test Forwarding Ring” test VI that creates “actor rings” of N actor that forward messages around the ring.  Bursts of M messages are timed as they take a full circuit.  I see no difference in message-passing time with rings of 30, 100, 1000, or 10000 actors.  CPU usage is about 95%.  So I would be surprised if the AF had issues with only 300.  Someone should make a AF “Actor Ring” and experiment.   — James

0 Kudos
Message 6 of 16
(5,291 Views)

drjdpowell, it sounds like the difference is that all of the actors in his app are generating those messages. I know the AF performs very well when you have a long chain of actors and you are sending messages up and down the chain. What I don't have any ideas about is how it performs when every point along the chain generates messages at the same time in both directions.

Here's the comparable benchmark (I think):

How does your system fair when you have a ring of N actors and all N fire Mx2 messages at the same time and send half clockwise and half counterclockwise?

0 Kudos
Message 7 of 16
(5,291 Views)

This thread has peaked my interest as we typically have on the order of 100 Actors in an application.  In fact I was discussing with a coworker earlier this month the ramifications of having ~500 actors in an application, we decided that we didn't know what the limitations were, even worse we didnt have a great way to test for limitations.  As such I was excited to read about the idea of the actor ring, so I built it.  But I want to be sure that we are comparing apples to apples.  My version of the actor ring launches N of the same actor, where all the actor knows how to do is launch a version of itself and pass that nested version of itself a message (which it will pass to its nested version of itself...  An actor Ouroboros).  I wrote in the ability to pass the message down the tree (clockwise I guess), up the tree (counterclockwise?), and in both directions at once.  I also copied your idea of sending batch messaging.  When I launch 10000 actors my CPU jumps to ~15% during the launch, which takes about 30 seconds, but then drops back down to ~0%.  When I pass 1 message in 1 direction around the ring it takes ~0.3 seconds and the CPU spikes to ~12%.  When I pass 1 message in both directions around the ring simultaneously it takes ~.45 seconds and the CPU spikes to ~15%.  When I pass a batch of 100 messages around the ring in 1 direction it takes ~30 seconds and the CPU spikes to ~35%.  And when I pass a batch of 100 messages around the ring in both directions simultaneously the passing of messages from the top down takes ~50 seconds while the passing of messages from the bottom up takes ~40 seonds, and the CPU fluctuates between 50%-75%.

What do we make of this?  Does it line up with what you see with your implementation of Actors?  Is this really an analog for an AF project?

0 Kudos
Message 8 of 16
(5,291 Views)

jon_mcbee wrote:

But I want to be sure that we are comparing apples to apples...

When I launch 10000 actors my CPU jumps to ~15% during the launch, which takes about 30 seconds, but then drops back down to ~0%.   ...

When I pass a batch of 100 messages around the ring in 1 direction it takes ~30 seconds and the CPU spikes to ~35%. 

I launch the (non-AF) actors in a loop from the top level, and pass each actor the previous actors's address (sorry, "enqueuer") in an initial message.  Launching 10,000 takes 37 seconds and uses one core worth of CPU (25% on my 4-core machine). 

Sending 100 messages along the chain takes an average of 8 microseconds per pass (8 seconds for the full test of 100x10,000 passes).  This uses 95% of CPU (so most of the 4 cores).

0 Kudos
Message 9 of 16
(5,291 Views)

Out of curiosity, I launched 65536 actors and passed a message from the top of the chain to the bottom and it took 1.8 seconds.  Passing from bottom to top took about the same.  I launched a batch of 10 messages in each direction and it took ~30 seconds up and ~40 seconds down.  I really didnt expect this to work.

Message 10 of 16
(5,291 Views)