ACBR Proxy Caller/ Memory Leak problem

[Drew.Rankin]

That sounds like what I was suspecting; however, I'm not sure where to look within the framework. Is it proper to dispose of the queue reference after a read caller enqueuer or read self enqueuer? I haven't seen examples of this in any of the template code or any of the automatically generated code in the message maker.

Can you explain a bit or point to a post discussing the creation and disposing of queues within the AF or where the queue structure came from for AF?

Thanks

[AristosQueue (NI)]

Drew.Rankin wrote:

That sounds like what I was suspecting; however, I'm not sure where to look within the framework. Is it proper to dispose of the queue reference after a read caller enqueuer or read self enqueuer? I haven't seen examples of this in any of the template code or any of the automatically generated code in the message maker.

One of the nice points of the AF is that it completely takes care of the references for the queues themselves. When an actor launches, the queue is created. When the actor stops, the queue is destroyed. It does not matter how many times you acquire the enqueuer refnum in the interim -- you do not need to dispose/delete/release anything other than sending Stop to the actor itself.

If there's any leak, it is almost certainly going to be in your code, with a refnum that you're allocating, not in the framework. Are you dynamically opening VI Server references to VIs or applications? Are you opening your own queues/notifiers/user events for some part of your code?

[AristosQueue (NI)]

Drew.Rankin wrote:
That sounds like what I was suspecting; however, I'm not sure where to look within the framework. Is it proper to dispose of the queue reference after a read caller enqueuer or read self enqueuer? I haven't seen examples of this in any of the template code or any of the automatically generated code in the message maker.

For the record, there is no "Release" function that you even *can* call. The AF manages those references. All you can do is send messages. This is discussed in the AF white paper (create a sample project and it's the HTML file created in your project... also available online from the READ THIS FIRST page) when we talk about encapsulating the queue reference.

[Drew.Rankin]

The only place I create a reference that you list is in the serial actors. Below is a look at one of them.

Another part of the story is that I had developed the application using the Debug Actor Framework and was concerned that if I was not actively looking at the debugger messages, then the queues for the debug actor would get very large unless they were limited in size somehow. I rebuilt the project using the standard Actor Framework and have been monitoring the application for a couple of days and it seems to have leveled off at 104MB of memory when running and the customer has not seen the application exhibit symptoms of running low on memory or queues being quite full trying to process messages.

Unfortunately, I do not have the time to dive into IF using the Debug Actor and not dequeueing messages from its queues would cause the growth in memory footprint I was seeing or not. What is interesting to me and curious is that the memory footprint of the application seems to change when sending a message that has not been used by the application previously.

Can someone talk a bit about how memory is allocated by a queue when it is first created and how memory is allocated when objects are enqueued then dequeued? Does the memory footprint of the queue change dynamically or is it allocated incrementally? I guess what I am asking is if a queue is created and has objects enqueued, when those objects are dequeued, does the queue release allocated memory when objects are dequeued?

Sorry for so many questions; however, I see huge potential for the AF in many applications that will allow me to develop really robust applications more quickly. I admit that I am still coming up the curve on what is happening behind the scenes.

I also wanted to make sure that AF does in fact handle creation and destruction of all queues within the framework and that there wasn't some corner case where I would have to handle releasing queues manually.

Thanks,

Drew

[SteenSchmidt]

Can someone talk a bit about how memory is allocated by a queue when it is first created and how memory is allocated when objects are enqueued then dequeued? Does the memory footprint of the queue change dynamically or is it allocated incrementally? I guess what I am asking is if a queue is created and has objects enqueued, when those objects are dequeued, does the queue release allocated memory when objects are dequeued?

When memory is first allocated for a queue it won't be released again until that queue is destroyed.

The queue starts up with a minimum amount of memory allocated (how much will probably depend on LV version). Then you enqueue data. When you run out of memory in the original queue LV's sub-allocator will (hopefully be able to) hand out a new memory block for the queue, a memory block that is somewhat larger than "just enough" (how much larger I don't know, but it's the same when building arrays and such). In that process the old memory block will be released for use by the sub-allocator. The reason for the allocation of a completely new memory block is that queues need contiguous memory space just like arrays - that's also part of the reason behind the "allocate more memory than necessary right now", since memory fragmentation can stop your queue from getting the memory block it needs.

And as said, the allocated memory doesn't ever get released again, except when the queue is destroyed. The biggest size the queue ever was is how much memory it will occupy for the rest of its lifetime. But that's not necessarily bad (if you have enough memory), as it lowers jitter and boosts performance.

Also note that the queue will only hold handles to dynamic data structures such as arrays - the arrays themselves won't be inside the queue memory block, they exist in each their own contiguous memory block elsewhere. So arrays in a queue acts memory wise like a linked list not needing contiguous memory as such.

Cheers,

Steen

[AristosQueue (NI)]

Steen is correct. The only bit I'll add is that the queue size is (if I recall correctly) enough space for one element on a desktop system unless you wire a max size into the Obtain Queue function. If you wire max size, I don't remember how much we allocate on the desktop. On RT, wiring max size will always allocate that much space in the queue so as to avoid RT jitter. On desktop, we might do the same, but I remember we tried several variations, and I don't recall which one we went with as the best balance for performance vs memory.

When a queue needs to resize, we follow the algorithm used by many dynamic allocation data structures to achieve amortized constant time -- when the queue fills up, we reallocate with a queue that is double the current size (so 1 element queue becomes 2, 2 element becomes 4, etc) until we hit 2048 elements. After 2048 elements, we add 1024 elements each time. The relative rarity of systems with extremely large queues means that most systems amortize nicely up to their spike point and after that we try to stay conservative so as to not over flood the system for a given spike of throughput.

[David_Staab]

AristosQueue wrote:

If you wire max size, I don't remember how much we allocate on the desktop.

IIRC, it doesn't change the allocation immediately. You have to enqueue up to <max size> and flush before using the queue in order to force that allocation.

[Drew.Rankin]

Fantastic. Thanks for the explainations, it does make sense. Now to just figure out if I am leaving any messages hanging in a queue somewhere. Where to start?

Incidentally, the customer has been using the application for a few days now and the memory seems pretty stable.

So now the question becomes, if I am using the Debugging AF and not looking at the messages or activity it is creating, will there be a large queue of objects floating around with no one to receive them?

Thanks,

Drew

[AristosQueue (NI)]

You know, it occurs to me that the issue might be that the Debug AF maintains a log of events... that log just keeps getting longer. It might be that there is no leak, but the memory usage you're seeing is a direct result of the deubgging doing exactly what it was asked to do.

Switch back to the normal AF and just watch your memory usage in Windows over time -- does it level out after a while?

[Drew.Rankin]

AQ,

Yes, switching back to normal AF and the application levels out at 104MB. This was definitely not the case when Debug AF was used. Too bad there isn't an easy way to switch between the two versions.