Windows running out of memory all of a sudden

[cold_matt]

I have an executable built from some LabVIEW VIs which used to run just fine and would happily chug along for weeks on end, but now (and I'm not sure what I changed in the build to cause this) seems to cause Windows to fill all its memory after a few days. Opening task manager with the .exe running shows the application using only about 110MB of memory, and the machine has much, much more than this available. Is this a known phenomenon that can occur with certain configurations, and what are some simple ways I can avoid it happening?

I am currently on a Linux machine without a LabVIEW license, but I will update this post with example VIs when I have a chance.

[Sam_Sharp]

Something must be using up all the memory - if you're saying that your application is only using 100MB and it's not growing then it must be something else on your system?

Maybe try using the performance monitor (either the in-built one or something like process explorer) to view/log the memory usage over time and try to narrow down the application (or service) that's taking up all of your memory.

Typically, things like 'build array', enqueuing data into a queue with no maximum size or reference leaks (e.g. opening a .NET reference and never closing it) are common causes of memory leaks in applications.

[cold_matt]

Hi Sam, thanks for your reply.

I'll have another look at the performance monitor, but the last time I checked I sorted the processes by memory usage (in Task Manager) and the application came at the top, implying there was nothing else contributing more.

Although my code is fairly messy in places I have tried to avoid those pitfalls you've outlined. I certainly don't think I've got any arrays growing arbitrarily large, I haven't used any queues or .NET references.

Please find attached the codebase that I currently have. The top-level VI is "MainMachineTemperatureUpdateCleaning.vi".

[Sam_Sharp]

It's quite hard to look at your VI because of the size of the block diagram - SubVIs are your friend! It's generally recommended to have your block diagram take up just 1 screens worth of space.

You have a shift registers of an array in the bottom loop - does this get cleared at any point or do you keep adding data to it indefinitely?

I don't think that services show memory usage in task manager - it could be that your memory is being used up by a service. Again - a decent process monitor that shows service memory usage might help you figure out where the problem lies.

[VItan]

Exactly what behaviour do you observe when Windows "fill all its memory"?

To track down this kind of behavior on Windows, you probably are going to need a thorough understanding of memory management in Windows.

(I am assuming that you are working with Windows 7)

As you noticed, the "Memory (Private Working Set)" field in Windows Task Manager does not include all the information about allocated memory.  In Windows, a process can have memory in three ways:

Private Bytes, the actual memory that the process has to itself, which is accounted for in both the process's virtual memory space and the OS's commited memroy table.  This is the traditionally thought-of memory allocation.  Private Bytes count towards the system commit limit and the process’s virtual size and are reported by Task Manager.  Note that a process cannot allocate private bytes that extend beyond its address space, e.g. a 32-bit process cannot have private bytes that would make its total virtual space exceed the 4 GiB address space limit.

Reserved Virtual Memory, memory addresses that the process has asked the OS to reserve (due to something like specific address or contiguousness requirements), which is accounted for in the process’s virtual memory space but is not committed to by the OS.  This does not count towards the system commit limit and therefore does not contribute to Windows identifying / reporting a low memory condition or any performance problems.  However, the reservation bookkeeping itself does count towards the system commit limit.  Note that a process can not reserve virtual memory that extends beyond its address space, e.g. a 32-bit process cannot reserve virtual memory that would make its total virtual space exceed the 4 GiB address space limit.

Pagefile-Backed Virtual Memory Objects, memory that is shareable and that the process has a handle to, which is not accounted for in the process’s virtual memory space but is committed to by the OS.  These objects are called Sections, and they do count towards the system commit limit.  Because they are shareable, these Sections are not associated with the process for the purposes of reporting in Task Manager, Process Explorer, or the System Event Log.  However, the process will have to maintain a Handle to the Section.  Note that a process can create more Sections than it would normally be able to address, e.g. a 32-bit process can request allocations of Sections such that the total exceeds their 4 GiB address space limit.

What you see reported in Task Manager is only the Private Bytes of a process, which means two things:

1. You don't see the full virtual size of the process (however, this doesn't matter, because any excess virtual size that is not commited does not count towards the OS commit limit).
2. You don't see any pagefile-backed shared memory objects that the process created and is maintaining handles to.

I would suggest using a process explorer that allows you to monitor all the memory management details, such as Sysinternals Process Explorer.  If no one is apparently using up much memory, start monitoring their handles and see if someone has a ton of Sections where they are hiding memory allocations.

Also don't discount that it may not be your application that is responsible; if anything has been changed or installed on the target machine, even something like a driver for a USB device, then that may be the culprit.

[billko]

Curious... you mention nothing about out of memory errors.  Could it be that you are misinterpreting what you see?  "Free" memory no longer means "available" memory.  Typically a Windows (Vista or better) operating system will try to use up all available memory by doing stuff like preloading often used applications and other stuff, so truly free memory will be very, very small.  (The idea is "why have all that idle memory when it could be used to speed up the system?")  However, this so-called "Standby" memory is readily given back to the pool of available memory if needed.  So in task manager, allyou really care about is the "available" memory.  In the resource monitor, you want to look at "Standby" + "Free" to determine how much memory is available for use.