The way the DAQ buffer works in continuous mode with DAQmx

[vygr0]

Hello,

I have the below line from the NI examples:

DAQmxErrChk (DAQmxCfgSampClkTiming(taskHandle,"",10000.0,DAQmx_Val_Rising,DAQmx_Val_ContSamps,1000));

to basically instruct the DAQ card to continuously collect 10000 points every second using a buffer large enough for 1000 samples. I would like to understand what happens when the hardware acquires the 1001st point. Does it add it to the head of the buffer and pop the tail out? Or does the buffer get flushed every 1000 samples and the next 1000 fill it up?

Thank you 

[GerdW]

Hi vygr0,

---

@vygr0 wrote:

I have the below line from the NI examples:

DAQmxErrChk (DAQmxCfgSampClkTiming(taskHandle,"",10000.0,DAQmx_Val_Rising,DAQmx_Val_ContSamps,1000));

---

Which LabVIEW example are you talking about?

Anyway: how often (aka: at which intervals) do you read the samples?

The point is: to avoid a buffer overflow error you need to read samples from the buffer faster than DAQmx is placing new samples into the buffer…

[vygr0]

Hello GerdW,

Appreciate your response and apologies for the confusion, I was referring to NI's DAQmx code examples that come with the DAQmx installation. So I'm trying to run this in my C code rather than LabView.

---

@GerdW wrote:

Hi vygr0,

Anyway: how often (aka: at which intervals) do you read the samples?

The point is: to avoid a buffer overflow error you need to read samples from the buffer faster than DAQmx is placing new samples into the buffer…

---

---

---

Currently I have an EveryNSamplesEvent set up which fires every N samples, where N is equal to my buffer size, to call DAQmxReadAnalogF64() and read N samples from the buffer.

So when I call DAQmxReadAnalogF64() to request N samples be read from the buffer, does it transfer the first N samples from the buffer to my read array (in a FIFO manner)? Meaning that I would now have N spaces available in the buffer for the next N samples. Is this how it works?

Thank you

[ZYOng]

See Understanding and Avoiding NI-DAQmx Overwrite and Overflow Errors

I would like to understand what happens when the hardware acquires the 1001st point. Does it add it to the head of the buffer and pop the tail out? Or does the buffer get flushed every 1000 samples and the next 1000 fill it up?

The data got overwritten and an error is thrown. You should make sure that you call DAQmxReadAnalogF64 fast enough or don't put any code that will block the execution of DAQmx Read.

[vygr0]

---

@ZYOng wrote:

See Understanding and Avoiding NI-DAQmx Overwrite and Overflow Errors

I would like to understand what happens when the hardware acquires the 1001st point. Does it add it to the head of the buffer and pop the tail out? Or does the buffer get flushed every 1000 samples and the next 1000 fill it up?

The data got overwritten and an error is thrown. You should make sure that you call DAQmxReadAnalogF64 fast enough or don't put any code that will block the execution of DAQmx Read.

---

Thanks so much for the link! Those diagrams have what I wanted to know. I currently get no overwrite or overflow errors but I was trying to understand how the data was being transferred by DAQmx functions. So DAQmxReadAnalogF64() does the transfer from "PC buffer" to "ADE memory" as I understand it. 

My last question is about how the PC buffer works. Assume that PC buffer size is 5 float64s, and we currently have following data on PC buffer: [tail] 0.2, 0.15, 0.35, 0.40, 0.25 [head]

If read 2 samples with DAQmxReadAnalogF64 from this state of the PC buffer, it would return 0.25, 0.40 (the first two data points from buffer's head - does it work this way?)

And now the PC buffer would have two spaces available for two new data points: [tail] ..., ..., 0.2, 0.15, 0.35 [head] (right?)

Which get filled by the FIFO buffer on DAQ so assuming the new two points acquired are 0.11 and 0.13, PC buffer would now look like: 0.13, 0.11, 0.2, 0.15, 0.35

Is this how these buffers work?

[ZYOng]

I am not sure how the memory are managed internally in the DAQmx driver, but the buffer is working as a FIFO.

[Kevin_Price · Accepted solution]

[More discussion happened between the time I started this reply and the time I finished.  Ask followups as needed.]

Expanding a little bit on some details:

1. When acquiring in Continuous Sampling mode, the DAQmx driver will treat the task buffer as though it were circular.  If the buffer size is 1000, it will want to put the 1001st sample at the same location it previously used for the 1st sample.

2. However, the driver is also smart.  It not only keeps track of where it left off pushing new data into this circular buffer, it also keeps track of where your app left off pulling data out of the buffer with calls to the DAQmx Read function.  If your app is "keeping up", the driver will be free to push new data into the buffer because it'll only be overwriting old values you already pulled out.

    But if your app is NOT keeping up, DAQmx will throw an error rather than overwrite any previously-unread data.  This is harsh, but is also almost always the preferred behavior.  In this way, DAQmx guarantees that your repetitive reads will either deliver a lossless stream of data, or else result in a fatal DAQmx error.  Nothing in between and no ambiguity.

3. (DAQmx provides some settings that allow you to override this behavior if you have one of the less common use cases that call for it.  An example is when trying to capture data leading up to a rare and unpredictable event.  You can tell DAQmx to overwrite the buffer repeatedly without error, knowing that you'll occasionally want to retrieve the recent few seconds leading up to such an event.)

4. DAQmx is good at controlling timing without a lot of CPU waste.  If you sit in a tight loop doing nothing but requesting 1/10 sec worth of data per iteration, that loop will be paced by DAQmx waiting for the next 1/10 sec of data that it needs to return back to you.

    Reading 1/10 sec worth of data at a time is a good rule of thumb and starting point for a wide range of acquisition tasks.

5. There's another sneaky thing DAQmx does in Continuous Sampling mode.  Consider your config call:

DAQmxCfgSampClkTiming(taskHandle,"",10000.0,DAQmx_Val_Rising,DAQmx_Val_ContSamps,1000);

Here you have a requested sample rate of 10000 and a requested buffer size of 1000.  Some devices may not support an exact 10000 Hz sample rate though, so when necessary, DAQmx will instead use the next higher rate that the device *does* support.  DAQmx may also make a bigger (and possibly *much* bigger) buffer than you request.  Here's a KB article that's mostly true.  (There's an off-by-one ambiguity for sample rates of exactly 100 or 10000.  To be honest, I don't know if that's based on requested or actual sample rate.)

    My own practice (and recommendation) is to make my buffers big enough for 5-10 seconds of data for sample rates under 100kHz or so.  Memory is plentiful and cheap these days, no need to skimp.  (FWIW, even a 1 second buffer size will usually be more than enough if you have a tight loop reading 1/10 sec worth at a time).

-Kevin P

[vygr0]

Wow thanks so much for all the info Kevin, exactly what I was looking for. Also thank you for that point 5, I was suspecting of that behavior by what I understood from https://www.ni.com/docs/en-US/bundle/ni-daqmx-c-api-ref/page/daqmxcfunc/daqmxcfgsampclktiming.html

where it explains sampsPerChanToAcquire: The number of samples to acquire or generate for each channel in the task if sampleMode is DAQmx_Val_FiniteSamps. If sampleMode is DAQmx_Val_ContSamps, NI-DAQmx uses this value to determine the buffer size.

Thanks for clarifying!

[mcduff]

One last tip for continuous acquisition if you are also using the built in logging. Make the number of acquired points for each event an even multiple of the disk sector size. This greatly improves efficiency. I have used the same functions as you in LabVIEW instead of C and for high speed acquisition multiple channels 10MSa/s; having an even multiple of the disk sector size made all the difference.

[BertMcMahan]

---

@mcduff wrote:

One last tip for continuous acquisition if you are also using the built in logging. Make the number of acquired points for each event an even multiple of the disk sector size. This greatly improves efficiency. I have used the same functions as you in LabVIEW instead of C and for high speed acquisition multiple channels 10MSa/s; having an even multiple of the disk sector size made all the difference.

---

Here's a link on determining your sector size:

https://www.computerhope.com/issues/ch002288.htm