DAQ - Measurement - Extraction of DataSet - PCA Analysis

[inst_soleil]

Dear All,

I hope you are all doing well. I am programming in LabView for the following:

There is a device that conducts resistance measurements with an integrated sensor. Sensor resistance changes based on the input sources, which are several. Therefore, I need to do a principle component analysis (PCA) to identify these different sources.

To do so,

A. I need to extract some part of the data from the CSV file; basically, this extraction should start at the beginning of the change in the resistance and continue until there is no change in the resistance value. the change looks like  this on the image.

• I need to open the file.
• Find where the change starts. What tool should I use, the standard deviation?
• Subtract that data and send it to PCA sub vi.

B. Do a PCA analysis. As for the PCA, I think I can use the one that LabView provides. I attached it to this message.

•  Integrate the PCA.vi into the program (as a sub.vi, right?
•  Read the extracted data and perform PCA using a proper model.

I want to do everything in the measurement program. I attached the part of the program that I am interested in. Of course, I am open to your recommendations if you have any.

I used to program in Labview, but I forgot almost everything since I had not used it for a long time. Therefore, please accept my apology for the stupid questions.

I thank you in advance for your messages, time, and effort.

Cheers

The part of the program. I suppose I should start from here.

[LVNinja]

If the process is not slow, I would recomend you create a "circular buffer in memory, as you acquire the data send it to a circular buffer so you have the most recent data on memory, instead of reading from the CSV file. The buffer will orverwrite oldest data with the newest data. 

The Buffer is a whileloop with a shift register, the first time you run (attach the i from the while loop) it innitializes the buffer and every new data will be filling the buffer, once the buffer is full, you can then start using the data for you PCA analysis. 

[inst_soleil]

Hi LVNinja,

I agree. You've suggested a way that will allow me to run the PCA while gathering the data for the next measurement, a dynamic way. makes sense more.

I appreciate the code. I'll make two remarks. Buffer Size was used to specify the buffer. In such scenario, I must specify its size, and I like to do it by utilizing the standard deviation. I can utilize the PCA when the standard deviation value approaches its former value, meaning that I start acquire the data and send it to a circular buffer. Next, when the standard deviation value approached to the its former value, or stabilize, I can perform the PCA. I believe that by doing this, I would be able to avoid worrying about how much information I would have to read. Additionally, it might read the raw value from the second experiment. What do you Think?

I am not sure how can I integrate the Standard deviation into my system.

[LVNinja]

That's sounds interesting. 

The standard deviation will be used from the data acquired in the buffer ? In that case you need to wait until the buffer fills then calculate STD Dev. 

[inst_soleil]

I don´t think so.  I need first to select  which data to process suing PCA, and than  index the data, or buffer.

Steps are like this:

1- select the data

2- index selected data

3- pca

4- move to the second data

5- ...

[Henrik_Volkers]

---

@inst_soleil wrote:

Dear All,

I hope you are all doing well. I am programming in LabView for the following:

There is a device that conducts resistance measurements with an integrated sensor. Sensor resistance changes based on the input sources, which are several. Therefore, I need to do a principle component analysis (PCA) to identify these different sources.

To do so,

A. I need to extract some part of the data from the CSV file; basically, this extraction should start at the beginning of the change in the resistance and continue until there is no change in the resistance value. the change looks like  this on the image.

• I need to open the file.
• Find where the change starts. What tool should I use, the standard deviation?
• Subtract that data and send it to PCA sub vi.

I would try a derivation (pts by pts) . all gray blocks are positive , followed by a detect zero crossing to cut out the blocks.

[inst_soleil]

Can you give a bit more details Henrik?

thanks

[Henrik_Volkers]

Do have the data for this graph?

faster to draw the code 😉

and simulation doesn't catch noise 😉

[inst_soleil]

Hi Henrik,

Sure, please find the example data attached to this message.

Thanks a lot

Beset

[Henrik_Volkers]

here is something to play 🙂

the data is noisy .. so derivation need heavy filtering

If you add all channels noise is better 😉

I used a Savitzky-Golay filter ..followed by a peak and valley search , sorted them and cut out the positive step

run the vi and play with the parameter. read help to get details of the vis used.