Manage aquisitions/ data in large scale project

[AhmedJ]

Hi all ,

Well im managing the development of a medium/large scale labview application witch collect data from DAQs an control machines via DAQs and VISA ...  All aquisitions are a part of a "session" that the user create by entering informations about aquisition like "infos ,and stuff like this ... " and also the user can select hardware config (channels , comm protocols with the machine .. ) as a part of an "Advanced config" ... then the user can make his tests/aquisitions and save his data ... also the infos can be entered by user from lists (drop downs .. ) (default configs data ) and add/rem/edit items to these lists for further selections ... its mostly an aquisition software but with a lot oof infos that are entered by user ... i wanna make it the most generic/flexble that it can  (so further version of software can support previous session data saved and so on .. and ofcourse for making a more confidents code ... )

Actually we are using clusters to define the data to be saved , but i m not really confident with this cuz i don think it s generic .. is there any advices that u can give me .. so i can have a clearer view ...as i said it s basicy an aquisition software with infos entered by user (it can be manually or by predefined lists that he can choose items from them or edit items )  saving waveforms  with some data processin results , and with the possibility to configure hardware for aqu ...

It sounds pretty basic as a problem , but when it become a lot of infos and configs to it with the flexibility kept in mind it s becoming more n more confusing ...  i dont know it LVOO (classes) can make this clearer .. so please i m waiting for suggestions 🙂

Thanks ! 

[josborne]

I am running into the same issues now.  I am acquiring lots of data, and need to customize my recording software to make the mounds of data manageable and usable.

If your waveforms are large, (e.g. lots of datapoints), I'd store all my data in TDMS files.  See here for some good tips:

     http://www.ni.com/labview/whatis/data-reporting/

     http://www.ni.com/data_management/

This is a great discussion of measurement storage options:

     http://www.ni.com/white-paper/3727/en

The TDMS file format was designed to address the challenges you described.  But if your waveforms are short, then I'd probably just dump everything into a simple CSV/TXT file with a simple header at the top.  Simply because the files are universal and human-readable. 

The biggest downside to the TDMS file "standard" is that other people can't open them without first installing LabVIEW or at least a data-plugin.  On this page, NI claims the files are "exchangeable" but they are only exchangeable if the user installs some code on their PC first.