DATA AHEAD toolkit support Discussions

cancel
Showing results for 
Search instead for 
Did you mean: 

Two more things I would like to see

It is very common here to have long term tests that run for months at a time and for people to open a data file for a quick check while a test is running.

I have noticed that XLR8 does not lock the files it has open, then the LabView program will crash next time XLR8 tries to write to the file because it is open in Excel.

XLR8 should lock the file as long as the file reference is open, so the file can only be opened by Excel in read-only mode until the file is closed.

------

Also for longterm testing it would be better if XLR8 actually wrote to the file on disk every time it wrote to a spreadsheet.

Our data files can be very large and contain 24 hours of 60 channels polled every minute just from one data logger.

From my testing of XLR8 it would appear that XLR8 is holding all this data in memory until you either use Save Workbook.vi or close the file.

In my tests large workbooks can take over a minute to be written to disk when you close the file.

That could lead to a memory leak and out of memory errors

========================
=== Engineer Ambiguously ===
========================
0 Kudos
Message 1 of 5
(5,302 Views)

Locking the file:

We are going to introduce a file sharing method in v 2.1, where you are able to open the file "read only" in Excel while it is reserved by LabVIEW. I am pretty sure this will suit your needs.

Writing to disk:

Unfortunately,  XLSX files cannot be written incrementally: Underlying XML files have to be changed, then compressed. Therefore, there is no advantage in saving with every write cycle, it would really just slow down the process.

The XLS format has less overhead as opposed to the XLSX format and will be written and read quicker. Consider trying the old format for large workbooks. Of course, the old format has its own limitations such as max 256 columns and 65,536 rows.

In general, I would not recommend the Microsoft Excel format for very large datasets. Loading them in Excel will be slow, too. Maybe splitting up the files could be an option in specific use cases.

Peter

CLD | Alliance Partner | Web data storage and visualization | daq.io
0 Kudos
Message 2 of 5
(4,101 Views)

Yes  I know Excel is probably not the right program when dealing with large data sets. But it is the defacto standard since everybody has it installed and knows how to use it. We occasionally deal with some stupidly large Excel files in the 100+ MB range and newer versions of Excel seem to handle them just fine and modern computers load them fast enough. I have used the LabView TDMS files but we are not doing any high speed data acquisition and the files need to be converted to Excel for analysis anyway adding another step in the process and requiring the Excel conversion plugin to be installed.

Everybody wants a test to output a file they can just double click on and open to view and analyze.

========================
=== Engineer Ambiguously ===
========================
0 Kudos
Message 3 of 5
(4,101 Views)

It's been a while but I was wondering if the "file lock" function has been added yet.

I am currently working on a purchace request and this would be a disadvantage if it is not implemented yet.

========================
=== Engineer Ambiguously ===
========================
0 Kudos
Message 4 of 5
(4,101 Views)

Yes, we implemented the "file lock" functionality as mentioned.

In the upcoming release, we will also have an  automated file backup that makes sures files are only replaced when they can be written successfully. For instance, when you edit an existing file and something goes wrong during editing, XLR8 will have the backup copy of the file instead of a corrupted 0-byte file.

Glad to hear you are pursuing to purchase XLR8 - Thanks!

Peter

CLD | Alliance Partner | Web data storage and visualization | daq.io
0 Kudos
Message 5 of 5
(4,101 Views)