Camera Sync Start Not Happening

[GeoffF]

Platform: TestStand 4.2, LabVIEW 8.6.1 (also on LabVIEW 2010).

We have a system performing vision testing.  It is currently deployed at two client sites (Japan and Europe) for testing UUTs and at our Australian office (my site) for development.  It was working OK (apparently) a month or so ago (before my colleague left), but has now ceased working - almost certainly due to something I've done in the meantime.  Unfortunately, there have been many changes and I need some ideas where I should probably be looking.

The systems have two camera sync tasks, one at 15Hz and the other at 200Hz.  If I start these using MAX, images can be obtained from the cameras with no apparent problems.

However, when we run from TestStand, the sync pulses are starting (as shown by a CRO) but no images are coming from the cameras.

During the initialisation sequence, for each task, I am calling the VI referenced in the first attachment as "Initialise Camera Sync".  (This VI has not been changed since February 2010.)  The cases shown are being executed.  Initialise Camera Sync then calls the "Start Camera Sync" VI as shown in the second attachment.  (This VI was last changed in August this year - the comment says "VIs recompiled" and the previous comment from June says "Disabled debugging".) The first sub-VI in that clause is attached as "Get Terminal Name".  (This VI has also not been changed since February 2010.)

At the end of the initialisation sequence (the 76th step of it), we're starting the CAN system, which also kicks off the DAQ tasks - at least, it's meant to.  The CRO shows that the camera sync pulses are starting.  However, our vision system receives nothing from the cameras.  If I pause TestStand and use our other vision utilities, no images appear.

If I start the tasks using MAX, then run the sequence, I get errors during the initialisation (as one would expect), but images do appear.

If anybody is still with me by here, I would appreciate any ideas or tips.  I don't have enough hair left to tear much out...

Thanks,

Geoff

[GeoffF]

Our European office (running LV2010) is experiencing slightly different behaviour to the other two sites (each running LV8.6.1).  In Europe, the sync pulses are not being sent, while the other two sites have sync pulses but no images.

[Jesse_D]

Hey Geoff, 

I can't say right off the bat what might be wrong, but we should be able to narrow down the problem further. 

1)Taking teststand out of the picture may simplify things for you also. It would be a simple step to create a new VI, using your pre-made subVIs, to send the trigger and read the camera image. I would use this to make sure the LabVIEW code is working correctly. 

2)Also, you might consider turning on highlight execution on the VIs you are calling from TestStand. This would wreck your timing but you would be able to see what values are being read in LabVIEW. Your notes mentioned disabling debugging, in order to use high-execution you may need to turn it on by going to File>>VI Properties>>Execution and check "Allow Debugging"

3)If you know you are sure that you are sending your triggers to the camera correctly you might look at your vision VIs instead. I didn't see any info the camera setup you have, how are you reading the cameras? If you are using imaq you may not be initializing the task correctly, something so simple as re-naming the camera could have caused this. 

Can you clarify what you mean by the images aren't being read, and about the vision side of things in general? Are you embedding images in a TestStand report or just storing them to a disk? Are you using IMAQ? 

[GeoffF]

Thanks for the reply Jesse,

1) I'll give the VI route a try.  I have to do a release of the system today (even though the vision side is not likely to work), so it might be next week before I can report on it.

2) I've been using some execution highlighting.  That's how I've been able to track the values that go in and out.

3) I agree that it's quite likely to be the vision VIs.  We're using an IMAQdx system and analysing the resulting images to perform various tests.  Analysis includes pattern matching, OCR, angle determination and/or brightness levels, depending on the specific test being run at the time.  Some of the cameras are Firewire, while others run on dedicated GigE lines.  The system in Japan has two cameras, while the European and Australian systems have 3 cameras each.

We have a separate, LabVIEW-only application for calibrating the cameras.  This is what I've been using to view the images.  This utility also reads and writes the configuration file used for the TestStand application.

When the system works, I can put a breakpoint on the test after the setup, fire up the calibration utility and view the images.  A CRO on the pulse outputs from the DAQ card that is generating them shows pulses.  If I let the tests run, the image-processing based steps get images and either pass or fail depending on the actual images read.

When the system is not working on the LV8.6 systems (Japan and Australia), the sync pulses show up on the CRO, but no images can be retrieved from the cameras.  The calibration utility shows a blank screen.  If I let the tests run, all the image-processing based steps time out and fail.

On the European (LV2010) system, the sync pulses are not being produced at all.  This is possibly a different issue.

On all systems, if the camera sync tasks are started in MAX, the calibration utility will produce images.  On the European system, the image-based tests also pass (or fail) as would be expected with the tasks running (although the sync start produces an error due to the resource being used by a different application - as expected).  On the Australian system, though, starting the tasks in MAX will still not cause the image-based tests to receive an image, even though the cameras can be viewed in our calibration utility.

Even though the camera names are pretty simple (camA, camB, camC in most cases), there is a layer of indirection (to allow for different acquisition rate requirements from the same camera).  I would not be at all surprised if you are correct to point the finger at a naming issue.

Regards,

Geoff

[Jesse_D]

Hi Geoff, 

It sounds like the calibration utility is accessing the camera in "listener mode," which is a mode set on the "IMAQdx Open Camera.vi" A blank screen in the configuration utility could be caused by a flawed configuration file in TestStand

You may want to put together a VI that tests your configuration files, based on the technique here. 

[GeoffF · Accepted solution]

Problem solved:

In Australia and Japan, it was a flawed configuration file. (Our calibration utility had written the strings with spurious quote marks.)

In Europe, the problem was due to the triggering configuration across the busses of their larger PXI backplane.