Aerospace Indicators - Artificial Horizon / Primary Flight Display and more using 2D picture controls - thoughts?

[ChrisLudwig]

Greetings fellow LabVIEW people!

I spent some time last week getting to know the 2D picture and picture plot VIs to make a better artificial horizon for flight test UIs. My main goal, besides making it look nice, was to make a resizable control that also looked as nice with as much functionality as we would expect on transport category aircraft. What I produced is I think as good as the best of what you will find on most airliners, at least as good as any aircraft not implementing synthetic vision or terrain depictions on their PFD. The trickiest bit was being able to rotate text in a picture control. I ended up converting coordinates from a single line font to rotate and translate each number and then used the draw multi-line vi to put the numbers on the picture. It isn't perfect as x,y coordinates round into the nearest pixel, but it is fully readable.

Here is the best of what I've created so far:

My inspirations was this, more or less, with some simplifications for ease of developing:

I have one more element to add, that being a roll angle indicator like the one at the top of the this PFD:

I think though that I will add that feature not as an overlaid meter indicator, but rather drawn in the picture control as well.

Having gained some serious appreciation for the 2D picture indicator, I am thinking I would like to tackle making some serious aerospace-grade vertical tapes and maybe eventually some dial indicators as well. I have made these vertical tapes using LabVIEW native indicators, but I am less than perfectly happy with them:

My biggest issue is that these are not easily resizable for the developer, and are not at all resizable at runtime. I also don't like that there isn't an easy way to indicate to the user where a tape will transition from normal (green band on an aircraft, white band for engineers that might be color blind), to caution (yellow), and warning (red).

I am thinking I could build vertical tapes using 2D picture control that would be resizable at runtime and also would allow me to make them look exactly as desired. I would be aiming for vertical tapes which are more in line with this, minus the yellow line joining tapes:

Those tapes (from a Sikorsky S-92A) add horizontal red and yellow lines to show transition points from normal to precautionary and warning levels. I could implement a single vertical tape and then allow the developer to create an array of tapes, or I could also make some simple VIs to build multiple tape pictures into a single image for display.

Here is my main question. Does anyone have a good understanding of how well these 2D picture controls would scale? Would redrawing a bunch of these (20+) at say 10-20Hz be a significant processing load compared to using native indicators? What about on PXI controllers that do not have discrete graphics cards? We'd be using stuff like the PXI-8881, but we have a significant workload processing data and can't really afford to make inefficient UIs to boot. My artificial horizon doesn't seem to be driving CPU too much, but then my dev computer has a great CPU and graphics card. Thoughts on performance with 2D picture controls compared to native controls? 

-Chris.

PS: My future state dream is to make a framework to build synoptic displays from a config file at runtime, stuff like this where lines change colors based on if it is an active flow path, and valves change symbols for open, transitioning, and closed, etc:

Send a PO if you need something like that! 😁

[ChrisLudwig]

Well, tapes were easily done in picture controls:

Resize is easy:

[billko]

I've found that a large picture box can bring a UI to its knees if it is refreshing fast enough.  This is because LabVIEW does not use accelerated graphics and uses the GDI to draw its stuff.  However, the alternative, the .NET picturebox uses DirectDraw, offloading the picture drawing to the GPU(s) drastically increasing UI response.  However, the .NET picturebox is a pain to implement and just the act of turning the LabVIEW video string to a .NET picturebox can limit your throughput.  Also, since the .NET picturebox is technically not part of LabVIEW, it is a pain to integrate with a LV front panel.

[ChrisLudwig]

Completely understood about the lack of video accel on native picture controls. I would recommend a ~5 Hz update rate for mechanical gauges and tapes, probably 10 Hz for an artificial horizon. I'll do worst case load test and see how the CPU handles it.

[ChrisLudwig]

I added a roll readout to the artificial horizon...

[billko]

By the way, these are awesome.

[ChrisLudwig]

I also made a reconfigurable dial gauge for an engine display:

Getting close to a useful set of aerospace indicators!

[Darren]

Chris, this is one of the coolest applications of the picture control I've seen in a long time. Well done.

[Darren]

Oops, I was so impressed by the screenshots I missed your question on first read...

---

@ChrisLudwig wrote:

Here is my main question. Does anyone have a good understanding of how well these 2D picture controls would scale?

---

I've worked on lots of picture control-based UIs in the past, but almost every single one was a single picture or an array of pictures (like a grid-based UI). For a UI that will have a bunch of separate picture controls I don't have any specific guidance. My general guidance is always "only draw what you have to". For most of my UIs, I have 'Erase First' turned off so that I only need to draw over the part of the image that changes. For your images, however, it looks like that would be of minimal use, as these images seem like they're going to be redrawing most of the content. Based on your block diagram it looks like you're not maintaining the picture value in a shift register, which is good... in general you don't want to add drawing instructions on top of drawing instructions... that will slow things down for sure.

Going back to the grid idea... is that a possibility? Because I found that writing to a single (1D or 2D) picture array terminal is more performant than writing to multiple (scalar) picture terminals, if image refresh rate is something that you're trying to maximize. If you end up exploring this option, let me know... it's been a while, but I feel like somebody had to hack me a 0-pixel border picture control to put in an array so that there wasn't any separation between the array elements, and I could probably find that picture control if I had to.

Of course, as mentioned previously you'll definitely want to benchmark your current approach on the deployed system to see if it accomplishes the desired loop rate before looking into any of my crazy ideas. 🙂

[NJKirchner]

@ChrisLudwig

Regarding performance of a picture indicator, and how to get the most out of it, I've got a ton of experience on this front. 
Was able to get 45fps on a Pentium 1 100MHz pc back in 2004 displaying a 800x600 image

As well as updating annotations on top if it.

I'd love to help you figure out how to do this and see some of the code implemented.

The tl;dr of the concept, is that the picture control is a glorified byte array and each element is an op-code, length, and then data.

I created code to keep a simple database of elements on the picture control, and then just modify the data of the specific elements.

also because it's a byte array, you can do in-place operations on the array.

That way you're not re-drawing things over and over again.

I developed that approach after creating something that should look pretty familiar.