CognoscentUI is a user interface framework intended to host modules within a SubPanel within an XControl. Instances of modules are able to run concurrently, and the framework allows for easy development of new modules to perform any sort of UI task.
CognoscentUI was developed as a contest entry for the LV2010 Coding Challenge (LVXCC), and it is specifically distributed as a reference source code example, demonstrating the following:
CognoscentUI is composed of three discrete frameworks (plus some eye candy):
This is the heart of the framework. The container is made of three distinct parts: the Titlebar, the SubPanel, and the Status Bar
This plugin displays pictures in a ring the user can spin around using a mouse (or finger if using a touchscreen).
This module plug-in displays a linear list of text and icons. The user can pan the list using a mouse (or finger if using a touchscreen).
LabVIEW 2010 - No additional drivers, add-ons, or toolkits required.
http://bit.ly/CognoscentUI-Demo
It's embarrassing how choppy the animations are in the the video. I would recommend that as you watch the video on one monitor, recreate the demonstration yourself on your other monitor. (Harder said than done if, like me, you only have one monitor ) Use the link below to download the 2.2MB Zip file, then run "CognoscentUI_Example.vi"
From the XControl Façade. When the XControl receives a message (from the Val(Sgnl) on the 'Interprocess Message' class control), the message is interpreted and handled by a case structure. The case shown handles the action for a module requesting to be inserted into the container.
The base messaging class has a method called "Send Message", and this screenshot shows the override implemented in the User Event child class. (At first, I only had one messaging class based on User Events, and it ended up genericized into a messaging base class since XControls won't "wake up" for User Events.)
When the "Activate Module" method is called, this method launches a reentrat clone of the module. Ugh, if only we had better dynamic launching.
Notice this guy has no Error terminals - it's a dynamically launched process that runs whenever a "Mouse Down?" event is detected on objects that can be manipulated with the mouse (the title bar moving the Container, the resize handle on the Container, the Pic Ring rotation...). The Object Kinematics class defines (overrides) how an object will move, and as the mouse moves "SetTarget" is called on the class to continually update the set point of the animation controller. The process continues until the button is no longer held.
One goal for this project was to not require anything more than a clean LabVIEW 2010 installation. The result? A few homebrew image processing routines. Note the interface only allows an input called "Sharpness (0.0 to 1.0)". One theme I tried to carry around this project is abstraction: develop interfaces through the eyes of the end user. The end user only cares about how blurry or sharp the image is: a value of 1.0 means no blurriness whatsoever, and a value of 0.0 means the entire image is smeared to oblivion.
I'm glad you asked!
1. Save a new instance of the CogMod_Template.lvclass
2. Give it a kickin' new name. Create a "Copy" of the Template - don't "Rename" the Template! Add it to your project.
3. At this point, you have the skeleton of a new module that's ready to be instantiated and used! Just drop an Invoke Node "Activate Module" for the container on your calling diagram and insert an instance of your Kickin' New Module Class. To add functionality to the new module, edit the "CogMod_Display.vi" class member. Consider this VI loosely analogous to the Façade member of an XControl - it must be present, and it contains the FP display of the module. Within this small VI, you will find brief instructions on how the VI should react to messages sent to it by the containing CognescentUI XControl. Feel free to add whatever class members you want, but don't change the inheritance of the class (the Template already made your new class inherit from Module base class).
There are compelling arguments against using typedefs. Madness?! You decide. I think I pulled it off with this project, and if someone can show where typedefs would produce more robust code, I'd love to discuss it in the comments.
Based on demand, I have added an additional download for a downconvert to LV2009.
This project is not complete and buggy. Like any framework, if it's going to be legit, it's got to be vetted.
The future of this project is demand-driven.
Kindly disregard all profane comment blocks I forgot to erase from source code prior to distribution.
Released under Creative Commons Attribution Unported 3.0 license - please be aware of the terms prior to further development or redistribution of this framework.
Example code from the Example Code Exchange in the NI Community is licensed with the MIT license.
interesting take on a Multi Document Interface.
I haven't thought of this framework as an MDI, but you're right, good comparison.
Alternatively... you could hide the Active Module Selector on the top-left title bar and have multiple instances of dedicated Containers running one module apiece (SDI).
Thinking aloud - It would also be straightforward to put an array of pictures on the title bar, which would effectively turn the Container into a TDI. Each module then would provide a title bar icon as part of it's contract. (Using the Pic Ring as the Module Selector is impractical, yet makes an attractive demo. I would lean toward TDI given more time.)
(And "document" is used loosely here, meaning module or plugin or extension or app)
I wonder if the blurring would be simpler with a direct 2D convolution (DBL) with a small gaussian kernel and the output size set to size=x. You can create the kernel with a plain 1D Gaussian fed to both inputs of "outer product".
Try it.
altenbach, I first started with Convolution, and it was MUCH (!!!) slower. As in, when I realized it wasn't going to respond within 30 seconds, I went and made some tea, and came back and it was still crunching.
Perhaps I was doing something wrong.
Then, lo and behold, the next day Andrey posted his Deblur contest entry. It was the answer to a much faster Gaussian blur (~50msec for a few hundred pixels square) using Fourier transforms. I spent a few hours wondering why I could not adapt the method, and then found it it only works for 2D arrays where both dimensions are a power of 2
Still, I'm unhappy with how long it takes to render the Pic Ring. I think the answer is a Box Blur, since it does not even require a trek into the Frequency Domain (but it comes with the price of a lower fidelity blur). Given more time, I definitely would have experimented with the Box Blur.
Looks very interesting. Do you have a version that works in Labview 2009 ?
Using Andreys images, I get a loop rate of about 20ms for B&W and about 120ms for color using convolution (after removing the wait, on a 4+ year old Laptop :D). I wonder why yours took so long....
Here are two very quick and dirty, "back of the envelope" drafts using a 2D cosine (but a 2D gaussian would be equally easy), modify as needed. I am sure many improvements are possible. For example, each color plane could be done on a different CPU core.
Paul and others - if you send me a PM on the LabVIEW Forums (http://bit.ly/LabVIEW-PM-to-JRD) containing your email address, I will send you a zip file saved for LV2009. I don't want to widely distribute older source code versions, but I have no problem sending it to individuals.
Jack, can you give me an example of when to use a framework like this for automation and test? It seems like this is a framework most often used in something like a mobile app. Have you used this for a project/customer?
I've built 1 unusual Multi Document Interface for a client of mine where they needed multiple windows open at once. Other than that, i've tried to steer clear of it in LV
>> It seems like this is a framework most often used in something like a mobile app.
I ought to update the document to better define the three distinct frameworks: one is for animating objects (the Object Kinematics class hierarchy), one is for asynchronous interprocess messaging (the Interprocess Message class hierarchy), and the final framework is a general purpose container meant to host plugins (the Module Manager class, the CognescentUI XControl, and Module class hierarchy). All three frameworks are intertwined to form the "CognescentUI package".
Tuning down the animations to a reasonable level (high Speed, low Overshoot) works great for touchscreen HMI's at test or control stations. I've had a high level of success using suble animations to make user interaction more intuitive. I work in an environment where that "namby pamby" graphics BS isn't required or tolerated, but undeniably the subtle animations are preferred over "things just popping up and disappearing from nowhere". (Note: the benefit-to-cost tips positive when hundreds or thousands of people might need to become acquainted with your interface, not just a few dedicated operators. Also, if you're optimizing for speed - taking orders on a touchscreen at a fast-food restaurant - animations might not be a good choice)
As for stylized controls in industry, one requirement for my largest embedded touchscreen project to date was "Don't make it look like Windows, and don't make it look like LabVIEW".
>> Have you used this for a project/customer?
All the code was written from scratch after LVXCC was announced, so it's unencumbered (hence the CC license). Recreational programming for this contest was a good creative release to test out some concepts and test out some beers.
>> can you give me an example of when to use a framework like this for automation and test?
I stuck loosely to the contest theme of "example code and iPAD" and used that as a springboard for the project. A contest has a dramatically different scope than a contract, so I just ended up settling on an entry that could showcase the widest smattering from my bag-o-tricks. I think an important take-away from this example for the ATE apps is how easily extensibility can be achieved with LVOOP.
>> I've built 1 unusual Multi Document Interface for a client of mine where they needed multiple windows open at once. Other than that, i've tried to steer clear of it in LV
I really hesitate to classify this framework as an MDI. Fulfilling the role of a MDI as a subset of the framework's capabilties? Sure, that sounds better. (Also, if they have multiple windows open at once, that's not an MDI, which only has one window) And I'll say again, the Pic Ring is an annoying method of switching through plugins, I would definitely go TDI with icons on the the title bar.
Alright, altenbach, I haven't been ignoring you, I finally got the results from Convolution vs. Fourier in this unusually messy snippet. On my computer, I see Fourier taking between 65 and 80 msec dependent upon blur level, and Convolution takes between 80-260msec, highly sensitive to the multiple of the kernel size. I'm sure we could optimize both methods even further, but I doubt either method would smoke a box blur. I was disappointed I couldn't get up and make tea while using your convolution method.
Since the results were not very close, I just used the "Run Continuously" button for this benchmark, without taking any of the usual benchmarking precautions.
Yes, its a matter of code simplicity vs a slight speed edge.
However, you are comparing apples with oranges! For my method, don't do the padding (just use the raw picture as upper input!) and skip the trimming at the end and it will be about twice as fast as Andrey`s.
In any case, try the FFTs from the NI Labs High Performance Analysis library 2.0 on Andreys solution, on my dual core, I improve about 20+% with them.
I am also puzzled by your parallel loop configuration. I would probably only parallelize the outer loop. Stacked parallel loops don't make sense to me.
EDIT:
OK, the final verdict: convolution is up to 3x faster!
Have a look at the attached modification, comparing different equivalent blur levels.
(not sure why the forum does not rescale the snipped....)
The plot thickens: I wrote the padding routine to go ahead and upscale it to the greater power of two when the dimension already was a power of two. In other words, the picture we have been starting with - 256x256 - was actually getting resampled to 512x512, which is why Convolution was faster than the Fourier. (The reason I wanted the extra padding is so I could specify the blend color for the edges. Given more time, I was going to add a "Background Color" input to this blur routine)
Now, if we shave off a pixel, so that Convolution is running on a 255x255 and Fourier is padded up a pixel to 256x256 (fair enough) Fourier again beats Convolution by about 30%.
One thing I have learned for sure: the speed of both methods is highly sensitive to input array size.
Yes, you don't need to pad. The FFT has been optimized for virtually all sizes quite a while ago (6.0 maybe?), so, YES, without padding Andrey's algorithm is again faster for that particular size.
Quote from Wikipedia:
"The most well known FFT algorithms depend upon the factorization of N, but (contrary to popular misconception) there are FFTs with O(N log N)complexity for all N, even for prime N."
Very long ago, I extensively studied it for 1D fft and made a nice graph of speed vs size for all sizes. Typically, it's probably not worth the extra rescaling. You could make a lookup table of all speed vs. sizes and then pad to the next higher fastest size, also accounting for the rescale penalty.
Both methods basically do the same things. The convolution method probably costs slightly more because it also involves an FFT of the broadening function. I still like it better for code clarity.
As mentioned above, don't forget to substitute the high performance library from NI Labs for Andrey's code.
>> Very long ago, I extensively studied...
My last comment fully documents the extents of my studies (), and at this point I defer to you. Thanks for the helpful criticisms - you've given me enough good ideas to probably decrease the rendering time of the Pic Ring by 50%.
>> I still like it better for code clarity.
Totally agree. The code is clearer, and the sliding blur kernel concept is more intuitive.
>> don't forget to substitute the high performance library from NI Labs for Andrey's code.
Based on future demand, will do!
Also, Andrey's code works especially well for the "special case" of a Gaussian, because the FFT of a Gaussian is again a Gaussian. The convolution approach lets you more easily play with other broadening functions (try a motion blur ;)).
Anyway, congratulations for being a finalist. Well deserved!
I haven't studied your code yet (probably will vote on the last day) after looking at everything.
Since you are dealing with relatively small images and a limited set of blur levels, it's probably worth to generate and cache e.g. 5 blurs for each image. In this case you only need one forward transform and N back transforms, saving you again almost half (over N forward and N back transforms).
(Here we get the real advantage over the convolution where forward and back FFT are merged and we cannot tap into the "middle").
Nice background image of the Melbourne Citylink freeway: http://maps.google.com/maps?f=q&source=s_q&hl=en&geocode=&q=Melbourne,+Victoria,+Australia&sll=37.06...
This is an amazing tool you've created.
The runtime resize-ability/moveability feature was something I was going to look into creating, and you've done a superb job of accomplishing this.
Any particular bugs I should be aware of?
Awesome example
~Jon
>> This is an amazing tool you've created.
Thanks!
>> Any particular bugs I should be aware of?
Here's what I know about:
Alright, full disclosure, that's all the dirty laundry I got on myself. Feel free to comment when you find more bugs
I've built the example into an executable, but I've lost some of the xcontrol's features. I can no longer move/resize the xctntrl by clicking on it. I can still use the controls at the bottom of the screen however. Any thoughts on why this is happening?
~Jon
Something was not being picked up by the compiler. Always included for whole project fixed this. I bet you are loading something off the path somewhere.
JMkokott2 wrote:
Something was not being picked up by the compiler. Always included for whole project fixed this. I bet you are loading something off the path somewhere.
Can you report back which VI was the culprit (I have not investigated)? I thought there were static references to each of the important modules, but it's clear I need to add more. +1 for the buglist.
I had a way-homer about the Messaging class hierarchy. It makes more sense to differentiate "Message" from "Messenger". It never felt right having the child classes ("Messengers") inherit from Message base class, with the most noticeable "code smell" coming from the "Send" child implementations (see the screenshot above, and also the class hierarchy. It's strange how the Messenger Ref is unbundled from the same data that is being sent).
While maybe not a bug, this should be considered a design flaw that needs reconsideration.