Actor Framework Discussions

cancel
Showing results for 
Search instead for 
Did you mean: 

Best way to handle config files in AF

Hi all,  I'm wondering how everyone handles config files when developing with AF.  Currently, I have my top level actor pass the config file folder path to each nested actor on launch. The nested actors then open or create their own personal config file in that folder.  The reason I did this was as a way to avoid losing config data due to more than one actor trying to read/write the config file at the same time.  Unfortunately, this results in dozens of individual config files floating around in my app directory, and it's just sloppy.  How does the community avoid this problem without blocking? My current idea is to have the controller make a copy of the config file for each nested actor and that is merged with the real config file on shut down.

CLAD
0 Kudos
Message 1 of 27
(12,003 Views)

I create a single config file and have each piece of my system implement reading it's configuraiton. Each piece knows its own file section. Reading the section keys can be a non-reentrant method so that only one thing has access to the file at a time.

Casey

Casey Lamers


Phoenix, LLC


casey.lamers@phoenixwi.com


CLA, LabVIEW Champion


Check Out the Software Engineering Processes, Architecture, and Design track at NIWeek. 2018 I guarantee you will learn things you can use daily! I will be presenting!

Message 2 of 27
(7,242 Views)

How can you make the section reads non-reentrant?

CLAD
0 Kudos
Message 3 of 27
(7,242 Views)

Hey testing,

I have begun using David Staab's extensible session framework for almost everything that requires sharing a reference across application boundaries (i.e. asynchronous processes as in AF).  The awesomeness of ESF is that it utilizes inversion of control.  This means that the callers are not responsible for destroying a session so you don't run into a potential race condition where one actor quits configuration and closes the reference before others have finished (or even begun!) configuration.  It also gets rid of deciding which actor should be in charge of the configuration file as the Session will have control of opening and closingthe reference.

Here is what my configuration might look like (the Config Data object is a child of the Session - Root object):

config data example.png

You can find out more about the ESF here.

And, regarding Casey's comment - regardless how you approach this, you should not have to have a non-reentrant VI as access to the reference should be blocking (i.e. the config data VIs rely on queues and the dequeue function in the Read Key timeout is set to -1).

Hope this helps.

Cheers, m

Message 4 of 27
(7,242 Views)

I'm not familiar with ESF.  Are you opening a reference to the config file and leaving it open for the entire session, or are you opening the file each time an actor calls that read refnum method?

CLAD
0 Kudos
Message 5 of 27
(7,242 Views)

Ah yes.  You answered that question in the image.   Got it working in an AF project.  Works perfectly!  Thanks, m!

CLAD
0 Kudos
Message 6 of 27
(7,242 Views)

I'm using AQ's Character Lineator (also presented here in the AF community).

My config file format is *.json, giving a nice hierachical possibility and simple readable text. I especially like the OOP features there - such as dynamically loading child classes directly from the configuration (see the Log part):

{

    "ClassName":"SAM Settings",

    "SAM":{

        "QueryNodesOnBus":false,    //    disabled for debugging

        "QueryNodesTime":5000,

    },

    "Comm":{

        "Instance":{

            "ClassName":"SAM Settings Ethernet Instance",

            "name":"Ethernet",

            "uC port":45175,

            "SAM port":45174

        },

        "gui update":5E-1

    },

    "Node":{

        "poll Nodes":false,    //    disabled for debugging

        "poll period":1E+0,

        "gui update":5E-1

    },

    "Log":{

        "Providers":[

            {

                "ClassName":"Log Provider Display",

                "Level":"debug"

            },

            {

                "ClassName":"Log Provider File Txt",

                "Level":"debug",

                "Path":"./data//"

            }

        ]

    }

}

I use a simple algortihm to strip the comments in the json file (borrowed from: https://github.com/sindresorhus/strip-json-comments)

To overcome the issue with changes, I use the following approach:

  • Each actor (see SAM, Comm, Node, Log above) has its own configuration part, so they also have their own classes.
  • When they child actors update some values, only this part/class is send upstream, where it is replaced/merged in the configuration (until it goes to the root actor).
  • File access is only done in the root actor.

-Benjamin

_________________________
CLA
Message 7 of 27
(7,242 Views)

I’ve also started to use configuration messages rather than sharing a file reference, with the final format being JSON**.  JSON is more useful than INI-format because in is more flexible and can match the actor nesting structure; subActor’s configurations can be contained inside their caller’s configuration, and one can reuse the same actor on different branches of the tree without a name collision in the INI file-section.   For example, my Config JSON contains a JSON Array of a variable number of analysis-step actors where individual actors can appear multiple times with different configurations.  I also note that such a by-value message-based solution can easily work distributed over the network, while a reference-based one cannot. 

— James 

**Note: I use a different actor-like framework and use clusters for config info rather than Objects.

0 Kudos
Message 8 of 27
(7,242 Views)

OK - so I understand using json for network communication, but I have to ask - why would you use json for configuration?  As with Benjamin, I went down the road of using the AQ Lineator for serialization/deserialization of classes and using the resulting product for configuration.  I found that I had serval issues with this attempt:

  1. I was having to store more information in my class than I would have normally carried around so that parameters were available for serialization.  These paraemters might be associated with running hardware tasks, timeouts etc.; properties that were not relevant to the operational aspects of the object that I might be configuring.
  2. Use of the AQ Lineator drags in a lot of objects that may or may not be used into your project.  For larger projects, this just adds to the drag associated with loading and building.
  3. I was unable to get the AQ Lineator to function properly on embedded (i.e. RT) systems.  The issue was associated with what Benjamin pointed out as one of the benefits - dynamically loading child classes.  I posted on it in that discussion but I do not believe this was ever addressed.
  4. I have had a hellava time getting anything to build with the AQ Lineator.  At best, it has been incosistent - sometimes things build ok and other times I seem to have something in the Lineator that causes my builds to barf.

I have since ditched the attempt to use the AQ Lineator for configuration.  To me it makes no sense - it's a LOT of overhead for little payoff.  You want hiearchical?  The INI format provides this.  You want human readable and interpretable?  The INI format provides this.  In fact, I would suggest that the INI format is more easily readable and interpretable than the json format (the INI format has few requirements for representation and when you write one of these by hand, you will not have to be concerned with properly bracketing an object as there is no bracketing).

All this being said, I think the json format (and xml for that matter) is wonderful when you are exchanging data between programming languages (such as when you respond to an http request and need to decode the data on the client side) and I have used the AQ Lineator for this simply because I didn't want to invest my time writing class specific json code.  But, again, I ask the same question as above - why use something that was designed for a specific problem that is not necessarily relevant to configuration when it seems a simpler solution is available?

0 Kudos
Message 9 of 27
(7,242 Views)

> why use something that was designed for a specific problem that
> is not necessarily relevant to configuration when it seems a simpler solution is available?

I don't mind that you ditched the AQCL. It isn't a silver bullet. It isn't the right solution for everyone. But I think I can answer your question. You see, configuration is EXACTLY the use case that AQCL was designed for. Transmission between machines in space or from one use of an application to another across time is the same data transmission problem (the latter being configuration). Using INI files doesn't solve the problem for a lot of use cases.

I've been thinking of adding the INI format to the AQCL (or you could if you want to use the AQCL -- that's why it is extensible for whatever format you want to use). The AQCL is not about a particular file format. It is about all the problems that ANY file format can give you. Just because you are using INI does not save you from the data mutation, dynamic loading, partial serialization and general mayhem of data preservation. The AQCL is designed to help with object serialization. The file format is almost irrelevant (thus the junky XML implementation that I threw together just to use as a test case). In other words, if you have any of the other data mutation problems, you're going to build something like the AQCL yourself around your INI file instead of letting the AQCL manage your INI file.

> you will not have to be concerned with properly bracketing an object as there is no bracketing

I assume you either have no nested objects in your configuration OR you've created a minilanguage for all of your nested objects in your configuration. If not, you will have to create bracketing there.

Once you start getting into "This chasis, channel 1, and this chassis, channel 2, and that chassis, channel 1, and that chassis, channel 2" and the number of chasses and channels is dynamic, you'll find that the simple INI file breaks down quickly. The AQCL would be able to manage the dynamic sections and do a read of an entire chassis even though its subobjects (channels) have their own data to write out. Again, that may not be your situation, but if that situation is on your horizon, tread carefully.

Regarding your specific difficulties..

  1. This particular difficulty can strike any serialization effort, whether you use the AQCL or not.

    When you have such a difficulty, it suggests you don't have your classes divided up properly or you specified the wrong info in your To String/From String overrides. You don't have to write down the fields of your object exactly as they are in the object. Write down those fields that are needed to reconstruct the object later. Frequently, you should strongly consider separating your configuration object and your execution object into two separate classes, where the execution object either contains or is initialized from the configuration object (on serialization, the execution object may be used to initialize a configuration object if the configuration can be changed through execution -- often the execution cannot change the configuration so this isn't an issue).

    In other words, in your case, you probably shouldn't serialize your execution class. Use a separate class that represents configuration only.

  2. This is noted in the description of the AQCL. It solves the worst of the worst class of serialization problems. Many serialization use cases are far simpler than the worst case. However, the latest version will not load the vast majority of classes unless you actually need them IF you don't use the Poly VIs in your To String and From String methods. Drop direct calls to the underlying subVIs instead of the Poly VIs. It's something I'm still trying to figure out how to deal with.

  3. I never saw your previous post, but multiple people have done the dynamic loading successfully. You have to download the dynamic classes to the RT target as part of your deployment -- you can't do deployment through the IDE unless you have forcibly loaded the classes into memory in the IDE first.

  4. This comment concerns me. Did you file a bug report with National Instruments? Are you willing to do so and share your code so we can try to replicate your problems? There's absolutely nothing special about the AQCL -- it's just a large block of G code. What kind of "barf" are you seeing?
0 Kudos
Message 10 of 27
(7,242 Views)