How to write certain mathematical functions in the formula string of the downhill simplex nD i

[blessedk]

I am just looking into using the downhill simplex nD vi and would like to know how to enter certain mathematical functions into the formula string ( functions like integral over a certain interval, derivative ) in this formula string. I am thinking that's why the levenberg M Nonlinear fit vi has an option to add a vi to describe the model. 

Can am anyone please help explain how to enter certain mathematical functions ( as described above) Into the formula string. Thanks

[Bob_Schor]

Wow, BlessedK, you must work for some highly-classified organization, as most of your posts that I've seen ask very detailed questions about very specific topics (such as "How to write certain mathematical functions in the formula string of the downhill simplex nD VI") (you actually wrote "nD i", but I'm guessing the "V" was silent).  There's no context, no code so we can see what you are really trying to do, and no examples of the "certain mathematical functions".

Have you looked at the Downhill Simplex nD VI?  Wow, what horrible LabVIEW code -- it even has a Stacked Sequence structure in it whose only purpose is to record the number of millisecond Ticks that the algorithm takes to converge (at least in the LabVIEW 2016 implementation).

Some students were doing a project involving an analog triaxial accelerometer whose voltage on each of the three orthogonal axes was a linear function of the acceleration, with a 1.5v ± 0.3v offset and a gain of 0.3v per "g", ± 10%.  Well, with such "slop" in the scaling offset and bias for the three axes, you could hardly expect to get accurate data on acceleration unless you calibrated the accelerometer.  But how to do that?  Inexpensively?  With no equipment for positioning it in six precisely-known orientations (and letting earth's 1g field be the stimulus)?

My solution was simple -- hold the accelerometer still for 0.1 second (easy to do) in six different orientations ("face up", "face down", "right edge up", "left edge up", "front edge up", and "front edge down", where the positions are by no means "exact", just "sufficiently different from each other").  You'll get 6 sets of readings from the 3 axes, and the resultant acceleration has to be 1g.  This becomes 6 non-linear equations in 6 unknowns (the three offsets and three gain settings of the 3 accelerometer channels).  I coded up my own implementation of the Simplex Algorithm, and got a calibration that was demonstrably correct (using it for the accelerometer, I got a 1g resultant for any orientation as long as the accelerometer was stationary).

My "solution" to handling representing the function was that I had a VI that calculated the "predicted response" given the "parameters" and the "input values".  It is much easier to have a VI that you write do the functional computations than having to input a function as a string, depend on an interpreter to parse the string and produce code to implement the function (do you trust the parser?  particularly when you are using "certain mathematical functions"?).

I think you will get more help, and better help (in the sense of being directed to the real problem you are trying to solve) if you provide clearer, more complete information on exactly what you are trying to do, and also provide code to show us what you have tried, and where the difficulties arise.

For my part, I can say that (given the vagueness of your question and my casual examination of the LabVIEW method in your question) I would stay away from the Downhill Simplex nD VI, unless you were prepared to do some extensive simulation and testing to be sure that your implementation produced reasonable results.

Bob Schor

[blessedk]

Hi Bob,

i actually specified the type of mathematical functions I was talking about ( integrals and derivatives). In general the objective is to determine the best values of n parameters in a given function F(x,a1,a2,a3,a4) such that this given function best matches another discrete function. As we speak there is no code and nothing being done. I was simply trying to understand how to use the vi. You said it is best to use the Vi to specify the function. Yes I agree and I mentioned that in my post too. So I guess it's obvious how to use a vi instead of the formula string for the levenebrg  M but not Downhill Simplex. My question is strictly on how to specify certain mathematical functions in a formula string and if a vi can be used instead for the downhill simplex, then great! Please illustrate how to use a Vi to specify the model ( any arbitrary model if it is F(x,a) = ax^2). My question is not how to use use the entirety of the downhill simplex vi but simply How to specify the formula involving integrals and derivatives either using the string or a vi. Thanks.

[blessedk]

Your solution for the 3-axis accelerometer calibration was great. Quite familiar, only longer than 0.1 Sec

[Bob_Schor]

Hi, BlessedK.  Generally, when you are fitting a function, you need to know the function, say (as per your post), F(x, a) = a * x^2.  Some fitting functions, such as Levenberg-Marquardt, use a Gradient Descent method, so they request that you also provide the derivative (with respect to the fitting parameter) of the function, e.g. dF(x, a)/da = x^2.  The Simplex (Nelder-Mead) algorithm, however, uses a Simplex, and doesn't require or use derivatives.

Oops.  I just re-read your response where you say "In general the objective is to determine the best values of n parameters in a given function F(x,a1,a2,a3,a4) such that this given function best matches another discrete function".  This is a completely different type of problem than I was thinking about -- what I believe both Levenberg-Marquardt and the Simplex method are designed to accomplish is a least-squares fit of a model to experimental data, not a somewhat-different question of "approximation of a mathematical function over a specified range using other functions" (such as polynomials).  Problems of the "fit a model to experimental data" kind are often called "curve-fitting problems", while coming up with simpler functions to approximate a (more complex) function I think of as "approximation theory".  For example, LabVIEW (probably) uses polynomial approximations to generate the functions sin(x), exp(x), ln(x), etc. -- textbooks have been devoted to this topic, many (most?) of which I have not read ...

Bob Schor

[blessedk]

Not sure if I was clear enough in the objective but I think we may be talking about the same thing. So let's say my model is F(x,a1,a2) = a1x^3+a2x^2. And I also have experimental results for the function F. So what i want is the values of the unknowns a1 and a2 such that my model is as close as possible to the discrete experimental values of F. Still looking forward to an example of using a vi to specify the model for the downhill simplex vi

[blessedk]

If it works differently than I thought ( looks like it seeks to minimize the function)

Then maybe I need to change my model formula from a1x^3+ a2x^2 to the Absolute value of the difference between F modelled and F experimental (ie ABS(Fmodelled - F experimental) or maybe via the regular least square: Summation { (Fmodelled(xi, a1,a2) - Fexperimental(at xi))^2 } from xi = 0 to xi= N-1, where N is experimental data array size, so that by minimizing the difference, I still achieve my intended goal?

[Bob_Schor]

Well, most of the fitting and modelling techniques use something called "The Method of Least Squares", which minimizes the square of the difference between Observed and Theoretical instead of the Absolute Different (ever heard of Gauss?  I don't think he "invented" this, but he sure wrote a lot about it).  The virtue of Least Squares is that the Square function is mathematically "nicer" (it is everywhere continuously differentiable, where as Abs() is not, having a non-linearity at 0).  However, you are certainly allowed to develop new mathematical theories and techniques, but it might be a better use of your time and abilities to learn more about the current State of the Art.

Do you have access to either a Mathematics or Statistics Department, or to mathematicians or statisticians?  Talk to them and get their input.

Bob Schor

[blessedk]

Hahaha. Thanks Bob, I have edited my post just before I saw this as I thought you may talk about that. See the edited one. 

I still want to see an example of using a vi to specify the model/formula input to a downhill simplex vi.

Please anyone who can or has any alternative with that vi should please help. Thanks

[altenbach]

---

@blessedk wrote:

In general the objective is to determine the best values of n parameters in a given function F(x,a1,a2,a3,a4) such that this given function best matches another discrete function.

---

Yes, if the "discrete function" is experimental data, we are just talking about data fitting. How else are you defining a "discrete function"? LabVIEW does not do symbolic math.

Yes, I agree that the current formula parsing and Nelder-Mead implementation probably has not been touched in over 20 years and most of the code is quite ugly by modern standards. Levenberg-Marqaurdt was in a similar state prior to a major overhaul in LabVIEW 8.0.

Long ago I have written a "drop-in" replacement for nonlinear curve fit that uses Nelder-Mead downhill simplex, where the VI model is defined as the same strictly typed subVI as used by nonlinear curve fit. Clearly this is something that NI needs to offer. (My code is from 2006 and has some special mods it it that requires certain fields in the data variant, e.g. to define the initial size of the simplex. ;)). The NI version does a simple +1 to define the initial simplex which might not be a great starting point if the parameters are all 1E9, 1E-9, or a mix of both), for example.

(Note that all these fitting VIs can be used for multiple dimensions in the independent variables. (see my NI Week presentation and other discussions in my group).

I have found Nelder-Mead to be significantly inferior to Levenberg-Marquardt in terms of speed of convergence. It also does not parallelize well. (Given enough CPU cores, you can calculate all partial derivitaves in parallel when using LM!). The only thing it has going for it is the absence of partial derivatives.

As an example, my EPR spectral fitting program has a nice selection of fitting agorithms. They are all "pin compatible" with nonlinear curve fit.. One of these days I might release the code ...)

You are still not very specific with your model. Is it something that would be easy to implement as a VI?