LVAD package: Implementation of Forward Mode Automatic Differentiation in LabVIEW using Operator Overloading Abhishek Kumar Gupta Department of Electrical Engineering Indian Institute of Technology Kanpur

Amrit Agrahari Department of Mechanical Engineering Indian Institute of Technology Kanpur

Category: Student Abstract The paper presents the implementation of forward mode Automatic Differentiation in the LabVIEW Automatic Differentiation (LVAD) package using object oriented approach of LabVIEW 8.5. Automatic Differentiation (AD) is the technique allowing one to calculate the exact numerical derivatives of a function defined by the computer programs. The implementation uses LVAD class to store variables. Each class variable contains two sub variables - (its value and derivatives). Most of the arithmetic operations and functions are overloaded for this class. This technique works over the concept that ‘As a function or operation is applied over some variables, each variable’s derivative is also got propagated according the chain rule. By the automatic propagation of derivatives, we can get the output variable’s derivative. This implementation is further extended to get the Jacobean J of function of the form : (ie the function with matrixes or arrays as input). The package also includes its practical applications particularly VIs used for function Optimization.

1. Introduction Exact and fast Calculation of Derivative of a function is desirable in many applications particularly controls and optimization problems. In such problems, error in the value of derivative affects the final error significantly. Many algorithms are developed to get the differential like difference method, symbolic differentiation. Difference method is very popular method but it does not provide exact differentials. Symbolic differentiation has limitation and it is not fast. Also it involves hand coding for each functions. It is not able to handle control flows in many cases efficiently. Automatic Differentiation is a tool which provides the numerical value of differential up to the precision limit of underlying computer platform/software. It is based on the fact that every function can be broken in to elementary operations whose differentials are known to us and simple to get. These differentials values can be combined according to chain rule to get the function’s differential.[1] Automatic differentiation (AD) can be implemented using two main techniques - forward mode or reverse mode which is according to the way in which chain rule is applied. The LabVIEW Automatic Differentiation (LVAD) package described here is implementation of AD in forward mode using operation overloading approach and object oriented concepts. The LVAD package consists of the LAVD class with its overloaded operations. The complexity analysis of AD has been done by Griewank et al which is described in later section.[2]

2. Mathematic Behind Forward mode AD Forward Mode AD is very easy to understand, It basically involves breaking the complex function to the elementary or simple operations/functions whose differential is known and easy to get and then combing these differentials according to chain rule. Suppose g(x)=x2*sin(x) is a function whose differential is required at x=a. sin

Let us assume sin Then g(x) can be written as Differential of g(x) is

`

` sin

`

2 cos

Expression for g`(x) is unique for multiplication of two functions. Similarly for addition and division of two functions f1(x) and f2(x) we have `

` `

`

` `

To implement AD we introduce a two field variable X as , ′ Now if We define the multiplication for this new variable as , ′ , ′ , ′ , ` ` This is known as operator overloading for classes in the programming language.

3. Implementation of AD in LabVIEW: LVAD Package Since AD involves the use of Overloaded Operations, the basic need of Class structure comes in to picture. To implement we have used the object oriented principles of LabVIEW 8.5. While LabVIEW 8.5 doesn’t provide function overloading in straight way, we tried to overload the operator by defining a LVAD class and defining its own operators’ definition giving them the same name of the original operator. We take each variable to be an array so that the class can be generalized for most of the cases. These subsections describe the package: 3.1 LAVD class: The LVAD control for the LAVD class is shown in the fig 3.1. It contains two class variables: i. Variable value: the first component array stands for the variable. ii. Derivative value: the second component array stands for the derivative of the variable. If there are multiple input variable arrays, the dimension of derivative changes accordingly. For one input variable case, it size matches with the variable size.

Fig 3.1 LVAD obj control The default control has 0 has its value and 1 as its derivative which represent X={x=0,x`=1} The class contains following methods to access its class variables:

Get Set

Returns the value of variable and derivative Sets the value of variable and derivative

Fig 3.2 Get and Set methods for LVAD class 3.2 Oveloaded Operators: Most of the arithmetic operators are defined for LVAD class in such the manner that output variable contains its value and derivative value correctly. Let’s consider a simple operation: multiplication of two LVAD objects X, Y ′

`

`

Fig 3.3 describes the implementation of the above operator definition which is very straight forward.

Fig 3.3 overloaded times method for LVAD class Similarly Z=sin(X) can be overloaded in the following way: sin sin z′ cos Similarly for Z=log(x) log

′

′

log 1 ′

Implementations are shown in figure 2.4a and b.

Fig 2.4 implementation of overloaded a) sin and b) ln functions for LVAD class 3.3 Package Hierarchy Package hierarchy is shown in fig 3.5.

Fig 3.5 Hierarchy of LVAD Package.

4. How to use : Demos and Examples: The use of LVAD package is very simple. We have provided some example demo cases for the user for standard cases. 4.1 One variable Scalar functions Fig 4.1 describes the method to differentiate a function f(x) at x=xo. 4 sin √ where x is scalar.

Fig 4.1a Front panel of demo showing inputs and results for f’(x) F’(x) comes out to be -0.4538 at x=3 for c=a=2;

Fig 4.1b block diagram for the differentiation of function f(x) The following steps describe the procedure in more detail:

(i) Define A LVAD object X and set its value to {xo,1}. (ii) Define three LVAD constants a, c and π and set their value respectively to {a,0},{c,0} and {π,0}. Second argument with zero value signifies that the variable is constant and its derivative is zero. (iii) Wire the variables and apply operations defined for LVAD class according to the function f(x) to get the output Y. at x=xo. (iv) Extract the derivative component of Y which the numerical value of ′ 4.2 Two variable functions Now consider a more general case. Consider a

,

:

,

function. sin

,

,

.

This function’s derivative can be described as Jacobian J

(i) Construct two LVAD object X, Y with , ,

1 0 0 1

(ii) Wire these objects to get f1 and f2 as in the fig 4.2.

Fig 4.2 block diagram and front panel showing results for two-variable example . (iii) Extract the derivative components of the (iv) First row refers to the derivative with respect to x and second row refers to derivative w.r.t. to y. Other examples are also included in the package for users.

5. Complexity Analysis: According to Griewank, the computation time for the forward mode, is equal to : function.

, to calculate the Jacobian of a function ), where t(f(x)) is the run-time for the original

The coefficient r [1 + n, 1 + 1.5n] is dependent on machine characteristics where n is number of input variables. So the complexity of AD is of O(n). [2]

6. Applications This package can be used for variety of application where exact numerical derivative value is needed. Some main applications in which LVAD is helpful are following: (i) for finding the roots for an equation We have built an implementation based on Newton method which can find the root of a function. It uses LVAD package to evaluate the derivatives at each iteration. Suppose if we want to find the roots of sin

0

With initial guess x=0.5, The above implementation is shown in the figure 6.1 and 6.2 for the fixed number of iterations. The results we get is x=.869474 after 10 iterations. (ii) Optimization For Newtonic gradient based optimization, gradients of the objective function are to be evaluated at each iterations. The given implementation can be easily used to evaluate gradients. (iii) Control Systems This implementation can be easily used to implement a derivative controller and many other applications in control systems.

Fig 6.1 Block Diagram for Newton’s method to solve a function for its root.

Fig 6.2 Front panel showing results for Newton method to solve for root of an equation.

7. Conclusion In this way, our implementation of LVAD gives the exact numerical differential value which has been tested for various test cases. The LVAD package can be helpful in many problems of controls and optimization where gradient values are required. The implementation can be further extended to achieve higher order derivatives and support Jacobian calculations.

References [1] Shaun A Forth (2005) An Efficient Overloaded Implementation of Forward Mode Automatic Differentiation in MATLAB, ACM Transactions on Mathematical Software, Volume 32, Issue 2 (June 2006) pp 195-222 [2] Griewank, A. (2000) Evaluating Derivatives: Principles and Techniques of Algorithmic Differentiation. Number 19 in Frontiers in Appl. Math. SIAM, Philadelphia, Penn.