Self-Organizing Networks

Multifunctional

Neuro-Mechanical

L I TH-I KP -R-1231 Petter Krus*

Matts Karlsson**

*Department of Mechanical Engineering **Department of Biomedical Engineering Linköping University Sweden

ABSTRACT Network systems have been subject to increasing interest during the recent decades. One of the more striking features of networks is that complex properties can emerge although the basic elements are very simple. This is particularly true in neural networks that has become a firmly established discipline for signal processing and control systems. There are also cellular automata, which is another branch where a great deal of interest is devoted. There is a strong element of inspiration from biological systems in these fields as there are examples of networks made up from simple elements with spectacular properties in nature. In this study the scope is extended to networks where the elements can have several properties beyond signal processing and transmission such as mass and energy transmission, and actuation. This would have the potential of creating structures that could vary their shape. No such component exists at present. There is no doubt, however, that these kinds of components can be made very cheaply as new manufacturing techniques, such as layered manufacturing, are introduced. The objective here is to study what they could be like, how they could be configured and what requirements they need to fulfil in order to be useful. INTRODUCTIONS Hyper functional technical products There is a general trend for functional density of products to increase. This is a result of a combination of customer requirements and the fact the prices of microprocessors, sensors and actuators is rapidly decreasing. The cost C of a system is in some way the sum of the cost of its parts. Assuming a system that is required to perform a function with a certain performance that is a function of the scaling variable s. Assuming that the performance P can be described by eq 1. (This could for instance be the total force a system of n actuators can produce)

n

P = ∑ c p si

γp

i =1 n

C = ∑ cc s iγ c i =1

If all s are the same 1

n

P = ∑ c p s = nc p s i =1

γp

then

 P γ p  s=  nc  p  

(3)

In the same way the cost can be calculated γc

γ

γc

1− c  γ p  P γ p  = n γ p cc  P  C = ∑ cc sγ c = ncc sγ c = ncc  c   nc  i =1  p  p n

(4)

Minimising the system cost with respect to the number of components n and their size s yields n=1 if the exponent γ <1. In a more general case a cost optimisation will minimize the number of components if γ <1. This is true in most cases of present technology. If, however, the exponent becomes larger than one, the optimal number of components would tip to infinity. This may also be the case if the exponent is smaller but very close to one and there are other aspects such as redundancy that can tip the balance. This is what can be seen in biological systems, where the only “cost” is material. This means that γ becomes one. In reality the exponent is a function of s and for very a small system it becomes less than one again to prevent the infinity. A consequence of this relation is that future products are likely to behave more "organically" with sensors, actuators and processing power that are integrated in the structure (structronics). This can be used to accomplish massive redundancy. With new manufacturing techniques, such as layered manufacturing, where products are built bottom-up in something like a 3D“printing” process, the cost of the product is not depending on the complexity, but only on the material spent. As the price of sensors, actuators and processors are coming down to the same level as the price of the rest of the structure, and can be made in the same process, the structure can be entirely made from these. The product then becomes hyper-functional. At that stage, not all functionality needs to be defined in advance. The product can be configured to fit each situation. At any point in time, only a fraction of the functionality will be used. An important component in this context is the integrated actuator, several of which can be linked to form a neuro-mechanical network (NMN).

The Generic Integrated Actuator A generic integrated actuator has the following characteristics: • • • • •

Actuation, through mechanical connections Energy supply and removal of energy conversion residues Control and/or sensory input Sensory output Simple signal processing

This means that sensors, energy supply and control are integrated within each actuator. These integrated actuators can then be connected to form a neural network that can be used for several purposes • • •

As an explanation model for biological systems As a generic structure using optimisation for mechatronic system design As a model for future massively redundant mechatronic systems

Control input Energy supply Mechanical connector Integrated sensor input

Mechanical connector Removal of energy conversion products

Control outputs

Figure 1. Generic integrated actuator. This representation is valid for both bio-mechanical actuators e.g. (muscles) and technical systems. An actuator can in general be described as a transformer that transforms some kind of power (electrical, fluid, chemical or mechanical) into mechanical power. A general representation of the equations for an actuator is: F = η F Aeα f 1−α v=

ηv A

e1−α f α

(5) (6)

The heat generated in the actuator is Q = (1 − η Fη v )ef

(7)

Here F is the force, v is the speed of the actuator, e is the effort variable i.e. force, torque, pressure, voltage etc, f the flow variable i.e. speed, angular speed, flow, current etc, and D is a constant. For a hydraulic linear motor (piston), these equations become:

F = η F Ap p v=

ηv

(8)

q

(9)

Q = (1 − η Fη v ) pq

(10)

Ap

where p is pressure and q is the flow. Consequently α =1. For an electrical motor, however, the relations are T = η F ki

ω=

ηv

(11)

u

(12)

Q = (1 − η Fη v )ui

(13)

k

where i is the current and u the voltage, and k is a constant. For this case α =0. Furthermore it is reasonable also to include a capacitive effect in actuators. Power supply system There must be a power supply to each element in the network. When the elements become many the problem of providing the necessary power supply becomes important. Nature provides examples of solutions in the elaborate tree patterns that are used, for instance, in the cardio-vascular system. The control problem When a large number of actuators are used it becomes necessary to study alternative ways of control. For the same reason as for the power supply system, the wiring of the system becomes very complex if every part of the system needs to be controlled individually. Instead it is useful to study means of distributed self-organising control mechanisms. An example of this is the neural network. The central idea here is to integrate physically the neural network with the actuation network in such a way that each GIA also is a connection in a neural network. The result is a neuro-mechanical network or NMN. Neural network with one hidden layer. In a neural network every neuron consist of a summation point and transfer function g(h,t) that normally contains a non-linearity as indicated by the dependency of the input variable h. It can also be dynamic as indicated by the dependency of t. Between the connections

(neurons) there are weights wij that are adjusted to give the appropriate response for the neural network.

Figure 2. Neural network with one hidden layer.

wi1 Σ

h

gi(h)

xj

wi2

wij Figure 3. Neuron

In contrast to the neural network the neuro-mechanical network, as defined here, has no crossing connection lines (actuators) except at junctions. To achieve the connectedness of the neural network, using a two-dimensional neuro-mechanical network with no crossing elements, the number of layers has to be increased so that the number of layers becomes equal to the number of inputs for each layer in the corresponding neural network. A special case is the cellular automata, where the behaviour of one element depends solely on the behaviour of adjacent elements.

Figure 4 Neuro-mechanical network corresponding to the first two layers in the neural network.

Another difference compared to the ordinary neural network is that there is no clearly defined input and output side. Any node can be connected to an input signal. This means that the structure of the connections between nodes has to be symmetric, passing signals in both

directions. The ordinary neural network, however, is still a special case of the neuromechanical network. Furthermore, if all computations are to be made locally, time delays between the nodes have to be introduced in order to avoid algebraic loops. These can be introduced as pure time delays or, more preferably, as low pass filtering of the signals. The scheme for each junction and connection is shown below. The main difference compared to the neural network is that there is also a layer of a mechanical system with an actuator in each connection. Furthermore, the signal paths are bi-directional, allowing signals to be exchanged in all directions, back and forth to adjacent elements. A more detail description of components for a neuro-mechanical network is shown in figure 5.

Connection

Connection Connection Junction

Connection Σ

k1i Gi(x,t)

xj

k1i Σ

Σ

w1j

g1i

w1i

Σ

Actuator

Connection

Connection

Figure 5. A neuron and a connection/actuator in the neuro-mechanical network.

The G filter can then be described as a low pass filter followed by a non-linearity that limits the output signal. A common choice in neural networks is the sigmoid function.

Gi Non-linearity

GLP,i(s)

Figure 6. The transfer function G(x,t)

The low-pass filter is needed in order to introduce at least a small delay between neurons in order to avoid algebraic loops when the system is executed. It can also be adjusted to modify the dynamic properties of the system. The non-linearity is placed before the filter in order to avoid distortion of the frequency characteristics of the filter. The steady state characteristics of a linear bi-directional dynamic neural network. To get an idea of the behaviour of the control part of the neuro mechanical network a simple system with only four neurons is studied analytically. y2.

y2.

The absolute position X of the neurons can be expressed as a function of the actuator positions as: X = F(X a )

(14)

where F is a non-linear function. Furthermore, the actuator positions are a function of the control signals U from the neurons.

X a = G (U )

(15)

The control signals U are a function of the input signals R

U = H (R )

(16)

If that the inverse of F and G exists then the control problem can be concentrated to H. Assuming that it should be possible to map several different sets of positions on the same number of set of input signals means that there must be at least one solution of H that satisfies all these mapping conditions. U 1   H ( R1 )      U 2   H ( R2 )   .  = .      .  .     U n   H ( Rn ) 

(17)

H does, however, become larger as the system grows so it does not mean that each element has to be more complex as the number of input case increases. It should, however, mean that the structure has to grow in complexity in order to accommodate the needed functionality. This suggests that there should be possible to find a balance between the complexity of the actuator system and that of the control system, which can be transferred to the individual neurons and actuators. The system can be solved for the parameters in H either analytical if the system is small, or using an iterative solver such as Newton-Raphson to solve the system. There is also another possibility of solving the system. This is to use optimisation based on simulation. Using this method the system is simulated using a set of input signal sequences. The objective function could be to minimize the error in a certain node or nodes compared with a reference trajectory. Although this method may sound very time consuming it has some attractive features. Perhaps most important is that it is possible also to include dynamic properties that can be optimised and that there is no restriction on using non-linearities in the neural functions.

EXAMPLE: A TWO DIMENSIONAL ACTUATOR SYSTEM As an example a NMN-system with five neurons and ten actuators is simulated. This is used to study the basic behavior. The system is subjected to two stimuli acting on the two outer neurons. The system can be regarded as a system of valve-controlled hydraulic (or pneumatic) actuators. The objective is to train the system in such a way that the upper one will result in a horizontal displacement of the outmost node and the lower one should result in a vertical displacement of that same node. The system is trained using system optimisation on a simulation model of the system. This is done using the COMPLEX optimisation algorithm and using a simulation model to evaluate the objective function as described in Krus, Jansson and Palmberg 1991. and J Andersson 2001.

x,y xref yref

Figure 7. A neuro-mechanical network with five neuron, eight actuators and two inputs. The simulation is done using the HOPSAN simulation package developed at Linköping University, Department of mechanical engineering. The simulation model is shown in Fig 8.

Figure 8. Simulation model of a neuro-mechanical network with five neurons.

In this model, the object function for optimisation of the system parameters is included. The object function is the sum of the integrals of the absolute values of the error in the x and ydirection. It can be written as:

objfcn = ∫ x − x ref dt + ∫ y − y ref dt

(18)

The system parameters that are optimised are all w1 , w2 and k1 values, which is a total of 30 parameters. The optimisation was performed using the COMPLEX optimisation algorithm. It took 3074 simulation runs to optimise the system. Fig 9 shows the horizontal and vertical displacement of the outer neuron. The tracking is acceptable although the vertical position is disturbed when the input for the horizontal reference position is excited. This is, however, a purely dynamic effect. The steady state behaviour, however, is perfect.

Fig 10 shows the individual actuator positions. The result shows that it is possible to train the neuro-mechanical network to perform a desired task. It clearly demonstrates the potential of the system as a general structure that can be trained to obtain a certain responses from arbitrary stimuli. In this example no attempt has been made to improve the dynamic behaviour. The system is lightly damped with pure position feedback in the actuators. A more elaborate scheme would include adjustment also of actuator internal gains and other control parameters.

Figure 9. The reference and actual positions in x- and ydirections at the tip.

Figure 10. The corresponding actuator positions

DISCUSSION The NMN system can be regard as a functional extension of the neural network NN in that it has other networked properties. In this paper mainly the control aspects of a NMN system have been described. The NMN is a network in the following respects: Neural network Actuation network Mass transport network Energy transport network Sensory network Although the NMN systems are partly aimed towards future technologies, the concept of neuro-mechanical networks can be useful much sooner as it has meaning also at a micro or even macro level in order to realise highly robust flexible actuator systems. Promising technologies for implementing this kind of system are layered manufacturing (or rapid prototyping). There are ongoing efforts to include multi-materials that make it possible to mix both conductive and non-conductive materials in order to achieve functional products directly. One application could be to achieve continuously variable adaptive airfoils for aircraft, to realise morphing wings discussed in Golding, Venneri and Noor, 2001. Another potential use is for design of more conventional system, with a minimum of components. Using the proper cost function for the system, optimisation of the neuromechanical network would reduce all actuators not necessary for the functioning of the system to almost zero, using only the signal processing capability of them thus becoming an analogue of the brain. Some parts might lose their actuation capability but still be retained as purely structural members, thus representing “bone tissue”. With this viewpoint, the generic integrated actuator can be regarded as the equivalence of stem cells, and conversely, all biological organisms can be regarded as neuro-mechanical networks. Incidentally, neuromechanical networks share a fundamental limitation with biological systems. Freely rotating parts, like wheels, are difficult to achieve. Although the number of parameters to optimise is large, (in the example with five neurons 30 parameters where adjusted) the optimisation algorithm manages to find good solutions every time. No attempt has been made to see weather a truly global optimum is reached. This is, however, less interesting. In this kind of system a distinction should be made between complexity in requirements and complexity in the number of parameters. As much as there is a general curse of dimensionality if the requirements are highly complex, there is also a blessing of dimensionality if the design space is large but the requirements are relatively simple. This is because there are many different parts of the design space that can give good solutions due to the many degrees of freedom in the parameters, although in a strict since these are local optimums. The COMPLEX method used here (a modified version of the original COMPLEX by Box 1965), is also very suitable for this kind of problems since the number of objective function evaluations only grows linearly with the number of parameters.

The distributed simulation concept used here (Krus, Jansson, Palmberg and Weddfelt 1990), with separate solvers in each component, based on bi-directional delay lines (Auslander 1968), also makes the simulation time grow no more than linearly with simulation time. Finally, the simulation time step used is fixed which leads to deterministic simulation times. This is a rather important for simulation-based optimisation where some parameter sets can cause highly pathological system behaviours, which can cause excessive simulation times if a variable time step is used, although they are usually far from the optimum.

CONCLUSIONS The neuro-mechanical network is a very interesting subject for research in the area of mechatronic systems and system design in general. The main feature is that a neuromechanical network can provide almost any functionality. It brings together the concepts of neural networks, cellular automata with actuation networks. It can be used as a basic concept for mechatronic systems as we are approaching the nano-age, as well as an explanatory model of biological systems. Another use is as a general structure that can be used with optimisation to form a macroscopic system, such as a robotic arm, where all but a few elements are retained to form an optimal system of actuators, structural members and control system. All redundant parts are reduced to become insignificant.

REFERENCES (1) J Andersson, Multiobjective Optimization in Engineering Design. Linköping Studies in Science and Technology. Dissertations. No. 675. Linköping 2001, Sweden. (2) D M Auslander, 'Distributed System Simulation with Bilateral Delay-Line Models' Journal of Basic Engineering, Trans. ASME p195-p200, June 1968. (3) M. J. Box. A new method of constrained optimisation and a comparison with other methods. Computer Journal, 8:42--52, 1965. (4) DS Golding, SL Venneri and AK Noor: Fresh air, wide-open space, Mechanical Engineering, November 2001 (pp48-55). (5) N Gershenfeld, The Nature of Mathematical Modeling. Cambridge University Press, 1998. (6) P Krus, A Jansson, J-O Palmberg, 'Optimization for Component Selection in Hydraulic Systems' Presented at 'Fourth Bath International Fluid Power Workshop', Bath, UK 1991. (7) P Krus, A Jansson, J-O Palmberg, K Weddfelt. 'Distributed Simulation of Hydromechanical Systems'. Presented at 'Third Bath International Fluid Power Workshop', Bath, UK 1990.