NN-based software tool for wireless communications simulation B. Romero, M. Rubiolo , G. Stegmayer, Member, IEEE, and O. Chiotti

Abstract—This paper presents a free software toolbox to support the task of modeling and simulation of mobile communications. The toolbox supports the task of automatic neural network-based model generation of devices or components of a wireless communication chain. The models can be trained, validated and simulated, directly from measurement data, using an interface specially designed for non-expert users in the neural network theory. The resulting models can be automatically exported into a commercial CAD simulator for simulation under different scenarios. Index Terms—neural networks, wireless communications, simulation, software toolbox

N

I. INTRODUCTION rd

generation (3G) wireless communications ew 3 standards, such as WCDMA (Wideband Code Multiple Division Access) and UMTS (Universal Mobile Telecommunications System), may introduce changes in the behavior of the devices and components that participate in a communication chain (i.e. mobile phones). These changes are mainly due to the algorithms they apply to the modulated signal, generating nonlinearities and memory effects (when an output signal depends on past values of an input signal) that may result in a deterioration of the system performance [1]. Therefore, inside the communication system design cycle, components modeling has become a critical task.

Fig. 1. Simplified block diagram of a digital wireless transmitter.

In this work we are interested in modeling the nonlinear behavior that an amplifier can have inside a wireless transmission. Amplifiers are a major building block of modern Radio Frequency (RF) digital wireless transmitters (i.e. mobile phones). Figure 1 shows a simplified block diagram of what a cellular phone communication would be. The voice coming from the phone speaker (analog signal) has to be digitalized with an Analog/Digital converter to be transmitted through the wireless network. The digitalized voice has to be compressed to reduce bit rate and bandwidth. It is also codified, to format the data so the receiver can detect and minimize errors by doing the reverse operation. After that, a modulator adds the carrier signal to the data signal. The signal has to reach an Manuscript received October 5, 2006. M. Rubiolo, B. Romero, G. Stegmayer and O. Chiotti are with the R+D Center CIDISI, Universidad Tecnológica Nacional-Regional Santa Fe, Lavaise 610, 3000, Sta. Fe, Argentina (e-mail: [email protected]).

antenna from the cellular phone with enough strength to guarantee the communication. But it suffers from attenuation and needs amplification before that. Therefore, the final element of the chain is a power amplifier (PA) that amplifies the signal before it travels to the nearer antenna and to the receiver side of the communications chain. Nonlinear behavior modeling has been object of investigation and increasing interest in the last years [2]-[4] since the classical techniques that were traditionally applied for modeling are not flexible and fast enough for taking into account these nonlinear characteristics. That is why new techniques and methodologies have been recently proposed, such as Artificial Neural Networks (NNs). Nowadays, the modeling and simulation of nonlinear elements inside a wireless communication system using NN-based models [5] is receiving increasing attention for a large variety of industrial applications [6], since model development only needs a training procedure based on experimental data or measurements. In the dynamic wireless market, NN-based models are specially suitable for communications components modeling, because they would allow for fast device model development and characterization. For example, neural models could contribute to a reduction of the design cycle when new semiconductor device technologies appear, being of particular interest for the industry. A neural model can be used during the stage of system design for a rapid evaluation of its performance and main characteristics. Neural models can be an efficient link between measurements and simulations, allowing for anticipating the consequences of technological choices for circuit performance. The model can be directly trained with measurements extracted from the real system, thus speeding up the design cycle and time-to-market of new products. However, the process of neural model development is not trivial and involves many critical issues such as data generation, normalization, NN topology definition, number of hidden neurons, learning rules, among others. As NN techniques are relatively new to the microwave community, it is often not easy for electronic engineers to make decisions regarding these issues. Therefore, a software tool for supporting NN-based models development could be of great interest to RF/microwave engineers, whose knowledge about NN theory may be limited. For this purpose, this paper presents a software toolbox that gives support to automatic neural model generation and simulation, directly from measurement data, which can also be exported into a traditional circuit simulator. The organization of the paper is the following: in Section II the toolbox main features are described. Section III presents

the neural network models supported by the proposed toolbox. In Section IV some implementation details appear and Section V shows a complete case study simulated with the software tool. Finally, the conclusions can be found on Section VI. II. TOOLBOX FEATURES There exist commercial and non-commercial products available on the market that allow for the creation and use of neural models, for example, for a communication system and components modeling. Nevertheless, most of them assume that the user is an expert in NN theory, because a rather deep understanding about how the NN paradigm works is needed to define a model. To be able to exploit all the features of existing available neural toolboxes, it is necessary to know very well the toolbox specific commands and language. Deep knowledge about neural models parameters, learning rules, etc., is required, which makes its use restrictive only to expert users. Finally, another important point is that, if one has been able to create a neural model, it cannot be exported automatically to be used inside any available commercial circuits simulator. Therefore practically they cannot be used for real simulations, i.e. of a whole communications system. Recently, MatLab® has incorporated a Neural Network Toolbox®[7] that allows creation, training and use of several kinds of Neural Networks to be applied for a large variety of problems. Among its advantages we can mention the ability to graphically visualize a neural model topology and a unique window with several tabs for training, simulation and testing options. However, to be able to use it, it is necessary to know very well the language and specific MatLab commands. This requires a good knowledge of neural network models creation and training, which restricts its use to expert users. Another important point is that these models cannot be exported inside a commercial circuit simulator, therefore it is not possible to use them in simulations of a whole communications system or chain. Another commercial alternatives are NeuralPlanner [8], NeuroSolutions [9]; among non-commercial products we can mention Lens Neural Network Simulator [10], PDP++ [11] and SNNS [12]. However, all of these tools are too general to be used in the mobile communications field, because they have been designed for the creation of generic neural networks models, and require an expert user. Considering only tools developed for neural black-box models of electronic device, there is MπLog [13] whose use is specifically limited to modeling analog-digital drivers, and it has been developed using the MatLab compiler, therefore requires their libraries to execute. A software that tries to overcome these problems is NeuroModeler®[14]. With this tool, electronic devices can be modeled. The main problem of this tool is its poor interface design. The menus design generates cycles on opening new windows, which superimpose on each other generating confusion, the user does not know on which model and on which window it is working at any time. It also requires quite deep knowledge of the neural theory. Beside this, it generates model that can be exported inside a specific circuit simulator, which requires expensive licences of installation and use.

In summary, to be able to use at maximum all the characteristics of existing neural network software tools, it is necessary to know very well the language and commands of each tool. Deep knowledge on neural parameters, topologies, learning rules, among others, is requires, which limits their use to expert users. Furthermore, if the user manages to create a neural model, it is highly likely that it would not be exported automatically into any available circuits simulator In this context, appears the need for a simpler software toolbox easy to use for an electronic engineer which must create and manage behavioral models based on NNs, but has the minimum knowledge required for making a black-box model and setting some parameters in an easy way. This paper presents a software toolbox which tries to fulfill these requirements. It has been designed bearing in mind a non-expert user trying to use a toolbox for neural model creation, but having no deep understanding on the NN theory. The software toolbox has been implemented in Java and some of its features are support for model creation, editing, training and simulation, selection of different activation functions, testing and plotting the results. Its main advantage over other existing tools, besides the user-friendly interface design for non-expert users, is its capability to automatically extract from measurement data the input/output variables of the neural model, and after the training phase, the trained neural model can be directly exported as a full-black-box model into a simulator, i.e. RF/Microwave circuit simulator, that allows a full wireless communication chain simulation with embedded neural models treated by the simulator as black-box models. Measurements data files generated at the laboratory are directly loaded into the toolbox. They have to follow a specific format, which allows for the automatic detection of number of input/output variables. From the detection of the variables in the measurements, the neural model is automatically created, which simplifies the design task and gives a starting point to the designer, who can change the original network design proposed by the toolbox and set some model parameters, such as number of hidden neurons and activation functions, if desired. Once the model has been defined, it can be trained up to a user-defined accuracy, or simulated (tested) with the available measurements. If after training the desired accuracy has been reached, the model can be save (exported) as a text file that includes the neural model and its parameters values. Once saved into a file, the NNmodel can be easily loaded as a device or circuit black-box model inside a commercial circuit simulator. This procedure will be illustrated with a case study. III. NEURAL NETWORK MODELS SUPPORTED The software toolbox that we have developed supports the creation of some classical and widely-used feedforward neural network topologies such as the multilayer-perceptron (MLP). This model is of particular interest for the RF/Microwave community for modeling components that are part of a wireless communication chain [15]. Time-delayed neural network (TDNN) models are specially suited for modeling dynamic behavior. A TDNN is based on the MLP

model with the addition of tapped delay lines (Z−1) which generate delayed samples of the input variables. They are used to add the history of the input signals to the model, needed for memory effects modeling in transceivers components such as power amplifiers (PAs) [16]. The patterns needed to train a TDNN model include not only the current value of the input signal x(t), but also its previous values, as illustrates Fig. 2. The memory depth (n) of the element or system analyzed is reflected on the length of the taps. The strategy followed to set the system memory is dictated by the bandwidth accuracy required. Our toolbox, differently from existing tools, gives specific support for the automatic creation of such a model, which requires data patters organized in a particular way. This will be exemplified in the next Section.

Fig. 2. Time-Delayed neural network (TDNN) model and its corresponding input data

The good generality property of a NN model says that it must perform well on a new dataset distinct from the one used for training. Even a excessive number of epochs or iterations on the learning phase could make performance to decrease, causing the over-training phenomena. That is why, to avoid it, the total amount of data available from measurements is divided into training and validation subsets, all equally spaced. The ”early-stopping” technique [17] for training NN models avoiding the mentiones phenomena, has been included into the toolbox features. This algorithm states that, if there is a succession of training epoch in which accuracy improves only for the training data and not for the validation data, over-training has occurred and the learning is terminated.

IV. CASE STUDY The software toolbox proposed in this work has been implemented in JAVA using the development tool IBM Eclipse [18] and the JOONE Framework (Java Object Oriented Neural Engine) for NN model creation [19] providing an LGPL (Lesser General Public License) license use. It has a modular architecture that allows easy extension and modification of its features, i.e. adding new training algorithms. The framework represents a NN model as composed by a number of objects, such as layers, connected among them by synapses. Depending on how these objects are related among them, several neural architectures can be created (feedforward, recurrent, self-organized maps, etc.). This section presents a case study which has been modeled and simulated with the developed software toolbox. With the example we would like to illustrate how it helped improving the design cycle by simplifying the tasks of model design and simulation. A more complete analysis of neural models for modeling nonlinear electronic devices can be found in [20]. As a case study, a TDNN model is used to characterize PA dynamic effects adding several time-delayed inputs. This model has been already successfully applied to model the nonlinear behavior in presence of high-order nonlinearity with medium-to-strong memory effects once trained with inputoutput time-delayed data samples at different power levels. Complete characterization was performed for different classes of operation. We report here some results for a class A pa of 1 mm total gate peripher GaN HEMT based on SiC at VDS=30 V,with IDSS = 700 mA, measured at 1 GHz. Figure 3 shows how the output signal of a device, as response to an input stimulus i.e. a modulated signal to be amplified by a PA, is measured by a system at the laboratory, which produces the data that is used to train the TDNN model in a supervised way. Once the TDNN model has learnt the nonlinear and dynamic behavior of the device under study, it can be used as a black-box model inside a circuits simulator and test it under different working conditions.

Fig. 3. TDNN model of an electronic device directly trained with device measurements.

Figure 4 shows schematically how our toolbox gives support for the creation of such a model directly from

measurement data, translating the data into the internal java object oriented representation of a neural model; and figures

5, 6 and 7 are snapshots of the screens that appear to the user when he accepts the help of a wizard to guide him in the model definition. The input/output files containing the measurements made at the lab, are loaded into the toolbox (fig. 5). Next, the user must decide which type of model (fig. 6) he wants to create (traditional MLP or TDNN). Finally, in step three the user must name the model that has already been created into the software, simply using the training data files. If he wants to, the user can change some model parameters that are suggested by the software, such as number of hidden neurons and delays of the input variables. The number of input/output variables cannot be changed because this is automatically detected from the data files. Up to this point, the measurements obtained at the laboratory have been loaded directly into the toolbox with txt files, simply formatted into columns which helped recognizing different input variables automatically.

user can change. The suggested number of hidden neurons is calculated with a common heuristics in the neural field, to be it equal the number of inputs.

Fig. 6. Model definition step 2: select model type.

Fig. 4. Neural network model definition from measurements.

Fig. 7. Model definition step 3: model parameters.

Fig. 5. Model definition step 1: select data files.

Once the toolbox recognized the input/output patterns, a feed-forward TDNN model with three layers has been created: an input layer composed of the input time-domain voltage discrete samples and their delayed replies, a hidden layer with nonlinear activation functions, and a linear output layer. For this example, the two input variables are the drain and gate time varying voltages VDS and VGS, with memory level N=4. This is a default number of delays to be considered for each input variable in the case if a TDNN model, which the

The model parameters (weights and biases) can be randomly o statically initialized, according to the user preferences. The tool provides the capability of extending the heuristics used inside the toolbox, this way, if in the future a better rule is found for deciding the number of hidden neurons in a MLP model, it could be included into the toolbox, transparently to the user. The system outputs are the time samples of the drain current IDS. The procedure used inside the toolbox for creation of the patters necessary for training a TDNN model is shown in figure 8. Samples have been collected on a 2 ns window. Input data vectors for training from all different input power levels have been first joined together, and the resulting vector copied and delayed as many times, to represent the NN input, as necessary to account for memory. Once the model has been created, it appears in a model-base manager at the left of the main interface (figure 9) where all the active models can be quickly identified and accessed. The main window in the center shows at any time the options

selected in the model base on the left. In figure 9, the “train” option is selected for “Model 1” and therefore a window with all the train options is shown to the user.

access in many alternative ways (main menu, right click, icon). I. CONCLUSIONS

Fig. 8. TDNN training patterns creation from input data files.

There are default values for the training algorithm (second order backpropagation) that the user can change, furthermore, if he has more knowledge about training algorithms, he can access the advanced options tab to configure learning rate, a momentum parameter, etc. Figure 10 shows a detail of the waveforms used for training the TDNN model presented in this case study. At any time, the user has, at a glance, all the loaded model available and the options associated to each one, which he can

In this paper we have presented a software toolbox designed to bring support for electronic engineers in creating NN-based components models and then using them as a black-box model inside a circuits simulator. It supports the automatic creation of NN-based models by means of an appropriately designed model base manager. The main advantage of this software tool over existing software, is that it is free, it helps improving the design cycle, simplifying model design and simulation tasks, even when users have not deep knowledge of NN-theory. The case study shows some details of the support provided by this tool . The software has been designed for a specific domain (wireless communications modeling) and the interface has been oriented towards satisfying electronic engineers designers. Engineers from Politecnico di Torino have participated in the toolbox interface design and they have tested it. The software will be soon freely available at http://cidisi.frsf.utn.edu.ar for download and to receive feedback from different users. II.

ACKNOWLEDGMENTS

The authors thank Banco RIO and Portal Universia for the financial support under project number INV-1334, and Politenico di Torino for the measurements used in this work.

. Fig. 9. Snapshot of the simulation software toolbox developed.

Fig. 10. Training set: time domain IDS waveforms at 1 GHz at increasing power for class A at optimum Pout. Measurements (top), TDNN simulations (bottom).

I. REFERENCES [1] [2]

[3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18] [19] [20]

C. Evci, U. Barth, P. Sehier and R. Sigle, “The path to beyond 3G systems: strategic and technological challenges”, in Proc. 4th Int. Conf. on 3G Mobile Communication Technologies, pp. 299-303, 2003. A. Ahmed, M.O. Abdalla, E.S. Mengistu and G. Kompa, “Power Amplifier Modeling Using Memory Polynomial with Non-Uniform Delay Taps”, in Proc. IEEE 34th European Microwave Week, pp. 14571460, 2004. H. Ku and J.S. Kenney, “Behavioral Modeling of Nonlinear RF Power Amplifiers Considering Memory Effects”, IEEE Trans. Microwave Theory Tech., vol. 51, no. 12, pp. 2495-2504, 2004. H. Qian and G.T Zhou, “A Neural Network Predistorter for Nonlinear Power Amplifiers with Memory”, in Proc. 10th IEEE DSP Workshop, pp. 312-316, 2002. Q.J. Zhang, K.C. Gupta and V.K. Devabhaktuni, “Artificial Neural Networks – From Theory to practice”, IEEE Trans. Microwave Theory Tech., vol. 51, no. 12, pp. 1339-1350, 2003. M.R.G. Meireles, P.E.M. Almeida and M.G. Simoes, ”A comprehensive review for industrial applicability of Artificial NN”, IEEE Trans. Industrial Electronics, vol. 50, no. 3, pp. 585-601, 2003. http://www.mathworks.com/products/neuralnet/ http://www.tropheus.demon.co.uk/nplan.htm http://www.nd.com/download.htm http://tedlab.mit.edu/~dr/Lens/ http://www.cnbc.cmu.edu/Resources/PDP++//PDP++.htm http://www-ra.informatik.uni-tuebingen.de/SNNS/ I. Stievano, I. Maio and F. Canavero, “M[pi]log, Macromodeling via Parametric Identification of Logic Gates”, IEEE Trans. on Advanced Packaging, vol. 27, no. 2, pp. 15-23, 2004. http://web.doe.carleton.ca/~qjz/qjz.html D. Root and J. Wood. Fundamentals of nonlinear behavioral modeling for RF and microwave design. Ed Artech House, Boston, 2005. G. Stegmayer, “Neural-based identification for nonlinear dynamic systems”, IEEE Int. Conf. on Computational Intelligence for Measurements Systems and Applications (CIMSA), pp. 290-294, 2005. J. Sjoberg and L. Ljung, “Overtraining, regularization and searching for a minimum, with application to neural networks”, Int. J. Control, vol. 62, pp. 1391-1407, 1995. http://www.research.ibm.com/eclipse/ http://www.jooneworld.com/ G. Stegmayer, “Nonlinear RF-Microwave device and system modeling using neural networks”, PhD Thesis dissertation, Politecnico di Torino, Italy, 2006.