Abstract. In this work a format for neural network models description is introduced. Its main purpose is to provide a unified way for neural network model definition. Format allows interchanging neural models as well as documentation, store and manipulating them independently from the simulation system that produced it. We propose to use XML notation for full description of neural models, including data dictionary, properties of training sample, preprocessing methods, details of network structure and parameters, method for network output interpretation. The first version of DTD for neural model description language is developed. A model description structure, contents of main issues in XML document and example of software structure for handling files with neural model description are presented.
 Key words: Neural networks, Standart, XM

1. INTRODUCTION

Within the last decade artificial neural networks became a wide used technique for solving variety pattern recognition problems. Today there is known huge amount of software concerning with neural networks simulation. But in our opinion there is the problem of low integration between simulation systems that have such consequence as absence of unified form to describe of an arbitrary neural network model independently from system that produced it. The reason for this is that the field of artificial neural networks is evolved independently in and among many scientific communities such as neurobiology, mathematics and applied statistics, artificial intelligent and pattern recognition. As a result, a variegated terminology and synonymic nomenclature was developed.

One of possible ways to raise effectiveness of neural network application is introducing a unified format for description of neural network models. The problem of development of unified format is closely connected to the investigation of standards for the description of various neural network model components. There have been a few publications that devoted to neural network standardization and network description language development [1,2,3,4]. In [5] is offered XML-based specification for definition multilayer backpropagation networks as part of Predictive Model Markup Language. In contrast to them we concentrate on developing a unified electronic format for description of neural network models that allow interchanging neural nets information between different simulation systems. This publication offers a declarative language for unified description of any trained multilayer neural network. We propose to use XML notation for full description of neural models, including the types of data input to and output from the models and its preprocessing, details of network structure and parameters, and how to interpret their results.

Used method of decomposition allows to describe some of most widespread types of neuron units: with various sigmoid functions, with threshold, linear and piecewise-linear transfer functions, radial basis functions, Pade-neurons, neurons of high degrees [6] etc. With proposed format multilayer networks (possible, pruned or networks with overlayer connections) and recurrent networks can be definite.
 
 

2. DECOMPOSITION OF NEURAL NETWORK MODEL

In the figure 1 main components of an information structure of neural network models description is presented. The neural network model is considered as sequential combination of preprocessing component, neural network and output signal interpreter.

Components of neural network model (the same as data fields, preprocessing modules for discrete and continuos data, neurons, and interpreter functions) are represented by a collection of connected objects. Identifier of functional transformation and set of pairs <parameter – value> determine each object. For instance, preprocessing component is presented as a set of parallel functioning objects, that transforms input from one or several data fields to signals, which then are sending to an input layer of a network. As a transformation in this case normalization (for continuos data), coding (for discrete data) or more sophisticated functions may be used.

Each neuron is represented as a node, which consists of two consecutive combining modules. First module combined inputs of neuron with the weights to yield a single value to which the activation function is applied. This module is called a neuron combination function [7]. Second module is represented a neuron activation function. Varying these functions we can get a different types of neurons. For example, using scalar product of inputs and weights as combination function and varying activation function we can describe some main types of neuron units, used in multi-layer perceptrons: with various sigmoid functions, with threshold, linear and piecewise-linear transfer functions. The application of various types combination functions permits to describe such kinds neurons, as radial basis functions (Euclidean distance), Pade-neurons (ratio of two scalar products), neurons of high degrees (high orders polynomial) [6] etc.

Neural network is defined as a sequence of neuron layers. First layer called “input layer” transmits signals from preprocessing units to neurons on hidden layers. The network connection structure is specified by set of parameters of neuron combination functions. In common case, each parameter is described by two fields: by an identifier of source of input signal and a scalar connection weight. The identifier of neuron, that sending its output signal to given neuron, or identifier of preprocessor module (for neurons of an input layer) is used as reference to sources of signals for given neuron. It is possible to use two scalar (for example in the description of bias input, where one scalar is bias level and second is weight) or two identifiers (in case, when the connection weight is output signal from other neuron). Thus, the structure of connections in a network are specified implicitly, through the list of the references, and the topology of a network is described locally.

The various combination functions can have various quantities of parameters and various structure of their description. So, for example, for n input signals, the standard scalar product with bias input has n + 1 parameter, Pade-neurons has 2n + 1 parameters, and each parameter for a quadratic function is set by three fields: by two references to sources of signals and scalar weight.

The name of transformation and list of parameters describe the neuron activation function. Quantity and sense of parameters are various for different types of transfer functions.

The interpreter is considered as a device, which on the basis of an output vector of neural network determines value of each of target parameters of neural network model. For each target parameter, in such case, the independent scheme of interpretation is used, hence it is possible to describe the interpreter as assembly of several (according to number of target parameters) independent modules.

3. DESCRIPTION OF NEURAL NETWORK BASED ON XML NOTATION

As a tool for creation of electronic format of the neural network models description the language XML [8] was chosen. XML notation permits to describe structured data and is widely used as the universal basis for development of specialized languages of various nature objects description. As advantage of XML usage can be referred standardized access to a data, stored in XML documents, not dependent from their structure and subject area.

DTD for neural network description format that specifies dictionary of language, rules of creating neural network models descriptions, allowable structures of elements, is developed. The XML document has root section <NEURALMODEL>. Main subsections of the document are: <Task> - contains problem and task description, information about author and creation tool, key words, copyright; <Data> - contains description of sample, that used for neural network learning, description of properties of input and target variables and preprocessing methods; <NeuralNet> - includes the description of general neural network parameters, report about network learning process, description of a neural network structure and neuron units parameters and constraints; <Interpretator> - contains methods for producing interpretable values from neural network answers.

Section <Declarations> is also allocated, which aimed for containing user-defined parameters of model.
The example of formal neuron description in developed language is indicated on fig. 2.
Instance of processing of files with neural model description is presented on fig. 3. Direct access to files of the description is provided by XML-parser. Its main function are: reading / writing of XML files, reconstruction an objects tree of neural model description, verifying XML document with DTD, granting access to elements of the description with set of methods and objects.
More detailed discussion about development and realization of unified format for neural network description may be found in [9].
With use of the developed unified format, and also part of the specification PMML 1.1 [5] (tree classification models) the development of the intellectual system shell is supposed. The given shell will share symbolical (first-order logic) and neural network (connectionist) paradigms for knowledge representation of expert. The shell will be used for construction of the knowledge bases of the distributed intellectual systems. The chart of components interaction of the given system is shown at fig. 3 [10]. The basic difference of the given system from systems such as MIX [11,12] consists that through Internet not the external interface of system (fields filling form) is transferred to the user, and also the interpreter of the hybrid knowledge base is transferred. The given technology has the following basic advantages: (i) the user data are not transferred through global network, that provides their security; (ii) the given approach allows to ensure independence of server software and hardware realization.

In the article we have offered the unified format of neural network model description. The goal of format creation is a unification of keeping and issue of neural network models. One of the possible practical applications of has given approach - an applied intelligent information system building. On the base of developed standard is expected building of distributed intelligent system shell. Using Internet for the transfer of learned neural network and tree classification models will allow vastly reducing expenses on the renovation of specified system knowledge bases.

1. Leighton, R.R. The Aspirin/MIGRANES neural network software. Users manual. MITRE Corp. 1992. 119p.
2. Mirkes, Е.М. Neurocomputer. Project of standard. Novosibirsk: Nauka, 1998.- 188 p (in Russian).
3. Fiesler, E. Neural network classification and formalization. – Computer Standarts and interfaces. v.16, Elseiver Science publishers, Amsterdam, 1994. 13p.
4. Atencia, M. A., G. Joya and F. Sandoval. A formal model for definition and simulation of generic neural networks. Neural Processing Letters 11: 87-105, 2000. Kluwer Academic Publishers.
5. The Data Mining Group. PMML 1.1 Neural Network http://www.dmg.org/html/neuralnetwork.html
6. Bishop, C.M. Neural Networks for Pattern Recognition. Oxford: Oxford University Press. 1995.
7. Sarle W. Frequent asked question on neural network. ftp://ftp.sas.com/pub/neural/FAQ.html
8. Extensible Markup Language (XML) 1.0. W3C Recommendation. http://www.w3.org/TR/1998/REC-xml-19980210.
9. Rubtsov D.V. Development of technology of artificial neural network application in applied information systems. Ph.D. thesis. Barnaul: Altay state university, 2000. - 140 p.
10. Butakov S., Rubtsov D. Interaction an intellectual component of information systems through Internet / Information Technologies, 2000, № 4., P. 52-53 (in Russian)
11. Fernando S. OsoHrio, Bernard Amy. INSS: A hybrid system for constructive machine learning / Neurocomputing 28 (1999). P. 191-205.
12. B. Amy et al., MIX medical application (Final Report } Deliverable D16), Technical Report, Project Esprit- BRA &MIX' }European Community, May 1997.