Neural gas

Neural Gas - a biologically inspired adaptive algorithm, coined by Martinetz and Schulten, 1991. It sorts for the input signal according to how far away they are. A certain number of them are selected by distance in order, then the number of adaptation units and strength are decreased according to a fixed schedule.

Algorithm

The rough steps of the Neural Gas algorithm can be specified as

Assuming that we have a distribution p(ζ)for which a Neural Gas model has to be created. The following parameters are needed for the Algorithm initialization.

 λi,λf and Ei,Ef, and tmax

 λi,λf are used to set the rate at which learning rate E converges

 Ei,Ef are the initial and final learning rate E respectively

 tmax is the time till which the process continues.

Step 1. Create a Set A to contain N units each with a vector reference from p(ζ). Also initialize the time parameter to 0.

A={C₁,C₂,...C_N}

 t=0

Step 2. Get a random value from the distribution p(ζ) and call it X.

Step 3. Line up all the elements from A in relation with their nearness to X, with the nearest coming first and the farthest the last.

 Thus line up A's vectors such that for Cp,Cm,Co... the corresponding vectors Wp,Wm,Wo,...

      ||Wp-X|| <=||Wm-X||<=||Wo-X|| holds true

 The norm || || usually taken is the square norm.

Step 4. Change the vectors for Cp,Cm,Co...

 ΔWi = E(t)*hλ(ki(X,A))*(X-Wi)

 Where,

   λ(t) = λi(λf/λi)(t/tmax)

   E(t) = Ei(Ef/Ei)t/tmax

   hλ = e(-k/λ(t))

Step 5. Increment t

 t=t+1

Step 6. tmax return to step 2.

Comments

1. The neural gas model does not delete a node and also does not create new nodes.

2. The neural gas model will require fine tuning of the λ parameters especially to achieve a good convergence rate and stable model.

3. It is topology representing neural network, that is after reaching convergence (>t_max), the network node's vector would be representing the distribution being modelled.

References & External Links

• T. M. Martinetz and K. J. Schulten. A ``neural-gas'' network learns topologies. In T. Kohonen, K. Mäkisara, O. Simula, and J. Kangas, editors, Artificial Neural Networks, pages 397-402. North-Holland, Amsterdam, 1991.
• Neural Gas Algorithm
• Java applet. It show evolution of Neural Gas, Growing Neural Gas and several other methods related to competitive learning.
• Growing Neural Gas videos.
• Java Competitve Learning Application A suite of Unsupervised Neural Networks (including Self-organizing map) written in Java. Complete with all source code.