Answer My Searches

A competitive learning network (CLN) is an unsupervised learning technique that is used to find similarities in examples in a dataset and therefore suggest groupings of the examples. This is known as a clustering algorithm since it “clusters” the data examples together into similar groups. It is called unsupervised because we do not tell the algorithm how to group the examples together. It is left to its own devices to figure out any similarities in the data.
A CLN is made up of two layers of nodes; an input layer and an output layer. Each node in the input layer connects to every node in the output layer. Each connection has a weight assigned to it. The values assigned to these weights are where knowledge is stored in this network. The each node in the input layer is associated with a feature in the dataset. Each node in the output layer corresponds to a potential grouping of the data.

To provide a clearer picture of how this all works, below are a table showing our animal dataset, and a diagram of how the CLN is set up.

Image from Knight[1]

The CLN learns by being having each record (stimuli) fed into the input layer and having the output nodes compete for control of that input. It may sound complicated but it’s not. In this sense “competing” just means that the output node with the highest weighted sum of its inputs “wins”.
The winner gets all of his incoming weights from active nodes increased while the losers do not get their weights changed. The effect of updating the weights like this is that the winner then becomes more likely to win a future input with similar attributes.
There are a few different methods for increasing the weights and distributing the weights. The most popular method is to increase each of the winner’s weights by following this formula :

weight_value = (learning_rate)*(input_value/number_non_zero_input_nodes)-(learning_rate*previous_weight_value)

The learning rate should be a small constant between 0 and 1. I used 0.25 in my experiments.

————————————

This is from a portion of a paper I wrote for a Neural Networks class I recently took. Read the full paper here.

I know this material is tough when you’re starting out, so if you’re still confused just leave a comment and I’ll try to help you out.

References:
1. K. Knight, “Connectionist ideas and algorithms,” Communications of the ACM, vol. 33, no. 11, pp. 59–74, 1990.

[tags]Knight, Connectionist ideas, ACM, Neural Networks, Competitive Learning Network, Competitive Learning Networks, Cluster, Clustering, Unsupervised Learning[/tags]