skewed class classificationn with neural network with not much data; procedure to follow

[Patrick Pintelon]

Hello,

This is my first implementation (trial) for a Neural Network.

I use Octave with neural network  to classify (3 classes).

99% of expected network output fall in class 3 (42 'non-class 3' results) .

There are 245 features at the input layer.

I have 4300 examples to train, so I divided the Train, Cv, and Test sets in a way that there are an equal number of 'non-class 3' results in each set.(so => in each about 17 ).

maxIter was fixed at 200)

1- I train the network with my X and y Train set and get the 2 theta's (1 hidden layer) back from fmincg.

2- Then I use those theta's from the Train set as initial theta's for the fmincg function when doing the Cv training with eg. 2 different number of nodes in the hidden layer (350 and 400) and use 3 lambda's so I get 6 combinations  (n° of hidd nodes, lambda, cost, theta-pairs). 

3- Those 6 are used on the Testset 'predict' function which makes the hypothesis, applies it to each example of the 6 combinations on Testset and checks the accuracy op the prediction in % correct results.

So I have not used fmincg on the Testset.

In order to have a less than 1 failure on the 42 non-class 3 predictions I need a accuracy of 99,99% since only 1% is non class 3.

Is the procedure I followed the correct one, or have I missed something?

Thanks!

-- 
Patrick Pintelon
address@hidden