# of neurons in Hidden layer.

Efeff1a01dbd2c5632a316cc52584dbf
0
greenleaf 101 Oct 14, 2005 at 03:58

Hi there,

I am trying to build a classification problem solution using ANN. I have to select one of six different choices hence I am using 6 neurons in the output layer while four inputs allow for four neurons in the input layer.

Hidden Layer: Now here I just want to have two neurons(for simplicity). Are 2 neurons enough(4-2-6 ANN)? Or should I have more neurons in the hidden layer aswell for proper results.
In general is there a criteria by which one should decide the number of neurons in hidden layer.

Thank you in advance.
-GL

11 Replies

Please log in or register to post a reply.

4a93ba032c357d782afb0820f328d14e
0
zavie 101 Oct 14, 2005 at 12:41

@greenleaf

In general is there a criteria by which one should decide the number of neurons in hidden layer.

Some non deterministic function using experience, chance, and testing as parameters… :->

6cad28055d4af37574fe0d4064555e53
0
GroundKeeper 101 Dec 28, 2005 at 13:05

I think what Zavie is saying is that there is no one that really knows. My hint would be not to many and enough to be able to learn the concept! ;P

B7257332ac4ff66d7a57708860c30904
0
cheng 101 Dec 28, 2005 at 18:09

I have a limited amount of experience with this, and what I do have can hardly be called rigorous. I’ve only done relatively simple research.

I’m not sure you can call my approach academic, and it doesn’t always work out the way I think it will, but I try to break down the learning process into steps, and start my network with a neuron for every step (not counting the input and output layers.) The arrangement isn’t necessarily a linear one though - that is to say, you’ll usually find some steps need to be made concurrently. Other steps that appear to be a single evaluation point, end up being multiple evaluation points (the reverse is also sometimes true.) Then tweak until it works

A word of warning about my advice though, I tend to be a little to boolean about things, overtrain, and spend a lot of time reinventing the wheel.

There is surprisingly little literature available on ANNs, at least that is understandable by a reasonably well-educated lay person.

Da26e799270ce5e8b62659ed77b11cef
0
Axel 101 Jan 15, 2006 at 18:07

You can also use a genetic algorithm to grow a neural network.

2a316b135fe9b24a09f153f6b9ee413a
0
bubu_LV_ 101 Jan 15, 2006 at 18:30

I think it is no point of using lesser number of neurons in hidden layer than in output layer. Because it is no point getting more data (6 values) from less available data (2 values).

6cad28055d4af37574fe0d4064555e53
0
GroundKeeper 101 Jan 18, 2006 at 08:30

Hmmm, a large number of neurons don’t make the network smarter! A neural network (this is baed on the fact that we do not know really) is very hard to analyze and the attempts that have been done have been on simple functions. I wouldn’t assume nothing and instead run more tests. There are ways of calculating much more precise numbers of neurons bu it requires more knowledge then I have in the area. I suggest take a course in Information Theory since they of some means of interpeting information and information gain.

6f0a333c785da81d479a0f58c2ccb203
0
monjardin 102 Jan 18, 2006 at 14:37

For a 3-layer feed-forward network, I’ve heard that the square root of the product of the inputs and outputs is a good number. You can start tweaking it from that point. So, your magic number that may or may not work for you is:
sqrt(6*4) = 4.8989794855663561963945681494118 ;)
Try 5?

6cad28055d4af37574fe0d4064555e53
0
GroundKeeper 101 Mar 04, 2006 at 08:34

Okey and to what extent does it relate to any analytically shown fact? I mean why is such a guess better then lets say log(input * output) which I could give reasoning to? Just wondering interesting thought!

6f0a333c785da81d479a0f58c2ccb203
0
monjardin 102 Mar 06, 2006 at 22:30

There are no analytically shown facts. I read it in an O’Reilly book and thought that it was as good a starting point as any… However, you have to admit that it is superior to the log of the product since log10(6*4) rounds to 1, but log2(6*3) gives you 3. That might not be too bad depending on which log you were talking about. :D

6cad28055d4af37574fe0d4064555e53
0
GroundKeeper 101 Mar 12, 2006 at 10:55

Just an exemple of how easy it is to give a unjustified statement of correctness if you only consider the elements and not the conceptual model. In neural networks you can’t generalize the number of neurons and layers because they are a function of the expected “intelligence”.

6f0a333c785da81d479a0f58c2ccb203
0
monjardin 102 Mar 13, 2006 at 17:56

:worthy: I appologize. :worthy: