Timeline for Why is my neural network not working?
Current License: CC BY-SA 3.0
52 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Nov 30, 2016 at 18:38 | history | edited | harry lakins | CC BY-SA 3.0 |
deleted 3186 characters in body
|
| Nov 29, 2016 at 18:32 | vote | accept | harry lakins | ||
| S Nov 29, 2016 at 18:32 | history | bounty ended | harry lakins | ||
| S Nov 29, 2016 at 18:32 | history | notice removed | harry lakins | ||
| Nov 29, 2016 at 14:22 | answer | added | A. STEFANI | timeline score: 0 | |
| Nov 28, 2016 at 20:23 | history | edited | harry lakins | CC BY-SA 3.0 |
deleted 2 characters in body
|
| Nov 28, 2016 at 19:50 | history | edited | harry lakins | CC BY-SA 3.0 |
added 2 characters in body
|
| Nov 28, 2016 at 19:44 | history | edited | harry lakins | CC BY-SA 3.0 |
added 2 characters in body
|
| Nov 28, 2016 at 19:44 | comment | added | harry lakins |
I have tested the forward feeding many times manually and it is 100% correct, using the sigmoid return 1/(1+(math.exp(-value))) . i have also manually tested the back prop, and it is defiantly working the way described in the tutorial i linked
|
|
| Nov 28, 2016 at 17:47 | comment | added | user2570465 | It's completely possible that something outside the neural network went wrong and it's possible that you have multiple errors in and out of your neural network code. For example, you could be messing up the training data processing in addition to having wrong backprop code. Can you post your entire code? It would also help me a lot if you post whether the output is correct when you substitute your neural network with a neural network from a well-known library (e.g. Tensorflow, Skflow, Keras). | |
| Nov 28, 2016 at 17:39 | history | edited | harry lakins | CC BY-SA 3.0 |
deleted 2 characters in body
|
| Nov 28, 2016 at 17:06 | history | edited | harry lakins | CC BY-SA 3.0 |
added 2 characters in body
|
| Nov 28, 2016 at 14:31 | history | edited | harry lakins | CC BY-SA 3.0 |
deleted 2 characters in body
|
| S Nov 28, 2016 at 14:03 | history | bounty started | harry lakins | ||
| S Nov 28, 2016 at 14:03 | history | notice added | harry lakins | Authoritative reference needed | |
| Nov 28, 2016 at 11:35 | answer | added | viceriel | timeline score: 1 | |
| Nov 28, 2016 at 11:26 | history | edited | harry lakins | CC BY-SA 3.0 |
edited title
|
| Nov 28, 2016 at 9:50 | comment | added | harry lakins | I have already tried many learning rates :) | |
| Nov 28, 2016 at 9:15 | comment | added | Dennis Soemers | @harrylakins Yeah ok in your case, when input is typically only positive, I dont expect scaling it to [-1, 1] would help. I still expect using a softmax function as activation function for the output layer to help a lot. If learning initially seems too slow, you can start out with a learning rate of 0.5, but you will want to gradually decrease the value of the learning rate then. | |
| Nov 27, 2016 at 22:37 | history | edited | harry lakins | CC BY-SA 3.0 |
added 2 characters in body
|
| Nov 27, 2016 at 21:04 | history | edited | harry lakins | CC BY-SA 3.0 |
deleted 2 characters in body
|
| Nov 27, 2016 at 19:40 | history | edited | harry lakins | CC BY-SA 3.0 |
added 2 characters in body
|
| Nov 27, 2016 at 19:26 | history | edited | harry lakins | CC BY-SA 3.0 |
added 62 characters in body
|
| Nov 27, 2016 at 19:24 | comment | added | harry lakins | Okay - so changing the weight ranges to -1,1 made the error output seem completely random - without any positive or negative correlation. | |
| Nov 27, 2016 at 19:17 | comment | added | harry lakins | Good question - so by default inputs can be between 0-255 - however, I divide all values by 255 to get a value < 1 for inputting. So, inputs are between 0 and 1. I will try the -1, 1 weight range now! Cheers. | |
| Nov 27, 2016 at 19:08 | comment | added | Dennis Soemers | @harrylakins What values can the nodes in your input layer take? If they're not in a small range such as [-1, 1] or [0, 1], you'll probably want to normalize them to such a range (for example, divide by maximum value to map it to the [0, 1] range). Also, you'll want to look into implementing the output layer as what's called a ''softmax'' layer. This ensures that your output values always sum up to one, which you want because you want the output to be probabilities. Finally, randomly initializing weights to [-1, 1] may be slightly better than [0, 1], but the difference should be small | |
| Nov 27, 2016 at 18:48 | history | edited | harry lakins | CC BY-SA 3.0 |
deleted 2 characters in body
|
| Nov 27, 2016 at 18:40 | comment | added | harry lakins |
I am not using tensor flow because i want the challenge - and the weights start as the result of random.uniform(0,1)
|
|
| Nov 27, 2016 at 18:33 | comment | added | Martin Thoma | How do you initialize the weights? | |
| Nov 27, 2016 at 18:32 | comment | added | Martin Thoma | Why aren't you using Tensorflow or Theano? | |
| Nov 27, 2016 at 18:30 | history | edited | harry lakins | CC BY-SA 3.0 |
added 2 characters in body
|
| Nov 27, 2016 at 16:47 | comment | added | harry lakins | 10,000 ...which should be enough right? but 5000 way through, the error gets to 0.2 and stays there (doesnt go beyond) | |
| Nov 27, 2016 at 16:33 | comment | added | erip | How many epochs are you training over? There's quite a lot of code being shown -- sorry if I missed it. | |
| Nov 27, 2016 at 16:31 | comment | added | erip | I agree with @DennisSoemers; 0.5 is quite high. | |
| Nov 27, 2016 at 16:29 | history | edited | harry lakins | CC BY-SA 3.0 |
deleted 2 characters in body
|
| Nov 27, 2016 at 13:35 | history | edited | harry lakins | CC BY-SA 3.0 |
added 2 characters in body
|
| Nov 27, 2016 at 12:16 | history | edited | harry lakins | CC BY-SA 3.0 |
deleted 2 characters in body
|
| Nov 27, 2016 at 12:09 | history | edited | harry lakins | CC BY-SA 3.0 |
deleted 2 characters in body
|
| Nov 26, 2016 at 20:02 | history | edited | harry lakins | CC BY-SA 3.0 |
added 2 characters in body
|
| Nov 26, 2016 at 19:44 | history | edited | harry lakins | CC BY-SA 3.0 |
deleted 2 characters in body
|
| Nov 26, 2016 at 19:26 | history | edited | harry lakins | CC BY-SA 3.0 |
added 2 characters in body
|
| Nov 26, 2016 at 19:22 | comment | added | harry lakins | When i try a much lower rate, all it seems to do is make all outputs >0.9 and the error total takes much much slower to get to anywhere < 0.5 | |
| Nov 26, 2016 at 18:45 | comment | added | Dennis Soemers | did you try modifying the learning rate? I wouldn't be surprised if 0.5 is way too high, and would expect something like 0.01 or 0.001 to work better | |
| Nov 26, 2016 at 16:34 | history | edited | harry lakins | CC BY-SA 3.0 |
deleted 2 characters in body
|
| Nov 26, 2016 at 15:24 | history | edited | harry lakins | CC BY-SA 3.0 |
deleted 2 characters in body
|
| Nov 26, 2016 at 15:05 | history | edited | harry lakins | CC BY-SA 3.0 |
added 2 characters in body
|
| Nov 26, 2016 at 14:03 | comment | added | harry lakins | thank you - ill be very impressed if someone manages to answer this one! | |
| Nov 26, 2016 at 14:02 | comment | added | user6035379 | Good luck getting an answer. I'll check back and see what kind of feedback you get. | |
| Nov 26, 2016 at 14:00 | comment | added | harry lakins | debugging a neural network is a nightmare! I have tried doing exactly that - and as far as i can tell, it does work. So i worry i am doing something conceptually wrong | |
| Nov 26, 2016 at 13:59 | history | edited | harry lakins | CC BY-SA 3.0 |
added 243 characters in body
|
| Nov 26, 2016 at 13:58 | comment | added | BlackBear | When I was debugging my neural net code, I simply built some simple examples and performed the computations by hand, then compared them with the results given by the code. Tedious, but it worked ;) | |
| Nov 26, 2016 at 13:54 | history | asked | harry lakins | CC BY-SA 3.0 |