Friday, December 04, 2009

Project idea: two neural nets, one data set

Here's a project idea.

Train two neural networks on the same data, but with different initial random weights. They will come out differently, but probably equally good at whatever classification task they are learning to do.

Come up with a way to determine that they were trained on the same data.

I expect that this finding will shed some light on how this might eventually work in brains. Neurally, your belief that 'a' is th first letter of 'apple' probably looks a lot different from mine.

I'm not a neural net scientist, so I'll probably never run this study. If you do, let me know.

Bookmark and Share


Anonymous said...


Cindy said...

I always assumed that the different connections formed by those two networks would ultimately result in different strengths and weaknesses across a variety of tasks.
When I first learned of this work, I thought it was a potential analog to how our brains develop. I also thought that it might be a good explanation of individual differences in the ability to perform various cognitive tasks (i.e., language vs. math).
Even though the two neural networks might achieve the same level of performance on the classification task on which they were trained, it is possible that they would differ in how quickly they were able to reach criterion on a different type of task (of course, this is assuming neural networks vastly more complex than what we have now).