Project idea: two neural nets, one data set
Here's a project idea.
Train two neural networks on the same data, but with different initial random weights. They will come out differently, but probably equally good at whatever classification task they are learning to do.
Come up with a way to determine that they were trained on the same data.
I expect that this finding will shed some light on how this might eventually work in brains. Neurally, your belief that 'a' is th first letter of 'apple' probably looks a lot different from mine.
I'm not a neural net scientist, so I'll probably never run this study. If you do, let me know.
Train two neural networks on the same data, but with different initial random weights. They will come out differently, but probably equally good at whatever classification task they are learning to do.
Come up with a way to determine that they were trained on the same data.
I expect that this finding will shed some light on how this might eventually work in brains. Neurally, your belief that 'a' is th first letter of 'apple' probably looks a lot different from mine.
I'm not a neural net scientist, so I'll probably never run this study. If you do, let me know.
Comments
When I first learned of this work, I thought it was a potential analog to how our brains develop. I also thought that it might be a good explanation of individual differences in the ability to perform various cognitive tasks (i.e., language vs. math).
Even though the two neural networks might achieve the same level of performance on the classification task on which they were trained, it is possible that they would differ in how quickly they were able to reach criterion on a different type of task (of course, this is assuming neural networks vastly more complex than what we have now).