Distribution of Processes over multiple machines.

[JoshuaPortelance]

Also, make it so we can see some things to make the demo more interesting.

[Vealor]

@jwrg any leads?

[jwrg]

Research so far suggests that distribution could potentially follow one of two routes. The first involves distribution of the training of the network, while the second involves distributing the inputs and summing the outputs. In both cases, in a sense we create a network of neural networks, which owing to how a neural network operates, would be inherently centralized (as it would require synchronization and communication between network instances).

There is a decent amount of extant research on distributing the training of neural networks, particularly by Microsoft Research. This is IMO the best option for distributing our system and fits the above prototype of being a centralized network of networks.

Distributing the inputs is trivial, while sanely compiling pieces of output poses some difficulty. There is less research on this technique, but with a centralized network could be realized if the compilation of outputs into a single coherent output can be done consistently and potentially modelled using some formal methods or other. No reason though why we shouldn't be able to do both, considering NeurX's modular design in elixir.

The effort involved would be substantial, and I don't see any way we can get NeurX distributed by the demo date. We can however, discuss at length the potential avenues we have at our disposal for distributing the system. More research pending.