A systems software architecture for training distributed neural fuzzy neural and genetic networks and their relevant information models have been developed. Principles of on-line architecture building training managing and optimization guidelines are provided and extensively discussed. Qualitative comparisons of neural training strategies have been provided.