FractionalRadix commented on Neural Networks: Algorithms, Applications, and Programming Techniques by James A. Freeman (Computation and Neural Systems Series)
Chapter 2 discusses the Adaline and the Madaline. The Adaline is the most basic neural network I've encountered. In fact, with a single element, you can hardly call it a network.