Reviews and Comments

Chapter 5 discusses the Boltzmann network - yes, that's Boltzmann from the law of entropy.

In previous chapters, artificial neural networks were considered to have entropy and energy. The question then arises - shouldn't they also have temperature?

From this, a link is made to simulating the annealing process in metallurgy: heating up and then cooling down in a controlled process.
The Boltzmann network learns and retrieves this way: the simulated temperature is set to a high value, then slowly lowered, with a number of training/retrieval steps taken at each temperature.
The idea is that this controlled process will help us avoid local minima. There does not seem to be a guarantee of finding a global minimum, but we do improve our chances.
Notable about the Boltzmann network is that it is more about probability than about exact steps. The simulator code at the end of the chapter shows this: nodes (called Processing Elements, or PE's) have a probability of being activated or de-activated. A random number generator then determines whether they are. It's also observed that the Boltzmann network is slow, even in comparison to other neural networks. The code presented shouldn't be too difficult for a computer nowadays, but must have taken a long time on a computer in the early 1990's. Despite this, or maybe even because of it, it's an interesting read.

Chapter 4 discusses the Bidirectional Associative Memory (BAM) and the Hopfield memory, which is presented as a variation of the BAM. It is noted that it was probably not designed as a variation.
The BAM is different from the networks discussed earlier in several ways.
First, it does not have training; the weights are initialized to the desired values from the start. Re-training would imply calculating an entirely new weight matrix.
Second, its inputs and outputs are either +1 or -1, rather than a real number.
When applying a value to either layer of the BAM, changes to the nodes are propgated until the network stabilizes. The eventual values on the "input" and " output" layers are then one of the vector pairs stored in the BAM.
(Note that "input layer" and "output layer" is a bit of a misnomer here; the book rightly prefers vectors named x and y).
This seems useful mostly for de-noising. The book gives an example where you store name - phone number pairs. If you do not quite recall the name, put it on the network and let it stabilize on the name that is stored in the network, together with the phone number. A proof is given that it always will stabilize, but the proof depends on the nodes having values of either +1 or -1. I'm not convinced the network would always stabilize if the nodes were real numbers instead.
Another interesting aspect is that when we store any pair of vectors in this network, we implicitly store its complement as well.
The book also notes that if vector pairs are too close to each other, "crosstalk" may occur: the network starts to confuse the vector pairs.
(I'm guessing this will also happen if we try to store more pairs than the network can handle).

The Hopfield memory has a single layer, where each node is connected to every other node. An example is then given where it is applied to the Traveling Salesperson Problem (TSP), noting that we will get a good solution but not necessarily the best solution. This is a reasonable expectation for NP-complete problems.

Like the previous chapters, chapter 4 concludes with an implementation of the network in pseudo-Pascal. This is one of the things I like about the book. Modern books would tell you to install a library such as PyTorch and use that. In the 90's, you were supposed to build a lot of things from scratch. Sometimes it's fun to get your hands dirty on the actual implementation.

RabbitMQ is a message broker, a piece of software that can be used to pass messages between systems - or between components of systems. By doing this via a message broker, we can accomplish loose coupling and asynchronous interaction. As an added bonus, we can see the messages passed between the systems in a uniform way, which helps diagnosis when an error occurs.

As the name suggests, the book does indeed go in depth. It explains not just how to use RabbitMQ, but goes so far as to explain the strucuture of the messages sent. It contains examples in Python, but one does not need to know Python to benefit from the book.

RabbitMQ implements the AMQP protocol; however, it also offers some functionality beyond that. The book points out what parts of RabbitMQ are AMQP, and what parts are extensions. This is useful if there's a chance you'll be using another implementation of AMQP at some point.

All in all, I've found this book useful for my work.