User Profile

The premise of this book is wonderful: translations between languages are always imperfect, and those slight mismatches give them power. In the world of this book, actual magical power. This is used to cause effects large and small in the industrial-revolution alternative-history setting of the book. The author, who herself was born in one country and grew up in another, and is a scholar of Chinese literature and language, brings that perspective to the book with force and with nuance.

I have mixed feelings about the world that the author creates; in many ways, it's very much like our world at the same point in our history. The same colonial empires, the same power structures, etc., just for slightly different reasons. I feel like this is not really fully taking advantage of the premise. On the other hand, key elements of the plot very much hinge on interactions with real historical figures and events - I suppose that only really works if the world has a history similar to our own. It also sets up a point of possible divergence from our history.

The story she tells is powerful, wide-ranging, and provocative. It touches on issues of race, class, and imperialism, and makes some of the power structures implicit in them explicit by giving them literal powers. It's a strong tale of struggle and empowerment, and certainly worth a read.

This book starts with the idea of accountability sinks: parts of a system that no one is responsible for, so there is no accountability for their actions. These are processes, algorithms, etc. that by design are not the responsibility of any one person, and which cannot be overridden so that no one can be held accountable for the outcomes that they produce. Davies makes the case that to some degree these are necessary in large systems, because we cannot really cope with systems in which there is personal responsibility for every decision. However, large systems can build so many accountability sinks that eventually no one is accountable for anything, and the system constantly produces outcomes that everyone involved claims not to want, and eventually break down.

From there, he goes into cybernetics, and the ways in which cybernetic theory describes the functioning and non-functioning of systems. One of the ways that accountability sinks make systems dysfunctional and unstable is that they break the flow of information: when you're unhappy with an outcome, there is, by design, no one you can express that unhappiness to that can do anything about it. At a small scale, this is tolerable and maybe even helpful to the health of the system. But when the accountability sinks are too numerous or in the wrong spots, the system becomes incapable of absorbing new information, eventually ceasing to respond to changing conditions, new needs, signs that it is failing, or growing dissatisfaction.

Eventually, all that is left is for those who are dissatisfied with the system's functioning is to pull the fire alarm (emergency brakes in Davies' examples). Davies' core thesis is that one of the problems with our modern large systems is that they are too full of accountability sinks, there are no nuanced ways of communicating what's going wrong, how it could be done better, what we actually want, etc. All we are left with is these single-bit messages where people signal 'something is going wrong' using the emergency stop mechanisms because that's all they can do.

And yes, this connects to AI, and so much more that's going wrong, but at this point you should probably just stop reading my review and read the book!

This book is an interesting look at the process that produced the FastLane grant administration system that was in use at NSF for over 20 years. The authors did an admirable job interviewing hundreds of people, getting details about how it was designed, developed, received by the user community, etc.

A few interesting tidbits that I learned: * The first review submitted via FastLane was a few months before the first purchase made on Amazon - so it was in fact quite the pioneering system (and then proceeded to change very little for what is essentially eternity in Internet time - I liked to call it "the finest in Web 0.2 Technology" and in fact that wasn't all the wrong) * The book credits FastLane as a major force moving academia towards PDF as a file format, since it required it; apparently a very large fraction of the problems with it had to do with the fact that in 2000, it was very hard to create PDFs and/or you had to have a licensed copy of Adobe Acrobat and/or Distiller. * There was some amount of contact between the team developing FastLane and the team developing Mosaic/Netscape (the first web browser), so in some ways FastLane actually helped shape the web itself. * A large chunk of it was originally written in Perl. Because of course it was. And that part was later re-written in Java. Because of course it was. * It was pretty interesting to see other sides of the system, such as the post-award administration bits and eJacket (the part used internally at NSF). The organization basically screwed up and developed the external-facing parts from a separate pot of money than the internal-facing parts, and as a result eJacket was 5-10 years behind and for a long time stuff submitted to FastLane got printed out on paper for internal circulation. * The book was far, far, too kind to the Interactive Panel System - yes, it did add some interactivity that used to be hard but it's implementation was and still is hot garbage that deserves to be cast into the fires of Mount Doom.

If you interacted a lot with FastLane or interact much with 'NSF, you might find this an entertaining read.