Book Review: Space Cadet

[Content Note: Intentionally exacting ethics, extensive quotation, casual discussion of nuclear warfare. Considerable spoilers for Space Cadet, but not in the way that you’d think.]


I’m probably going to regret trying to review Space Cadet because Heinlein is always about morality and writing about morality always frustrates me no end.

To be clear, it’s not morality that frustrates me, but writing about it, because I don’t have the time to dash off a three hundred page introduction to whatever idea it is that I’m trying to communicate. Learning to think in aesthetics was probably a mistake, because then you have to concretize and suddenly see that you’ve leapt over all the supporting framework.

If this seems a little dramatic for a slim YA novel, well, this book can be read on multiple levels. My initial reading, back in elementary school, mostly just took away the science fiction story of Matt Dodson joining the Patrol and his subsequent adventures as a cadet traveling the solar system.

Matt is a convenient character for this sort of story, because he has almost no defining features. He was raised in Iowa, North American Union, Terra. He struggles in mathematics but ultimately succeeds, enjoys playing space polo, studied Basic but not tensor calculus in high school, makes several friends and an enemy. Note that those friends have more features than him: “Tex” Jarman has a personality as big as his home state, Oscar from Venus tells us all about the Venerian culture and customs, Pete from Ganymede has an emotional episode of homesickness. Even the hate sink has a better-defined backstory. We’re intended to step easily into Matt’s shoes.

Heinlein, meanwhile, self-inserts into the various Patrol officers mentoring the young men as they attend Annapolis in space. The Patrol is not just a military organization, or a research organization, or a humanitarian organization. It’s all of these and more. Crafting boys into the sort of supermen who can keep the peace between the various nations of Terra and the inhabitants of Mars and Venus is no mean feat.

The first half of the novel is a standard Bildungsroman on the making of a spaceman. Consider this passage, during Matt’s orientation aboard the P.R.S. Randolph in geosynchronous orbit, where each cadet begins his education. Lieutenant Wong, Matt’s mentor, is explaining a cadet’s curriculum:

“Everything that can possibly be studied under hypno[sis] you will have to learn that way in order to leave time for the really important subjects.”

Matt nodded. “I see. Like astrogation.”

“No, no no! Not astrogation. A ten-year-old child could learn to pilot a spaceship if he had the talent for mathematics. That is kindergarten stuff, Dodson. The arts of space and warfare are the least part of your education. I know, from your tests, that you can soak up the math and physical sciences and technologies. Much more important is the world around you, the planets and their inhabitants—extraterrestrial biology, history, cultures, psychology, law and institutions, treaties and conventions, planetary ecologies, system ecology, interplanetary economics, applications of extraterritorialism, comparative religious customs, law of space, to mention a few.”

Matt was looking bug-eyed. “My gosh! How long does it take to learn all those things?”

“You’ll still be studying the day you retire. But even those subjects are not your education; they are simply the raw materials. Your real job is to learn how to think—and that means you must study several other subjects: epistemology, scientific methodology, semantics, structures of languages, patterns of ethics and morals, varieties of logics, motivational psychology, and so on. This school is based on the idea that a man who can think correctly will automatically behave morally—or what we call ‘morally.’ What is moral behavior for a Patrolman, Matt? You are called Matt, aren’t you? By your friends?”

“Yes, sir. Moral behavior for a Patrolman . . .”

“Yes, yes. Go on.”

“Well, I guess it means to do your duty, live up to your oath, that sort of thing.”

“Why should you?”

Matt kept quiet and looked stubborn.

“Why should you, when it may get you some messy way of dying? Never mind. Our prime purpose here is to see to it that you learn how your own mind works. If the result is a man who fits into the purposes of the Patrol because his own mind, when he knows how to use it, works that way—then fine! He is commissioned. If not, the we have to let him go.”

Matt remained silent until Wong finally said, “What’s eating on you, kid? Spill it.”

“Well—look here, sir. I’m perfectly willing to work hard to get my commission. But you make it sound like something beyond my control. First I have to study a lot of things I’ve never heard of. Then, when it’s all over, somebody decides my mind doesn’t work right. It seems to me that what this job calls for is a superman.”

“Like me.” Wong chuckled and flexed his arms. “Maybe so, Matt, but there aren’t any supermen, so we’ll have to do the best we can with young squirts like you. Come, now, let’s make up the list of spools you’ll need.”

Thus begins Matt’s theoretical education as a Patrolman. The process isn’t easy for him, and he struggles. That aspect of the story is far more relatable to me now that when I read this book as a kid, because I’ve been there. Honestly, if I could make 2013!me read a particular book, I’d probably ask myself to reread Space Cadet. It might just have bent the trajectory of my life a different direction.

Matt, too, struggles with trajectories—he’s so frustrated by the coursework in astrogation that he asks Lieutenant Wong for a transfer to the space marines. Wong refuses, saying that Matt is too far removed from the appropriate mindset:

“People tend to fall into three psychological types, all differently motivated. There is the type, motivated by economic factors, money . . . and there is the type motivated by ‘face,’ or pride. This type is a spender, fighter, boaster, lover, sportsman, gambler; he has a will to power and an itch for glory. And there is the professional type, which claims to follow a code of ethics rather than simply seeking money or glory—priests and ministers, teachers, scientists, medical men, some artists and writers. The idea is that such a man believes that he is devoting his life to some purpose more important than his individual self.

[. . .]

“The Patrol is meant to be made up exclusively of the professional type. In the space marines, every single man jack, from the generals to the privates, is or should be the sort who lives by pride and glory.”

“Oh . . .”

Wong waited for it to sink in. “You can see it in the very uniforms; the Patrol wears the plainest of uniforms, the marines wear the gaudiest possible. In the Patrol all emphasis is on the oath, the responsibility to humanity. In the space marines the emphasis is on pride in their corps and its glorious history, loyalty to comrades, the ancient virtues of the soldier. I am not disparaging the marine when I say that he does not care a tinker’s damn for the political institutions of the Solar System; he cares only for his organization.

“But it’s not your style, Matt. I know more about you than you do yourself, because I have studied the results of your psychological tests. You will never make a marine.”

Rejected by Lieutenant Wong, Matt returns to astrogation, planning secretly to not return from his first leave.

The next chapter opens waiting for the rocket back to P.R.S. Randolph, wondering just when he changed his mind. The narrative alternates between the rocket flight and Matt’s vacation, illustrating the ways in which he is no longer a civilian:

Great-aunt Dora was the current family matriarch. She had been a very active woman, busy with church and social work. Now she was bed-fast and had been for three years. Matt called on her because his family obviously expected it. “She often complains to me that you don’t write to her, Matt, and—”

“But, Mother, I don’t have time to write to everyone!”

“Yes, yes, but she’s proud of you, Matt. She’ll want to ask you a thousand questions about everything. Be sure to wear your uniform—she’ll expect it.”

Aunt Dora had not asked a thousand questions; she had asked just one—why had he waited so long to come see her? Thereafter Matt found himself being informed, in detail, of the shortcomings of the new pastor, the marriage chances of several female relatives and connections, and the states of health of several older women, many of them unknown to him, including the details of operations and post-operative developments.

I’m glad I’ve never had that experience with my older relatives, though when Dad talks about his coworkers….

Yes, maybe that was it—it might have been the visit to Aunt Dora that convinced him that he was not ready to resign and remain in Des Moines. It could not have been Marianne.

Marianne was the girl who had made him promise to write regularly—and, in fact, he had, more regularly than she. But he had let her know that he was coming home and she had organized a picnic to welcome him back. It had been jolly. Matt had renewed old acquaintances and had enjoyed a certain amount of hero worship from the girls present. There had been a young man there, three or four years older than Matt, who seemed unattached. Gradually it dawned on Matt that Marianne treated the newcomer as her property.

It had not worried him. Marianne was the sort of girl who never would get clearly fixed in her mind the distinction between a planet and a star. He had not noticed this before, but it and similar matters had come up on the one date he had had alone with her.

And she had referred to his uniform as “cute.”

He began to understand, from Marianne, why most Patrol officers do not marry until their mid-thirties, after retirement.

This passage, and several like it, were why I decided to reread Space Cadet after all these years. The disconnect between specialist and layman grows too large and it becomes impossible to talk meaningfully about your work. So far, I’ve managed to keep Mom and Dad up to speed, but we’ll see how long that lasts.

Matt is in a much worse state, trying to describe missile maintenance to his parents, who neither understand orbital mechanics, nucleonics, nor the political motivations of the Patrol.

Nuclear weapons are kept in polar orbits, he explains, so that the entire planet is covered by the Patrol’s watchful eye. They are regularly serviced by ships—physically caught by a cadet, disarmed, and reeled in for inspection and repositioning. Matt casual mentions that J-3 will be passing over Des Moines in a few minutes, which gives his mother a fit of anxiety. “What if it should fall?” she demands.

Objects in orbit don’t fall, of course, as Matt explains—they would have to instantaneously lose 7,800 m/s of velocity to drop straight down. If the Patrol needed to nuke Des Moines that night, they would use a missile requiring a more moderate change of trajectory, like I-2 or H-1.

This doesn’t comfort her.

Matt’s father tries to argue that the Patrol would never bomb the North American Union, because the majority of Patrol officers are from North America. Matt refuses to commit, insisting later that the Patrol absolutely would. But he has doubts.

For the first few weeks after leave, Matt was too busy to fret. He had to get back into the treadmill, with more studying to do and less time to do it in. He was on the watch list for cadet officer of the watch now, and had more laboratory periods in electronics and nucleonics as well. Besides this he shared with the other oldsters the responsibility for bringing up the youngster cadets. Before leave his evenings had usually been free for study, now he coached youngsters in astrogation three nights a week.

He was beginning to think that he would have to give up space polo, when he found himself elected captain of [the deck’s] team. Then he was busier than ever. He hardly thought about abstract problems until his next session with Lieutenant Wong.

“Good afternoon,” his coach greeted him. “How’s your class in astrogation?”

“Oh, that—It seems funny to be teaching it instead of flunking it.”

“That’s why you’re stuck with it—you still remember what it was that used to stump you and why. How about atomics?”

“Well . . . I suppose I’ll get by, but I’ll never be an Einstein.”

“I’d be amazed if you were. How are you getting along otherwise?” Wong waited.

“All right, I guess. Do you know, Mr. Wong—when I went on leave I didn’t intend to come back.”

“I’d rather thought so. That space-marines notion was just your way of dodging around, trying to avoid your real problem.”

“Oh. Say, Mr. Wong—tell me straight. Are you a regular Patrol officer, or a psychiatrist?”

Wong almost grinned. “I’m a regular Patrol officer, Matt, but I’ve had the special training required for this job.”

“Uh, I see. What was it I was running away from?”

“I don’t know. You tell me.”

“I don’t know where to start.”

“Tell me about your leave, then. We’ve got all afternoon.”

“Yes, sir.” Matt meandered along, telling as much as he could remember. “So you see,” he concluded, “it was a lot of little things. I was home—but I was a stranger. We didn’t talk the same language.”

Wong chuckled. “I’m not laughing at you,” he apologized. “It isn’t funny. We all go through it—the discovery that there’s no way to go back. It’s part of growing up—but with spacemen it’s an especially acute and savage process.”

Matt nodded. “I’d already gotten that through my thick head. Whatever happens I won’t go back—not to stay. I might go into the merchant service, but I’ll stay in space.”

“You’re not likely to flunk out at this stage, Matt.”

“Maybe not, but I don’t know yet that the Patrol is the place for me. That’s what bothers me.”

“Well . . . can you tell me about it?”

Matt tried. He related the conversation with his father and his mother that had gotten them all upset. “It’s this: if it comes to a showdown, I’m expected to bomb my own hometown. I’m not sure it’s in me to do it. Maybe I don’t belong here.”

“Not likely to come up, Matt. Your father was right there.”

“That’s not the point. If a Patrol officer is loyal to his oath only when it’s no skin off his own nose, the whole system breaks down.”

Wong waited before replying. “If the prospect of bombing your own town, your own family, didn’t worry you, I’d have you out of this ship within the hour—you’d be an utterly dangerous man. The Patrol doesn’t expect a man to have godlike perfection. Since men are imperfect, the Patrol works on the principle of calculated risk. The chance of a threat to the System coming from your own hometown in your lifetime is slight; the chance that you might be called upon to carry out the attack is equally slight…But if you did hit the jackpot, your commanding officer would probably lock you up in your room rather than take a chance on you.”

Matt still looked troubled. “Not satisfied?” Wong went on. “Matt, you are suffering from a disease of youth—you expect moral problems to have nice, neat, black-and-white answers. Suppose you relax and let me worry about whether or not you have what it takes. Oh, some day you’ll be caught in a squeeze with no one around to tell you the right answer. But I have to decide whether or not you can get the right answer when the problem comes along—and I don’t even know what your problem will be! How would you like to be in my boots?”

Matt grinned sheepishly. “I wouldn’t like it.

From thereon out, it’s a fairly standard science fiction story. If the last hundred page feel like an entirely different novel, well, the earlier drafts went in a rather different direction. In the final version, however, Matt is assigned to a ship, continuing his education while on search-and-assist in the asteroid belt, before being sent to Venus. There, Matt, Tex, and Oscar find themselves stranded, their commanding officer incapacitated, and must keep the peace with the local Venerians while rescuing themselves—exactly the sort of experience Lieutenant Wong was preparing Matt for. If only all college guidance counselors had the time and training to take such interest in their students’ psychological development!

What draws me to Space Cadet again after so many years is that it is not just a fun adventure in space (though that certainly doesn’t hurt). It’s a vision of how to live as human beings.

This story was written immediately after the war, copyright 1948. The specter of fascism still hung over the western world, that Russia would be our geopolitical enemy for next forty years was still largely unthinkable.

Heinlein was looking ahead to a world of nuclear weapons and nuclear war. Remember, Uncle Joe still didn’t have the bomb—if we’d acted quickly, the entire planet could have been a democracy (or a dictatorship). Even before America entered the war, Heinlein was thinking about the threat that nuclear weapons posed to world peace and world freedom.

In various forms, the Patrol was his fictional attempt to answer this problem. A quasi-military organization, with unlimited funds and unlimited firepower at its disposal, and each officer committed to the safety of every nation but his own. Lieutenant Wong is no accident: the Patrol’s multicultural character is made clear throughout the book. In a classic Heinlein twist, only after the boys are stranded on Venus do we learn that one of their commanders was of African descent.

(Those who mistakenly believe Sixth Column accurately represent Heinlein’s views on race should consider that he wrote this, for kids, at the same time.)

A decade before the beatniks, we’re told to stand up tall and proud in the shadow of the mushroom cloud and conduct ourselves as men.

Let’s do the responsible thing here and quote from William Patterson’s biography:

An incident witnessed on a family outing in Swope Park in 1912 stayed with [Heinlein] for the rest of his life. He would take it out of memory and turn it over in his mind again and again, examining it with wonder:

A young couple was walking along a set of railroad tracks that cut through the park in those days when the woman got her heel caught in a switch—a nuisance, until they heard a train whistle approaching at speed. Another younger man—the newspapers later said he was a tramp—stopped to help them get free. As the train bore down on them, the husband and the tramp struggled to get the woman free and were struck, all of them. The wife and the tramp were killed instantly, the husband seriously injured.

Why did he do it? Not the husband, who was, after all, simply (simply!) doing his duty by his wife—but the tramp, who had no personal stake in their welfare and could have jumped aside, even at the last minute, to save himself. Why did he do it? wondered little Bobby and then Adolescent Bobby—and so, repeatedly, did Midshipman Bob and politician Bob and adult Robert, understanding a bit more, a bit differently, every time he looked at it.

An artist works in images and articulates images even when he can’t necessarily articulate the meaning. This incident became a core image for [Heinlein], one that showed him in a way beyond words what it means to be a human being. At the end he still could not articulate it. All he could say about it was: “This is how a man dies. This is how a man lives!” And that was enough.

This is what I love about Space Cadet, what I love about the Patrol, and what I love about Heinlein.

Maybe thinking with aesthetics isn’t so bad after all.

Wrong, Wrong, Wrong


[You’ll have to take my word that this post was planned before Megan left a comment to much the same effect on Facebook. I still wrote it, because I love sharing space trivia, but because it took several more days to finish, my summer schlamperei cannot be questioned.]

Last week I discussed a mistaken explanation in one of my engineering textbooks. The specific explanation was wrong, but there’s two other issues with that innocuous caption in chapter one.

Hibbeler attributes a NASA image to a PhotoShop artist on Shutterstock. Now there’s nothing wrong with getting images from there. I find them a bit overthetop personally, but when has that ever been an impediment to the authors of textbooks on serious topics? Never, that’s when.

But why would you go through a stock image company when NASA images are public domain? It seems unnecessary, strongly suggests that author or graphic artist did a quick Google search, rather than having a clear image in mind. That brings us to the other problem.

This is a very specific, very recognizable image of Bruce McCandless operating the Manned Maneuvering Unit during STS-41B. You even see the same cloud patterns on the NASA website. And the last I heard, Bruce McCandless is a man.

Misgendering McCandless was obviously done for inclusion, but would it have been so hard to get a picture of a woman astronaut? It certainly would go further toward fixing the image of female spacetravelers in the reader’s mind.

Unfortunately for Hibbeler, there aren’t any pictures of women EVAing without a tether because no woman has ever operated the MMU. If that seems problematic, consider that the MMU was an experimental piece of technology that was really too dangerous to use regularly. Only six astronauts ever wore them, the last in November of 19841. After the Challenger disaster, NASA cancelled all the potential flights which might use the MMU and unofficially retired them from use.

But, we don’t really need a picture of a female astronaut operating untethered, because Hibbeler got it wrong. Astronauts aren’t weightless in orbit because they’re far removed from Earth’s gravity; they’re weightless because they’re falling around Earth at the same rate as their spacecraft and everything in it2. We can use an interior photo, this classic picture of Mae Jemison aboard Endeavour being a logical choice:


Now was that really so hard?

1To be explicit about it: including that final mission, only four American women had reached orbit, and only one had performed an EVA. The Soviet Union, meanwhile, only ever sent up two female cosmonauts. Tereshkova, in 1963, was basically a publicity stunt: she never returned to space and the Soviets resumed all-male crews until Svetlana Savitskaya launched in 1982, by which point it was clear that American women were going to space to stay. Nevertheless, Savitskaya beat Kathryn Sullivan to the first female spacewalk by 78 days.

2If we wanted pictures where the inverse squared law has kicked in, our selection is limited to the deep space EVAs during Apollo 15, 16, and 17. Performed during the coast back to Earth, terrestrial gravity had lessened, though their weightlessness was still attributable to free fall. However, our selection is all-male: Al Worden, Ken Mattingly, and Ron Evans.

Beware Scientific Metaphors

I’m about a quarter finished with Isabel Paterson’s The God of the Machine, which I’m finally reading after several years of intending to. So far, it’s been both pleasurable and interesting. My main reservation, however, has been an extended metaphor which both illustrates the central idea and potentially undermines it.

Paterson develops a notion of energy to describe the synthesis of material resources, cultural virtue, and human capital which results in creativity and production. As metaphors go, this is not a bad one. That said, my engineering background gives me cause for concern. It isn’t clear that Paterson has a clear understanding of energy as a scientific concept, and her analogy may suffer for it. Complicating matters, she sometimes also phrases “energy” as if it were electricity, which is another can of worms in and of itself.

Mechanical energy behaves oddly enough for human purposes, being generally conserved between gravitational potential and kinetic energy, and dissipated through friction and heating. It emphatically does not spring ex-nihilo into cars and trains. Coal and oil have chemical potential energy, which is released as thermal energy, then converted into kinetic energy and thus motion to drive an internal combustion engine.

Electrical energy is even weirder. It’s been enough years since I finished my physics that I won’t attempt to explain the workings in detail. (My electronics class this spring bypassed scientific basis almost entirely.) Suffice to say that the analogy of water moving through a pipe is not adequate beyond the basics.

Atomic energy, the most potent source yet harnessed, does create energy, but at a cost. A nuclear generating station physically destroys a small part of a uranium atom, converting it via Einstein’s famous relation to useful energy. But more on that in later posts.

I won’t say that the “energy” metaphor is strictly-speaking wrong, because I haven’t done the work of dissecting it in detail. Paterson was a journalist and writer, but she was also self-educated, and therefore we cannot easily assess the scope and accuracy of her knowledge of such phenomena. But I don’t think it matters: even if the metaphor is faulty, the concept which it tries to communicate seems, on the face, quite plausible without grounding in the physical sciences.

I bring this up now, well before I’ve finished the book, because I’ve seen much worse analogies from writers with much less excuse to make them. The God of the Machine was published in 1943. Authors today have a cornucopia of factual knowledge at their fingertips and still screw it up. For instance, take this caption from my statics textbook:


Hibbeler, R. C., Engineering Mechanics: Statics & Dynamics, 14th ed., Pearson Prentice Hall, Hoboken, 2016.

There is no excuse for a tenured professor (or, more plausibly, his graduate students) to screw this up. The correct equation is on that very page and they couldn’t even be bothered to run the numbers and see that, no, you’re not significantly lighter in low Earth orbit. From my perspective, such a blatant error is unconscionable in the opening pages of a professional text.

Now that isn’t exactly a metaphor, but it illustrates the risks of discussing fields nominally close to your own which nevertheless you know very little about. Imagine the danger of using metaphors from totally different fields you’ve never formally studied.

So, I would advise writers to be sparing with scientific metaphors. If you can learn the science correctly, that’s great: you’ll construct metaphors that are both interesting and accurate. But as we’ve seen above, even PhDs make stupid mistakes. Err on the side of caution.

Book Review: The Signal and the Noise

Supposedly Nate Silver’s credibility took a major hit last November, which will no doubt discourage many potential readers of his book. This interpretation is wrong, but palatable, because the sorts of commentators who would come to such conclusions shouldn’t be trusted with it. This book is about how to be more intelligent when making predictions and be wrong less often. Such an attitude is not common—most “predictions” are political pot-shots or, as discussed previously, avaricious attempts to put the cart before the horse.

Let’s begin with a discussion of a few major tips. Most of these things should be taught in high school civics (how can you responsibly vote without a concept of base rates?!), but aren’t. Perhaps the most important thing is to limit the number of predictions made, so you can easily come back and score them. Calibration is recommended—nine out of ten predictions made with 90% confidence should come true.

Political pundits are terrible about these sorts of things. Meteorologists are actually great at it. Now your local weatherman is regularly wrong, but the National Weather Service makes almost perfectly calibrated forecasts1. This is, in part, because their models are under constant refinement, always seeking more accuracy. And it pays off: NWS predictions have improved drastically over the last few decades, due to improved models, more data collection, and faster computers. But more on that later.

Local meteorologists, on the other hand, are incentivized to make outlandish forecasts which drive viewership (and erode trust in their profession). One might see this as evidence that public entities make better predictions than private ones, but we quickly see that that is no panacea when we turn to seismology and epidemiology.

Part of the problem, in those fields, is that government and university researchers are under considerable pressure from their employers to develop new models which will enable them to predict disasters. This is a reasonable enough desire, but a desire alone does not a solution make. We can quite easily make statistical statements about approximately how frequently certain locations will experience earthquakes, for instance. But attempts beyond a simple logarithmic regression have so far been fruitless, not just failing to predict major earthquakes but specifically prediction that some of the most destructive earthquakes in recent memory would not occur.

Silver’s primary case study in this comes from the planning for Fukushima Daiichi Nuclear Power Plant. When engineers were designing it in the 1960s, it was necessary to extrapolate what sort of earthquake loads it might need to withstand. Fortunately, the sample size of the largest earthquakes is necessarily low. Unfortunately, there was a small dogleg in the data, an oh-so-tempting suggestion that the frequency of extremely large earthquakes was exceedingly low. The standard Gutenberg-Richter model suggests that a 9.0-magnitude earthquake would occur in the area about once every 300 years; the engineers’ adaptation suggested every 13,000. They constructed fantastical rationalizations for their model and a power station able to withstand 8.6. In March of 2011 a 9.0-magnitude earthquake hit the coast of Japan and triggered a tsunami. The rest, as they say, is history.

The problem in seismology comes from overfitting. It is easy, in the absence of hard knowledge, to underestimate the amount of noise in a dataset and end up constructing a model which predicts random outliers. Those data points don’t represent the underlying reality; rather, they are caused by influences outside the particular thing you’re wishing to study (including the imprecision of your instruments).

And it can take awhile to realize that this is the case, if the model is partially correct or if the particular outlier doesn’t appear frequently. An example would be the model developed by Professor David Bowman at California State University-Fullerton in the mid-2000s, which identified high-risk areas, some of which then experienced earthquakes. But the model also indicated that an area which soon thereafter experienced an 8.5 was particularly low-risk. Dr. Bowman had the humility to retire the model and admit to its faults. Many predictors aren’t so honest.

On the other hand, we see overly cautious models. For instance, in January of 1976, Private David Lewis of the US Army died at Fort Dix of H1N1, the same flu virus which caused the Spanish Influenza of 1918. The flu always occurs at military bases in January, after soldiers have been spread across the country for Christmas and New Year’s. The Spanish Influenza had also first cropped up at a military base, and this unexpected reappearance terrified the Center for Disease Control. Many feared an even worse epidemic. President Ford asked Congress to authorize a massive vaccination program at public expense, which passed overwhelmingly.

The epidemic never materialized. No other cases of H1N1 were confirmed anywhere in the country and the normal flu strain which did appear was less intense than usual. We still have no idea how Private Lewis contracted the deadly disease.

Alarmism, however, broke public confidence in government predictions generally and on vaccines particularly. The vaccination rate fell precipitously in the following years, opening the way to more epidemics later on.

Traditionally, this category of error was known as crying wolf. Modern writers have forgotten it and have to be reminded to not do that. Journalists and politicians make dozens if not hundreds of “predictions” each year, few if any of which are scored, in no small part because most of them turn out wrong or even incoherent.

Sadly, the pursuit of truth and popularity are uncorrelated at best. As Mr. Silver has learned, striving for accuracy and against premature conclusions is a great way to get yourself berated2. Forecasting is not the field for those seeking societal validation. If that’s your goal, skipping this book is far better than trying to balance its lessons and the public’s whim.

But let’s suppose you do want to be right. If you do, then this book can help you in that quest, though it is hardly a comprehensive text. You’ll need to study statistics, history, economics, decision theory, differential equations, and plenty more. Forecasting could be an education in its own right (though regrettably is not). The layman, however, can improve vastly by just touching on these subjects.

First and foremost is an understanding of probability, specifically Bayesian statistics. Silver has the courage to show us actual equations, which is more than can be said for many science writers. Do read this chapter.

Steal an example from another book, suppose two taxi companies operate in a particular region, based on color. Blue Taxi has a larger market share. If you think you see a Green Taxi, there’s a small chance that it’s really Blue and you’re mistaken (and a smaller chance if you see Blue, it’s really Green). The market share is the base rate, and you should adjust up or down based on the reasons you might feel uncertain. For instance, if the lighting is poor and you’re far away, your confidence should be lower that if you’re close by at mid-day. Try thinking up a few confounders of your own.

To better develop your Bayesian probability estimate of a given scenario, you need to assess what information you possess and what information you don’t possess. These will be your Known Knowns and Known Unknowns. The final category is Unknown Unknowns, the thing you aren’t aware are even a problem. A big part of rationality is trying to consider previously ignored dangers and trying to mitigate risk from the unforeseen.

This is much easier to do ex post facto. By that point, the signal you need to consider stands out against hundreds you can neglect. Beforehand, though, it’s difficult to determine which is the most important. Often, you’re not even measuring the relevant quantity directly but rather secondary and tertiary effects. Positive interference can create a signal where none exists. Negative interference can reduce clear trends to background noise. There’s a reason signal processing pays so well for electrical engineers.

The applications range from predicting terrorist attacks to not losing your shirt gambling. An entire chapter discusses the Poker Bubble and how stupid players make the game profitable for the much smaller pool of cautious ones. In addition to discussing the mechanics and economics of the gambling, I got a decent explanation of how poker is played. Certainly interesting.

Another chapter tells the story of how Deep Blue beat Gary Kasparov. Entire books have been written on the subject, but Silver gives a good overview of the final tournament and what makes computers so powerful in the first place.

Computers aren’t actually very smart. Their strength comes from solving linear equations very, very quickly. They don’t make the kinds of arithmetic mistakes which humans make, especially when the iterations run into the millions. Chess is a linear game, however, so it was really a matter of time until algorithms could beat humans. There’s certainly a larger layer of complexity and strategy than many simpler games, but it doesn’t take a particularly unique intelligence to look ahead and avoid making mistakes in the heat of the moment.

Furthermore, the stating position of chess is always the same. This is not the case for many other linear systems, let alone nonlinear ones. Nonlinear systems exhibit extreme sensitivity to initial conditions; the weather a classical example. The chapter on meteorology discusses this in detail—we have very good models of how the atmosphere behaves, but because we don’t know every property at every location, we’re stuck making inferences about the air in-between sampling points. Add to this finite computing power, and the NWS can only (only!) predict large-scale weather systems with extreme accuracy a few days ahead.

With more sampling points, more computing capacity, or more time, we could get better predictions, but all of these factors play off one another. This dilemma arises throughout prediction. More research will allow for more accurate results but delays your publication data. (This assumes that the data you need is even available: frequently, it isn’t3.)

Producing useful predictions is not about having the best data or the most computing power (though they certainly help). It is primarily about constraining your anticipation to what the evidence actually implies. Nate Silver lays out several techniques for pursuing this goal, with examples. It’s a good introduction for us laymen; experienced statisticians will probably find little they didn’t already know.

I would not recommend this book, however, unless you’re willing to do the work. Prediction is a difficult skill to master, and those without the humility to accept their inexperience can get into a lot of trouble. Should you want to test your abilities, try doing calibrated predictions and see how accurate you are. Julia Galef has a number of mostly harmless suggestions for trying this out.

If you are serious, however, The Signal and the Noise offers a quality primer on several important rationality techniques, and a good deal of information about a variety of other topics. I found it an enjoyable read and hope Nate Silver writes more books in the future.


1Major aggregators like the Weather Channel and AccuWeather tend to take the NWS predictions and paste an additional layer of modelling on top of it, for better or for worse.

2In the week before the 2016 election, several liberal commentators accused Mr. Silver of throwing the nation into unwarranted fear for only having Hillary Clinton’s odds of winning at ~70%. As it turns out, his model was one of the most balanced of mainstream predictions, yet everyone then acted as if he had reason to be ashamed for getting it wrong.

3The data may be concealed in confidential documents, nominally available but out of sight, or sitting right under your nose. Most often, however, it’s hiding in the noise. Economic forecasts suffer from this last problem. There’s econometric data everywhere, but basically no one has found more than rudimentary ways to make predictions with it. Perverse incentives complicate matters for private sector analysts, who often then ignore the few semi-reliable indicators we’ve got.