The Baffling Popularity of Randomized Linear Network Coding
As I write this chapter, I’m attending a computer science conference in San Jose, California. Earlier today, something interesting happened. I attended a session in which four different professors from four different universities presented their latest research. Surprisingly, all four presentations tackled the same narrow problem—information dissemination in networks—using the same narrow technique—randomized linear network coding. It was as if my research community woke up one morning and collectively and spontaneously decided to tackle the same esoteric problem.
This example of joint discovery surprised me, but it would not have surprised the science writer Steven Johnson. In his engaging 2010 book, Where Good Ideas Come From, Johnson explains that such “multiples” are frequent in the history of science.1 Consider the discovery of sunspots in 1611: As Johnson notes, four scientists, from four different countries, all identified the phenomenon during that same year. The first electrical battery? Invented twice in the mid-eighteenth century. Oxygen? Isolated independently in 1772 and 1774. In one study cited by Johnson, researchers from Columbia University found just shy of 150 different examples of prominent scientific breakthroughs made by multiple researchers at near the same time.
These examples of simultaneous discovery, though interesting, might seem tangential to our interest in career mission. I ask, however, that you stick with me, as the explanation for this phenomenon is the first link in a chain of logic that helped me decode what Pardis did differently than Sarah and Jane.
Big ideas, Johnson explained, are almost always discovered in the “adjacent possible,” a term borrowed from the complex-system biologist Stuart Kauffman, who used it to describe the spontaneous formation of complex chemical structures from simpler structures. Given a soup of chemical components sloshing and mixing together, noted Kauffman, lots of new chemicals will form. Not every new chemical, however, is equally likely. The new chemicals you’ll find are those that can be made by combining the structures already in the soup. That is, the new chemicals are in the space of the adjacent possible defined by the current structures.
When Johnson adopted the term, he shifted it from complex chemicals to cultural and scientific innovations. “We take the ideas we’ve inherited or that we’ve stumbled across, and we jigger them together into some new shape,” he explained. The next big ideas in any field are found right beyond the current cutting edge, in the adjacent space that contains the possible new combinations of existing ideas. The reason important discoveries often happen multiple times, therefore, is that they only become possible once they enter the adjacent possible, at which point anyone surveying this space—that is, those who are the current cutting edge—will notice the same innovations waiting to happen.
The isolation of oxygen as a component of air, to name one of Johnson’s examples of a multiple discovery, wasn’t possible until two things happened: First, scientists began to think about air as a substance containing elements, not just a void; and second, sensitive scales, a key tool in the needed experiments, became available. Once these two developments occurred, the isolation of oxygen became a big fat target in the newly defined adjacent possible—visible to anyone who happened to be looking in that direction. Two scientists—Carl Wilhelm Scheele and Joseph Priestley—were looking in this direction, and therefore both went on to conduct the necessary experiments independently but at nearly the same time.
The adjacent possible also explains my earlier example of four researchers tackling the same obscure problem with the same obscure technique at the conference I attended. The specific technique applied in this case—a technique called randomized linear network coding—came to the attention of the computer scientists I work with only over the last two years, as researchers who study a related topic began to apply it successfully to thorny problems. The scientists who ended up presenting papers on this technique at my conference had all noticed its potential around the same time. Put in Johnson’s terms, this technique redefined the cutting edge in my corner of the academic world, and therefore it also redefined the adjacent possible, and in this new configuration the information dissemination problem, like the discovery of oxygen many centuries earlier, suddenly loomed as a big target waiting to be tackled.
We like to think of innovation as striking us in a stunning eureka moment, where you all at once change the way people see the world, leaping far ahead of our current understanding. I’m arguing that in reality, innovation is more systematic. We grind away to expand the cutting edge, opening up new problems in the adjacent possible to tackle and therefore expand the cutting edge some more, opening up more new problems, and so on. “The truth,” Johnson explains, “is that technological (and scientific) advances rarely break out of the adjacent possible.”
As I mentioned, understanding the adjacent possible and its role in innovation is the first link in a chain of argument that explains how to identify a good career mission. In the next section, I’ll forge the second link, which connects the world of scientific breakthroughs to the world of work.