My second book, The Big Switch: Rewiring the World, from Edison to Google, celebrates its tenth birthday this year. The book, which came out in January 2008, heralds the coming of the cloud and speculates on its consequences. It’s hard to imagine now, but in 2008 cloud computing was a new and largely unproven concept, and the common wisdom was that it wouldn’t work. Software programs running in centralized server farms and delivered over the internet to users would be too slow and balky, it was thought, to displace the programs running on hard drives inside personal computers or on servers in data centers owned by individual companies. The naysayers were wrong. The technical barriers fell, network latency evaporated, and in short order computing went from being a decentralized resource to being a centralized one — a utility, essentially. The computer scientists, engineers, and programmers who made this monumental technical shift possible still haven’t received their due, and they probably never will. The real work went on behind the scenes, anonymously, in companies like Google, Salesforce.com, Amazon Web Services, Akamai, and Facebook, among many others.

The Big Switch has two parts. The first, “One Machine,” draws a parallel between the building of the electric grid a hundred years ago and the building of the cloud today. In both cases, a decentralized resource essential to society (power, data processing) was centralized through the construction of a distribution network (electric grid, internet) and central plants (generation stations, server farms). The stories of the electric grid and the computing grid are both stories of technical ingenuity and fearlessness. The book’s second part, “Living in the Cloud,” is darker. In fact, it was during the course of writing it that my view of the future of computing changed. I began The Big Switch believing that the new computing grid would democratize the use of computing power even as it centralized the machinery of data processing. That is, after all, what the electric grid did. By industrializing the generation and distribution of electricity, it made power a cheap resource that everyone could use simply by sticking a plug into a wall socket. But data is fundamentally different from electric current, I belatedly realized, and centralizing the provision of computing would also mean centralizing control over information. The owners of the server farms would not be faceless utilities; they would be our overseers.

Here’s an excerpt from “The Inventor and His Clerk,” a chapter in the first half of The Big Switch:

Thomas Edison was tired. It was the summer of 1878, and he‘d just spent a grueling year perfecting and then promoting his most dazzling invention yet, the tinfoil phonograph. He needed a break from the round-the-clock bustle of his Menlo Park laboratory, a chance to clear his mind before embarking on some great new technological adventure. When a group of his friends invited him to join them on a leisurely camping and hunting tour of the American West, he quickly agreed. The trip began in Rawlins, Wyoming, where the party viewed an eclipse of the sun, and then continued westward through Utah and Nevada, into Yosemite Valley, and on to San Francisco.

While traveling through the Rockies, Edison visited a mining site by the side of the Platte River. Seeing a crew of workers struggling with manual drills, he turned to a companion and remarked, “Why cannot the power of yonder river be transmitted to these men by electricity?” It was an audacious thought—electricity had yet to be harnessed on anything but the smallest scale—but for Edison audacity was synonymous with inspiration. By the time he returned east in the fall, he was consumed with the idea of supplying electricity over a network from a central generating station. His interest no longer lay in powering the drills of work crews in the wilderness, however. He wanted to illuminate entire cities. He rushed to set up the Edison Electric Light Company to fund the project and, on October 20, he announced to the press that he would soon be providing electricity to the homes and offices of New York City. Having made the grand promise, all he and his Menlo Park team had to do was figure out how to fulfill it.

Unlike lesser inventors, Edison didn‘t just create individual products; he created entire systems. He first imagined the whole, then he built the necessary pieces, making sure they all fit together seamlessly. “It was not only necessary that the lamps should give light and the dynamos generate current,” he would later write about his plan for supplying electricity as a utility, “but the lamps must be adapted to the current of the dynamos, and the dynamos must be constructed to give the character of current required by the lamps, and likewise all parts of the system must be constructed with reference to all other parts, since, in one sense, all the parts form one machine.” Fortunately for Edison, he had a good model at hand. Urban gaslight systems, invented at the start of the century, had been set up in many cities to bring natural gas from a central gasworks into buildings to be used as fuel for lamps. Light, having been produced by simple candles and oil lamps for centuries, had already become a centralized utility. Edison‘s challenge was to replace the gaslight systems with electric ones.

Electricity had, in theory, many advantages over gas as a source of lighting. It was easier to control, and because it provided illumination without a flame it was cleaner and safer to use. Gaslight by comparison was dangerous and messy. It sucked the oxygen out of rooms, gave off toxic fumes, blackened walls and soiled curtains, heated the air, and had an unnerving tendency to cause large and deadly explosions. While gaslight was originally “celebrated as cleanliness and purity incarnate,” Wolfgang Schivelbusch reports in Disenchanted Night, his history of lighting systems, its shortcomings became more apparent as it came to be more broadly used. People began to consider it “dirty and unhygienic”—a necessary evil. Edison himself dismissed gaslight as “barbarous and wasteful.” He called it “a light for the dark ages.”

Despite the growing discontent with gas lamps, technological constraints limited the use of electricity for lighting at the time Edison began his experiments. For one thing, the modern incandescent lightbulb had yet to be invented. The only viable electric light was the arc lamp, which worked by sending a naked current across a gap between two charged iron rods. Arc lamps burned with such intense brightness and heat that you couldn‘t put them inside rooms or most other enclosed spaces. They were restricted to large public areas. For another thing, there was no way to supply electricity from a central facility. Every arc lamp required its own battery. “Like the candle and the oil lamp,” Schivelbusch explains, “arc lighting was governed by the pre-industrial principle of a self-sufficient supply.” However bad gaslight might be, electric light was no alternative.

To build his “one machine,” therefore, Edison had to pursue technological breakthroughs in every major component of the system. He had to pioneer a way to produce electricity efficiently in large quantities, a way to transmit the current safely to homes and offices, a way to measure each customer‘s use of the current, and, finally, a way to turn the current into controllable, reliable light suitable for normal living spaces. And he had to make sure that he could sell electric light at the same price as gaslight and still turn a profit.

It was a daunting challenge, but he and his Menlo Park associates managed to pull it off with remarkable speed. Within two years, they had developed all the critical components of the system. They had invented the renowned Edison lightbulb, sealing a thin copper filament inside a small glass vacuum to create, as one reporter poetically put it, “a little globe of sunshine, a veritable Aladdin‘s lamp.” They had designed a powerful new dynamo that was four times bigger than its largest precursor. (They named their creation the Jumbo, after a popular circus elephant of the time.) They had perfected a parallel circuit that would allow many bulbs to operate independently, with separate controls, on a single wire. And they had created a meter that would keep track of how much electricity a customer used. In 1881, Edison traveled to Paris to display a small working model of his system at the International Exposition of Electricity, held in the Palais de l‘Industrie on the Champs-Elysées. He also unveiled blueprints for the world‘s first central generating station, which he announced he would construct in two warehouses on Pearl Street in lower Manhattan.

The plans for the Pearl Street station were ambitious. Four large coal-fired boilers would create the steam pressure to power six 125-horsepower steam engines, which in turn would drive six of Edison‘s Jumbo dynamos. The electricity would be sent through a network of underground cables to buildings in a square-mile territory around the plant, each of which would be outfitted with a meter. Construction of the system began soon after the Paris Exposition, with Edison often working through the night to supervise the effort. A little more than a year later, the plant had been built and the miles of cables laid. At precisely three o‘clock in the afternoon on September 4, 1882, Edison instructed his chief electrician, John Lieb, to throw a switch at the Pearl Street station, releasing the current from one of its generators. As the New York Herald reported the following day, “in a twinkling, the area bounded by Spruce, Wall, Nassau and Pearl Streets was in a glow.” The electric utility had arrived.

And here’s an excerpt from “A Spider’s Web,” a chapter in the second half:

The most far-reaching corporate use of the cloud [will be] as a control technology for optimizing how we act as consumers. Despite the resistance of the Web‘s early pioneers and pundits, consumerism long ago replaced libertarianism as the prevailing ideology of the online world. Restrictions on the commercial use of the Net collapsed with the launch of the World Wide Web in 1991. The first banner ad—for a Silicon Valley law firm—appeared in 1993, followed the next year by the first spam campaign. In 1995, Netscape tweaked its Navigator browser to support the “cookies” that enable companies to identify and monitor visitors to their sites. By 1996, the dotcom gold rush had begun. More recently, the Web‘s role as a sales and promotion channel has expanded further. Assisted by Internet marketing consultants, companies large and small have become much more adept at collecting information on customers, analyzing their behavior, and targeting products and promotional messages to them.

The growing sophistication of Web marketing can be seen most clearly in advertising. Rather than being dominated by generic banner ads, online advertising is now tightly tied to search results or other explicit indicators of people‘s desires and identities. Search engines themselves have become the leading distributors of ads, as the prevailing tools for Web navigation and corporate promotion have merged into a single and extraordinarily profitable service. Google originally resisted the linking of advertisements to search results—its founders argued that “advertising-funded search engines will be inherently biased towards the advertisers and away from the needs of the consumers”—but now it makes billions of dollars through the practice. Search-engine optimization—the science of using advanced statistical techniques to increase the likelihood that a person will visit a site or click on an ad—has become an important corporate function, which Google and other search engines promote by sharing with companies information on how they rank sites and place ads.

In what is perhaps the most remarkable manifestation of the triumph of consumerism on the Web, popular online communities like MySpace encourage their members to become friends with corporations and their products. During 2006, for example, more than 85,000 people “friended” Toyota‘s Yaris car model at the site, happily entangling themselves in the company‘s promotional campaign for the recently introduced vehicle. “MySpace can be viewed as one huge platform for ‘personal product placement,'” writes Wade Roush in an article in Technology Review. He argues that “the large supply of fake ‘friends,‘ together with the cornucopia of ready-made songs, videos, and other marketing materials that can be directly embedded in [users’] profiles, encourages members to define themselves and their relationships almost solely in terms of media and consumption.” In recognition of the blurring of the line between customer and marketer online, Advertising Age named “the consumer” its 2007 Advertising Agency of the Year.

But the Internet is not just a marketing channel. It‘s also a marketing laboratory, providing companies with unprecedented insights into the motivations and behavior of shoppers. Businesses have long been skilled at controlling the supply side of their operations, thanks in large part to earlier advances in information technology, but they‘ve struggled when it comes to exerting control over the demand side—over what people buy and where and when they buy it. They haven‘t been able to influence customers as directly as they‘ve been able to influence employees and suppliers. Advertising and promotion have always been frustratingly imprecise. As the department store magnate John Wanamaker famously said more than a hundred years ago, “Half the money I spend on advertising is wasted. The trouble is, I don‘t know which half.”

The cloud is beginning to change that. It promises to strengthen companies‘ control over consumption by providing marketers with the data they need to personalize their pitches precisely and gauge the effects of those pitches accurately. It optimizes both communication and measurement. In a 2006 interview with the Economist, Rishad Tobaccowala, a top executive with the international ad agency Publicis, summed up the change in a colorful, and telling, metaphor. He compared traditional advertising to dropping bombs on cities—a company can‘t be sure who it hits and who it misses. But with Internet ads, he said, companies can “make lots of spearheads and then get people to impale themselves.” […]

“As every man goes through life he fills in a number of forms for the record, each containing a number of questions,” Alexander Solzhenitsyn wrote in his novel Cancer Ward. “A man‘s answer to one question on one form becomes a little thread, permanently connecting him to the local center of personnel records administration. There are thus hundreds of little threads radiating from every man, millions of threads in all. If these threads were suddenly to become visible, the whole sky would look like a spider‘s web. … Each man, permanently aware of his own invisible threads, naturally develops a respect for the people who manipulate the threads.”

As we go about our increasingly digitized lives, the threads that radiate from us are multiplying far beyond anything that even Solzhenitsyn could have imagined in the Soviet Union in the 1960s. Nearly everything we do online is recorded somewhere in the machinery of the cloud. Every time we read a page of text or click on a link or watch a video, every time we put something in a shopping cart or perform a search, every time we send an email or chat in an instant-messaging window, we are filling in a “form for the record.” Unlike Solzhenitsyn‘s Everyman, however, we‘re often unaware of the threads we‘re spinning and how and by whom they‘re being manipulated. And even if we were conscious of being monitored or controlled, we might not care. After all, we also benefit from the personalization that the Internet makes possible—it makes us more perfect consumers and workers. We accept greater control in return for greater convenience. The spider‘s web is made to measure, and we‘re not unhappy inside it.

That was the view, or at least one view, from 2008.