Over the last few months, I’ve made a number of observations, starting here, about the growing dominance of Wikipedia over search engine results for common terms. My findings were anecdotal, drawn from the searches I do day-in and day-out as well as a few dozen random ones I did for the express purpose of checking Wikipedia’s rank. Everything I saw seemed to point to an emerging Wikipedian search supremacy, but of course I was seeing only a tiny fraction of searches. It would have been nice to have a more rigorous sample.

Well, now we have such a sample, thanks to a student in Slovenia by the name of Jure Cuhalev. In a research project, Cuhalev gathered a random sample of about 1,000 of the 1.4 million topics covered by Wikipedia. He then ran the terms through the Google, Yahoo, and MSN search engines. He found that Wikipedia did in fact appear with remarkable consistency in the upper reaches of search results. On average, the online encyclopedia appeared in the top-ten search results 65% of the time – and 26% of the time it actually had two results in the top ten. (Cuhalev has posted a summary of his findings on his blog, and the full report can be downloaded here.)

But the findings get more interesting when you look beyond the averages to the particular results turned in by each of the three engines. It turns out that Google’s algorithm absolutely adores Wikipedia and that Yahoo’s passion for the online encyclopedia is nearly as ardent. But Microsoft’s MSN algorithm seems strikingly less enchanted by Wikipedia’s charms. Wikipedia turned up in Google’s top ten a whopping 81% of the time and in Yahoo’s 77%, but it appeared in MSN’s top ten just 38% of the time. What’s up with that?

Cuhalev also found that when Wikipedia does turn up in the top ten it tends to rank very highly indeed. It’s in the top three results 76% of the time at Yahoo, 66% at Google, and 54% at MSN.

I hope other researchers will look at this phenomenon from different angles, and also track changes over time, but in the meantime we now have the first solid evidence that, for Google and Yahoo at least, Wikipedia rules.

Mark Cuban, on his blog, quotes at length an “anonymous source” who claims to have the inside dope on the Google-YouTube deal and in particular its legal angles. It’s a fascinating read, though given that it’s completely uncorroborated one has no choice but to assume it’s a fantasy. As far as fantasies of acquisition negotiations go, though, it’s very well done. As Cuban says, “it rings true.” (Good fiction rings true, too.)

Regardless of its veracity, there’s one passage that particularly intrigued me because it reminded me of a question I remember having when the deal was announced. If you remember, a few hours before the news broke, there was a “separate” announcement of deals between YouTube and three of the largest music companies – Universal, Sony BMG, and Warner – in which, according to press reports at the time, the media groups took equity stakes in YouTube in exchange for some kind of assurance that they wouldn’t sue the company for copyright infringement. The equity stakes turned into a nice pile of cash when, immediately thereafter, Google announced it would buy YouTube. What I wondered at the time is whether or not the musicians whose work was being broadcast via YouTube without compensation would share in the windfall that the record companies seemed to have received.

Well, here’s the relevant passage from Cuban’s anonymous source, which discusses why the media company deals might have been structured the way they were:

> The media companies had their typical challenges. Specifically, how to

> get money from Youtube without being required to give any to the

> talent (musicians and actors)? If monies were received as part of a

> license to Youtube then they would [be] contractually obligated to share a

> substantial portion of the proceeds with others. For example most

> record label contracts call for artists to get 50% of all license

> deals. It was decided the media companies would receive an equity

> position as an investor in Youtube which Google would buy from them.

> This shelters all the up front monies from any royalty demands by

> allowing them to classify it as gains from an investment position. A

> few savvy agents might complain about receiving nothing and get a

> token amount, but most will be unaware of what transpired.

All I can say is that if I was one of those musicians I’d be asking somebody some questions.

If Wikipedia were a for-profit company, what would it be worth? It’s a question I’ve been thinking about – and, apparently, others have, too. HipMojo did some back-of-the-envelope calculations last Thursday and came with $580 million as an estimated hypothetical valuation. That got Jason Calacanis drooling, and he upped HipMojo’s valuation by a factor of ten, writing that the online encyclopedia would be worth a cool $5 billion.

Calacanis’s number is insane, but there’s no doubt that, in today’s market, a for-profit, ad-running Wikipedia would be worth a whole lot – and then some.

Just consider the latest Comscore report on the world’s top ten web sites. Wikipedia comes in at #6, having attracted 155 million unique visitors during the month of September. More striking still is that its traffic increased by a whopping 12% over August’s total – the same rate of increase posted by YouTube, the #14 site. But Wikipedia’s growth is much more impressive than YouTube’s because Wikipedia was starting from a much larger base – twice as large, in fact.

Beyond its huge popularity, Wikipedia is by its nature a search-advertiser’s dream. Visitors to the site, after all, are seeking information about particular topics, many of which – from diseases to porn stars – are also highly prized targets for search advertisers. (Compare Wikipedia to, say, a site like Digg, where users are not seeking out information on a particular topic, and individual pages present a hodge-podge of information that frustrates ad-placement algorithms.) If you do a Google search on a high-priced AdWords keyword like “asbestos,” Wikipedia comes up #3 and it’s the first nongovernmental result. It’s also #3 for “lasik,” #3 for “lawyer,” #3 for “Viagra,” #4 for “mesothelioma,” #6 for “poker,” and #2 for “sex.” And, of course, it’s #1 for hundreds of other terms, including “lawsuit” and “personal computer.”

For a search company like Google, in other words, Wikipedia would almost certainly be the most valuable single property for displaying search ads on the entire internet (with the exception, of course, of its own search site). Because of Wikipedia’s increasing dominance over search results for common terms, moreover, its lack of ads turns it into a vast black hole for Google and other search-ad syndicators. It sucks in huge numbers of web surfers without spitting out any ad revenue whatsoever. It’s not hard to see, therefore, how valuable a property it would become if it began to run ads.

I trust the Wikipedians will defend the company’s nonprofit status, and I hope they resist Calacanis’s fatuous suggestion that it’s “unconscionable to not monetize the Wikipedia” by sticking ads on the site and giving the money to charity. By that logic, you’d put billboards along the sides of the Grand Canyon – and on Calacanis’s forehead. There’s nothing immoral about choosing not to commercialize something.

But the huge theoretical value of Wikipedia’s site does makes me wonder why no devious entrepreneurs have made a concerted effort to take Wikipedia’s content, which is of course free to be reused in any way, and reformat and rebrand it as an attractive commercial site. Why, in other words, hasn’t anyone done to Wikipedia what Larry Ellison last week did to RedHat?

It’s not like Wikipedia’s site is perfect. The design’s mediocre, the user interface is often confusing, and the search tool is just plain awful. While it’s true that some sites, like Answers.com, already syndicate Wikipedia’s content, they mix it up with a whole bunch of other stuff and don’t really add any value beyond the existing Wikipedia site. I have to believe that a couple of talented coders could pretty easily hack together an improved version of Wikipedia, with a better design, a better interface, and a much better search engine. They could also strip out the site’s endless supply of arcana – the history and talk pages, for instance, and all those “warning” boxes – which are beloved by the Wikipedians but are just distracting screen junk for pretty much everyone else on earth. That would simplify the site and also reduce storage costs. Then, once the site was up, you’d advertise the hell out of it, positioning it as a superior version of the free encyclopedia.

Even if you siphoned off just a fraction of Wikipedia’s traffic, it would still be a pretty darn lucrative site – and, if it really was easier to use than Wikipedia, the traffic would tend to grow naturally. Now, granted, such a move would be a pretty slimy thing to do – and would no doubt earn the entrepreneurs some really bad karma. But, hey, it’s not like the market’s invisible hand is known for the gentleness of its caress. So tell me: What am I missing? Why hasn’t anyone made a real effort to do a Larry on Wikipedia?

The brouhaha that surrounded my Harvard Business Review article “IT Doesn’t Matter,” published in May 2003, seems to have scared HBR away from the topic of IT altogether. I can’t for the life of me think of a single substantial article that it’s published on the topic in the succeeding three and a half years. Now, I’d like to believe that’s because the academic crowd embraced my message lock, stock and barrel – if IT doesn’t matter, why write about it in executive journals? – but, given the apoplexy with which some of Harvard’s elder IT statesmen greeted my argument, something tells me there are other forces at work.

Anyway, I was glad to see that one my favorite academic thinkers about IT, Andrew McAfee, has a big article in the new edition of HBR. (And I was even happier to discover that HBR is allowing free access to the piece, at least for a few weeks.) McAfee, unlike some of his colleagues, doesn’t try to fit the post-internet IT world into tired pre-internet frameworks. He comes at it fresh. And that’s good.

But McAfee’s HBR article, alas, lacks the freshness, the intellectual zip, of his best work. It feels like it may have been left in HBR’s blandification machine a little too long. Compared with the recent piece he wrote on “Enterprise 2.0” for the MIT Sloan Management Review, which argued that Web 2.0 technologies may require a basic rethinking of the way businesses approach IT, the new piece, called “Mastering the Three Worlds of Information Technology,” is more, well, academic. As its title suggests, it’s one of those dry scholarly exercises in categorization that spend a lot of time explaining stuff that most managers understand intuitively.

Echoing the work of other IT scholars, most notably MIT’s Erik Brynjolfsson, McAfee argues that IT success in business today is less about technology than about good old-fashioned management: “Everyone who has studied companies’ frustrations with IT argues that technology projects are increasingly becoming managerial challenges rather than technical ones.” Success hinges, in particular, on how well you manage IT’s “organizational complements” such as the design of processes, the rules of governance, and the talents of people. What’s new in McAfee’s piece is the idea that the importance of such complements, and hence management’s role in IT management, varies according the type of IT a company is installing.

McAfee identifies three categories of IT: “Function IT” (FIT) is the kind that, like a word-processing or computer-aided-design program, “make[s] the execution of stand-alone tasks more efficient.” “Network IT” (NIT) is the stuff that, like Lotus Notes or wikis, helps employees communicate and collaborate. Finally, “Enterprise IT” (EIT) consists of the heavy-duty systems that automate big corporate processes – CRM and ERP systems, for instance. While FIT and NIT don’t “impose complements” on companies, EIT does. In other words, EIT forces you to make certain changes to your processes, your governance structure, and so on. And while FIT and EIT typically require a strong top-down management push to encourage their adoption by employees, NIT tends to be, McAfee argues (not entirely convincingly), welcomed by employees. Management’s main role is to get the hell out of the way.

This kind of categorization can be useful in adding precision to the language we use to discuss complex subjects. It helps us get beyond big, ill-defined generalizations. But there’s a drawback. It can prevent us from seeing how categories blend together. By drawing bright lines between things, it can give the illusion that those things are more distinct than they really are. I sense that problem here (even while granting the usefulness of McAfee’s categorization). Take the identification of CRM as an enterprise information technology. Isn’t that assumption exactly what doomed so many big CRM projects? The projects lost sight of the fact that CRM is as much a functional tool, a tool that helps individual employees, like salespeople, do their work better, as an enterprise system. CRM, in other words, is as much FIT as EIT. And, in fact, there’s a lot of NIT in it as well.

What’s exciting about IT today, I’d argue, is that we can finally begin to get beyond old dysfunctional categories. Software-as-a-service products, like the CRM systems provided by Salesforce.com or NetSuite or RightNow, are compelling not simply because they allow companies to avoid big capital expenses, but because they begin to break down the monolithic complexity of traditional enterprise systems and give more control to the individual user – they turn EIT into FIT. We certainly need to appreciate the differences in IT tools and systems, but, even more important, we need to begin to see beyond the distinctions that weren’t true differences at all, but were merely the byproducts of immature technology. From this perspective, McAfee’s article may not be quite as clarifying as it is intended to be.

The Ask.com blog has a funny retort to Google’s almost unbearably annoying blog-lecture about the proper usage of its name. Michael Ferguson, the Ask blogger, nails Google’s faux-chatty tone. A taste:

Q: Do people Google on Ask?

A: Looking at our logs, people do seem to google on Ask.com. They type in “google” and go to Google. They also do this with Yahoo, eBay, Amazon etc. So you may find that you Yahoo on Google, eBay on Yahoo, and even Amazon on Amazon (if you were looking for books about the rainforest on that wonderful shopping site). Just this morning I Microsoft’d on the Ask.com Blog Search. Felt good.

It makes me think I should start doing more of my googling over at Ask.

A couple of days ago, I reread Ray Ozzie’s famous memo to top Microsoft’s top executives and engineers on “the internet services disruption,” as well as Bill Gates’s cover note. The sense of urgency in both men’s words remains striking. Gates spoke of the software-as-a-service model as being a “wave” that could swamp the traditional software business. Ozzie said that “the most important step is for each of us to internalize the transformative and disruptive potential of services. We must then focus on the need for agility in execution.”

It’s been a year, exactly, since those memos went out. A lot’s gone on in that time. It basically covers, for instance, YouTube’s entire lifespan, from early startup mode to explosive viral growth to being bought out for more than a billion and a half dollars by Microsoft’s nemesis in Internet services, Google. A lot of startups have jumped pell-mell into the so-called “Office 2.0” business, hoping to create online alternatives or complements to Microsoft Office. While their services, for the most part, remain rudimentary, they’ve progressed significantly over these last 12 months.

So where’s Microsoft? I think it’s fair to say that the company’s been distracted. It’s been struggling to get what seem likely to be the last major upgrades (in the traditional sense of upgrades) of its two major products, Windows and Office, out the door. It’s been working through organizational changes, not least Gates’s cutting back of his role in the company. It’s been continuing its seemingly permanent negotiations with European regulators. (“Another croissant, Mr. Ballmer?”) It’s been constructing big-ass utility data centers. And, needless to say, it’s been sucking in a ton of money. Oh, and there’s that Zune thing, too. Big companies tend to have a lot on their plates, and Microsoft’s is heaping full.

It’s not surprising, therefore, that the company hasn’t yet demonstrated the “agility in execution” that, as Ozzie argued, it needs if it’s going to make a successful transition to the services model. Yes, it’s been expanding the services, or “Live,” side of its business. But the steps have felt tentative, and the products, so far, have been underwhelming and ill-branded. (Like Google, Microsoft might really want to hire someone with a little talent in naming products.) The Live stuff hasn’t exactly been turning any heads.

Early today, Dan Farber reported on a conversation he had with Ozzie yesterday evening. He asked Microsoft’s new chief software architect about what he thought of the Office 2.0 crowd and the potential threat it posed. Ozzie’s response was as thoughtful as you’d expect, but it also was surprisingly tepid. It lacked the urgency you heard in his memo:

From Ozzie’s response, Microsoft is not in a hurry to deliver a pure Web Office, nor does it have its head in the sand. “People have been trying to create applications with Web technology since the Web began,” Ozzie said. “Just because you can do it, doesn’t mean you should. We are looking at Google Docs & Spreadsheets, and paying attention to Office 2.0 and Zoho. We are also putting those in front of customers and seeing what makes sense.[“] Ozzie said that Microsoft [is] taking a holistic view of how to proceed from a customer point of view, and modeling various user scenarios. ”

“Modeling various user scenarios” does not seem like a recipe for “agility in execution” in the new world of software – a world in which “user scenarios” are defined not in laboratory settings but in the rough-and-tumble of the web.

Farber continues:

I probed a bit more on the topic of whether a tipping point had been reached for browser-based suites. Ozzie said that no announcement is forthcoming. However, I would guess that Microsoft [is] busy coding browser-based Office components and could pull the trigger rapidly if it were deemed necessary to compete. Nonetheless, Microsoft could miss the window of opportunity, as it did in Web search, and have to play catch up in a category that is a major cash cow for the company. Developing the ultimate hybrid Office platform, beyond the forthcoming Office 2007, that accommodates all kinds [of] online and offline user scenarios could also take a long time versus the faster moving upstarts adding new features every few months and taking advantage of improvements in bandwidth and network reliability.

I’ve argued here in the past that Microsoft is in the catbird’s seat when it comes to leading the shift of personal productivity applications from the desktop to the web. We’re not at a “tipping point,” so far as the mainstream business market is concerned. We’re at the start of a transitional period in which the capabilities of desktop applications will be extended by, and slowly replaced by, the capabilities of services. As the dominant player – the only major player, for that matter – in desktop productivity apps, Microsoft has a strong natural advantage (some would call it an unnatural advantage) in coming up with the interim hybrid productivity apps that will be broadly adopted by businesses.

But while there’s no particular reason for Microsoft to act rashly, there is a big danger in relying on “the holistic view.” The new world is not going to be created holistically. It’s going to be created piecemeal. Farber’s right that Microsoft could easily come to find itself playing catch up. That’s not the ideal position to be in when it comes to software services that can take over the market with YouTube-like speed. You have to be in the fray, if only to keep your competitors off balance.

I’ve been fiddling around with SlideShare recently. It’s a service that makes it simple to publish PowerPoint presentations on web sites and blogs. As Ross Mayfield has said, it applies the YouTube model to PowerPoint. It’s one of those products that you look at and say, “This is really useful. I need this.” There’s no doubt in my mind that there’s a big market for a service that makes it as easy to syndicate presentations as YouTube makes it to syndicate videos. There’s a problem, though. SlideShare, which is in beta, doesn’t work very well yet. It can’t handle a lot of the graphics and charts and other elements routinely found in PowerPoint presentations. Which means it’s still a long way from being “easy.”

And yet this is such an obvious extension of the usefulness of PowerPoint that you have to wonder why Microsoft isn’t out there with its own innovation. It can’t be a matter of resources and coding expertise. If a dinky firm like SlideShare can do it, Microsoft can certainly do it. And this service doesn’t threaten to cannibalize PowerPoint sales at all. In fact, it would make PowerPoint more valuable – and it would also open up to Microsoft an opportunity to gain additional revenues by hosting the syndicated presentations on its own servers. You give away a little storage space with every version of PowerPoint, and then you charge companies for additional space. It’s kind of a no-brainer, so far as I can see.

Why this kind of piecemeal service is dangerous to Microsoft is that it does have a YouTube-like quality. It solves a common problem. Once it gets good, it could take off rapidly. And if it’s somebody else’s service that takes off, it would put a dent in Microsoft’s control over the market. It would begin the process of erosion that’s the biggest threat Microsoft faces with Office.

Sure you should think holistically. But you shouldn’t let the whole blind you to the importance of the pieces.

As open-source software programs mature and become commercial products, the work of developing them naturally shifts, to one degree or another, from the original community of unpaid volunteers to professional programmers who are employed and paid by companies, in particular the companies that profit from selling services related to the installation and upkeep of the software. What Yochai Benkler calls “social production,” the system of creating goods through freely donated labor rather than through labor that’s purchased and controlled by corporations, begins, inevitably, to break down. You get, instead, a system in which paid workers and volunteers labor together, though not necessarily toward the same goal, in an uneasy alliance. Some may call this a hybrid system. Others may call it a corrupted one.

It’s always been clear that the system, however you view it, imposes an economic vulnerability on the profit-making companies that engage in it. Those companies have to pay labor costs for developing a free good, a public good that that they have no proprietary control over. Their rivals can reap the fruits of that labor without having to pay for it. That creates, in theory, a dangerous asymmetry in competition. But what hasn’t been clear is whether that vulnerability actually matters, whether the danger that exists in theory also exists in reality. Are there economic or other barriers that prevent competitors from capitalizing on the investments of the open-source companies?

We’re about to get a lot closer to an answer to that question, thanks to that great clarifying force in the technology business, Larry Ellison. Yesterday, Ellison announced that his company, Oracle, fully intends to eat the fruits of the labor of Red Hat, the leading for-profit supplier of the open-source Linux operating system. Oracle is taking the version of Linux developed by Red Hat and distributing it under its own brand, as “Unbreakable Linux.” And, in a stab at Red Hat’s very heart, Ellison claims that Oracle will substantially undercut the open-source firm’s prices for supporting the software. It seems like a claim that shouldn’t be hard to fulfill. After all, Oracle doesn’t have to pay those labor costs.

Once open source became a business, rather than a movement, the rules changed. Larry Ellison, whos’s nothing if not a non-sentimentalist, understands that, and he doesn’t particularly care what “the community” thinks. His attack on Red Hat would never be called neighborly, but it is, as Business Week’s Steve Hamm puts it, “a ruthless and brilliant act of capitalism.”

It’s also something more. It illuminates a much broader and deeper tension in the digital world, a fault line that runs not only through the software industry but through every industry whose products or services exist, or can exist, as software. The tension is between social production and the profit motive. Volunteer labor means something very different in the context of a community than it does in the context of a business. In the context of a community, it’s an expression of fellowship, of the communal value of sharing. But in the context of a business, as Ellison’s move illustrates, it’s nothing more than a cheap input. Many of the most eloquent advocates of social production would prefer it if this tension didn’t exist. But it does, and it’s important.