Milestone Group Quarterly: October 2006
Articles
By Invitation:
Tim O'Reilly, O'Reilly Media Founder and Web 2.0 Author
The bursting of the dot-com bubble in the fall of 2001 marked a turning point for the Web. Many people concluded that the Web was over-hyped, when in fact bubbles and consequent shakeouts appear to be a common feature of all technological revolutions. Shakeouts typically mark the point at which an ascendant technology is ready to take its place at center stage. The pretenders are given the bum's rush, the real success stories show their strength, and there begins to be an understanding of what separates one from the other.
Like many important concepts, Web 2.0 doesn't have a hard boundary, but rather, a gravitational core. You can visualize Web 2.0 as a set of principles and practices that tie together a veritable solar system of sites that demonstrate some or all of those principles, at a varying distance from that core.
The first of those principles is "The Web as platform” (though that was also a rallying cry of Web 1.0 darling Netscape, which went down in flames). If Netscape is the standard bearer for Web 1.0, Google is most certainly the standard bearer for Web 2.0, if only because their respective IPOs were defining events for each era.
Netscape framed "the Web as platform" in terms of the old software paradigm: their flagship product was the Web browser, a desktop application, and their strategy was to use their dominance in the browser market to establish a market for high-priced server products. Control over standards for displaying content and applications in the browser would, in theory, give Netscape the kind of market power enjoyed by Microsoft in the PC market. In the end, both Web browsers and Web servers turned out to be commodities, and value moved "up the stack" to services delivered over the Web platform.
Google, by contrast, began its life as a native Web application, never sold or packaged, but delivered as a service, with customers paying, directly or indirectly, for the use of that service. None of the trappings of the old software industry are present. No scheduled software releases, just continuous improvement. No licensing or sale, just usage. Google requires a competency that Netscape never needed: database management. Google isn't just a collection of software tools, it's a specialized database. Without the data, the tools are useless; without the software, the data is unmanageable. In this case, the value of the software is proportional to the scale and dynamism of the data it helps to manage.
While both Netscape and Google could be described as software companies, it's clear that Netscape belonged to the same software world as Lotus, Microsoft, Oracle, SAP, and other companies that got their start in the 1980's software revolution, while Google's fellows are other internet applications like eBay, Amazon, Napster, DoubleClick and Akamai.
This leads to a central principle behind the success of the giants born in the Web 1.0 era who have survived to lead the Web 2.0 era - they have embraced the power of the Web to harness collective intelligence. Yahoo!, the first great Internet success story, was born as a catalog, or directory of links, an aggregation of the best work of thousands, then millions of Web users. While Yahoo! has since moved into the business of creating many types of content, its role as a portal to the collective work of the Net's users remains the core of its value.
Google's breakthrough in search, which quickly made it the undisputed search market leader, was PageRank, a method of using the link structure of the Web rather than just the characteristics of documents to provide better search results.
Now, innovative companies that pick up on this insight and perhaps extend it even further, are making their mark. Wikipedia, an online encyclopedia based on the unlikely notion that an entry can be added by any Web user, and edited by any other, is a radical experiment in trust, applying Eric Raymond's dictum (originally coined in the context of open source software) that "with enough eyeballs, all bugs are shallow," to content creation. Wikipedia is already in the top 100 Websites, and many think it will be in the top ten before long. This is a profound change in the dynamics of content creation!
The lesson: Network effects from user contributions are the key to market dominance in the Web 2.0 era.
Now, of course, "dynamic Web sites" (i.e., database-backed sites with dynamically generated content) replaced static Web pages well over ten years ago. What's dynamic about the live Web are not just the pages, but the links. A link to a Weblog is expected to point to a perennially changing page, with "permalinks" for any individual entry, and notification for each change. An RSS feed is thus a much stronger link than, say a bookmark or a link to a single page.
But RSS is only part of what makes a Weblog different from an ordinary Web page. Tom Coates remarks on the significance of the permalink:
"It may seem like a trivial piece of functionality now, but it was effectively the device that turned Weblogs from an ease-of-publishing phenomenon into a conversational mess of overlapping communities. For the first time it became relatively easy to gesture directly at a highly specific post on someone else's site and talk about it. Discussion emerged. Chat emerged. And - as a result - friendships emerged or became more entrenched. The permalink was the first - and most successful - attempt to build bridges between Weblogs." In many ways, the combination of RSS and permalinks adds many of the features of NNTP, the Network News Protocol of the Usenet, onto HTTP, the Web protocol. The "blogosphere" can be thought of as a new, peer-to-peer equivalent to Usenet and bulletin-boards, the conversational watering holes of the early Internet. Not only can people subscribe to each others' sites, and easily link to individual comments on a page, but also, via a mechanism known as trackbacks, they can see when anyone else links to their pages, and can respond, either with reciprocal links, or by adding comments.
If an essential part of Web 2.0 is harnessing collective intelligence, turning the Web into a kind of global brain, the blogosphere is the equivalent of constant mental chatter in the forebrain, the voice we hear in all of our heads. It may not reflect the deep structure of the brain, which is often unconscious, but is instead the equivalent of conscious thought. And as a reflection of conscious thought and attention, the blogosphere has begun to have a powerful effect.
First, because search engines use link structure to help predict useful pages, bloggers, as the most prolific and timely linkers, have a disproportionate role in shaping search engine results. Second, because the blogging community is so highly self-referential, bloggers paying attention to other bloggers magnifies their visibility and power. The "echo chamber" that critics decry is also an amplifier.
If it were merely an amplifier, blogging would be uninteresting. But like Wikipedia, blogging harnesses collective intelligence as a kind of filter. What James Suriowecki calls "the wisdom of crowds" comes into play, and much as PageRank produces better results than analysis of any individual document, the collective attention of the Blogosphere selects for value.
And that value is characterized by information. Database management is a core competency of Web 2.0 companies, so much so that we have sometimes referred to these applications as "infoware" rather than merely software. This fact leads to a key question: Who owns the data?
The now hotly contested Web mapping arena demonstrates how a failure to understand the importance of owning an application's core data will eventually undercut its competitive position. MapQuest pioneered the Web mapping category in 1995, yet when Yahoo!, and then Microsoft, and most recently Google, decided to enter the market, they were easily able to offer a competing application simply by licensing the same data.
Contrast, however, the position of Amazon.com. Like competitors such as Barnesandnoble.com, its original database came from ISBN registry provider R.R. Bowker. But unlike MapQuest, Amazon relentlessly enhanced the data, adding publisher-supplied data such as cover images, table of contents, index, and sample material. Even more importantly, they harnessed their users to annotate the data, such that after ten years, Amazon, not Bowker, is the primary source for bibliographic data on books, a reference source for scholars and librarians as well as consumers. Amazon also introduced their own proprietary identifier, the ASIN, which corresponds to the ISBN where one is present, and creates an equivalent namespace for products without one. Effectively, Amazon "embraced and extended" their data suppliers.
Imagine if MapQuest had done the same thing, harnessing their users to annotate maps and directions, adding layers of value. It would have been much more difficult for competitors to enter the market just by licensing the base data.
The recent introduction of Google Maps provides a living laboratory for the competition between application vendors and their data suppliers. Google's lightweight programming model has led to the creation of numerous value-added services in the form of mashups that link Google Maps with other internet-accessible data sources. Paul Rademacher's housingmaps.com, which combines Google Maps with Craigslist apartment rental and home purchase data to create an interactive housing search tool, is the pre-eminent example of such a mashup.
At present, these mashups are mostly innovative experiments, done by hackers. But entrepreneurial activity follows close behind. And already, one can see that for at least one class of developer, Google has taken the role of data source away from Navteq and inserted themselves as a favored intermediary. We expect to see battles between data suppliers and application vendors in the next few years, as both realize just how important certain classes of data will become as building blocks for Web 2.0 applications.
The race is on to own certain classes of core data: location, identity, calendaring of public events, product identifiers and namespaces. In many cases, where there is significant cost to create the data, there may be an opportunity for an “Intel Inside” style play, with a single source for the data. In others, the winner will be the company that first reaches critical mass via user aggregation, and turns that aggregated data into a system service. As noted above in the discussion of Google vs. Netscape, one of the defining characteristics of Internet era software is that it is delivered as a service, not as a product. This fact leads to a number of fundamental changes in the business model of such a company:
- Operations must become a core competency. Google's or Yahoo!'s expertise in product development must be matched by an expertise in daily operations. So fundamental is the shift from software as artifact to software as service that the software will cease to perform unless it is maintained on a daily basis. Google must continuously crawl the Web and update its indices, continuously filter out link spam and other attempts to influence its results, continuously and dynamically respond to hundreds of millions of asynchronous user queries, simultaneously matching them with context-appropriate advertisements.
It's no accident that Google's system administration, networking, and load balancing techniques are perhaps even more closely guarded secrets than their search algorithms. Google's success at automating these processes is a key part of their cost advantage over competitors.
It's also no accident that scripting languages such as Perl, Python, PHP, and now Ruby, play such a large role at Web 2.0 companies. Perl was famously described by Hassan Schroeder, Sun's first Webmaster, as "the duct tape of the Internet." Dynamic languages (often called scripting languages and looked down on by the software engineers of the era of software artifacts) are the tool of choice for system and network administrators, as well as application developers building dynamic systems that require constant change.
- Users must be treated as co-developers, in a reflection of open source development practices (even if the software in question is unlikely to be released under an open source license.) The open source dictum, "release early and release often" in fact has morphed into an even more radical position, "the perpetual beta," in which the product is developed in the open, with new features slipstreamed in on a monthly, weekly, or even daily basis. It's no accident that services such as Gmail, Google Maps, Flickr, del.icio.us, and the like may be expected to bear a "Beta" logo for years at a time.
Real time monitoring of user behavior to see just which new features are used, and how they are used, thus becomes another required core competency. A Web developer at a major online service remarked: "We put up two or three new features on some part of the site every day, and if users don't adopt them, we take them down. If they like them, we roll them out to the entire site."
Cal Henderson, the lead developer of Flickr, recently revealed that they deploy new builds up to every half hour. This is clearly a radically different development model! While not all Web applications are developed in as extreme a style as Flickr, almost all Web applications have a development cycle that is radically unlike anything from the PC or client-server era. It is for this reason that a recent ZDnet editorial concluded that Microsoft won't be able to beat Google: "Microsoft's business model depends on everyone upgrading their computing environment every two to three years. Google's depends on everyone exploring what's new in their computing environment every day."
While Microsoft has demonstrated enormous ability to learn from and ultimately best its competition, there's no question that this time, the competition will require Microsoft (and by extension, every other existing software company) to become a deeply different kind of company. Native Web 2.0 companies enjoy a natural advantage, as they don't have old patterns (and corresponding business models and revenue sources) to shed.
Another feature of Web 2.0 that deserves mention is the fact that it's no longer limited to the PC platform. In his parting advice to Microsoft, long time Microsoft developer Dave Stutz pointed out that "Useful software written above the level of the single device will command high margins for a long time to come."
Of course, any Web application can be seen as software above the level of a single device. After all, even the simplest Web application involves at least two computers: the one hosting the Web server and the one hosting the browser. And as we've discussed, the development of the Web as platform extends this idea to synthetic applications composed of services provided by multiple computers.
But as with many areas of Web 2.0, where the "2.0-ness" is not something new, but rather a fuller realization of the true potential of the Web platform, this phrase gives us a key insight into how to design applications and services for the new platform.
To date, iTunes is the best exemplar of this principle. This application seamlessly reaches from the handheld device to a massive Web back-end, with the PC acting as a local cache and control station. There have been many previous attempts to bring Web content to portable devices, but the iPod/iTunes combination is one of the first such applications designed from the ground up to span multiple devices. TiVo is another good example. iTunes and TiVo also demonstrate many of the other core principles of Web 2.0. They are not Web applications per se, but they leverage the power of the Web platform, making it a seamless, almost invisible part of their infrastructure. Data management is most clearly the heart of their offering. They are services, not packaged applications (although in the case of iTunes, it can be used as a packaged application, managing only the user's local data.) What's more, both TiVo and iTunes show some budding use of collective intelligence, although in each case, their experiments are at war with the IP lobby's.
There's only a limited architecture of participation in iTunes, though the recent addition of podcasting changes that equation substantially.
This is one of the areas of Web 2.0 where we expect to see some of the greatest change, as more and more devices are connected to the new platform. What applications become possible when our phones and our cars are not consuming data but reporting it? Real time traffic monitoring, flash mobs, and citizen journalism are only a few of the early warning signs of the capabilities of the new platform.
In exploring the principles above, we've highlighted some of the principal features of Web 2.0. Each of the examples we've explored demonstrates one or more of those key principles, but may miss others. Let's close, therefore, by summarizing what we believe to be the core competencies of Web 2.0 companies:
- Services, not packaged software, with cost-effective scalability
- Control over unique, hard-to-recreate data sources that get richer as more people use them
- Trusting users as co-developers
- Harnessing collective intelligence
- Leveraging the long tail through customer self-service
- Software above the level of a single device
- Lightweight user interfaces, development models, AND business models
The next time a company claims that it's "Web 2.0," test their features against the list above. The more points they score, the more they are worthy of the name. Remember, though, that excellence in one area may be more telling than some small steps in all dimensions.
Excerpted from "What is Web 2.0", available in full at http://www.oreilly.com/go/Web2.
Tim O'Reilly is the founder and CEO of O'Reilly Media, thought by many to be the best computer book publisher in the world. The company also publishes online through the O'Reilly Network and hosts conferences on technology topics. Tim is an activist for open source, open standards, and sensible intellectual property laws.
Since 1978, Tim has led the company's pursuit of its core goal: to be a catalyst for technology change by capturing and transmitting the knowledge of "alpha geeks" and other innovators. His active engagement with technology communities drives both the company's product development and its marketing. Tim has built a culture where advocacy, meme-making, and evangelism are key tenets of the business philosophy.
|
