kottke.org posts about 'anildash'

Growth of Twitter vs. Blogger

Important update: I've re-evaluated the Twitter data and came up with what I think is a much more accurate representation of what's going on.

Further update: The Twitter data is bad, bad, bad, rendering Andy's post and most of this here post useless. Both jumps in Twitter activity in Nov 2006 and March 2007 are artificial in nature. See here for an update.

Update: A commenter noted that sometime in mid-March, Twitter stopped using sequential IDs. So that big upswing that the below graphs currently show is partially artificial. I'm attempting to correct now. This is the danger of doing this type of analysis with "data" instead of data.
--

In mid-March, Andy Baio noted that Twitter uses publicly available sequential message IDs and employed Twitter co-founder Evan Williams' messages to graph the growth of the service over the first year of its existence. Williams co-founded Blogger back in 1999, a service that, as it happens, also exposed its sequential post IDs to the public. Itching to compare the growth of the two services from their inception, I emailed Matt Webb about a script he'd written a few years ago that tracked the daily growth of Blogger. His stats didn't go back far enough so I borrowed Andy's idea and used Williams' own blog to get his Blogger post IDs and corresponding dates. Here are the resulting graphs of that data.¹

The first one covers the first 253 days of each service. The second graph shows the Twitter data through May 7, 2007 and the Blogger data through March 7, 2002. [Some notes about the data are contained in this footnote.]

As you can see, the two services grew at a similar pace until around 240 days in, with Blogger posts increasing faster than Twitter messages. Then around November 21, 2006, Twitter took off and never looked back. At last count, Twitter has amassed five times the number of messages than Blogger did in just under half the time period. But Blogger was not the slouch that the graph makes it out to be. Plotting the service by itself reveals a healthy growth curve:

From late 2001 to early 2002, Blogger doubled the number of messages in its database from 5M to 10M in under 200 days. Of course, it took Twitter just over 40 days to do the same and under 20 days to double again to 20M. The curious thing about Blogger's message growth is that large events like 9/11, SXSW 2000 & 2001, new versions of Blogger, and the launch of blog*spot didn't affect the growth at all. I expected to see a huge message spike on 9/11/01 but there was barely a blip.

The second graph also shows that Twitter's post-SXSW 2007 growth is real and not just a temporary bump...a bunch of people came to check it out, stayed on, and everyone messaged like crazy. However, it does look like growth is slowing just a bit if you look at the data on a logarithmic scale:

Actually, as the graph shows, the biggest rate of growth for Twitter didn't occur following SXSW 2007 but after November 21.

As for why Twitter took off so much faster than Blogger, I came up with five possible reasons (there are likely more):

1. Twitter is easier to use than Blogger was. All you need is a web browser or mobile phone. Before blog*spot came along in August 2000, you needed web space with FTP access to set up a Blogger blog, not something that everyone had.

2. Twitter has more ways to create a new message than Blogger did at that point. With Blogger, you needed to use the form on the web site to create a post. To post to Twitter, you can use the web, your phone, an IM client, Twitterrific, etc. It's also far easier to send data to Twitter programatically...the NY Times account alone sends a couple dozen new messages into the Twitter database every day without anyone having to sit there and type them in.

3. Blogger was more strapped for cash and resources than Twitter is. The company that built Blogger ran out of money in early 2001 and nearly out of employees shortly after that. Hard to say how Blogger might have grown if the dot com crash and other factors hadn't led to the severe limitation of its resources for several key months.

4. Twitter has a much larger pool of available users than Blogger did. Blogger launched in August 1999 and Twitter almost 7 years later in March 2006. In the intervening time, hundreds of millions of people, the media, and technology & media companies have become familiar and comfortable with services like YouTube, Friendster, MySpace, Typepad, Blogger, Facebook, and GMail. Hundreds of millions more now have internet access and mobile phones. The potential user base for the two probably differed by an order of magnitude or two, if not more.

5. But the biggest factor is that the social aspect of Twitter is built in and that's where the super-fast growth comes from. With Blogger, reading, writing, and creating social ties were decoupled from each other but they're all integrated into Twitter. Essentially, the top graph shows the difference between a site with social networking and one largely without. Those steep parts of the Twitter trend on Nov 21 and mid-March? That's crazy insane viral growth², very contagious, users attracting more users, messages resulting in more messages, multiplying rapidly. With the way Blogger worked, it just didn't have the capability for that kind of growth.

A few miscellaneous thoughts:

It's important to keep in mind that these graphs depict the growth in messages, not users or web traffic. It would be great to have user growth data, but that's not publicly available in either case (I don't think). It's tempting to look at the growth and think of it in terms of new users because the two are obviously related. More users = more messages. But that's not a static relationship...perhaps Twitter's userbase is not increasing all that much and the message growth is due to the existing users increasing their messaging output. So, grain of salt and all that.

What impact does Twitter's API have on its message growth? As I said above, the NY Times is pumping dozens of messages into Twitter daily and hundreds of other sites do the same. This is where it would be nice to have data for the number of active users and/or readers. The usual caveats apply, but if you look at the Alexa trends for Twitter, pageviews and traffic seem to leveling out. Compete, which only offers data as recently as March 2007, still shows traffic growing quickly for Twitter.

Just for comparison, here's a graph showing the adoption of various technologies ranging from the automobile to the internet. Here's another graph showing the adoption of four internet-based applications: Skype, Hotmail, ICQ, and Kazaa (source: a Tim Draper presentation from April 2006).

[Thanks to Andy, Matt, Anil, Meg, and Jonah for their data and thoughts.]

[1] Some notes and caveats about the data. The Blogger post IDs were taken from archived versions of Evhead and Anil Dash's site stored at the Internet Archive and from a short-lived early collaborative blog called Mezzazine. For posts prior to the introduction of the permalink in March 2000, most pages output by Blogger didn't publish the post IDs. Luckily, both Ev and Anil republished their old archives with permalinks at a later time, which allowed me to record the IDs.

The earliest Blogger post ID I could find was 9871 on November 23, 1999. Posts from before that date had higher post IDs because they were re-imported into the database at a later time so an accurate trend from before 11/23/99 is impossible. According to an archived version of the Blogger site, Blogger was released to the public on August 23, 1999, so for the purposes of the graph, I assumed that post #1 happened on that day. (As you can see, Anil was one of the first 2-3 users of Blogger who didn't work at Pyra. That's some old school flavor right there.)

Regarding the re-importing of the early posts, that happened right around mid-December 1999...the post ID numbers jumped from ~13,000 to ~25,000 in one day. In addition to the early posts, I imagine some other posts were imported from various Pyra weblogs that weren't published with Blogger at the time. I adjusted the numbers subsequent to this discontinuity and the resulting numbers are not precise but are within 100-200 of the actual values, an error of less than 1% at that point and becoming significantly smaller as the number of posts grows large. The last usable Blogger post ID is from March 7, 2002. After that, the database numbering scheme changed and I was unable to correct for it. A few months later, Blogger switched to a post numbering system that wasn't strictly sequential.

The data for Twitter from March 21, 2006 to March 15, 2007 is from Andy Baio. Twitter data subsequent to 3/15/07 was collected by me. ↩

[2] "Crazy insane viral growth" is a very technical epidemiological term. I don't expect you to understand its precise meaning. ↩

 May 8, 2007  •  tags: weblogs www nostalgia statistics twitter evanwilliams anildash blogger andybaio

It's ok if you enjoy pretending to talk like a cat, but don't sucker yourself into thinking that it's anything more than April Fools' Day non-humor on every single day of the year.

 Apr 27, 2007  •  tags: aprilfoolsday anildash

The blog commentor's gaze

The cover story of the December 9th issue of Science News, The Predator's Gaze, is about psychopathy. The whole article is worth a read, but the brief description of psychopathy at the beginning got me thinking about something that Anil Dash wrote the other day. He highlighted a review of a B&B made by a potential guest that was upset that his many attempts to persuade the owners to accept his expired gift certificate. Anil labeled this person a sociopath:

As a public service, I offer you my analysis. This quote is how you can tell this guy is a sociopath. Not that he merely went online and vented to random strangers about his greediness. No, rather, that he was willing to concede his own willful ignorance (or illiteracy?) while complaining. The web is littered with these chuckleheads who point out their own sociopathic behavior while complaining about others.

At dinner the other night, a group of us were talking about a particularly irksome message board contributor and the subject of sociopathy came up again. This particular person seemed to be oblivious to the rules of the board, didn't pick up on the social cues of other participants or moderators to modify his behavior, and was making public personal attacks against others while complaining that others were doing the same to him, even though they were not. Anyone who runs a community site, has comments on their blog, or participates on a message board knows this guy -- and it usually is a guy. He's the fly in everyone else's ointment, screaming in the middle a quiet conversation, and then says things like "if you hate me, I must be doing something right".

With that in mind, some quotes from the Science News article:

Psychopaths lack a conscience and are incapable of experiencing empathy, guilt, or loyalty.

People with psychopathy don't modify behaviors for which they're punished and don't learn to avoid actions that harm others, Blair proposes in the September Cognition. As a result, they fail to develop a moral sense, in his view. Blair's theory fits with previous observations that psychopaths have difficulty learning to avoid punishments, show weak physiological responses to threats, and don't often recognize sadness or fear in others.

He views psychopathic personalities as the product of an attention deficit. Psychopaths focus well on their explicit goals but ignore incidental information that provides perspective and guides behavior, Newman holds. Most other people, as they take action, unconsciously consult such information, for instance, rules of conduct in social settings and nonverbal signs of discomfort in those around them.

Sounds a lot like the fellow we were discussing at dinner. I don't think most of the people that demonstrate antisocial behavior in comment threads are actually psychopaths or sociopaths (there is a difference) in real life. Rather, interacting via text strips out so much social context and "incidental information" that causes some people to display psychopathic behavior online and fail to develop an online moral sense.

Thinking about disruptive commenters in this way presents an interesting challenge. According to the article, psychopathy seems to be genetic in nature and curing people of this extreme antisocial behavior can be difficult. An Australian study cited in the article found that boys with behavioral problems reacted better to rewards for good behavior than to punishments for bad behavior. Maybe looking for ways to reward bad online community members for their good behavior as well as trying to replace some of the stripped away social context is the way forward. (A quick idea for replacing some social context: add a graphic of eyes to the text-posting interface?)

 Dec 15, 2006  •  tags: science weblogs sociology anildash

Snowclone watch watch: Anil discovers the multivalent story of America.

 Nov 1, 2006  •  tags: usa language snowclones anildash

Dealing with stuff that sucks

If you've ever used any of the various menu sites out there, you may have noticed that the menus are occasionally not as up-to-date or complete as they could be. A typical response in the blogosphere to a situation like this is to fire off a snarky missive about how menu sites suck, wish harm on the site's owners and their children, and why don't they just die already, those sucking bastards, and basically overreact in such a way as to make the writer feel temporarily better and all but ensures that nothing constructive comes of it.

Since its launch last year, I've admired the tone of Eater, a site about New York city food and dining. The site strikes the right balance between criticism, enthusiasm, insider knowledge, and detatched reportage while covering a topic where too much of any one of these is deadly for the reader. Last week, Eater took note of the menu site situation, but instead of just complaining, they went looking for some evidence and reported the results:

Last week, Eater began an exhaustive investigative series called MenuGate. For those who think we'd forgotten about it, ten-hut. Tomorrow morning, we'll be conducting a SPOT INSPECTION of the major menu site players, then scoring them on how accurate (or inaccurate) their menus are. The benchmark will be the menu that's freely available, at this very moment, on the restaurant's official website.

In canning the snark, offering fair criticism, and letting the results speak for themselves, Eater made it possible for the menu sites to respond in a congenial fashion:

We saw you chose 11 Madison Park this morning to do a menu comparison and our menu was out of date. To be fair, we waited to let you investigate the differences before we updated the menu, even though we noticed the menu had changed. In any event, now that you've written your piece, we have updated the menu as we do for restaurants everyday. We have a team specifically assigned to update menus and we receive user submissions as well to let us know about restaurant changes.

The end result? The situation improved for everyone. A small improvement perhaps, but MenuGate is an ongoing Eater feature so we can expect future improvements. And perhaps when the menu sites get tired of taking their lumps each time around, MenuGate may lead them to think of better ways to keep their menus up-to-date and useful. Anil Dash wrote a post two years ago about how bloggers could take positive action against "Stuff That Sucks":

I'm proud of what [bloggers have] done in creating so many different weblog communities, and I don't want our legacy to be one of having the positives overshadowed by our frequent, though understandable, tendency to be unkind or uncivil to those we're communicating with.

The way Eater has approached the menu sites issue is certainly a good example of what Anil was talking about. Good show.

 Oct 2, 2006  •  tags: food nyc restaurants eater weblog anildash

Anil Dash writes that "intellectual dishonesty is a powerful tool, and should only be used in service of important and valuable causes". He and I have argued about this before so I think he's being serious (and not just being intellectually dishonest), but I think that statement is pure horse plop.

 Aug 10, 2006  •  tags: anildash

Anil Dash interviews Paul Lindner, one of the creators of Gopher and current Six Apart employee. Here's my recent post on Gopher.

 Aug 7, 2006  •  tags: gopher paullindner sixapart interviews anildash

You can use iTunes and a little AppleScript to make custom ringtones for Mail.app. I could have it play When Doves Cry everytime I get email from Anil.

 May 5, 2006  •  tags: anildash music ringtones email mailapp itunes osx apple

Over the past two weeks, David Jacobs, Anil Dash and I have attempted to reproduce (in some halting way) Jason Kottke, while the actual Jason Kottke was in rehab on his honeymoon. The attempt, on my part at least, has been an abject failure. Or haven't you noticed all the crappy links with "GK" at the end of them? Go-kart magazines? What the hell?

Like most of the disasters I've had a hand in, I've got a theory that both explains what happened and exonerates me. Ducking responsibility sounds better if you put on academic airs about it.

The theory: There are two kinds of bloggers, referential and experiential. Kottke is one. I, now two weeks too late in realizing this, am another.

The referential blogger uses the link as his fundamental unit of currency, building posts around ideas and experiences spawned elsewhere: Look at this. Referential bloggers are reporters, delivering pointers to and snippets of information, insight or entertainment happening out there, on the Intraweb. They can, and do, add their own information, insight and entertainment to the links they unearth -- extrapolations, juxtapositions, even lengthy and personal anecdotes -- but the outward direction of their focus remains their distinguishing feature.

The experiential blogger is inwardly directed, drawing entries from personal experience and opinion: How about this. They are storytellers (and/or bores), drawing whatever they have to offer from their own perspective. They can, and do, add links to supporting or explanatory information, even unique and undercited external sources. But their motivation, their impetus, comes from a desire to supply narrative, not reference it.

There's nothing here to imply that one type of blogger is better than the other. There are literally thousands -- OK, hundreds... OK, at least a dozen -- of both kinds that are valuable additions to the on-going conversation/food-fight/furry-cuddle that is the Internet. My point is that Jason Kottke is a very, very good referential blogger and I am a very, very bad one. And I'm sure I wouldn't have trouble finding a link that expresses this sentiment (many, many times over, with varying degrees of vehemence), but I'd rather say it from my own experience:

Welcome back, Jason. You've been missed. -- GK

 Apr 13, 2006  •  tags: weblogs gregknauss anildash davidjacobs jasonkottke kottkedotorg

Guests

I'm going to be away for a couple of weeks, but my pal Greg Knauss is taking over posting some remaindered links while I'm gone, aided by special guests David Jacobs and perhaps even Anil Dash.

Greg was the very first guest blogger here on kottke.org (and perhaps the first guest blogger ever anywhere) back in March of 2000 when I went to SXSW and they didn't have wifi at the conference (nor did I have a laptop). Good times, back then.

When I get back, house on fire.

 Mar 30, 2006  •  tags: weblogs kottkedotorg gregknauss anildash davidjacobs conferences sxsw

Anil on the conservatism of liberalism (by way of explaining why Craigslist is taking away everyone else's classifieds business). "A complete unwillingness to be critical, an almost astoundingly low set of criteria for acceptance -- these aren't the traits that encourage a community or a culture to improve."

 Feb 6, 2006  •  tags: craigslist anildash journalism

Anil Dash offers a list of dos and don'ts for beating the iPod and iTunes.

 Jan 6, 2006  •  tags: ipod apple itunes anildash lists business

You can now post from Microsoft Word to your Blogger blog. More interesting to me is how former Pyra folks remember this old idea. Matt says it was "something we talked about building back when the blogger api was brand new" and that Anil Dash, then a Blogger enthusiast, knocked up a working prototype (which I also remember). Ev says it's "a product that I first thought about five years ago". Both accounts are no doubt accurate, but how they're remembered is interesting.

 Aug 18, 2005  •  tags: blogger pyra word microsoft weblogs matthaughey anildash evanwilliams

I'm off on vacation for a few days, but Anil Dash and David Jacobs will be posting remaindered links while I'm gone.

 May 20, 2005  •  tags: travel kottkedotorg anildash davidjacobs