Archive for the ‘Architecture’ Category

Of DDoSs and SPOFs: How Twitter and Facebook violate “The Internet Way”

Thursday, September 24th, 2009

As Twitter experienced yet another period of severe slowness the other week I found myself thinking of the Distributed Denial of Service (DDoS) that took out Twitter, Facebook and a few other sites for a period of time last month, and found myself asking again:

Why do we continue to rely on services like Twitter and Facebook that violate the fundamental ways of the Internet?

Think about it for a moment…

Does the entire Web ever “go down“?

No, because the Web is distributed and decentralized.

You don’t have to ask anyone to set up a web server. You just go ahead and do it yourself – or use someone else’s service. You are not locked into any one company’s service. Now, a large web hosting provider may fail, taking hundreds or thousands of web sites offline, but the “Web” does not completely fail.

Does “e-mail” completely and entirely fail?

No, because email is distributed and decentralized.

You don’t have to ask anyone to set up an email server. You just go ahead and do it yourself – or use someone else’s service. You are not locked into any one company’s service. Now, a large email hosting provider, like GMail, may fail, taking millions of email accounts offline, but email in general does not completely fail.

Does Instant Messaging based on the open standards of XMPP/Jabber completely fail?

No, because Jabber/XMPP-based IM is distributed and decentralized.

You don’t have to ask anyone to set up an XMPP server. You just go ahead and do it yourself – or use someone else’s service. You are not locked into any one company’s service. Now, a large Jabber/XMPP provider, like GTalk, may fail, taking millions of IM accounts offline, but XMPP IM in general does not completely fail. (Note that the same is not true of proprietary IM systems such as AIM, Yahoo, MSN, etc.)

Does content syndication via RSS/Atom completely fail?

No, because RSS syndication is distributed and decentralized.

You don’t have to ask anyone to set up a RSS feed. You just go ahead and do it yourself – or use someone else’s service. You are not locked into any one company’s service. Now, a large content provider may fail, taking many RSS feeds offline, but RSS syndication in general does not completely fail.

Does Voice-over-IP based on the open standards of SIP completely fail?

No, because SIP-based VoIP is distributed and decentralized.

You don’t have to ask anyone to set up a SIP server. You just go ahead and do it yourself – or use someone else’s service. You are not locked into any one company’s service. Now, a large VoIP provider may fail, taking hundreds or thousands of VoIP accounts offline, but SIP-based VOIP in general does not completely fail.

DO YOU SENSE A RECURRING THEME HERE?

THE INTERNET WAY

The “Internet Way” is fundamentally about distributed and decentralized architectures and services. Note that both of those aspects are important. Skype is a massively distributed system, but as we found out with the multi-day disruption a few years back, it relies on centralized enrollment/authorization services.

Distributed and Decentralized.

And yet…

… here we all are heavily using and coming to rely on services such as Twitter and Facebook. Now don’t get me wrong – I’m a huge fan of Twitter and have been using it since back in late 2006. I’ve been on Facebook since around then and we do use Voxeo’s Facebook Page as another way of communicating with customers and developers.

But when Twitter starts throwing Fail Whales around, you can almost feel the collective twitching of the Twitterati (myself included) as they are unable to post updates. When Facebook starts rejecting updates or is slow to get into the site, millions of people start experiencing high blood pressure.

Here’s the problem:

Both Twitter and Facebook are Single Points of Failure (SPOFs).

They are single services operated by single companies. Yes, the services themselves may in fact be running on thousands of actual servers. Perhaps those servers are even spread out in multiple data centers. But at the end of the day, both services are:

Concentrated and Centralized

I can’t set up my own Twitter server. I can’t run my own copy of Facebook. Nor are there alternatives to the single sites/services. There’s not another Facebook-like site that I can go to and enjoy full interoperability with Facebook users.

So we are beholden to the direction of those companies… and their struggles… and their whims (like, oh, Facebook deciding to ditch regional networks… or Twitter randomly suspending a bunch of heavy users). We can’t innovate on the core service… sure, we can build apps that interact with the APIs provided by both services, but the fundamental core service is entirely controlled by the single company running the service.

Ultimately I see this as bad for us as users of the services. Without the open ability to control our own destiny, we are severely LOCKED into these services.

Now, to be fair, both Twitter and Facebook have been receptive to feedback from users and have made changes based on user feedback and suggestions. But at the end of the day, they are still SPOFs.

HOW DO WE GET DISTRIBUTED/DECENTRALIZED?

Good question.

On the Twitter side – for public microblogging, there are several potential options:

There’s also the open source code for Jaiku floating around, and any number of other projects.

The greatest challenge any of these services have is “discovery” – how do I find someone on the service? On the web, we’ve solved that issue with, well… Google. To move to a distributed architecture, you need some way for people to discover others. It’s easy with a centralized system because that system can run its own search. Far harder with a decentralized system.

Who knows, perhaps Twitter themselves may come up with some option. The point, after all, is not necessarily to replace Twitter, but rather to come up with a distributed and decentralized microblogging infrastructure. Twitter could easily be the “first among equals”… I might still choose to use Twitter.com, but I’d like a choice.

On the Facebook side, it’s a bit more of a challenge. Facebook really is the classic “walled garden” online service (which I wrote about 2.5 years ago), albeit with perhaps a few more holes in the walls that the classic services of a few decades ago. Facebook is quite complex – it’s far more than simple messaging like Twitter. It’s not clear to me that you could easily replicate the service in all its capabilities. But it would be great as a consumer/user if there was another choice… another Facebook-like service that had full interoperability with Facebook.

THE INTEROPERABILITY IMPERATIVE

In both these cases, Twitter and Facebook, you’ll note that the ability for those services to move to a distributed / decentralized architecture comes down to one word:

Interoperability

And for interoperability to occur, at the end of the day you need to have standards that multiple vendors can use. Either formal industry standards through an organization like the IETF – or “de facto” standards set by the marketplace. Either way, you need agreed-upon APIs, protocols, etc.

WILL WE GET THERE?

Probably not.

Or at least… not for a long while.

There is basically zero incentive for either Twitter or Facebook to open up and look at a distributed / decentralized architecture. Both are for-profit companies who have received substantial investments. Both have incredible momentum and are doing extremely well. We can argue about how it can be best in the long-term, how they can grow a lasting service, etc, etc…. but what’s in it for them?

And what it is in it for the hundreds of millions of users of the system who find the user experience so incredibly simple and easy? Sure, we can point out the dangers as I do here… but it’s so easy to use today…

The best we can do, I think – we who care about the issue – is continue to experiment… to develop… to architect… to promote alternatives… to make simple user experiences… to work on the discovery/directory issue so it’s drop-dead simple to find other people on services… to build the alternatives.

We have to. Back in the days of CompuServe, AOL, Prodigy, etc., it became clear that email was a form of communication that was powerful and useful – and needed to break out of the walled gardens. We collectively tore down those walls with SMTP and DNS.

A couple of decades later, we’ve found that “status messaging” or “microblogging” or whatever we want to call it is a powerful communication tool. We need now to tear down those walls and move it to a distributed and decentralized architecture.

It’s the Internet Way.

If you found this post interesting or helpful, please consider either subscribing via RSS, becoming a fan on Facebook, or following us on Twitter.

Tags: , ,
Posted in Architecture, Facebook, Internet, Twitter | 11 Comments »