Tech Industry News
Technology, Widgetry, Gadgetry and Controversy Online and Off

Could Digg Do Something About Crashing Other Websites?

The strangest thing about – which sets it apart from all other social networking sites – is how much load it sends to newly popular articles. On Reddit, things crawl slowly up the ladder and get more and more hits along the way. On StumbleUpon, referrals are space out over time, reducing overall load even if sending more hits than Digg. Couldn’t Digg introduce more intermediate steps to deal with this problem? Semi-popular stories pages that also get lots of views?

Take this site for example – taken down three days in a row by making the front page of Digg. Lots of visitors never made it to the site as a result, which benefits neither party: the site loses out on readers while the Diggers lose out on content. Duggmirror can deal with some, but not all of these issues.

This story illustrates the problem rather ironically. It concerns a site named for its pride in scalability that couldn’t stand up to being on the Digg front page in the middle of a busy day!


5 Responses to “Could Digg Do Something About Crashing Other Websites?”

  1. This happens ALL the time. The only option is to have digg redirect to a page explaining the site is down to please bookmark it”s URL and try later.

  2. I don’t think digg could do anything because limiting how you can visit a site is not possible, and there would be modifications to stop this. StumbleUpon doesn’t sent visitors gradually over time by their own choosing, it just sends visitors according to their preference of content, and not everyone will see everything that is available – they only see what’s there and then.

    Digg has a similar type feature, basically you can always go back in a day, week, month or even a year back and check out that article you missed. Ultimately it’s up to the reader on whether they take that initiative or not. A while back I created a little php script that would store all sites that went offline on Digg, and would check every day if the site was back – if it was, it would cache a copy – and e-mail me with a link to the cache (which was on my server).

  3. […] Could Digg Do Something About Crashing Other Websites? The strangest thing about – which sets it apart from all other social networking sites – is how much load it […] […]

  4. There are a couple of things that can be done. But first, let me clarify a couple of points.

    First off, this is officially known as The Slashdot Effect (, so it’s not *that* “strange”. Let’s call the larger site (i.e. Digg, Slashdot, NYTimes, Curbed, etc.) the “indexer”, and the site that hosts the content the “destination site”.

    *Any* indexing site that has a huge sustained daily traffic volume can overload a small site with a direct link on the front page. Doesn’t have to be a juggernaut like Digg or Slashdot. Case in point: several months ago, I created a PR piece that made it to the front page of, a real estate newsblog in New York City. Within *minutes*, our admin was calling me asking what the heck was up with the 40X traffic increase 😉 My boss was happy – the admin wasn’t. Server slowed to a crawl, photos weren’t loading, etc. I learned a lesson that day 😉

    If the destination site is *intentionally* releasing a content item that is expected to generate a lot of attention, they would do well to call their webhost and request a temporary increase in bandwidth capacity, i.e. upgrade their service plan by 1-2 levels. Since traffic will fade eventually, they won’t have to go over budget that much.
    If the destination site is in the habit of generating news stories that get attention, they should pay attention to their net stats, as well as check the rankings of their stories if they appear on an indexer. Digg has a “waterfall effect” – things flow smoothly until they hit a point (i.e. link gets on the front page), then become a mess. The way to guard against this is to monitor the story’s rank, and when you see it approaching the front page – increase the bandwidth on your host, or mirror the content somewhere.
    IMHO, it’s not Digg’s responsibility to mirror content or control traffic. That burden rests squarely on the shoulders of the content creator, and their webhost admin.

  5. […] phrased), and was buried right upon hitting the front page. It concerned a Digg feature that would help both Diggers and site owners, but the headline was phrased in such a way that Diggers thought it was an attack on Digg. Oh […]

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: