Archive for September, 2005

Blog Pings are stupid

Wednesday, September 28th, 2005

Kevin Burton Calls it a Pending Ping Crisis (Via Chris). I believe that Blog Pings are just a bad idea.

I only see two groups who derive any real value from pings, the Content Publishers (or ‘Bloggers’) and the Content Aggregators (Including Search Engines).

I believe that neither of these groups can derive significant value to running a ping service. A service takes money to operate. To spend money, you have to make it somehow, and you can’t make money by redistributing packets over the world, and not charge anyone for it. If its a free service, your company still needs to derive value from it. There is value to users using your website. There is not any value in you telling everyone about their website for free.

For the Content Publishers, the value of pinging is that you make sure everyone you care about knows about your new content. But, if they really care, they will be crawling your RSS Feed anyways, because pings are not reliable. As a content Aggregator, you cannot just trust pings. If for whatever reason you miss a ping, regardless of it was your fault or the pingers fault, you just missed content, which is the Cardinal Sin of an Aggregator. People want their content, and any failure to reliably Aggregate content will drive people away from your service.

For the Content Aggregators, Pingging doesn’t provide many advantages. Google already indexes my RSS Feed every 15 minutes. Bloglines tries to do it once an hour. Even if you could update your search index in under an hour, no one cares, yet. The speed that information arrives to most people hasn’t become critical. If I read an article at 5:45 or 6:30, it doesn’t matter much. A full 24 hours, yes, that can matter, but the times we are talking about are already sub-1-hour. I believe there isn’t much value to getting the data a few minutes earlier.

For a Content Aggregators, aka the people who make any money off this, there is little value from Pinging, because you can’t trust the pings, and because your infrastructure is already built to crawl every RSS Feed ever, every hour. But, these Aggregators are the only ones who derive enough value from pinging, that one of them might try running a ping service, except for one major problem.

The content publishers want lots of people to know about their Feeds. They don’t want only Yahoo or just Ask Jeeves to know. They want all the people they care about. That means your ping service needs to send pings out to everyone, and guess what, everyone includes your direct competition. And important people don’t like their company helping their competitors, for free.

This isn’t a new type of problem. In the conventional world, this is where the government steps in. They build roads, goto war, and build other public works/utilities. Because while the individual value to the individual user or corporation is minimal, the combined value is potentially huge. The problem is there is no utility on the internet. No group to take responsibility to ensure some things just work. Of course, everyone knows that roads aren’t free, we all pay taxes.

But wait, the internet is awesome you say! Its special and we can solve these problems!

Guess what, there’s more!

Smart people already solved this problem. Its called reliable IP multicasting. It is a whole different subject, that of replacing the current ping system, but it could be done. There are much much much much much better systems and designs for a single entity to distribute reliably a message to hundreds or thousands of other nodes, without causing excessive load on the system. I am sure someone has a PHD thesis out there on it too.

In closing my point is that the economics of pure blog ping services will never work. There is not enough value to either of the major groups to support these services. This means in the long run, all of them will die. I believe there could be a technological solution, but it would be a massive departure from our current system, and would require actual brain power to make it work. It isn’t impossible, but services that require brain power to work have a tendency to not succeed on the Internet. This means we will walk around in our crap with some groups trying to make ping services work, until they collapse and we can all wait for more groups try to reinvent them as Web 6.0 in a few years.

Need Beta Testers

Sunday, September 25th, 2005

Okay, so I don’t have a cool form embedded into this blog entry asking for your email address to get sent an email message about a super-secret and revolutionary Web 3.0 product, BUT, I really do need people to help testing.

Apache HTTPD 2.1.8-BETA is available for testing. I would love feedback, both positive and negative. Please install it on your production servers, see what breaks. It wouldn’t hurt to try some of the new features too.

One of the coolest features that was just recently added is graceful-stop. This allows you to drain connections to your web server, and do an upgrade without any downtime. Thanks to Colm for adding it.

Full ChangeLog: CHANGES_2.1

blah blah blah

Tuesday, September 20th, 2005

Jono Bacon has an interesting post on O’Reilly called ‘Opening the potential of OpenOffice.org‘ (Via Colm).

I was going to write a huge post on why he is right and wrong on many points, but I am way too tired and lazy. The basic point is that OpenOffice isn’t unique, even though he tries to assert it since the code base is soooo huge.

The reality is that many major open source projects shifted to longer development cycles, and are learning that its bad (FreeBSD and Apache HTTPD come to mind).

FreeBSD has learned their lesson, and are back on track for 6-8 month release cycles with 6.xx. Long releases do more than increase the feedback cycle, but they discourage new developers. New developers like to see users using their stuff. It is a gratification thing. Every open source project needs a constant flow of new developers. Projects that have a hard time getting new developers…. die.

Apache HTTPD is re-learning this lesson. The 2.1.xx branch has been around for years, never being released to users. Lucky for you, 2.1.7-beta is available now. I hope to get apache into a 6-12 month stable/dev cycle. I think that this type of cycle will increase user feedback, and more importantly encourage new developers to get involved.

ENOMEM?

Monday, September 19th, 2005

Dear Lazy Web,

Why does calling pthread_mutex_lock and pthread_mutex_unlock in a tight loop cause pthread_create to error out with ENOMEM?

I could accept something is wrong with my code changes, but by removing the lock/unlock around the shared variables it works correctly.(*) The lock/unlock are in tight loops, but putting them in doesn’t change the flow of the program at all. Any ideas on how this could be causing a pthread_create to fail with ENOMEM?

* Correctly. It crashes randomly when starting because the variables aren’t locked correctly, but sometimes it finishes.

I would like to blame linux somehow. I wish this code ran on freebsd so I could…. But testing is for wimps anyways.

Thanks,

Paul

YARR

Sunday, September 18th, 2005

< rik> remember

< rik> tomorrow is International Talk Like A Pirate Day.

< DrBacchus> Who could forget?!

< chipig> tomorrow.

< fajita> tomorrow is “Kill A Spammer” day. I expect you all to participate.

< rik> hahah

< chipig> YARR!

IRC Bots are so smart.

Herbal Essences

Friday, September 9th, 2005

I have been using Herbal Essences for the past 3 days, but I haven’t found any females screaming “Yes, Yes Yessss” in my bathroom yet. Is there a secret code to turning that feature on?

Async IO in HTTPD 2.3

Monday, September 5th, 2005

In the last couple days ideas and code for doing asynchronous writes to clients has started to flow. Brian Pane has posted a cool diagram of how the different connection states might work. On Saturday I created a subversion branch at https://svn.apache.org/repos/asf/httpd/httpd/branches/async-dev/ to house the development. If anyone is interested in helping shape Async IO development, please go and criticize our plans on dev@httpd.

I hope to have time to post a more detailed description of the work and ideas on this blog…. Maybe later this week.

Now Running Apache 2

Friday, September 2nd, 2005

Bloglines was upgraded to Apache 2.0 today. Woohoo.