Wednesday, 07-Sep-11 22:25:00 CEST

Dead blog posts rests in peace

I promised to keep an eye on how Google handles a transition like I made with this blog. It gives a kind of insight into how sites are crawled. At least this is interesting for the near future; Google changes things continuously so there is no great wisdom to carve into stone for the eternal future.

I started the blog on Blogger on 11th of June this year. I ported the blog to another platform on 28th of August. Regularly I check out Google Webmaster tools to see what it reports as missing pages. For missing pages I arrange permanent redirects to the new location. This way Googles search result will point to the moved page.

For June I had made 57 posts. Yesterday Google had reported that 27 of these were missing.  That is 47%.

For July I had made 94 posts. Yesterday Google had established that 87 of these were missing. That is 93%.

For August I had made 57 posts before the transition and yesterday Google had found out that 44 of these were missing. That is 77%.

All that were reported missing have their redirects done now. That is fine. Google should be proud of me.

What is more interesting to find out is the opposite figure. Namely, how many percent Google still have not find out about since 28th of August: There are 50 blog posts that Google still need to find out that they are missing. That is 24%. After ten days I am on a loss of 24 percent. I am not complaining, it just a lesson to be learned. A couple of years back I should have been deleting and not reflect about what is going on at Google.

These days everybody is surfing through Google so the entire experience on the web is integrated with Google. So Webmasters need to be nice to Google for their own sake.

So what happens when you delete a blog post? Google robots cannot find the page and sees a status 404. It is not so that the search result at Google connected to the vanished page is deleting directly because it is common that web servers are down for a while. My guess. Google keeps the missing page for a while. How long is that? The irritating thing about porting a blog where the internal structure is entirely replaced by a new structure that is that apparently it takes much longer for Google to find the start of the website again and figure out what it holds and where.

It is as if you create a website then you get one chance to let Google discover it quickly. When that is done and you are “known” then Google take it easy with the crawling.

I suspect that if Google found out that “hey, here is a page with the same content as that page that we cannot find anymore” is a bit complex to make the decision that ok it has been replaced. So you will end up with a dead page competing with a new replacement page. For as long as Google thinks that the dead page should be held in their indexes. I suppose they are so clever at Google to give up at some point and let pages die.