Tech talk, social media, blogging, computing tips and tricks

Force Google to spider your site

February 3rd, 2010 by David Bradley >> 4 Comments

If you have a static website, one with content that doesn’t change often and that has no RSS newsfeed (so not a blog), you’ve probably noticed (if you care about these things) that Google doesn’t bother to visit, or spider, your static that often.

After an initial flurry of activity when you first created the site, the search engine will have recognized that nothing changed between consecutive visits and will have settled itself down to visiting just once a week, month, every few months, or whatever…what happens next is that Google may consider your static site to be unimportant to its users and lower your ranking in the in its search engine results pages (SERPs).

With the handful of static sites I manage, I’ve not actually seen this de-ranking effect, but it may happen so a new feature in Google’s online RSS Reader tool could be of interest. Google has added the capability to “create” a virtual feed for any static site and add it to the site updates you follow and let you know of any changes should they ever happen on the site.

Logically, that means that even though a site may be static, by adding it to your reader, you are telling Google to spider your site on a regular if not frequent basis nevertheless. After all, if it were not doing so it would not be able to give you an RSS alert to let you know that the site had been changed.

Whether this trick actually has any benefits for sites that have not suffered de-ranking despite being static remains to be seen. But, it’s worth a try. It would also give you a useful way of keeping an eye on other people’s backburner sites you don’t visit often. You could even check whether any of the sites you run have been changed by hackers. That said, more acute “zero-day” observation of those sites would make more sense if they are at all important to you despite being static.

Thanks to the omnipresent and seemingly omniscient Ann Smarty for the tip off to this new feature. Brian Shih made the official announcement of this feature on 25th January in the Google Reader Blog.


Leave a comment ↓

  • David Bradley // Feb 3, 2010 at 9:59 am

    Belated hat tip to Heidi Cool for spotting the original Smarty post…

  • uncoolbob // Feb 3, 2010 at 10:21 am

    It doesn’t necessarily follow that the same spider is used to update the search index and monitor pages for the RSS feed. And even if it was, and there were no changes being detected, wouldn’t the “importance” of the page/site continue to drop?

  • David Bradley // Feb 3, 2010 at 10:39 am

    Yes. Agreed. On both counts. I advise all my web clients to have active sites first off if they want any kind of traffic. But, like I say I do have a couple that have been the same for years and still rank on page 1 in Google for their keywords, so maybe this is a total red herring. It’s definitely useful for keeping an eye on other people’s static sites just in case things do change though.

  • pens parker // Feb 6, 2010 at 8:09 pm

    You can also request google to reassess your site through webmaster tools that amy help too. But also change the content on you site a bit particularly on your home page google wants to see an update and constantly improving site with new pages added