RSS Feed
Blog posts  Blog RSS

Moving the web forward: A look at Google's modernized indexer

Moving the web forward: A look at Google's modernized indexer
A review of the article "Understanding Webpages Better"

Posted By: Matt Wojas
Job Title: Web Developer

Link to article: Understanding Webpages Better

The web has been evolving in tandem with technology at considerable speeds over the last few years. Looking back to websites pre-2000 era we are reminded of very different times.  Standards were different, design has become responsive and simplified stylistically, and most pertinently the very way we as developers use the tools of the web has drastically changed.  Advances in memory utilization and storage, telecommunications mediums, content delivery networks just to name a few have enabled much more involved, rich, and dynamic web browsing experiences.  Namely javascript has gone from basic tasks to become heavily relied upon to both serve and control content as well as manipulate and generate design/layout.  

Historically search spiders have not been able to parse javascript.  Best practice has been to exclude javascript files from the bots radars via a robots.txt file.  The purpose of this was to save the bots needlessly indexing content it could not use, and restrict their concerns strictly with textual content. Because so much of our online experience is now mediated via javascripting, the potential for bots and crawlers (user agents whose purpose is to index the web for search engines) to miss out on content has become a serious issue.

Google is making an update to its Google bot algorithms to enable consumption of javascript files (as well as CSS) and various other improvements.  This is great news as it means content will be represented in a more modern and realistic way to the indexer.  There are of course some issues with this but the Google webmaster blog post outlines some best practices for how to configure your web pages to be best served to Google bots for indexing purposes.

The onus isn’t too heavy for the developer (basic tips are to let Google index javascript and CSS files rather than excluding bots from the robots.txt file, and make sites which degrade gracefully -  which should already be part of the development process).  These steps should mean better indexed sites and more meaningful search results.

Photo credit: Luna, Carlos via Flickr 
Creative Commons Attribution 2.0 Generic (CC BY 2.0)
Posted by Maddison on 5/26/2014 11:03:25 AM

Trackback URL: https://akanewmedia.com/trackback/9af65aa5-34dc-43fd-8fec-3196a1e2b458/Moving-the-web-forward-A-look-at-Google-s-modernized-indexer.aspx?culture=en-US

Blog post currently doesn't have any comments.
Leave comment

Is nine < than two? (true/false)


Let's Talk
we deliver results that we can all take pride in.

Interactive & Branding Solutions

Jassmin Nicoloff
Business Development Associate
416 323 1981 X 233

Online & Mobile Fundraising Platform
Haley Clift 
Business Development Director
416 323 1981 X 231

Human Resources

A.K.A. New Media Inc.
20 Maud Street, Suite 100
Toronto, Ontario, M5V 2M5

T    416 323 1981
F    416 323 1982
TF  866 766 3336

Top of Page