Back to Search Engine Basics: How Search Engines Work (Crawling and Indexing)
Last time I discussed the two factors search engines use to rank websites and thus produce search results. Before the relevance and popularity of a website is taken into account though there are other steps that must be taken by search engines in order to get the required data and process them. It is important to understand how search engines work in order to realise how and why search engine optimisation is important. It is also important if you want to understand the methods used for search engine optimisation.
There are four basic steps to explain how search engines work.
Step 1: Crawling – Search engines get information about websites through crawling. Crawling is done by search engine spiders/bots, which are simply programs designed to wander through cyberspace in order to perform the next step – Indexing. Crawling is done repeatedly in order to update the search engines’ index. In a previous post of mine (Are All Your Webpages Search Engine Spider Friendly? I & II), I mentioned instances wherein a web page might be missed by a search engine spider. To ensure that your webpages can be crawled please reread that post.
Step 2: Indexing – Once search engine spiders crawl a web page the contents of that page is indexed or stored in the search engine’s database. Some pages though crawled, however, are not indexed. The reasons for this varies, from a website being banned to poor web design practices resulting in the spider being unable to view/determine the contents of a web page. To ensure that all the contents in your website that needs to be indexed are, it is best to hire the services of a professional SEO consultant.
to be continued…
Popularity: 9% [?]
Similar Posts:


0845 838 7448 
No comments yet.