Search Engine Spiders
"The search engine sends the spiders out into the internet to index web pages. Spiders are programmed by the search engines to bring back bits of information about the web page."
Search Engine Spiders
When researching search engine optimization knowing what a search engine spider is, and what they do is very important. Search engine’s sort through billions of web pages and index the content. So when a keyword search is conducted, the search engine’s algorithm sorts through the index, finds and displays the web pages that have the highest relevancy score.
What The Spiders Do
The search engine sends the spiders out into the internet to index web pages. Spiders are programmed by the search engines to bring back bits of information about the web page.
Examples of “Bits of Information”
- What the main words are within the webpage/website (Keyword density)
- What the main words are in the Web pages/websites title
- What text is used in the links (if any) that point to the webpage/website
The spiders are programmed to find hundreds of “Bits of Information” with in the webpage/website. This gathered information lets the search engine know how to index the webpage/website. The search engine can determine what webpage ranks where on the result listing, based on point values given to each bit of information.