How Does Search Engine Works?

The term "search engine" is often used generically to describe both crawler-based search engines and human-powered directories. These two types of search engines gather their listings in radically different ways.

Search Engine are the special sites on the web that are designed to help people find information stored on that site. Here are differences in the ways various search engines work, but they all perform three basic tasks:

1. They search the Internet or select pieces of the Internet based on important words.
2. They keep an index of the words they find, and where they find them.
3. They allow users to look for words or combination's of words found in that index. 

The spider returns to the sites in its index on a regular basis, scanning for any changes. How often the spider returns is up to the search engines to decide. Website owners do have some control in how often a spider visits their site by making use of a robot.txt file.

Spiders start their journeys with a list of page URLs that have previously been added to their index database. As it visits these pages, crawling the code and copy, it adds new pages links that it finds on the page to its index.