Origin: Early search engines

Webmasters and content providers began optimizing sites for search engines in the mid-1990s, as the first search engines were cataloging the early Web. Initially, all a webmaster needed to do was submit a site to the various engines which would run spiders, programs that "crawled" a page and stored the collected data in a database.

By 1996, SEO related email spam was commonplace.[2][3] The earliest known use of the phrase "search engine optimization" was a spam posted on Usenet on July 26, 1997.[4]

The process involves a search engine spider downloading a page and storing it on the search engine's own server, where a second program, known as an indexer, extracts various information about the page, such as the words it contains and where these are located, as well as any weight for specific words, as well as any and all links the page contains, which are then placed into a scheduler for crawling at a later date.

At first, search engines were supplied with information about pages by the webmasters themselves. Early versions of search algorithms relied on webmaster-provided information such as the keyword meta tag, or index files in engines like ALIWEB. Meta-tags provided a guide to each page's content. But indexing pages based upon meta data was found to be less than reliable, mostly because webmasters abused meta tags by including keywords that had nothing to do with the content of their pages, to artificially increase page impressions for their Website and increase their Ad Revenue. Cost Per Impression was at the time the common means of monetizing content websites. Inaccurate, incomplete, and inconsistent meta data in meta tags caused pages to rank for irrelevant searches, and fail to rank for relevant searches. [5] Search engines responded by developing more complex ranking algorithms, taking into account additional factors including:

Text within the title tag
Domain name
URL directories and file names
HTML tags: headings, emphasized () and strongly emphasized () text
Term frequency, both in the document and globally, often misunderstood and mistakenly referred to as Keyword density
Keyword proximity
Keyword adjacency
Keyword sequence
Alt attributes for images
Text within NOFRAMES tags
Web content development
Sitemaps
Today the only major search engine that says it considers meta keywords in its ranking algorithms is Yahoo, though most experts feel that even there the attention paid to meta keywords is minimal.[citation needed] Explicit facts about the effectives of meta keywords are, however, not readily known, because of the secrecy used during the ranking of algorithms by the search engines. One could therefore recommend the use of meta keywords in webpages, and limit them to 20 keywords or less.[citation needed] For example, the source code of this page shows that Wikipedia uses meta keywords. The "description" tag is, however, claimed by most SEO-experts to be more important. One could therefore say to use both meta tags in webpages.

Web content providers also manipulated a number of attributes within the HTML source of a page in an attempt to rank well in search engines.[6]

By relying extensively on factors that were still within the webmasters' exclusive control, search engines continued to suffer from abuse and ranking manipulation. In order to provide better results to their users, search engines had to adapt to ensure their SERPs showed the most relevant search results, rather than useless pages stuffed with numerous keywords by unscrupulous webmasters using a bait-and-switch lure to display unrelated web pages. This led to the rise of a new kind of search engine.