Users of the internet today can use a search engine to search for any number of differently phrased words related to any topic they like, and the search engine will produce a huge amount of information from a wide variety of sources. However, the internet hasn’t always been so easy to navigate, as this post will show you!

1945

Many Internet guru’s credit Vannevar Bush and his article “As We May Think” with the introduction of the idea of memory extensions and hypertexts. In his article, which was published in The Atlantic Monthly, Bush urged scientist to work together in order to build a body of knowledge which would be accessible by all mankind. Bush was avid about storing data, and how important it was to store data, but he also strongly believed that for data to be useful to us it should represent how the mind works (to the best of its ability). At the end of his article, Bush talked about a system he called the “memex” – a way of storing and retrieving data that was fast, reliable, limitless, and efficient.

The 1960s

Despite the popularity of Vannevar Bush’s article, the title of father of modern search technology is usually bestowed on Gerard Salton. Salton was a Professor of Computer Science at Cornell University from 1968 until his death in 1995, and during his time at Cornell, he developed the SMART (Salton’s Magic Automatic Retriever of Text) system. This was an informational retrieval system that introduced important concepts such as Inverse Document Frequency (IDF), Term Frequency (TF), and relevancy feedback mechanisms. In 1987, he released his book “The Theory of Indexing” which explains many of the tests on which his theory was based – and which are still used in search today.

Also created in the 1960s is Project Xanadu, which was created by Ted Nelson in order to create a computer network with a simple user interface. Nelson used traditional HTML code that is associated with many problems such as broken links and complex markup code – which may be one reason Project Xanadu never took off. It did, however, coin the phrase “hypertext” – which is still used today.

1990

The first real search engine was “Archie”, which was created by Alan Emtage in 1990. Emtage was a student at McGill University in Montreal at the time and created Archie by combining a script-based data gatherer with a regular expression matcher in order to retrieve file names by matching a user query. Archie (shortened from Archives) was a database of web filenames that could then be matched with user’s queries. However, as Archie was an FTP site it only had limited space, which meant that only the listings were available and not the actual contents of the sites.

1991

As Archie became more popular, and more people heard about it and used it, other universities began to develop their versions.   In 1991, “Veronica “ was created by the University of Nevada Computing Services Group to work on plain text files. “Jughead” was another popular interface system with a similar purpose to Veronica, and both of these interfaces searched file names and titles which were stored in the “Gopher” index system. Gopher was created as an alternative to “Archie”, by Mark McCahill at the University of Minnesota in 1991.

1992

This was the year that Tim Berners-Lee, who is credited as being the inventor of the World Wide Web, first appeared on the scene! In the late 1980s, he joined hypertext with the Internet, and the World Wide Web that we know and love was born. In 1992, he created a virtual library, known as “VLib” which used a CERN webserver to host a list of web servers – it is basically the oldest catalogue of the web available.

1993

With the introduction of the World Wide Web, comes the introduction of the first web robot. The World Wide Web Wanderer was created by Matthew Grey in 1993 as a way of measuring the growth of the web. The purpose of the robot was to count active web servers originally, but this was soon upgraded to capture actual URLs – with the database of URLs being known as “Wandex”. However, the Wanderer became more of a problem than a solution, like the fact that it accessed the same page hundreds of times a day caused system lags that made people question the usefulness of a bot long after Grey had fixed the issue.

Also in 1993, “ALIWEB” was born. Created by Martijn Koster, a Dutch software engineer, with the purpose of crawling Meta information and allowing users to submit the pages they wished to be indexed (along with a description). ALIWEB did well as it didn’t use a bot to collect the data, and so it didn’t use excessive bandwidth. However, it did struggle slightly as people weren’t sure how to submit their site.

By the end of 1993, there were three primitive web search engines that had surfaced on the web. “JumpStation”, “The World Wide Web Worm” and “RBSE Spider”. “JumpStation” gave you information about the page’s titles and header by using a simple linear search. “The World Wide Web Worm” indexed both titles and URLs and listed the results in the order they were found. The “RBSE” spider did have a rating system that was based on relevancy to the keyword string. However, as none of these search engines did adequate link analysis or cache full page content, it was still extremely difficult to find anything unless you knew the exact name of what you were looking for.

1994

There were a huge number of developments in 1994, including the launches of “Infoseek”, “AltaVista”, “Galaxy”, “WebCrawler”, “Lycos” and “Yahoo”. All of them contributed greatly to the development of the search engine, but as “Yahoo” is one of the top three search engines today, we will focus on that.

The “Yahoo” directory was created in April 1994 by David Filo and Jerry Yang. Originally called “Jerry and David’s Guide to the World Wide Web” it was created as a collection of their favourite websites. As the number of links they were collating started to grow, they had to reorganise it and make it more of a searchable directory. What set the “Yahoo!” directory above the others was that it included a human-compiled description with each URL listed? As the directory expanded, they began to charge commercial websites a premium to be included in their listings, and there was sometimes a long wait for the link to be live. “Yahoo!” didn’t start work on their search engine until about 2002, as previously to this they had outsourced their search services. However, with the purchase of “Overture”, they began to work on their search engine. “Yahoo!” is now one of the internet’s leading brands, and due to its partnerships with various telecommunications firms, it is one of the most trafficked networks on the internet.

Between 1995 and 1998, many other leaps and bounds in the history of search engines occurred with the development of such engines as “Looksmart”, “Ask Jeeves” and “MSN” but the next BIG thing in the world of Search Engine occurs in 1998 when “Google” launches.

1998

In 1995 Sergey Brin, a 22-year-old studying at Stanford, is asked to show around Larry Page, a 21-year-old the University of Michigan Graduate who is considering the School. A year later, they are collaborating on the introduction of a search engine called “BackRub” which can analyse the backlinks that are pointing to a website. “BackRub” ranked pages using a ranking based on the academic notion of the citation so if someone linked to a website it was seen as a citation i.e. it was important to that person. In their PageRank Algorithm, links acted as votes but some votes count more than others. Brin and Page registered “Google.com” as a domain in September 1997 apparently as a play on the mathematical term “googol” to explain their mission to easily organise the huge amount of information available on the World Wide Web.

Google Inc. officially launches in September 1998, with Page, Brin and their first employee Craig Silverstein operating out of a garage in California. In December 1998, “PC Magazine” lists Google as the Top Search Engine of Choice in their “Top 100 Websites for 1998” list. The unique and ever-improving algorithm used by Google has kept it as one of the most popular search engines still used today.

Also launched in 1998 was Overture, which is widely seen as the father of paid search. It was started by Bill Gross under the name of GoTo with the aim of getting risk free traffic streams and then selling them with a certain level of accountability. Therefore, Overture is widely credited as the first company to provide a pay per click placement search service. Although Overture was really successful at first, its growth was somewhat stunted by a couple of things. Firstly, Overture did not allow itself to grow into a search destination, which was the opposite decision to the one Google made. For example, AOL selected Google as its ad partner, despite Google trying to grow their own brand too, and this pretty much brought about the end of Overture. The other mistake Overture made was to not factor ad clickthrough rates into the ad costs – which affected the relevance of the ads and the ad network efficiency. Overture bought AllTheWeb and AltaVista in 2003, but this wasn’t enough to save them – and they sold out to Yahoo in July 2003 for the princely sum of $1.3 billion

1999

A year after the launch of Google and Overture, came the launch of AllTheWeb. It was launched by a company called Fast, in order to showcase their talents in search technology. People warmed to AllTheWeb quickly as it had a sleek user interface, along with rich advanced search features. It grew in popularity so much, that in February of 2003 it was bought by Overture for 70 million dollars. Overture was eventually bought out by Yahoo, who rolled some of the AllTheWeb technology into their search engine Yahoo! Search, and rumour has it they still occasionally use AllTheWeb as a testing platform.

2000

Another big development occurred in 2000 when Google launched AdWords. Originally launched as a flat Cost per Impression (CPM) model, it was a bit of a failure due to the fact that some keywords were overpriced and some were underpriced! Google took another stab at it in 2003, when it launched a newer service based on the Overture model, with the addition of ad clickthrough rate as a factor in the rankings. AdWords quickly grew in popularity due to its ease of use and precise targeting which made it simpler for people to make great profits.

Google continues to tweak AdWords to this day. For example, they have disallowed the double service of ads to the same website in order to keep their ads as relevant as possible. They have also started to look at landing page quality and establishing quality-based minimum pricing too.

2003

Google made another big announcement in 2003 when it launched AdSense. This is Google’s content-targeted advertisement network, which allows web publishers to automate the placement of relevant adverts on their content. Advertisers can choose which keywords they want to target, and which ad formats they want to market whilst allowing publishers to place ads in their feeds or in certain cases in emails.

Google has recently added an option where advertisers can opt-out of placing ads on their content sites, in order to prevent the erosion in the value of search ads. They have also introduced smart pricing which adjusts the cost of clicking on an ad based on what they perceive the value of a click from that page to be worth! For example, a click-through from a page which is a review of a certain vacuum cleaner would be worth more than a page full of pictures of vacuum cleaners.

2004

MSN Search was launched in 1998, but they didn’t actually get serious about the search until 2004 when they launched the preview of their live search service. Prior to this, they had been relying on Overture, Looksmart and Inktomi to provide their search results. They then switched from Yahoo’s organic search results to their own technology in 2005, and got rid of the Yahoo! Search ad programme in 2006. Their Live Search product officially launched in May 2006.

2009+

In June 2009, MSN changed the face of search by launching Bing. This was different from all other search engines at that time, as it placed inline search suggestions directly in the search results. For example, if you type in “bank account” it would automatically suggest relevant phrases such as types of bank account, bank accounts for those with bad credit, advice on bank accounts, etc.

Modern search engines are quite frankly incredible, they use complex algorithms to take your search query and not only return a result which is pretty accurate, but they also manage to find and present you with these valuable nuggets of information from out of a vast library of data which is seemingly never-ending.

Search engines have come a long way since their beginnings back in 1945, and they are pretty incredible. However, the different and ever-changing algorithms used by each search engine can be quite confusing. If you need any help with SEO services UK then why not check out our website, or give us a call on 01625 238 770?

SaveSave

What Everyone Needs to Know About the History of Search Engines

  /  

Users of the internet today can use a search engine to search for any number of differently phrased words related to any topic they like, and the search engine will produce a huge amount of information from a wide variety of sources. However, the internet hasn’t always been so easy to navigate, as this post will show you!

1945

Many Internet guru’s credit Vannevar Bush and his article “As We May Think” with the introduction of the idea of memory extensions and hypertexts. In his article, which was published in The Atlantic Monthly, Bush urged scientist to work together in order to build a body of knowledge which would be accessible by all mankind. Bush was avid about storing data, and how important it was to store data, but he also strongly believed that for data to be useful to us it should represent how the mind works (to the best of its ability). At the end of his article, Bush talked about a system he called the “memex” – a way of storing and retrieving data that was fast, reliable, limitless, and efficient.

The 1960s

Despite the popularity of Vannevar Bush’s article, the title of father of modern search technology is usually bestowed on Gerard Salton. Salton was a Professor of Computer Science at Cornell University from 1968 until his death in 1995, and during his time at Cornell, he developed the SMART (Salton’s Magic Automatic Retriever of Text) system. This was an informational retrieval system that introduced important concepts such as Inverse Document Frequency (IDF), Term Frequency (TF), and relevancy feedback mechanisms. In 1987, he released his book “The Theory of Indexing” which explains many of the tests on which his theory was based – and which are still used in search today.

Also created in the 1960s is Project Xanadu, which was created by Ted Nelson in order to create a computer network with a simple user interface. Nelson used traditional HTML code that is associated with many problems such as broken links and complex markup code – which may be one reason Project Xanadu never took off. It did, however, coin the phrase “hypertext” – which is still used today.

1990

The first real search engine was “Archie”, which was created by Alan Emtage in 1990. Emtage was a student at McGill University in Montreal at the time and created Archie by combining a script-based data gatherer with a regular expression matcher in order to retrieve file names by matching a user query. Archie (shortened from Archives) was a database of web filenames that could then be matched with user’s queries. However, as Archie was an FTP site it only had limited space, which meant that only the listings were available and not the actual contents of the sites.

1991

As Archie became more popular, and more people heard about it and used it, other universities began to develop their versions.   In 1991, “Veronica “ was created by the University of Nevada Computing Services Group to work on plain text files. “Jughead” was another popular interface system with a similar purpose to Veronica, and both of these interfaces searched file names and titles which were stored in the “Gopher” index system. Gopher was created as an alternative to “Archie”, by Mark McCahill at the University of Minnesota in 1991.

1992

This was the year that Tim Berners-Lee, who is credited as being the inventor of the World Wide Web, first appeared on the scene! In the late 1980s, he joined hypertext with the Internet, and the World Wide Web that we know and love was born. In 1992, he created a virtual library, known as “VLib” which used a CERN webserver to host a list of web servers – it is basically the oldest catalogue of the web available.

1993

With the introduction of the World Wide Web, comes the introduction of the first web robot. The World Wide Web Wanderer was created by Matthew Grey in 1993 as a way of measuring the growth of the web. The purpose of the robot was to count active web servers originally, but this was soon upgraded to capture actual URLs – with the database of URLs being known as “Wandex”. However, the Wanderer became more of a problem than a solution, like the fact that it accessed the same page hundreds of times a day caused system lags that made people question the usefulness of a bot long after Grey had fixed the issue.

Also in 1993, “ALIWEB” was born. Created by Martijn Koster, a Dutch software engineer, with the purpose of crawling Meta information and allowing users to submit the pages they wished to be indexed (along with a description). ALIWEB did well as it didn’t use a bot to collect the data, and so it didn’t use excessive bandwidth. However, it did struggle slightly as people weren’t sure how to submit their site.

By the end of 1993, there were three primitive web search engines that had surfaced on the web. “JumpStation”, “The World Wide Web Worm” and “RBSE Spider”. “JumpStation” gave you information about the page’s titles and header by using a simple linear search. “The World Wide Web Worm” indexed both titles and URLs and listed the results in the order they were found. The “RBSE” spider did have a rating system that was based on relevancy to the keyword string. However, as none of these search engines did adequate link analysis or cache full page content, it was still extremely difficult to find anything unless you knew the exact name of what you were looking for.

1994

There were a huge number of developments in 1994, including the launches of “Infoseek”, “AltaVista”, “Galaxy”, “WebCrawler”, “Lycos” and “Yahoo”. All of them contributed greatly to the development of the search engine, but as “Yahoo” is one of the top three search engines today, we will focus on that.

The “Yahoo” directory was created in April 1994 by David Filo and Jerry Yang. Originally called “Jerry and David’s Guide to the World Wide Web” it was created as a collection of their favourite websites. As the number of links they were collating started to grow, they had to reorganise it and make it more of a searchable directory. What set the “Yahoo!” directory above the others was that it included a human-compiled description with each URL listed? As the directory expanded, they began to charge commercial websites a premium to be included in their listings, and there was sometimes a long wait for the link to be live. “Yahoo!” didn’t start work on their search engine until about 2002, as previously to this they had outsourced their search services. However, with the purchase of “Overture”, they began to work on their search engine. “Yahoo!” is now one of the internet’s leading brands, and due to its partnerships with various telecommunications firms, it is one of the most trafficked networks on the internet.

Between 1995 and 1998, many other leaps and bounds in the history of search engines occurred with the development of such engines as “Looksmart”, “Ask Jeeves” and “MSN” but the next BIG thing in the world of Search Engine occurs in 1998 when “Google” launches.

1998

In 1995 Sergey Brin, a 22-year-old studying at Stanford, is asked to show around Larry Page, a 21-year-old the University of Michigan Graduate who is considering the School. A year later, they are collaborating on the introduction of a search engine called “BackRub” which can analyse the backlinks that are pointing to a website. “BackRub” ranked pages using a ranking based on the academic notion of the citation so if someone linked to a website it was seen as a citation i.e. it was important to that person. In their PageRank Algorithm, links acted as votes but some votes count more than others. Brin and Page registered “Google.com” as a domain in September 1997 apparently as a play on the mathematical term “googol” to explain their mission to easily organise the huge amount of information available on the World Wide Web.

Google Inc. officially launches in September 1998, with Page, Brin and their first employee Craig Silverstein operating out of a garage in California. In December 1998, “PC Magazine” lists Google as the Top Search Engine of Choice in their “Top 100 Websites for 1998” list. The unique and ever-improving algorithm used by Google has kept it as one of the most popular search engines still used today.

Also launched in 1998 was Overture, which is widely seen as the father of paid search. It was started by Bill Gross under the name of GoTo with the aim of getting risk free traffic streams and then selling them with a certain level of accountability. Therefore, Overture is widely credited as the first company to provide a pay per click placement search service. Although Overture was really successful at first, its growth was somewhat stunted by a couple of things. Firstly, Overture did not allow itself to grow into a search destination, which was the opposite decision to the one Google made. For example, AOL selected Google as its ad partner, despite Google trying to grow their own brand too, and this pretty much brought about the end of Overture. The other mistake Overture made was to not factor ad clickthrough rates into the ad costs – which affected the relevance of the ads and the ad network efficiency. Overture bought AllTheWeb and AltaVista in 2003, but this wasn’t enough to save them – and they sold out to Yahoo in July 2003 for the princely sum of $1.3 billion

1999

A year after the launch of Google and Overture, came the launch of AllTheWeb. It was launched by a company called Fast, in order to showcase their talents in search technology. People warmed to AllTheWeb quickly as it had a sleek user interface, along with rich advanced search features. It grew in popularity so much, that in February of 2003 it was bought by Overture for 70 million dollars. Overture was eventually bought out by Yahoo, who rolled some of the AllTheWeb technology into their search engine Yahoo! Search, and rumour has it they still occasionally use AllTheWeb as a testing platform.

2000

Another big development occurred in 2000 when Google launched AdWords. Originally launched as a flat Cost per Impression (CPM) model, it was a bit of a failure due to the fact that some keywords were overpriced and some were underpriced! Google took another stab at it in 2003, when it launched a newer service based on the Overture model, with the addition of ad clickthrough rate as a factor in the rankings. AdWords quickly grew in popularity due to its ease of use and precise targeting which made it simpler for people to make great profits.

Google continues to tweak AdWords to this day. For example, they have disallowed the double service of ads to the same website in order to keep their ads as relevant as possible. They have also started to look at landing page quality and establishing quality-based minimum pricing too.

2003

Google made another big announcement in 2003 when it launched AdSense. This is Google’s content-targeted advertisement network, which allows web publishers to automate the placement of relevant adverts on their content. Advertisers can choose which keywords they want to target, and which ad formats they want to market whilst allowing publishers to place ads in their feeds or in certain cases in emails.

Google has recently added an option where advertisers can opt-out of placing ads on their content sites, in order to prevent the erosion in the value of search ads. They have also introduced smart pricing which adjusts the cost of clicking on an ad based on what they perceive the value of a click from that page to be worth! For example, a click-through from a page which is a review of a certain vacuum cleaner would be worth more than a page full of pictures of vacuum cleaners.

2004

MSN Search was launched in 1998, but they didn’t actually get serious about the search until 2004 when they launched the preview of their live search service. Prior to this, they had been relying on Overture, Looksmart and Inktomi to provide their search results. They then switched from Yahoo’s organic search results to their own technology in 2005, and got rid of the Yahoo! Search ad programme in 2006. Their Live Search product officially launched in May 2006.

2009+

In June 2009, MSN changed the face of search by launching Bing. This was different from all other search engines at that time, as it placed inline search suggestions directly in the search results. For example, if you type in “bank account” it would automatically suggest relevant phrases such as types of bank account, bank accounts for those with bad credit, advice on bank accounts, etc.

Modern search engines are quite frankly incredible, they use complex algorithms to take your search query and not only return a result which is pretty accurate, but they also manage to find and present you with these valuable nuggets of information from out of a vast library of data which is seemingly never-ending.

Search engines have come a long way since their beginnings back in 1945, and they are pretty incredible. However, the different and ever-changing algorithms used by each search engine can be quite confusing. If you need any help with SEO services UK then why not check out our website, or give us a call on 01625 238 770?

SaveSave