Search engines, list and resources

They are no longer just a means to find a site on a topic, but  they want to go beyond, answer questions, for example and became for the webmaster  the essential element of the environment. See the Future of search engines for more...

Technology of search engines

For years, the technique was to attach a set of keywords to a page, and display in the results pages links corresponding to the keywords of the user query.

To improve the relevance, Google invented PageRank: pages are ranked by the number and quality of links pointing to them, for a group of keywords.

Besides PageRank, we see the BrowseRank which ranks pages based on user activity, TrustRank for the confidence  of the source. The FreshRank aims to evaluate the freshness of the contents and pertinence in the news.

Knowledge base of objects

Bing tries to move from text to the object, just like Google. In 2012, 300 million object have a card in the database of Microsoft. When a request is made, the engine attempts to identify the relevant object and returns the results in relation to this object. All information collected on the Web, on the pages or on social sites, are associated with an object. This provides information on it when it is identified in the request.

Google has a similar tool that begins to take shape in the SERPs in 2012, and that it calls Knowledge Graph. This is to display in the results page next to the link list, information about the object of research: people, places, works etc. ... This in text and image.

Data come from Freebase, Wikipedia, CIA World Factbook and other sources. It contains in 2012 500 million objects and 3.5 billion of facts about these objects. As an example, if the query is identified as Marie Curie, the page displays a photo of her, a biography, pictures of people in connection with her. If it is a painter, there will be images of his most famous paintings.

Video of Knowledge Graph.
This type of result page is the subject of an experiment by Google on www.wydl.com since 2011 and it was expected that it passes to the main search engine, which now began to in 2012.

See also The future of search engines.

List of search engines

You can register for free and add your site on search engines listed below, the most notables are in the list. Note that the inscription on the Dmoz.org directory add a site automatically in all engines ... however it may be useful to save time referencing the engines (not directories) the description of a site under construction ...

• About.com (8)
  Lets ask a question to which experts answer.
• Alexa.com (7)
  Also provides traffic statistics of websites.
  Knowing how to see the traffic of a site with Alexa.
• Altavista.com (9)
  The ancestor of engines that once owned 80% market share - until 1998.
• Ask.com (4)
• Bing.  (9)
  Replaces Microsoft Live Search and is intended as a decision engine. Research is done by category.
• Blekko.com (5)
  Provides partial information of SEO about the sites.
• Cnn.com (10)
  Search in the news. Like all sites in fact, but very developed.
• DuckDuckGo (5)
  A different view on the Web. No private data used.
• Entireweb.com (6)
• Europeana.eu (10)
  Competitor to Google's Book Search, is a virtual library.
• Excite.com (8)
• Exalead.com (7)
  Offers special services such as traffic analysis for a site based on statistics from other engines.
• Gigablast.com (7)
• Google (10)
  Note that Google uses the Dmoz directory to its own directory, and indexing sites, like other engines. Dmoz descriptions can be included in the results.
• Ixquick Does not use private data.
• Lycos.com (8)
• Mamma.com (7)
• Startpage Does not use personal data.
• Virgilio.it (8)
  Italian.
• Answers.wikia.com (6)
  Given the failure of the conventional search engine, which should be weighted by the users but was only by spammers, has become a site of question/answers.
  
• Wolfram Alpha. (8).
  Engine knowledge to answer scientific questions.
• Yahoo (8)
  To be replaced by Bing for the engine.

The number in parentheses is the PageRank.

Source code search

After the closure of  Google code search, it remains alternatives ...

• Koders.com.
• Grepcode.com.
• Antepedia.com.

Resources

• CommonCrawl. This non-profit foundation provides an index of 5 billion of links on Web pages. Enough to create its own search engine!
• IndexTank. The search engine of LinkedIn is open sourced. This include a framework to maintain the index.
• DuckDuckGo. The search engine that is going up is open sourced too on GitHub.

Tweet