Hacks by Mekineer

User Tools

Site Tools

You are here: decentralized encyclopedia » information technology » 2019 search engines
information-technology:2019-search-engines

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision		Previous revision
information-technology:2019-search-engines [2019/07/03 04:17] – marcosinformation-technology:2019-search-engines [2024/05/23 03:32] (current) – marcos
Line 1: Line 1:
 +// Motivation:\\
 +\\
 +It's hard to find results that are not mass media or commerce, aside from sources like reddit, quora, and other forums.  Perhaps non-commercial sites, despite producing good content, don't make it to top results.   I looked for alternative search engines, that I might find better results.\\
 +\\
 +I found many articles about search engines, but they were not focused on independent search engines, with their own crawlers and indexing methods.  If they were about alternatives, they were misinformed and non-comprehensive.\\
 +\\
 +Since I couldn't find the article I wanted, I started writing my own.  I have combed through what's available and give a brief summary of what I found.  I was also curious about how much alternative search engines are being used.//
 +
 +Update 2020-10: Commentary from [[https://www.quora.com/How-is-Google-s-PageRank-algorithm-being-consistently-outsmarted-such-that-Google-search-results-are-in-decline/answer/Borislav-Agapiev |Borislav Apagiev]]
 +
 +Update 2021-11: New search engine from Huawei named [[https://en.wikipedia.org/wiki/Petal_Search |Petal Search]] at [[https://petalsearch.com |petalsearch.com]]
 +
 +Update 2023-05: https://search.marginalia.nu.  Its mission is excellent, but the results are rarely relevant.  Still worth using when other search engines don't deliver.
 +
 +Update 2024-05: Google has become all about selling products.  It's harder than ever to find relevant results if your search terms overlap with some product or service. 
 +
 +
 +\\
 +====== Search Engines: A Quest for Alternatives ======
 +
 +===== General Purpose Search Engines =====
 +
 +The most popular alternative to Google, is Bing.  In this review, I will avoid counting those that re-brand someone else's search engine results.  One example is [[https://en.wikipedia.org/wiki/Yahoo!_Search |Yahoo search]], which uses the Bing search engine since 2009.  I made exception with [[https://en.wikipedia.org/wiki/DuckDuckGo |DuckDuckGo]] and [[https://en.wikipedia.org/wiki/Searx |SearcX]] since I thought they were notable, and as [[https://en.wikipedia.org/wiki/Metasearch_engine |meta search]] engines they combine search results from as many engines as possible.
 +
 +
 +Method:
 +
 +In my first pass, I ruled out search sites where other articles, including Wikipedia, state that the search site uses someone else's search engine.  It's possible that any of those articles could be mistaken or even outdated.  I verified infrequently.
 +
 +In the second pass, I tested search sites that claimed to have hybrid results.  I have yet to see an abundance of **unique results**, if any, for any engine that supposedly "enriches" results from bing and/or google.  I wasn't trying to be thorough: I tried at least 2 different searches looking for unique results.  Some results were exact copies, some offered the same results but the order was randomized.  Or mixed and randomized from both Google and Bing <del>as in the case of DuckDuckGo's</del>.
 +
 +Update 2020: Sometime since I ran tests in 2019, DuckDuckGo stopped using Google search results, and aside from Bing results, had 5 unique results in the first 30.
 +
 +I am still going through the third pass, which is to test each remaining engine for individuality of search results.  Some are crossed out below.  They are crossed out, if I did not find sufficient unique search results.  Sufficient unique results means that the first twenty results were not listed in the first twenty results of bing and the first sixty results of google.
 +
 +During the third pass, I found a directory of international search engines, which lead me to expand my results toward the bottom of the page.  I like Seznam and use it regularly.  As opposed to Baidu, English search returns mostly English results.  When I use Baidu, I use a browser extension to translate the results into English.
 +
 +[[http://google.com |Google]]\\
 +[[http://bing.com |Bing]]\\
 +[[http://yandex.com |Yandex]] (unique)\\
 +[[http://baidu.com |Baidu]] (unique)
 +
 +Lesser known search engines providing unique results.\\
 +I'm only promising unique results.  I'll leave it to you to decide if the results are good.:
 +
 +[[http://mojeek.com |Mojeek]] (unique)\\
 +[[https://metager.de |Metager]] (bing, scopia)\\
 +[[http://gigablast.com |GigaBlast]] (unique)\\
 +[[http://lookseek.com |LookSeek]] (unique)\\
 +[[https://millionshort.com |MillionShort]] (no crawler but unique)\\
 +[[http://www.exalead.com/search |Exalead]] (unique)\\
 +
 +Search engines that are mostly rebranded Google or Bing results, or are not working:
 +
 +<del>[[http://duckduckgo.com |DuckDuckGo]] (bing, google)</del>\\
 +<del>[[http://lycos.com |Lycos]] (bing)</del>\\
 +<del>[[http://qwant.com |Qwant]] (bing, google) just one unique at result #16</del>\\
 +<del>[[http://spiderline.net |SpiderLine]] (bing) unique 1st-try #5 #14, unique 2nd-try #3 #8</del>\\
 +<del>[[http://gibiru.com |Gibiru]] (google) unique 1st-try #12 #14 #18, unique 2nd-try #10 #11 #12 #13 #14, hides visited links color</del>\\
 +<del>[[http://yacy.com |Yacy]] (not working)</del>\\
 +<del>[[http://iseek.com |iSeek]] (bing)</del>\\
 +<del>[[http://meekd.com |MeekD]] (bing, google)</del>\\
 +
 +Update: I found www.searchenginemap.com that covers where a lot of search engines get their results from.\\
 +The following are still to be tested (with searchenginemap.com indicator in parenthesis):
 +
 +[[http://carrot2.org |Carrot2]] (bing -> clustering)\\
 +[[http://exactseek.com |ExactSeek]] (unique)\\
 +[[http://dothop.com |DotHop]]\\
 +[[http://izsearch.com |iZsearch]]\\
 +[[http://amidalla.de |Amidalla]]\\
 +[[http://wiby.me |Wiby]]\\
 +[[http://wbsrch.com |Wbsrch]]\\
 +[[http://search.thunderstone.com |ThunderStone]]\\
 +[[http://www.activesearchresults.com |ActiveSearchResults]] (unique)\\
 +\\
 +\\
 +\\
 +
 +===== Estimating Usage of Search Engines =====
 +
 +Using Alexa or Similarweb to determine the popularity of search engines makes for a very rough estimate.  Another rough estimate, would be to visit the Alternativeto.net website.  Another would be to look at the most popular search engine plugins at https://mycroftproject.com/dlstats.html.  Lastly, there is the google trends site.
 +
 +The charts below are from [[https://trends.google.com |Google Trends]], so please take this conflict-of-interest into account.  If you browse to Google Trends, make sure and remember to select "Worldwide", if that is what you seek, because it defaults to the country you are searching from.  I use separate charts so that the relative scale of lesser known search engines can be viewed.  If Google were placed in the chart with the lesser known search engines, they would all show as a flat line at the bottom of the graph, as is the case with Bing, the search engine that had the second highest peak in the timeline.
 +
 +The only search engines that may get more traffic than google, in a specific region, are the localized search engines listed below the charts.
 +
 +{{ :information-technology:google-bing.jpg |google bing}}\\
 +{{ :information-technology:bing-baidu-yandex-duckduckgo-lycos.jpg |bing baidu yandex duckduckgo lycos}}\\
 +{{ :information-technology:duckduckgo-qwant-exalead-gigablast.jpg |duckduckgo qwant exalead gigablast}}\\
 +{{ :information-technology:gigablast-searx-yacy-mojeek.jpg |gigablast searx yacy mojeek}}\\
 +\\
 +\\
 +\\
 +
 +===== Interesting Reviews =====
 +
 +https://restoreprivacy.com/private-search-engine\\
 +https://www.deepwebsiteslinks.com/uncensored-search-engines-for-anonymous-searching\\
 +https://www.wired.com/2009/04/wikipedia-found\\
 +https://en.wikipedia.org/wiki/List_of_search_engines\\
 +https://www.dailydot.com/news/rip-altavista-search-engine-alltheweb-snap-infoseek\\
 +https://12bytes.org/articles/tech/alternative-search-engines-that-respect-your-privacy\\
 +https://www.briskbard.com/blog/?q=node/29\\
 +http://www.idcloak.com/learning-center/gibiru-review-is-gibiru-a-genuine-private-search-engine/a565.html\\
 +https://searchenginearchive.com/general_overview.php
 +\\
 +\\
 +\\
 +
 +===== Localized Search Engines =====
 +
 +Localized search engines, with independent or semi-independent engines.  What is amazing is that on google trends, the popularity of every single local engine is declining.  They are ranked here by overall google trends traffic since 2004:
 +
 +https://www.seznam.cz Czechoslovakia\\
 +https://www.rambler.ru Russia\\
 +http://www.najdi.si Slovenian\\
 +https://www.search.ch Switzerland\\
 +http://naver.com South Korea\\
 +http://www.daum.net South Korea\\
 +http://ant.com Bangladesh\\
 +http://so.com China\\
 +http://leit.is Iceland\\
 +http://sogou.com China
 +
 +Same Localized search engines as above, by current google trends ranking:
 +
 +https://www.seznam.cz Czechoslovakia 1\\
 +https://www.rambler.ru Russia 2\\
 +http://naver.com South Korea 5->3\\
 +https://www.search.ch Switzerland 4\\
 +http://www.daum.net South Korea 6->5\\
 +http://www.najdi.si Slovenian 3->6\\
 +http://so.com China 8->7\\
 +http://ant.com Bangladesh (low traffic; on par with Qwant)\\
 +http://leit.is Iceland (low traffic; on par with Qwant)\\
 +http://sogou.com China (low traffic; on par with Qwant)
 +
 +Localized search engines with barely any traffic, like, ever:
 +
 +https://www.sputnik.ru Russia\\
 +https://www.pipilika.com Bangladesh\\
 +http://parsijoo.ir Iran\\
 +https://yooz.ir Iran\\
 +http://www.istella.it Italy\\
 +https://www.13tabs.com India\\
 +http://egerin.com Kurdish\\
 +http://coccoc.com/search Vietnam\\
 +
 +International directory of search engines:\\
 +http://www.searchenginecolossus.com\\
 +http://www.searchenginesoftheworld.com
 +\\
 +\\
 +\\
 +
 +===== Academic Search Engines =====
 +
 +<WRAP center round box 95%>
 +www.refseek.com - Academic Resource Search. More than a billion sources: encyclopedia, monographies, magazines.\\
 +www.worldcat.org - a search for the contents of 20 thousand worldwide libraries. Find out where lies the nearest rare book you need.\\
 +https://link.springer.com - access to more than 10 million scientific documents: books, articles, research protocols.
 +www.bioline.org.br is a library of scientific bioscience journals published in developing countries.\\
 +http://repec.org - volunteers from 102 countries have collected almost 4 million publications on economics and related science.\\
 +www.science.gov is an American state search engine on 2200+ scientific sites. More than 200 million articles are indexed.\\
 +www.pdfdrive.com is the largest website for free download of books in PDF format. Claiming over 225 million names.\\
 +www.base-search.net is one of the most powerful researches on academic studies texts. More than 100 million scientific documents, 70% of them are free.\\
 +
 +[[https://www.facebook.com/groups/ironcuriosity/posts/2764187810557820 |Tayo Redondo in the science book club]]
 +</WRAP>
 +
 +https://www.sweetsearch.com\\
 +http://www.chemistryguide.org\\
 +https://en.wikipedia.org/wiki/List_of_academic_databases_and_search_engines\\
 +\\
 +\\
 +\\
 +
 +===== Other Search Engines =====
 +
 +P2P peer to peer search engine:\\
 +https://www.majestic12.co.uk\\
 +http://yacy.com
 +
 +Human edited directory:\\
 +https://curlie.org
 +
 +Independent News:\\
 +https://www.goodgopher.com
 +
 +Wiki and forum search:\\
 +http://wiki.com\\
 +http://boardreader.com\\
 +http://omgili.com
 +
 +Encyclopedic search:\\
 +http://www.factbites.com\\
 +http://wolframalpha.com
 +
 +Metasearch engines:\\
 +http://www.infospace.com (WebCrawler, MetaCrawler, DogPile)\\
 +http://yippy.com\\
 +<del>http://searx.me (not working)</del>
 +
 +Tor / Dark Web ([[https://www.torproject.org/download |tor web browser]] required to view results):\\
 +https://ahmia.fi aka http://msydqstlz2kzerdg.onion\\
 +https://onion.link\\
 +not Evil http://hss3uro2hsxfogfq.onion\\
 +torch http://xmh57jrzrnw6insl.onion\\
 +Also duckduckgo, searx, metager, yippy\\
 +Unlike duckduckgo, some search engines include *only* onion results.\\
 +In 60 minutes of searching and reading on the TOR network, results were really slow in loading.  Each search engine had their own set of unique results.  Some had rather distasteful results.
 +
 +http://teoma.com resurrected?\\
 +\\
 +\\
 +\\
 +
 +===== Browser Search Engine Plugins =====
 +
 +If you visit a search engine's website, click the down arrow in the web browser's search box.  You may find the option to add the search engine's plugin to your web browser (tested on Pale Moon).  Most search engine's homepages offer this feature, but a few don't.  Update: For those that don't, on Pale Moon web browser, you can use the extension [[http://legacycollector.org/firefox-addons/3682/index.html |Add to Search Bar]], by right clicking over the search field and selecting this extension's option.
 +
 +You can also create your own search engine plugins here:\\
 +http://ready.to/search/en
 +
 +I've downloaded the following from both https://mycroftproject.com and from https://addons.palemoon.org/search-plugins.  Click on any of the following to install them on your web browser.
 +
 +<html>
 +    <script type="text/javascript">
 +        function installSearchPlugin(pluginname) {
 +            window.external.AddSearchProvider(pluginname);
 +        }
 +    </script>
 +    <a href="#browser-search-engine-plugins" onclick="installSearchPlugin('https://nerdondemand.com/sep/baidu.xml');">Install Baidu</a><br>
 +    <a href="#browser-search-engine-plugins" onclick="installSearchPlugin('https://nerdondemand.com/sep/curlie.xml');">Install Curlie</a><br>
 +    <a href="#browser-search-engine-plugins" onclick="installSearchPlugin('https://nerdondemand.com/sep/gigablast.xml');">Install Gigablast</a><br>
 +    <a href="#browser-search-engine-plugins" onclick="installSearchPlugin('https://nerdondemand.com/sep/ip-location.xml');">Install IP-Location</a><br>
 +    <a href="#browser-search-engine-plugins" onclick="installSearchPlugin('https://nerdondemand.com/sep/metager.xml');">Install Metager</a><br>
 +    <a href="#browser-search-engine-plugins" onclick="installSearchPlugin('https://nerdondemand.com/sep/qwant.xml');">Install Qwant</a><br>
 +    <a href="#browser-search-engine-plugins" onclick="installSearchPlugin('https://nerdondemand.com/sep/searx.xml');">Install Searx</a><br>
 +    <a href="#browser-search-engine-plugins" onclick="installSearchPlugin('https://nerdondemand.com/sep/seznam.xml');">Install Seznam</a><br>
 +    <a href="#browser-search-engine-plugins" onclick="installSearchPlugin('https://nerdondemand.com/sep/whoiscom.xml');">Install WhoIs</a><br>
 +    <a href="#browser-search-engine-plugins" onclick="installSearchPlugin('https://nerdondemand.com/sep/wolframalpha.xml');">Install Wolframalpha</a><br>
 +    <a href="#browser-search-engine-plugins" onclick="installSearchPlugin('https://nerdondemand.com/sep/yacy.xml');">Install Yacy</a><br>
 +</html>
 +\\
 +\\
 +\\
 +
 +===== Browser Search Extensions =====
 +
 +1) You can have a search bar set up with your search engine plugins using this extension on Pale Moon web browser:\\
 +https://addons.palemoon.org/addon/searchbuttonsbar\\
 +Using Search Buttons Bar, you can search the same search query on a different engine with one click.  It can look like this:\\
 +
 +{{:information-technology:search-bar.jpg?900|Search Buttons Bar}}\\
 +
 +2) Context Search X https://addons.palemoon.org/addon/context-search-x\\
 +Or alternatively, I'm using Quick Context Search https://legacycollector.org/firefox-addons/502176/index.html\\
 +On Firefox: https://addons.mozilla.org/en-US/firefox/addon/contextual-search\\
 +If you highlight a word on the page, these extensions give you a righ-click context menu option to search with your list of search engine plugins.
 +
 +{{:information-technology:quick-context-search.jpg?240|Quick Context Search}}
 +\\
 +\\
 +\\
 +
 +===== Greasemonkey User Scripts =====
 +
 +If you have one of Greasemonkey, Tampermonkey, or Violentmonkey on your web browser, you can use the script [[https://greasyfork.org/en/scripts/383270-alternative-search-engines-2-1 |Alternative Search Engines]] to quickly perform a search with the same query on any of the search engines: google, bing, yandex, and duckduckgo.  It works by showing links to the other engines on any search results page, as you can see in the image below.  I find this one useful on my mobile device, using a web browser that allows extensions, such as Kiwi, with the drawback that I have to request the desktop site in order to see the options.
 +
 +<html>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</html>{{:information-technology:alternative-search-engines.jpg?450|alternative search engines}}
 +
 +Other user scripts include:\\
 +  * [[https://userscripts-mirror.org/scripts/show/121261 |Straight Google]]: Gives google search results-list the actual URLs instead of googlified URLs.  Important when searching on other search engines, because the URLs you have already been to will be marked purple instead of blue.
 +  * [[https://greasyfork.org/en/scripts/22737-remove-yandex-redirect |Remove Yandex Redirect]]: Same as above, but for Yandex.
 +  * [[https://nerdondemand.com/remove-google-people-also-search-for.user.js |Remove Google 'People Also Search For']]: This section expands after opening one of the google URLs.  It is a big annoyance for me, because it usually expands while I'm trying to click on another link, moving the link out from under my mouse.
 +\\
 +\\
 +\\
 +
 +===== Self Hosted Search Engine =====
 +
 +Since SearX is open source, you can use the software and have your own search engine server:\\
 +https://www.howtoforge.com/tutorial/how-to-install-searx-meta-search-engine-on-ubuntu-1804
 +\\
 +\\
 +\\
 +
 +===== Search Engine Filtering =====
 +
 +I decided to do something and create my own alternative search engine, of sorts.  Search results from search engines, filtered based on the resource domains a website uses to construct a web page.  The project is here: https://github.com/mekineer/single_domain_search
 +
 +It has potential, but obtaining search results from search engines is difficult because they guard their stores.
 +\\
 +\\
 +\\
 +
 +===== Audacity of Google =====
 +
 +And Google said "[[https://support.google.com/webmasters/answer/76329?hl=en |let there be hyphens]]", and there were hyphens.  Dokuwiki defaults to using underscores as word separators.  Wikipedia uses underscores also, but if you aren't Wikipedia, I'm going to have a hard time finding your site unless you go through the arduous process of mass-converting your site to hyphens.  Maybe there's a good reason?  [[https://www.youtube.com/watch?v=AQcSFsQyct8 |Matt Cutts says]] it's because programmers use underscores in variable names, which are treated as one word.
 +
 +Google does have a conflict of interest: money. So without any other bias, it will provide results that are backed by money. Google makes money on advertising. If a site isn't paying money for advertising, or involved in their advertising program, then it will rank lower in Google results.
 +
 +Also, sites will rank higher if they can hire someone in SEO, to meet Google's artificial requirements for being a "good quality" site. While many of the requirements aren't necessarily bad, they have to be studied in order to be followed. Sites that don't follow the requirements will rank lower, even if they have content that may be of high quality.
 +
 +So your top Google search results will come from sites that are trying to rank well on Google. While this includes producing good content, it is not the only factor.
 +
 +The other conflict of interest is in politics, but I can't speak with any certainty other than repeating different news sources.  The only certainty is the existence of bias, because humanity.  Some are better at [[https://en.wikipedia.org/wiki/Doublethink |doublethink]] than others.  I hope we can still all [[https://www.breitbart.com/tech/2019/06/25/google-executive-jen-gennai-i-used-imprecise-language-in-project-veritas-video |think the best of each other]].
 +
 +[[https://www.businessinsider.com/google-manipulates-search-results-report-2019-11 |Google reportedly manipulates search results to hide controversial subjects and favor big business, businessinsider.com]]
  

Page Tools

Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Share Alike 4.0 International
CC Attribution-Share Alike 4.0 International Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki