Recently James Corbett did an episode of Corbett Report on tools for online researchers. He listed some he uses and then his site members listed more in the comments.
Here I've compiled them all together with a brief explanation. You can now find this list on the links page in it's own "Tools" section. Suggestions for additions to the list are welcome, here in the comments, or sent to modmail anytime.
-
youtube-dl - comprehensive program for downloading youtube videos - terminal or GUI - with many options
-
Yandex Image Search - excellent image search & reverse image search too
-
convertcase - convert the case of copy/pasted text
-
highlighter - extension for an online highlighter
-
HTtrack or webhttrack - download and preserve entire websites
-
Web to PDF - online webpage to pdf convertor
-
Million Short - Million Short makes it easy to discover sites that just don't make it to the top of the search engine results for whatever reason
-
Archive web pages: https://archive.ph/ or https://archive.is
-
https://ricks-apps.com/osx/sitesucker/ (like Htrack for OSX)
-
nsfwyoutube.com - watch age-restriced videos on youtube without signing in. Just paste nsfw in front of youtube in addressbar of video
-
Bypass Paywalls - extension for Chrome, Brave, Chromium - also available for Firefox
-
Open Broadcaster Software - download in-progress live streamed videos
-
ArchiveBox - powerful, self-hosted internet archiving solution to collect, save, and view sites you want to preserve offline
-
Wallabag - self hostable application for saving web pages: Save and classify articles. Read them later. Freely.
-
Keepnote - note taking application that works on Windows, Linux, and MacOS X
-
WayBack Machine Downloader - Download archived webpages from wayback machine
Sci-Hub - https://sci-hub.st/
Ever found a paper you want to read but you're no longer a student so you can't use your Uni login to get it, or maybe you've never even had that service
Example: https://www.bmj.com/content/343/bmj.d8348.full
The feminisation of nature
Access this article for 1 day for: £30 / $37 / €33 (excludes VAT)
(this paper is about turning the frogs gay - for real)
it has a DOI number
so put that into Sci-Hub and it gives you free access and a link
https://sci-hub.st/10.1136/bmj.d8348
At it has links to its sources in another DOI
doi:10.1039/B815741N
so pop that one in and
https://sci-hub.st/10.1039/B815741N
10th Anniversary Perspective: Reflections on endocrine disruption in the aquatic environment: from known knowns to unknown unknowns (and many things in between)
If you need a place to search for papers Scopus.com is good
and also Google Scholar if you have to
Piggybacking on this.
Also try libgen.is
You can download tons of books for free, and scientific papers. Fiction, non-fiction, you name it.
That's a great one, thanks!
Losing access to Papers was the worst thing about graduating!
Sci-hub button for GreaseMonkey browser extension. Automatically detects DOI: numbers on most pages and takes you directly to the sci-hub download page for that : https://greasyfork.org/en/scripts/370246-sci-hub-button
Similar as a Chromium extension: https://chrome.google.com/webstore/detail/sci-hub-x-now/gmmnidkpkgiohfdoenhpghbilmeeagjj?hl=en-US
Do note that due to USA/Globalist censorship, sci-hub keeps moving domains.
Keep searching for "sci-hub" working current domains to find out the current, if your mostly used goes down.
or go to /r/scihub as long as it is still there and not taken down:
https://www.reddit.com/r/scihub/
the sci hub one is russian, whats up with that? russian hackers/trolls/scammers?
Hacker in the old sense. The founder is Russian
https://en.wikipedia.org/wiki/Alexandra_Elbakyan
She is also in hiding to avoid extradition to the US
That's a good observation, because Russian things are clearly made by hackers/trolls/scammers.
Do not forget to use a Printer and hide the documents well.
When the big genocide comes, digital info is basically pointless. There will be no video proof when video streaming becomes patented and anyone suspected to have the files are sent to camps or hacked into pieces with machetes.
The only reason why we even know much of the Khmer Rouge genocide (that anti covid policies/mandates eerily resemble) or the Maoist China Cultural Revolution is people take notes on that day, be it executioners, pawns or just people burying their notes under vases or cabinets.
TL:DR; Print everything you think is important out and mothball them. When the big day comes you WILL need it.
To that end: For printing a PDF you'll want to print 2 pages per sheet, side by side, and then cut them in half with a paper guillotine. Half the amount of paper used, and still a good size for reading.
This can be finicky to do in free word processors (adobe acrobat pro seems to do it perfectly) - personally I use Libre Office, and find I get the best result by setting the format of the PDF to B5(ISO) and then under print settings set paper size to A4 (the size of paper I'll be printing on) and then set "2 pages per sheet". Then you must scroll through a few preview pages just to make sure no text has been cut, and maybe print a test page.
Printing initially in small ranges is good too, just in case you encounter an error need to reprint. That way you have not printed an entire long document in an unsuitable format.
Even if this seems like excessive precaution; it could be handy, if you are predominantly an ebook reader, to have a few books / articles you want to read printed out like this for blackouts or bug-outs.
Your comment brings to mind the CIA's Dumbo project.
https://wikileaks.org/vault7/#Dumbo
The Video DownloadHelper extension is by far the best method to download videos.
Sometimes, Facebook videos can stubbornly resist downloaders. To download those videos:
(1) Click play on the facebook video
(2) Click the full screen button. This will change your url to the video's url.
(3) Edit the url by replacing "www." with "mbasic."
(4) You're still on facebook but a dumbed down version, so click play on your video again.
(5) You can now download the playing video by right clicking on it and selecting "Save Video"
Also, Print Friendly is a great extension to get rid of all the border frames and clutter when saving webpages, so you only get the stuff you want
Great list, my additions. You learn all on this page (all posts) and your OSINT skills are better than 90% of people out there.
Archive.org is already compromised. do NOT store there, only look for already stored stuff there. They have started retroactively censoring. Use Archive.IS for now.
eTools.CH meta-search engine (google, bing, duckduckgo, exalead, fastbot, qwant, yahoo, yandex, etc.) that allows you to rank different search engines (rank google down, rank others up, make Results 40/page, make 40 results per engine, save your own preferences, learn to use + and "" and site: in your search terms): https://www.etools.ch/searchAdvancedSubmit.do?query=
MetaGer - another meta search engine, that does NOT automatically put google results on top. You can find different hits with this search engine than with the big two (google and bing, which have their own indexes, but rank/censor heavily) : https://metager.org/
Brave Search - a new search engine, with no censoring and privacy and does NOT use the Google index by default, but is slowly building it's own (it will take time, so don't expect this to cover nearly what Google / Bing cover now) : https://search.brave.com/
Torch - TOR + search but with lots of ads - you need a TOR browser first : http://xmh57jrzrnw6insl.onion/
Ahmia - Another Tor network search, but you can initially search with a basic browser, but need a TOR browser to follow links to TOR results : https://ahmia.fi/
Forensically - Alternative Photo Forensic tool with clone detection, ELA, NA, LLS, LG, PCA, MD ,GT and other analyses: https://29a.ch/photo-forensics/#forensic-magnifier
META.ORG scientific paper search along with pre-prints. Learn to use filters on the right column : https://www.meta.org/
BioRxiv & MedRxiv : Biomedicine/medicine preprint servers for uncensored pre-published scientific papers: https://www.biorxiv.org/search https://www.medrxiv.org/search
ThreadReader - Read and PDF print full twitter threads (they don't properly print out from twitter.com) : https://threadreaderapp.com/
PubMed - Really. Learn to use it. Powerful advanced search. The shit you can find for free is amazing. https://pubmed.ncbi.nlm.nih.gov/advanced/
Espacenet - worldwide patent search : https://worldwide.espacenet.com/
NewsNow - news links / headline aggregator and search engine NOT controlled by google : https://www.newsnow.co.uk/h/?search=
Odysee , Rumble , Brighteon , Bitchute , BrandNewTube - Alt video platforms - learn to use their searches properly : https://odysee.com/ https://rumble.com/ https://www.bitchute.com/ https://www.brighteon.com/ https://brandnewtube.com/
Baidu.com - yes really. It is CCP censored, but mainly about CCP / China critical stuff. Think it like this way : If you wanted to read shit on USA that was censored in USA during cold war you read TASS/Izvestia. But if you wanted to read shit on USSR, you would have to go to non-Soviet sources. Everybody does propaganda, just find a away around local propaganda. Same logic works here. https://baidu.com/
SearX - another meta-search engines with various nodes (URLs that you search from). Very very handy .Does not censor, but doesn' t have an index of it's own (uses other engines, like all other meta search engines): https://searx.me/
Bypass Paywalls Clean - a better browser extension than the standard "Bypass Paywalls" (Mozzilla and chromium derivatives ) to bypass paywalls on most for-pay major newspapers sites : https://github.com/qnoum/bypass-paywalls-chrome-clean-magnolia1234#installation-instructions
GreaseMonkey - browser extension with tons of good installable JavaScript extensions from greasyfork that make google, searching, linking, archiving, youtube, bitchute, etc tons better/faster: https://www.greasespot.net/ https://greasyfork.org/en
RevEye Reverse Image search extension (chromium) : search by image on Google, Bing, Yandex and TinEye simultaneously, saving time on separate searches : https://chrome.google.com/webstore/detail/reveye-reverse-image-sear/keaaclcjhehbbapnphnmpiklalfhelgf?hl=en
Discussion button Search Extension for Chromium on Google searches - bring back the forum search feature that Google removed (in a roundabout way). Kludgy, but sometimes helpful: https://chrome.google.com/webstore/detail/discussions-button-for-go/igjiggoeheaondbmhmilpmbdkpgcjmdn
Scribd - tons of books and PDFs and saved web pages. Very, very useful for older reference materials that have disappeared from elsewhere on the web. https://www.scribd.com/search
Will add more later if I have time. This should help most to get up to speed.
https://archive.org/details/usfederalcourts
US Federal Court Documents
The documents in this collection are from the US Federal Courts. A large collection come from the federal government's project for Public Access to Court Electronic Records (PACER). The PACER Service Center is the Federal Judiciary's centralized registration, billing, and technical support center for electronic access to U.S. District, Bankruptcy, and Appellate court records.
Dynamite, didn't know that one, thanks!
An important thing to keep in mind:
Sometimes, to truly get to the bottom of things, you need to repeat the SAME exact search keyword search in MULTIPLE search engines (because they have different coverage/indexes) AND then do multiple variations (synonyms) of your keywords (again to all search engines), so that you get maximum coverage of information. Tedious, but very very useful at times.
WolframAlpha : a math , chemistry and physics computational engine. You can do chemical reactions, see radioactive isotope breakdowsn, do symbolic math solving, etc. Great for doing more complex calcs quickly https://www.wolframalpha.com/
BookFinder - the best used (and new) books search. Yeah, books. Those antique "things" that contain a lot of distilled information (I have a few thousand myself): https://www.bookfinder.com/
ResearchGate - sometimes you can find academic papers you can't download elsewhere (not even on sci-hub) here. Esp. non-peer-reviewed and from more obscure journals. https://www.researchgate.net/
Academia.edu - as ResearchGate, although you can't always download (depends on your academic/organization affiliations), but at least you can find papers, researchers and often read abstracts. https://www.academia.edu/
BASE - Universität Bielefeld Search - Academic search with high granularity and indexing also non-English papers. https://www.base-search.net/Search/Advanced
CORE - Search of Open Access research papers. https://core.ac.uk/search?q=%22
SemanticScholar - Another academic peer review paper search but with date range, academic field, author, journal and PDF filters. https://www.refseek.com/documents?q=
Science.Gov - Find out what the US Federal Agencies do in the name of science. Reports, papers, guidelines, etc. Very, very useful to find out what Federal science agencies are thinking/publishing. https://www.science.gov/scigov/desktop/en/search.html
DeepDyve - a meta-search engine for peer-reviewed, preprint and other papers (has also it's own chromium plugin that interfaces with PubMed and Google Scholar). https://www.deepdyve.com/search?query=
Publish or Perish - Best Desktop search tool for all for-paid scientific publications / peer review articles via CrossRef , Google Scholar , PubMed , Microsoft Academic , Scopus , Web of Science (you have to install API keys or have institution access for some of these). You get boolean searches, phrase searches, h-index , g-index ranking, sorting by date, citations, etc. The best tool for serious researchers, esp. if you are academic, or need to revisit certain keyword searchers regularly. https://harzing.com/resources/publish-or-perish
PDFDrive - another book download alternative to Scribd.com (and GenLib) https://www.pdfdrive.com/search?q=
Zotero - if you do writing and academic research, this is a must for reference and endnote management. OR if you have insane amounts of references, pdfs and keywords, like some of us. https://www.zotero.org/
Project Gutenberg - old, historically important, out-of-print and copyright free books scanned and searchable. https://www.gutenberg.org/
Worldcat.org
Tor Browser https://www.torproject.org/download/
This lets you do your research privately.
tor is so risky its not even worth it for anything related to conspiracy theories
Please don't discredit our research by calling it what the US government wants people to believe they are.
As for Tor, what risks are you talking about?
EDIT: There has been a lot of government disinformation concerning Tor. Bear that in mind.
you need to have a good understanding of cybersecurity to be truly safe. so its not a good idea for most people.
Thanks for your input. What I see here is something of a perfect solution fallacy. What this does is keep a lot of your information private. More people absolutely should use it.
the reason you arent getting hacked on the clearnet is because the service providers have cybersecurity experts protecting their systems. when u leave that section and use tor you are doing the equivalent of representing yourself in court. if your not prepared u could be victimized
https://en.wikipedia.org/wiki/Fear,_uncertainty,_and_doubt
Tor is not less or more secure (unless you control exit nodes), but you can access data with Tor that you can't access on the public web.
That's why Tor can be useful: not for privacy, but wider access to obscure data.
A little dated but prob still has good tips for research NSA Untangling the Web A Guide To Internet Research https://archive.org/details/Untangling_the_Web
Thank you. It's good to know of tools for activist research.
Thank you I’m saving this
It's appreciated!
Awesome post! Thank you for this!
Thank you!
I just created a sub for something like this.
Check it out: c/Toolbox !
This is a pretty awesome list. Thanks
awesome! Switched from FuckFuckNo to Brave!
Thanks!!
ok, boys...grannypede here...where/how do I find other .win subs??
nsfwyoutube doesn't work for me to unblock content for reasons unrelated to age. I think that's kind of weird.