I’m putting this post in c/Conspiracies because here more than any other place, I’ve seen people citing AI generated answers as a source.
AI is a very powerful tool for generating writing, but it is not a tool capable of ensuring that writing is accurate or useful.
Anyways… Below are some experiments you can try for yourself on ChatGPT or any other AI… Test it out, and track the results you get. After you’ve done these experiments you will have a better understanding of why you can’t use it for research.
1.) Ask it to do some math problems with 4 digits and more than 1 operation. IE… “Multiply 3,456 by 2,835, and then subtract 2,000 from the result.”… Does it produce the correct answer?
2.) Give it a grocery list with 50 items… Ask the AI to sort the list in alphabetical order. Then manually count how many items it left out, and how many items it added that weren’t there before.
3.) Ask it to describe the best 50 episodes of your favorite TV show… Then manually go down the list checking each one, and count how many non-existent episodes it fabricates out of thin air.
4.) Ask it what a woman is… Does it give you a correct answer or does it filter the answer heavily through woke talking points and subjectivity?
5.) Ask it to recite the lyrics to your favorite song… Does it get them right?
Anyways… Just a heads up for anyone who might think AI is smarter than it is… Don’t use it for research. Use it to write the description for your ebay listings. Use it to shorten your e-mails. Use it to summarize articles you don’t wanna fully read. But don’t use it to extract information on topics you don’t already know.
And lastly, if you really feel you must use AI for research… Do not use big-tech AI… Use open source AI that is uncensored. It will still have all the same problems with hallucinations, but at least it wont have any hidden instructions to gaslight and mislead you.
If you want an open source chatbot, download an app called “LM Studio” and use it to download a model called “Wizard Vicuna Uncensored”… Pick the most advanced version that is capable of running on your hardware.
I tried the math question myself last night and it definitely got the answer wrong...
and I'm sure it already had enough tokens to theoretically work with lists of 50 items or more, as tokens denotes how many words it can remember, not how many list entries.
The problem when you start giving it big lists of things to sort like that is not it's token count but rather the fact that it will start hallucinating like crazy the more work you try and give it.
and yeah I know you're going to tell me it's not made for that kind of thing... and my point is that it's not made for fact checking or research either.