Elon clearly doesn't know how AI works if he claims it'll be smarter than a human so quickly. But this statement is probably relying on the ignorance of the masses. What we call AI now doesn't think, it doesn't understand what it's being asked, it extracts sentiment from a prompt then goes through all the data its been trained on to calculate a response word by word based on the frequency of those words being used in the context of the given sentiment and alongside the previous calculated words.
And perhaps he doesn't actually know how it works, because I'm more and more of the mind that he's just a frontman or character - but regardless the simpler explanation is musk has always been pushing the "dangers of AI" but not in a way to shut down development but in a way to justify development on brain chips, claiming it's the only way humans will be able to compete. So him fluffing AI right now is just a gateway to transhumanism, like most of his actions despite the SUPER BASTE MEMES
User: All men are mortal. Socrates is a man. What else do we know about Socrates?
ChatGPT: We can conclude that Socrates is mortal.
This looks like reasoning. It isn't. It's picking (based on "attention" to the input prompt, and its trained word-probabilities) the next word to output. So it picks "We can conclude that Socrates is" and at this point it could have completed with "n't mortal", "mortal", "a dog" etc. but "mortal" has the highest score so that's what's picked. It's also kind of a bad example of reasoning because the context (input text by the user/previous replies by the bot) is clearly weighting "is mortal" for "Socrates" very highly when the attention mechanism reads that context.
Regardless, there was no thought process or reasoning happening. Even "chain of thought" and other methods in use for making LLMs "reason better" are basically a trick, because outputting text that describes (the most likely based on weights from training and context) some plausible reasoning process is then attended to and is more likely to produce a weighting that gives a better result.
That said, it can be used "like reasoning" in practice: For automating things that require a little bit of fuzziness, like if you want to ask a question about some text but don't have the exact word to search. It's decent at that.
It fails at planning, hard.
It's also (as of this moment) impossible to get these to actually learn new things dynamically, because the model's weights are fixed after training (training is slow and takes lots of memory). Naive methods of context implementations are quadratic to the number of tokens, so a super long context (needed for long term learning) is currently out of reach, though there are techniques to extend this, but even then we're talking about extending to 1m ish tokens, which will still require hundreds of GBs of very fast RAM (VRAM or similar) and is still inadequate for a long term memory.
What we will see instead is narrow expert models that are good at some task or job and can deal with that specific thing adequately (like automating basic computer tasks like sorting files, drafting boilerplate emails etc.).
The vision enabled ones are useful for recognition tasks etc.
They're going to be very useful and already are in natural language tasks, but "AGI" this is not.
Well like I said, tasks, not reasoning or understanding.
Example: I have 10GB of memes broadly separated into folders, but otherwise unorganized - names of files are random. Given a multi-modal LLM - i.e., can "see" (really describe) images - it's quite capable of going through each image and renaming them to a descriptive file name based on the image content, just given a prompt telling it to do that.
Same for some of the UI interaction models, where you can say "Click the button that says 'OK' in this program" and it still works even if the text on the button is "Okay", "Ok" or even "Accept" etc.
Yes you can manually do a lot of those things as well but it's more cumbersome than without LLMs.
Reasoning and "intelligence" isn't actually required for it to be useful for NLP tasks.
I actually am doing a lot of this stuff in my day job, and I can say the ability to do fuzzy type searching, instruction etc is light years ahead of where it was just a couple years ago. Doesn't mean they're intelligent or are "AI", just that they're good at attending the input context in a way that makes generating the right (or at least, usually good enough) response much easier.
Kudos to you. It's just that "natural language task" has also divergent meanings, dependent on domain where it is used, hence the clarification on my part.
Elon clearly doesn't know how AI works if he claims it'll be smarter than a human so quickly. But this statement is probably relying on the ignorance of the masses. What we call AI now doesn't think, it doesn't understand what it's being asked, it extracts sentiment from a prompt then goes through all the data its been trained on to calculate a response word by word based on the frequency of those words being used in the context of the given sentiment and alongside the previous calculated words.
And perhaps he doesn't actually know how it works, because I'm more and more of the mind that he's just a frontman or character - but regardless the simpler explanation is musk has always been pushing the "dangers of AI" but not in a way to shut down development but in a way to justify development on brain chips, claiming it's the only way humans will be able to compete. So him fluffing AI right now is just a gateway to transhumanism, like most of his actions despite the SUPER BASTE MEMES
Yep, came to post much the same.
Was reading this: https://github.com/neurallambda/neurallambda yesterday and it has a really good example of what these LLMs actually do:
https://github.com/neurallambda/neurallambda/raw/master/doc/socrates.png
Text for the lazy:
This looks like reasoning. It isn't. It's picking (based on "attention" to the input prompt, and its trained word-probabilities) the next word to output. So it picks "We can conclude that Socrates is" and at this point it could have completed with "n't mortal", "mortal", "a dog" etc. but "mortal" has the highest score so that's what's picked. It's also kind of a bad example of reasoning because the context (input text by the user/previous replies by the bot) is clearly weighting "is mortal" for "Socrates" very highly when the attention mechanism reads that context.
Regardless, there was no thought process or reasoning happening. Even "chain of thought" and other methods in use for making LLMs "reason better" are basically a trick, because outputting text that describes (the most likely based on weights from training and context) some plausible reasoning process is then attended to and is more likely to produce a weighting that gives a better result.
That said, it can be used "like reasoning" in practice: For automating things that require a little bit of fuzziness, like if you want to ask a question about some text but don't have the exact word to search. It's decent at that.
It fails at planning, hard.
It's also (as of this moment) impossible to get these to actually learn new things dynamically, because the model's weights are fixed after training (training is slow and takes lots of memory). Naive methods of context implementations are quadratic to the number of tokens, so a super long context (needed for long term learning) is currently out of reach, though there are techniques to extend this, but even then we're talking about extending to 1m ish tokens, which will still require hundreds of GBs of very fast RAM (VRAM or similar) and is still inadequate for a long term memory.
What we will see instead is narrow expert models that are good at some task or job and can deal with that specific thing adequately (like automating basic computer tasks like sorting files, drafting boilerplate emails etc.).
The vision enabled ones are useful for recognition tasks etc.
They're going to be very useful and already are in natural language tasks, but "AGI" this is not.
Upvoted ! You get it, except the last part is not accurate:
" very useful and already are in natural language tasks"
Depends on how you define "task".
If you mean mimicking natural language patterns based on trained material, then yes.
If we mean "natural language understanding" and "natural language based task reasoning"; then no.
All it can do is pattern re-match and re-generate.
Well like I said, tasks, not reasoning or understanding.
Example: I have 10GB of memes broadly separated into folders, but otherwise unorganized - names of files are random. Given a multi-modal LLM - i.e., can "see" (really describe) images - it's quite capable of going through each image and renaming them to a descriptive file name based on the image content, just given a prompt telling it to do that.
Same for some of the UI interaction models, where you can say "Click the button that says 'OK' in this program" and it still works even if the text on the button is "Okay", "Ok" or even "Accept" etc.
Yes you can manually do a lot of those things as well but it's more cumbersome than without LLMs.
Reasoning and "intelligence" isn't actually required for it to be useful for NLP tasks.
I actually am doing a lot of this stuff in my day job, and I can say the ability to do fuzzy type searching, instruction etc is light years ahead of where it was just a couple years ago. Doesn't mean they're intelligent or are "AI", just that they're good at attending the input context in a way that makes generating the right (or at least, usually good enough) response much easier.
Kudos to you. It's just that "natural language task" has also divergent meanings, dependent on domain where it is used, hence the clarification on my part.