Anyone who actually understands the architecture of an LLM knows how it works. Because it's a non-deterministic system, you cannot easily step through it as you would normal code, but it is still possible not only to conceptually understand it but to actually step through the system if you have access to the datasets and meta-parameters (temperature, top-P, etc), and logging information about the RNG that occurred (any company working on foundation models would have all this or they couldn't reliably improve the product).

Systems with transformer architecture (what "AI" is using) are going to be used for what they're best at - pattern recognition. This is shat allows the system to very quickly extract sentiment from extremely large sets of data. Pattern recognition being essentially THE function of transformer architecture also makes it good at facial recognition. A dream for surveillance.

There are two sets of people who are running this "AI is going to become super smart and end us all!" Narrative:

• the business people: Note how it's almost always someone who stands to benefit economically from AI that is in the public eye talking about how amazing AI is and how it's just doing all these crazy and mysterious things (which are not at all crazy and mysterious if you even halfway understand how it works)
• the narrative seeders: pick your poison on who these elites are, but they want people to both humanize AI and see it as something amazingly intelligent so they willingly submit to upcoming AI surveillance and governance.

Aside from that you have true believers, grifters, and those who have been fooled, but the two sets above are the ones pushing it from the top down. And there is bleedover between the two, as I believe Elon Musk, for example, belongs in both camps