Chomsky had 60+ years to develop his theories and a lot of effort has been spent by him and his followers on formalizing grammar and on creating idealized, symbolic approaches to cognition. But nothing that emerged from this research has come even remotely close to the progress of approaches inspired by neural networks that we are seeing now. Despite enormous amounts of money spent into this area of research, the results of symbolic AI are not in an infant, but in fact have never left the amoebia stage. It should be clear by now that this approach leads nowhere. Chomsky has certainly had his chance. He would never admit that his thoughts have been wrong even if faced with one of the next releases of neurally inspired systems that may pass the Turing test.
Developing symbolic AI was never Chomsky’s field of work. Rather it is undestanding and modelling how we, humans, acquire a language, a process which is clearly diffent from the way something like ChatGPT is trained. A child, for example, is exposed to a truly minuscule amount of data by comparison. I therefore don’t see how the success of LLMs has any bearing on the validity of Chomsky’s work and his approach to cognition.
In the essay Chomsky bases himself on such fundamental differences to make his points. Whether his arguments are sound can of course be discussed (I, for instance, don’t think he provides particularly good examples), but I’m afraid you merely misrepresent his carreer. With regard to symbolic AI Chomsky would probably claim that as long as we don’t fully understand how we humans acquire a language, an essential source of insight will be missing from all AI engineering efforts.
I think this is an embarrassingly bad take from Chomsky.
I’m getting relatively old myself, and sometimes find it hard to accept significant change. But I’ve also been thinking about AI for a couple decades and for me it has -always- come back to a model directly in line with how LLMs operate. No matter how you pose a theory of mind, you can’t escape the web of correlations between symbols, correlations which can be recursively seen as symbols themselves. We’ve known for decades that the ground of physical reality as we know it is intrinsically probabilistic. I don’t see any reason to expect thought is any different.
Reason / logic are likely more akin to cosmic filaments – emergent patterns in a vast universe of relational probabilities. But people like Chomsky may never let go of what IMO amounts to a dated Newtonian view of the mind, because they’re simply too deeply invested.
Don’t use ChatGPT for JUCE stuff. Use Grok instead. Always much better results.
I think this doesn’t primarily show that AI software doesn’t work yet, but just that license models aren’t up to the challenge yet. i mean, think of it. what are LLMs doing? autocompleting your texts based on your own inputs. it doesn’t just spit out code that makes people rich automatically. it’s more like something that trys to be you. it copies your style and writes what you would most likely want to write. the things that you are already planning out in your head anyway. it might use licensed code to accomplish that but it is not spilling secrets, if that makes sense. licenses can’t see the difference between that yet, but humans can. also, but that is specifically for the copilot model, when you install copilot you are asked if your code should contribute to the AI as well, so you can say no there, very transparently. only problem: the governments didn’t create institutions yet whose job it is to check if those systems break their rules. but it’s not the AI developers’ fault that they are so slow. it’s not that this happened just yesterday