Can LLMs be used to discover new laws of logic?

Question

Can LLMs be used to discover new laws of logic?

Stephen Wolfram seems to claim this in What Is ChatGPT Doing … and Why Does It Work?, § "What Really Lets ChatGPT Work?":

is there a general way to tell if a sentence is meaningful? There’s no traditional overall theory for that. But it’s something that one can think of ChatGPT as having implicitly “developed a theory for” after being trained with billions of (presumably meaningful) sentences from the web, etc.

What might this theory be like? Well, there’s one tiny corner that’s basically been known for two millennia, and that’s logic. And certainly in the syllogistic form in which Aristotle discovered it, logic is basically a way of saying that sentences that follow certain patterns are reasonable, while others are not. Thus, for example, it’s reasonable to say “All X are Y. This is not Y, so it’s not an X” (as in “All fishes are blue. This is not blue, so it’s not a fish.”). And just as one can somewhat whimsically imagine that Aristotle discovered syllogistic logic by going (“machine-learning-style”) through lots of examples of rhetoric, so too one can imagine that in the training of ChatGPT it will have been able to “discover syllogistic logic” by looking at lots of text on the web, etc. (And, yes, while one can therefore expect ChatGPT to produce text that contains “correct inferences” based on things like syllogistic logic, it’s a quite different story when it comes to more sophisticated formal logic—and I think one can expect it to fail here for the same kind of reasons it fails in parenthesis matching.)

Answer 1

In the quoted text, Stephen Wolfram uses scare quotes to signify loose, non-technical analogies. So by my reading, he is not claiming that ChatGPT has discovered any new type of logic. Instead he is comparing ChatGPT's ability to output meaningful sentences loosely to classical logic.

The new "logic" that the model may have then, would be: Is this sentence meaningful? I.e. does it represent an idea that can be parsed and understood?

He then makes a comparison to existing classical logic (whether a statement is true or not), which ChatGPT does arguably approximate, and definitely parses and outputs simple logically correct statements better than chance.

Answer 2

The short answer is no, it can't.

LLM like we have today are really less complicated than some people think (well it's definetly not simple to make one, but the task it's doing is fairly simple, just difficult to optimize). All it really does is, given some text, guess what's more likely to be the next word in that text if a human was writing it.

What it usually learns first is syntax, like how to build words with some tokens (tokens could be characters, common combinations of characters or even full words, but then it would already have syntax done). It's fairly easy for it to learn to complete the following: "improvemen", as the word to be completed is probably improvement, so it would guess a "t". Next it would learn some more complex speech patterns, like how an open bracket usually get's closed in the future, or how words coordinate with each other. It's as if it learns what are verbs, nouns, adjectives etc.

The cool part comes now, because if you train a large enough model with enough data for enough time, it will start to identify complex subjects structures, like how when we are teaching about LLM we might start explaining RNNs, than from RNNs we explain about BPTT and exploding/vanishing gradient, and around there we move from RNNs from LSTM, and so on. Notice how each subject has layers (teaching -> LLM -> RNN -> BPTT -> exploding/vanishing gradient ...), and depending on what layers have come before we know in what layer we will enter next, and each layer is really more complicated than what I said, with common words or sentence structures, and the LLMs start learning about those in each layer and how it should "navigate" through the layers.

In the end, it can mimic how we talk about various subjects, but still, it's just mimicking the texts it has seen. It doesn't "think" or use logic at all in what it's saying. It's "logic" consists of copying our pattern of speech in different subjects, but no actuall logic of what it's saying exists. Nothing new can come from just following patterns. Humans can extrapolate common sense and create new things, and that's what makes us special.

Also note that my explanation of how it learns things is just a human interpretation. In the end it's just a lot of numbers marching down a complicated cost function, and really have no guarantee that itcan be reasoned from a human perspective.

Answer 3

Ian Goodfellow, Yoshua Bengio, & Aaron Courville, Deep Learning (2016), § 6.1 is an example on using deep feed-forward networks to learn the XOR logic operation.