This reminded me of when I first started to learn about complex numbers. It seemed like a bit of a miracle that you could take a problem, usually with sines and cosines, and (seemingly) make it more complicated by turning everything into complex numbers. Once you'd got the complex exponentials in place, the algebra fell into place, and you could do the work and then 'recover' the answer you were originally looking for by taking the real (or imaginary) part as necessary. Magic.
Of course, when I learned more about complex numbers it became clear that this 'magic' went much deeper. The Cauchy Integral Formula where the function 'inside' some closed boundary is determined entirely by the properties AT the boundary, a kind of 'holographic' principle, always seemed like magic of the highest order to me.
Does AI have the potential to construct this kind of 'magic'? Are we going to see beautiful theorems constructed by it?
I'm not convinced. I share Penrose's view (which he didn't fully nail down the logic on) that, in some sense, the way humans think is "non-algorithmic". In other words, you can't replace what human beings are capable of by some Universal Turing Machine. Penrose thinks you might need quantum mechanics to achieve human-like levels of intuition, but I'm not convinced by that, either.
On the other hand, if we really are just "computers made of meat" then there's no reason, in principle, why we can't (one day) create true AI.
This reminds me of one of my favorite quotes, which is also my life motto. "All that is too complex is unnecessary, and it is simple that is needed." (Mikhail Kalashnikov) ;-)
This reminded me of when I first started to learn about complex numbers. It seemed like a bit of a miracle that you could take a problem, usually with sines and cosines, and (seemingly) make it more complicated by turning everything into complex numbers. Once you'd got the complex exponentials in place, the algebra fell into place, and you could do the work and then 'recover' the answer you were originally looking for by taking the real (or imaginary) part as necessary. Magic.
Of course, when I learned more about complex numbers it became clear that this 'magic' went much deeper. The Cauchy Integral Formula where the function 'inside' some closed boundary is determined entirely by the properties AT the boundary, a kind of 'holographic' principle, always seemed like magic of the highest order to me.
Does AI have the potential to construct this kind of 'magic'? Are we going to see beautiful theorems constructed by it?
I'm not convinced. I share Penrose's view (which he didn't fully nail down the logic on) that, in some sense, the way humans think is "non-algorithmic". In other words, you can't replace what human beings are capable of by some Universal Turing Machine. Penrose thinks you might need quantum mechanics to achieve human-like levels of intuition, but I'm not convinced by that, either.
On the other hand, if we really are just "computers made of meat" then there's no reason, in principle, why we can't (one day) create true AI.
This reminds me of one of my favorite quotes, which is also my life motto. "All that is too complex is unnecessary, and it is simple that is needed." (Mikhail Kalashnikov) ;-)
Intriguing. And, no doubt, true.
"Complexity can be found in LLMs only because these are parasitic on human complexity"
Very well said. Loved it.