This reminded me of when I first started to learn about complex numbers. It seemed like a bit of a miracle that you could take a problem, usually with sines and cosines, and (seemingly) make it more complicated by turning everything into complex numbers. Once you'd got the complex exponentials in place, the algebra fell into place, and you could do the work and then 'recover' the answer you were originally looking for by taking the real (or imaginary) part as necessary. Magic.
Of course, when I learned more about complex numbers it became clear that this 'magic' went much deeper. The Cauchy Integral Formula where the function 'inside' some closed boundary is determined entirely by the properties AT the boundary, a kind of 'holographic' principle, always seemed like magic of the highest order to me.
Does AI have the potential to construct this kind of 'magic'? Are we going to see beautiful theorems constructed by it?
I'm not convinced. I share Penrose's view (which he didn't fully nail down the logic on) that, in some sense, the way humans think is "non-algorithmic". In other words, you can't replace what human beings are capable of by some Universal Turing Machine. Penrose thinks you might need quantum mechanics to achieve human-like levels of intuition, but I'm not convinced by that, either.
On the other hand, if we really are just "computers made of meat" then there's no reason, in principle, why we can't (one day) create true AI.
The magic of complex numbers has many facets indeed. My own graduate and post-graduate work was about varieties (objects in n-dimensional space described by algebraic equations) over finite fields (with prime number of elements) and their connection to what happens over the complex numbers. A variety over a finite field has itself a finite number of points, which you can count. And for certain types of varieties (Calabi-Yau ones, for example), these numbers are connected to coefficients of expansions of certain functions (modular forms) with special symmetries over the complex numbers. More magic.
I think that now we have gone very far with the computers, we have to seriously start thinking about the meat, and not in the sense of how to make it obsolete ("gnostic meat lego", as the wonderful Mary Harrington puts it) but how essential it is.
Final remark regarding Penrose: aperiodic tilings (like his kite/dart one) are amazing. A machine could do the tiling (because it is algorithmic), and yet it is guaranteed that the final result has no translational symmetry. Recently, someone constructed aperiodic tilings with a single shape ("einstein"): https://leisureguy.wordpress.com/2023/03/21/aperiodic-tiling-with-a-single-shape/
This reminds me of one of my favorite quotes, which is also my life motto. "All that is too complex is unnecessary, and it is simple that is needed." (Mikhail Kalashnikov) ;-)
Oh, yes, but consider H.L. Mencken: "For every complex problem there is an answer that is clear, simple, and wrong." And Pseudo-Einstein: "Everything should be made as simple as possible, but no simpler." :)
This reminded me of when I first started to learn about complex numbers. It seemed like a bit of a miracle that you could take a problem, usually with sines and cosines, and (seemingly) make it more complicated by turning everything into complex numbers. Once you'd got the complex exponentials in place, the algebra fell into place, and you could do the work and then 'recover' the answer you were originally looking for by taking the real (or imaginary) part as necessary. Magic.
Of course, when I learned more about complex numbers it became clear that this 'magic' went much deeper. The Cauchy Integral Formula where the function 'inside' some closed boundary is determined entirely by the properties AT the boundary, a kind of 'holographic' principle, always seemed like magic of the highest order to me.
Does AI have the potential to construct this kind of 'magic'? Are we going to see beautiful theorems constructed by it?
I'm not convinced. I share Penrose's view (which he didn't fully nail down the logic on) that, in some sense, the way humans think is "non-algorithmic". In other words, you can't replace what human beings are capable of by some Universal Turing Machine. Penrose thinks you might need quantum mechanics to achieve human-like levels of intuition, but I'm not convinced by that, either.
On the other hand, if we really are just "computers made of meat" then there's no reason, in principle, why we can't (one day) create true AI.
The magic of complex numbers has many facets indeed. My own graduate and post-graduate work was about varieties (objects in n-dimensional space described by algebraic equations) over finite fields (with prime number of elements) and their connection to what happens over the complex numbers. A variety over a finite field has itself a finite number of points, which you can count. And for certain types of varieties (Calabi-Yau ones, for example), these numbers are connected to coefficients of expansions of certain functions (modular forms) with special symmetries over the complex numbers. More magic.
I think that now we have gone very far with the computers, we have to seriously start thinking about the meat, and not in the sense of how to make it obsolete ("gnostic meat lego", as the wonderful Mary Harrington puts it) but how essential it is.
Final remark regarding Penrose: aperiodic tilings (like his kite/dart one) are amazing. A machine could do the tiling (because it is algorithmic), and yet it is guaranteed that the final result has no translational symmetry. Recently, someone constructed aperiodic tilings with a single shape ("einstein"): https://leisureguy.wordpress.com/2023/03/21/aperiodic-tiling-with-a-single-shape/
This reminds me of one of my favorite quotes, which is also my life motto. "All that is too complex is unnecessary, and it is simple that is needed." (Mikhail Kalashnikov) ;-)
Oh, yes, but consider H.L. Mencken: "For every complex problem there is an answer that is clear, simple, and wrong." And Pseudo-Einstein: "Everything should be made as simple as possible, but no simpler." :)
Fascinating! Who is that guy? In the cases that are interesting to me, I'd certainly rather trust Kalashnikov. ;-)
Intriguing. And, no doubt, true.
"Complexity can be found in LLMs only because these are parasitic on human complexity"
Very well said. Loved it.