Complex systems emerge from simple rules

OMG one must wonder about a system that produces this article and especially these comments. If you are interested in this area you might consider:

* Hidden Order by John Holland. Especially if you are technical, then read it from the perspective of "how would I build one of these"? (See RIP).

* Signs Of Life: How Complexity Pervades Biology. An overwhelming book. At the very least you should come out aware of how little we understand.

* What is money? Really. We all know the mythical person month perspective, but according to that New York City can not exist.

* The RIP routing protocol.

* A New Kind of Science.

Gestalts, created from complex adaptive systems, surround us. (Inside joke). I think this is an example of the old adage "I won't see it until I believe it" which is just a simplified version of Sapir Whorf.

[edit: add Wolfram's "A New Kind of Science" ]

Utterly banal. For one thing, to say that life supervenes upon chemistry which supervenes upon physics is actually taking the _opposite_ position than "life is emergent," at least to the extent that philosophers agree about what emergent means. I understand emergence to mean what Deacon means: the properties of the emergent system are independent from the properties of the system underneath it at least to the extent that you couldn't predict the behavior of the emergent system with just the physics, at least not without a lot of work and the specific initial conditions required. The idea being that many possible systems could serve as the basis for the emergent system and that the emergent system is some kind of set of self-reinforcing constraints imposed on the underlying system.

As it happens, I don't believe this argument if taken all the way. Sure, emergence has the property that, in some sense, one knows more about the emergent system if it is described at its own level rather than at the level of the underlying system (eg, if you tell me you are hungry I'm much more likely to correctly predict you will order pizza than if you give me a catalog of the state of all your neurons), but your behavior still supervenes upon the neurons inexorably. Even the constraints that maintain the emergent system ultimately are expressed in terms of the underlying system.

Look at it this way: life on Earth is composed primarily of C, H, O, N , and a number of trace atoms.

If life evolved here on Earth, then somehow, CHON+trace all self-organized into all of Earth life today, including us, and all we humans have done and will do in the future.

Now: say we could go back just before life evolved. Even with very very good data, and with whatever talent (AI, science, anything) and technology from today, would we be able to truly describe the emergence paths for those CHON+trace atoms?

Impossible. This would be far beyond our level of science and technology. We can't even do this for much simpler systems and shorter time horizons. You'd have to predict the emergence of life, the full properties and behavior of all forms of life in the last 4 billion years, and last but not least, humans and all that humans have done and thought since then.

Yet clearly nature emerged all this from perhaps small amounts of 11 elements or so. Complexity is one of the greatest unknowns in our present civilization.

One way of thinking is that the rules are designed by humans. The game is not a game until we establish the rules. At any point in time in this realm of life, if we try to establish some control, or try to create a frame of "simple" rules around it, it becomes complex.

So it doesn't matter if the ChatGPT is designed by simple rules, the minute we try to control the randomness, it becomes complex.

Most systems don’t have interesting stable states like Conway’s Game of Life, they either collapse or explode. You can verify this yourself by changing the rules of the Game of Life.

LLMs aren’t simple systems either, they are density estimators pretrained on terabytes of text, then fine tuned with human feedback. The simple fact that the author believes this suggests he doesn’t know much about LLMs.

Not a great analogy, since Game of Life in Game of Life didn't emerge, it was programmed by a human.

Other people might be interested in Stephen Wolfram's approach to physics[0].

[0]: https://wolframphysics.org/