We might think of the world and ourselves as a layered system, like an onion. These layers have no fixed borders, they are just a way to orient ourselves in the matter, like a map, but looking at the matter this way might be useful.
We may think of the human being as the core and the world as the outer layer. We might divide the world into an outer layer not influenced by us (“nature”) and an inner layer of artifacts and things influenced by our actions, containing technological objects and systems, waste and destruction, buildings and clothes, and so on (“culture” or “civilization”).
The human being has traditionally been divided into a body and a mind. The body contains sensors (ears, eyes etc.) and effectors (hands, legs, etc.) that can act upon the world. It also contains the brain in which the mind is implemented, and some “support infrastructure” to keep all of that running, but for the sake of our simple onion-model, we might view it just as a set of sensors and effectors, providing the interface between the mind and the world.
The mind could also be viewed as a layered, onion-like system that we can conceptually describe as a set of nested spheres. The outer layer consists of data. The data either enter the mind through the senses (sense data) or they are the product of cognitive processes that act on other data.
The next layer, further inside, consists of what is doing that processing. We might think of it as a set of programs. They may take data as inputs and produce data as outputs or influence the effectors.
Up to this point, our model can just as well describe a human being or a house fly, even a simple machine like an automatic vacuum cleaner, or a simple organism like a paramecium whose simple actions and reactions can also be described as simple processes of information processing that can be modelled as simple programs.
The next level makes things more complex. It consists of meta-programs, i.e. programs that produce other programs. It enables the system to learn something. Such meta-programs can be very simple, combining data and programs into simple new programs, or they can be very sophisticated. Animals that can learn, i.e. develop new patterns of behavior or of cognitive processing, can be thought of as having some meta-programs that combine data and programs into new programs. This includes the possibility to form new meta-programs as well. However, each meta-program contains only a limited amount of programming-knowledge, so such systems are limited.
I am going to try to point out the reason for this limitation in a later article in more detail. I will only hint at it here, knowing that the following section of this article might only be partially understandable at this moment:
Each program on the program level runs with limited resources of storage space and time. It will always come to an end after some time. Otherwise, the system would run into infinite loops. This does not happen in actual cognitive systems like animals and humans (although it might happen in computers). Since the programs always yield a result after a limited time (even if it might be a kind of “error message”), one can view them as calculating what is called total functions, i.e. functions that are defined on all of their inputs. Now it can be shown (and I will try to explain the proof in a later article) that the set of programs calculating total functions is not Turing-enumerable, i.e. there is no algorithm that can produce all of them. No matter what algorithm you are using to produce such programs, it is going to turn out to be incomplete.
That means that any system that can be described as an algorithm consisting of meta-programs as its components, i.e. a system that produces new programs on the program level, is incomplete.
This is the core problem of intelligence. The programs constitute the knowledge or know how of the system. An intelligent system must be able to produce arbitrary new knowledge, and this cannot be done by an algorithm. So an intelligent system must be non-algorithmic in some sense.
However, it is possible to have a process that produces new meta-programs. However, such a process will have a paradoxical property: you may integrate it into the algorithm, but the resulting algorithm will be incomplete again. However, you can apply that meta-meta-process “from the outside” of the algorithm and get a new, extended version.
That meta-meta-process is the core of the onion. It can be viewed as creativity. In order to be truly intelligent, an animal or other system needs such a creative core. You may view the program layer and the meta-program layer together as an algorithm. The creative core can nudge this system out of its fixed algorithmic lane and put it on a new one, it can transform it into a more powerful system by extending it with new meta-programs that where not part of the previous capabilities of the system. A creative core of this kind is what is required to make a system truly intelligent.
One way to do so is by using what is known in mathematics (in the theory of computability) as a “productive function” (this possibility was pointed out to me by Kurt Ammon). It is possible to construct for any given set of meta-programs to construct a new one that can do something the programs in the original set could not (i.e. a meta-program that can produce new programs on the program level). The construction works through what is called a “diagonalization”. I hope to be able to make these concepts understandable in a future article.
Another possibility might involve random processes. The processes of genetic evolution that change the brains of animals during their evolution might also be put here. This process of genetic evolution forms the creative core of simple animals and simple organisms, but in more complex organisms, the creative core may be a neurological entity.
So let’s go back from the inside of the onion to the outside in order to sum things up: At the core of the system is a creative core that can extend the meta-programs, i.e. the mechanisms that produce new programs in the system, and extend them in such a way that the system can learn new things (i.e. generate new programs) it could not generate before, i.e. it can learn totally new ways of learning. Let’s call this the level of creativity or Level 0.
The next layer is the layer of meta-programs, i.e. programs that produce other programs, e.g. by combining programs and data into new ones. One could say that these programs contain programming-know-how. We can call this the meta-(program)-layer or learning-layer or Layer 1.
The next layer is the layer of programs that act on data. These represent both skills in doing something with the effectors or the system (i.e. motor knowledge (movement skills etc.) and knowledge of how to produce language) and cognitive or thinking skills, i.e. ways or patterns of thinking, e.g. the know-how of doing logical inferences or doing calculations. We can call this the program layer or skill layer, or Layer 2.
The programs in Layer 2 act on data that forms the data layer or Layer 3. The data in the data layer originates in the senses or results from the execution of programs in layer 2. We may think of the data as representing information or knowledge about the world, including the cognitive processes inside the system itself.
Layers 1, 2, and 3 together may be modelled as an algorithm. In each stage of its development, it might be described by a formal theory. The system that consists of Layers 0 to 3, however, cannot be modelled as an algorithm or formal theory. The extension process of layer 0 might depend on a “trigger” or a random event that originates in layer 4 (and ultimately 5 and 6) and lies outside the mathematical structure of layers 1, 2, and 3, i.e. it might depend on the physicality of the system.
Layer 4 then consists of the sensors and effectors through which we interact with our surroundings.
Layer 5 consists of the world around us that we influence. We can call this the layer of culture or of civilization.
Layer 6 is what is outside that sphere of influence. We may call it the layer of nature.
(The picture is from https://commons.wikimedia.org/wiki/File:Red_onion_rings_closeup.jpg.)