Information Storage Capacity and Creativity


In my previous article, I have introduced some aspects of the theory of creativity developed by Kurt Ammon. In this article, I am going to develop these thoughts a bit further.

Kurt Ammon provided a mathematical proof showing the possibility of computer programs to develop out of the scope of any given formal theory about them. I viewed these formal theories as boxes and the creative process as a process of “thinking out of the box”.

The mathematical proof given by Ammon assumes an unlimited amount of memory. This is the kind of assumption you will always meet in mathematics. In reality, however, you always have a limited amount of memory.

The idea of this article is that while that limited memory of real computers also means there is some “box”, you can always add more memory and thus leave that box.

Any real computer has only a limited information storage capacity (the Giga- or Terabytes of its storage device). The historical hard drive on the picture above, for example, had a (once staggering) capacity of 512 Megabytes.

It is possible to show that a computer with a limited storage capacity can in principle be completely described by a formal theory.[1]

Now, Ammon defined creativity as the ability to leave the scope of any given formal theory. If there is a formal theory describing completely what the computer can do, it cannot be really creative, or can it?

A picture might make it easier to understand the situation. The blue oval symbolizes the amount of memory available for a given computer. The green oval symbolizes a formal theory capable of describing what certain computer programs are able to do, including everything that can happen inside a computer with the amount of memory represented by the blue oval.


The red line symbolizes the development of a creative computer program. In the “blue” computer, it would hit the memory limit and abort. If you add memory that fits into the green oval, the theory symbolized by the green area can still describe what the program is doing (the red line inside the blue and green area). But that formal theory is limited. The program can leave the “box” described by it. If the program is creative, then by adding more memory still (denoted here by the lavender colored ellipse), the program can develop out of the scope of the “green theory”.

So for any given formal theory (green) that is describing what programs will do on computers with a certain amount of memory (or less, i.e. the blue area) there is a larger amount of memory so that the program can “escape” from the limits of the theory. What happens in the “lavender” area is no longer predicted or described by the “green theory”. It is something qualitatively new with respect to the “green theory”.

Of course you could think of another, larger theory around the “lavender” area, describing anything inside it. For such a more comprehensive theory, events in the lavender area would not be qualitatively new, but by adding still more memory (another, still larger ellipse around that one), the system can leave that outer theory as well, and so on.

This means that although there is a limit at any time, by adding memory to a computer, creative programs running on it may be made to behave in qualitatively new ways. The system can develop innovations.

The qualitatively new is new only with respect to a preexisting theory. It is always possible, in hindsight, to create larger theories in which these developments are ordinary processes, so there is nothing mysterious about them.

If we generalize this idea, it means that if we add enough information storage capacity to a system, the system can show phenomena or behaviors that are new with respect to any previously given theory about it, i.e. they are qualitatively new phenomena. I hypothesize that this does not only apply to computers, it applies to physical systems in general, to organisms, nervous systems (brains), as well as groups of humans (societies, cultures).

The information storage capacity of physical systems can be increase by adding energy or making the system larger (increasing the number of particles in the system or increasing the number of possible arrangements of these particles in space). For example, a small dust cloud of gas and dust will never do anything interesting, but the large cloud from which the solar system emerged was able to develop planet earth, life, humans, civilizations, you and me, and this blog article. It was able to do so because with its many particles and its large size, it can store a much larger amount of information. So the “creative capacity” of a system depends on its information storage size.

The information storage capacity of organisms can be increased by adding more genes or by making the organism multicellular. The information storage capacity of nervous systems can be increased by adding neurons or increasing the number of connections between them.

When humans developed, I think what essentially happened was that the information storage and processing capacity of their brains became larger. I don’t think that a lot of sophisticated structure arose. There is, I think, very little pre-programmed structure in our brain. The sophisticated structures arise through learning processes, through creativity, but by increasing the storage capacity of the brain beyond some threshold, qualitatively new things could happen. Humans arose, not by the genetic evolution of some super-intelligent complex and intricate neural algorithm, but by increasing the capacity of the brain, making room for the creative processes to develop into something new, intricate and complex. It looks like the capacity of our brains is so large that we cannot exhaust it within our lifetimes. And the societies and cultures we form can store more information still.

In the course of cultural developments, there is a trend to increase the information storage capacity of the culture. First, humans developed larger vocabularies and mnemonic aids like songs, poetry and rituals to make sure important information was not lost. Then, division of labor in larger communities increased the total amount of information a culture could pass on to the next generation, although now every single person knew only a fraction of the total information. The inventions of writing, printing and finally electronic storage media further increased the storage capacity. Each time the information storage capacity was increased, completely new phenomena could develop. Today’s computers with their vastly increased storage capacity allow for applications unthinkable a few decades ago and the internet with its virtually unlimited storage capacity has led to completely new cultural phenomena.

[1] The reason you can describe what a computer can do in a finite formal theory is that there is only a finite (although extremely large) number of different states the computer can have. This means you can fill the computer’s storage with information in a very large number of ways, but this number is finite. Since the computer is a deterministic machine, it will go from one state to the next in a predictable way. As a result, all possible processes in it can in principle be described with a finite theory (although in practice, it is not possible because the number of possibilities is extremely high).

(The picture of the historical hard drive is from



  1. Reblogged this on The Asifoscope and commented:

    I have added another article about the theory of creative systems to my philosophy blog. This article is a little bit more technical. The basic idea is that if information storage capacity of a system is increased, qualitatively new phenomena can arise in the system.

  2. This works well with my idea about non-physical or non-adaptive evolution in humans.

    We stopped using physical change/mutation as our evolutionary method. We don’t change physically to adapt. Instead we use our cognitive/creative abilities to alter the environment to suit our requirements.

    Not necessarily the best thing to do without restraint or forethought, as is becoming painfully obvious.

  3. Exactly. simple animals might be completely described by some formal theory. You can think of them as an algorithm. The have a limited way of interacting with and impacting the environment. Creativity allows to find ever new ways of interacting and thus of using (up) resources. Since the resources are limited, this is becoming destructive.

  4. The question now becomes, what do we do about it? Will we be able to use that same “creativity” to make the necessary fundamental changes in our relationship with Life? Or will we persist in our present self-destructive actions?

  5. […] (INFORMATION STORAGE CAPACITY AND CREATIVITY) I have argued that for systems with a fixed information storage capacity, it should be possible to […]

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: