The deeper reason for the possibility of creativity, I suppose, is the possibility of information storage and the possibility that the processing of information can depend on other information.
Physical systems can store information. They can change their state or configuration under the influence of interactions with other systems or with parts of themselves. The change in their configuration allows inferring something about the state of the influencing system. Therefore, the change can be viewed as storing information about the other (observed) system.
We observe information storage in animals and humans, specifically in their brains, as well as in computers, but generally, a large variety of physical systems can store information in some way.
If a system that has stored some information is exposed to additional information, what happens then may depend on the information stored before. For example, the reaction of an animal to some stimulus may depend on its previous experiences.
Stored information that influences how further information is processed may be viewed as a program. The information stored in the system first programmed it and this program then operates on the information received later. The information received later is transformed by the program and in the process the program might be modified itself.
In order to operate this way, a system needs to consist of parts that can play the role of storing information and of parts that can modify signals representing information depending on signals representing other information. Examples of such components are transistors and neurons. To use a terminology introduced by the German computer scientist Carl Adam Petri, such a component allows for the modification of a “Fluss” (flow) of information by some “Einfluss” (influence).
If the original system is described by a set of laws, one might expect that everything such a system can do would be describable in terms of a formal theory describing those laws. However, if the system can store information and this information can influence what the system does with other information afterwards – in other words: if the system is programmable – the system will be able to record some information from the outside that can then have an influence on its behavior. This information can be new to the system, i.e. it may be information that is not derivable from any theory describing the system initially. If the system can be described by a formal theory (algorithm), this formal theory is a finite amount of information describing what the system does. If the system can store information, it could absorb information from the outside (“outside” here includes random processes inside the system that are not determined by its laws) that cannot be derived from the original formal theory, i.e. it can be reprogrammed and extended. Since the new information can influence what the system does, it can then behave in ways not covered by the original theory.
The proof presented by Dr. Kurt Ammon in his paper http://arxiv.org/pdf/1302.1155v1.pdf shows that it is possible to write computer programs that can be extended by new information in such a way that the scope of applicability of any formal theory that describes such a system can be left. Ammon shows that such a system can calculate a function that cannot be calculated by a Turing machine. The proof shows that this is possible. The ability to calculate functions that are not calculable by Turing machines is what Ammon calls creativity. It means the ability of a system to get out of any given formal theory.
What is ultimately required for this to be possible is just the possibility to store additional information and to process information depending on that new information. Such a system no longer has fixed laws describing it. You start with a simple system that can be described by some fixed laws but then you can add information and by doing so, reprogram the behavior of the system. The original core system with the original laws is still there but as long as you don’t exhaust the information storage capacity of the system (or as long as you are able to add more information storage) you can reprogram the system and change its behavior.
Human beings seem to be able to do things that cannot be described by Turing machines. For example, they can do math in ways for which it can be shown that no Turing machine is able to do this. It is therefore plausible to assume that human beings are creative in the sense defined by Ammon. This means that there are no fixed laws of thinking and that human cognition cannot be described completely by a single formal theory. For each formal theory describing human cognitive processes, the human mind can be reprogrammed by acquiring additional information that cannot be derived from the give theory. The mind is extensible.
If this is true, there is no way to describe human cultures by formal theories. While formal theories are capable of covering sub-sections of culture, humans would always be able to invent or discover things outside the realm of the given theory.
One consequence of this is that the type of explanations used in physics or other sciences, where phenomena are derived from laws, cannot be extended to psychology and the study of culture. The development of cognition and the development of cultures are necessarily historical, being subject to change. The methods of science, as a result, are only special cases. Science cannot be extended into the study of more complex areas, especially into the study of the human being and its psychology and culture, without fundamentally changing the concept of science itself, because in psychology and cultural studies (as well as in evolutionary biology) we are dealing with system whose “laws” are subject to historical change. If you have a complete step by step record of what is happening in such a system, you can describe its history exactly and there is nothing mysterious about it. However, there is no way to describe all its possible developments in terms of a single formal theory or set of laws. If you describe the processes on the lowest level of physical processes, the description becomes meaningless and is too large to be practically treatable. Moreover, a complete record of events will normally not be available. While this is no problem in the classical sciences, where experiments can always be repeated and refined, it is problematic in disciplines where you are dealing with unique historical developments. Attempting to use statistical laws, as is done in parts of psychology and the social sciences, is only partially helpful because these “laws” can only be approximations and are themselves subject to historical change.
Programmability of systems thus results in a situation in which the classical epistemological model of the sciences is not applicable. This is the deeper reason, I think, for the divide between sciences and other academic disciplines. Humans and cultures are programmable and thus creative systems that have a history and changeable “laws”. No theory is possible that can describe them in their entirety. However, their study must not be considered as unscientific. It is possible, I think, to develop a more general concept of “science” that encompasses all of these disciplines, although in this generalized concept of “science” we would have to deal here with incompleteness, historicity and vagueness. Since a fixed concept and methodology of science would then no longer be possible, instead science would have to be defined as an attempt to describe phenomena as exactly and generally as possible, knowing that both of these goals cannot be achieved completely at the same time. As a result, we would have to critically reflect our concepts and methods all the time. The classical sciences would only appear as simple special case (of describing non-historical systems), while in more complex areas where scientist are not evading the complexity of reality, the generalized science would look more like the humanities.
These considerations point at the possibility of an ontological reductionism: phenomena of the human mind and of human cultures are ultimately physical processes (describable as processes of information processing). But methodological reductionism would be impossible because the one physical reality produces more phenomena than can be described or derived in any single formal theory. Scientists are describing one reality, but in doing so they have to use a multiplicity of concepts and methods.
(The pictures are from