Limits of Complete, Exact Descriptions

A complete, exact description of an object is one from which all properties of the object can be reconstructed. If the object is information, e.g. a text, a data structure, an image or sound file etc., this means that the original data structure can be reconstructed from the description.

If the description is shorter than what is described, the described data must necessarily contain some regularity, order or repetition. The description is then a compression of the original data. The compression is possible because of the order. A completely unordered, random data object cannot be compressed.

If you have some channel through which you receive the information (e.g. a music stream from the internet), you might have a complete description of the signal coming through that channel (e.g. you know every detail of the music coming through the channel). You can then use this description to predict the signal. The knowledge or description may then be viewed as a formal theory about the signal that can be used to derive predictions about it.

Now, if the signal is longer than the description (the knowledge you have about it), it must contain some regularity or redundancy because if the description is exact and complete (i.e. the original information can be recovered from it) and it is shorter than the original information, it is a compressed form of that information. But if the original information can be compressed, it must contain some order.

So for any signal channel for which there is a finite complete description of its content, if the length of observation (say m bytes) exceeds the length of the description (say n bytes), the signal must contain some kind of order because the data can be compressed (into the description).

This means that the signal cannot exhaust the bandwidth of the channel completely if its length exceeds the length of the description. If there is order, there must be some unused bandwidth. You could compress the signal into a shorter signal andthus  free some time slot through which additional information can be sent. So the signal described by the description cannot use up the channel’s bandwidth completely.

An unordered signal, on the other hand, would be a random signal; it would be something like what is called “white noise”. It would not be compressible. There can never be a complete description of such a signal because any description valid for a limited stretch of the signal can be broken by additional observation .

A signal containing order, on the other hand, has a structure. It is not completely random. A channel that contains an ordered signal would not exhaust the channel completely. There would be room for additional signals. But these would not be covered by the description of the original signal.

We can describe our interface with reality (through our senses) as a channel carrying a signal. If we have limited knowledge of reality, i.e. a limited and finite description of it, that knowledge could never exhaust the bandwidth of the channel. It would only describe a subset of reality (except reality had a simple, ordered structure). So whatever knowledge we have, there is room for additional, new or surprising experiences.

If our cognitive system was an algorithm, i.e. a finite set of rules about how to interact with the environment, this algorithm could be viewed as a finite description of reality. It could then only cover part of the bandwidth of our interface with reality. So there could be additional signals (i.e. things happening in reality) not covered by this algorithm. We would then be limited in our ability to understand reality. We could understand reality only if it was very well behaved and regular, but that is not what we are finding.

If, on the other hand, our ability to understand reality (i.e. to discover structures or order in the signal) is unlimited, our cognition cannot be an algorithm. It would instead have to be extensible, so the description of the world could be augmented by new parts. Our cognition would have to be able to change beyond the limits of any single description (or algorithm or formal theory).

If there is a complete description of cognition, i.e. cognition is understandable completely, then cognition would have to be limited and we could only understand an ordered subset of the world. If cognition is universal, there cannot be a complete description of it, i.e. of ourselves. We must be creative in order for our cognition to be universal.

Related articles:

(The picture is from It shows a triangle-shaped signal and the spectrum of that signal, i.e. a plot of the frequencies and amplitudes from which the signal could be reconstructed by adding them. The gaps in the spectrum clearly show that the signal described by it is an ordered structure. The bandwidth is not exhausted, additional frequencies could be squeezed in between. The signal could be described by a simple short text or a small programm.)



  1. interesting.
    the signal must be understood by the creator and the receiver.
    in most compression information is thrown away
    A few thoughts off the top of my head.
    information gathered by the creator of the signal/information may not be sufficient for a receiver to recreate an object exactly but only a facsimile. e.g. I could send you the description, a photograph, and other detail that I perceive about the camera body on my desk but you would not be able to recreate it.
    Star Trek teleporters are doomed.

    1. Thanks for the feedback, it is useful for me to see where I have not yet succeeded to express my ideas clearly. I am currently tryin to get these ideas into a clearer form. From one article to the next, I am circling arround these topics, trying to find a clearer description, because the basic ideas are simple. Your feedback shows that I have not yet found the optimal way to express them, but bit by bit, I think I manage to find a form that is more understandable.

      I was mainly thinking of data objects, like files downloaded from a server, for example.
      I was not very clear here. Obviously, I have to distinguish more clearly. between phyical objects and data opbjects (that might be representing them).

      And I did not yet think here of a sender. If we think of our “sense chanels”, the physical world is the sender. Here, our knowledge (set of all descriptions at a certain time) is always incomplete.

      I was thinking here of compression that is not lossy. Of course, in our perception, a lot of information is simply discarded, but that is something I did not take into account here at this point.

      Since I had not physical objects but data objects in mind, I was not thinking here of teleportation, which I don’t think is possible (or only with limitations).

      The main motive here with this line of thought is to get an argument against certain approaches of “Artificial Intelligence” and approaches to cognitive science based on this type of AI, where it is assumed that there is a fixed structure of thinking processes and a fixed representation language to represent knowledge. I don’t think such a fixed structure exists. My idea is that if the mind has a fixed structure, it can only cover a subset of the world. It would be systematically blind to certain aspects of reality. It would not be able to discover certain instances of order. If the mind has a fixed strcuture it would be limited.

      I think instead that the mind is always extended by additional information comming from the environment. This information becomes active, becomming part of the “software” of the mind. The “software” therefore is changing and extending itself. Any finite exact description of cognitive processes is therefore incomplete. I am thinking of the mind as a system that processes information. It is not a dualist approach. The mid is something physical. But it is not completely describable because it can extend or reporgram itself.

      1. (I’m not sure there is a compression system that is not lossy – at the very least ‘digitisation’ is lossy, at some point an analogue value must become either one digital value or another, ie not what it is.)

        On the question of AI – I agree with you. The mind of man is an analogue beast. We may try to represent the mind in some sort of digital format, but that is the only format we have to represent the mind. (I’m not sure if ‘they’ have managed to represent a ‘maybe’ decision point using digital processing – but I know little of this area.)

        “The mind is something physical. But it is not completely describable because it can extend or reporgram itself.” – absolutely. This very afternoon, I have been throwing darts at a dartboard – I used to play a bit 30 years ago – I could have stood there for a couple of hours and got a little better maybe, but I only spent 15 minutes practicing – I know that during the next 24 hours or so away from the board my mind will be starting to relay/build pathways within itself and I will be better tomorrow than I was today. In a couple of weeks, longer practice sessions will become useful as I fine tune my muscle control with the help of the new pathways in my mind. I don’t need to become frustrated with my seeming lack of progress – may age means that my mind will take a little longer to take care of itself. Oh, and this evening, I’ve had a bit of a conversation with you and in a few minutes I will go and watch a film on TV – not sure AI is anywhere nearer in understanding ‘pleasure’ as such.
        sorry rambling –
        Thank you for the pleasure of exercising my mind.

        1. There are lossy compression schemes which, for example, might compress a picture. The result might be something in lower resolution, so some detail is lost. But that is not exactly what I mean here. There are loss-free compression algorithms. For example, it you ZIP a file, you usually get a smaller file (however, if you do that to a picture, the immeiate result will not be a picture). You can then unzip the file again, i.e. restore the original. There is software where you can increase the storage capacity of a medium like a hard drive. Everything you put into it is compressed and when you take it out, it is restored again. Software for download is often also stored in such ways to make the download faster. Sometimes the compressed files are self-extracting, i.e. the decompression software is bundled with the compressed files into one program. When you execute it, it will write the original files back onto your hard drive. However, there is no optimal compression algorithm. No algorithm can discover and therefore exploit every instance of order in order to always yield the shortest possible compressed form. This means that discovery of order (wich is a large part of what we do in perception as well as in science) requires creativity. It cannot be completely formalized, i.e. every formal description of it is incomplete. If you improve it to cover more cases, the resulting description will in turn be incomplete.

          1. Would it be fair to say you are looking at an closed system in data – but the brain is an open system?
            I think we must also accept that the brain is selective in it’s retention.
            Of course, a fundamental difference between a ‘data system’ based on digital files, drives etc. is that you can switch it off and it’s likely to be intact after a period of time. If you switch of a brain it dies and all is lost. Also a hard drive can be swapped between systems, a brain can’t.
            Good luck with your journey. and thank you for giving me the opportunity to shake my brain cells up a bit – no point in having them if we don’t exercise them.

            1. The brain is certainly an open system, and I agree with you in what you write here.
              Thanks for the discussion, it has shown me some points where I have to express my ideas more clearly so they are better understandable.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: