Please enable JavaScript.

Coggle requires JavaScript to display documents.

Sense of randomness (we might consider we have an intuitive grasp of…

- - - - which might be incomprehensibe to us
  - - - Ignorance is the quality of the observer
        
        Hence it is subjective
      - But randomness if it exists is the quality of the thing itself, not the observer
- - - - it lowers the entropy
      - otherwise it increases the entropy
      - this is how Shannon links information to a measure of order or randomness
- - - - redundancy means less information content
    - - this will save us characters
      - cost of sending message will be lower
  - - - Probablity theory treats events statistically
  - - - which makes it random if we follow the idea randomness is opposite of predictability
      - but we know how to compute pi
        
        that is we can arrive at the same string repeatedly after computation
        
        can a particular number be random?
  - - - An ideal language free from real world messiness
      - it would run on all computers
    - - Chaitin used this program size to measure the degree of randomness of a number
        
        how some numbers are random than the other
        
        this was Chaitin's algorithmic definition of randomness
      - but the program size could also express the measure of information
        
        this was Chaitin's algorithmic definition of information
    - - It was titled Three Approaches to the Definition of the Concept 'Amount of Information'
        
        It dealt with measuring information in a finite object
        
        The object could be message or data
        
        He suggested three approaches
        
        Combinatorial #
        
        Probabilistic
        
        These two approaches extended on Shannon's work
        
        Probability of one object from the ensemble of possible objects
        
        But these methods were limiting
        
        if the message was highly complex
        
        2 more items...
        
        How would measure the amount of information in Tolstoy's War and Peace?
        Is it possible to include this novel in a reasonable way in the set of 'all possible novels' and further to postulate the existence of a certain probability distribution in this set.
        
        Algorthmic
        
        Focused on the entity itself
        
        instead of the ensemble of objects
        
        Kolmogrov associated a new term with the object
        
        Complexity
        
        The complexity of an object is the size of the smallest computer program needed to generate it.
        
        3 more items...
      - But Chaitin couldn't read Russian. So, he was not much aware of Kolmogrov's work
        
        Similarly Kolmogrov was under the socialist regime of Stalin
        
        Study of information and cybernetics were considered ideological weapons
        
        Kolmogrov did a lot of redundant work unaware of Chaitin's work
    - - but we cannot prove a number to be uninteresting
      - this led to Chaitin's version of Godel's incompleteness
        
        Complexity defined in terms of program size is generally uncomputable. Given an arbitrary string of million digits, a mathematician knows that it is almost certainly random, complex and patternless – but cannot be absolutely sure.
      - we could also view this as an information packing problem
        
        Shannon had already worked on it
        
        Ray Solomonoff took interest in it
        
        How much information could one pack into a given number of bits, or conversely, given some information, how could one pack it into the fewest possible bits.
        
        This resembles scientific deductions
        
        Laws of science are akin to data compression
        
        How do people create theories to account for their experience in the world? They have to make generalisations, find patterns in data that are always influenced by randomness and noise.
        
        But can we teach this method of deductions to computer?
        
        In other words, could a computer be made to learn from experience?
        
        Through his quest, Solomonoff had actually anticipated the essential characteristics of the algorthmic information theory
- - - - It contained music, sounds of modernity, nature and humanity
      - But to aliens what would it all mean
        
        They are just patterns etched on a disc
        
        Would they find the well formed patterns of a piece by Bach different from noisy patterns of birds chirping
        
        Can this well thought patterns convey presence of intelligent life
        
        Does the music carry its own logic with it, independent of geography and history?
  - - - It is simple the way a Rembrandt drawing is simple. It does a lot with little.
      - Complexity of music as in any art remains deeply unsatisfying if measured in a minimal program size.
        
        If we consider Bach's C major prelude , much of its contents, the tempo, timbre and melody has tremendous redundancies
        
        The very repetition creates expectations and breaks them.
        
        Everything we care about lies somewhere in the middle, where pattern and randomness interlace.
        
        It means it is not necessary that if a message or an art contains more information it will be more enjoyable. The art lies in building expectation through redundant patterns and then suddenly breaking it
        
        Hence complexity is not a good measure of value of a message
        
        Charles. H Bennett introduced a new mesaure
        
        Logical Depth of a message
        
        He proposed the value of a message lies in "what might be called its buried redundancy – parts predictable only with difficulty, things the receiver could in principle have figured out without being told, but only at considerable cost in money, time and computation.
- - - - The string exhibits quality of randomness
        
        so is it information rich?
    - - what if we were sending the message to the aliens
      - sending the algorithm / the symbol is making the assumption that the sender and receiver share a same mental codebook