Posts Tagged ‘Information’

Information Entropy

February 8, 2019

Is there a maximum amount of data about any given subject above which the incremental value of any additional data begins to approach zero?  Even more starkly is there a point where more data about a given subject may actually begin to have a negative effect, in that it actually decreases the amount of information about the subject?

I don’t mean that newer data may prove that old data about the subject is no longer accurate and therefore render the old data out of date. In which case the old data would still exist but no longer be relevant.  I’m talking about a situation where the informational content, i.e. the payload of the data, will actually decrease.  This would be a situation when we know less about a subject at some point in the future than we knew in the past, based on all the data there is on the subject.  I think it would have to have something to do with context, with the passage of time, and the associations between units of data about the subject and data about other subjects.  The more connections, the more information a body of data has.

This is a tricky speculation.  I mean, is it possible to actually know less at some point than was previously known?  Not for just a single person, as sometimes happens as we age and simply forget the information we previously could recall about a subject.  I am talking about the accumulated knowledge about a subject.  It is kind of like a subject becoming simpler over time rather than more complex.  Which is pretty much the opposite of observed reality.

In fact, this just may be what happens as we approach absolute and universal entropy.  I am not a physicist but why would this not be the case?  There would be two aspects to this, not only would universal entropy eliminate any differences between things, but the differences between parties, places and time would cease to exist as well.  Not only would there be less information on all subjects, but there would be less and less subjects to have information about.  Nor would there be anyone to have and be responsible for information, even if it did still exit.