A file gets smaller when you can guess what is in it
A file gets smaller when you can guess what is in it.
You zip a file and it gets smaller. Your phone guesses the next word in your message. An essay published yesterday shows those two obey the same rule.
A bit is one yes-or-no answer, and likely things need fewer of them
A bit is one yes-or-no answer, and likely things need fewer of them.
Picture five animals, each half as likely as the one before. A bird takes one yes-or-no answer to pin down. A bear takes four. Those answers are the bits, and every file on your computer is priced that way.
Only one message fits the range that is left
Only one message fits the range that is left.
A method from nineteen seventy-six writes a whole message down as one number. Seven characters cost ten bits. A twelve-character string that is mostly one repeated letter costs the same ten. That method lands within about one percent of the smallest size the odds allow. The writing-down half is finished work.
The guess changed, and the phrase came out half the size
The guess changed, and the phrase came out half the size.
Take the phrase TO BE OR NOT TO BE. Written out plainly it costs a hundred and forty-four bits. Price each letter by how often it appears and the phrase costs forty-seven. Price each letter by the letter in front of it and the phrase costs twenty-one. A U is a rare letter in English. After a Q, the U is close to certain. So the guesser is the only part left worth improving. The best guessers we have for text are language models, trained to guess what word comes next. Give that model the text. The real next word is already known, so the model only prices it.
The 2019 model more than halved the letter-by-letter price
The 2019 model more than halved the letter-by-letter price.
The author ran the same test on a famous sentence from Charles Dickens. Pricing each letter by the letter before it got the quote down to a quarter of its plain size. Then GPT-2, a language model from twenty nineteen the author calls archaic, wrote the same quote at a tenth.
Sending the guesser costs more than the guessing saves
Sending the guesser costs more than the guessing saves.
Your browser is not using any of this. Both ends of the connection need the same model to open the file. So every visitor downloads several gigabytes to save a few kilobytes.
A file gets smaller when you can guess what is in it
The writing down is finished work. The guessing is not. Every bit you save from here comes from a better guess.
