Theoretical Approaches to Compression under Modern Constraints

Overview

Classical source coding theory assumes long blocklengths, i.i.d. sources and a decoder whose only job is to reconstruct the data. Autonomous driving, the Internet of Things, distributed sensing and federated learning break these assumptions. Data must be sent under strict delay constraints, sources have memory, devices compress correlated data separately, and the receiver may need an inference as well as a reconstruction.

This project extends compression theory to these settings, with an emphasis on logarithmic loss, a distortion measure for probabilistic predictions that connects to the information bottleneck framework. For single-shot compression with joint inference and reconstruction, we bound the probability of excess distortion, with an easily computable achievability bound when reconstruction is measured by logarithmic loss. For the one-shot Wyner–Ziv problem, we derive upper and lower bounds on the entropy–distortion function of a uniform source with quantized or noisy side information. Further work studies stationary sources under logarithmic loss.