Prototype and analysis based on the Information Bottleneck by Tishby et al. (1999)
The Information Bottleneck (IB) is a statistical framework which aims at obtaining an intermediate representation of an 'input' random variable, in relation to an 'output' random variable. The IB has been applied to discrete-valued data (such as text), continuous-valued data (such as image data), as well as serving as a formalism in deep learning models. This code implements the information bottleneck algorithm introduced by Tishby et al. (1999).