MSE Research Project Database

Information theoretic approaches to deep learning


Project Leader: Tansu Alpcan
Staff: Margreta Kuijper
Primary Contact: Tansu Alpcan (tansu.alpcan@unimelb.edu.au)
Keywords: communications and networks; complex and intelligent systems; machine learning; optimisation
Disciplines: Electrical & Electronic Engineering
Domains:

The goal of this project is to investigate information and coding theory-based approaches to analyse existing Deep/Machine learning schemes as well as use the insights obtained to develop novel learning methods. The Information Bottleneck Principle by Tishby et al. provides an excellent starting point in this direction by building a link between rate distortion theory and deep learning. However, the area is still in its infancy with many fundamental open research questions.