FEIT Research Project Database

Information-theoretic analysis of machine learning algorithms

Project Leader: Jingge Zhu
Primary Contact: Jingge Zhu (jingge.zhu@unimelb.edu.au)
Keywords: machine learning
Disciplines: Computing and Information Systems,Electrical & Electronic Engineering

Information theory and statistical learning theory are closely related, as both fields are rooted in statistics. Numerous works have shown that information-theoretic analysis and techniques provide useful insight to machine learning algorithms. Examples include that information measures (eg, mutual information) provide tight bounds for generalisation errors of learning algorithms, and Fano inequalities can be used to give lower bounds on non-parametric regression problems. In this project, we will use both information-theoretic measures and techniques to study learning algroithms for different scenarios.