MSE Research Project Database

Information-theoretic analysis of machine learning algorithms


Project Leader: Jingge Zhu
Primary Contact: Jingge Zhu (jingge.zhu@unimelb.edu.au)
Keywords: information theory; machine learning
Disciplines: Computing and Information Systems,Electrical & Electronic Engineering
Domains:

Information theory and statistical learning theory are closely related, as both fields are rooted in statistics. Numerous works have shown that information-theoretic analysis and techniques provide useful insight to machine learning algorithms. Examples include that information measures (eg, mutual information) provides tight bounds for generalisation errors of learning algorithms, and Fano inequalities can be used to give lower bounds on non-parametric regression problems. In this project, we will use both information-theoretic measures and techniques to study learning algroithms for different scenarios.