Model Growth

As a result of a brain storm session on learning theory I wrote a paper on Model Growth and submitted it to BeNeLearn 2002. I presented this paper using hand-written slides. I converted those slides into electronic form using OpenOffice.org Impress:

• Model Growth presentation - PowerPoint version. (There are a few minor tabulation problems, but it is otherwise okay.)

PDF

Contents

1. Introduction
2. Example
3. The algorithmic approach
   Application to the example
4. The Minimum Description Length approach
   Application to the example
5. Algorithmic statistics
6. Conclusion
   Further research

References

Abstract

The algorithmic method of inductive inference that Ray Solomonoff proposes in [Sol64] is not interactive. Marcus Hutter defines how to add interactivity to the inductive method based on the assumption that the environment supplies a utility function [Hut00]. This paper discusses the possibility of a framework based on a utility function that is internal to the learning subject and independent of the environment. The internal utility function should measure the amount of information extracted from the interaction with the environment. The Minimum Description Length principle (MDL) proposed by Jorma Rissanen [Ris89] supplies a framework that clearly separates a statistical model that represents the extracted information from the exact representation of the data. Algorithmic Statistics [GTV01] should be able to bridge the gap between the algorithmic approach of Solomonoff and the statistical approach of Rissanen.

BibTeX entry:

@InProceedings{Maa02,
  author =       "van~Maanen, Jeroen",
  title =        "Model Growth",
  booktitle =    "Proceedings of the Twelfth Belgian-Dutch
                  Conference on Machine Learning",
  editor =       "Marco Wiering",
  year =         "2002",
  publisher =    "Utrecht University",
  address =      "Utrecht"
}

Visit the LEIA project page on GitHub for project details.