From Course 6.867

Jump to: navigation, search


Data repositories

Machine learning algorithm libraries

Journals and conferences

Journal papers are usually easier to read, because they are longer and have better exposition and motivation.

Possible papers to read and replicate

Supervised Classification

  • Fei Sha and Lawrence K. Saul, Large Margin Hidden Markov Models.
Advances of Neural Information and Processing System, 2006. paper
  • Ryan Rifkin and Aldebaro Klautau, In Defense of One-vs-All Classification.
Journal of Machine Learning Research, Volume 5 (Jan): 101-141, 2004. paper
  • Ben Taskar, and Carlos Guestrin and Daphne Koller, Max-Margin Markov Networks.
Advances of Neural Information and Processing System, 2003. paper
  • Andrew Y. Ng and Michael I. Jordan, On Discriminative vs. Generative Classifiers: A Comparison of Logistic Regression and Naive Bayes.
Advances of Neural Information and Processing System, 2001. paper
  • Yoav Freund and Robert E. Schapire, Large Margin Classification Using the Perceptron Algorithm.
Machine Learning, Volume 37, Issue 3, 1999. paper
  • Robert E. Schapire, Yoav Freund, Peter Bartlett, and Wee Sun Lee, Boosting the Margin: A New Explanation for the Effectiveness of Voting Methods.
The Annals of Statistics, Volume 26, Issue 5, 1998. paper
  • Thorsten Joachims, Text Categorization with Suport Vector Machines: Learning with Many Relevant Features.
Proceedings of the European Conference on Machine Learning, 1998. paper
  • Rich Caruana, Multitask Learning.
Machine Learning, Volume 28, 41-75, 1997. paper
  • Nick Littlestone, Learning Quickly when Irrelevant Attributes Abound: A New Linear-Threshold Algorithm.
Machine Learning, Volume 2, Issue 4, 1988.
And see later work on multiplicative update algorithms.

Semi-supervised Classification

  • Andreas Argyriou, Mark Herbster, and Massimiliano Pontil, Combining Graph Laplacians for Semi–Supervised Learning.
Advances of Neural Information and Processing Systems, 2005. paper
  • Thorsten Joachims, Transductive Inference for Text Classification Using Support Vector Machines.
Proceedings of International Conference on Machine Learning, 1999. paper
  • Avrim Blum and Tom Mitchell, Combining Labeled and Unlabeled data with Co-Training.
Proceedings of the 11th Annual Conference on Computational Learning Theory, 1998. paper

Unsupervised Learning

  • Lawrence K. Saul and Sam T. Roweis, Think Globally, Fit Locally: Unsupervised Learning of Low-Dimensional Manifolds,
Journal of Machine Learning Research, Volume 4, 119-155, 2003. paper
  • Michael E. Tipping and Christopher M. Bishop, Probabilistic Principal Component Analysis,
Journal of Royal Statistics Society (B), Volume 61, Part 3, 611-622, 1999. paper

Reinforcement Learning

  • T. G. Dietterich, Hierarchical Reinforcement Leanring with the MAXQ Value Function Decomposition.
Journal of Artificial Intelligence Research, Volume 13, 227-303, 2000. paper

Graphical Models and Inference

  • Chong Wang, Bo Thiesson, Christopher Meek, and David Blei, Markov Topic Models.
Proceedings of the 12th International Conference on Artificial Intelligence and Statistics (AISTATS) 2009. paper
  • David Blei and Jon McAuliffe, Supervised Topic Models.
Advances of Neural Information and Processing Systems, 2007. paper
  • David M. Blei, Andrew Y. Ng, and Michael I. Jordan, Latent Dirichlet Allocation.
Journal of Machine Learning Research, Volume 3, 993-1022, 2003. paper
  • Zhe Chen, Bayesian Filtering: From Kalman Filters to Particle Filters, and Beyond.
Statistics, 2003. paper
  • Jonathan S. Yedidia and William T. Freeman, and Yair Weiss, Understanding Belief Propagation and Its Generalization.
Exploring Artificial Intelligence in the New Millennium, Chap. 8, pp. 239-236, 2003. paper
  • Matthew J. Beal and Zoubin Ghahramani, The Variational Bayesian EM Algorithm for Incomplete Data: with Application to Scoring Graphical Model Structures.
Bayesian Statistics 7, 2003. paper
  • John Lafferty, Andrew McCallum, and Fernando Pereira, Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data.
Proceedings of 18th International Conference on Machine Learning, 2001. paper
  • C.K.I. Williams, Prediction with Gaussian Processes: From Linear Regression to Linear Prediction.
Learning in Graphical Models, pages 599-621. The MIT Press, 1999. paper

Model Selection

  • Michael Kearns, Yishay Mansour, Andrew Y. Ng, and Dana Ron, An Experimental and Theoretical Comparison of Model Selection Methods.
Proceeding of 18th Annual Conference on Computational Learning Theory, 1995. paper


  • Guy Shani, Davic Heckerman, and Ronen I. Brafman, An MDP-Based Recommender System.
Journal of Machine Learning Research, Volume 6, 1265-1295, 2005. paper
  • Thomas Gaertner, John Lloyd, and Peter Flach, Kernels and Distances for Structured Data.
Machine Learning, Volume 57 Issue 3, 2004. paper
You'd need to know something about logic to pursue this one.
Personal tools