6.899 Term Paper: Yuan Qi yuanqi@media.mit.edu March 20, 2002

1 Introduction

The past decade has witnessed many new theories and applications for statistical machine learning. However, most of statistical machine learning techniques are for a predetermined situation; it is static and inflexible; it has flat structure and only deal with attribute (random variable) without any concept of objects. To some extent, these limitations make it hard to apply Bayesian networks to representing complex, structured, and flexible domains. On the other hand, first order logic or relational data model have been used in AI and database communities to have a more expressive representation for data relations. But for learning and inference purposes, they suffer from the disability of modeling the uncertenty of the world.

To address those problems, an interesting research direction is combining relational data model or first order logic with probablity to have probablistic relational learning. On one side, researchers have introduced probability directly into the proof procedure of logic programmings, for example, stochastic logic programs (SLP) []. On the other hand, researchers have combined relational data model with probablity in a way more like extensions of Bayesian networks, for example, probabilistic relational models (PRM) [3] and Bayesian logic programs [1]. In this paper, we try to utilize these new models to address some machine learning problems.

The paper contains two slightly diverse topics. First, we propose new methods of sampling the Bayesian network structures using stochastic logic programs. Then we present a new algorithm for classifying texts based on probabilistic relational models. This two topics will be discussed in part 1 and 2 of this paper respectively.

2 Part 1: Sampling BN Structures using SLP

An SLP can efficiently define a prior over Bayesian nets as follows [].

bn([],[],[]). bn([RV—RVs],BN,AncBN):- bn(RVs, BN2, AncBN2, connect_{n}o_{c}ycles(RV, BN2, AncBN2, BN, AncBN).

% An edge: RV parent of H 1/3:: which_{e}dge([H|T], RV, [H −RV |Rest])‘ : −choose_{e}dges(T, RV, Rest).%Anedge : HparentofRV 1/3 :: which_{e}dge([H|T], RV, [RV − H|Rest])‘ : −choose_{e}dges(T, RV, Rest).%Noedge1/3 :: which_{e}dge([_{H}|T], RV, Rest)‘ : −choose_{e}dges(T, RV, Rest).

Cussens [], and Angelopoulos and Cussens [] use SLP for sampling Bayesian model structures. They use the Metropolis-Hasting algorithm to sample from the

Second, I plan to study some existed approaches for combing attribute and relational learning [1, 3, 2]. Among them, Bayesian logic programs [1] and probabilistic relational models [3] are especially interesting to me.

Third, I can either write a review paper on investigating those approaches and related learning methods [4], or try to develop classification and clustering techniques for relational data based on these existed models.

In short, I think it is promising to combine attribute and relational learning from a Bayesian perspective. Some initial works have been done in this direction. I hope my work could add more interesting results to this research direction.

3 Part 2: Relational Bayesian Text Classification

4 Conclusions

5 Alternative Plans

References

[1] Kristian Kersting and Luc de Raedt, ”Bayesian Logic Programs.” In J. Cussens and A. Frisch, editors, Work-in-Progress Reports of the Tenth International Conference on Inductive Logic Programming, London, 2000.

[2] Liem Ngo and Peter Haddawy, ”Answering Queries from Context-Sensitive Probabilistic Knowledge Bases.” Theoretical Computer Science, 1996.

[3] Daphne Koller and Avi Pfeffer, ”Probabilistic Frame-Based systems.” In Proceedings of the Fifteenth National Conference on Artificial Intelligence, 1998.

[4] L. Getoor, N. Friedman, D. Koller, and A. Pfeffer , Learning Probabilistic Relational Models,Invited contribution to the book Relational Data Mining, S. Dzeroski and N. Lavrac, Eds., Springer-Verlag, 2001.

[5] E. A. Wan and R. van der Merwe, ”The Unscented Kalman Filter for Nonlinear Estimation,” in Proc. of IEEE Symposium 2000 (AS-SPCC), Lake Louise, Alberta, Canada, Oct. 2000. http://citeseer.nj.nec.com/wan00unscented.html

[6] D. J. MacKay, Probable Networks and Plausible Predictions - A Review of Practical Bayesian Methods for Supervised Neural Networks, Network (IOPP), 2000.

[7] D. J. MacKay, Probable Networks and Plausible Predictions - A Review of Practical Bayesian Methods for Supervised Neural Networks, Network (IOPP), 2000.