Forêts Aléatoires PAC-Bayésiennes

Authors: Zirakiza, Brice
Advisor: Laviolette, FrançoisMarchand, Mario
Abstract: In this master's thesis, we present at first an algorithm of the state of the art called Random Forests introduced by Léo Breiman. This algorithm construct a uniformly weighted majority vote of decision trees built using the CART algorithm without pruning. Thereafter, we introduce an algorithm that we called SORF. The SORF algorithm is based on the PAC-Bayes approach, which in order to minimize the risk of Bayes classifier, minimizes the risk of the Gibbs classifier with a regularizer. The risk of Gibbs classifier is indeed a convex function which is an upper bound of the risk of Bayes classifier. To find the distribution that would be optimal, the SORF algorithm is reduced to being a simple quadratic program minimizing the quadratic risk of Gibbs classifier to seek a distribution Q of base classifiers which are trees of the forest. Empirical results show that generally SORF is almost as efficient as Random forests, and in some cases, it can even outperform Random forests.
Document Type: Mémoire de maîtrise
Issue Date: 2013
Open Access Date: 19 April 2018
Grantor: Université Laval
Collection:Thèses et mémoires

Files in this item:
29815.pdf1.79 MBAdobe PDFView/Open
All documents in CorpusUL are protected by Copyright Act of Canada.