Natural language processing (NLP) went through a profound transformation in the mid-1980s when it shifted to make heavy use of corpora and data-driven techniques to analyze language. Since then, the use of statistical techniques in NLP has evolved in several ways. One such example of evolution took place in the late 1990s or early 2000s, when full-fledged Bayesian machinery was introduced to NLP. This Bayesian approach to NLP has come to accommodate for various shortcomings in the frequentist approach and to enrich it, especially in the unsupervised setting, where statistical learning is done without target prediction examples.
We cover the methods and algorithms that are needed to fluently read Bayesian learning papers in NLP and to do research in the area. These methods and algorithms are partially borrowed from both machine learning and statistics and are partially developed "in-house" in NLP. We cover inference techniques such as Markov chain Monte Carlo sampling and variational inference, Bayesian estimation, and nonparametric modeling. We also cover fundamental concepts in Bayesian statistics such as prior distributions, conjugacy, and generative modeling. Finally, we cover some of the fundamental modeling techniques in NLP, such as grammar modeling and their use with Bayesian analysis.
Table of Contents
Bayesian Grammar Models
About the Author(s)Shay Cohen
, University of Edinburgh
Shay Cohen is an Assistant Professor at the Institute for Language, Cognition and Computation at the School of Informatics at the University of Edinburgh. He received his PhD in Language Technologies from Carnegie Mellon University (2011), his MSc in Computer Science from Tel-Aviv University (2004) and his BSc in Mathematics and Computer Science from Tel-Aviv University (2000). He was awarded a Computing Innovation Fellowship for his postdoctoral studies at Columbia University (2011-2013). His research interests are in natural language processing and machine learning, with a focus on problems in structured prediction, such as syntactic and semantic parsing.