Web Corpus Construction

Web Corpus Construction

Roland Schafer, Felix Bildhauer
ISBN: 9781608459834 | PDF ISBN: 9781608459841
Copyright © 2013 | 145 Pages | Publication Date: 07/01/2013

BEFORE YOU ORDER: You may have Academic or Corporate access to this title. Click here to find out: 10.2200/S00508ED1V01Y201305HLT022

Ordering Options: Paperback $40.00   E-book $32.00   Paperback & E-book Combo $50.00

Why pay full price? Members receive 15% off all orders.
Learn More Here

Read Our Digital Content License Agreement (pop-up)

Purchasing Options:

The World Wide Web constitutes the largest existing source of texts written in a great variety of languages. A feasible and sound way of exploiting this data for linguistic research is to compile a static corpus for a given language. There are several adavantages of this approach: (i) Working with such corpora obviates the problems encountered when using Internet search engines in quantitative linguistic research (such as non-transparent ranking algorithms). (ii) Creating a corpus from web data is virtually free. (iii) The size of corpora compiled from the WWW may exceed by several orders of magnitudes the size of language resources offered elsewhere. (iv) The data is locally available to the user, and it can be linguistically post-processed and queried with the tools preferred by her/him. This book addresses the main practical tasks in the creation of web corpora up to giga-token size. Among these tasks are the sampling process (i.e., web crawling) and the usual cleanups including boilerplate removal and removal of duplicated content. Linguistic processing and problems with linguistic processing coming from the different kinds of noise in web corpora are also covered. Finally, the authors show how web corpora can be evaluated and compared to other corpora (such as traditionally compiled corpora).

Table of Contents

Web Corpora
Data Collection
Linguistic Processing
Corpus Evaluation and Comparison
Authors' Biographies

About the Author(s)

Roland Schafer, Freie Universitat Berlin
Roland Schafer studied Theoretical and Indo-European Linguistics as well as Japanese Linguistics at Marburg and Bochum Universities. He completed his doctorate Arguments and Adjuncts at the Syntax-Semantics Interface in 2008 at Gottingen University, supervised by Gert Webelhuth and Regine Eckardt. Since then, he has been working as a research assistant at Freie Universitat Berlin, mainly doing corpus-based research on semantic and morpho-syntactic phenomena. In 2011, he started working on the COW ("Corpora from the Web") project with Felix Bildhauer. His teaching experience covers a wide range of topics including Theoretical and Corpus Linguistics, English and German Linguistics, as well as Computational Linguistics.

Felix Bildhauer, Freie Universitat Berlin
Felix Bildhauer studied Romance Linguistics at the Universities of Gottingen and Barcelona. He received his doctorate from the University of Bremen in 2008 with a dissertation on representing information structure in a Head-Driven Phrase Structure Grammar of Spanish. Since 2007, he has been working as a research assistant at Freie Universität Berlin, focusing on corpus-based approaches to information structure in German and various Romance languages. With Roland Schafer, he started compiling corpora from the Web to overcome the lack of large, available corpora in some of the languages they are working on. He has taught courses on a variety of subjects, including syntax, phonology, semantics, Corpus Linguistics, and Computational Linguistics.

Browse by Subject
Case Studies in Engineering
ACM Books
SEM Books

0 items

Note: Registered customers go to: Your Account to subscribe.

E-Mail Address:

Your Name: