Workshop on Crowdsourcing Technologies for Language and Cognition Studies

This workshop was held on Jul7 27th, 2011, in conjunction with the 2011 Linguistic Society of America Summer Institute at the University of Boulder, Colorado. The workshop brought together linguists who are utilizing crowdsourcing technologies and those who wanted to know more about them, with ‘how-to’ tutorial sessions and a workshop where researchers presented results, ideas, and experimental design and evaluation strategies.


Languages spoken by crowdsourced workers (from

Robert Munro, Stanford, and Harry Tily, MIT. The Start of the Art: An Introduction to Crowdsourcing Technologies for Language and Cognition Studies

Lukas Biewald, CrowdFlower. Keynote.

T. Florian Jaeger, Harry Tily, Michael C. Frank, Jacqueline Gutman and Andrew Watts, University of Rochester
A web‐based (iterated) language learning
paradigm with human participants

Pranav Anand, Caroline Andrews and Matt Wagers, University of California, Santa Cruz
Assessing the pragmatics of experiments with crowdsourcing: The case of scalar implicature

David Clausen and Chris Potts, Stanford University
Collecting task-oriented dialogues

Jordan Boyd-Graber, University of Maryland
Trivial Classification: What features do humans use for classification?

Alessandra Zarcone, Universität Stuttgart and Sebastian Pado, Universität Heidelberg
A crowdsourcing study of logical metonymy

Andrew Watts and T. Florian Jaeger, University of Rochester
Balancing experimental lists without sacrificing voluntary participation

Nicholas Duran and Rick Dale, University of Memphis
Creating illusory social connectivity in Amazon Mechanical Turk

Marie-Catherine de Marneffe and Chris Potts, Stanford University
A case study in effectively crowdsourcing long tasks with novel categories

Workshop Organizers

Robert Munro (Stanford Linguistics)
Harry Tily (MIT Brain and Cognitive Sciences)

Review Committee

Lukas Biewald (CrowdFlower)
Chris Callison-Burch (Johns Hopkins Computer Science & Center for Language and Speech Processing)
Mike Frank (Stanford Psychology & Language and Cognition Lab)
Florian Jaeger (University of Rochester Human Language Processing Lab)
Victor Kuperman (McMaster University Department of Linguistics & Languages)
Marie-Catherine de Marneffe (Stanford Linguistics)
Rada Mihalcea (University of North Texas)
Steven Piantadosi (MIT Brain and Cognitive Sciences)
Chris Potts (Stanford Linguistics)
Philip Resnik (University of Maryland Department of Linguistics & Institute for Advanced Computer Studies)
Neal Snider (Nuance Communications)

Organizing committee

David Clausen (Stanford Linguistics)
Mollie Allick (CrowdFlower)
Vaughn Hester (CrowdFlower)


The workshop was sponsored by CrowdFlower.


Comments are closed.

Set your Twitter account name in your settings to use the TwitterBar Section.