Results

You can find a detailed report in our paper or peruse the results yourself here.

This page allows you to do cross-sectional analysis on each pair of questions to understand how various demographics and opinions relate to each other among survey respondents. To preserve people’s privacy, answers with less than 10 respondents for gender and location are combined into larger buckets.

You can link directly to the current visualization using the URL of this page, which will update as you explore the data.

Overview

View responses alongside demographics. When viewing a single demographic variable and a single survey section, click on a bar to filter responses only to that demographic group. Shift-click to add or remove from your selection.

Demographics: Career position Job sector Publication-oriented job Attended a recent ACL event Number of recent NLP publications Number of NLP publications Location Gender Underrepresented minority Twitter use for NLP Heard about the survey Meta-question confidence Donation All None

Section: State of the Field Scale, Inductive Bias, and Adjacent Fields AGI and Major Risks Language Understanding Promising Research Programs Ethics All None

Analyze

Explore the relationship between two survey questions.

Group by: =============== State of the Field =============== Private firms have too much influence Industry will produce the most widely-cited research NLP winter is coming (10 years) NLP winter is coming (30 years) Most of NLP is dubious science Author anonymity is worth restricting preprints META: Private firms have too much influence META: Industry will produce the most widely-cited research META: NLP winter is coming (10 years) META: NLP winter is coming (30 years) META: Most of NLP is dubious science META: Author anonymity is worth restricting preprints === Scale, Inductive Bias, and Adjacent Fields === Scaling solves practically any important problem Linguistic structure is necessary Expert inductive biases are necessary Ling/CogSci will contribute to the most-cited models META: Scaling solves practically any important problem META: Linguistic structure is necessary META: Expert inductive biases are necessary META: Ling/CogSci will contribute to the most-cited models ============== AGI and Major Risks =============== AGI is an important concern Recent progress is moving us towards AGI AI could soon lead to revolutionary societal change AI decisions could cause nuclear-level catastrophe META: AGI is an important concern META: Recent progress is moving us towards AGI META: AI could soon lead to revolutionary societal change META: AI decisions could cause nuclear-level catastrophe ============= Language Understanding ============= LMs understand language Multimodal models understand language Text-only evaluation can measure language understanding META: LMs understand language META: Multimodal models understand language META: Text-only evaluation can measure language understanding ========== Promising Research Programs =========== There's too much focus on scale There's too much focus on benchmarks On the wrong track: model architectures On the wrong track: language generation On the wrong track: explainable models On the wrong track: black-box interpretability We should do more to incorporate interdisciplinary insights META: There's too much focus on scale META: There's too much focus on benchmarks META: On the wrong track: model architectures META: On the wrong track: language generation META: On the wrong track: explainable models META: On the wrong track: black-box interpretability META: We should do more to incorporate interdisciplinary insights ======= Likely Sources of Future Advances ======== Most influential near-future advances =============== Ethics =========================== NLP's past net impact is good NLP's future net impact is good It is unethical to build easily-misusable systems Ethical and scientific considerations can conflict Ethical concerns mostly reduce to data quality and model accuracy It is unethical to predict psychological characteristics Carbon footprint is a major concern NLP should be regulated META: NLP's past net impact is good META: NLP's future net impact is good META: It is unethical to build easily-misusable systems META: Ethical and scientific considerations can conflict META: Ethical concerns mostly reduce to data quality and model accuracy META: It is unethical to predict psychological characteristics META: Carbon footprint is a major concern META: NLP should be regulated =============== Demographics ===================== Career position Job sector Publication-oriented job Attended a recent ACL event Number of recent NLP publications Number of NLP publications Gender Underrepresented minority Twitter use for NLP Heard about the survey =============== Conclusion ======================= Meta-question confidence Donation ============== Demographics (extra) ============== Region Year of first NLP publication =============== Subfield ========================= Subfield = Computational Social Science and Cultural Analytics Subfield = Dialogue and Interactive Systems Subfield = Discourse and Pragmatics Subfield = Ethics and NLP Subfield = Generation Subfield = Information Extraction Subfield = Information Retrieval and Text Mining Subfield = Interpretability and Analysis of Models for NLP Subfield = Language Grounding to Vision, Robotics and Beyond Subfield = Linguistic Theories, Cognitive Modeling, and Psycholinguistics Subfield = Machine Learning for NLP Subfield = Machine Translation and Multilinguality Subfield = NLP Applications Subfield = Phonology, Morphology, and Word Segmentation Subfield = Question Answering Subfield = Resources and Evaluation Subfield = Semantics: Lexical Subfield = Semantics: Sentence-level Semantics, Textual Inference, and Other Areas Subfield = Sentiment Analysis, Stylistic Analysis, and Argument Mining Subfield = Speech and Multimodality Subfield = Summarization Subfield = Syntax: Tagging, Chunking and Parsing Subfield = None of the above Subfield = Prefer not to say

Measure: =============== State of the Field =============== Private firms have too much influence Industry will produce the most widely-cited research NLP winter is coming (10 years) NLP winter is coming (30 years) Most of NLP is dubious science Author anonymity is worth restricting preprints META: Private firms have too much influence META: Industry will produce the most widely-cited research META: NLP winter is coming (10 years) META: NLP winter is coming (30 years) META: Most of NLP is dubious science META: Author anonymity is worth restricting preprints === Scale, Inductive Bias, and Adjacent Fields === Scaling solves practically any important problem Linguistic structure is necessary Expert inductive biases are necessary Ling/CogSci will contribute to the most-cited models META: Scaling solves practically any important problem META: Linguistic structure is necessary META: Expert inductive biases are necessary META: Ling/CogSci will contribute to the most-cited models ============== AGI and Major Risks =============== AGI is an important concern Recent progress is moving us towards AGI AI could soon lead to revolutionary societal change AI decisions could cause nuclear-level catastrophe META: AGI is an important concern META: Recent progress is moving us towards AGI META: AI could soon lead to revolutionary societal change META: AI decisions could cause nuclear-level catastrophe ============= Language Understanding ============= LMs understand language Multimodal models understand language Text-only evaluation can measure language understanding META: LMs understand language META: Multimodal models understand language META: Text-only evaluation can measure language understanding ========== Promising Research Programs =========== There's too much focus on scale There's too much focus on benchmarks On the wrong track: model architectures On the wrong track: language generation On the wrong track: explainable models On the wrong track: black-box interpretability We should do more to incorporate interdisciplinary insights META: There's too much focus on scale META: There's too much focus on benchmarks META: On the wrong track: model architectures META: On the wrong track: language generation META: On the wrong track: explainable models META: On the wrong track: black-box interpretability META: We should do more to incorporate interdisciplinary insights ======= Likely Sources of Future Advances ======== Most influential near-future advances =============== Ethics =========================== NLP's past net impact is good NLP's future net impact is good It is unethical to build easily-misusable systems Ethical and scientific considerations can conflict Ethical concerns mostly reduce to data quality and model accuracy It is unethical to predict psychological characteristics Carbon footprint is a major concern NLP should be regulated META: NLP's past net impact is good META: NLP's future net impact is good META: It is unethical to build easily-misusable systems META: Ethical and scientific considerations can conflict META: Ethical concerns mostly reduce to data quality and model accuracy META: It is unethical to predict psychological characteristics META: Carbon footprint is a major concern META: NLP should be regulated =============== Demographics ===================== Career position Job sector Publication-oriented job Attended a recent ACL event Number of recent NLP publications Number of NLP publications Gender Underrepresented minority Twitter use for NLP Heard about the survey =============== Conclusion ======================= Meta-question confidence Donation ============== Demographics (extra) ============== Region Year of first NLP publication =============== Subfield ========================= Subfield = Computational Social Science and Cultural Analytics Subfield = Dialogue and Interactive Systems Subfield = Discourse and Pragmatics Subfield = Ethics and NLP Subfield = Generation Subfield = Information Extraction Subfield = Information Retrieval and Text Mining Subfield = Interpretability and Analysis of Models for NLP Subfield = Language Grounding to Vision, Robotics and Beyond Subfield = Linguistic Theories, Cognitive Modeling, and Psycholinguistics Subfield = Machine Learning for NLP Subfield = Machine Translation and Multilinguality Subfield = NLP Applications Subfield = Phonology, Morphology, and Word Segmentation Subfield = Question Answering Subfield = Resources and Evaluation Subfield = Semantics: Lexical Subfield = Semantics: Sentence-level Semantics, Textual Inference, and Other Areas Subfield = Sentiment Analysis, Stylistic Analysis, and Argument Mining Subfield = Speech and Multimodality Subfield = Summarization Subfield = Syntax: Tagging, Chunking and Parsing Subfield = None of the above Subfield = Prefer not to say

Summary

Click on a bar to filter the summary to a subset of the data. Shift-click to add or remove from your selection, and click outside of a bar or double-click anywhere on the graph to reset.

Joint Distribution

Shown is the joint distribution of answers to the selected questions. Circles are sized according to answer frequency and colored according to normalized PMI, where positive values indicate that answers cooccur more frequently than they would by chance.

Comparisons

This shows how the "Measure" outcomes differ between "Group by" groups. On each measured outcome, we show a 95% confidence interval around the mean for each group.