news

한어Русский языкEnglishFrançaisIndonesianSanskrit日本語DeutschPortuguêsΕλληνικάespañolItalianoSuomalainenLatina

Machine Heart Report

Synced Editorial Department

Sound the alarm or move towards closure?

"ACL is not an AI conference." At ACL 2024 held in Bangkok, Thailand, this year's ACL President Emily M. Bender made a very controversial conclusion.

The ACL conference is a top international conference in the field of computational linguistics and natural language processing, organized by the Association for Computational Linguistics and held annually. The ACL conference has always been ranked first in academic influence in the field of NLP, and it is also a CCF-A recommended conference.

In recent years, as methods such as deep learning have become the mainstream of NLP research, more and more people regard this conference as an AI conference, and most of the submissions are related to AI. Emily M. Bender seems to have seen some drawbacks of this tendency.

In her speech, she emphasized that ACL, the Annual Meeting of the Association for Computational Linguistics, is not centered on artificial intelligence, but is more focused on language technology and computational linguistics.

Bender made it clear that she does not use AI (artificial intelligence) as a synonym for ML (machine learning). She believes that machine learning (including deep learning) provides many useful techniques for language technology and computational linguistics, but problems arise when the focus shifts to AI.

Computational linguistics and natural language processing (CL/NLP) are concerned with the similarities/differences of languages, how information is represented in languages, how to build technologies to help with transcription, translation, summarization, information acquisition, etc. in different languages, as well as how to evaluate these technologies, which intermediate representations are useful for these technologies, how effective different ML techniques are for different tasks, and how language technologies interact with existing power systems.

Problems in the field of AI include how to build thinking machines that can perform human-like reasoning, how to make these machines surpass humans in cognitive tasks, how to automate the scientific method, and how to automate creative work.

The AI ​​field also puts forward some ideas, such as that humanity’s destiny is to merge with machines to become superhuman, that the arrival of the singularity is inevitable, and that AI (effectively synthetic text machines) can replace the services we should provide to each other (education, health care, legal representation).

The AI ​​field has several problems, including intense interest from venture capital and billionaires.

Bender also criticized some bad research practices in the field of AI, such as improper use of benchmarks, requiring comparative evaluation with SOTA closed models, and lack of reserved data due to large data sets. Bender said that if your research question focuses on "how to prove that my machine is intelligent", then this focus may distort research practices.

She also noted that the focus of the AI ​​field has led to poor reviewing practices, such that papers that do not use large language models (LLMs) or do not provide SOTA-sized LLM results may be considered uninteresting.

In contrast to poor practices in AI, research best practices in CL/NLP include applicability of the technology, understanding of human language behavior, well-defined evaluations, intrinsic and extrinsic assessments, solid baselines, retained test data, and detailed error analysis.

Bender said CL/NLP research is built on an understanding of data, including knowledge of how language works (i.e., linguistics) and documentation of datasets.

In terms of replicability and reproducibility, Bender emphasized that science is about building on previous research rather than just reaching SOTA.

In terms of social impact, CL/NLP research focuses on the impact of its technology on society, including ethics and the history of NLP research, as well as understanding who, for whom, and to whom language and technology will be used, and who may be harmed by being excluded or included.

Bender believes that ACL should be a place to focus on language technology, a community that promotes interdisciplinary research, a research field that cares about the language community, and a space where we can reason about the impact of our research and technology on society.

This view has caused great controversy on social platforms.

Some people think that this is a manifestation of lack of inclusiveness. "The best moments in the history of NLP occurred when people were open to ideas from other disciplines: learning statistical methods from language researchers and learning how to look at the world from social scientists. These slides make me feel that some people want us to be closed."

Others believe that there is no need for such a division because the two have been organically integrated.

However, some people expressed their understanding. After all, AI is too popular. Once a conference is "contracted" by AI papers, research in other fields will inevitably be neglected, which will make the conference lose its original interest.

Regarding the signal conveyed by this speech, people began to speculate: Does this mean that ACL no longer welcomes AI papers?

What do you think about this?

Reference link: https://faculty.washington.edu/ebender/papers/ACL_2024_Presidential_Address.pdf