NLP News Cypher | 11.10.19

Echoes from EMNLP and GPT-2 Strikes Back

Wow, what a week it was. The EMNLP conference gave us many treats to chew on such as the growing popularity of cross-lingual learning and the continued adoption of knowledge graphs in language models.

Because of all this action, this week’s Cypher will be a bit longer than usual.

🤯 EMNLP 2019 🤯

New QA Leaderboard Attempting to Mitigate SQuAD Problems

GPT-2 Doesn’t Bring Armageddon

Chollet’s New Formulation of Intelligence

Unsupervised Cross-lingual Representation Learning

Compute Growth Goes Hyperbolic

What were some of the top keywords in EMNLP papers?

Image for post
Image for post

🤯EMNLP 2019🤯

Stephen Mayhew et al were live tweeting during the conference (thank you) and sharing all the action. Here are a few threads that caught our eyes:

1.Chris Manning discusses the GQA dataset, which takes natural language questions generated from graphs (based on the visual genome project) and delivers new leaderboard results from his neural state machine paper to be presented at NeurIPS next month. Full thread and link to GQA below:

2. Allen Institute’s Matt Gardner shares his slides on the limitations of reading comprehension task in NLP. He posits an open reading benchmark that can evaluate multiple problems in reading comprehension (e.g. Sentence-level linguistic structure, Discrete Reasoning Over Paragraphs, Question-based coreference resolution, Reasoning Over Paragraph Effects in Situations, time, grounding and others) all at once. Slides:

Allen Institute

3. If you haven’t heard of GNN’s, (Graph Neural Networks) you should get familiar. Below is the presentation Graph Neural Networks for Natural Language Processing by Shikhar Vashishth, Naganand Yadati and Partha Talukdar. Warning it’s 315 slides long. Great work!

Github: https://github.com/svjan5/GNNs-for-NLP

4. My prayers were answered when Michael Galkin summarized all the knowledge graph insights from EMNLP. I won’t even bother discussing it since he did such a great job in the column below, part 2 is dropping soon!

New QA Leaderboard Attempting to Mitigate SQuAD Problems

TechQA, a new leaderboard based on questions posted in IBM DeveloperWorks, is introduced by IBM Research for enterprise Question Answering systems. Sobering insight we already knew:

“Natural Questions was created by harvesting users’ questions of Google’s search engine and then finding answers by using turkers. When a SQuAD system is tested on the Natural Questions leaderboard the F measure drops dramatically to 6% (on short answers — it is 2% for a SQuAD v1.1 system) illustrating the brittleness of SQuAD trained systems.”

GPT-2 Doesn’t Bring Armageddon

OpenAI finally unveiled their 1.5 billion parameter transformer to the world. In addition, they also unveiled a detection model for detecting AI written text that thinks all my stuff is written by AI. 😂😂

…55 minutes later, Adam King, the creator of Talktotransformer.com put GPT-2 up. 🧐🧐

… 80 minutes later, Hugging Face put it up…🧐🧐

Me:

Image for post

Chollet’s New Formulation of Intelligence

Francis Chollet (of the Keras fame) dropped his thesis on defining and measuring intelligence and a new eval dataset called ARC (Abstraction and Reasoning Corpus). Apparently he was working on this for the past 2 years.

“ARC can be seen as a general artificial intelligence benchmark, as a program synthesis benchmark, or as a psychometric intelligence test. It is targeted at both humans and artificially intelligent systems that aim at emulating a human-like form of general fluid intelligence.”

Unsupervised Cross-lingual Representation Learning

Last week we showed a picture we took during Sebastian Ruder’s talk at NYU. In his blog post, he shares some the slides he used and more:

Compute Growth Goes Hyperbolic

Nothing to see here, move along…

Image for post
Image for post

This column is a weekly round-up of NLP news and code drops from researchers worldwide.

Follow us on Twitter for more Code & Demos: @Quantum_Stat

Image for post
Image for post
www.quantumstat.com

Written by

We Build NLP for the Bravehearts ✌ quantumstat.com

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store