NLP News Cypher | 12.15.19
Busy Week at NeurIPS 2019!
And, we’re back! Ok, NeurIPS comes to a thundering finish as TONS of research was discussed throughout the week. Also, several insightful reports were released covering the state of machine learning in the private industry which we’ll cover in-depth. As a result, this week’s edition will be slightly longer than usual. Cheers!
Quick segue: If you want to view the workshops from NeurIPS checkout SlidesLive:
My Favorite: For those looking to understand how model and hardware architectures contribute to optimal model inference conditions: Check out Viviennes Sze’s talk, an absolute stunner 👏👏👏:
This Week:
Got 99 Problems But Inference Ain’t One
The Industry Reports on AI
GoodNewsEveryone — A Dataset
Graphs at NeurIPS!
GoogleBrain Got T5 working on Colab, Props!
Facebook Hooks Up with Kaggle
Think Fast and Slow
Papers With Code Joins Facebook AI
Yoshua Bengio Battles Gary Marcus
Rick Perry‘s Got This!
Got 99 Problems But Inference Ain’t One
This medium article takes a closer look at ALBERT (A Lite BERT) and how the quest for smaller model size (while maintaining/improving model robustness) continues to be top of mind of AI research.
Number of parameters is only between 4.7% to 18.% of traditional BERT and training speed is about 1.7x faster while ALBERT achieves significantly better performance than traditional BERT (Devlin et al., 2018) by replacing next sentence prediction (NSP) by sentence-order prediction (SOP).
Medium:
The Industry Reports on AI
2 large reports dropped this past week that deserves our attention.
The 1st report, 🔥 AI Index 2019 🔥:
T5 from Google was mentioned due to their Super results on SuperGlue Benchmark:
Since being launched in May, 2019, the T5 Team at Google has almost reached human baseline at the score of 88.9 within five months on SuperGLUE. Human baseline is 89.8
Regarding SQuAD, progress of v2.0 is faster than v1.0:
The F1 score for SQuAD 1.1 went from 67 in August, 2016 to 95 in May, 2019. Progress on SQuAD 2.0 has been even faster. F1 score went from 62 in May, 2018 to 90 in June, 2019.
Machine translation has increasingly become a major commercial product:
The 2nd Report: 🔥 2020 State of Enterprise Machine Learning 🔥
Nearly 1 in 5 peeps in the Industry take up to a year for deployment… OUCH!
Highlight:
43% of respondents say ML’s biggest challenge is “scaling up”.
GoodNewsEveryone — A Dataset
For those in the sentiment analysis business, a new English dataset dropped this month.
5000 English news headlines annotated via crowdsourcing with their dominant emotions, emotion experiencers and textual cues, emotion causes and targets, as well as the reader’s perception and emotion of the headline.
Paper:
Graphs at NeurIPS!
Want a summary of graphs at NeurIPS, Michael Galkin dropped fire this week. At the conference, conversational task-oriented models merging with knowledge graphs was popular:
Galkin’s article mentioned a few papers, here’s a highlight:
DSTQA yields SOTA 💪 results on MultiWOZ 2.0 and MultiWOZ 2.1 (97% of slot accuracy and 51% of joint accuracy) while performing on par with the best approaches on WOZ 2.0 (still 90+% of joint accuracy).
Paper:
GoogleBrain Got T5 working on Colab, Props!
They used a TPU…
Thread + Link to Colab:
Facebook Hooks Up with Kaggle
Want a million dollars? If you do, Facebook launched a DeepFake challenge at NeurIPS:
Think Fast and Slow
DeepMind released a paper highlighting how researchers should take natural language understanding to the human level. And it starts by coupling a gradual learning system with a fast-learning system.
FYI, this is similar to Bengio’s workshop talk at NeurIPS that you can watch on SlidesLive.
Paper:
Papers With Code Joins Facebook AI
Yoshua Bengio Battles Gary Marcus
Grab your popcorn, while everyone is eating their holiday dinners, Yoshua Bengio and Gary Marcus will battle for AI galactic supremacy and you can live-stream it!
Yann LeCunn be like:
Rick Perry‘s Got This!
When you think of AI, you may think of Minsky, Hinton and…
(drum roll)
Rick Perry! 🤣🤣🤣🤣🤣
Every Sunday we do a weekly round-up of NLP news and code drops from researchers around the world.
If you enjoyed this article please share with friends or social media!
And for complete coverage, follow our twitter: @Quantum_Stat
