Sometimes… cool things happen. A new chatbot from Facebook AI was released this Friday with remarkable features. This chatbot, BlenderBot 2.0, is an improvement on their previous bot from last year. The bot has better long-term memory and can search the internet for information during conversation! This is a convenient improvement versus traditional bots since information is not statically “memorized” but instead has the option to be dynamic and “staying up to date” via the internet. 🤯
I’ve recently tested the model and trialed the smaller 400M variant. Currently, there exists two variants:
Welcome back! Hope you had a great week. We have a new leader on the SuperGLUE benchmark with a new Ernie model from Baidu comprising of 10 billion parameters trained on on a 4TB corpus. FYI, human baseline was already beat by Microsoft’s DeBERTa model at the beginning of the year… time for a new SuperSuperGLUE benchmark???
BTW, if you are still interested in GitHub’s CoPilot, I stumbled upon the Codex paper this week:
DeepMind’s Perceiver transformer allows it to take a variety of modalities (vision, audio, text) as its input and able to achieve competitive outcomes in…
Hey Welcome back! Want to wish everyone in the US a happy 4th of July🎆🎇! Also, want to quickly mention that the NLP Index has doubled in size (since its inception) with now housing over 6,000 repos, pretty cool!!! 😎 And as always, it gets updated weekly. But first, this week we asked 100 NLP developers: Name one thing Microsoft got for paying $7.5 billi for GitHub, and $1 billi to OpenAI? SURVEY SAYS:
7.5B + 1B = GitHub CoPilot 👍
If you want to hear GitHub’s take on their new code generating assistant read here:
Also… it turns…
Welcome back! EleutherAI has a brand new (and big) GPT model that was open-sourced over this past week. The model (JAX-based) was trained for 5 weeks on the Pile dataset, Eleuther’s own ~800GB data dump. The model is called GPT-J, a 6 billion parameter model that rivals the performance of GPT-3 of the same size. And apparently it performs well on code generation:
Here’s a comparison of all the major language models on various datasets:
Hey, Welcome Back! This is probably a good time to take you down the rabbit hole on the state of the Super Duper NLP Repo (SDNR)😎 .
If this is your first time hearing about the SDNR, it’s a handy repository of more than 300 Colab notebooks (and counting) focusing on natural language processing (NLP). Colab is essentially a Jupyter notebook that one can use and share via a web-based kernel. The best part of these notebooks, is that you can use a free GPU, usually a K80 or a T4 or even a TPU (if you are feeling dangerous)…
Welcome back to the simulation ✌ . So ACL 2021 data dump happened and now we have a huge list of repos to get through in the Repo Cypher this week. 😁
Also, we are updating the NLP index very soon with 100+ new repos (many of which are mentioned here) alongside 30+ new NLP notebooks like this one 👇 . If you would like to get an email alert for future newsletters and asset updates, you can sign-up here.
So let us start with incoming awesomeness. Heard of the Graph4NLP library??? If you want to…
ArXiv has out-of-this-world research if you take your time to search for it. One interesting paper dating back to 2015 explores UAP (aka unidentified areal phenomena aka 🛸’s) activity over French territory. Yep, that’s right! And according to authors, aliens are very interested in atom splitting. UFO interest in nuclear facilities has been a long held belief among UFO nerds who continue to divide across two camps: camp A) aliens want to save us from nuclear destruction. And camp B) aliens want to destroy us with our own nukes. Hopefully, ET will be fully immunized to COVID prior to nuclear…
Hey Welcome back, another week goes by and so much code/research has been released into the wild.
Oh and btw, The NLP Index is on 🔥🔥🔥 , and I want to thank all contributors!
Here’s a quick glimpse at the awesome contributions: A collection of Spanish Medical NLP datasets brought to you by Salvador Lima in Barcelona. 🙌🙌 Will update the NLP Index with these and other assets by tomorrow.
Cantemist (oncology clinical cases for cancer text mining): https://zenodo.org/record/3978041
PharmaCoNER (Pharmacological Substances, Compounds and proteins in Spanish clinical case reports): https://zenodo.org/record/4270158
I mostly know dark.fail as an onion site with a great collection of urls for parasailing tor-land (aka darknet). To be honest, I didn’t even know dark.fail had a clearnet site. And very recently, its clearnet mirror was phished for a total of 4–5 days. 👀
Apparently a threat actor presented a fake court order to dark.fail’s domain registrar. And in return, they obtained access to the dark.fail’s hosting and rerouted traffic to the bad actor’s mirrored web page. It phished the pages URLs with the intention on fooling people into thinking they were buying products on the dark markets…
As an applied machine learning engineer (aka hacker 👨💻 aka flying ninja 🐱👤), I’m consistently looking for better and faster ways to stay on top of the deep learning and software development circuit. After comparing various sources for research, code, and apps. I’ve discovered that a significant amount of awesome NLP code is not on arXiv and not all NLP research is on GitHub. To obtain a wider scope of current NLP research and code, I’ve created the NLP Index! A search-as-you-type search engine containing over 3,000 NLP repositories (updated weekly) 🔥. …