Dive into recent scientific breakthroughs and thoroughly-explained fundamentals in natural language processing.

Symbolic for BERT models

Keeping up with the BERTs: a review of the main NLP benchmarks

April 27, 2020 

What can NLP do better than humans and where is there still room for improvement? We present the different benchmark tasks for which models are already better than humans and discuss the remaining challenges for NLP systems.

Read More
David vs. Goliath

David over Goliath: towards smaller models for cheaper, faster, and greener NLP 

March 27, 2020 

Is bigger always better? When looking at the evolution of language model size in the past few years, one may think the answer is yes. In this week’s blog post, we present a recent NLP trend favoring smaller, faster, and greener models while preserving performance.

Read More
Sound control panel with a hand moving a slider.,

“Mind your language, GPT-2”: how to control style and content in automatic text writing

March 9, 2020 

Despite the impressive fluency automatic text writing has exhibited in the past year, it is still challenging to control the content of machine-written text. In this blog post, we discuss the latest research improvements allowing us to control how machines write text.

Read More
Old typewriter on a table.

How do machines write text?

February 21, 2020 

The past year has seen a tremendous rise in the capabilities of language models when it comes to writing text, the most popular being GPT-2 by OpenAI. This post aims at empowering everyone to use these models and better understand how the text output is generated, from beam search to the latest nucleus sampling method. 

Read More