Bert from Sesame Street as the NLP champion, standing victorious over defeated LLM robots

Beating BERT? Small LLMs vs Fine-Tuned Encoders for Classification

I ran 32 experiments comparing small LLMs to BERT on classification tasks. Turns out 2018-era BERT is still really good at what it does.