1 Comment

I would add the pertinent question AT WHAT COST can RLHF address some of the most pressing challenges of LLMs (considering the explotation of ghost workers at minimal wages)?

And if non-transparent, gigantic datasets are one issue (e.g. as illustrated by ChatGPT), what value lies in creating and improving Little Language Models, as possibly the better LLMs based on highly contextual, curated, high-quality datasets?

Credit for this last point goes to Lelapa.AI and a conversation I had with Pelonomi Moiloa.

Expand full comment