Wait, you guys run evals?

Comes in my mind a meme with this expression but clearly cannot find the image related to. my question folks of this community is: whenever u build a system or a product or anything that contains a model in the backend that takes actions and is in charge of decisions that require rigor, u search few good peer reviewed benchmarks run the hardest tasks to grant ur self a bon bon of antisycophancy and see where u stand above or below. great, but still some metrics are not built for ur exact use ca...