Mine is less fun, but hopefully more useful https://erikwiffin.github.io/memory-reasoning-eval/
Selling to labs is more than I'm looking for. I'm aiming for a couple hundred dollars so I don't have to finance a Fable vs Sol run out of my own pocket. It would be cool to have my benchmark be one of the ones referenced in a model card!
Mine is less fun, but hopefully more useful https://erikwiffin.github.io/memory-reasoning-eval/
Selling to labs is more than I'm looking for. I'm aiming for a couple hundred dollars so I don't have to finance a Fable vs Sol run out of my own pocket. It would be cool to have my benchmark be one of the ones referenced in a model card!