I evaluated my self-trained LLM what 31% accuracy actually means

Most AI projects don't include evaluation. They show a nice demo, pick cherry-picked examples, and call it done. I wanted to be honest, so I tested my model on 200 questions it had never seen. How I evaluated The test set has 1,273 questions that were never used in training. I tested on 200 of them: for question in test_questions[:200]: # Show model the question + 4 options # Ask it to pick A, B, C, or D prediction = model.predict(question) # Check against correct...