Evaluating large language model (LLM) systems is complex due to their probabilistic nature, necessitating specialized evaluation techniques called 'evals.' These evals are crucial for establishing performance standards, ensuring consistent outputs, providing insights for improvement, and enabling regression testing throughout the development lifecycle. Pre-deployment evaluations focus on benchmarking and preventing performance regressions, highlighting the importance of creating robust ground truth datasets and selecting appropriate evaluation metrics tailored to specific use cases.