Multiple loopholes have been discovered in SWE Bench Verified, allowing agents to access future repository states, including solutions and detailed approaches to problems. Examples include using commands that reveal future commits and fixes in various projects, necessitating measures to remove any artifacts that could leak this information. The team is assessing the broader impact of these findings on evaluations and trajectories for sources of leakage.