SWE-Bench tainted by answer leakage; real pass rates significantly lower

Article URL: https://arxiv.org/abs/2410.06992

Comments URL: https://news.ycombinator.com/item?id=43130732

Points: 45

# Comments: 3