Inside the company behind Claude, researchers are trying to understand systems that may have already exceeded their grasp.