6 Comments
User's avatar
NIA's avatar

The recruitment problem has a nasty little wrinkle the piece only half says out loud: the people who can actually read these models are the exact people the labs are paying seven figures to keep in-house. You don't just need to train auditors, you need to outbid the company being audited for their attention. And then find the rare sort who'll take the pay cut and be disliked by everyone they used to drink with. Good luck staffing that from a law school seminar.

Oscar Farrera's avatar

Governance and auditing comes together before anything gets built. This is where you build model harness through on top of the physical infrastructure and security.

Yupanqui's avatar

advance access doesn't mean much if the auditor can't hire enough people to use it before the next release

Steven Adler's avatar

Helpful reporting and explanations, thank you for writing it

AI Delivery Brief's avatar

Capacity is the bottleneck I keep coming back to. A law can demand an audit, but it cannot conjure the people who can test a frontier model and stand up to the firm that built it. The stopgap matters here. Without funding, training, and a clear demand signal, the inspector role stays a good idea on paper.

Lorzs's avatar

social science humanities clinical LPC here. I see the gap and see how my skill set bridges this gap. Yet only ML and CS engineers are considered even for positions, where the job itself is about qualitative creative and different modes of working and evaluating. I've been monitoring for a year and still not much change or opportunity.