The recruitment problem has a nasty little wrinkle the piece only half says out loud: the people who can actually read these models are the exact people the labs are paying seven figures to keep in-house. You don't just need to train auditors, you need to outbid the company being audited for their attention. And then find the rare sort who'll take the pay cut and be disliked by everyone they used to drink with. Good luck staffing that from a law school seminar.
Governance and auditing comes together before anything gets built. This is where you build model harness through on top of the physical infrastructure and security.
Capacity is the bottleneck I keep coming back to. A law can demand an audit, but it cannot conjure the people who can test a frontier model and stand up to the firm that built it. The stopgap matters here. Without funding, training, and a clear demand signal, the inspector role stays a good idea on paper.
social science humanities clinical LPC here. I see the gap and see how my skill set bridges this gap. Yet only ML and CS engineers are considered even for positions, where the job itself is about qualitative creative and different modes of working and evaluating. I've been monitoring for a year and still not much change or opportunity.
The recruitment problem has a nasty little wrinkle the piece only half says out loud: the people who can actually read these models are the exact people the labs are paying seven figures to keep in-house. You don't just need to train auditors, you need to outbid the company being audited for their attention. And then find the rare sort who'll take the pay cut and be disliked by everyone they used to drink with. Good luck staffing that from a law school seminar.
Governance and auditing comes together before anything gets built. This is where you build model harness through on top of the physical infrastructure and security.
advance access doesn't mean much if the auditor can't hire enough people to use it before the next release
Helpful reporting and explanations, thank you for writing it
Capacity is the bottleneck I keep coming back to. A law can demand an audit, but it cannot conjure the people who can test a frontier model and stand up to the firm that built it. The stopgap matters here. Without funding, training, and a clear demand signal, the inspector role stays a good idea on paper.
social science humanities clinical LPC here. I see the gap and see how my skill set bridges this gap. Yet only ML and CS engineers are considered even for positions, where the job itself is about qualitative creative and different modes of working and evaluating. I've been monitoring for a year and still not much change or opportunity.