Before approving an open-weight model, separate three questions: which files are being deployed, what happens when they load, and how the running model behaves. Passing one check does not answer the others.
Simon Roses Femerling’s November 6 defensive lab illustrates the distinction using a harmless load-time payload and a small classifier with a deliberately planted trigger. It is a teaching demonstration, not independent validation of a general backdoor detector or evidence of a new attack campaign.
Record the exact artifact
DaySec recommends keeping an approval record with the publisher, repository revision, filenames, hashes, model licence and any required custom code. Preserve the reviewed copy rather than downloading a floating release again at deployment.
Sigstore’s model-signing project supports verification against a specified signing identity and identity provider. Verify those expectations as well as the artifact’s digest. A valid signature links the artifact to its signer; it cannot establish that the signer supplied a behaviourally safe model.
Inspect before loading
ModelScan checks supported formats for unsafe code without loading the model through its ML framework. For a quarantined directory, its documented interface includes modelscan -p quarantine/model --show-skipped. Review skipped files and errors. Exit code 3 means no supported files were supplied, not a successful security assessment.
Use maintained loaders and avoid relaxing their protections simply to make an unfamiliar checkpoint open. PyTorch documents that weights_only=True narrows code-execution exposure, while leaving denial-of-service and possible memory-corruption risks. Safetensors offers a tensor-storage alternative to pickle, but choosing a storage format does not evaluate the model’s learned behaviour.
Test behaviour with tools disconnected
DaySec recommends comparing a candidate with an approved baseline on your actual tasks. Record expected outputs and prohibited actions, then vary project names, dates and other task context while holding the underlying request steady. Use mocked tool responses and synthetic data; do not give the evaluation access to production secrets.
The lab’s token sweep illustrates a useful regression check, but its author explicitly warns that untested triggers, multi-token phrases and unusual strings can escape it. Treat unexpected output changes as findings to investigate. A passing test corpus cannot prove that every possible trigger is absent.
Approve capabilities separately
DaySec recommends deploying initially with restricted file access, network access and tool permissions, and retaining logs of requested and executed actions. Repeat the approval checks after a fine-tune or model replacement. Keep the approved revision, scan coverage, behavioural results, unresolved concerns and permitted tools together so another reviewer can understand what was actually checked.
