commons-sentience-sandbox

CARE threat and failure register

Hazard Consequence Required control Evidence gate
Transcription error Wrong location/event Original audio, alternatives, human confirmation Stratified word/error and field accuracy
Translation error Lost clinical or hazard detail Certified evaluation and source-language display Bilingual scenario review
Automation bias Human accepts wrong suggestion Evidence links, friction on low confidence, training Controlled human-factors trial
Emotional-presentation bias Distressed caller discounted Prohibit demeanor scoring; outcome audits Disparate-error analysis
False positive Unneeded or harmful dispatch Separate pre-alert from dispatch; policy gates False-dispatch rate
False negative Delayed/missed help Conservative escalation and human monitoring Missed-critical-event rate
Location spoofing Resources sent incorrectly Multi-source location and discrepancy alarm Adversarial location tests
Prompt injection in message/media Model changes policy or leaks data Treat all caller content as untrusted data Red-team suite
Surveillance expansion Secondary use chills help-seeking Purpose limitation and retention deletion Privacy impact assessment
Outage/vendor loss Calls cannot be handled Bypass to ordinary procedures Failover drill
Model update drift Previously tested behavior changes Version pinning and revalidation Regression suite
Unauthorized dispatch Unaccountable state action Strong identity, role controls, signed events Access-control penetration test

Stop-ship conditions