HL-CLM-1095 FLAG
On 2026-09-20, claims-adjud v5.1 (an internally developed system) produced 'partial pay' in a NY health claims transaction. Inputs included medical records, claims history. No human reviewed the output before it took effect. The outcome was adverse to the consumer; no notice to the consumer is recorded. RegBench applied 1 rule(s) in force on 2026-09-20 and recorded 1 finding(s); the most serious is 'Adverse outcome from a high-risk AI system with no human review' (medium, L6).
FLAG: 1 finding(s) under rules 2026.10+c4982313, corpus 2026-10-30-9c5dad5a. Routed to the carrier's compliance reviewer with cited findings. Coverage unsupported: No black-letter rule in the pack covers a NY health claims transaction. Nothing binding was checked, so no disposition here says anything about compliance.
Evidence (carrier asserted). Supplied by the carrier and not verified by RegBench: ai_system.governance_attested, ai_system.model_doc_ref, ai_system.testing_report_ref.
Findings (1)
Adverse outcome from a high-risk AI system with no human review
Finding. A high-risk system produced an adverse outcome that took effect without human review.
Basis. Doctrine: examiners applying the NAIC bulletin's governance and control expectations, and the AI Risk Evaluation Supplement's Exhibit B question on whether there is a human in the loop for direct-consumer-impact uses, ask first about human oversight of adverse automated outcomes; an unreviewed adverse decision is the case file most likely to be sampled.
Required action. Route to the carrier's reviewer; record the review and any override on the transaction.
Excerpt: naic-ai-bulletin-s3-1-ais-program · NAIC model guidance; binding in a state only as adopted (see adoptions register) · in force from 2023-12-04
"all Insurers authorized to do business in this state are expected to develop, implement, and maintain a written program (an 'AIS Program') for the responsible use of AI Systems that make, or support decisions related to regulated insurance practices. The AIS Program should be designed to mitigate the risk of Adverse Consumer Outcomes ... The Department also encourages the development and use of verification and testing methods to identify errors and bias in Predictive Models and AI Systems, as well as the potential for unfair discrimination in the decisions and outcomes resulting from the use of Predictive Models and AI Systems."
Point in time
- Law as of
- 2026-09-20
- Rules version
- 2026.10+c4982313
- Corpus snapshot
- 2026-10-30-9c5dad5a
- Engine
- regbench-engine/0.5.0 · deterministic-1
- Fingerprint
- 21ba960d3e066c3778002a8b (pattern grouping, not an integrity check)
- Input digest
- db48b9deed5405aae7ddf8427dc8cc3cd81c97b4a52f129eea06e9642740670a
- Evaluated
- 2026-10-31 05:42:56 UTC
- Exhibit C case file
- yes
Ledger
- Entry
- #6598 · assessment · recorded 2026-09-20T15:35:47+00:00
- Hash
- 79339c5e2c1824b8a9fbbb2c72a144f90d54ed882a86110419c8cef68d8efee2
- Previous
- 19a9054c8e8bbbe4c5a4910bb851ace4d30fed6cc9fdd4958aad145f4057f7b0
- Stored copy intact
- yes, hashes match
- Assessed input intact
- yes, revision 1 matches the input digest
Carrier review
Resolved by adjuster · outcome remediated · 16.5 hours after it opened · Resolved in the carrier's review queue (synthetic).
Recorded facts revision 1
As asserted by the carrier. References are recorded, not verified: RegBench holds no copy of the documents they name.
- AI system
- claims-adjud v5.1 · internal · high-risk class: claims
- Governance attested
- True
- Testing report
- test://harborline/claims-adjud/2026Q1
- Model documentation
- doc://harborline/claims-adjud/5.1
- Feature classes
- medical_records, claims_history
- Protected factors used
- none
- External data (ECDIS)
- False
- Consumer report / credit
- False / False
- Data sources
- none recorded
- Decision
- partial pay · score 0.489
- Human review
- none
- Adverse to consumer
- True
- Notice
- not recorded · reasons: True · specific: False · external data disclosed: False
- Appeal channel
- –
- Consumer token
- tok_9c36462677