SR-UW-0619 FLAG
On 2026-10-15, rx-score v3 (a third-party system from RxData Labs (fictional)) produced 'decline' in a CT life rating transaction. Inputs included rx history, credit, consumer report, application data. A consumer report was used. No human reviewed the output before it took effect. The outcome was adverse to the consumer; a notice was sent on 2026-10-16 with specific reasons. RegBench applied 7 rule(s) in force on 2026-10-15 and recorded 4 finding(s); the most serious is 'Adverse high-risk decision from a system with no adverse-outcome testing on record' (medium, L1).
FLAG: 4 finding(s) under rules 2026.10+dbe5ddf6, corpus 2026-10-30-5f07c521. Routed to the carrier's compliance reviewer with cited findings.
Findings (4)
Adverse high-risk decision from a system with no adverse-outcome testing on record
Finding. The decision was adverse to the consumer and came from a high-risk AI system for which no testing for errors, bias or unfair discrimination is referenced.
Basis. The bulletin encourages verification and testing methods to identify errors and bias in predictive models and AI systems and the potential for unfair discrimination in their outcomes, and Section 4 lists validation, testing and auditing records among what an examiner may request.
Required action. Attach the system's latest testing report reference (errors, bias, unfair discrimination, drift).
Excerpt: naic-ai-bulletin-s3-1-ais-program · NAIC model guidance; binding in a state only as adopted (see adoptions register) · in force from 2023-12-04
"all Insurers authorized to do business in this state are expected to develop, implement, and maintain a written program (an 'AIS Program') for the responsible use of AI Systems that make, or support decisions related to regulated insurance practices. The AIS Program should be designed to mitigate the risk of Adverse Consumer Outcomes ... The Department also encourages the development and use of verification and testing methods to identify errors and bias in Predictive Models and AI Systems, as well as the potential for unfair discrimination in the decisions and outcomes resulting from the use of Predictive Models and AI Systems."
Excerpt: naic-ai-bulletin-s4-oversight · NAIC model guidance; binding in a state only as adopted (see adoptions register) · in force from 2023-12-04
"in the context of an investigation or market conduct action, an Insurer can expect to be asked about its development, deployment, and use of AI Systems ... and its outcomes (including Adverse Consumer Outcomes)". Requests may cover the written AIS Program, "inventories and descriptions of Predictive Models, and AI Systems used ... to make or support decisions that can result in Adverse Consumer Outcomes", documentation for any specific model including data source, provenance, lineage and bias analysis, and "validation, testing, and auditing, including evaluation of Model Drift"; and for third-party systems, "2.1 Due diligence conducted on third parties ... 2.2 Contracts with third-party AI System, model, or data vendors ... 2.3 Audits and/or confirmation processes performed regarding third-party compliance".
Third-party AI system without an attested due-diligence and contract record
Finding. A third-party AI system contributed to a high-risk decision, and the carrier has not attested that the system is covered by its AIS Program's third-party due-diligence standards and contract terms.
Basis. The bulletin, as adopted by this state, expects the AIS Program to address due diligence on third-party AI systems and data and contract terms providing audit rights and cooperation with regulatory inquiries; the carrier remains responsible for the outcomes.
Required action. Attach the vendor due-diligence record and contract terms (audit rights, regulatory cooperation) to the system's inventory entry and attest governance coverage.
Excerpt: naic-ai-bulletin-s3-4-third-party · NAIC model guidance; binding in a state only as adopted (see adoptions register) · in force from 2023-12-04
"Each AIS Program should address the Insurer's process for acquiring, using, or relying on (i) third-party data to develop AI Systems; and (ii) AI Systems developed by a third party, which may include, as appropriate, the establishment of standards, policies, procedures, and protocols relating to ... 4.1 Due diligence and the methods employed by the Insurer to assess the third party and its data or AI Systems ... to ensure that decisions made or supported from such AI Systems that could lead to Adverse Consumer Outcomes will meet the legal standards imposed on the Insurer itself. 4.2 Where appropriate and available, the inclusion of terms in contracts with third parties that: a) Provide audit rights and/or entitle the Insurer to receive audit reports ... b) Require the third party to cooperate with the Insurer with regard to regulatory inquiries and investigations ..."
Excerpt: naic-ai-bulletin-s4-oversight · NAIC model guidance; binding in a state only as adopted (see adoptions register) · in force from 2023-12-04
"in the context of an investigation or market conduct action, an Insurer can expect to be asked about its development, deployment, and use of AI Systems ... and its outcomes (including Adverse Consumer Outcomes)". Requests may cover the written AIS Program, "inventories and descriptions of Predictive Models, and AI Systems used ... to make or support decisions that can result in Adverse Consumer Outcomes", documentation for any specific model including data source, provenance, lineage and bias analysis, and "validation, testing, and auditing, including evaluation of Model Drift"; and for third-party systems, "2.1 Due diligence conducted on third parties ... 2.2 Contracts with third-party AI System, model, or data vendors ... 2.3 Audits and/or confirmation processes performed regarding third-party compliance".
Adverse outcome from a high-risk AI system with no human review
Finding. A high-risk system produced an adverse outcome that took effect without human review.
Basis. Doctrine: examiners applying the NAIC bulletin's governance and control expectations, and the AI Risk Evaluation Supplement's Exhibit B question on whether there is a human in the loop for direct-consumer-impact uses, ask first about human oversight of adverse automated outcomes; an unreviewed adverse decision is the case file most likely to be sampled.
Required action. Route to the carrier's reviewer; record the review and any override on the transaction.
Excerpt: naic-ai-bulletin-s3-1-ais-program · NAIC model guidance; binding in a state only as adopted (see adoptions register) · in force from 2023-12-04
"all Insurers authorized to do business in this state are expected to develop, implement, and maintain a written program (an 'AIS Program') for the responsible use of AI Systems that make, or support decisions related to regulated insurance practices. The AIS Program should be designed to mitigate the risk of Adverse Consumer Outcomes ... The Department also encourages the development and use of verification and testing methods to identify errors and bias in Predictive Models and AI Systems, as well as the potential for unfair discrimination in the decisions and outcomes resulting from the use of Predictive Models and AI Systems."
High-risk AI system with no model documentation reference
Finding. The AI system that produced this decision is in a high-risk class but the transaction carries no reference to model documentation.
Basis. Section 4 of the bulletin tells insurers to expect requests for inventories and descriptions of the predictive models and AI systems used to make or support decisions that can result in adverse consumer outcomes, together with validation, testing and auditing documentation.
Required action. Link the system to its model documentation (purpose, inputs, validation, monitoring) in the inventory.
Excerpt: naic-ai-bulletin-s4-oversight · NAIC model guidance; binding in a state only as adopted (see adoptions register) · in force from 2023-12-04
"in the context of an investigation or market conduct action, an Insurer can expect to be asked about its development, deployment, and use of AI Systems ... and its outcomes (including Adverse Consumer Outcomes)". Requests may cover the written AIS Program, "inventories and descriptions of Predictive Models, and AI Systems used ... to make or support decisions that can result in Adverse Consumer Outcomes", documentation for any specific model including data source, provenance, lineage and bias analysis, and "validation, testing, and auditing, including evaluation of Model Drift"; and for third-party systems, "2.1 Due diligence conducted on third parties ... 2.2 Contracts with third-party AI System, model, or data vendors ... 2.3 Audits and/or confirmation processes performed regarding third-party compliance".
Excerpt: naic-ai-bulletin-s3-1-ais-program · NAIC model guidance; binding in a state only as adopted (see adoptions register) · in force from 2023-12-04
"all Insurers authorized to do business in this state are expected to develop, implement, and maintain a written program (an 'AIS Program') for the responsible use of AI Systems that make, or support decisions related to regulated insurance practices. The AIS Program should be designed to mitigate the risk of Adverse Consumer Outcomes ... The Department also encourages the development and use of verification and testing methods to identify errors and bias in Predictive Models and AI Systems, as well as the potential for unfair discrimination in the decisions and outcomes resulting from the use of Predictive Models and AI Systems."
Point in time
- Law as of
- 2026-10-15
- Rules version
- 2026.10+dbe5ddf6
- Corpus snapshot
- 2026-10-30-5f07c521
- Engine
- regbench-engine/0.4.0 · deterministic-1
- Fingerprint
- d76c272f49069ce338d10014
- Evaluated
- 2026-10-30 11:23:24 UTC
- Exhibit C case file
- yes
Ledger
- Entry
- #619 · assessment · recorded 2026-10-15T18:05:50+00:00
- Hash
- fae3609788947139afe1cc9ef17e5be1f29cdcbc4d2e09dd0ce72d213017cc45
- Previous
- c3c0ee50043dc7a28063db5d21a8992704a9bb645dd0adfd8e53c7174e4952c8
- Stored copy intact
- yes, hashes match
Carrier review
Resolved by compliance · outcome escalated · 32.3 hours after it opened · Resolved in the carrier's review queue (synthetic).
Recorded facts
- AI system
- rx-score v3 · third party (RxData Labs (fictional)) · high-risk class: underwriting
- Governance attested
- False
- Testing report
- none recorded
- Model documentation
- none recorded
- Feature classes
- rx_history, credit, consumer_report, application_data
- Protected factors used
- none
- External data (ECDIS)
- False
- Consumer report / credit
- True / True
- Data sources
- vendor://rxdata/score, cra://demo-bureau/attributes
- Decision
- decline · score 0.934 · codes RX4, CR2
- Human review
- none
- Adverse to consumer
- True
- Notice
- sent 2026-10-16 · reasons: True · specific: True · external data disclosed: True
- Appeal channel
- –
- Consumer token
- tok_a539569742