AI Disclaimer

1. What AI Workforce is

MahCare contains a governed AI layer called AI Workforce, composed of a set of named workers with clear, narrow responsibilities — for example, drafting discharge summaries, preparing DSAR packs, triaging inbound referrals, or summarising change notes. AI Workforce is used to reduce clinician and administrative burden, not to replace clinical judgment.

Every AI Workforce worker operates under a written policy that defines: its scope, the data it may read, the actions it may take, who reviews its output, how errors are handled, and the retention of its provenance. Policies are versioned and enforceable; the Service will refuse to run a worker outside its policy.

2. What AI Workforce never does

• It does not autonomously diagnose. AI Workforce workers do not issue diagnoses as clinical decisions without a clinician in the loop.
• It does not autonomously prescribe or discontinue medication. Medication suggestions require an explicit action by an authorised prescriber.
• It does not silently write to the legal clinical record. All AI-originated content is marked as such, reviewed by an authorised user, and committed with clear attribution.
• It does not bypass consent. AI Workforce operates within the tenant’s configured consent model, retention rules and access controls.
• It does not train on Customer PHI. Foundation model providers used by MahCare are contractually prevented from using Customer PHI to train general-purpose models. In-tenant fine-tuning, where offered, is opt-in, isolated and controlled by the Customer.

3. Human-in-the-loop by default

Every clinically influential output is routed for human review before it becomes authoritative. The review interface presents:

• The model identity and version;
• The prompt and inputs (with PHI masking where appropriate);
• The sources the worker drew from, linked into the Care Graph;
• A confidence indicator with calibrated meaning;
• Actions the reviewer can take: accept, edit, reject, escalate.

The reviewer’s decision is recorded in the Evidence Ledger alongside the output, so that any later reader can see what the AI said, what the human decided, and why.

4. Provenance and auditability

Every AI Workforce output is stored with its full provenance: the worker policy in effect, the model and version, the prompt, the retrieved sources, the reviewer, the outcome and any downstream action. Provenance is written to the append-only Evidence Ledger and can be surfaced in audit exports and incident reviews. See the security overview for how the Ledger works.

5. Customer responsibility

Customers remain responsible for:

• Deciding which AI Workforce workers are enabled for their tenant and under what policy;
• Clinical governance, including any safety case or impact assessment required by local regulation (for example, DCB0129 / DCB0160 in England);
• Reviewing AI outputs before acting on them, and applying clinical judgment;
• Training authorised users to understand AI strengths, failure modes and escalation paths;
• Reporting suspected AI errors or near-misses through the incident channels described below.

6. Not a medical device on day one

MahCare is designed as a non-device clinical decision support tool. It is not currently marketed as a medical device under the UK Medical Devices Regulations 2002 or under the US Food, Drug, and Cosmetic Act as interpreted by FDA guidance on Clinical Decision Support software. We track relevant guidance, including the MHRA’s Software and AI as a Medical Device Change Programme, and will re-evaluate classification if we introduce features that cross that boundary. If and when specific features become medical devices, they will be clearly marked, separately governed and subject to the appropriate regulatory controls.

7. Model providers and data flows

MahCare uses a combination of first-party and reputable third-party foundation models. For each third-party model used:

• The provider is on the sub-processor list in the Data Processing Agreement;
• The provider is bound by data protection and confidentiality terms;
• The provider may not train on Customer PHI;
• Deployment region preferences (UK or US) are honoured wherever the provider supports regional residency;
• Prompt and response logs are retained inside the Customer’s tenant for governance, not by the model provider.

8. Opt-out and configuration

AI Workforce features are configurable per tenant and, where supported, per team, role and workflow. Customers may disable specific workers or the entire AI Workforce module at any time. Disabling AI Workforce does not affect the underlying Care Graph or Evidence Ledger.

9. Transparency metrics

ITLOX publishes evaluation and override metrics to Customer administrators, including: the number of AI outputs generated, the acceptance and override rates by worker, the time-to-review distribution, and known failure modes identified from feedback. These metrics are intended to inform clinical governance and continuous improvement.

10. AI incident reporting

If you believe that MahCare has produced an AI output that poses a clinical, ethical or legal concern, please report it immediately to ai-safety@mahcare.ai. Include the tenant, the worker and output identifiers (not PHI), the expected versus actual outcome and any immediate harm or near-miss. ITLOX will triage AI incidents within one business day and engage Customer governance as appropriate.

11. Regulatory commitments

• UK: we monitor MHRA guidance on Software and AI as a Medical Device, ICO guidance on AI and data protection, and NICE evidence standards for digital health technologies.
• US: we monitor FDA guidance on Clinical Decision Support software, HHS OCR guidance on HIPAA and AI, and applicable state AI disclosure laws.
• EU (where applicable via Customer scope): we monitor the EU AI Act’s requirements for high-risk AI systems and their relationship to MDR.

12. Contact

AI safety and governance: ai-safety@mahcare.ai
Legal: legal@mahcare.ai
General: hello@mahcare.ai