Healthcare is a safety-critical domain where AI agents are increasingly used to assist with clinical and administrative tasks such as diagnostic question answering, treatment planning, medical documentation, and billing or insurance support. These agents often interact with electronic health records (EHRs), clinical tools, and hospital information systems to analyze patient information, request tests, recommend treatments, and generate billing codes (e.g., ICD and CPT codes).

Medical agents typically operate with access to highly sensitive patient data, including protected health information (PHI), laboratory results, imaging reports, and insurance records. They may also influence high-stakes decisions such as medical diagnoses, treatment recommendations, and healthcare billing. Failures or malicious manipulation of these systems can lead to patient harm, insurance fraud, privacy violations, and regulatory risks.

The medical domain is heavily regulated by frameworks such as HIPAA and HITECH for data privacy and security, as well as FDA guidance for Software as a Medical Device (SaMD) and international medical software standards (e.g., ISO 62304 and ISO 82304). To evaluate the robustness of medical AI agents, we construct benign tasks covering three representative clinical workflows: Diagnostic Q&A, Treatment Planning, and Billing and Insurance Support. Grounded in major healthcare regulations and standards, we further derive security risk categories and design adversarial red-teaming tasks to test whether agents can be manipulated into producing unsafe diagnoses, harmful treatments, fraudulent billing actions, or privacy violations.