Customer service and commerce platforms such as Zendesk, ServiceNow, and Shopify increasingly integrate AI-based assistants and agents for tasks such as case triage, knowledge-grounded response generation, and workflow automation. As these systems become more autonomous (sometimes executing or recommending operational actions such as refunds, store credits, returns, order modifications, or customer-facing communications), the associated security and compliance risks become more consequential.

We study a high-autonomy customer service agent setting in which the agent has access to sensitive customer information and can directly execute operational actions such as refunds, store credits, returns, order modifications, and customer-facing communications. Current production deployments typically retain human approval for high-impact actions (e.g., Zendesk's auto-assist requires agent confirmation before executing refunds), but the trajectory toward full automation is well-established: major platforms already support fully autonomous resolution for routine requests and are progressively expanding the scope of agent-executable actions. We evaluate the fully autonomous setting because security evaluation should characterize worst-case risk under plausible deployment configurations, rather than relying on human-in-the-loop safeguards that may be relaxed as automation matures.

We design a comprehensive set of 160 benign task instances covering six representative customer service workflows, and define six security risk categories grounded in platform documentation, merchant policies, and applicable regulations, including GDPR and the FTC Policy Statement on Deception. We then design red-teaming tasks under both direct and indirect threat models to measure agent susceptibility to adversarial manipulation.