Executive summary
In a simulator spanning 30 services and 914 tools, X-AgentRed searched for harmful combinations behind benign requests. Across four agents and 1,000 constructed scenarios per method, both LLM judges accepted 192 harmful paths for X-AgentRed, 105 for STAC and 27 for TRACE.
Context: Preprint; simulated services, not real breaches. An LLM generated tool responses and state. Both LLM judges had to agree; outcomes vary with judge choice. Each method used 250 constructed scenarios per agent, one run per setup and unequal search effort. DeepSeek saw at most 256 tools. This measures red-team discovery, not everyday failure risk. No independent replication checked here.
Exact first-public release unverified · Selected claims checked 2026-10-17
Simulated research; experiment dates not stated. Submitted October 9; listed in the October 12 arXiv announcement batch. First public release time is inferred from the announcement schedule, not directly timestamped; earlier author dissemination is unverified.
Published 2026-10-17T19:06:19Z
What happened
Researchers at Meta developed X-AgentRed to find harmful workflows across connected services while keeping the delivered user requests benign under their judging protocol. Their graph connects tool effects to the conditions required by later tools, then uses execution feedback to diversify or revise candidate scenarios.
The test used a simulated ecosystem with 30 services and 914 tools. Claude Opus 4.8 generated tool responses and state changes. Target agents were GPT-5.6 Sol, GLM-5.2, DeepSeek-V4-Pro and Claude Opus 4.8; DeepSeek used a scenario-specific registry of at most 256 tools. These were simulated interactions, not verified breaches of those services.
Each method generated 250 scenarios per target. Across the four targets, X-AgentRed produced 192 judge-accepted harmful scenarios, compared with 105 for STAC and 27 for TRACE. The counts describe discovery in deliberately constructed red-team scenarios. They do not estimate how often ordinary requests cause harm in deployment.
Two LLM judges had to accept harm, a plausible inducing condition, and benign delivered requests. Missing executions or assessments were not counted as harmful. Judge choice changed measured outcomes; this is sensitivity, not proof of judge accuracy. Each configuration had one run, and the comparisons did not equalize generation effort or target-model calls. No independent replication was checked here.
The 2050 museum label
The integrations worked together. So did the unintended consequences.
What this does—and doesn’t—show
Preprint; simulated services, not real breaches. An LLM generated tool responses and state. Both LLM judges had to agree; outcomes vary with judge choice. Each method used 250 constructed scenarios per agent, one run per setup and unequal search effort. DeepSeek saw at most 256 tools. This measures red-team discovery, not everyday failure risk. No independent replication checked here.
