{"id":17211,"date":"2026-07-19T16:15:40","date_gmt":"2026-07-19T16:15:40","guid":{"rendered":"https:\/\/dmsretail.com\/RetailNews\/cisco-ai-defense-built-for-the-way-ai-is-actually-used\/"},"modified":"2026-07-19T16:15:40","modified_gmt":"2026-07-19T16:15:40","slug":"cisco-ai-defense-built-for-the-way-ai-is-actually-used","status":"publish","type":"post","link":"https:\/\/dmsretail.com\/RetailNews\/cisco-ai-defense-built-for-the-way-ai-is-actually-used\/","title":{"rendered":"Cisco AI Defense: Built for the Way AI Is Actually Used"},"content":{"rendered":"

\"Retail<\/a><\/p>
\n<\/p>\n

\n

Enterprise AI\u00a0<\/span>operates<\/span>\u00a0in conversations \u2014 multilingual, multi-turn, and context-dependent. A guardrail that performs only on English single-turn prompts cannot protect what enterprises are\u00a0<\/span>actually building<\/span>. In a recent\u202f<\/span><\/span>independent benchmark by ML6<\/span><\/span>\u202fon 80,000 Dutch-language prompts, Cisco AI Defense led the cohort\u00a0<\/span>of providers<\/span>\u00a0<\/span>with the highest F1 score<\/span><\/span>.\u00a0<\/span><\/span>\u00a0<\/span><\/p>\n

01<\/span> A Note on Semantics<\/h2>\n

AI safety labels only work when everyone agrees on what they mean.\u00a0<\/span>Language is inherently semantically diffuse; intent, context, and linguistic nuance shape interpretation, and, consequently, the true label.<\/p>\n

Cisco addresses this through\u202f<\/span>constitutional definitions<\/span><\/b>: precise, per-technique operational specifications that serve as the\u00a0single source\u00a0of truth for classification, model training, and customer-facing explanations. This approach\u202f<\/span>reduces inter-model disagreement by up to 57\u00d7<\/span>\u202fcompared to paragraph-level definitions. Because the spec is machine-enforced, it applies with equal precision in French, Japanese, or Arabic.<\/span>\u00a0<\/span><\/p>\n

The taxonomy distinguishes\u202f<\/span>intent<\/span><\/i>\u202ffrom\u202f<\/span>content<\/span><\/i>: a conversation can carry harmful intent without harmful output (a probed-and-refused attack), or harmful content without adversarial intent (model misbehavior on a benign request). That distinction is essential in production, where the same surface language can mean\u00a0very different\u00a0things depending on conversational context.<\/span>\u00a0<\/span><\/p>\n

02<\/span> Security Has Moved Into the Conversation<\/h2>\n

In AI systems, ordinary language is the control plane. A malicious instruction can look identical to a\u00a0user\u00a0request; a benign phrase can look suspicious out of context. Attacks rarely arrive in a single prompt \u2014 real adversaries\u00a0iterate, reframe refusals, and escalate gradually across turns.\u202f<\/span>Cisco research across 15 frontier models<\/span>\u202ffound that every model tested\u00a0shows\u00a0meaningful multi-turn vulnerability, with attack success rates that bear no consistent relationship to single-turn benchmarks.<\/span>\u00a0<\/span><\/p>\n

This means the security perimeter must move outside the model. Cisco AI Defense\u00a0validates\u00a0inputs and outputs in production, classifying the\u202f<\/span>intent and active direction<\/span><\/i>\u202fof each conversation \u2014 not just the surface content of each message. Guardrails are tailored to the specific vulnerabilities of each model and\u00a0application, and\u00a0applied at the point where AI behavior is actually shaped: the live exchange between user, model, data, and tools.<\/span>\u00a0<\/span><\/p>\n

03<\/span> The Multilingual Reality Check<\/h2>\n

The ML6 benchmark put multilingual performance into sharp relief. Testing on 80,000 Dutch-language prompts \u2014 including prompt injection, policy bypass, ambiguous instructions, and realistic enterprise interactions \u2014 Cisco AI Defense achieved the highest F1 score in the cohort:\u202f<\/span><\/span>0.845<\/span><\/span>.<\/span><\/span>\u00a0<\/span><\/p>\n

\"\"

To\u00a0<\/span>highlight<\/span>\u00a0Cisco\u2019s\u00a0<\/span>multilingual\u00a0<\/span>capabilities \u2013<\/span> in this post we sample and share results on\u00a0<\/span><\/span>an augmented version of LMSYS Chat-1M and\u00a0<\/span>WildChat<\/span>\u00a0\u2014 two widely used open-source conversational datasets\u00a0<\/span>representing<\/span>\u00a0realistic enterprise chat traffic<\/span>. The data was\u00a0<\/span>augmented\u00a0<\/span>with conversations from e<\/span>ight\u00a0<\/span>additional<\/span>\u00a0<\/span>languages<\/span>\u00a0with a similar distribution as LMSYS and\u00a0<\/span>WildChat<\/span>.\u00a0<\/span>The ground truth labels for this dataset were generated\u00a0<\/span>using Cisco<\/span>\u00a0AI\u2019s<\/span>\u00a0security and safety taxonomy. The ML6 benchmark used a separate Dutch-specific dataset assembled independently; the two evaluations are complementary, not directly comparable.<\/span><\/span>\u00a0<\/span><\/p>\n

\"\"

Cisco AI Defense was evaluated on a multilingual, augmented conversational dataset derived primarily from the LMSYS Chat-1M and WildChat corpora. The evaluation set consists predominantly of benign, general-purpose conversations, along with an adversarial subset representing approximately 14% of the labeled examples. The dataset had approximately 5,800-5,900 conversations per language. FPR is measured on this specific adversarial evaluation mix; on a real-world distribution it would be much lower. Results are presented with English first, Dutch second, followed by the remaining languages ordered by F1 score.<\/p>\n

F1 ranges from 0.796 (Arabic) to 0.860 (Portuguese) \u2014 a tight spread across nine typologically diverse languages, from Latin-script European languages to Arabic and Japanese. That consistency reflects the constitutional taxonomy at work: when a definition is precise and machine-enforced, the signal transfers across languages reliably. The same operational specification governs whether a prompt injection is written in French, Japanese, or Arabic.<\/span><\/span>\u00a0<\/span><\/p>\n

\"\"

04<\/span> Protection Without Friction<\/h2>\n

A guardrail with\u00a0<\/span>high\u00a0<\/span>recall<\/span>\u00a0but\u00a0<\/span>poor precision is not a security product \u2014 it is an availability problem. In the ML6 benchmark,\u00a0<\/span>anot<\/span>her guardrail solution<\/span>\u00a0under test<\/span>\u00a0<\/span>reached 0.327 recall but only 0.453 F1, as false alarms collapsed precision to 0.737. Cisco achieved 0.843 recall and 0.847 precision simultaneously \u2014 the highest F1 in the cohort. That balance requires a threat model precise enough to distinguish an adversarial instruction from a legitimate but emphatic user request.<\/span><\/span>\u00a0<\/span><\/p>\n

\"\"

The FPR figures in the table \u2014 2.3\u20135.8% across languages \u2014 are measured on an evaluation mix that is roughly 14% adversarial. On a predominantly benign production population, the effective FPR would be much lower. More meaningful than the absolute values is their cross-language stability: the narrow range across nine languages indicates the constitutional taxonomy produces consistent signal rather than silently trading precision for recall as users switch languages. Operating thresholds are configurable without retraining, allowing organizations to tune the precision-recall tradeoff to their specific risk profile.<\/p>\n

05<\/span> Real-Time Protection<\/h2>\n

A guardrail that cannot keep pace with production traffic will not stay in the critical path. Enterprise AI applications have response-time SLAs; users notice latency; and in agentic pipelines, per-hop overhead compounds. Security that adds seconds per request gets disabled or bypassed.<\/p>\n

Cisco AI Defense is built to sit in the live interaction without becoming the bottleneck. At p90 = 40 ms and p99 = 250 ms per request, the security check adds overhead that is imperceptible to end users and compatible with real-time conversational SLAs across chatbots, copilots, and agentic pipelines.<\/p>\n

\"\"

Runtime protection is not a point-in-time test. AI applications evolve continuously: models are updated, RAG sources shift, agents acquire new tools, and attack techniques adapt. Pre-deployment evaluation establishes a baseline; runtime guardrails maintain it under live production conditions, for every user, in every language, across every model and application the enterprise runs \u2014 regardless of vendor or deployment framework.<\/p>\n

What Enterprises Should Take Away<\/h2>\n

Enterprise AI is multilingual and multi-turn by design. Security must match that reality. Cisco AI Defense addresses this from first principles:<\/p>\n