\n
Every technology leader faces the enterprise AI paradox: as AI adoption accelerates, organizations run headfirst into external power, cost, and security constraints. Cisco IT proved that infrastructure is the ultimate control point.\u00a0<\/span><\/i>\u00a0<\/span><\/p>\nKeep reading and watch the short video below, or tune into the full podcast discussion, to learn how we built a full-stack, multi-tenant AI factory\u2014tripling GPU utilization and delivering mission-critical reliability for 90,000 employees.\u00a0<\/i><\/p>\n<\/button><\/div>\nThe inflection point: From project-specific builds to mission-critical enterprise scale<\/h2>\n Like most enterprises, Cisco is on an ongoing AI journey. In the beginning, our mission was simple: get AI capabilities into the hands of teams so they could prove value quickly for specific use cases. We started with dedicated, project-specific AI infrastructure clusters for individual engineering and product teams. This approach was sufficient for experimental prototypes, but then we reached a crucial inflection point as AI evolved from isolated projects to mission-critical operations.<\/span>\u00a0<\/span><\/p>\nThe primary catalyst was the global deployment of MyAgent, Cisco\u2019s enterprise super agent rolled out across our 90,000-person workforce. At the same time, we scaled dedicated capacity for Cisco IQ, enabling hosting live production workloads for thousands of enterprise accounts. Cisco IQ provides unified, AI-powered capabilities through Cisco Support and Professional Services.<\/span>\u00a0<\/span><\/p>\nThis significant surge of agentic workflows required us to modernize our infrastructure to deliver high performance, embedded security, and economic control at enterprise scale.<\/span>\u00a0<\/span><\/p>\nFour barriers to scaling AI every CIO will face<\/h2>\n As we scaled our AI usage enterprise-wide, we knew we needed to address four critical barriers:<\/p>\n
\n Stranded capacity and low GPU utilization: <\/span><\/b>Static, project-based allocations created fragmented compute silos. Because GPUs were isolated by network and location, resources could not be pooled, leaving average utilization at 10%-15%. This forced teams to wait for entire GPU nodes to be vacated by other projects before they could begin their own workloads.<\/span><\/li>\n Unpredictable token costs and workload placement: <\/span><\/b>Relying solely on close-sourced <\/span>cloud models worked well for early pilots, but became less cost-efficient as AI scaled enterprise-wide. Compounding this challenge, a lack of visibility into token consumption across teams and tools made forecasting expenses and proving ROI nearly impossible.<\/span> Cisco\u2019s research with Omdia<\/span> found that cloud spending jumped 30% on average after AI deployment.<\/span><\/li>\n <\/span><\/b>Expanding security attack surfaces:<\/span><\/b> Scaling autonomous agents and multi-tenant workloads introduced new threat surfaces, from prompt injection and proprietary data loss to network threats that could move laterally without detection.<\/span><\/li>\n Thermal and power density ceilings: <\/span><\/b>Power availability is the true binding constraint. Even if an enterprise received 2,000 brand-new GPUs tomorrow, traditional air-cooled enterprise data centers hit thermal limits long before they can energize that compute density.<\/span><\/li>\n<\/ol>\nW<\/span><\/span>hile p<\/span>ower limits, <\/span>and supply chain <\/span>shortages, and cloud pricing volatility are external realities<\/span>,<\/span><\/span>Cisco IT <\/span>realized <\/span>we have a critical<\/span> internal control point: <\/span><\/span>our infrastructure<\/span><\/span>.<\/span><\/span><\/strong><\/p>\nPooling compute can reduce stranded capacity.<\/span> Real-time visibility into token spend \u2014 connected to business value \u2014 helps make costs predictable. \u00a0Building security into every layer of the stack delivers stronger protection than bolting it on as an afterthought. And optimizing rack density addresses power constraints. <\/span>\u00a0<\/span><\/p>\nOur infrastructure was the key to scaling efficiently, securely, and economically.<\/span>\u00a0<\/span><\/p>\nA full-stack solution: How we built our AI Factory<\/h2>\n To overcome these barriers and scale AI economically, <\/span>we<\/span> deployed a unified, full-stack, multi-tenant AI <\/span>f<\/span>actory across our global enterprise data centers<\/span>.<\/span> This architecture brings together <\/span>compute<\/span>, networking, security, and observability to give us the visibility and consistency needed to <\/span>operate<\/span> workloads across the stack from silicon to the applications:<\/span><\/span> <\/span><\/p>\n\nHigh-bandwidth pooled network fabric: <\/span><\/b><\/b>To reduce stranded capacity, we are consolidating previously fragmented backend networks into a single, high-bandwidth pooled fabric across our major data centers. <\/span>Cisco switches<\/span> powered by <\/span>Cisco<\/span> Silicon One<\/span> eliminate bottlenecks and ensure zero GPU packet-drop stalls. Because Silicon One is fully programmable, our network fabrics can quickly adapt to evolving AI demands, standards, and protocols without requiring costly hardware refreshes. Now, multi-tenant workloads dynamically and seamlessly distribute to available capacity in real time, preventing GPUs from sitting idle.<\/span><\/span><\/li>\nHeterogeneous compute: <\/span><\/b><\/b>No single hardware configuration fits every enterprise workload. We standardized on Cisco Compute with <\/span>UCS-X Series servers<\/span> supporting a strategic mix of NVIDIA and AMD GPUs. This heterogeneous <\/span>architecture provides us the foundation to evaluate and align compute profiles to workload demands \u2014 evaluating cost-effective silicon for high-volume inference while sizing capacity for intensive training. This gives us the flexibility to optimize token economics and adapt to evolving hardware supply.\u00a0<\/span><\/li>\nHigh-density rack cooling: <\/span><\/b><\/b>To address power and thermal limitations within our existing footprint, Cisco IT engineered rear-door heat exchangers directly onto our racks. This absorbs thermal exhaust at the source, dramatically boosting cooling efficiency and doubling our compute density to two full AI nodes per cabinet without expanding physical real estate.<\/span><\/span><\/li>\nNatively embedded AI security: <\/span><\/b><\/b>For security integrated at every layer, we deployed <\/span>Cisco AI Defens<\/span>e to apply runtime guardrails that inspect prompts and responses to prevent data leakage and stop threats before they reach our models and agents. <\/span>Cisco Hypershield<\/span> will provide segmentation across the fabric, and with <\/span>Live Protect<\/span> on our <\/span>Nexus switches<\/span>, we dynamically apply vulnerability shielding between patch windows, protecting our infrastructure against critical vulnerabilities without requiring reboots or taking production infrastructure offline.<\/span><\/span><\/li>\nReal-time observability, tokenomics, and closed-loop routing: <\/span><\/b><\/b>Splunk Observability Cloud<\/span> serves as Cisco IT\u2019s core metrics and traces repository for infrastructure, integrated with <\/span>Splunk Cloud Platform<\/span> for log analytics and Splunk IT Service Intelligence (ITSI) for event management. Together, this unified foundation will deliver deep operational visibility, rapid Mean Time to Resolution (MTTR), and enable autonomous observability and closed-loop operations. We are currently in the process of expanding that to <\/span>Agent Observability<\/span> and Tokenomics, and will share more on this journey soon.<\/span><\/li>\n<\/ol>\nTogether, these capabilities have helped Cisco IT scale AI while improving the economics, security, and operational support of our infrastructure.<\/p>\n
![\"Retail](\"https:\/\/dmsretail.com\/RetailNews\/wp-content\/uploads\/2022\/05\/RETAIL-ONLINE-TRAINING-728-X-90.png\")
<\/a><\/p>