{"id":16987,"date":"2026-04-24T14:39:26","date_gmt":"2026-04-24T14:39:26","guid":{"rendered":"https:\/\/dmsretail.com\/RetailNews\/how-cisco-it-innovates-with-less-risk\/"},"modified":"2026-04-24T14:39:26","modified_gmt":"2026-04-24T14:39:26","slug":"how-cisco-it-innovates-with-less-risk","status":"publish","type":"post","link":"https:\/\/dmsretail.com\/RetailNews\/how-cisco-it-innovates-with-less-risk\/","title":{"rendered":"How Cisco IT Innovates with Less Risk"},"content":{"rendered":"<p> <p><a href=\"https:\/\/dmsretail.com\/online-workshops-list\/\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-496\" src=\"https:\/\/dmsretail.com\/RetailNews\/wp-content\/uploads\/2022\/05\/RETAIL-ONLINE-TRAINING-728-X-90.png\" alt=\"Retail Online Training\" width=\"729\" height=\"91\" srcset=\"https:\/\/dmsretail.com\/RetailNews\/wp-content\/uploads\/2022\/05\/RETAIL-ONLINE-TRAINING-728-X-90.png 729w, https:\/\/dmsretail.com\/RetailNews\/wp-content\/uploads\/2022\/05\/RETAIL-ONLINE-TRAINING-728-X-90-300x37.png 300w\" sizes=\"auto, (max-width: 729px) 100vw, 729px\" \/><\/a><\/p><br \/>\n<\/p>\n<div>\n<blockquote>\n<p>Every IT leader faces the same paradox: innovate faster while maintaining rock-solid stability. At Cisco IT, we were deploying AI systems and new technologies at breakneck speed\u2014and watching our incident rate climb. Then we turned it around. Here\u2019s how we reduced major incidents by 25% in one year while <em>accelerating\u00a0<\/em>our pace of innovation.<\/p>\n<\/blockquote>\n<p style=\"text-align: center;\"><iframe class=\"lazy lazy-hidden\" loading=\"lazy\" data-lazy-type=\"iframe\" data-src=\"https:\/\/players.brightcove.net\/1384193102001\/41XYD7gTx_default\/index.html?videoId=6388828645112\" allowfullscreen=\"\" webkitallowfullscreen=\"\" mozallowfullscreen=\"\" width=\"640\" height=\"360\"><\/iframe><noscript><iframe loading=\"lazy\" src=\"https:\/\/players.brightcove.net\/1384193102001\/41XYD7gTx_default\/index.html?videoId=6388828645112\" allowfullscreen=\"\" webkitallowfullscreen=\"\" mozallowfullscreen=\"\" width=\"640\" height=\"360\"><\/iframe><\/noscript><\/p>\n<h2><strong>The innovation tax: When speed becomes your enemy<\/strong><\/h2>\n<p><span data-contrast=\"auto\">Like most IT organizations, we were adding AI capabilities, deploying cloud services, and modernizing applications at an unprecedented pace. Innovation was our mandate.<\/span><span data-ccp-props=\"{&quot;134233117&quot;:false,&quot;134233118&quot;:false,&quot;335559738&quot;:240,&quot;335559739&quot;:240}\">\u00a0<\/span><\/p>\n<p><span data-contrast=\"auto\">But with each new system came hidden costs:<\/span><span data-ccp-props=\"{&quot;134233117&quot;:false,&quot;134233118&quot;:false,&quot;335559738&quot;:240,&quot;335559739&quot;:240}\">\u00a0<\/span><\/p>\n<ul>\n<li><b><span data-contrast=\"auto\">Visibility gaps:<\/span><\/b><span data-contrast=\"auto\"> New technologies brought new dashboards \u2014 each siloed, none talking to each other. Our operations team was drowning in alerts with no unified view of actual business impact.<\/span><span data-ccp-props=\"{&quot;134233117&quot;:false,&quot;134233118&quot;:false,&quot;335559738&quot;:240,&quot;335559739&quot;:240}\">\u00a0<\/span><\/li>\n<li><b><span data-contrast=\"auto\">Change-driven instability:<\/span><\/b><span data-contrast=\"auto\"> We discovered a direct correlation; the more changes we pushed, the more incidents we experienced. Innovation was causing outages.<\/span><span data-ccp-props=\"{&quot;134233117&quot;:false,&quot;134233118&quot;:false,&quot;335559738&quot;:240,&quot;335559739&quot;:240}\">\u00a0<\/span><\/li>\n<li><b><span data-contrast=\"auto\">AI uncertainty:<\/span><\/b><span data-contrast=\"auto\"> While AI promised efficiency, it also introduced new failure modes. How do you monitor what you don\u2019t fully understand?<\/span><span data-ccp-props=\"{&quot;134233117&quot;:false,&quot;134233118&quot;:false,&quot;335559738&quot;:240,&quot;335559739&quot;:240}\">\u00a0<\/span><\/li>\n<\/ul>\n<p><span data-contrast=\"auto\">The question became urgent: How do we innovate without disruption?<\/span><span data-ccp-props=\"{&quot;134233117&quot;:false,&quot;134233118&quot;:false,&quot;335559738&quot;:240,&quot;335559739&quot;:240}\">\u00a0<\/span><\/p>\n<p><span data-contrast=\"auto\">To address this, Cisco IT has made observability a cornerstone of our approach.<\/span><span data-ccp-props=\"{&quot;134233117&quot;:false,&quot;134233118&quot;:false,&quot;335559738&quot;:240,&quot;335559739&quot;:240}\">\u00a0<\/span><\/p>\n<h2><strong>Our North Star: Innovation without disrupt\u00a0<\/strong><\/h2>\n<p aria-level=\"3\"><span data-contrast=\"auto\">Rather than slow down innovation, we made a different choice: become radically better at observability.<\/span><span data-ccp-props=\"{&quot;134233117&quot;:false,&quot;134233118&quot;:false,&quot;134245418&quot;:true,&quot;134245529&quot;:true,&quot;201341983&quot;:0,&quot;335551550&quot;:1,&quot;335551620&quot;:1,&quot;335559685&quot;:0,&quot;335559737&quot;:0,&quot;335559738&quot;:0,&quot;335559739&quot;:160,&quot;335559740&quot;:278}\">\u00a0<\/span><\/p>\n<p><span data-contrast=\"auto\">Our Service Operations team and Enterprise Operations Center (EOC) set three clear objectives:<\/span><span data-ccp-props=\"{&quot;134233117&quot;:false,&quot;134233118&quot;:false,&quot;335551550&quot;:1,&quot;335551620&quot;:1,&quot;335559738&quot;:240,&quot;335559739&quot;:240}\">\u00a0<\/span><\/p>\n<ol>\n<li><b><span data-contrast=\"auto\">Detect faster<\/span><\/b><span data-contrast=\"auto\"> \u2013 Spot issues before users report them, with complete business impact context<\/span><span data-ccp-props=\"{&quot;134233117&quot;:false,&quot;134233118&quot;:false,&quot;335551550&quot;:1,&quot;335551620&quot;:1,&quot;335559738&quot;:0,&quot;335559739&quot;:0}\">\u00a0<\/span><\/li>\n<li><b><span data-contrast=\"auto\">Assign smarter<\/span><\/b><span data-contrast=\"auto\"> \u2013 Route problems to the right experts immediately, no handoffs<\/span><span data-ccp-props=\"{&quot;134233117&quot;:false,&quot;134233118&quot;:false,&quot;335551550&quot;:1,&quot;335551620&quot;:1,&quot;335559738&quot;:0,&quot;335559739&quot;:0}\">\u00a0<\/span><\/li>\n<li><b><span data-contrast=\"auto\">Resolve proactively<\/span><\/b><span data-contrast=\"auto\"> \u2013 Fix issues automatically when possible, communicate clearly when not<\/span><span data-ccp-props=\"{&quot;134233117&quot;:false,&quot;134233118&quot;:false,&quot;335551550&quot;:1,&quot;335551620&quot;:1,&quot;335559738&quot;:0,&quot;335559739&quot;:0}\">\u00a0<\/span><\/li>\n<\/ol>\n<p><span data-contrast=\"auto\">The goal wasn\u2019t just faster incident response. It was to make our environment so observable that we could innovate <\/span><span data-contrast=\"auto\">faster, and with less risk.<\/span><span data-ccp-props=\"{&quot;134233117&quot;:false,&quot;134233118&quot;:false,&quot;335551550&quot;:1,&quot;335551620&quot;:1,&quot;335559685&quot;:0,&quot;335559738&quot;:0,&quot;335559739&quot;:0}\">\u00a0<\/span><\/p>\n<h2><span data-ccp-props=\"{&quot;134233117&quot;:false,&quot;134233118&quot;:false,&quot;201341983&quot;:0,&quot;335551550&quot;:1,&quot;335551620&quot;:1,&quot;335559685&quot;:0,&quot;335559737&quot;:0,&quot;335559738&quot;:0,&quot;335559739&quot;:160,&quot;335559740&quot;:278}\">\u00a0<strong>Cisco IT\u2019s observability approach and technology<\/strong><\/span><\/h2>\n<p><span data-contrast=\"auto\">For Cisco IT, observability is critical to delivering end-to-end visibility, actionable insights, and AI-driven automation to enable us to detect, address, and even prevent issues\u202fbefore they impact the business.<\/span><span data-ccp-props=\"{}\">\u00a0<\/span><\/p>\n<p><span data-contrast=\"auto\">Cisco IT\u2019s <\/span><span data-contrast=\"none\">observability strategy<\/span><span data-contrast=\"auto\"> is built on a layered approach spanning three teams. In the first two \u2018layers\u2019, dedicated teams are responsible for end-to-end observability across our network, applications, services, and infrastructure. Leveraging critical solutions like ThousandEyes and Splunk, they aggregate telemetry from our global environment and transform raw data into meaningful insights.<\/span><span data-ccp-props=\"{}\">\u00a0<\/span><span data-ccp-props=\"{}\">\u00a0<\/span><\/p>\n<ul>\n<li><span data-contrast=\"none\">Splunk<\/span><span data-contrast=\"auto\">: Our central nervous system for IT health. By aggregating logs, metrics, and events across our global infrastructure, Splunk gave us something we\u2019d never had: a single source of truth. When an issue emerges, our team sees correlated signals across system \u2014 not isolated alerts \u2014 enabling us to understand root cause in minutes, not hours.\u00a0<\/span><\/li>\n<li><span data-contrast=\"none\">Cisco ThousandEyes<\/span><span data-contrast=\"auto\">: <\/span><span data-contrast=\"none\">Our eyes on the end-user experience. ThousandEyes provides deep visibility into network paths and application performance from the user\u2019s perspective \u2014 pinpointing exactly where and why slowdowns occur. When a critical application underperforms, our Service Operations team doesn\u2019t guess whether it\u2019s our network, a third-party provider, or the application itself. We know immediately, isolate the issue, and engage the right team to fix it \u2014 often before users open a ticket.<\/span><\/li>\n<\/ul>\n<p>Our Service Operations team is where these insights are put into action to quickly identify, address, and even prevent issues before they impact the business.<span data-ccp-props=\"{&quot;335559685&quot;:40}\">\u00a0<\/span><\/p>\n<p><span data-contrast=\"auto\">To enable our team to use the data and insights from these solutions even more effectively,<\/span><span data-contrast=\"none\"> we deploy AI-driven automation across a variety of incident management use cases:<\/span><span data-ccp-props=\"{}\">\u00a0<\/span><\/p>\n<ul>\n<li><b><span data-contrast=\"auto\">Predict assignment groups:<\/span><\/b><span data-contrast=\"auto\"> AI analyzes incident descriptions against historical patterns to route issues to the right team immediately. <\/span><span data-contrast=\"auto\">This has resulted in a 19% reduction in reassignments and faster time-to-expertise.<\/span><span data-ccp-props=\"{&quot;134233117&quot;:false,&quot;134233118&quot;:false,&quot;335559738&quot;:240,&quot;335559739&quot;:240}\">\u00a0<\/span><\/li>\n<li><b><span data-contrast=\"auto\">Suggest resolution options:<\/span><\/b><span data-contrast=\"auto\"> By matching current issues to our knowledge base of 100,000+ resolved incidents, AI surfaces proven fixes instantly.\u00a0<\/span><span data-ccp-props=\"{&quot;134233117&quot;:false,&quot;134233118&quot;:false,&quot;335559738&quot;:240,&quot;335559739&quot;:240}\">\u00a0<\/span><\/li>\n<li><b><span data-contrast=\"auto\">Automate resolution:<\/span><\/b><span data-contrast=\"auto\"> Self-healing systems now handle routine issues like storage cleanup and session resets without human intervention. A<\/span><span data-contrast=\"auto\">I-automations now handle 99.998% of ~4 million daily alerts that represent potential issues\/incidents.<\/span><span data-ccp-props=\"{&quot;134233117&quot;:false,&quot;134233118&quot;:false,&quot;335559738&quot;:240,&quot;335559739&quot;:240}\">\u00a0<\/span><\/li>\n<\/ul>\n<p><span data-contrast=\"auto\">While observability platforms and automation provide a critical foundation, technology alone isn\u2019t enough. That\u2019s where our team and established best practices make the difference.<\/span><span data-ccp-props=\"{&quot;335559685&quot;:0}\">\u00a0<\/span><\/p>\n<h2><strong>Beyond the technology: the human element of observability<\/strong><\/h2>\n<p><span data-contrast=\"auto\">The true value of our team goes beyond technology \u2014 it lies in the people and processes that convert information and insights into action. We work to quickly detect, analyze, assign, and resolve issues to minimize disruption.\u00a0<\/span><span data-ccp-props=\"{}\">\u00a0<\/span><\/p>\n<p><span data-contrast=\"auto\">To do this effectively, we\u2019ve recognized 3 best practices are key to our success:<\/span><span data-ccp-props=\"{}\">\u00a0<\/span><\/p>\n<ul>\n<li><b><span data-contrast=\"auto\">Intelligent change management: <\/span><\/b><i><span data-contrast=\"auto\">N<\/span><\/i><i><span data-contrast=\"auto\">ot all changes carry equal risk. Treat them accordingly.<\/span><\/i><span data-contrast=\"auto\">We didn\u2019t slow down changes \u2014 we got smarter about them. By categorizing changes based on risk, we automated approvals for 80% of standard, low-risk tasks while intensifying our focus and monitoring for higher-risk initiatives. The takeaway here is that not all changes carry equal risk. Treat them accordingly.<\/span><\/li>\n<\/ul>\n<p>\u00a0<\/p>\n<ul>\n<li><b><span data-contrast=\"auto\">Data quality and accuracy<\/span><\/b><span data-contrast=\"auto\">: <\/span><i><span data-contrast=\"auto\">Quality AI requires quality data. Prioritize CMDB hygiene.<\/span><\/i><span data-contrast=\"auto\">Our foundation for AI effectiveness. AI is only as intelligent as the data feeding it \u2014 garbage in, garbage out. We built a comprehensive data quality framework around our Enterprise Service Platform (ESP), with our Configuration Management Database (CMDB) serving as the single source of truth for our entire technology environment. Through automated quality reporting and workflows, we continuously identify gaps, flag stale information, and trigger updates in real-time. When our AI predicts assignment groups or suggests resolutions, it\u2019s working from accurate, current data \u2014 not outdated records from three months ago.\u00a0<\/span><span data-ccp-props=\"{}\">\u00a0<\/span><\/li>\n<\/ul>\n<p>\u00a0<\/p>\n<ul>\n<li><b><span data-contrast=\"auto\">Effective communications:<\/span><\/b> <i><span data-contrast=\"auto\">In a crisis, clarity is as valuable as speed.<\/span><\/i><span data-contrast=\"auto\">Our bridge between technical chaos and business clarity. During critical incidents, technical teams understand the problem, but business stakeholders need to understand the <\/span><i><span data-contrast=\"auto\">impact<\/span><\/i><span data-contrast=\"auto\">. Our Service Operations team translates complex technical issues into clear business language: which services are affected, how many users are impacted, what we\u2019re doing to fix it, and when normal operations will resume. This disciplined communication approach keeps executives informed without overwhelming them, enables business units to make contingency decisions quickly, and maintains trust even during disruptions.\u00a0<\/span><span data-ccp-props=\"{}\">\u00a0<\/span><\/li>\n<\/ul>\n<h2><strong>The bottom line: Measurable business impact<\/strong><\/h2>\n<p><span data-contrast=\"auto\">Over 18 months, our observability transformation delivered results that directly enabled business agility:<\/span><span data-ccp-props=\"{&quot;134233117&quot;:false,&quot;134233118&quot;:false,&quot;201341983&quot;:0,&quot;335551550&quot;:1,&quot;335551620&quot;:1,&quot;335559685&quot;:0,&quot;335559737&quot;:0,&quot;335559738&quot;:0,&quot;335559739&quot;:160,&quot;335559740&quot;:278}\">\u00a0<\/span><\/p>\n<ul>\n<li><b><span data-contrast=\"auto\">25% reduction in major incidents<\/span><\/b><span data-contrast=\"auto\"> \u2013 Fewer disruptions to employee productivity and customer-facing services<\/span><span data-ccp-props=\"{&quot;134233117&quot;:false,&quot;134233118&quot;:false,&quot;335559738&quot;:240,&quot;335559739&quot;:240}\">\u00a0<\/span><\/li>\n<li><b><span data-contrast=\"auto\">20% fewer change-related incidents<\/span><\/b><span data-contrast=\"auto\"> \u2013 Innovation without instability<\/span><span data-ccp-props=\"{&quot;134233117&quot;:false,&quot;134233118&quot;:false,&quot;335559738&quot;:240,&quot;335559739&quot;:240}\">\u00a0<\/span><\/li>\n<li><b><span data-contrast=\"auto\">45% faster mean time to restore<\/span><\/b><span data-contrast=\"auto\"> \u2013 From hours to minutes for critical service recovery<\/span><span data-ccp-props=\"{&quot;134233117&quot;:false,&quot;134233118&quot;:false,&quot;335559738&quot;:240,&quot;335559739&quot;:240}\">\u00a0<\/span><\/li>\n<li><b><span data-contrast=\"auto\">80% of changes now auto-approved<\/span><\/b><span data-contrast=\"auto\"> \u2013 Faster deployment, lower risk<\/span><span data-ccp-props=\"{&quot;134233117&quot;:false,&quot;134233118&quot;:false,&quot;335559738&quot;:240,&quot;335559739&quot;:240}\">\u00a0<\/span><\/li>\n<\/ul>\n<p><b><span data-contrast=\"auto\">What this means:<\/span><\/b><span data-contrast=\"auto\"> Cisco employees experience fewer disruptions, IT teams spend less time firefighting and more time innovating, and the business moves faster with confidence.<\/span><span data-ccp-props=\"{&quot;134233117&quot;:false,&quot;134233118&quot;:false,&quot;335559738&quot;:240,&quot;335559739&quot;:240}\">\u00a0<\/span><\/p>\n<p>\u00a0<\/p>\n<h2><strong>Ready to transform your IT operations?<\/strong><\/h2>\n<p><span class=\"TextRun SCXW109945358 BCX4\" lang=\"EN-US\" xml:lang=\"EN-US\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW109945358 BCX4\">The lessons from Cisco IT\u2019s observability journey are clear: you <\/span><span class=\"NormalTextRun SCXW109945358 BCX4\">don\u2019t<\/span><span class=\"NormalTextRun SCXW109945358 BCX4\"> have to choose between innovation and stability. With the right approach to observability, AI-driven automation, and operational discipline, you can have both.<\/span><\/span><span class=\"EOP SCXW109945358 BCX4\" data-ccp-props=\"{&quot;134233117&quot;:false,&quot;134233118&quot;:false,&quot;335559738&quot;:240,&quot;335559739&quot;:240}\">\u00a0<\/span><\/p>\n<p>\u00a0<\/p>\n<p>\u00a0<\/p>\n<p><strong>Next Steps:\u00a0<\/strong><\/p>\n<p><span data-ccp-props=\"{&quot;134233117&quot;:false,&quot;134233118&quot;:false,&quot;335557856&quot;:16777215,&quot;335559685&quot;:720,&quot;335559738&quot;:220,&quot;335559739&quot;:220}\">\u00a0<\/span><\/p>\n<\/p><\/div>\n<p><p><a href=\"https:\/\/dmsretail.com\/online-workshops-list\/\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-496\" src=\"https:\/\/dmsretail.com\/RetailNews\/wp-content\/uploads\/2022\/05\/RETAIL-ONLINE-TRAINING-728-X-90.png\" alt=\"Retail Online Training\" width=\"729\" height=\"91\" srcset=\"https:\/\/dmsretail.com\/RetailNews\/wp-content\/uploads\/2022\/05\/RETAIL-ONLINE-TRAINING-728-X-90.png 729w, https:\/\/dmsretail.com\/RetailNews\/wp-content\/uploads\/2022\/05\/RETAIL-ONLINE-TRAINING-728-X-90-300x37.png 300w\" sizes=\"auto, (max-width: 729px) 100vw, 729px\" \/><\/a><\/p><br \/><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Every IT leader faces the same paradox: innovate faster while maintaining rock-solid stability. At Cisco IT, we were deploying AI systems and new technologies at [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":16988,"comment_status":"","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[5],"tags":[],"class_list":["post-16987","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-technology"],"_links":{"self":[{"href":"https:\/\/dmsretail.com\/RetailNews\/wp-json\/wp\/v2\/posts\/16987","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/dmsretail.com\/RetailNews\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/dmsretail.com\/RetailNews\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/dmsretail.com\/RetailNews\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/dmsretail.com\/RetailNews\/wp-json\/wp\/v2\/comments?post=16987"}],"version-history":[{"count":0,"href":"https:\/\/dmsretail.com\/RetailNews\/wp-json\/wp\/v2\/posts\/16987\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/dmsretail.com\/RetailNews\/wp-json\/wp\/v2\/media\/16988"}],"wp:attachment":[{"href":"https:\/\/dmsretail.com\/RetailNews\/wp-json\/wp\/v2\/media?parent=16987"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/dmsretail.com\/RetailNews\/wp-json\/wp\/v2\/categories?post=16987"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/dmsretail.com\/RetailNews\/wp-json\/wp\/v2\/tags?post=16987"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}