\n
Alright, my friends, I\u2019m back with another post based on my learnings and exploration of AI and how it\u2019ll fit into our work as network engineers. In today\u2019s post, I want to share the first (of what will likely be many) \u201cnerd knobs\u201d that I think we all should be aware of and how they will impact our use of AI and AI tools. I can already sense the excitement in the room. After all, there\u2019s not much a network engineer likes more than tweaking a nerd knob<\/em> in the network to fine-tune performance. And that\u2019s exactly what we\u2019ll be doing here. Fine-tuning our AI tools to help us be more effective.<\/p>\nFirst up, the requisite disclaimer or two.<\/p>\n
\nThere are SO MANY nerd knobs in AI.<\/strong> (Shocker, I know.) So, if you all like this kind of blog post, I\u2019d be happy to return in other posts where we look at other \u201cknobs\u201d and settings in AI and how they work. Well, I\u2019d be happy to return once I understand them, at least. \ud83d\ude42<\/li>\nChanging any of the settings<\/em> on your AI tools can have dramatic effects on results.<\/strong> This includes increasing the resource consumption of the AI model, as well as increasing hallucinations and decreasing the accuracy of the information that comes back from your prompts. Consider yourselves warned. As with all things AI, go forth and explore and experiment. But do so in a safe, lab environment.<\/li>\n<\/ol>\nFor today\u2019s experiment, I\u2019m once again using LMStudio running locally on my laptop rather than a public or cloud-hosted AI model. For more details on why I like LMStudio, check out my last blog, Creating a NetAI Playground for Agentic AI Experimentation.<\/p>\n
Enough of the setup, let\u2019s get into it!<\/p>\n
The impact of working memory size, a.k.a. \u201ccontext\u201d<\/strong><\/h2>\nLet me set a scene for you.<\/p>\n
You\u2019re in the middle of troubleshooting a network issue. Someone reported, or noticed, instability at a point in your network, and you\u2019ve been assigned the joyful task of getting to the bottom of it. You captured some logs and relevant debug information, and the time has come to go through it all to figure out what it means. But you\u2019ve also been using AI tools to be more productive, 10x your work, impress your boss, you know all the things<\/em> that are going on right now.<\/p>\nSo, you decide to see if AI can help you work through the data faster and get to the root of the issue.<\/p>\n
You fire up your local AI assistant. (Yes, local\u2014because who knows<\/em> what\u2019s in the debug messages? Best to keep it all safe on your laptop.)<\/p>\nYou tell it what you\u2019re up to, and paste in the log messages.<\/p>\nAsking AI to assist with troubleshooting<\/figcaption><\/figure>\nAfter getting 120 or so lines of logs into the chat, you hit enter, kick up your feet, reach for your Arnold Palmer for a refreshing drink, and wait for the AI magic to happen. But before you can take a sip of that iced tea and lemonade goodness<\/em>, you see this has immediately popped up on the screen:<\/p>\nAI Failure! \u201cThe AI has nothing to say\u201d<\/figcaption><\/figure>\nOh my.<\/p>\n
\u201cThe AI has nothing to say.\u201d<\/em>!?! How could that be?<\/p>\nDid you find a question so difficult that AI can\u2019t handle it?<\/p>\n
No, that\u2019s not the problem. Check out the helpful error message that LMStudio has kicked back:<\/p>\n
\u201cTrying to keep the first 4994 tokens when context the overflows. However, the model is loaded with context length of only 4096 tokens, which is not enough<\/strong>. Try to load the model with a larger context length, or provide shorter input.\u201d<\/em><\/p>\nAnd we\u2019ve gotten to the root of this perfectly scripted storyline and demonstration. Every AI tool out there has a limit to how much \u201cworking memory\u201d it has. The technical term for this working memory is \u201ccontext length.\u201d <\/em>If you try to send more data to an AI tool than can fit into the context length, you\u2019ll hit this error, or something like it.<\/p>\nThe error message indicates that the model was \u201cloaded with context length of only 4096 tokens.\u201d What is a \u201ctoken,\u201d you wonder? Answering that could be a topic of an entirely different blog post, but for now, just know that \u201ctokens\u201d are the unit of size for the context length. And the first thing that is done when you send a prompt to an AI tool is that the prompt is converted into \u201ctokens\u201d.<\/p>\n
So what do we do? Well, the message gives us two possible options: we can increase the context length of the model, or we can provide shorter input. Sometimes it isn\u2019t a big deal to provide shorter input. But other times, like when we are dealing with large log files, that option isn\u2019t practical\u2014all of the data is important.<\/p>\n
Time to turn the knob!<\/strong><\/h3>\nIt is that first option, to load the model with a larger context length, that is our nerd knob. Let\u2019s turn it.<\/p>\n
From within LMStudio, head over to \u201cMy Models\u201d and click to open up the configuration settings interface for the model.<\/p>\nAccessing Model Settings<\/figcaption><\/figure>\nYou\u2019ll get a chance to view all the knobs that AI models have. And as I mentioned, there are a lot of them.<\/p>\nDefault configuration settings<\/figcaption><\/figure>\nBut the one we care about right now is the Context Length. We can see that the default length for this model is 4096 tokens. But it supports up to 8192 tokens. Let\u2019s max it out!<\/p>\nMaxing out the Context Length<\/figcaption><\/figure>\nLMStudio provides a helpful warning and probable reason for why the model doesn\u2019t default to the max. The context length takes memory and resources. And raising it to \u201ca high value\u201d can impact performance and usage. So if this model had a max length of 40,960 tokens (the Qwen3 model I use sometimes has that high of a max), you might not want to just max it out right away. Instead, increase it by a little at a time to find the sweet spot: a context length big enough for the job, but not oversized.<\/p>\n
As network engineers, we are used to fine-tuning knobs for timers, frame sizes, and so many other things. This is right up our alley!<\/p>\n
Once you\u2019ve updated your context length, you\u2019ll need to \u201cEject\u201d and \u201cReload\u201d the model for the setting to take effect. But once that\u2019s done, it\u2019s time to take advantage of the change we\u2019ve made!<\/p>\nAI fully analyzes the logs<\/figcaption><\/figure>\nAnd look at that, with the larger context window, the AI assistant was able to go through the logs and give us a nice write-up about what they show.<\/p>\n
I particularly like the shade it threw my way: \u201c\u2026consider seeking assistance from \u2026 a qualified network engineer.\u201d <\/em>Well played, AI. Well played.<\/p>\nBut bruised ego aside, we can continue the AI assisted troubleshooting with something like this.<\/p>\nThe AI Assistant puts a timeline together<\/figcaption><\/figure>\nAnd we\u2019re off to the races. We\u2019ve been able to leverage our AI assistant to:<\/p>\n
\nProcess a significant amount of log and debug data to identify possible issues<\/li>\n Develop a timeline of the problem (that will be super useful in the help desk ticket and root cause analysis documents)<\/li>\n Identify some next steps we can do in our troubleshooting efforts.<\/li>\n<\/ol>\nAll stories must end\u2026<\/strong><\/h2>\nAnd so you have it, our first AI Nerd Knob\u2014Context Length. Let\u2019s review what we learned:<\/p>\n
\nAI models have a \u201cworking memory\u201d that is referred to as \u201ccontext length.\u201d<\/li>\n Context Length is measured in \u201ctokens.\u201d<\/li>\n Oftentimes times an AI model will support a higher context length than the default setting.<\/li>\n Increasing the context length will require more resources, so make changes slowly, don\u2019t just max it out completely.<\/li>\n<\/ol>\nNow, depending on what AI tool you\u2019re using, you may NOT be able to adjust the context length. If you\u2019re using a public AI like ChatGPT, Gemini, or Claude, the context length will depend on the subscription and models you have access to. However, there most definitely IS a context length that will factor into how much \u201cworking memory\u201d the AI tool has. And being aware of that fact, and its impact on how you can use AI, is important. Even if the knob in question is behind a lock and key. \ud83d\ude42<\/p>\n
If you enjoyed this look under the hood of AI and would like to learn about more options, please let me know in the comments: Do you have a favorite \u201cknob\u201d you like to turn? Share it with all of us. Until next time!<\/p>\n
PS\u2026 If you\u2019d like to learn more about using LMStudio, my buddy Jason Belk put a free tutorial together called Run Your Own LLM Locally For Free and with Ease that can get you started very quickly. Check it out!<\/em><\/p>\n
![\"Retail](\"https:\/\/dmsretail.com\/RetailNews\/wp-content\/uploads\/2022\/05\/RETAIL-ONLINE-TRAINING-728-X-90.png\")
<\/a><\/p>![\"Asking](\"https:\/\/storage.googleapis.com\/blogs-images-new\/ciscoblogs\/1\/2025\/07\/ai-knobs-context-01.png\")
![\"AI](\"https:\/\/storage.googleapis.com\/blogs-images-new\/ciscoblogs\/1\/2025\/07\/ai-knobs-context-02.png\")
![\"Accessing](\"https:\/\/storage.googleapis.com\/blogs-images-new\/ciscoblogs\/1\/2025\/07\/ai-knobs-context-03.png\")
![\"Default](\"https:\/\/storage.googleapis.com\/blogs-images-new\/ciscoblogs\/1\/2025\/07\/ai-knobs-context-04.png\")
![\"Maxing](\"https:\/\/storage.googleapis.com\/blogs-images-new\/ciscoblogs\/1\/2025\/07\/ai-knobs-context-05.png\")
![\"The](\"https:\/\/storage.googleapis.com\/blogs-images-new\/ciscoblogs\/1\/2025\/07\/ai-knobs-context-06.png\")
![\"AI](\"https:\/\/storage.googleapis.com\/blogs-images-new\/ciscoblogs\/1\/2025\/07\/ai-knobs-context-07.png\")