AI Fluency for Utility Leaders

Every week, another vendor walks into a utility boardroom with a pitch deck full of AI promises. Predictive maintenance that will save millions. Customer service chatbots that will cut call volumes in half. Automated rate analysis that will transform how you do business.

The technology is real, and the potential is genuine. But here's the uncomfortable truth: most utility executives and board members don't yet have the vocabulary to evaluate these claims. Not because they aren't smart — they are — but because AI has its own language, and nobody has bothered to translate it for the people who actually make the buying decisions.

This guide is that translation. We're not going to teach you how to build a neural network. We're going to give you the conceptual toolkit you need to walk into any AI conversation and hold your own — to ask the questions that separate a serious proposal from a glossy slide deck.

What AI Actually Is (and Isn't)

The AI Family Tree

People use "AI" to mean everything from a simple spreadsheet formula to a science fiction robot. In reality, there's a clear hierarchy, and understanding it matters because each layer does fundamentally different things.

Artificial Intelligencethe broadest category. It's any system that performs tasks normally requiring human intelligence. Your spam filter is AI. Your utility's demand forecasting model is AI. The term covers a huge range of sophistication.

Machine Learninga subset of AI where systems learn from data rather than following hand-coded rules. Instead of a programmer writing "if temperature exceeds X, flag this pipe," a machine learning model looks at thousands of pipe failures and figures out the patterns on its own. Think of it like the difference between giving someone a recipe versus letting them taste a hundred dishes and figure out how to cook.

Deep Learninga further subset of machine learning that uses layered mathematical structures called neural networks. These models can find incredibly complex patterns — like recognizing a water main break from acoustic sensor data, or detecting anomalies in SCADA readings that no human would catch. The "deep" refers to the many layers of processing, not any philosophical depth.

Generative AIthe newest branch, and the one making all the headlines. While traditional AI analyzes and classifies data, generative AI creates new content: text, images, code, analysis. When you're chatting with an AI assistant about your rate case strategy, that's generative AI at work.

What a Large Language Model (LLM) Is

A large language model is the engine behind tools like Claude, ChatGPT, and Gemini. Think of it as a system that has read an enormous portion of the text ever written — books, articles, technical papers, code, regulatory filings — and learned the statistical patterns of how language works. When you ask it a question, it's not looking up the answer in a database. It's generating a response based on the patterns it learned during training.

An analogy for the utility world: imagine if you could take every rate case filing, every AWWA manual, every PUC order ever published, and distill all of that knowledge into a single analyst. That analyst wouldn't have perfect memory of every document, but they'd have an extraordinarily good sense of how these things work, what language regulators use, and what arguments tend to appear. That's roughly what an LLM does — at scale, and imperfectly.

What "Training" Means

Training is the process of feeding massive amounts of data into a model so it can learn patterns. This happens before you ever interact with the model. The training data for major LLMs includes publicly available text from the internet, books, academic papers, and more. This is an important distinction: the model was trained once (or periodically updated) on general data. It's not learning from your specific conversations in real time.

This matters because of a common fear: "If I put my utility's financial data into an AI, will it learn from it and share it with others?" On enterprise plans from reputable providers, the answer is definitively no. Your conversations and data are not used to train the model. We'll cover this more in the security section, but the short version is: enterprise AI agreements explicitly prohibit using your data for model training.

What "Hallucination" Means

This is perhaps the most important concept for any utility leader to understand. AI hallucination is when a model generates information that sounds completely plausible but is factually wrong. It's not lying — it doesn't have the concept of truth. It's producing text that statistically fits the pattern of a correct answer.

For utility work, this is a serious concern. An AI might confidently cite a regulatory precedent that doesn't exist, produce a financial calculation with a subtle error, or reference an AWWA standard number that's close to real but slightly off. The output reads like it was written by an expert, but the facts may be invented.

How You Interact With AI

Prompts and Prompt Engineering

A prompt is simply the input you give to an AI — the question you ask, the instruction you write, the context you provide. Prompt engineering is the practice of crafting those inputs to get better results.

Think of it like writing a work order for a contractor. "Fix the leak" will get you very different results than "Repair the 6-inch ductile iron main break at the intersection of Oak and Elm, isolate the section, notify affected customers in the 400 block, and document the repair for the asset management system." Both are valid instructions, but the second one produces dramatically better outcomes.

The same principle applies to AI. A vague prompt produces a vague answer. A specific prompt with context, constraints, and clear expectations produces output that's actually useful. This is a skill your staff can learn in hours, not months — and it has an outsized impact on the value you get from AI tools.

Context Windows

A context window is the amount of information an AI can "hold in its head" during a single conversation. Think of it like the working memory of your analyst. If you hand someone a 300-page rate case filing and ask them to discuss it, they can reference any part of it because it's all in front of them. But if you hand them a 5,000-page document, they can only focus on so much at once.

Today's leading models have context windows ranging from about 200,000 to over 1 million tokens (roughly 150,000 to 750,000 words). This means you can feed an AI an entire cost-of-service study, a full set of tariff schedules, or years of board meeting minutes — and it can reference all of it in a single conversation. This is a transformative capability for utility work, where decisions often require synthesizing information scattered across dozens of documents.

AI Memory

A context window is short-term memory — it lasts one conversation and then it's gone. AI memory is the newer, persistent kind: the assistant retains facts, preferences, and project context across conversations. Tell it once that your utility operates on a June 30 fiscal year-end, and it applies that knowledge next month without being reminded. Between late 2025 and mid-2026, every major assistant — ChatGPT, Claude, Gemini, Copilot — added some form of persistent memory.

For leaders, the governance question matters more than the convenience. Memory is a new data store living inside your AI tool. What is being remembered about your projects, your finances, your customers? Can administrators review and delete it? Is it on or off by default? On most enterprise plans, memory ships off by default and requires an administrator to enable it — which is exactly the kind of setting your AI governance committee should own, rather than leaving it to individual employees to discover.

Tokens

Tokens are the unit of measurement for AI. Everything the model reads and writes gets broken into tokens — roughly one token per three-quarters of a word. "Water utility rate case" is about five tokens.

Tokens matter for two practical reasons. First, they determine cost: API access to AI models is priced per token. More tokens in and out means higher bills. Second, they define the limits of a conversation: that context window we just discussed is measured in tokens. Understanding tokens helps you evaluate vendor pricing and understand why some AI requests cost more than others.

Temperature

Temperature is a setting that controls how creative or predictable an AI's responses are. At low temperature (close to zero), the model gives you the most statistically likely response — safe, predictable, consistent. At high temperature, it takes more creative risks, producing more varied and sometimes surprising output.

For utility work, this has practical implications. If you're using AI to draft regulatory language or produce financial calculations, you want low temperature — consistency and accuracy matter. If you're brainstorming new approaches to customer engagement or exploring innovative rate structures, higher temperature can generate ideas you wouldn't have considered.

Reasoning Models and Extended Thinking

The newest generation of models can "think" before they answer — working through a problem step by step internally before producing a response. These reasoning models (you'll also hear the term "extended thinking") are slower and cost more per query, but they are dramatically better at hard analytical work: multi-step financial modeling, untangling how a rate design interacts with a regulatory requirement, checking a complex calculation.

The leadership takeaway is about matching the model to the task. You don't pay reasoning-model prices to summarize meeting notes, and you don't hand a lightweight model your cost-of-service allocation. Providers now offer tiers of capability at very different price points — the practical question for your team is which tier does which work, at what cost.

How AI Connects to Your Systems

APIs: How Software Talks to AI

An API (Application Programming Interface) is the mechanism that lets one piece of software communicate with another. When your CIS system pulls data from your billing database, it's using an API. When a mobile app displays your utility's outage map, it's using an API.

In the AI context, APIs are how your internal systems can send data to an AI model and receive analysis back — without anyone manually copying and pasting information. This is the difference between having an employee ask an AI chatbot a question and having your work order management system automatically analyze incoming service requests and route them intelligently.

MCP: Model Context Protocol

Model Context Protocol is an emerging standard that lets AI models securely connect to your databases, documents, and internal systems. Think of it like giving the AI a read-only library card to your organization's information. Instead of you manually uploading a document and asking the AI about it, MCP allows the AI to directly access the data it needs — with proper permissions and security controls.

For utilities, this means an AI assistant could potentially pull real-time data from your SCADA system, reference your current tariff schedules, and access your capital improvement plan — all within a single conversation — without you having to gather and upload each document manually.

AI Agents

Most AI today is reactive: you ask a question, it gives an answer. An AI agent goes further — it can take actions. An agent can break a complex task into steps, use tools, access systems, and execute multi-step workflows with minimal human direction.

Imagine telling an AI: "Analyze last quarter's water loss data, compare it to the three-year trend, flag any distribution zones where non-revenue water increased by more than 5%, and draft a summary memo for the operations team." A basic AI chatbot would give you a generic explanation of how to do that. An AI agent would actually go do it — pulling the data, running the analysis, and producing the memo.

Agents moved from concept to production during 2026 — you'll now find them in coding tools, customer service platforms, and back-office automation, and vendors increasingly describe their products as "agentic AI." Because an agent takes actions rather than just answering questions, it introduces a new class of risk: it needs defined permissions, an audit trail, and human checkpoints before consequential steps. The vendor question has shifted from "what can your AI tell us?" to "what can your agent actually do, who approved that list, and who reviews its work?"

RAG: Retrieval-Augmented Generation

RAG is a technique that dramatically reduces hallucinations by connecting an AI model to a specific, trusted set of documents. Instead of relying solely on what it learned during training, the model first searches your documents for relevant information and then generates its response based on what it actually found.

This is particularly powerful for utility applications. A RAG-enabled system connected to your regulatory filings, engineering standards, and policy documents can answer questions with specific citations — pulling exact passages from your own documents rather than generating plausible-sounding but potentially inaccurate responses.

Fine-Tuning

Fine-tuning is the process of taking a general-purpose AI model and training it further on your specific data to improve its performance in your domain. Think of it like hiring a generalist consultant and then immersing them in your industry for six months before putting them to work.

For most utilities, fine-tuning is not the first step. It's expensive, requires significant technical expertise, and the general-purpose models are already remarkably capable for most utility tasks. RAG is usually a better starting point because it achieves domain-specific accuracy without the cost and complexity of model customization. Fine-tuning makes sense when you need the model to adopt very specific patterns — like matching your utility's writing style for customer communications or learning the nuances of your state's regulatory language.

The AI Landscape for Utilities

The Major Models

The AI market moves fast — leading providers now ship new versions every few months, so specific model numbers go stale almost as soon as they're printed. What's durable is the model family and the provider behind it. As of mid-2026, here are the major players your organization is most likely to encounter:

Claude (by Anthropic)known for strong reasoning, long context windows, and a focus on safety and accuracy. Popular in professional and enterprise environments where reliability matters more than novelty.

GPT (by OpenAI)the model behind ChatGPT, the product that brought AI into the mainstream. Widely used across consumer and business applications, with a broad ecosystem of integrations and plugins.

Gemini (by Google)Google's flagship model, tightly integrated with Google Workspace. If your utility runs on Google's ecosystem, Gemini has a natural integration advantage.

Llama (by Meta)the model family that popularized open-weight AI — models you can download and run on your own servers. This matters for utilities with strict data sovereignty requirements who want AI capability without sending any data to an external provider. Llama now shares that space with open-weight releases from OpenAI, Google, Microsoft, and others (more on this below).

Copilot (by Microsoft)Microsoft's AI assistant, embedded directly into Office 365, Teams, and other Microsoft products. For utilities heavily invested in the Microsoft stack, Copilot offers AI capabilities without requiring a separate tool.

You will also encounter Grok (by xAI) and capable open-weight models from Chinese labs such as DeepSeek and Qwen (by Alibaba). The latter are technically impressive and inexpensive — but where a model was trained, and who maintains it, is a legitimate procurement question, particularly for critical infrastructure. One more question worth asking any vendor: providers retire older models on roughly twelve- to eighteen-month cycles, so what happens to your workflows when the model behind their product changes or disappears?

Small Language Models (SLMs)

Not every task needs a frontier model. A small language model is exactly what it sounds like — a model with a few billion parameters instead of hundreds of billions. It trades breadth for economy: its general knowledge is shallower, but a small model tuned for a specific domain can match or beat the giant generalists on that narrow task, at a fraction of the cost, and it's compact enough to run on a single server, a laptop, or a device in the field.

The utility relevance is direct. An SLM can run analytics at a substation with no cloud connection, power a document assistant that lives entirely inside your network, or handle high-volume routine work — classifying service requests, summarizing call transcripts — for pennies. Names you'll see include Microsoft's Phi, Google's Gemma, and the smaller variants of Llama and OpenAI's gpt-oss. SLMs are a big part of why on-premise AI has become economically realistic for a mid-size utility, which brings us to the deployment question.

Deployment Options: The On-Premise to Off-Premise Spectrum

Understanding where AI runs is as important as understanding the technology itself, because it determines your security posture, your cost structure, and your operational flexibility. In 2026 the choice is no longer a simple cloud-or-not decision — it's a spectrum, and the middle options are the ones most utility leaders don't know exist.

Provider Plans (Team and Enterprise)subscription-based access through a provider's platform. Your staff logs in through a web interface or desktop application and uses AI directly. This is the simplest option and is appropriate for most utility use cases. Enterprise plans include contractual guarantees about data privacy, security certifications, and administrative controls.

API and Cloud-Tenancy Accessprogrammatic access that lets your developers embed AI into the systems staff already use — your work order system, your CIS, your GIS platform. Increasingly this happens inside your own cloud environment: services like AWS Bedrock, Microsoft Azure AI Foundry, and Google Vertex AI run frontier models within your cloud tenancy, so prompts and data stay under your existing cloud security agreement, with FedRAMP-authorized configurations available. For utilities that have said "we can't send our data to an AI company," this is usually the actual answer — and most don't know it exists.

Hybrid Deploymentthe pattern emerging as the enterprise default: a small open-weight model running on your own servers handles sensitive, high-volume daily work, while a cloud frontier model handles the hardest analytical tasks. You get the economics and data control of on-premise with the capability of the frontier — but it only works with clear rules about which data is allowed to go where.

Self-Hosted / On-Premiserunning AI entirely on your own servers, up to and including fully air-gapped. This gives you complete control over your data and eliminates any dependency on external providers. It used to be impractical for all but the largest organizations; with open-weight models approaching frontier quality and hardware costs falling, it is now a genuine option — and especially relevant where NERC CIP or other requirements constrain cloud connectivity to operational systems.

Choosing a point on this spectrum comes down to four questions: how sensitive is the data, where does your OT/IT boundary sit, what volume of AI work are you running, and what technical talent do you have in-house? For a deeper treatment of deployment strategy, see our strategic guide to the AI transition.

What "Open Source" and "Open Weight" Mean in AI

In software, open source means the code is publicly available for anyone to inspect, modify, and use. In AI, the more precise term is open weight: the model's weights — the mathematical parameters that define its behavior — are publicly released for anyone to download and run, while the training data and methods usually remain private. You'll hear "open source" used loosely for these models; open weight is what's almost always meant.

For utilities making security decisions, open-weight AI has a specific advantage: you can run it entirely within your own network, so no data ever leaves your infrastructure. And the old objection has weakened considerably — the capability gap between open-weight and frontier commercial models has narrowed to the point where, for most business tasks, a well-chosen open-weight model is more than capable. OpenAI, Google, Microsoft, NVIDIA, Meta, and Mistral all now publish serious open-weight models. The tradeoff today is less about capability and more about operational burden: your infrastructure, your maintenance, your expertise.

Security and Trust

Enterprise Agreements and Data Privacy

The single most important security concept to understand is this: on enterprise plans from major AI providers, your data is not used to train the model. This is a contractual guarantee, not just a marketing promise. It means that when your staff pastes a financial model into an AI conversation, that data is processed to generate a response and then effectively discarded. It does not become part of the model's knowledge. It cannot surface in another customer's conversation.

This is where the distinction between consumer and enterprise AI matters enormously. The free version of ChatGPT, for example, may use your conversations to improve the model (unless you opt out in settings). An enterprise agreement with Anthropic, OpenAI, or Google includes explicit contractual language prohibiting this. Read the agreement. Confirm the terms. This single distinction is the foundation of safe AI use in a utility environment.

Security Certifications

SOC 2an audit framework that verifies a company's controls for data security, availability, processing integrity, confidentiality, and privacy. This is table stakes for any enterprise software vendor and the minimum certification you should expect from an AI provider.

FedRAMPthe Federal Risk and Authorization Management Program. If your utility has any federal reporting requirements, receives federal funding, or handles data subject to federal regulations, FedRAMP certification ensures the AI provider meets federal security standards.

StateRAMPsimilar to FedRAMP but designed specifically for state and local government use. This is increasingly relevant for municipal utilities and state-regulated entities that need to demonstrate compliance with government security standards.

Data Residency

Data residency refers to where your data is physically stored and processed. When you use a cloud-based AI service, your prompts and the AI's responses travel through data centers. For most utilities, the key question is whether that data stays within the United States. Major AI providers offer data residency guarantees that keep processing within specific geographic regions.

This matters particularly for utilities handling sensitive infrastructure data, customer personally identifiable information (PII), or data subject to state-specific privacy regulations. Ask your AI provider: where are the data centers that will process our queries? Is there a guaranteed data residency option?

Consumer AI vs. Enterprise AI

This distinction cannot be overstated. When an employee signs up for the free version of an AI tool using their personal email and starts pasting utility data into it, that is fundamentally different from the same employee using an enterprise-licensed AI tool with proper security controls.

Consumer AI typically has weaker (or no) contractual data protections, may use your inputs for model training, lacks administrative oversight tools, and provides no audit trail. Enterprise AI includes contractual data privacy guarantees, administrative controls for user management, usage monitoring and audit capabilities, single sign-on integration, and compliance with industry security standards. If your utility is using AI in any meaningful capacity, it should be on an enterprise plan.

Air-Gapped Deployments

For utilities with the most stringent security requirements — particularly those operating critical infrastructure — air-gapped deployment means running AI on servers that have no connection to the internet whatsoever. Data goes in, analysis comes out, and nothing ever touches an external network.

This is currently only feasible with open-source models and requires significant on-premise computing infrastructure. It's the most secure option available, but also the most expensive and technically demanding. Most utilities don't need this level of isolation, but it's important to know it exists for sensitive operational technology environments.

The Regulatory Picture

As of mid-2026, no state public utility commission has issued formal guidance on utilities' operational use of AI — but the frameworks that will shape future rules already exist. The NIST AI Risk Management Framework has become the de facto governance anchor for U.S. organizations, and federal agencies including DOE and FERC have published their own AI guidance. The practical move is to align your AI policy and governance practices with the NIST framework now, so that when regulators formalize expectations, you can demonstrate you were already there.

The Vocabulary of AI Strategy

Technology is only half the equation. The other half is organizational strategy — how you plan for, govern, and adopt AI across your utility. Here are the terms you'll encounter when building that strategy.

Digital Transformation vs. AI Transformation

Digital transformation is the broader journey of moving from paper-based, manual processes to digital systems — implementing a modern CIS, deploying AMI meters, adopting GIS platforms. AI transformation is a specific subset: using artificial intelligence to enhance or automate decision-making, analysis, and operations. Many utilities are still in the digital transformation phase, and that's okay. AI transformation generally requires a solid digital foundation to be effective — AI is most powerful when it has good data to work with.

AI Readiness Assessment

Before deploying AI, smart organizations assess their readiness. This includes evaluating data quality and availability, technical infrastructure, staff skills and culture, governance frameworks, and specific use cases where AI can deliver measurable value. An AI readiness assessment isn't a pass/fail test — it's a roadmap that tells you where you are, where you want to go, and what gaps need to be filled along the way.

AI Governance Committee

An AI governance committee is a cross-functional group responsible for overseeing your utility's AI strategy, policies, and risk management. Think of it like your capital planning committee, but for technology decisions. The committee typically includes representatives from IT, legal, operations, finance, and executive leadership. Their job is to evaluate AI use cases, approve tools and vendors, set policies for acceptable use, and monitor for risks.

Crawl-Walk-Run Adoption

This is the framework most successful utility AI adoptions follow. Crawl means starting with low-risk, internal-facing use cases — drafting internal memos, summarizing meeting notes, researching regulatory questions. Walk means expanding to higher-value applications with appropriate oversight — analyzing rate structures, preparing first drafts of regulatory filings, building financial models. Run means deploying AI in operational and customer-facing applications where it can drive significant efficiency and quality improvements.

The mistake most organizations make is trying to run before they've learned to crawl. Start small, learn what works, build institutional knowledge and trust, and then expand.

AI Policy

Every utility needs a formal AI policy — a written document that defines how AI tools can and cannot be used within the organization. At minimum, this policy should address which AI tools are approved for use, what types of data can and cannot be shared with AI systems, review and approval requirements for AI-generated work product, roles and responsibilities for AI governance, and how AI decisions will be documented and audited.

An AI policy doesn't have to be long or complicated. But it does need to exist. Without one, you're relying on individual judgment calls about data security, output quality, and appropriate use — and that's a risk no utility should accept.

Shadow AI

Shadow AI is what happens when your policy lags your people: employees using unapproved consumer AI tools — personal ChatGPT accounts, free browser extensions — for utility work, outside any security controls or oversight. It is the most common AI governance failure in 2026, and it rarely comes from bad intent. It comes from capable staff trying to be productive with tools the organization hasn't provided.

The fix is not prohibition alone. Organizations that ban AI without offering a sanctioned alternative simply push usage underground, where it's invisible. The pattern that works: provide an enterprise-licensed tool that is genuinely good, make the approved path easier than the workaround, and pair the policy with training. Prohibition without provision creates shadow AI; provision with guardrails eliminates most of it.

Change Management for AI

The biggest obstacle to successful AI adoption isn't the technology. It's the people. Staff members may fear that AI will replace their jobs, distrust AI-generated output, or simply resist changing workflows they've used for years.

Effective change management addresses these concerns head-on through clear communication about what AI will and won't replace (hint: it replaces tasks, not jobs), training that builds confidence and competence, early wins that demonstrate tangible value, and channels for feedback and continuous improvement. The utilities that succeed with AI are the ones that invest as much in change management as they do in the technology itself.

AI Champion

An AI champion is the internal advocate who drives adoption within your organization. This isn't necessarily a technologist — it's someone with credibility, enthusiasm, and enough understanding of both the technology and the business to bridge the gap between IT and operations. The champion identifies use cases, builds consensus, runs pilot projects, and evangelizes successes. Without a champion, AI initiatives tend to stall in the pilot phase.

What Comes Next

You've just built a working vocabulary for the most important technology shift your utility will face in the coming decade. You don't need to remember every term — that's what this guide is for. What matters is that you now have the conceptual framework to engage with AI conversations from a position of informed confidence rather than uncertainty.

The utilities that will thrive in the AI era won't be the ones with the fanciest technology. They'll be the ones whose leaders understood the concepts well enough to make smart, strategic decisions about adoption — to know when a vendor's promise is credible, when a security concern is real, and when the right answer is "let's start with a pilot."

At NewGen Strategies & Solutions, we help utilities navigate the intersection of technology and strategy. Whether you need an AI readiness assessment, help developing an AI policy, or a trusted advisor to evaluate vendor proposals, we'd love to hear from you.