Monday morning. An AI receives the task: “Analyze all customer complaints from the past week, check if the most common issues appear in the system logs, and write a summary for management.” Two hours later, the report is ready—structured, complete, with source references from the ticketing system.

No human clicked, copied, or monitored anything.
This is Agentic AI—and many are amazed as if it were magic. It’s not magic. It’s three concrete capabilities that modern Large Language Models (LLMs) now possess. Understanding these fundamentals reveals why the current hype is technically justified—and where the limits lie.

What an LLM Really Is—and How It Relates to Agency

At its core, an LLM is a machine that, based on a vast training dataset, learns which word (or more precisely, which “token”) is most likely to follow in a given context. From this seemingly simple mechanism—given sufficient model size and training data—emergent abilities arise that no one explicitly programmed: reasoning, planning, reflection.

These are called emergent capabilities—properties that “appear” beyond a certain threshold without being directly trained.

What was missing until recently: hands. The model could think—but it couldn’t act.

The Three Pillars of Agentic AI

Agentic AI emerges when an LLM is equipped with three capabilities: tools, planning, and memory. None of these alone makes an agent. Together, they create a system that can autonomously solve multi-step tasks.

Pillar 1: Tools – The Agent’s Hands

Tool use (also called “function calling”) is the technical mechanism that transforms LLMs from chatbots into agents. The principle is surprisingly simple and can be broken down into three questions:

How does the model know which tools are available?
How does it select the right tool for a task?
How does it use the tool correctly?

Early tools were simple, like internet search via search engines. But as LLMs proved adept at writing code, it became clear they could also manipulate files and other data sources. The combination has proven more powerful than initially expected.

The Menu: How the LLM Knows Its Tools

At the start of a task, the LLM is provided with a list of available tools—not as a technical API documentation, but as readable descriptions in natural language. Each tool gets a name, an explanation of what it’s for, and a description of the expected parameters. Conceptually, it might look like this:

TOOL NAME	DESCRIPTION	PARAMETERS
lade_tickets	Loads support tickets from the CRM system	Time period, type (complaint/request), status
search_logs	Searches system logs for a keyword	Search term, time period, log level
send_email	Sends an email via the company’s mailing list	Recipient, subject, content
create_document	Formats and saves a document	Title, content, storage location

The LLM reads these descriptions like a menu—and decides, based on the task and tool descriptions, whether a tool fits the current situation. The quality of these descriptions is critical: a vaguely described tool will either be misused or ignored entirely.

The Decision: How the LLM Chooses the Right Tool

The model doesn’t select tools through separate logic or a rule system—it’s pure language understanding. The LLM simultaneously reads the task and the tool descriptions, then writes a plan to decide which tool best fits the situation. It considers:

Does the tool match the task? → “I need current data” → No reliance on training knowledge, but a tool call.
What parameters does the tool require? → The model extracts needed values from the task description or prior context.
Is the tool even useful right now? → The model may decide not to call a tool and respond directly if it already has all necessary information.

Important: The model can select the right tool from a list of ten or twenty—but it only chooses from the tools explicitly provided to it.

The Execution: How the LLM Activates a Tool

When the model wants to use a tool, it doesn’t return a response in natural language. Instead, it generates a structured request: the tool name and parameters in a machine-readable format. Conceptually:

Tool = load_tickets

Parameters = Time period = “last 7 days”

Type = “complaint”

Status = “open”

The surrounding application receives this call, executes it (e.g., queries the CRM system), and returns the result to the LLM. The model processes the result and decides:

Is this enough?
Do I need another tool?
Can I now complete the task?

This cycle—task → tool call → result → next decision—can repeat multiple times until the task is fully resolved.

A critical detail for security-conscious readers: The LLM does not execute tools itself. It writes a structured call request—the surrounding application executes it, deciding whether and how to fulfill the request. This isn’t a technical limitation but a deliberate architecture: it ensures full control over which actions are possible, which systems are accessible, and what permissions are used.

Pillar 2: Planning – The Agent’s Thinking

Tool use alone isn’t enough. A complex task consists of multiple steps that must be executed in the correct order, depend on each other, and influence one another. That’s where planning comes in.

Modern LLMs can improve measurably on complex tasks through a simple technique: explicit “step-by-step thinking” (also called Chain of Thought). Instead of just asking for the answer, the model is asked for the path to the answer. This sounds trivial but, in benchmarks, doubles the quality for multi-step problems.

The most important architectural pattern for agents is the ReAct pattern (Reasoning + Acting). It describes a thought-action-observation loop:

Thought: What do I need next?

Action: [Tool call with parameters]

Observation: [Result of the tool]

Thought: What does this result mean? What’s the next step?

Thought: What do I need next?

Action: [Tool call with parameters]

Observation: [Result of the tool]

Thought: What does this result mean? What’s the next step?

A concrete example from our opening scenario:

Thought: I need the customer complaints from the past week.

Action: [Tool: load_tickets, Time period: last 7 days, Type: complaint]

Observation: [247 tickets, most common category: “Login issues” (38%)]

Thought: Login issues are the most common category.

I’ll check if there are entries in the system logs.

Action: [Tool: search_logs, Search term: “authentication error”, Time period: 7 days]

Observation: [3,240 error entries—clear spike starting Tuesday]

Thought: Connection found. I can now write the report.

Action: [Tool: create_document, Content: …]

The agent “thinks aloud”—and this isn’t just for show. It’s technically essential because the model can explicitly document intermediate steps and build upon them, rather than generating a (poorly thought-out) answer directly for a complex problem.

Pillar 3: Memory – The Agent’s Knowledge

LLMs do not have persistent memory by default. Once a session ends, everything is forgotten. Agentic systems therefore build artificial memory at different levels.

Working Memory (Context Window)

The context window is the agent’s short-term memory: everything the model sees in the current “task”—the original prompt, all prior tool results, and intermediate thoughts. The larger this window, the more complex tasks an agent can theoretically handle. Top models today offer context windows of hundreds of thousands to over a million tokens—enough for entire books. Three years ago, this was unthinkable.

Long-Term Memory (Retrieval-Augmented Generation & Beyond)

Through databases (RAG), file storage, or other systems, agents can access knowledge bases far larger than their working memory: internal documents, manuals, historical project data. The agent searches for what it needs just in time, much like a human who doesn’t memorize everything but knows where to look. Beyond knowledge, these systems can also store processing rules.

Conclusio: Why Is This Only Possible Now?

This isn’t a given. Just three to four years ago, the same mechanisms would have mostly failed with weaker models. What changed?

Scaling: More parameters, more training data—beyond a certain threshold, emergent abilities like reliable instruction-following emerge.
RLHF (Reinforcement Learning from Human Feedback): Models were trained via human feedback to prioritize useful and precise responses, improving reliability in tool use.
Tool-Use Training: Top models today are explicitly trained to reliably generate structured calls and process results correctly.
Larger Context Windows: Only with sufficient working memory can multi-step agent tasks be meaningfully executed.

The breakthrough wasn’t a single moment. It was the gradual crossing of multiple thresholds simultaneously—in model size, training quality, and context length.