11

In the context of Artificial Intelligence, sometimes people use the word "agent" and sometimes use the word "model" to refer to the output of the whole "AI-process". For examples: "RL agents" and "deep learning models".

Are the two words interchangeable? If not, in what case should I use "agents" instead of "models" and vice versa?

nbro
  • 42,615
  • 12
  • 119
  • 217
malioboro
  • 2,859
  • 3
  • 23
  • 47

4 Answers4

9

Agent

The other answer defines an agent as a policy (as it's defined in reinforcement learning). However, although this definition is fine for most current purposes, given that currently agents are mainly used to solve video games, in the real world, an intelligent agent will also need to have a body, which Russell and Norvig call an architecture (section 2.4 of the 3rd edition of Artificial Intelligence: A Modern Approach, page 46), which should not be confused with an architecture of a model or neural network, but it's the computing device that contains the physical sensors and actuators for the agent to sense and act on the environment, respectively. So, to be more general, the agent is defined as follows

agent = body + policy (brain)

where the policy is what Russell and Norvig call the agent program, which is an implementation of the agent function.

Alternatively, it can be defined as follows

An agent is anything that can be viewed as perceiving its environment through sensors and acting upon that environment through actuators.

This is just another definition given by Russell and Norvig, which I also report in this answer, where I describe different types of agents. Note that these definitions are equivalent. However, in the first one, we just emphasize that we need some means to "think" (brain) and some means to "behave" (body).

These definitions are quite general, so I think people should use them, although, as I said above, sometimes people refer to an agent as just the policy.

Model

In this answer, I describe what a model is or what I like to think a model is, and how it is different from a function.

In AI, a model can refer to different but somehow related concepts.

  • For example, in reinforcement learning, a model typically refers to $p(s', r \mid s, a)$, i.e. the joint probability distribution over the next state $s'$ and reward $r$, given the current state $s$ and action $a$ taken in $s$.

  • In deep learning, a model typically refers to a neural network, which can be used to compute (or model) different functions. For example, a neural network can be used to compute/represent/model a policy, so, in this case, there would be no actual difference between a model and an agent (if defined as a policy, without a body). However, conceptually, at a higher-level, these would still be different (in the same way that biological neural networks are different from the brain).

  • More generally, in machine learning, a model typically refers to a system that can be changed to compute some function. Examples of models are decision trees, neural networks, linear regression models, etc. So, as I also state in the other answer, I like to think of a model as a set of functions, so, in this sense, a model would be a hypothesis class in computational learning theory. This definition is roughly consistent with $p(s', r \mid s, a)$, which can also be thought of as a (possibly infinite) set of functions, but note that a probability distribution is not exactly a set of functions.

  • In the context of knowledge bases, a model is an assignment to the variables, which represents a "possible world". See section 7.3, page 240, of the cited book.

There are possible other uses of the word model (both in the context of AI, e.g. in the context of planning, there's often the idea of a conceptual model, which is similar to an MDP in RL, and in other areas), but the definitions given above should be more or less widely applicable in their contexts.

What is the difference between an agent and a model?

Given that there are different possible definitions of a model depending on the context, it's not easy to briefly state what the difference between the two is.

So, here's the difference in the context of RL (and you can now find out the differences in other contexts by using the different definitions): an agent can have a model of the world, which allows it to predict e.g. the reward it will receive given its current state and some action that it decides to take. The model can allow the agent to plan. In this same context, a model could also refer to the specific system (e.g. a neural network) used to compute/represent the policy of the agent, but note that people usually refer to $p(s', r \mid s, a)$ when they use the word model in RL. See this post for more details.

nbro
  • 42,615
  • 12
  • 119
  • 217
7

In game AI context:

  • An Agent is a player that plays the game. Basically, its a function that gets the current state of the game and returns the next action.
  • A Model is a representation of the game.

For example, I have made a Gin-Rummy game + AI-agents. One aspect in the model was the representation of the deck as a $4*13$ matrix, where each entry in the matrix is the card status (location, whether the card has been seen by opponent).

One can model the same game in different ways, most times there is a tradeoff between representability and simplicity.

Franck Dernoncourt
  • 3,473
  • 2
  • 21
  • 39
Cohensius
  • 423
  • 3
  • 15
2

Agents act

The key property of agents is that they act. Quoting one possible definition (Russel&Norvig, AI: A modern approach), "An agent is anything that can be viewed as perceiving its environment through sensors and acting upon that environment through actuators." The environment can be the physical world around us, or some computer system, or a simulation, however, the key part of an agent that distinguishes it from non-agents is that it acts to affect that environment - as opposed to merely observing it and/or making some calculations.

Within the field of artificial intelligence there are many contexts where various models are used to decide upon a course of action. For example, reinforcement learning, or world models that are used in planning systems. In this case the whole system would be an agent, but the model would be just a part of that system.

However, that does not apply for all contexts - there are systems that are used for making decisions not related to any action (e.g. an OCR system recognizing particular letters), and there are systems that make a decision about some action, but are not intended to actually act on that decision (e.g. they show the proposed decision to a human which might act on that information). Such systems may contain models but in this case also the whole system should not be called an agent, because the actual act of acting is out of scope for that system.

Peteris
  • 883
  • 5
  • 8
-1

Andrew Ng differentiates agents from non-agents as follows:

enter image description here

Example:

enter image description here

Typical agentic design patterns:

enter image description here

And here are a bit more details on each of these agentic design patterns:

enter image description here

enter image description here

enter image description here

enter image description here

enter image description here

Source: https://youtu.be/KrRD7r7y7NY


Another definition from https://www.anthropic.com/research/building-effective-agents (thanks Carl Dockhorn for the pointer to it):

"Agent" can be defined in several ways. Some customers define agents as fully autonomous systems that operate independently over extended periods, using various tools to accomplish complex tasks. Others use the term to describe more prescriptive implementations that follow predefined workflows. At Anthropic, we categorize all these variations as agentic systems, but draw an important architectural distinction between workflows and agents:

  • Workflows are systems where LLMs and tools are orchestrated through predefined code paths.

  • Agents, on the other hand, are systems where LLMs dynamically direct their own processes and tool usage, maintaining control over how they accomplish tasks.


Another definition from https://www.kaggle.com/whitepaper-agents:

In its most fundamental form, a Generative AI agent can be defined as an application that attempts to achieve a goal by observing the world and acting upon it using the tools that it has at its disposal. Agents are autonomous and can act independently of human intervention, especially when provided with proper goals or objectives they are meant to achieve. Agents can also be proactive in their approach to reaching their goals. Even in the absence of explicit instruction sets from a human, an agent can reason about what it should do next to achieve its ultimate goal.


Another definition from Peter Gostev:

enter image description here

Franck Dernoncourt
  • 3,473
  • 2
  • 21
  • 39