Goal-Based/Knowledge-Level Agent (Reasoning Agent)

As shown in Fig. 6-5, a goal-based/knowledge-level agent maintains a store of general knowledge, K. The agent also maintains a set of models, M, the internal representations, and G, a set of goals the agent is trying to achieve (Russell and Norvig, 2009; Genesereth and Nillson, 1987). The agent uses the knowledge, models, and goals to reason about the next action to perform.

The set of actions the agent can perform is A. Each action, when taken, (do function) causes the environment to change to another state (A x S —> S). During each cycle, the environment exists in a state (s <= S) however

Fig. 6-5: A goal-based/knowledge-level agent.

the agent can detect only one of a set of partitions (groups of states) in S called T. The agent perceives the environment (see function) and detects which partition (tcT) the environment is in (S —» T). The agent modifies its internal representation (model function) based on its new perceptions, its knowledge of the world, and its goals (M x G x К x T —> M). Having perceived the environment and updated its internal model, the agent selects an action (a <= A) (action function) using the internal models, goals, knowledge, and its perceptions (M x G x К x T —> A). The agent then performs the selected action (do function) causing the environment to transition to a new state (A x S —* S).

Here, general knowledge, K, and goals, G, are logically separated from the models, M. One could use the same data structure to represent all three if desired, but this is not necessarily the case in every implementation. In more sophisticated models, discussed later, different kinds of knowledge will be separated out into independent structures. It is important to note this type of agent is not able to modify its knowledge or goals (it is not a learning agent), but it can modify/update its set of models by incorporating information from recent perceptions.

In general, M captures representations about the dynamic nature of the environment while К captures static, always-true, concepts. For example, К might contain the knowledge "unsupported objects will fall downward in the presence of a gravitational field." This piece of information is true no matter where objects in the environment are positioned so this is a static piece of knowledge about the world. However, objects can move around in the environment, so M is used to capture location of objects in real-time as the agent is able to perceive their locations and positions. If the agent perceives an object moved to a position where it is no longer supported, combining that perception with К allows the agent to conclude the object will begin to fall downward. The fact the object is going to fall downward is a new piece of knowledge about the environment and allows the agent to choose actions accordingly. For example, if that situation is undesirable, the agent could choose an action to mitigate the object from hitting the floor.

The more general information (common sense knowledge) an agent knows about itself, its tasks, and the environment, the better it can function in real world scenarios. How many pieces of information does an agent need in К to be able to handle all of the uncertainty of operating in the real world? This is a famous unsolved problem in artificial intelligence research. No one knows how many pieces of information are required and no one knows if it is even possible to capture all relevant knowledge to facilitate common sense reasoning. Projects such a Cyc (Lenat and Guha, 1989), Open Mind Common Sense and ConceptNet (Havasi et al., 2007), and various projects at the Allen Institute for Artificial Intelligence (Allen Institute, 2019) are seeking to capture volumes of common sense knowledge and bring common sense reasoning to reality, but this is still a work in progress.

An agent possessing substantive К is much more robust than an agent without K. For example, in the example given above, just one piece of knowledge allows the agent to recognize the problem of an object falling because of the location of an object. More pieces of knowledge might have enabled the agent to prevent the object from being placed in a precarious position in the first place. Still more knowledge might have avoided the entire situation by alternative action.

Utility-Based/Knowledge-Level Agent (Rational Agent)

As shown in Fig. 6-6, a utility-based/knowledge-level agent, also called a rational agent, maintains a store of general knowledge, K, a set of models, M (internal representations), a set of goals the agent is trying to achieve, G, and a set of utility values, U. The agent uses the knowledge, models, and goals to reason about the next action to perform (Russell and Norvig, 2009; Genesereth and Nillson, 1987).

Fig. 6-6: A utility-based agent.

The set of actions the agent can perform is A. Each action, when taken (do function) causes the environment to change to another state (AxS-> S). During each cycle, the environment exists in a state (s <= S) however the agent can detect only one of a set of partitions (groups of states) in S called T. The agent perceives the environment (see function) and detects which partition (t <= T) the environment is in (S —> T). The agent modifies its internal representation (model function) based on its new perceptions, its knowledge of the world, its goals, and its utility values (M xGxKxlJxT-* M). Having perceived the environment and updated its internal model, the agent selects an action (a <= A) (action function) using the internal models, goals, knowledge, utility values, and its perceptions (M x G x К x U x T —> A). The agent then performs the selected action (do function) causing the environment to transition to a new state (AxS-> S).

The utility values allow priorities to be assigned to items, especially goals and actions. Utility values allow the agent to choose a preferred course of action when it must decide between two or more alternatives. For example, consider two goals with "take pictures of animals" has a lower priority than "collect samples of water." If the agent then finds itself in a situation where it must decide what to do next and both taking pictures and collecting samples are possible next actions, the agent will collect the water sample first.

Utility values also affects the agent's model building performance. For example, if the agent's task is to photograph birds, then bird-objects will have a higher priority than other objects. When the agent perceives a bird, it will record information about the bird into its models. However, if the agent perceives an animal different than a bird, it can decide to ignore it and not update its models with non-bird information.

This type of agent is called a rational agent because incorporating utility values into decision processes allows the agent to make decisions yielding the best possible overall performance. When an agent takes the optimal course of action its behavior is considered rational.

Note this agent is not able to change its knowledge, K, goals, G, or utility values, U. That is not to say these values could not be changed by other entities. For example, while the agent is operating, a human operator could assign different utility values thereby altering the agent's behavior. Likewise, another entity could transmit new knowledge to the agent. Once the new knowledge is in К it can be used in all subsequent reasoning and affect the agent's behavior. The next two architectures allow the agent to change its own knowledge, goals, and utility values thereby making them far more autonomous.

General Learning Agent (Adaptive Agent)

As shown in Fig. 6-7, the set of actions the agent can perform is A. Each action, when taken, (do function) causes the environment to change to another state (A x S —> S). During each cycle, the environment exists in a state (s <= S) however the agent can detect only one of a set of partitions (groups of states) in S called T. The agent perceives the environment (see function) and detects which partition (t <= T) the environment is in (S —> T). The agent modifies its internal representation (model function) based on its new perceptions, its knowledge of the world, its goals, and its utility values (Mx G xKxUxT—>M). Having perceived the environment and updated its internal model, the agent selects an action (a <= A) (action function) using the internal models, goals, knowledge, utility values, and its perceptions (M x G x К x U x T —> A). The agent then performs the selected action (do function) causing the environment to transition to a new state (AxS-> S). The agent's knowledge, K, can be modified by the agent (learn function). Pieces of knowledge can be added, removed, or modified (MxGxKxlfxT-> Ю- The agent's goals, models, and utility values affect learning.

Fig. 6-7: A general learning agent.

The difference between this architecture and the rational agent is the learn function. Learning allows the agent's knowledge to evolve over time. Many kinds of machine learning have been developed over the decades in artificial intelligence research but this architecture does not distinguish different types of learning. All forms of learning result in the modification of the knowledge store, K.

Allowing the agent's knowledge to change over time affects behavior of the agent since К is a factor in the model and action function as well. For example, new or modified knowledge can change the way the agent builds its models, M. Imagine a bird-photographing agent. The agent maintains many models of birds since it is important for the agent to realize a bird is there to be photographed. However, while photographing birds, imagine the agent learns a new piece of knowledge "bears are dangerous" possibly from observing a bear attacking an animal. This new knowledge may cause the agent to create a new model ("bear danger") and start collecting perceived information related to bears. Now, the agent is maintaining bird models and also the bear danger model.

New knowledge can also affect the agent's selection of the next action to perform. Previously, the agent, when sensing a nearby bear, would have ignored the bear. However, with the new knowledge about bears being dangerous, the agent may choose the "flee" or "avoid" actions instead of ignoring the bear.

This is an example of how learning agents adapt to the environment. The ability to adapt is a key skill in real world scenarios where all possible situations cannot be predicted. However, this architecture is not able to autonomously update utility values and goals. Those abilities are found in the next architecture.