Soar

Figure 6-12 shows the Soar cognitive architecture. Like much of artificial intelligence research, the Soar architecture is based on the idea everything can be represented as symbols and intelligence is the result of symbol manipulation. This is called the Physical Symbol System Hypothesis (PSSH) (Newell and Simon, 1976). Allen Newell was one of the founders of artificial intelligence and the PSSH. As John Laird's and Paul Rosenbloom's dissertation advisor, Newell, Laird, Rosenbloom developed the first version of Soar in the early 1980s (Laird and Newell, 1987). Laird is now the lead of Soar which has been in continual development for over 30 years (Laird, 2012).

Held in long-term memory, Soar maintains a set of procedural production rules (how to do things), a set of semantic representations (knowledge about things), and a set of episodic memory representations (snapshots of short-term working memory). These represent the agent's knowledge (К in the formal models).

Soar also features a learning module for each kind of long-term memory store. Reinforcement learning supports procedural memory, semantic learning supports semantic memory, and episodic learning supports episodic memory. This layer in Soar corresponds to the learn function in the formal models.

Fig. 6-12: The soar cognitive architecture.

Like other humanistic models, including EPIC and ACT-R described above, Soar makes a distinction between long-term memory and shortterm, or working, memory. Information, knowledge, and production rules are brought into short-term memory when needed. While Soar does not provide detail for sensory and effector modules, the agent's perception brings information into short-term memory to provide the agent with information about the environment. Short-term memory is where the production rules are executed using the information and knowledge to reason about the world. The agent's actions are performed based on information from short-term memory. The decision and appraisal functions support reasoning as well.

At any point in time, short-term memory contains all information of importance at that particular time. Snapshots of working memory are stored in a temporal fashion in episodic memory. At any later time, prior episodes can be retrieved into working memory through. This is how Soar agents are able to recall past experiences. Being able to retrieve knowledge and past experience is key to being an expert as described in later chapters.

Soar/Formal Model

Here, we combine Soar and the formal models. Soar is the most highly- regarded cognitive architecture developed over a period of more than 30 years (Laird, 2012). The formal models of agents presented earlier in the chapter were developed by artificial intelligence researchers over several decades as well but were developed from a different perspective (Russell and Norvig, 2009; Genesereth and Nillson, 1987). However, the two fundamentally represent the same thing—cognition. Here, we combine the Soar architecture with elements from the formal agent models two as shown in Fig. 6-13. The fit is not perfect but one can see how they complement one another.

The fundamental and classic perceive-reason-act-learn structure is present. All agents are able to perceive a subset of the environment and all agents can perform actions to alter the environment in some way. The formal agent models do not attempt to distinguish long-term from short-term memory. The humanistic models do because these structures appear in humans. The formal models name specific stores such as goals, models, and utility values whereas Soar does not. Soar identifies kinds of

Fig. 6-13: Soar/Formal model combination.

knowledge stores, procedural, semantic, and episodic because it is intended to explain human cognition. Any formal agent model can be reconciled with Soar by placing various types of knowledge long-term memory. The agent retrieves information and knowledge from these stores and brings them into short-term memory as it does its processing.

Soar also differentiates three different kinds of learning, again, because these kinds of learning are present in human cognition. The formal models do not differentiate and have only a generic learning element. However, the formal models could be extended with such detail as desired.

Standard Model of the Mind

The latest work from the "humanistic models" researchers, Laird, Rosenbloom, and Lebiere, is the Standard Model of the Mind (SMM) shown in Fig. 6-14 (Laird et al., 2017). SMM is seen as a coming together of Soar, Act-R, and several other humanistic cognitive architectures. SMM views the human mind as a collection of independent modules having distinct functionalities and exhibits the classic perceive-reason-act structure. Knowledge is broken up into declarative long-term memory and procedural long-term memory. Declarative memory contains general knowledge about things and procedural memory contains knowledge about how to do things.

SMM is based on symbol representation of knowledge and relations over those symbols. Items stored in memory also include metadata (data about data) such as frequency, recency, co-occurrence, similarity, and

utility information. This permits statistical treatment of knowledge and a form of episodic memory.

As in EPIC, ACT-R, and Soar, working memory in SMM is where dynamic symbol manipulation occurs. Items from perception and from long-term memory are brought into working memory where inferencing occurs.

Learning in SMM involves the automatic creation of new symbol structures, plus the tuning of metadata, in long-term memory. SMM assumes all types of long-term knowledge are learnable.

Subsumption Architecture

The next three cognitive architectures form the "subsumptive group." These architectures are interesting because they are based on multiple layers, or levels, which "execute" in parallel. Each successive layer constitutes a higher-level behavior. Overall behavior emerges from the individual actions at each layer.

As shown in Fig. 6-15, the subsumption architecture was created in the 1980s by Rodney Brooks as a reaction to the failures of traditional symbol-based cognitive architectures (the humanistic models described earlier) to achieve success in real-world environments (Brooks, 1986). Systems were "brittle" meaning they could not perform well when exposed to incomplete and sometimes contradictory information in the real-world. Unlike EPIC, ACT-R, CLARION, Soar, and SMM, there is no symbolic representation in the subsumption architecture. Neither is there internal storage of knowledge. Subsumptive agents are not able to store

general knowledge, procedural knowledge, episodic knowledge, nor are they capable of forming models and maintaining goals or utility values. Instead, each layer of behavior is autonomous (although there is feedback between the layers). All behavioral layers operate at the same time and a behavior can subsume (override) another behavior.

The architecture was applied to robotics. For example, an autonomous robot could move around an environment and avoid obstacles. At this layer, all the agent does is perceive the environment and act so as to avoid collisions. However, at a higher level, the agent is also wandering. Wandering implies a non-random movement. However, while wandering around a room, the agent also avoids collisions. For example, if the agent is wandering in a general northernly direction, it may deviate temporarily to avoid a collision and resume the northernly path after the avoidance maneuver is completed. Higher-level behaviors are explore and seek. However, as the agent performs these higher-level behaviors it still wanders and avoids.

This architecture is based on the Creature Hypothesis because it mimics insect and animal behavior as opposed to the Physical Symbol System Hypothesis common to the humanistic models. Subsumptive behavior yielded demonstrations of robustness. Lab-scale systems exhibited the ability to maintain overall behavior in the presence of unexpected situations and dynamic environments. However, the subsumptive architectures were not able to evolve complex actions such as high-level cognition, reasoning, and learning. It seems both subsumptive and symbolic approaches are needed to create a complete model of intelligence. Over the last twenty years, several other interesting subsumptive architectures have been created.

Emotion Machine—Model 6

Marvin Minsky created the Society of the Mind architecture in the 1980s, about the same time Brooks was creating the subsumption architecture described above (Minsky, 1986). Similar to the subsumption architecture, Minsky models different levels of thinking as layers. The Emotion Machine architecture shown in Fig. 6-16 is Minsky's latest version of this type of architecture (Minsky, 2007).

Humans possess instinctive reactions to perceptions (e.g., hunger, fear) but also learn reactions through experience (e.g., don't step in front of a car). At a higher level, the deliberative thinking level, humans can reason about their perceptions and knowledge and make informed decisions. Humans also analyze their own thinking (reflective thinking) and constantly examine what-if scenarios to improve future deliberative decisions. At higher levels, humans think about how their perceptions, actions, and reasoning

Fig. 6-16: The emotion machine—Model 6.

affects them personally. At the self-reflective level (the personal level), humans evaluate themselves against their beliefs, morals, and goals. At the self-conscious level (the social level), humans evaluate themselves with respect to others and society in general (e.g., what do others think about my behavior?).

None of these levels are possible without common sense and learning. This is called the "Common Sense Hypothesis." Recall, the humanistic models all held general and common sense knowledge in long-term memory stores. The same is presumably true for the Emotion Machine but details are not provided in the architecture. However, as noted earlier, the problem of collecting enough common sense knowledge to be effective is an unsolved problem in artificial intelligence. Projects such a Cyc (Lenat and Guha, 1989), Open Mind Common Sense and ConceptNet (Havasi et al., 2007), and various projects at the Allen Institute for Artificial Intelligence (Allen Institute, 2019) have sought to capture volumes of common-sense knowledge and bring common sense reasoning to reality.

Importantly, the Emotion Machine distinguishes different levels of human thinking and is ultimately a model of the human mind. A still unanswered question is whether or not artificial systems need all of these levels of thinking to be effective. This book involves synthetic expertise— systems capable of expert-level cognition in a given domain—so we expect an artificial expert to possess some of the same levels of thinking as a human expert.

Genesis Architecture

The Genesis architecture shown in Fig. 6-17 is Patrick Winston's latest subsumptive/ level-based description of interaction between intelligent agents or beings. The basis of this architecture is the "Strong Story Hypothesis" maintaining intelligence is the ability to tell stories and understand them (Winston, 2011). With humans, language is the mediator of perceiving the world and understanding it. Humans perceive the world and use language to describe it (general and perceptual knowledge) and partition it and time into events (episodic memory). Sequences of events constitute a story. Stories collected at various hierarchical levels such as the family/personal level (microlevel) and the state/country level (macro) constitute culture.

Fig. 6-17: The genesis architecture.

The architecture as shown has the perceive function represented but does not represent the effector function of an agent. Such details are not the purpose of the architecture but presumably act functions would either be done at the same level as the perception function. Like the Emotion Machine, the Genesis architecture is of interest to this book because synthetic experts will have to interact naturally with humans. We expect synthetic experts to have the capability of multi-level thinking like that shown in the Emotion Machine architecture and also be able to converse with humans on multiple levels as shown in the Genesis architecture.