Context Awareness and Context Recognition in Modern Decision-Making - 1

 

Section One: Modern Decision-Making

Introduction

Today's decision-making process takes place in an increasingly dynamic and complex environment. Today’s decision-making domains include not only the traditional physical environments but also digital age domains such as cyber and information. The speed at which the environment changes has increased and decision-makers require new ways to adapt to changing contexts in which they create change and fulfill intents.

This report presents the results of a scoping study in which the changing dynamics of context recognition and awareness are mapped and a decision-making model based on this mapping is outlined. It is divided into four main sections.

• The first section, which you are reading, surveys the concepts of context awareness and context recognition, outlines theories of modern decision-making, and concludes with a discussion of situation recognition and awareness.
• In the second section, context recognition is described as a process of representing complex environments as characteristic schemas, frames or models, and identifies the roles these abstractions play in decision-making in terms of data, information and capability.
• The third section presents research results based on both a literature review and interviews with Canadian Forces command personnel. A circular graph-based task model is proposed depicting the multiple simultaneous influences of task domains on each other, as an effect of context, and as a basis for context recognition.
• Finally, the fourth section suggests avenues of future research that may expand our understanding of context recognition and identify how the use of digital technologies such as artificial intelligence may assist decision-makers in context recognition and decision-making tasks.

Literature Review

This review identifies relevant materials in the federal science database, focusing on key terms within the field, in addition to related work found through expanded-search techniques on the World Wide Web. The review was conducted within the context of the pre-defined concepts of ‘command and control’, but may be applied to wider contexts.

The review followed considerations related to definitions of ‘context’ and subthemes such as ‘context awareness’ and ‘context recognition’, which depict ‘context’ as a type of generalisation that can lead to appropriate response. The review identified three major interpretations: context as schema, context as frame, and context as model.

• A schema is a semantical framework that helps individuals organise, process, and store information about their environment. It can be thought of as a semantic concept composed of two major parts: significant words or symbols interspersed with blank-space placeholders; and a method or condition specifying how the placeholders are to be filled to obtain instances.
• A frame is a psychological attitude defining a set of background assumptions and expectations. (Lakoff)
• A model is a scientific or computational construction defining an environment, process or system. Models may be deterministic, stochastic, or probabilistic. Types of model include a logic model, mental model, simulation, or multi-objective optimization.

Defining Context

Context may be defined as “any information that can be used to characterise the situation of entities” The term ‘context’ is frequently used in conjunction with terms related to capability or capacity, such as ‘context recognition’ or ‘context awareness’.

Context Awareness is the “current characterization (as described by pattern, scenario, type or template) of the situation of entities”, ie., “the ability to detect, sense, interpret, act and respond to relevant aspects of the environment, such as location, time, temperature or user identity,” where relevance is described by the current task or set of objectives, that could enable a prediction of actor intentions or future events.

Context Recognition is “recognition of a previously characterised context (as described by pattern, scenario, type or template)”, including possible interpretations, actions and responses, supporting a prediction of actor intentions or future events

To contrast context and situation, we say that a ‘situation;’ is the state of affairs in the environment relevant to a decision or an action, while by contrast we say a ‘context’ is a type of generalisation that can be inferred from the situation that is in turn associated with specific decisions or actions. For example, the situation ma be that orange and black stripes are present in a canopy of green, while the context is that you are near a tiger.

In this discussion, context is thought of as a range of possible situation classifications such that we are able to classify the situation as, eg. class 1 or class 2. There may be different context sets addressing various perspectives of context.

The Decision-Making Environment

Decision-making itself is a structured process, following a logical progression through:

• The identification and analysis of a problem;
• The development of options for solutions to the problem; and
• The translation of conceptual options into a plan that can be executed (CACSC, 2018).

Decision making is anticipated to occur in an extremely complex and interactive future environment (such that) future operating environments will require a real time, fully networked... capability” to support “integration and synchronisation of actions”. (Tucholski, 2021 5; JP 3-0, 2022). The eventual outcome will be a cloud-based decision-making environment that incorporates a large number of "relevant data feeds as well as artificial intelligence and machine learning to enable decision-makers to maintain detailed situational awareness of the environment.” (Gordon, 2023; Cliche, 2024).

Modern Decision Science

The focus of modern decision science “is on building a framework capable to offer an effective tool for decisions in the field of force planning and operations planning” (Yuan and Singer, 2021), which requires a capacity to respond to dynamic and changing environments.

Classic decision-making science approaches environments as systems to which qualitative methods such as scenario spinning, operational gaming, or Delphi techniques, may be applied (Davis et al., 2005, p. 33). However, in the face of increasingly complex environments, “Instead of seeking to “predict” effects on a system of various alternatives and then ‘optimizing’ choice, it may be far better to recognize that meaningful prediction is often just not in the cards and that we should instead be seeking strategies that are flexible, adaptive, and robust” (Ibid., p.46).

Some modern decision-making approaches found in the literature follow. Each of them highlights the role of context in decision making in a modern environment.

OODA Loop: John Boyd's observation-orientation-decision-action metaphorical decision-making cycle (or "OODA loop") is used, for example, to make fast and accurate decisions (Maccuish,2012, p. 67). “Because they’re developed and tested in the relentless laboratory of conflict, military mental models have practical applications far beyond their original context.” In the OODA model, context plays a key role in the ‘orientation’ stage.

Orientation “involves assessing the relevance and significance of the data, understanding how it fits into the larger context, and identifying potential opportunities or threats” (Wale, 2024). The OODA loop is recognizable in the Canadian Forces Operational Planning Process (OPP), which recognizes five stages: initiation, orientation, course of action development, plan development, and plan review (CACSC, 2018, pp. 11-16). The OPP is informed by descriptions of other actors, terrain, structures, capabilities, organizations, people, and events. (ASCOPE) (CACSC, 2018, p. 18).

Intent Model: David Marquet's intent-based leadership (IBL) model “is not based on the flow of power from one individual to another as in the leader-follower model, but is instead based on a goal, or intent, shared between individuals. By analogy, the leader-follower model is similar to command and control, but the IBL model is similar to mission command” (Fernandez-Salvador, 2017).

While IBL is most often discussed from a leadership perspective, as a training model it develops a learner’s sense of context. “With IBL, learners gain experience in making sense of a problem. As they develop the solution to a problem, the problem begins to make sense, and learners begin to problem solve and adapt” (Duffy and Raymer, 2010, p.v).

Joint Decision-Making: Complex environments often require multiple organizations and branches and hence entail joint decision-making. The joint decision-making model builds on the OODA to create constructs like joint operational planning and joint information and intelliggence preparation in order to enable a systems understanding of an information environment (Sylvestre, 2022, p. 14).

Robust Decision-Making (RDM): Robust decision making (RDM) “is a quantitative, decision support methodology designed to inform decisions under conditions of deep uncertainty and complexity (to) help defense planners make plans more robust to a wide range of hard-to-predict futures” (Lempert et al., 2016, p. 2).

In contrast to “agree-on-assumptions” (Kalra et al. 2014) or “predict-then-act” approaches to decision-making, RDM takes an “‘agree-on-decisions’ approach, which inverts these steps,” using “models and data to stress test the strategies over a wide range of plausible paths into the future.”

Decision Making under Deep Uncertainty (DMDU): In cases of deep uncertainty there is not agreement on how the system works nor what future outcomes may be. Accordingly, as (Kwakkel and Haasnoot, 2019, p. 357) argue, various representations or models may apply. Scenario thinking, exploratory modelling and adaptive thinking are methods of preparing for alternative situation types. DMDU proposes a taxonomy of such methods (of which RDM, above, is one) that apply in different cases.

Situation Recognition

Modern decision theory requires situation awareness (SA) in order to comprehend which, if any, representation or model may apply. “Determining exactly what constitutes SA is a very difficult task, given the complexity of the construct itself, and the many different processes involved with its acquisition and maintenance” (Banbury and Trembley, 2004, p. Xiii). Moreover, "...models of SA refer to cognitive processes in general terms, but do not specify exactly what processes are involved and to what extent" (Ibid).

An understanding of the mechanisms of arriving at situation awareness, here called ‘situation recognition’, is required. "The test of situation awareness as a construct will be in its ability to be operationalized in terms of objective, clearly specified independent (stimulus manipulation) and dependent (response difference) variables ... Otherwise, SA will be yet another buzzword to cloak scientists' ignorance” (Flach, 1995, p. 155).

In computer vision, “‘situation recognition’ is the task of recognizing the activity happening in an image, the actors and objects involved in this activity, and the roles they play. Semantic roles describe how objects in the image participate in the activity described by the verb” (Pratt et al., 2020, p.2).

This involves (per Wikipedia):

• “the perception of the elements in the environment within a volume of time and space, the comprehension of their meaning, and the projection of their status in the near future”. Endsley (1995b).
• An alternative definition is that situation awareness is adaptive, externally-directed consciousness that has as its products knowledge about a dynamic task environment and directed action within that environment.

According to Endsley (Ibid.) expert decision-makers act first to classify and understand a situation, then proceed to action selection, for example, matching to prototypical situations in memory: Dreyfus (1981) experts, deGroot (1965) chess, Mintzberg (1973) management, Kuhn (1970) science. (p 34). This process comprehends three major approaches developed in the years following Endsley’s work.

• Schemata, which are coherent frameworks for understanding information (information is lost but becomes more coherent and understandable). Scripts (a special type of schema) provide sequences of appropriate actions
• Mental models, which may be “mechanisms whereby humans are able to generate descriptions of system purpose and form” (Rouse and Morris, 1985, p. 356). Experts shift from representation to abstract codes (Ibid), for example, the situation model “provides a mechanism for the single-step ‘recognition primed’ decision-making” (Endsley, 1995b, p. 43)
• Development, whereby schemata and mental models are developed as a function of training and experience in a given environment - Holland, Holyoak, Nisbett and Thagard (1986).

As seen through the examples below, more recent models are based on graph analysis. “Existing situational awareness systems use prebuilt situational knowledge-based symbolic reasoning, making it very difficult to infer situational knowledge building or unexpected situations in complex, time-space dynamic environments such as battlefields” (Lee et al., 2023, p. 6057).

Examples of Situation Recognition Models

Following are a few examples of situation models drawn from the literature:

• Grounded Situation Recognition, developed by AllenAI, is “a task that requires producing structured semantic summaries of images describing: the primary activity, entities engaged in the activity with their roles (e.g. agent, tool), and bounding-box groundings of entities” (Pratt et. al., 2020, p. 1).
• Chmielewski and Sobolewski, (2019, p. 38) describe situation recognition as a “consistent data flow process, consisting of: data generation, data integration and filtration, data visualization, and knowledge acquisition and reasoning.”
• A situation awareness model with human-machine collaboration proposed by Meng et al. (2022, p. 1443) is comprised of a human cognitive part that includes situation perception and a machine part machine including situation recognition, where situation recognition “mainly corresponds to human's situation perception part, which is an intuitive analysis of the current situation from objective data in reconnaissance intelligence, trend intelligence and other data (p. 1444).
• Baek et al. (2022, p. 308) describe a distributed graph matching network “to classify multiple agents based on their graph semantic information” and a “hypergraph to analyze high-order relationship between agents.”
• Lee et al. (2023, p. 6041) describe an architecture based on multi‑modal data and graph neural network (p. 6042) comprising four key parts: multi-agent based manned-unmanned collaboration architecture, robust tactical map fusion technology, hypergraph based representation learning, and space-time multi layer model. “The proposed model provides collaborative intelligence-based real-time battlefield situation recognition technologies” (p. 6066).

Summary

Modern decision-making has evolved from a simple process, described by the OODA loop, to a complex process that involves the development and application of models based on prior knowledge, scenario building, and situation recognition.

This has shifted the emphasis in decision-making from being one in which being informed and aware is sufficient to one in which considerable pre-planning, including especially model-building, is required. The task of developing and applying such models is often a joint one, involving as well the development of collaborative processes and information networks.

--

Image source: Strategy Punk https://www.strategypunk.com/navigating-the-ooda-loop-mastering-adaptive-decision-making-in-complex-situations/

This article is based on work completed for Defence Research and Development Canada, Contract Report DRDC-RDDC-2025-C035

References will be listed after the series is complete.