All about Safety and Situational Awareness (Situational Awareness and…
All about Safety and Situational Awareness
Situational Awareness and Safety
by Stanton et al
History of the concept
The concept of situational awareness was identified during World War I by Oswal Boelke who realized "
the importance of gaining an awareness of the enemy before the enemy gained a similar awareness, and devised methods for accomplishing this
This idea of separation between the human operators understanding of system status and actual system status is at the crux of the definition of situational awareness.
This concept is getting more important because of the increased realisation that system design is no longer optimised for human operation and, under some condiitons, has overstepped the human's capability to keep track.
In order for people to maintain an adequate awareness of the system status, they need to track the development of events as they gradually unfold.
Definition of situational awareness
An appropriate awareness of a situation
Situational awareness is the perception of the elements in the environment within a volume of time and space, the comprehension of their meaning and a projection of their status in the near future
Situational awareness is the conscious dynamic reflection on the situation by an individual. It provides dyanimc orientation to the situation, the opportunity to reflect not only the past, present, and future, but the potential features of the situation. The dynamic reflection contains logical-conceptual, imaginative, conscious, and unconscious components which enables individuals to develop mental models of external events
(Bedny and Meister, 1999)
Situational awareness is the invariant in the agent-environment system that generates the momentary knowledge and behavior required to attain the goals specified by an arbiter of performance in the environment
(Smith and Hancock, 1995)
Elements of situational awareness
Reflection and projection
Knowledge and mental models
Perception and the representation
Things in the world
Theories of situational awareness
Perception of the elements in the current situation.
No interpretation of the data is performed at this stage, all it is intended to represent is the initial receipt of information in its raw form.
Comprehension of the current situation.
Comprehension is essential to understand the significance of the elements and to gain a picture of what is going on.
Projection of future status of the situation.
This is the highest level of situational awareness and is associated with the ability to project the future of the elements in the environment.
- Interpretation of information from world.
- Conceptual ‘image’ of information–task–goal.
- Dynamic reflection of situation and task.
- Comparing motivation and performance.
- Interacting with the world.
- Determining relevant criteria for evaluation.
- Modify experience to interpret new information.
- Modify world model to interpret new information.
The Perceptual Cycle
refers to the perceptual and cognitive activities
involved in revising the state of situational awareness.
refers to the state of situational awareness with regard to available information and knowledge.
Human thought is closely coupled with a persons interaction with the world.
Knowledge of how the world works (e.g. mental models) leads to the anticipation of certain kinds of information, which in turn directs behavior to seek out certain kinds of information and provides a ready means of interpretation.
The perceptual cycle can be used to explain human information processing in control rooms.
As an example, assuming that the control roo engineer has the correct knowledge of the system they are controlling, their mental model will enable them to anticipate events, search for confirmatory evidence, direct a course of action and continually check that the outcome is as expected is as expected.
Loss of Situational Awareness
Loss of situational awareness is correlated with poor system performance.
People who have lost situational awareness may be slower to detect problems with the system they are controlling as well as requiring additional time to diagnose problems and conduct remedial activities when they are finally detected.
Taxonomy of human errors
Level 1 SA
Data not available
Data hard to detect or discriminate
Failure to observe or monitor data
Misperception of data
Level 2 SA
Lack of, or incomplete, mental model
Use of incorrect mental model
Over-reliance on default values
Level 3 SA
Lack of, or incomplete, mental model
Over-projection of current trends
Underlying psychological mechanisms assigned to technical errors
- the person misread or misheard the information
- the person was distracted or not attentive
- the person could not recall, or confused the information
- there was poor interpretation, poor understanding, poor judgement, poor reasoning, or poor planning
Improving Situational Awareness
Reduce the requirement for people to make calculations.
Present data in a manner that makes level 2 SA (understanding) and level 3 SA (prediction) easier.
Organise information in a manner that is consistent with the persons goals.
Indicators of the current mode or status of the system can help the cue the appropriate situational awareness.
Critical cues should be provided to capture attention during critical events.
Global situational awareness is supported by providing an overview of the situation across the goals of the operator.
System-generated support for projection of future events and states will support level 3 SA.
System design should be multi-modal and present data from different sources together rather than sequentially in order to support parallel processing of information.
Situational Awareness Can Be Taught By:
Practice in scanning relevant displays to maximise perception.
Use of expanded checklists to ensure that relevant data are not lost.
Explicit training in allocation of attention.
Practising multi-tasking rather than performing isolated tasks.
Training in pattern recognition and pattern matching.
Situational Awareness and Safety: Contribution or Confusion
What is situational awareness?
Describes how individuals, teams, or systems develop and maintain awareness during task performance in safety critical systems
attempts to make inferences on what the content of that awareness is
attempts to assess the quality of that awareness against a normative ideal
Contribution or Confusion?
Although many safety-related applications exist, translation of
findings in the real world has been minimal.
Assessments of the impact of SA-related interventions in the real world are sparse as are publications describing the influence of SA-related research on safety and performance.
Why does this happen?
A large proportion of the studies presented in the literature
are related to theoretical and methodological development. Many articles focus on proposing or testing new theoritical models or measures of SA or examining existing models and measures.
Little attention has been given to the use of SA theory and principles to actually drive the system or interface or technology or training design process. Despite the existence of various SA design guidelines, SA is more often tested following the development or introduction of new systems rather than considered explicitly during the design process. Designs directly inspired by SA-related principles are few and far between (or at least literature describing them is sparse).
Problems associated with measuring the concept could be limiting its consideration as a key design component. It is difficult to assess SA in real world systems and more recent research applications focus less on assigning numerical values to the level of SA achieved in different systems and contexts, and more on the "content" of SA itself in terms of information and knowledge. This makes it difficult to discriminate between design concepts for their ability to support SA (which is something that stakeholders' desire). Since SA cannot be measured, stakeholders are less interested in it as a key design criteria.
Often cited as causal factor in accidents and incidents across the safety critical domains.
Loss of SA, lack of SA, or poor SA are now popular terms within accident investigations and have been identified as causal factors in all manner of incidents.
Its current popularity in safety circles is further evidenced by its continued appearance as a topic of investigation across the safety-critical domains.
Contemporary situation awareness and safety research
SA is likely to be different across different road users (e.g. drivers and motorcyclists), even when they are engaged in the same road situations.
Incompatibilities in SA lie at the root of collisions involving different road user groups such as "right of way" crashes between cars and motorcycles.
Road design is a key factor that influences driver SA both directly in terms of how drivers perceive road situations and behave and indirectly in terms of how it affects other road users and their behavior.
The more varied roadway experiences of motorcycle riders who also drive cars could enable them to develop enhanced levels of SA compared to other drivers.
Distraction is likely to degrade driver SA. When distracted, drivers were still able to integrate information in order to develop SA; however, engaging in the visually distracting task did change the content of their awareness in terms of the concepts underpinning SA.
Drivers are able to interact with secondary tasks in a situationally aware manner and that SA is a key factor in deciding to engage in secondary tasks,
Although SA is a key component of safe trackwork performance, to date SA has only been indirectly examined in this context.
SA of trackwork protection and its current state is a major contributory factor in trackwork incidents but that trackwork incidents are unlikely to be caused by poor SA regarding approaaching trains.
Various factors shaping SA, including shift handovers, interpretation of documentation, distraction, communication failures, poor planning, wrong assumptions, shift duration, vigilance decreements, and time pressure.
SA has ofen been cited as a key causal factor in fratricide incidents whereby friendly weapons are used with the intent to kill the enemy or destroy their assets, but result in unintentional death or injury to friendly personnel. However, Despite this, analyses focussing specifically on the role of SA in fratricide events have been scarce.
The less effective team made more communications than the more effective team, which questions the notion that more communcations equals better SA. The less effective team’s SA was more interconnected, which suggests that tight coupling and sharing of information may not be appropriate in military teams. The less effective team's SA was underpinned by concepts related to engagement, whereas the more effective team's SA was underpinned by concepts related to friendly units.
Inadequate team organisation along with poor communication can lead to poor SA, and point to the need to investigate the nature of SA exchanges between team members in collaborative systems as a keyline of future inquiry.
SA has become fashionable as a safety-related concept
within offshore drilling domain.
Offshore drillers reporting higher levels of stress, sleep disruption, and fatigue achieved lower SA scores on the Work Situation Awareness (WSA) scale.
Drillers who achieved lower scores on the WSA scale were also found to report greater levels of unsafe behaviour and had a higher involvement in previous offshore accidents.
Future team training programs should incorporate components focussed on SA, fatigue and stress management.
SA is likely to be substaintially different across human operators even when they are exposed to the same situation. When different situational understandings are incompatible, conflicts arise which in turn lead to safety-compromising incidents. System design (training, procedures, technologies) should take this into account and design for compatibility in SA across its users.
SA is heavily influenced by system design. Valid tools for assessing the affects of different design concepts on SA represent a critical methodological need in this area.
In multi-user or multi-role systems, experience of other roles
may lead to a heightened level of SA.
Further research is needed to clarify the relationship between other Human Factors concepts, such as distraction and fatigue, and SA. For example, the relationship between distraction and SA is complex and being distracted does not simply lead to diminished SA.
For safety research, the focus should be on the affects of different factors on the nature of SA in terms of the knowledge underpinning it, rather than on the quality of SA in terms of a comparison to a normative ideal.
Schemata have a strong influence on SA, shaping expectations
and directing situational attention and exploration.
In complex collaborative systems, more communications does not necessarily equate to better SA and tight coupling and sharing of information be not appropriate.
Stress, sleep disruption, and fatigue adversely influence SA.
Real time probes provide an appropriate means of assessing SA,
removing the need for scenario freezes.
There are ethical concerns associated with attributing ‘loss of
SA’ as a direct cause of accidents in safety critical systems.
The literature suggests that SA has yet to have achieved the level of impact of other Human Factors safety-related concepts, such as "human error" and "mental workload".
Does advanced training improve situational awareness?
Driver Situation Awareness
Situation focus vs. cognitive focus
The current state of the art in SA is "situation focused". It relies on the notion of information, that is, items and/or artifects in the situation about wich the driver requires awareness in order to act effectively.
It requires a mapping of the relevant information in the situation onto a mental representation of that information within the driver.
Secondly, it reflects the ‘‘hypothetical nature of perceptual experience" meaning that this representation is a model, "a representation that mirrors, duplicates, imitates, or in some way illustrates a pattern of relationships observed in data or in nature", "a characterisation of a process", meaning that in some form or other SA needs to provide the driver with "explanations for all attendants facts".
Firstly, information elements are structured and that SA is not merely about the presence or absence of discrete elemeents, but also their interconnection.
Linear vs. non-linear SA
The current state of the art in SA carries with it a certain ‘linear’
flavour. It assumes that ‘‘humans typically operate in a closed-loop manner’’.
Not only would knowledge in working memory be connected differently for advanced drivers compared to "normal" drivers, but that non linear feed-forward awareness in the form of anticipation will require and/or generate new types of information.
Advanced drivers should, therefore, have a greater proportion of new information having been subject to the training. In addition, it would be anticipated that such new information, post-intervention, will be of systematically greater importance compared to non-advanced drivers.
Normative vs. formative SA
The current state of the art in SA also carries with it a "normative" flavour, the tacit assumption being that the objective situation provides a reference point for judging "goodness" or "badness" of SA.
SA can be viewed as "a generative process of knowledge creation" in which the environment informs the agent, modifying its knowledge. Knowledge directs the agent's activity in the environment. That activity samples and perhapes anticipates or alters the environment, which in turn informs the agent and so on in a cyclical manner
Rather than linearly deterministic, in practice, SA emerges as rather more evolutionary and probabilistic which, in turn, raises interesting questions about judging "goodness" or "badness" of SA.
Is good awareness of a bad situation the same as poor awareness of a good situation?
If risks from hazards have been successfully minimized through the intelligent application of the system of car control, then does the more benign situation that the drier has created result in fewer critical variables to be aware of.
Conclusion from the user tests
Advanced driver training does improve SA but it is possible to go further.
Implicit knowledge is brought back into consciousness as a result of the intervention.
Significant increases in behaviour were noted, behaviours that can be implicated in the evolution towards better situations that the findings in terms of "hazard assessment and management" represent.
The network-based approach to the measurement of driver SA is compatible with, and responsive to, long standing knowledge about expert decision-making. Specifically, that experts chunck information and look for patterns of interrelationships and that SA is more than the sheer quantity and even type of knowledge possessed by an individual.
Assessment of operator's situation awareness for smart operation of mobile
by Fang et al
Crane lifting operations are unique among other heavy equipment as they demand huge workspaces and have a significant impact on the safety and efficiency of the entire construction projects.
The consequences of crane accidents are catastrophic as they very often result in a significant cost overrun, schedule delay, and serious injuries and fatalities.
Unlike other types of construction accidents, the victims in crane-related accidents are not necessarily limited to construction workers but also pedestrians walking-by as observed in many crane-related accidents.
Operating a crane is inherently a sophisticated job that requires the operators have extensive training and experience. Repetitive lift tasks and extended work hours make them vulnerable to distraction and fatigue. In addition to the errors of crane operators, lifting safety can also be jeopardized by poor coordination and communication with personnel such as riggers, signal persons, and ironworksers.
Crane operations can benefit from technologies similar to the advanced driver assistance systems (ADASs) deployed on automobiles that provide the real-time support to drivers based on surrounding situations.
With the goal of providing real-time assistance to crane operators, researchers in construction have explored the use of sensing, visualization, and simulation technologies. However, a holistic synthesis of existing technologies and a framework outlining further development is missing.
Previous efforts in evaluating a crane assistance system mainly focused on measuring accuracy, reliability, and ease of use of the introduced technologies. Very few emphasized on the system's effectiveness on helping the operators to understand the situation and the safety risks.
This is partly because that the SA of crane operators during lifting operations is difficult to define and measure in such complex and dynamic environment. This also leads to the fact that most of the techniques in previous research were only validated in a simulated environment instead of utilizing real lift tasks.
Critical components in operator assistance for mobile cranes
Accurately capture crane motions in real-time. The motions of a crane are essential to carry out a variety of spatial and temporal analyses for load capacity, crane stability, and collisions.
Cranes can be considered as giant robots with multiple degrees of freedom in a rigid body or kinematic relationships. Therefore, the entire notion of a crane can be capture by measuring critical motions (e.g., swing, lifting, and extension of the boom, extend/retract hoist cable).
Given the spatial and temporal scale of crane lifting operations, crane workspace is subject to constant changes in its surrounding environment (e.g., presence of vehicles and workers, newly erected structures). Therefore, modeling the as-is condition and dynamics of crane workspace is of great importance to successful operator assistance.
The UI needs to provide just-in-time alerts in multple forms (visual, auditory, haptic) with the right amount of information that supports operator to make decisions to mitigate hazards. Numerical feedback provided by traditional Load Limit Indicators (LMI) is limited. On the other hand, vivid pictures from crane camera systems may lead to distraction and increased mental workload when the operator struggles with the depth perception and limited field of view.
Although an array of safety devices are available in the market, the effectiveness and utilization of these devices in the industry are still unclear. It is important to evaluate the effectiveness of these safety devices in actual lift tasks in order to identify potential challenge and suggest further improvement.
Operator error and situation awareness
When a crane operator is exposed to one or more multiple hazards, they need to first perceive the presence of the hazards, mainly through the status, attributes, and dynamics of relevant elements in the environment.
Based on the information acquired and their understanding of hazards, the operator should recognize the type, severity, and consequences of the hazards.
Finally, the operator needs to make appropriate decisions and actions to mitigate or avoid the hazard from further development. Operator success in all cognition stages will result in safe behaviors. Failure in any of the stages will result in unsafe behaviors that may lead to accidents.
It should be noted that there is always a chance that the unsafe behavior may not lead to an accident and safe behavior may result in an accident as described in the model.
Successful hazard perception involves an acute state of alterness and high level of sensory skill from the operators, and it requires them to maintain a good SA during the operation.
During crane operations, crane operators' SA on an understanding of both the crane (crane motion, capacity, malfunctions) and the physical characteristics of the environment (wind speed, blind spots, clearances to obstructions).
Measurement of situation awareness
Examine the way subjects process information obtained from the environment, such as by analyzing gaze movement using eye tracking technology.
Another type of process indices is physiological measures such as electroencephalographic (EEG) activity, eye blinks, and cardiac activity, which represent the subject's overall functional state.
Although the changes in the subject's physiological states may be associated with cognitive activities, there is not necessarily a direct link between physiological states and the level of SA.
The level of SA is inferred from the performance outcomes based on the assumption that better performance indicates better SA.
Commonly used performance metrics include productivity level, time to perform the task, and the accuracy of the response, or converely, the number of errors committed.
The main advantage of performance measures is that they yield objective, quantitative results without disrupting task performance. Although in many cases there is a positive relation between SA and performance, this connection is not always direct and explicit.
Subjects are asked directly about their perception of certain aspects of the situation. The queries are usually designed by domain experts based on the characteristics of the tasks.
SAGAT (Situation Awareness Global Assessment Technique) is one of most popular query-based techniques. The operation is frozen at randomly selected times, and subjects are queries about their perception of the situation at that instant.
SAGAT is popular as it produces a quantitative assessment of SA and it can be benchmark the result with similar data in a similar context. However, SAGAT is criticized as it interrupts the natural flow of the task.
SPAM (Situation Present Assessment Method) can be used as an alternative to SAGAT based on the premise that SA involves simply knowing where to find a particular piece of information in the environment. In addition to be less intrusive, the benefits of using SPAM lie in its use of response time to indicate the level of SA so that the results reflect the real-time dynamic SA of the operator.
Theoritical Underpinnings of Situation Awareness: A Critical Review
The enhancement of operator situation awareness (SA) has become a major design goal for those developing operator interfaces, automation concepts and training programs in a wide variety of fields, including aircraft, air traffic control, power plants, and advanced manufacturing systems
One can easily see that situation awareness has always been needed in order for people to perform tasks effectively.
Due to achievements in various types of datalink and internet technologies, systems can also provide data on almost anything anywwhere in the world. The problem with today's systems is not a lack of information, but finding what is needed when it is needed.
Unfortunately, in the face of this torrent of data, many operators may be even less informed than ever before. This is because there is a huge gap between the tons of data being produced and disseminated and people's ability to find the bits that are needed and process them together with the other bits to arrive at the actual information that is required for their decisions. This information must be integrated and interpreted correctly as well.
In addition to designing systems that provide the operator with the needed information and capabilities, we must also insure that it is provided in a way that is usable cognitively as well as physically. This design objective and measure of merit has been termed situation awareness.
What is Situation Awareness (SA)?
Simply put, SA is knowing what is going on around you.
Many definitions of SA have been developed, some very closely tied to the aircraft domain and some more general (see Dominguez, 1994 and Fracker, 1988). Now SA is being studied in a variety of domains.
A general definition of SA that has been found to be applicable across a wide variety of domains is the definition of Endsley, 1988.
Temporal aspect of SA
A critical part of SA is often understanding how much time is available until some event occurs or some action must be taken. It is important to note that the interest is not only the space where the event will occur, but also the time when the event will occur.
The dynamic aspect of real-world situations is another important temporal aspect of SA. The rate at which information is changing is a part of SA regarding the current situation, which also allows for projection of future situations. The dynamic nature of situations dictates that as the situation is always changing, so the person's situation awareness must constantly change or be rendered out-dated and thus inaccurate.
It is entirely possible to have perfect SA, yet make an incorrect decision. As an example, the battle commander may understand where the enemy is and the enemy's capabilities, but select a poor or inappriopriate strategy for launching an attack.
Who needs SA?
It should be clearly noted, however, that technological systems do not provide SA in and of themselves. It takes a human operator to perceive information to make it useful.
No matter how a person does his or her job, the need for SA has always existed, even in what would not normally be considered heavily cognitive or technological fields.
The highlighted emphasis on SA in current system design has occurred because (a) we can now do more to help proivde good SA through decision aids and system interfaces, and (b) we are far more able to actually hinder SA through these same efforts if, as designers, we fail in adequately addressing the SA needs of the operator.
How do we get SA?
Cues may be received through visual, aural, tactile, etc. Some cues may be overt and some quite subtle which are registered only subscionciously.
As we move towards the instantiation of remote operators in many domains, a major challenge will be providing sufficient information through a remote interface to compensate for the cues once perceived directly.
It is important to note that while as system designers we tend to focus on the information provided through the system and its operator interface, this is not the only source of SA. In many domains, operators may be able to directly view and hear information form the environment itself, although in some cases they may not.
It is important that analyses of the SA provided by system designs also take into account that information which operators also derive from other means.
It is critical to note that his is not a passive process of receiving displayed information, but one in which the operator may be very actively involved. As an example, the operator in many systems can control which information is displayed and which information is attended to.
Theories of SA
Working Memory and Attention
Several factors will influence the accuracy and completeness of SA that individual operators derive from their environment.
Humans are limited y working memory and attention. The way in which attention is employed in a complex environment with multiple competing cues is essential in determining which aspects of the situation will be processed to form SA.
Once taken in, information must be integrated with other information, compared to goal states and projected into the future - all heavily demanding on working memory.
Much can be said about the importance of attention on SA. Numerous factors will dictate how people direct their attention in acquiring information, including learned scan patterns and information sampling strategies, goals, expectations and other information already processed.
Attention to information is prioritized based on how important that information is perceived to be. It should also be pointed out, however, that even experienced operators can err in this process, neglecting to attend to certain information over other information.
Correctly prioritizinf information in a dynamic environment remains a challenging aspect of SA. Good SA requires enough awaremeness of what is going on across a wide range of SA requirements to be able to determine where to best focus one's attention for more detailed information.
There are 4 strategies actively used by operators to reduce the working memory load associated with SA: (1) information prioritization, (2) chungking, (3) gistification of information, and (4) restructuring the environment to provide external memory cues.
Long-term Memory and Working Memory Connection
To view SA as either function of working memory or long-term memory would probably erroneous.
Information proceeds directly from sensory memory to long-term memory, which is necessary for pattern-recognition and coding. Those portions of the environment that are salient remain in working memory as a highlighted subset of long-term memory through either localized attention or automatic activation.
This way, information from the environment may be processed and stored in terms of the activated mental model or schema. Thus activated, these schema provide a richs source of data for bringing to bear on the situation including mechanisms for processing the data and default values for filing in missing information.
Sarter and Woods (1991) emphasize the importance of information that can be activated from long-term memory to support limited working memory. In this sense, SA is a unique product of external information acquired, working memory processes the internal long-term memory stores activated and brought to bear on the formation of the internal representation,
Long-term Memory, Mental Models and SA
With experience, operators develop internal models of the systems they operate and the environments in which they operate. These models serve to help direct limited attention in efficient ways, provide a means of integrating information without loading working memory, and provide a mechanism for generating projection of future system states.
Critical cues in the environment may be matched to such schema to indicate prototypical situations that provide instant situation classification and comprehension.
The use of mental models in achieving SA is considered to be dependent on the ability of the individual to pattern match between critical cues in the environment and elements in the mental model.
The SA or situational model captures not only the person's representation of the various parameters of the system, but also a representation of how they relate in terms of system form and fuction to create a meaningful synthesis - a gestalt comprehension of the system state.
In addition to containing the current value of engine temperature and its rate of change, the situation model also includes an understanding of the impact of that state on the system and on projected events (e.g. overheat).
The concept of a mental model is useful in that it provides a mechanism for (a) guiding attention to relevant aspects of the situation, (b) a means of integrating information perceived to form an understanding of its meaning, and (c) a mechanism for projecting future states of the system based on its current state and an understanding of its dynaimcs.
Pattern Matching and Other Cognitive Processes
Pattern matching goes hand-in-hand with schema and mental models in facitlitating the development of SA. There is considerable evidence that experienced decision makers use pattern matching to recognize perceived information as a particular exemplar of a known class of situations.
It should be noted, however, that SA is not solely dependent on pattern matching.
A combination of pattern matching, conscious analysis, story building, mental simulation, and meta-cognitive processes all may be used by operators at various times to form SA.
Goals are central to the development of situation awareness. Essentially, human information processing in operating complex systems is seen as alternating between data driven (bottom-up) and goal driven (top-down) processing. This process is viewed as critical in the formation of SA.
In goal driven processing, attention is directed across the environment in accordance with active goals. The operator actively seeks information needed for goal attainment and the goals simultaneously act as a filter in interpreting the information that is perceived,
In data driven processing, perceived environmental cues may indicate new goals that need to be active. Dynamic switching between these two processing modes is important for successful performance in many environments,
Smith and Hancock (1995) tahe the view that SA is purposeful behavior that is directed toward achieving a goal in a specific task environment. They furthermore point out that SA is therefore dependent on a normative definition of task performance and goals that are appropriate in the specific environment.
The alternating between bottom-up and top-down processing is one of the most important mechanisms underlying SA. Failures in this process can be particularly damaging to SA, including failures to process information needed to assure proper goal selection and resultant inattention to needed information or minsinterpretation of information perceived.
Preconceptions or expectations influence the formation of situation awareness. People may have certain expectations about what they expect to see or hear in a particular environment.
This may be due to mental models, prior experiences, instructions or other communcations. These expectations will influence how attention is daeployed and the actual perception of information taken in. There is a tendency for people to see what they expect to see (or hear).
Expectations may be formulated based on the active mental model and prior expectations. They also may be developed through instructions or other communications.
Contextual information, provided by early information, acts to trigger different information processing strategies, thus influencing which information is attended to and how conflicting information is explained.
Automaticity is another mechanism developed with experience that can influence situation awareness. With experience, the pattern.recognition/action selection sequence can become highly routinized and developed to a level of automaticity-
This mechanism provides good performance with a very low level of attention demand in certain well-understood environments. In this sense, automaticity can positively affect situation awareness by reducing demands on limited attention resources, particularly for demanding physical tasks.
SA can also be negatively impacted by automaticity of cognitive processes, however, due to a reduction in responsiveness to novel stimuli. Information that is outside the routinized sequence may not be attended to. Thus, situation awareness may suffer when that information is important.
While the advantage of automaticity is the low level of attention required by the task, it also has significant hazards for SA and performance, such as (1) does not allow for novel integration of information, (2) people under automaticity tend to be non-receptive to novel events, and (3) errors tend to occur when there must be a change in the learned pattern.
Measurement of SA
Why Measure SA?
One of the main reasons for measuring SA has been for the purpose of evaluatin new system and interface designs.
To determine the degree to which new technologies or design concepts actually improve (or degrade) operator SA, it is necessary to systematically evaluate them based on a measure of SA, thus providing a determination of which ideas have merit and which have unforeseen negative consequences. Explicit measurement of SA during design testing determines the degree to which design objectives have been met.
If SA is measured direclty, it should be possible to select design concepts that promote situation awareness, and thus increase the probability that operators will make effective decisions and avoid poor ones.
Problems with SA, such as data overload, non-integrated data, automation, complex systems that are poorly understood, excess attention demands, and many other factors, can be detected early in the design process and corrective changes made to improve the design
Evaluations of Training Techniques
A measure of SA may also be useful for evaluating the impact of training techniques on SA.
Although relatively little work has been conducted to date on the effects of different training approaches on operator SA, in theory the same principles apply as those discussed for evaluating design concepts.
Investigations of the Situation Awareness Construct
The measurement of SA is essential for conducting studies to empirically examine factors that may affect SA, such as individual skills, or the effectiveness of different processes and strategies for acquiring SA, and for investigateing the nature of the SA construct itself.
The theory can only be expanded upon and developed further through such effots.
Requirements for SA Measures
The veracity of available SA measures needs to be established. Ultimately, validity and reliability must be established for any SA measurement technique that is used.
It is neccessary to establish a metric that (a) indeed measures the construct it claims to measure and is not a reflection of other processes, (b) provides the required insight in the form of sensitivity and diagnosticity, and (c) does not substantially alter the construct in the process, providing biased data and altered behavior.
Implications of SA Theory for Measurement of SA
Processes vs States
SA as defined here is a state of knowledge about a dynamic environment. This is different than the processes used to achieve that knowledge.
Different individuals may use different processes to arrive at the same state of knowledge, or may arrive at different states of knowledge based on the same processes due to differences in comprehension and projection of acquired information or different mental models, schema, etc.
SA, Decision Making, and Performance Disconnect
Just as there may be a disconnect between the processes used and the resultant SA, there may also be a disconnect between SA and the decisions made.
With high levels of expertise in well-understood environments, there may be a direct situation awareness-decision link, whereby understanding what the situation is leads directly to selection of an appropriate action from memory. Although this is not always the case, individuals can still make poor decisions with good SA.
The relation between SA and performance can be viewed as a probabilistic link. Good SA should increase the probability of good decisions and good performance, but does not guarantee it and vice versa.
The way in which a person deploys his or her attention in acquiring and processing ifnromation has a fundamental impact on SA.
Design changes that influence attention distribution (intentionally or not) therefore can have a big impact on situation awarenesss.
Measurement techniques that artificially influence attention distribution should be avoided, as they may well change the construct that is being measured in the process.
Automaticity may influence memory recall. With automaticity, there is very little awareness of the processes used.
Direct measures of SA tap into a person's knowledge of the state of the dynamic environment. This information may be resident in working memory for a short period of time or in long-term memory.
Time also affects the ability of people to report information from memory. With time, there is a rapid decay of information in working memory, thus only long-term memory access may be available.
Real-time, immediate access of information from memory can also be difficult, however, as this process may influence ongoing performance and decision processes and SA itself.
Consideration of supply and demand of resources as central to SA. However, only when workload demands exceed maximum human capacity is SA necessarily at risk. SA problems may also occur under low workload. Although inter-related, SA and workload are independent constructs in many ways.
As people can make trade-off between the level of effort and how much they feel they need to know, it is important that both SA and workload be measured independently in the evaluation of a design concept.
How much SA is enough?
As SA and performance are only linked probabistically, there is really no set threshold of SA that can guarantee a given level of performance.
It is probably better to think of SA in relative terms. As the level of SA increases, the probability of making good decisions and performing well also increases. Therefore, it will be necessary to make relative comparisons when evaluating design concepts.
It is also fair to say that there is no such thing as too much SA. More SA is always better. As SA is operationally defined as that which one really needs to know, gaining more information and understanding of these elements is always better and increases the likelihood of effective decision making.
It should also be carefully noted that one should ensure that SA on one aspect of the situation is not gained at the cost of another, equally important aspect.
Measuring Situation Awareness in Complex Systems
by Salmon et al
Measuring situation awareness
The current theoritical contentnion surrounding the construct of SA makes measuring it a problematic task. The SA literature is disparate, and many academics disagree on what SA actually refers to.
Various definitions of SA
Bell and Lyon, 2000 view as SA as knowledge in working memory.
Endsley, 1995a views it as a cognitive product of information processing.
Smith and Hancock, 1995 view it as externally directed consciousness
Fracker, 1991 thinks that SA refers to the process of gaining awareness.
Varios SA models
Three-level model (Endsley, 1995a) is a cognitive theory that uses an information processing approach.
Smith and Hancock (1995) model is an ecological approach underpinned by the perceptual cycle model.
Bedny and Meister (1995) approach uses an activity theory model to describe SA.
Situation awareness measures
Freeze probe techniques
Freeze probe techniques involve the administration of SA related queries during the task performance during "freezes" in a simulation of the task under analysis.
Typically, a task is randomly frozen, all displays and screens are blanked and a set of SA queries regarding the current situation at the time of the freeze is administered.
Participants are required to answer each query based upon their knowledge and understanding og the situation at the point of the freeze.
Participants' responses are compared to the state of the system at the point of the freeze and an overall SA score is calculated at the end of the trial.
The main advantage of using freeze probe techniques is theirl alleged direct, objective nature, which removes the issues associated with collecting SA data post trial.
However, such approaches have also been heavily criticised for their level of intrusion on task performance, the difficulties associated with using such approaches during real world activities and have faced quesitons regarding their valididty.
SAGAT (Endsley, 1995b) is the most popular freeze probe technique and was developed to assess pilot SA based on the three level of SA.
Real-time probe techniques
Real-time probe techniques involve the administration of SA related queries on-line, but with no freeze of the task under analysis.
Subject Matter Experts (SMEs) develop queries either prior to the task or during task performance and administer them while the participant is performing the task under analysis. Answer content and response time are taken as a measure of participant SA.
The main advantage associated with real-time probe techniques is the reduced level of intrusiveness compared to freeze probe techniques, since no freeze of the task is required. However, these approaches still suffer from the same concerns associated with freeze probe approaches,
The Situation Present Assessment Method (SPAM) is a real time probe technique developed for use in the assessment of air traffic controllers SA. The query response time (for giving correct responses) is taken as an indicator of the operators' SA and workload.
Self-techniques are used to elicit subjective assessments of participant SA and typically administered post-trial.
Self-rating techniques involve participants providing a subjective rating of their perceived SA via a rating scale of some sort.
The primary advantage of self-rating techniques are their ease of application and their non-intrusive nature.
However, subjective self-rating techniques are heavily criticised for a plethora of reasons, including the various problems associated with the collection of SA data post-trial and also issues regarding their sensitivity.
The Situation Awareness Rating Technique (SART) is the most popular of these approaches and uses ten dimensions to measure operator SA.
Observer rating techniques
Observer rating techniques are most commonly used when measuring SA "in the field" due to their non-intrusive nature.
Observer rating techniques typically involve SMEs observing participants during task performance and then providing an assessment or rating of each participant's SA.
The SA ratings are based upon pre-defined observable SA related behaviors exhibited by participants during task performance.
Observer rating approaches are advantageous since they have no impact on the task being performed and can be used during real world activities.
However, these approaches also suffer from concerns regarding their validity.
The Situation Awareness Behavioural Rating Scale (SABARS) is an observer rating technique that has been used to assess infantry SA in field training exercises.
Using performance measures to assess SA involves measuring relevant aspects of participant performance during the task under analysis.
Depending upon the task, certain aspects of performance are recorded in order to determine an indirect measure of SA.
When measuring military infantry exercises, performance measures may be "kills", "hits" or mission success or failure. When assessing driver SA, hazard detection, blocking car detection, and crash avoidance are measured.
Process indices involve recording the processes that participants use in order to develop SA during the task under analysis.
Some examples of these techniques are using videos from eye-tracking devices and transcripts from thinking out loud, etc.
Comparing SA measures
The unavoidable suspicion is that different forms of SA measurement approach measure different aspects (or different things entirely) of operator SA. Despite this, only a handful of studies have sought to compare the different SA measurement apporaches available.
Endsley et al (2000) report that SART and the on-line probe approach did not show a significant difference in SA between conditions, whereas the SAGAT scores where sensitive to display changes.
No statistically significant correlation was found between total SAGAT scores and performance, however signifcant relationships between the individual SAGAT queries and performance were identified.
Moderate correlations between different SA measures were also identified: Level 1 SAGAT scores were correlated with the overall SART rating, the supply dimension rating and the Level 1 real time probe. However, level 2 and 3 SAGAT scores correlated negatively with the SART understanding and supply ratings.
Endsöey et al (1998) report that "the subjective assessment of SA derived via SART does not appear to be related to the objective measure of SA provided by SAGAT.
Jonas and Endsley (2000) reported that SAGAT, SART and NASA TLX demonstrated sensitivity to the differences in the two scenarios undertaken by teams and that there was a significant correlation between the real time probe measure and the SAGAT measure.
A relationship between NASA TLX and SART measures was also identified. Indeed, these two measures have previously been found to be highly correlated.
Endsley et al (2000) report a statistically significant correlation between level 1 SAGAT query and SART and level 1 real time query scores.
This leads us to believe that the different forms of SA measures discussed may be actually assessing different aspects of SA or even different things entirely.
A Taxonomy of Situation Awareness Error
The failures of human decision making are frequently cited in investigations of error in a wide variety of systems.
In aviation, failures in decision making are attributed as a causal in factor approximately 51.6% if all fatal accidents and 35.1% of non-fatal accidents, of the 80-85% of accidents which are attributed to human error.
While some of the incidents may represent failures in actual decision making, a high percentage of these errors are actually errors in situation awareness.
That is, the aircrew makes the correct decision for their perception of the situation, but their perception is in error. This represents a fundamentally different category of problem than a decision error - in which the correct situation is comprehended, but poor decision is made as to the course of action to take.
SA break-downs can occur due to either incomplete SA - knowledge of only some of the elements - or inaccurate SA - eroneous knowledge concerning the value of some elements or the integration and comprehension of those elements.
SA Error Taxonomy
Level 1 SA - Falure to correctly perceive the situation
Data not available
Data difficult to detect/perceive
Failure to scan or observe data
Misperception of data
Level 2 SA - Failure to comprehend situation
Lack of/poor mental model
Use of incorrect mental model
Over-reliance on devault values in model
Level 3 SA - Failure to project situation into the future
Lack of/poor mental model
Maintaining multiple goals
- some people are simply bad at doing multiple things at once
- doing tasks automatically