State actors, private influence operators and grassroots groups are exploiting the openness and reach of the Internet to manipulate populations at a distance. They are extending a decades-long struggle for "hearts and minds" via propaganda, influence operations and information warfare, often in the form of coordinated incidents that are part of longer-timescale narrative-based campaigns.

Our work on cognitive security extends information security principles, practices, and tools, to the detection and management of information harms including disinformation and disinformation. Specifically, we have adapted and extended frameworks used to describe information security incidents, to create the DISARM series of frameworks for understanding and responding to organized disinformation incidents; these have been in continuous use since 2019 for analysis, simulations, training, and country-level disinformation risk assessments.

In this paper, we describe how and why the DISARM frameworks were created, and discuss their components and uses, including analysis of ways, means, and ends to achieve influence goals.

We developed a Co-Awareness Analytic Framework to help ensure satisfactory performance of future teams consisting of humans and intelligent machines operating as peers. This Framework was designed to provide more precision than current tools to analyze the interactions among tasks, task environments, and the capabilities and dependencies among teams of humans and intelligent machines. A key aspect of this framework is reimagined definitions of situation awareness (SA) and option awareness (OA) so that they apply to both humans and machines, rather than defining them in purely cognitive terms that can only be applied to humans. Having appropriate SA and OA has been shown to benefit teams of humans, and there is a presumption that SA/OA will also benefit human-machine teams (HMTs). It is difficult to create requirements for, and subsequently measure, SA and OA in HMTs, however, when the current definitions for these concepts are so oriented towards humans. To address this issue, this paper provides a reconceptualization of awareness that is agnostic to team composition known as Informational Awareness definitions. It also includes a first look at the Co-Awareness Analytic Framework and a brief example of its use.

Driving intention recognition is an important aspect of Advanced Driving Assistance Systems (ADAS) for giving drivers suggestions to maneuver safely. The intention recognition algorithms in ADAS are often developed using Machine Learning-based models. The model's input, such as environmental (ENV) and eye-tracking (ET) features affect the model's recognition performance. In this contribution, an Artificial Neural Network-based state machine is used for lane changing intention recognition. Three lane changing behaviors are considered, left/right lane change and lane keeping. Here, data consisting of ENV and ET information are collected using a driving simulator and eye-tracker. The aim is to investigate the effect of different feature types on the model's intention recognition performance. First, a 10-cross validation is performed to evaluate the model's performance, using only ENV and both ENV and ET features. The validation results show that the model with only ENV features performs better with respect to different metrics. Thus in the test, only ENV features are used to evaluate the performance. Accuracy values of higher than 80 % are achieved. Furthermore, the recognition performance of the model is compared with other Machine Learning models. The approach introduced outperforms other models in most metrics.

Modern surveillance systems require to integrate coherently all available sources of information to compose an operational picture that is as complete as possible. While in the past surveillance had suffered from a lack of data, current technology transformed the problem into one of an overabundance of information, leading to an extreme need for automated analysis. Indeed, current surveillance sensors generate volumes of data that would have been unthinkable only a few years ago. Therefore, all the processing, algorithm calibration, parameter tuning, etc., need to be executed as much automatically as possible. This also requires novel paradigms for algorithmic design. In this respect, Artificial Intelligence (AI) and Data Fusion (DF) offer an unprecedented opportunity to strengthen the technological edge; however, the risk is to elevate, at the same time, the speed of the threats we face. Indeed, the surveillance task is complicated by the diversification of threats, whose nature and origin is most often unknown. AI and DF techniques have the potential to identify patterns emerging within these very large datasets, fused from a variety of sources and generated from monitoring wide areas on a day-to-day basis, and use the learned knowledge to anticipate the possible evolution(s) of the operational picture. The presentation will focus on both real-world scenarios and theoretical models, spanning from the underwater to the space domain, including the analysis of scenarios with heterogeneous surveillance sensors (such as radar, sonar and satellite). Finally, the opportunity will be taken to present a brief overview of the transitioning of some of these techniques for COVID-19 epidemiological curve monitoring and forecasting.

The increasing availability of Satellite technology for Earth observation enables the monitoring of land subsidence, achieving large-scale and long-term situation awareness for supporting various human activities. Nevertheless, even with the most-recent Interferometric Synthetic Aperture Radar (InSAR) technology, one of the main limitations is signal loss of coherence. This paper introduces a novel method and tool for increasing the spatial density of the surface motion samples. The method is based on Transformers, a machine learning architecture with dominant performance, low calibration cost and agnostic method. This paper covers development and experimentation on four-years surface subsidence (2017-2021) occurring in two Italian regions, Emilia-Romagna and Tuscany, due to ground-water over-pumping using Sentinel-1 data processed with P-SBAS (Parallel Small Baseline Subset) time-series analysis. Experimental results clearly show the potential of the approach. The developed system has been publicly released to guarantee its reproducibility and the scientific collaboration.

Swarms of Unmanned Aerial Vehicles (UAVs) are increasingly adopted to provide early situational awareness in environmental monitoring missions. Currently, a challenging problem is to manage swarms via responsive and adaptive coordination mechanisms. This study takes into account a cutting-edge swarm coordination algorithm called SFE, based on three strategies: stigmergy, flocking and evolution. Stigmergy is the release of digital pheromone by drones to generate a potential field that influences the steering in the spatial-temporal proximity. Flocking is a formation mechanism to spatially organize drones into local groups. Evolution is the parametrically adaptation of Stigmergy and Flocking to a specific type of mission. A novel algorithm called P-SFE is proposed, to overcome the limit of SFE related to the static priority of the three strategies. This prioritization is managed through an Artificial Immune System. A simulation testbed is developed and publicly released, on the basis of commercially available technology and real-world scenarios. Experimental results show that the proposed P-SFE extends and sensibly outperforms the SFE.

This paper explores inference patterns within and among higher data fusion levels. Systematic methods and examples are presented for reasoning about relations, relationships, situations, and scenarios in a dynamic, complex, and uncertain world. The series of JDL fusion levels often provides a useful and natural sequence for inferencing by component-wise composition: measurements to features to states of individuals to relationships, situations, and scenarios. However, there are instances in which other inference sequences are preferable. These include cases where contextual information, as provided by the encompassing situation and scenario, becomes necessary to provide expectations or to resolve ambiguities. A counter-piracy application is examined to illustrate methods for synergistic reasoning within and across fusion levels and the interplay between higher-level fusion and response management in managing dynamic threat situations.

We focus on identity as an equivalence relation (supporting Leibniz's Law and the Principle of Substitution) and how identity hypotheses are addressed and supported in law enforcement. The dominate notion of identify in computer science is of something like a profile; identifying in our terms can have such a profile as one side of an identity as long as it is interpreted as denoting. Our examples are from law enforcement, but we intend our framework to apply generally to identifying people. We use a modified version of Barwise and Perry's situation theory to capture how evidence accrues to identity hypotheses. We extend the relational theory of meaning of situation semantics to encompass an extended version of the causal theory of reference that regards traces of a person (e.g., fingerprints) and their copies as devices denoting that person. These objects stitch together the situations relevant to an investigation. We extend situation theory with higher-order elements motivated by our analysis of criminal investigations, event statements, and thematic relations taken from sentential semantics. We present a hypothetical criminal scenario (which we have presented in focus groups) and show how we encode it into RDF in terms of our ontologies.

Vibro-tactile interfaces were proposed as an alternative to support situation management and human-machine communication in information-rich domains. Tactile cues can capture the operator's attention in busy environments. However, like the visual and auditory channels, frequent use of tactile alerts may lead to high workload, impaired performance, or neglect. We examined the effectiveness of three levels of tactile alerts when combined with visual alerts. On the 'basic level,' the alert provides easy to interpret binary information with a "low threshold," on the 'simple level' the alert provides easy to interpret information about the occurrence of a pre-determined event with "high threshold," in the 'complex level' the alert requires more effort to interpret but includes specific information. Two experiments simulate an operational mission in which participants ride an autonomous ground vehicle patrol while looking for threats and targets in the area and monitoring two supporting unmanned systems (ground and aerial) in a MUM-T (Manned UnManned Teaming) setup. Response times to notifications, threat identification rates, and subjective workload were measured. Results indicate that tactile alerts, in addition to visual alerts in a visually loaded and auditory noisy scene, improve task performance. Moreover, a complex level of tactile alerts does not impair performance or increase the perceived workload compared to the basic and simple levels

Human activity recognition has attracted enormous research interest because it provides precious contextual information in several domains spanning from health-care to security, safety, and entertainment. So far, robust and consolidated literature focused on the automatic detection of activities performed by single individuals, with a great variety of approaches in terms of sensing modalities, recognition techniques, specific set of recognized activities, and final application objectives. However, much less research attention has been devoted to scenarios in which multiple subjects perform individual or joint activities forming groups to achieve common goals. This problem is often referred as multi-user activity recognition. With the advent of the Internet-of-Things, smart objects are being pervasively spread in the environment and worn on the human body, enabling contextual and distributed recognition of group and multi-user activities. This talk discusses the motivations and advantages of multi-user activity recognition and overviews sensing methods (including those based on Wearable Computing Systems and in particular Collaborative Body Sensor Networks, developed in the context of the SPINE Body-of-Knowledge), recognition approaches, and impact in practical, relevant applications. The multi-user activity recognition approaches are also analyzed from the perspective of situation-aware systems. In particular, an emerging trend analyzed during the talk is the definition of collaborative situation-aware wearable computing systems, i.e., wearable devices able to perceive and understand the situation in order to adapt their behavior and anticipate user's needs. The discussion also outlines important open research challenges and related future directions.

This paper focuses on the relationship between public satisfaction with COVID-19 pandemic situation management and the pandemic development based on the Estonian case. The focus is on whether public satisfaction is related to situation awareness within pandemic situations. For analysis, data were collected from open sources, namely, we analyzed data from 37 COVID-19 survey reports that were carried out at the request of the Estonian State Office (Riigikantselei). For time-series analysis, we collected data on weekly rates of new COVID-19 cases in Estonia from the World Health Organization (WHO COVID-19) homepage. The results of the analysis indicate that public satisfaction with situation management (SM) correlates strongly and negatively with infection rates, i.e., lower public satisfaction with higher COVID-19 infection rates, and vice versa.

There are some suggestions in the literature indicating a significant improve of the job satisfaction during the COVID-19 pandemic compared to the level of the job satisfaction pre COVID-19 pandemic period. The findings of our empirical studies of the periods pre COVID-19 (2018-2019) and during COVID-19 (2020-2021) indicate that the job satisfaction significantly improved during the COVID-19 pandemic. What the literature sources did not pay enough attention to the empirical evidence related to the changes in the determinants of the job satisfaction compared to pre and during COVID-19 pandemic or simplifying, which are the main sources for the job satisfaction improvement during COVID-19 pandemic. That is what we are trying to find an answer to in current paper.

In clinical practice, patient care flows are generally subject to recommended and standardized therapeutic interventions. Especially in a home care setting, situation-aware adherence to therapy can be both difficult for the patient to follow and difficult for the physician to assess. Process mining techniques may be useful artificial intelligence solutions for remotely assessing the compliance of patients' behavior with the corresponding care path, especially if adopted in a cognitive IoT Edge infrastructure, dedicated to the acquisition and analysis of daily routines in a form of event log. In this paper, we present an innovative method to measure in-home adherence to metabolic syndrome management with the aim of providing awareness of the patient's current situation. The analytical results demonstrate the validity of using process mining techniques to remotely evaluate patient behavior.

Situation Awareness (SA) and Internet of Things (IoT) can successfully be adopted with the aim of developing "smart" assistive technologies able to understand and interact with users' actions, adapting to several situations. A peculiar type of assistive technology is meant to promote seniors' active aging, providing assistance, companionship, and entertainment. The study presented in this work was carried out in the context of a European project named "EMPATHIC" aimed at developing virtual coaches for senior users. More specifically, the study analysed seniors' gaze behavior while interacting with a virtual agent, to investigate whether users' attentional engagement would have changed over the interaction and accordingly to the conversational topics. Results highlight senior users' considerable engagement during the interaction and an overall positive attitude toward the proposed virtual agents.

The use of wearable devices in daily activities is continuously and rapidly growing. Wearable technology provides seamless sensing, monitoring, multimodal interaction, without continuous manual intervention and effort for users. These devices support the realization of novel applications in many domains, from healthcare to security and entertainment, improving the quality of life of users. The situation awareness paradigm allows wearable computing systems to be aware of what is happening to users and in the surrounding environment, supporting automatic smart adaptive behaviors based on the identified situation. Although situation-aware wearable devices have recently attracted a lot of attention, there is still a lack of methodological approaches and references models for defining such systems. In this paper, we propose a reference architecture for situation-aware wearable computing systems grounded on Endsley's SA model. A specialization of the architecture in the context of multi-user wearable computing systems is also proposed to support team situation awareness. An illustrative example shows a practical instantiation of the architecture in the context of contact tracing using smart sensorized face masks.

The use of unmanned aerial vehicles (UAVs) has become commonplace in the military domain. They are used together with manned platforms (e.g. a helicopter) to form a tactical team. This concept is called manned-unmanned teaming (MUM-T). During the execution of a mission, pilots continuously assess the current situation and make decisions on how and where to deploy these UAVs. Thereby, the primary influence factor on this decision process is the threat assessment in the current situation. However, in high workload situations or due to the lack of situation awareness, the pilots may fail to deploy the UAVs effectively. In this article, we propose a software agent that performs a threat assessment on a given scenario and generates UAV task proposals to assist the pilots in the decision-making process. We conducted experiments with human operators who use the system in pre-defined scenarios to show that such an agent can be used to make decisions similar to the ones a human operator would make. The results presented in this article show that the system's outcomes on the threat assessment and UAV decisions are in alignment to the human operators' assessment and decision process outcomes in most cases. Finally, we analyze the results obtained from the experiment and outline future work.

There is potential great value in turning physiological and behavioral data into actionable information in tactical environments. However. the design of an appropriate system in terms of measurement accuracy, wearing comfort and technical feasibility, for instance for forces training in realistic conditions, therefore requires addressing multiple scientific and technical challenges. The current paper focuses on efforts realized to address four challenges identified in recent efforts in the development of an integrated wearable system. These four challenges pertain to: 1) Data management; 2) Wearable sensors; 3) Algorithms and models; and 4) Human factors considerations. Components were developed and integrated to tackle each of the four challenges and the integrated system was tested on participants during field trials. The system referred to "Readiness Evaluation: QUantified INdividuals" (REQUIN), offers a dashboard to visualize in real-time the various metrics collected and calculated by the system, including metrics derived from wearable sensors and models. Results of the field trials are discussed in regard with the four challenges addressed in the paper and recommendations for further research are presented, including using alternative sensing technologies for blood oxygenation measures and improving the models' specificity. This study represents a critical step in the integration of real-time sensing technologies for applications involving collective situation management and control.

Aiming to improve traffic safety, we investigated the effect of landscape and soundscape on driving behavior in urban roads using a VR driving simulator. While active and passive systems are implemented in vehicles and safer infrastructure is being proposed and built, previous research has suggested that the environment around roads also affects driving behaviors. Although changes in infrastructure and technological improvements are costly, well-implemented policies and better maintained road environments could also affect driving behaviors, especially with respect to risky driving. In this paper, we focus on traffic noise and road maintenance, which are examples of soundscapes and landscapes that can be improved relatively easily via appropriate policies. Specifically, we investigated the effects of soundscapes and landscapes on speed, lateral deviation from the road center, and behavior at crosswalks. The experimental results showed that both the soundscape and landscape affected driving behaviors; more precisely, horn noise and dirty environment enhanced aggressiveness in driving and driver distraction. This implies that policies targeting those aspects may improve safety without costly interventions.

Excessive intra-operative noise in cardiac surgery has the potential to serve as source of distraction and additional cognitive workload for the surgical team, and may interfere with optimal performance. The separation from bypass phase is a technically complex phase of surgery, making it highly susceptible to communication breakdowns due to high cognitive demands and requiring tightly coupled team coordination. The objective of this study was to investigate team cognitive workload levels and communication in relation to intra-operative time periods representative of infrequent vs. frequent peaks in ambient noise. Compared to 5-minute segments with no peaks in noise at all, segments with the highest percentage of noise peaks (≥10%) were significantly associated with higher team members' heart rate before, during, and after noise segments analyzed. These noisier segments were also associated with a significantly higher level of case-irrelevant communication events. These data suggest that case-irrelevant conversations associated with a greater degree of excessive peaks in noise may be associated with team workload levels, warranting further investigation into efforts to standardize communication during critical surgical phases.

Situational awareness (SA) at both individual and team levels, plays a critical role in the operating room (OR). During the pre-incision time-out, the entire OR team comes together to deploy the surgical safety checklist (SSC). Worldwide, the implementation of the SSC has been shown to reduce intraoperative complications and mortality among surgical patients. In this study, we investigated the feasibility of applying computer vision analysis on surgical videos to extract team motion metrics that could differentiate teams with good SA from those with poor SA during the pre-incision time-out. We used a validated observation-based tool to assess SA, and a computer vision software to measure body position and motion patterns in the OR. Our findings showed that it is feasible to extract surgical team motion metrics captured via off-the-shelf OR cameras. Entropy as a measure of the level of team organization was able to distinguish surgical teams with good and poor SA. These findings corroborate existing studies showing that computer vision-based motion metrics have the potential to integrate traditional observation-based performance assessments in the OR.

Defining a Challenge Problem for Cognitive Situation Awareness and Management

The development of shipping is witnessing increasing automation - from existing assistance systems to fully autonomous behaviour. In this article, we present a building block on the way to a fully autonomous passenger ferry for the Kiel Fjord in Germany by presenting a simulation-based approach to situation modelling of maritime environments and the behaviour therein. We show how this can be used for a posteriori analysis of the possible behaviour of the ship. This analysis then in turn flows into the decision-making process. We also use this environment to investigate the application of Deep Reinforcement Learning techniques to optimize navigation tasks and identify challenges and limitations.

The Engineer Research and Development Center (ERDC) has been using ship simulation technology since the 1980s as an engineering tool to assess proposed modifications to federal navigation channels. Some examples of channel modifications that can be assessed in the ship simulator include channel widenings, channel deepenings, turning basin modifications, bend easings, or a combination. The difference between ERDC and other organizations engaged in simulation is ERDC's primary goal: To provide safe, reliable, efficient, effective, and environmentally sustainable waterborne transportation systems for the movement of commerce, national security needs, and recreation.

In this dialogue with expert Dr. Jean Botev, we will first discuss different definitions of "sociotechnical" systems. This includes adding into our models and computational systems, the behavior, constraints on and policies of institutions and cultures, as well as social behaviors and social dynamics that are not institutionalized. Then we will address three hard questions: How does adding social, cultural and institutional information to our computational systems change our goals for and our approaches to these systems? How do we evaluate the success of sociotechnical systems? What are some of the key challenges to building sociotechnical systems?

In the context of the emergency of COVID-19, students have experienced moments of strong emotional stress, with the risk of generating a state of frustration and discouraging them from studying. The proposed adaptive e-learning system, based on situational awareness, and remodeled teaching were essential in limiting this phenomenon. The system has been designed and developed according to the design principles of SA. The feedback selection process, updated through the years to face the emergency situation, is driven by a Fuzzy Cognitive Map, implemented to identify the learners' situation, defined through their levels of engagement, motivation, and participation. The experimentation was conducted using the SAGAT methodology, involving students participating in classes during the courses held over the academic years 2018/2019, 2019/2020, and 2020/2021. The results show that the system is capable of increasing the level of situation awareness of the students even in a context of emergency.

Situation Awareness in health care involves two different sets of concerns that are only exacerbated in a pandemic.

First is the societal: who is infected, how they interact with others (epidemiological). Second is the personal: what happens or will happen to each person (individual). This paper is about the latter.

It describes an approach to building models of disease progression using several advanced mathematical methods, and using existing and arriving measurement data to refine models that use those methods to tune them for individual patients, as the disease is progressing (i.e., not after the fact). The idea is to build models of our initial understanding of the general form of the various progressions of the disease, based on medical expertise, and to use the data we can get to change those models: emphasize some aspects, de-emphasize others, add new elements that become interesting or possibly significant, remove others that have become irrelevant or redundant, and in all of these processes, run millions of ``excursions'', that is, models with slightly different formulations or structures to determine which one(s) better describe the available data and the known science.

This paper describes the technical approach, the purpose and style of modeling we propose, and what we can expect to learn from the application of these methods. The target disease for these first experiments is COVID-19.

An effective public health monitoring system is essential to detecting infectious disease outbreaks in time, i.e., before they spread, cost lives and become difficult to control. In this paper, we propose a novel Health Emergency Monitoring (HEM) system which detects and identifies an emerging health emergency and corresponding infectious disease characteristics from Open Source Data (OSD) using a taxonomy driven actionable knowledge extraction and fuzzy preference argumentation, while correlating the detections about the disease with georeferenced census data to infer any pertinent social, cultural, demographic, economic and geographic indicators of the disease spread. As part of HEM, a probabilistic epidemiological model is used to compute the estimates of disease transmission patterns and other epidemiological outcomes using data from public health records. The HEM system has been tested and validated using relevant COVID-19 and Ebola OSD, proving its feasibility.

The purpose of this short paper is to stimulate a community wide discussion on how situation awareness (SA) technologies may best support the decisions made during pandemics. Last year we discussed how much of SA is directed towards the critical information needed to make epidemiological and public health decisions; as important as this is, an epidemic also requires doctors and front line medical professional to make numerous decisions regarding individualized patient care. This year we emphasize some statistical analyses that better support individualized medicine. We believe that these methods will also support better analyses and understanding of the specific details of individual situations.