Note: All times specified in UTC/GMT!

  Friday, May 14 Saturday, May 15 Monday, May 17 Tuesday, May 18 Wednesday, May 19 Thursday, May 20 Friday, May 21 Saturday, May 22
12:45 ‑ 13:00     Conference Opening          
13:00 ‑ 14:00 Tutorial 1: "The Journey from Similar to Plausible Situations: Human and Mathematical Aspects". Tutorial 3: "Introduction to Mission-Centric Cyber Security Situation Management" OS1: Situation Management in Intelligent Transportation Systems OS2: Situation Management in Health Care and Education OS3: Analytical Modeling for Situation Awareness Quality Estimations OS4: Machine Learning for Human-Machine Interaction    
14:00 ‑ 14:15 Workshop 1: Workshop on CogSIMA Challenge Problems Workshop 2: Workshop on Interoperability for Situation Management and Decision-Making
14:15 ‑ 14:25 Break Break Break Break
14:25 ‑ 15:35 K1: Keynote by Ross Anderson, Professor of Security Engineering Cambridge University, UK: "Situational Awareness and Adversarial Machine Learning" K2: Keynote by Susan Stepney, Professor of Computer Science and Director of the York Cross-disciplinary Centre for Systems Analysis, University of York, UK: "Computation as a Dynamical System" K3: Keynote by William D. Casebeer, PhD Director, AI & ML Laboratory, Riverside Research Institute: "Human-Machine Teaming: Evolution or Revolution, and the Ethical Dimensions of Cyborgs" K4: Keynote by Katia Sycara, Edward Fredkin Research Chair in Robotics, School of Computer Science at Carnegie Mellon University, USA: "Towards Theory of Mind for Effective Human-Autonomy Teaming"
15:35 ‑ 15:45 Break Break Break Break
15:45 ‑ 16:30 FS1: Focus Session: Fake News FS2: Focus Session: Psychological Dimensions of Cognitive Situation Management FS3: Focus Session: Large-Scale Emergency Response PS: Poster Session
16:30 ‑ 16:40 Break Break Break
16:40 ‑ 17:00 OS5: Foundations of Cognitive Situation Management
17:00 ‑ 17:25 Tutorial 2: "Getting Support Right: User and Use System Testing using a Work-centered Approach". Tutorial 4: "Interdependence and Vulnerability in Systems: Applying Theory to Define Situations for Autonomous Systems"  
17:25 ‑ 17:30      
17:30 ‑ 18:00       Conference Closing
18:00 ‑ 20:30            

Friday, May 14

Friday, May 14 13:00 - 16:30 (UTC)

Tutorial 1: "The Journey from Similar to Plausible Situations: Human and Mathematical Aspects". Details

Instructor: Prof. Peeter Lorents, Estonian Business School, Estonia

Friday, May 14 16:30 - 17:00 (UTC)


Friday, May 14 17:00 - 20:30 (UTC)

Tutorial 2: "Getting Support Right: User and Use System Testing using a Work-centered Approach". Details

Instructors: Prof. Ann Bisantz, Ph.D. (University at Buffalo, Buffalo, NY) and Dr. Emilie Roth, Ph.D. (Roth Cognitive Engineering, Stanford, CA)

Saturday, May 15

Saturday, May 15 13:00 - 16:30 (UTC)

Tutorial 3: "Introduction to Mission-Centric Cyber Security Situation Management" Details

Instructor: Dr. Gabriel Jakobson, Chief Scientist, CyberGem Consulting, USA

Saturday, May 15 16:30 - 17:00 (UTC)


Saturday, May 15 17:00 - 20:30 (UTC)

Tutorial 4: "Interdependence and Vulnerability in Systems: Applying Theory to Define Situations for Autonomous Systems" Details

Instructor: Prof. William F. Lawless, Paine College, USA

Monday, May 17

Monday, May 17 12:45 - 13:00 (UTC)

Conference Opening

Prof. Leo Motus, Tallinn University of Technology, Estonia

Monday, May 17 13:00 - 14:15 (UTC)

OS1 (Oral Presentation): Situation Management in Intelligent Transportation Systems

Chair: Kenneth P. Baclawski (Northeastern University, USA)
13:00 Near Real-Time Situation Awareness and Anomaly Detection for Complex Railway Environment
Zan Chen, Cheng Wu and Yiming Wang (Soochow University, China)
Oral Presentation
With the development of artificial intelligence and computer vision technologies, the driverless technology of trains has received great attention. How to perceive the driving environment in front of the train and discover potential safety risks are very important issues in unmanned train technology. Aiming at the unmanned driving trains, this paper proposes a near real-time situational awareness and anomaly detection mechanism. In order to realize situation awareness of complex railway environment, deep semantic segmentation is used to extract real-time track area. For anomaly detection, random track obstacles with irregular shapes and types are detected based on the continuity track line. Experimental results show that the trajectory extracted by this method has good continuity and low noise, and it can detect tiny obstacles on straight lines, elbows and turnouts. Our method is proven to be suitable for changing weather conditions and has high practical feasibility.
pp. 1-8
13:25 Smart Elevator with Unsupervised Learning for Visitor Profiling and Personalised Destination Prediction
Mairo Leier, Andri Riid, Tanel Alumäe, Uljana Reinsalu, Rene Pihlak, Andres Udal and Risto Heinsar (Tallinn University of Technology, Estonia); Sven Vainküla (Tallinn University of Applied Sciences, Estonia)
Oral Presentation
Elevator manufacturing companies make great effort to provide better user experience and to reduce elevator waiting time. Deep learning for visitor profiling offers more options for personalised service and enables the elevator to learn about its usage. In this paper, we present a solution that integrates conventional elevator with facial recognition, voice assistant and unsupervised learning and discuss some insights gained during the development. In the 3-month testing period, we experienced different social reactions that help us to examine people's readiness to accept new technologies in their daily life. The novelty of the solution lies in a combination of different cognitive technologies like facial recognition, unsupervised classification of persons, recognition of voice commands and statistics-based prediction of passenger destinations.
pp. 9-16
13:50 Integrated Platform for sUAS Operations in Sensitive Areas with Improved Pilot Situation Awareness
Giulio Avanzini (University of Salento, Italy); Valerio De Luca (Corpo O, Campus Ecotekne, Via Monteroni & University of Salento, Italy); Claudio Pascarelli (University of Salento, Italy)
Oral Presentation
The introduction of small Unmanned Aircraft Systems (sUAS) in an increasingly wide range of civil applications implies several challenges for the integration of manned and unmanned traffic in the controlled and uncontrolled low-level airspace, both at a technological and procedural level. This paper discusses the design of an integrated platform for the management of contingency events and the improvement of situation awareness of the pilot. To this end, a head-up display, based on augmented reality (AR) technology, is designed to provide the pilot with visual cues that should improve his awareness of admissible flight volumes. A force feedback is also provided by a haptic joystick, which suggests manoeuvres to avoid collisions or violation of airspace constraints. Tests in an actual airport facility are expected to be conducted in 2021. In the meantime, a possible realistic scenario is analysed, illustrating the interactions among the various actors and components of the system.
Presenter bio: Valerio De Luca graduated magna cum laude from University of Salento, Italy, in Information Engineering in 2009 and got a PhD in 2014. His past research interests include grid computing, distributed systems for real-time audio-video streaming and QoE. His current research interests deal with human-computer interaction, augmented and virtual reality.
Valerio De Luca
pp. 17-24

Monday, May 17 14:15 - 14:25 (UTC)


Monday, May 17 14:25 - 15:35 (UTC)

K1: Keynote by Ross Anderson, Professor of Security Engineering Cambridge University, UK: "Situational Awareness and Adversarial Machine Learning" Details

Chair: Kellyn Rein (Fraunhofer FKIE, Germany)

Monday, May 17 15:35 - 15:45 (UTC)


Monday, May 17 15:45 - 17:25 (UTC)

FS1 (Oral Presentation): Focus Session: Fake News

Chair: Dorothea Thomas-Aniola (SAIL LABS Technology GmbH, Austria)
15:45 Building a Dynamic Corpus of Fake News Using Commercially Available Machine Translation and NLP Software
George Antoniu Bara (SDL & RWS, Romania)
Oral Presentation
Fake news is a global phenomenon with no language barriers. Online misinformation travels rapidly and in volumes across countries and languages, and tracking it poses a real challenge that can be solved with the use of AI-powered language technologies. Specific fake stories evolve with lifespans that go beyond a year, making it difficult to track the entire event from the beginning and across all its manifestations: when and where did the information originate, in what language? What websites & authors helped the spread of the information and how can a new story be verified against and previous stories on the same topic? A theoretical approach to building a publicly available dataset for fake news stories that is continuously updated and contains semantically comparable multilingual data is described conceptually and from a functional perspective: data collection, processing, analysis, and reporting. The potential use of commercially available Neural Machine Translation and Natural Language Processing (Semantic Comparison) for data processing and analysis purposes is also validated from a technical perspective.
16:10 Fake News in European History
Bettina Biron (University of Vienna, Austria)
Oral Presentation
Bettina Biron Faculty of Informatics University of Vienna The phenomenon of fake news has been with us since the development of the earliest writing systems, or even since information and lie exist. The lecture aims to approach the question of fake news in history, which methods were used and what lessons can we learn from them nowadays in the times of digitalization and information overflow. The contribution focuses on migration, as an example on the case study of Sudeten Germans. Here we can see the transformation of a close relationship between ethnic as well as linguistic groups over centuries into national hatred. News, fake news and propaganda played an essential role in these developments. In the break of culture of National Socialism and the resulting post-war violence, this culminated in the expulsion of the Sudeten Germans from Czechoslovakia. These developments led to friend and foe schemes over generations until today - which are expressed offline as well as online. Keywords - fake news, history, media, social media
16:35 Modus Operandi in Fake News
Edith Huber (Universität für Weiterbildung Krems & Donau-Universität Krems, Austria); Bettina Pospisil (Universität für Weiterbildung Krems, Austria); Wolfgang Haidegger (Kapsch, Austria)
Oral Presentation
With the proliferation of different forms of digital communication, the opportunities for spreading Fake News are increasing constantly. More often than not people inform themselves about news and current events in social networks or blogs. By doing so, they run the risk of falling for and also - often unconsciously - spreading false information from manipulative news sites. The distribution of fake news itself is not a new phenomenon, but thanks to the advances in digital communication their circulation accelerates, leading to major impacts for the whole society. This lecture will take a closer look at several aspects. (a) What exactly are fake news? Which legal and content-related criteria must be fulfilled in order to identify fake news as such? (b) Modus Operandi: What are the typical methods of committing a crime? Moreover, the typical motives for spreading fake news are going to be examined in more detail.
17:00 Challenges for Automatic Detection of Fake News Related to Migration
Werner Bailer (Joanneum Research, Austria); Georg Thallinger (JOANNEUM RESEARCH Forschungsgesellschaft mbH, Austria); Dorothea Thomas-Aniola (SAIL LABS Technology GmbH, Austria); Gerhard Backfried (HENSOLDT Analytics GmbH, Austria)
Oral Presentation
Fake news and misinformation is a widespread phenomenon these days, affecting social media, alternative and traditional media. In a climate of increasing polarization and perceived societal injustice, the topic of migration is one domain that is frequently the target of fake news, addressing both migrants and citizens in host countries. The problem is inherently a multi-lingual and multi-modal one in that it involves information in an array of languages, material in textual, visual and auditory form and often involves communication in a language which may be unfamiliar to recipients or which these recipients only may have basic knowledge of. We argue that semi-automatic approaches, empowering users to gain a clearer picture and base their decisions on sound information, are needed to counter the problem of misinformation. In order to deal with the scale of the problem, such approaches involve a variety of technologies from the field of Artificial Intelligence (AI). In this paper we identify a number of challenges related to implementing approaches for the detection of fake news in the context of migration. These include collecting multi-lingual and multi-modal datasets related to the migration domain, providing explanations of AI tools used in verification to both media professionals and consumers. Further efforts in truly collaborative AI will be needed.

Tuesday, May 18

Tuesday, May 18 13:00 - 14:15 (UTC)

OS2 (Oral Presentation): Situation Management in Health Care and Education

Chair: Mare Teichmann (Tallinn University of Technology, Estonia)
OS2.1 13:00 Cooperative Game for Hospitalization Management Under Pandemic
Alexander Smirnov (SPIIRAS, Russia); Nikolay Teslya (ITMO University, Russia); Nikolay Shilov (SPIIRAS, Russia); Elena Moll (ECMD, Management Training Center "MMK", Russia)
Oral Presentation
The problems of organizing medical care in the context of a pandemic COVID-19 associated with the uncertainty and limited of various resources have led to the need for refining management systems for hospitalizing patients. With the use of situational management, decision making can be improved to better suit the considered situation. Since the conditions for the development of a pandemic are difficult to predict, situational management becomes important. Taking into account psychological and behavioral factors makes the situation model more accurate and realistic. The paper proposes the use of coalition games for situational management during the hospitalization of sick patients. The game players and coalition participants are hospitals, ambulance teams, patients, and computed tomography centers. The goal of the game is to form a coalition of participants that gives the maximum payoff, which is assessed on the basis of minimizing the time to make a decision. The general scheme of hospitalization, the main sources of information about the situation, the formulation and formalization of the problem are considered. The model was tested using the example of the COVID-19 pandemic. Due to the small amount of data and the lack of proven models for the development of the situation, heuristic models of the situation development were formed for conducting experiments, based on the analysis of information from open information sources. The experiment tested the formation of a coalition at the time of hospitalization.
Presenter bio: Head of laboratory "computer aided integrated systems" of St.Petersburg intitute for informatics and automation of Russian Academy of Science and internation laboratory "intelligent technologies for socio-ciberphisical systems" of ITMO University. Graduated Leningrad state polytechnical university in 1979. Doctor of technical science, professor, honored scientist of Russian Federation. Author of more than 300 scientific publications. Research interests covers knowledge management, web services, group decicion support systems, virtual enterprises, supply chain management.
Alexander Smirnov
pp. 25-31
OS2.2 13:25 Enhancing Education in Underserved Schools: The Internet Backpack as Cyber-Physical Infrastructure
Lee McKnight and Danielle Taana Smith (Syracuse University, USA); Wednaud Ronelus (Timothy Dwight Elementary School, USA); Ryan Ondocin and Prasanta Ghosh (Syracuse University, USA)
Oral Presentation
This paper assesses the results of a STEM education pilot project bringing cyber-physical infrastructure for broadband connectivity, trusted devices, and secure cloud and privacy and rights-protecting edge cognitive computing and wireless services, to underserved New York City school children at Timothy Dwight PS 33X in the South Bronx, PS 91 Albany Avenue School and PS 316 Elijah Stroud Elementary School, both in Brooklyn, New York, USA. This pilot study, initiated in 2017, demonstrated how the innovative Internet Backpack could bring immediate connectivity and digital and physical (cyber-physical) resource-sharing including cognitive wireless networks to many school children simultaneously, much faster than previously thought possible, or affordable. Projecting from this successful pilot, in this paper we explain how we anticipate that the results will serve to focus further action by all stakeholders on the broadband underserved wherever they may be. We suggest utilizing the Internet Backpack to develop a gap-filling last few hundred feet road map of where broadband connectivity is otherwise lacking and hence hindering school children's education performance and opportunities to explore STEM learning topics. This model can guide future buildouts of broadband Internet and cyber-physical infrastructure to help address both the Covid-19 pandemic emergency and the ongoing, longstanding systemic societal emergencies exacerbated by limited Internet access in resource-constrained communities. Our initial pilot data shows improvement in both student scientific reasoning and science mastery when uninterrupted Internet connectivity is provided, allowing students to engage in both curricular and extracurricular science projects unimpeded by digital divides. Larger-scale studies, if replicating these results, could guide educators and policymakers towards utilizing cognitive systems such as the Internet Backpacks, and Science/IoT curricula, for efficient cloud to edge connectivity and innovative educational content, changing the equation for greater digital inclusion in urban and rural communities, quickly.
Presenter bio: Lee W.McKnight is an Associate Professor in the iSchool (The School of Information Studies), Syracuse University, Faculty Advisor to the Worldwide Innovation Technology and Entrepreneurship Club (WiTec), and an Affiliate of the Institute for National Security and Counterterrorism (INSTC). Lee lectures annually at MIT since 1998. Lee was Principal Investigator of the 2011 Technology Project of Year TACNY Award-winning National Science Foundation Partnerships for Innovation Wireless Grids Innovation Testbed (WiGiT) project 2009-2014. Lee is inventor of edgeware, a new class of software for creating secure ad hoc overlay cloud to edge applications, services, and Things. Lee's research focuses on blockchain and cloud management of cyber-physically secure dynamic edge services, virtual markets and wireless grids, the global information economy, national and international technology policy, and Internet governance.
Lee McKnight
pp. 32-38
OS2.3 13:50 Towards an AI Coach to Infer Team Mental Model Alignment in Healthcare
Sangwon Seo (Rice University, USA); Lauren R Kennedy-Metz (VA Boston Healthcare System & Harvard Medical School, USA); Marco A Zenati (Harvard Medical School & Brigham & Women's Hospital, USA); Julie Shah (Massachusetts Institute of Technology, USA); Roger Daglius Dias (Brigham and Women's Hospital - Harvard Medical School, USA); Vaibhav V Unhelkar (Rice University, USA)
Oral Presentation
Shared mental models are critical to team success; however, in practice, team members may have misaligned models due to a variety of factors. In safety-critical domains (e.g., aviation, healthcare), lack of shared mental models can lead to preventable errors and harm. Towards the goal of mitigating such preventable errors, here, we present a Bayesian approach to infer misalignment in team members' mental models during complex healthcare task execution. As an exemplary application, we demonstrate our approach using two simulated team-based scenarios, derived from actual teamwork in cardiac surgery. In these simulated experiments, our approach inferred model misalignment with over 75% recall, thereby providing a building block for enabling computer-assisted interventions to augment human cognition in the operating room and improve teamwork.
Presenter bio: I am a PhD student in Computer Science at Rice University. I received BS in Electrical and Computer Engineering and MS in Bioengineering at Seoul National University in South Korea.
pp. 91-96

Tuesday, May 18 14:15 - 14:25 (UTC)


Tuesday, May 18 14:25 - 15:35 (UTC)

K2 (Oral Presentation): Keynote by Susan Stepney, Professor of Computer Science and Director of the York Cross-disciplinary Centre for Systems Analysis, University of York, UK: "Computation as a Dynamical System" Details

Chair: Leo Motus (Estonian Academy of Science, Estonia)

Tuesday, May 18 15:35 - 15:45 (UTC)


Tuesday, May 18 15:45 - 17:25 (UTC)

FS2 (Oral Presentation): Focus Session: Psychological Dimensions of Cognitive Situation Management

Chair: Mare Teichmann (Tallinn University of Technology, Estonia)
15:45 Toward a More Integrated Framework for Psychology and Situation Management
Mare Teichmann (Tallinn University of Technology, Estonia); Gabriel Jakobson (CyberGem Consulting, Inc., USA)
Oral Presentation
In current paper we discuss about six core assumptions underpinning more integrated framework for psychology and situation management (SM). None of these assumptions have received special attention in situation management literature, but each of them is vital in psychological perspective. It worth considering whether and how it is appropriate it integrate discussed issues into future SM studies.
pp. 97-102
16:10 Empirical Study for Measuring the Mental States of Humans During the Interaction with Heavy-Duty Industrial Robots
Mohamad Bdiwi (Fraunhofer Institute for Machine Tools and Forming Technology IWU, Germany); Shuxiao Hou (Fraunhofer IWU, Germany); Lena Winkler (Fraunofer IWU, Germany); Steffen Ihlenfeldt (Fraunhofer IWU, Germany)
Oral Presentation
Industrial robots are widespread in today's production facilities. However, strict safety regulations are required for their operation. Therefore, the robots usually work spatially separated from humans. Only special designs - such as lightweight robots - can currently cooperate directly with humans and actively support them. The use of lightweight robots is nowadays limited to particular assistance tasks. In the case of heavy-duty robots, certain types of cooperation become necessary, where humans and robots can mutually exploit their advantages. Beside the Safety, the human fear is another obstacle in the interaction with heavy-duty robots. The robot is a fully dynamic machine that can move in all directions. Furthermore, by implementing intelligent and autonomous robots, humans could misinterpret the movements of the robot and act in wrong way. These unforeseen movements could endanger the human. This work performs an empirical study supported by a unique platform equipped with various sensors and flexible fusion-tool. The objectives of this study comprise 1. Measuring the mental states of the human objectively and subjectively during the interaction with heavy-duty robots. 2. Investigating the influence of robot parameters (speed, path, interaction level etc.) and various communication systems (e.g. visual and audio systems) on the human mental states in order to reduce the negative emotions during the interaction.
pp. 103-108
16:35 The Impact of New Technologies on Work Design - Case Study of the Work of Robot Operators from One Organisation
Anita Pollak, Mateusz Paliga and Barbara Kożusznik (University of Silesia, Poland)
Oral Presentation
Work design theory research has a long and rich tradition in the industrial and organisational psychology. Work design identifies tasks, activities, relationships, and responsibilities in a specific job. It provides a comprehensive job description useful for examining and creating optimal working conditions, also by developing empowering workers or by restructuring their responsibilities. Human-robot interaction is among the domains with a high demand for work design. It is related to the widespread human fear of being replaced by machines at work, as well as the still small number of publications in this field, mostly devoted to human-robot interaction analysis conditions. Scholars have identified an elaborated set of tasks by a human during working with a robot. From work design perspective, besides expanding what tasks are considered, it is crucial to consider its meaning for creating critical psychological states, which are important for individual and organisational results. Thus, the paper preparation's impetus describes and analyses the tasks carried out by robot operators in a more precise manner, by using psychological work analysis from the work design perspective. The research group consists of 32 robot operators from a large international manufacturing company. The Work Design Questionnaire (WDQ) by Morgeson and Humphrey (2006) was used to describe 21 of work characteristics. Results show that Task Variety, Information Processing, Task Identity, Specialisation, Skill Variety, Problem-solving, Task Significance, Received Interdependence, Feedback From Others, and Feedback from Job are the highest work characteristics of robot operators. It suggests that robot operators distinguish non-routine, cognitive, and interpersonal tasks as more substantial than other tasks. Alike, according to the work design theory, the work characteristics indicated by respondents are recognised as important job-specific factors for workers' critical psychological states i.e., experiencing meaning, feeling responsible for outcomes, and understanding the results of their efforts, and helping shape the context of the work. It is imperative that further and more extensive research is conducted to bring more attention to how work design can support individuals' cognitive, social, and identity development in human-robot interaction.
17:00 Ghost Injection Attack on Automatic Dependent Surveillance-Broadcast Equipped Drones Impact on Human Behaviour
Yazeed Haddad (Tallinn University of Technology, Estonia); Erwin Orye (Tallinn University of Technology & NATO Cooperative Cyber Defence Centre of Excellence, Estonia); Olaf M Maennel (Tallinn University of Technology, Estonia)
Oral Presentation
Automatic Dependent Surveillance-Broadcast (ADS-B) is a communication protocol that is used in aviation for situational awareness by sending out own position and altitude data. ADS-B IN devices are able to receive ADS-B messages. The ADS-B protocol is known to be vulnerable to many cyber attacks. In our work, we simulated a ghost injection attack on drones that are equipped with ADS-B IN equipment to identify its effect on human behaviour. Our aim is to study how users and drone pilots will respond while under attack. To investigate this, we conducted an experiment with 50 participants in a drone simulator where they had to fly a time critical mission. During the mission, an attacker fakes a nearby aircraft by sending out spoofed ADS-B OUT messages to alter the drone's flight path. Participants were interviewed after the mission. Our research was focused on anthropological links between groups of participants: what are the conditions for operators to comply with legislation and land the drone after a proximity warning or ignore legislation and prioritising the mission and continuing the time critical flight? The results of this experiment show that the decision to land (or not) the drone is linked somehow to being Estonian or coming from abroad. About 40\% of the participants were Estonian and all of them complied with the message from the ghost injection attack and landed the drone, while the large majority of the non-Estonian participants continued the mission.

Wednesday, May 19

Wednesday, May 19 13:00 - 14:15 (UTC)

OS3 (Oral Presentation): Analytical Modeling for Situation Awareness Quality Estimations

Chair: Christopher Landauer (Topcy House Consulting, USA)
13:00 Human Reliability Estimation Based on Fuzzy Logic-Modified CREAM Approach
Chao He, Yuan Lum and Kar Lee (University of Duisburg-Essen, Germany); Dirk Söffker (University Duisburg-Essen, Germany)
Oral Presentation
Human reliability is one of the key issues in driver-vehicle systems as human-related accidents accounts for the highest proportion of total accidents. Furthermore, the behaviors of drivers become increasingly essential for driving safety as the driving context is of increasing complexity. Cognitive reliability and error analysis method (CREAM) provides the evaluation method for human reliability in industrial fields, when it is applied to situated context, adaption is required. In this contribution, a modified fuzzy-based CREAM approach is introduced to evaluate human driver reliability in situated driving context using the data collected from driving simulator. Firstly, a new list of common performance conditions (CPCs) characterizing the situated driving context is generated due to the application limits of CPCs in CREAM. Secondly, to determine the levels in the new generated CPCs, fuzzy neighborhood density-based spatial clustering of application with noise (FN-DBSCAN) is applied to driving data defining the membership function parameters, which reduces reliance on expert knowledge and can better characterize human behaviors individually. Next, a new evaluation index, human performance reliability score (HPRS), is proposed for the quantitative and dynamic evaluation of human reliability. The results show that the new proposed method could quantify and evaluate human driver reliability in real time.
pp. 39-44
13:25 An Approach to Gamifying Acquisitions for Assessing Impact on Military Strategy of Nation States
Alicia Ruvinsky (USACE ERDC, USA); Megan Gonzalez (US Army ERDC, USA); James Richards (Engineer Research Development Center, USA); Richard Salter (US Army Engineer Research and Development Center, USA); Joshua Church and Deems McKee (US Army ERDC, USA); Maria Seale (United States Army Engineer Research and Development Center (ERDC) & Information Technology Laboratory, USA)
Oral Presentation
Complex Systems in which humans play a role, namely Human-Integrated Complex Systems (HICS), can be difficult to model or simulate due to the uncertainty introduced by the human component. Traditional modeling approaches such as physics-based modeling do not provide predictive insight towards situation awareness and management. War game designers, and game architects are familiar with HICS problem spaces, and use gamification of such complex contexts as a means of modeling human behavior to inform, predict, and manage an HICS style problem. The game play thereby becomes a means of providing situation awareness and management of the HICS by using human action during game play as a heuristic for pruning the intractable possibility space of the problem at large into a likely probability subspace based on the actions players actually take when playing an HICS game simulation. This paper explores the approach of gamification of real-world HICS problem spaces for situation awareness and management. A gamification methodology is introduced and investigated through the use case of military acquisitions.
pp. 45-53
13:50 Twitter Sentiment Analysis and Political Approval Ratings for Situational Awareness
Navid Shaghaghi and Andres Calle (Santa Clara University, USA); Juan Zuluaga (Universidad Pontificia Bolivariana, Medellín, Colombia); Mubashir Hussain and Yash Kamdar (Santa Clara University, USA); Smita Ghosh (University of Texas at Dallas, USA)
Oral Presentation
The rally around the flag effect is a political science concept used to explain increased, yet short-lived, popular support of a country's government or political leader(s) during periods of crisis such as wars. The effect is teased out through solicited public opinion surveys which reach a limited sub-sample of willing participants and are expensive to conduct, leading to a slow response rate and after-the-fact results more suitable for historical studies than for situational awareness or crisis management. On the other hand, on social media platforms such as Twitter, millions of users provide their unsolicited opinion on almost any topic, including politics. This paper aims to initiate the conversation around the question: Can social media be used to observe the rally around the flag effect in action as it happens and thus help increase situational awareness? This first study utilizes the Twitter social media platform as, at the time of this writing, Twitter is used for political discourse more frequently than other mainstream social media platforms are. Furthermore, Donald Trump, the 45th President of the United States of America was a staunch Twitter user who almost religiously used the platform for communicating to the public, more than and even some times in place of the regular established main stream media channels such as radio and television broadcasts and long standing processes such as white house press conferences and media releases. The study was conducted on tweets from the entire 4 years of Trump's presidency with a focus on the biggest crisis during his presidency: The COVID-19 global pandemic which did lead to the only, and exceptionally short-lived rally around the flag effect for his presidency.

Wednesday, May 19 14:15 - 14:25 (UTC)


Wednesday, May 19 14:25 - 15:35 (UTC)

K3: Keynote by William D. Casebeer, PhD Director, AI & ML Laboratory, Riverside Research Institute: "Human-Machine Teaming: Evolution or Revolution, and the Ethical Dimensions of Cyborgs" Details

Chair: Scott Fouse (Self Employed, USA)

Wednesday, May 19 15:35 - 15:45 (UTC)


Wednesday, May 19 15:45 - 17:00 (UTC)

FS3 (Oral Presentation): Focus Session: Large-Scale Emergency Response

Chair: Kirstie L Bellman (Topcy House Consulting, USA)
15:45 Pandemic Situation Awareness Must Support Both Individualized Medicine and Public Health
Jeffrey Galpin (Providence Specialty Medical Group, USA); Kirstie L Bellman (Topcy House Consulting, USA)
Oral Presentation
The purpose of this short position paper is to stimulate a community wide discussion on how situation awareness (SA) technologies may best support the decisions made during pandemics. Currently much of SA is directed towards the critical information needed to make epidemiological and public health decisions; as important as this is, an epidemic also requires doctors and front line medical professional to make numerous decisions regarding individualized patient care. The question is how can SA contribute to both the rapid collection, analysis, and sharing of critical patient information among medical personnel, and then across all levels of decision-making from individual patients to public health policy?
16:10 Rapid Establishment of Data Visualization Tools for Disaster Response Modeling - Invited
Brandon Randle and Michael Clement (United States Army Engineer Research and Development Center (ERDC), USA); Matthew Little (US Army ERDC, USA); David Jackson (US Army Engineer Research and Development Center (ERDC), USA)
Oral Presentation
As disaster scenarios unfold, flexibility is extremely important. Both tools and people need to rapidly adjust to changing situations. Simple, custom, and lightweight dependency solutions are the ideal position for such circumstances. Off-the-shelf solutions that are not easily modified or adapted can significantly impair making rapid changes and, ultimately, will not provide the response speed necessary to tackle the complexity and uncertainty characteristic of disasters. The result of this effort indicates that flexibility and performance are the two key metrics by which such tools should be gauged.
16:35 Lessons from Leading the ERDC COVID-19 Modeling and Analysis Team - Invited
Brandon J Lafferty (US Army Engineer Research and Development Center, USA)
Oral Presentation
This talk encompasses the successes, challenges, and lessons learned from engaging in a yearlong COVID-19 emergency response effort in the United States and beyond. Fusing advanced computational analytics and applied public health knowledge, pandemic response may be best understood as a 'marathon of sprints.' There are few simple answers, but through sound science and resilient organization, it is possible to tackle the marathon one day and one task at a time.

Thursday, May 20

Thursday, May 20 13:00 - 14:15 (UTC)

OS4 (Oral Presentation): Machine Learning for Human-Machine Interaction

Chair: Andrea Salfinger (Johannes Kepler University Linz, Austria)
13:00 Expression Recognition Based on Multiple Feature Fusion-Based Convolutional Neural Network
Danqing Qian, Liulei Zhou, Yiming Wang and Cheng Wu (Soochow University, China)
Oral Presentation
The process of traditional convolutional neural network to deal with human expression features is often one-way, and it cannot make full use of the features of each layer in the convolutional network to retain valuable facial expression features. This paper proposes an improved convolutional neural network structure with a multi-feature fusion module to optimize facial expression recognition. Within this structure, three convolutional layers are divided into one area module. For each area module, the input of each convolutional layer is composed of the outputs of the previous two convolutional layers. At the same time, a dense connection structure is added between two different area modules to realize feature reuse, thereby relying on feature diversity to improve the accuracy of facial expression recognition. Our experimental results on the CK+ and FER2013 databases show that compared with existing convolutional neural networks, the method gives higher accuracy and lower structure complexity.
pp. 54-60
13:25 Evaluation of Evolutionary Algorithms Under Frugal Learning Constraints for Online Policy Capturing
Alexandre Marois (Thales Research and Technology Canada, Canada); Loïc Grossetête (ENSEIRB-MATMECA, Canada); Bénédicte Chatelais (Thales Research and Technology Canada, Canada); Daniel Lafond (Thales Canada, Canada)
Oral Presentation
Decision making can be modeled in various ways for the design of decision-support systems. One strategy privileged for this purpose is policy capturing, i.e. using statistical techniques (and more recently machine learning) to model judgement policies. The Cognitive Shadow is a prototype tool suited for frugal learning that automatically learns a user's decision pattern in real time based on an ensemble of seven supervised learning algorithms. This tool can provide advisory warnings when the user decision is inconsistent with the predicted outcome. Evolutionary computation methods could reinforce the system's efficiency because of their ability to deal with computational complexity via evolution-inspired optimization mechanisms. The goal of this study was to assess the potential of evolutionary algorithms for frugal learning in an online policy capturing context. To do so, we tested three evolutionary algorithms on three different datasets (each split in three sizes), and compared both their prediction performance and training time with that of the other modeling techniques already implemented in the Cognitive Shadow system. Although all three evolutionary models were generally outperformed by non-evolutionary learning algorithms, one genetic programming method showed good prediction performance for the more complex use cases with the smaller datasets.
pp. 61-67
13:50 Emotion Regulation with Markov Decision Processfor Human-Robot Interaction
Kunye Chen (University of Texas at Austin, USA)
Oral Presentation
In this paper, an Emotion Regulation system with a multi-weighted Markov Decision Process (MDPER system) is created for human-robot interaction. The system is developed to maximize the transfer from negative emotion arousal to positive emotion while minimizing the service spendings in cost and time. To achieve this goal, the system is designed to take personality and emotion/intention degrees as the external input. The fuzzy analytic hierarchy process is used to regulate the weights of input factors. Actions taken by the robot are determined by the reward and punishment functions, which are affected by current personal emotional states, actions as well as the external input. Personality is defined by the 'Big-five test', which has a significant influence on human beings' emotions and actions. For example, one type of big five personalities is Neuroticism (N), which is the personality trait of being emotional. High scorers tend to have high emotional reactions to stress. To regulate each type's emotions, several emotion regulation strategies are used in this paper, including relaxing music, images, and gifts. A 3D grid world is proposed to validate and visualize the value function of the MDPER system. By using the input which simulates two types of people, one with high Neuroticism (N) score and one with a lower Neuroticism score, the 3D grid world simulation shows that the system successfully achieves the goal within three steps. The simulation of the high N scorer shows that in order to achieve the goal, the robot takes two actions determined by the MDPER system in three steps, one step of pictures and two steps of music. For the simulation of the low N scorer, the robot takes two actions determined by the MDPER system in two steps to achieve the goal. Then an experiment with 38 participants was implemented to test the MDPER system. Around 80 percent of participants are satisfied with the robot actions which are determined by the MDPER system. 93 percent of the participants experience a significant emotional improvement after interacting with the robot within three steps.
pp. 68-73

Thursday, May 20 14:15 - 14:25 (UTC)


Thursday, May 20 14:25 - 15:35 (UTC)

K4: Keynote by Katia Sycara, Edward Fredkin Research Chair in Robotics, School of Computer Science at Carnegie Mellon University, USA: "Towards Theory of Mind for Effective Human-Autonomy Teaming" Details

Thursday, May 20 15:35 - 15:45 (UTC)


Thursday, May 20 15:45 - 16:30 (UTC)

PS (Poster Presentation): Poster Session

Chair: Giuseppe D'Aniello (University of Salerno, Italy)
15:45 An End-To-End Track Line Obstacle Detection Framework for Unmanned Rail Transit
Zan Chen, Cheng Wu and Yiming Wang (Soochow University, China)
Poster Presentation
With the development of artificial intelligence and computer vision, the unmanned driving technology of trains has been greatly improved. How to determine the driving environment in front of the train for accurate sensing is a very important issue in the unmanned train technology. Aiming at the unmanned driving of trains, this paper proposes a track region extraction method based on semantic segmentation, and proposes track lines by using image processing algorithm. According to the continuity of track lines, random obstacles with irregular shapes and types are detected. Experimental results show that the track lines extracted by our method show good continuity and less noise. At the same time, it can extract track lines and detect obstacles on straights, bends and forks, and it is suitable for changing weather conditions, with high feasibility and innovation.
16:00 A Situation-Aware DSS to Support Assisted Reproductive Technology Outcome Prediction
Mario Lepore (CORISA, Italy); Antonio Petruzziello (University of Naples Federico II, Italy)
Poster Presentation
Assisted reproductive technology (ART) has been using increasingly over the years as infertility treatment globally among women of different ages since the first baby was born as a result of in vitro fertilization in the UK in 1978. Despite this diffusion, the success rate of assisted reproductive technology is relatively low. The main reasons are poor quality of embryos and eggs, age of the woman, small uterine cavity, immune system disorders and genetic factors such as chromosomal alterations. ART procedures are associated with risk for mothers and infants (e.g., miscarriage, premature delivery, low birth weight infants, and long-term disability among infants). Furthermore, since these treatments are highly expensive and time-consuming, their repetition is not affordable for most infertile couples. An adequate design of a decision support system and its capability to support ART outcome prediction can help infertile couples. In this work, we propose an approach to predict the outcome of ART procedures, based on the identification of the situation of the infertile couple (mainly in terms of physiological parameters), and to provide the doctor with a set of feedback aimed at supporting the doctor's decision to start or continue ART process. A prototypical system is proposed to preliminary verify the feasibility and utility of the approach.
16:15 Situational Decision Making Using Situation Modeling and Deep Learning
Georg Hägele (Husqvarna Group, Sweden); Arezoo Sarkheyli-Hägele (Malmö universitet, Sweden)
Poster Presentation
This paper addresses the problem of situation modeling and machine learning-based decision making in open and non-predictive environments. Situational decision making incorporates the determination of an action based on the current situation, represented by the situation model and trained system behavior using deep neural networks. Commonly, the situation modeling is not considered an intermediate step for decision making in situational action selection. This contribution introduces a novel approach for decision making using situation modeling and deep neural networks. It uses an information structuring and representation technique for the generation of situation spectra used as input to deep learning-based decision making. Simulation-based experimental results show the proposed approach's effectiveness and importance.

Thursday, May 20 16:30 - 16:40 (UTC)


Thursday, May 20 16:40 - 17:30 (UTC)

OS5 (Oral Presentation): Foundations of Cognitive Situation Management

Chair: Peeter Lorents (Estonian Business School, Estonia)
16:40 Situation Awareness, Mental Models, and Understanding
Mare Teichmann (Tallinn University of Technology, Estonia); Leo Motus (Estonian Academy of Science, Estonia)
Oral Presentation
This is an introductory discussion of links between collected situational awareness (e.g. data) and the cognitive processes required to create situation awareness that contributes to accomplish of missions. Basic cognitive processes (sensation, perception, attention, memory), higher cognitive processes (thinking, understanding and prediction) and operator competence are mentioned. The results will be merged under the name of comprehensive situation awareness model.
pp. 74-81
17:05 Stimulating Self-Organization in Human-Machine Collective Intelligence Environment
Alexander Smirnov (SPIIRAS, Russia); Andrew Ponomarev (SPC RAS, Russia)
Oral Presentation
Decision-making in complex situations often requires significant human involvement and remains an activity that is performed jointly by humans and various computational (and AI) tools. This paper develops a concept of human-machine collective intelligence environment for decision support, leveraging self-organization of human-machine teams in decision-support scenarios. Specifically, the paper proposes a set of methods and mechanisms to stimulate self-organization in the human-machine collective intelligence environment: a) team-forming method, b) the method of ensuring the development of the participants, contributing to the fulfillment of the condition of self-organization in the long term. Although the proposed methods and mechanisms are designed, first of all, for the environment being developed, we believe that they can be adapted and therefore are useful for a wide range of applications, supporting collaborative work on complex problems.
pp. 82-90

Thursday, May 20 17:30 - 18:00 (UTC)

Conference Closing

Friday, May 21

Friday, May 21 14:00 - 18:00 (UTC)

Workshop 1: Workshop on CogSIMA Challenge Problems Details

Chairs: Scott Fouse (Self Employed, USA), Giuseppe D'Aniello (University of Salerno, Italy)

The goal of the workshop is to foster the discussion regarding open challenge problems in the field of Cognitive Situation Management, both from the Cognitive Science as well as the Computer Science perspective. The aim of the discussion is to identify and discuss the main, open CogSIMA challenge problems that should be addressed by researchers and practitioners in the near future, paving the road to the organization of a competition on these open problems for the next edition of CogSIMA. The workshop will allow discussing the objectives and motivations of the CogSIMA challenge problems; to identify the main tasks to create one or more challenges for the competition; to recruit volunteer in the community to lead the tasks and to participate in the solution of these problems; to develop a schedule for the organization of the competition. Researchers, scholars, and practitioners from academia, industry, and government interested in the open challenges of Cognitive Situation Management, as well as those who would like to take an active role in the organization of the competition, are warmly invited to attend the workshop and share their ideas.

Saturday, May 22

Saturday, May 22 14:00 - 18:00 (UTC)

Workshop 2: Workshop on Interoperability for Situation Management and Decision-Making Details

Chair: Kenneth P. Baclawski (Northeastern University, USA)

We have seen the wealth of research and applications in CogSIMA 2020 as well as CogSIMA conferences that preceded this one. While these research and development projects have been valuable, the risk is that they will be unconnected "silos", a common problem throughout academics and industry. The purpose of this workshop is to examine the issue of interoperability of situation management and decision making, and to propose possible solutions. This workshop will be a continuation of the workshop held at the end of CogSIMA 2020. The workshop will begin with the presentation of a draft Communiqué that weaves together the many strands of the previous workshop. The intention is for this workshop to collaboratively produce a final Communiqué that will be submitted for publication. The goal is to start a process leading to interoperability and standardization. If you wish to submit a brief position paper to be presented at the workshop, please submit it to


Registration is now open.
Registration fees and info are available here
Author registration deadline is March 31 Please register here

New Conference Dates: May 14-22, 2021.

Mar 26: Prof. Peeter Lorents will instruct a tutorial on The journey from Similar to Plausible Situations: Human and Mathematical Aspects.
Feb 22: Dr. Gabriel Jakobson (CyberGem Consulting, USA) will instruct a tutorial on Introduction to Mission-Centric Cyber Security Situation Management.
Jan 7: Prof. Katia Sycara (Carnegie Mellon University, USA) will present a keynote address. 
Nov 19: Dr. William D. Casebeer (Riverside Research’s Open Innovation Center, USA) will present a keynote address on Human-Machine Teaming: Evolution or Revolution, and the Ethical Dimensions of Cyborgs.
Nov 16: Deadline Extension: Paper submissions are due Jan 11, 2021.
Nov 16: Prof. Susan Stepney (University of York, UK) will present a keynote address on Computation as a dynamical system.
Nov 15: Prof. Ross Anderson (University of Cambridge, UK) will present a keynote address on Situational awareness and adversarial machine learning.
Nov 02: Prof. Ann Bisantz (University at Buffalo, USA) and Dr. Emilie Roth (Roth Cognitive Engineering, USA) will instruct a tutorial on Getting Support Right: User and Use System Testing using a Work-centered Approach.
Oct 21: Prof. William Lawless (Paine College, USA) will instruct a tutorial on Interdependence and vulnerability in systems: Applying theory to define situations for autonomous systems. 
Sep 24: The IEEE CogSIMA official website is online! 



Sponsors and Patrons

IEEE logo
SMC logo

Related Conferences & Organizations