Tutorial Program
Tutorial 1: Detecting, Attributing, and Characterizing Generated and Manipulated Media Using Semantic Forensics
Location: tbd
Abstract: The threat of manipulated media has steadily increased as automated manipulation technologies become more accessible, and social media continues to provide a ripe environment for viral content sharing.
The speed, scale, and breadth at which massive disinformation campaigns can unfold require computational defenses and automated algorithms to help humans discern what content is real and what's been manipulated or synthesized, why, and how.
Through the Semantic Forensics (SemaFor) program, and previously the Media Forensics program, DARPA's research investments in detecting, attributing, and characterizing manipulated and synthesized media, known as deepfakes, have resulted in hundreds of analytics and methods that can help organizations and individuals protect themselves against the multitude of threats of manipulated media.
With SemaFor in its final phase, DARPA's investments have systemically driven down developmental risks - paving the way for a new era of defenses against the mounting threat of deepfakes. Now, the agency is calling on the broader community - including commercial industry and academia doing research in this space - to leverage these investments.
This tutorial walks practitioners through the process of developing a media forensics analytic compatible with the DARPA SemaFor system. Analytics within this ecosystem go beyond simply probabilistic classification, but must provide localized, evidence-based defense of hypotheses related to how a piece of media may have been generated or manipulated. These additional details will become critical as models go from academic exercises to court evidence where the output of these models results in actionable consequences. Practitioners will be given the opportunity to explore the SemaFor interface and see how other models approach the challenge and be guided through the developing and testing tools available to ease the process of interacting with the datamodel.
Intended Audience: Software developers and systems engineers with a specialization in AI will find this tutorial useful in designing, developing, and testing more robust models for the detection, attribution, or characterization of manipulated and generated media. Product Owners and BD Managers will benefit from learning about what goes into building an effective model.
This tutorial is recommended for researchers interested in advancing the state of the art of semantic forensics with an understanding of statistical media forensics or machine learning techniques, although other methods are equally applicable. It will also benefit researchers in being more prepared to compete for potential research grants offered by SemaFor stewards such as Underwriter Laboratories (UL) who is transitioning SemaFor from DARPA.
Instructor's biography: Mr. Michael Kozak is a Senior Software Engineer within Lockheed Martin's Advanced Technology Laboratories (LM ATL). Michael has nearly 20 years of experience managing, designing, and developing technologies related to single and multi-vehicle autonomy, planning and optimization, mission management, contingency management, crewed-uncrewed teaming, and media forensics. His research programs span across DARPA, ONR, ARL, ARO, CTTSO, IARPA, and AFRL, across low TRL basic research programs and high TRL demonstrations on live hardware at major events.
Most recently he worked as the Co-PI on the DARPA Semantic Forensics (SemaFor) program as part of the Systems Integration (TA2) team. This team was responsible for building the software architecture and user interfaces that connect analytics capable of detection, attribution, and characterization of generated and manipulated media across multiple modalities. Michael holds a Master's Degree in Computer Science from Drexel University.
Tutorial 2: Decision Making Under Risk, Uncertainty, and Ignorance
Location: tbd
Abstract: Decision theory, the study of rational choices under risk, has long been dominated by the principle of Maximization of Expected Utility (MEU). Despite its long and successful use in fields such as insurance, resource management, and engineering design, classical MEU Decision Theory has significant limitations since it is not consistently descriptive of human decision-making. The limitations of MEU as a descriptive theory suggest that human decision-makers may distrust decision support systems predicated on it, limiting its real-world usefulness. MEU assumes that all uncertainties in the problem domain be aleatoric rather than epistemic and that both the numerical probabilities and utilities for all decision consequences be specified. Probabilities are estimated based on human opinion and/or data available. Human opinions suffer various biases while available data can be of insufficient quality, unreliable, ambiguous, or irrelevant. Utilization of probability theory for decisions under risk requires taking into account problems with probability estimation.
Decision making under uncertainty and its extreme form, ignorance, is in an opposite epistemic state characterized by the lack of reliable information about the situation of interest. Uncertainty is described in the literature as a state of knowledge when the situation characteristics are known but the likelihood of consequences is not precisely described while ignorance is the state of knowledge when not all the situational characteristics are defined. A particularly important instance in which likelihood of consequences is not reliably known, or may even not be reasonably defined, is the category of decision problems in which, among the universe of discourse, lie events, which are quite rare but highly consequential (grey swan events). To deal with the problems of uncertainty and ignorance as well as various human decision making biases, multiple non-Bayesian decision making models based on non-additive beliefs have been introduced. Among them are Choquet Expected Utility, Cumulative Prospect Theories, and Multiple Quantile Theories, models utilizing fuzzy, possibility, and evidential theories. Consideration of the models of decision making under uncertainty and ignorance is especially important for risk assessment and management of dynamic uncertain and poorly characterized situations.
The tutorial will survey the ideas and methods of decision making under risk, uncertainty, and ignorance, with a focus on recent advances in non-Bayesian models. Examples of decision making and action selection in dynamic uncertain situations will be considered.
Intended Audience: This tutorial is intended for both researchers and practitioners from a wide variety of fields such as communication, intelligence, business processes, health care, and finance, who are interested in understanding of problems of decision making in highly uncertain dynamic environments and building methods for dealing with these problems.
Instructor's biography: Dr. Rogova is a research professor at the State University at Buffalo as well as an independent consultant (DBA Encompass Consulting). She is a recognized expert in information fusion, decision making, machine learning and information quality. Her other research expertise includes reasoning under uncertainty, and image understanding. She has worked on a wide range of defense and non-defense problems such as decision making involving low probability high consequence events, situation and threat assessment, information quality in information fusion, computer-aided diagnosis, and understanding of volcanic eruption patterns, among others. Dr. Rogova published numerous papers and lectured internationally on these subjects. She co-edited 7 books including "Meeting Security Challenges Through Data Analytics and Decision Support", "Fusion Methodologies in Crisis Management: Higher Level Fusion and Decision Making", and "Information Quality in Information Fusion and Decision Making." Dr. Rogova has been a tutorial lecturer and a member of organizing committees of multiple conferences on information fusion and decision support.
Location: tbd
Abstract: This tutorial will introduce the main modelling methods, paradigms and techniques involved in the design of cognitive artificial systems. Students will understand the theoretical and technical challenges involved in modelling and building systems that can reason, solve problems, acquire and use knowledge and make decisions in real-life environments. A particular emphasis will dedicated to the Cognitive Architectures: software adopted in a variety of fields (ranging from robotics to video games) explicitly implementing human-like heuristics and decision procedures coming from the experimental results of the cognitive science.
Participants will learn how the design of biologically and cognitively-inspired computational models can be used to help us in understanding human intelligence and in making progress towards more advanced AI systems.
Table of Contents of the Tutorial:
- Introduction to cognitive artificial systems and difference with standard AI technologies
- Design Paradigms in Cognitive Systems: from the Cybernetics to the origins of Artificial Intelligence (AI)
- Modelling Paradigms: Cognitivism and computationalism, connectionism, dynamical systems and enactivism
- Unified theories of cognition: Cognitive Architectures
- Role of a cognitive architecture in AI and Biological/Cognitive Modelling
- Design characteristics of cognitivist and emergent architectures
- Example of cognitive architectures (e.g. SOAR, ACT-R, LIDA, CLARION, and others)
- Towards a Computationally Grounded Standard/Common Model of the Mind
- Differences between Cognitive Artificial Systems and the so-called "Cognitive Computing" Systems
- Integration strategies of human-like computational systems in cognitive architectures
- Open problems in AI and Cognitive Systems communities
Among the questions subject to discussion are: What characterizes a cognitively inspired AI system? What are examples of cognitively inspired AI systems? What is the explanatory power of AI systems compared to AI systems designed with a machine-oriented approach? What can the Cognitive Design approach add to the development of the next generation of AI systems (going beyond deep learning and generative AI)? The course aims delve deeply on these themes by presenting different case studies showing the potential of the Cognitive Design Approach when applied to the realization of intelligent technologies.
Intended Audience: The tutorial is mainly targeted towards Ph.D. Students, post docs and and early-career researchers in the areas of Artificial Intelligence, Computational Cognitive Science, Cognitive and Social Robotics.
Selected Bibliography
Antonio Lieto, (2021) Cognitive Design for Artificial Minds, Routledge Taylor & Francis.
Brendon Lake, Tomer Ullman, Joshua Tenenbaum, & Gershman, S. J. (2017) Building machines that learn and think like people, Behavioral and Brain Sciences.
David Vernon, (2014) Artificial Cognitive Systems: A Primer, MIT Press.
Allen Newell (1990), Unified Theories of Cognition, Harvard University Press.
Instructor's biography:
Antonio Lieto, Ph.D., is an Associate Professor in Computer Science at the University of Salerno (Italy), where he leads the Cognition, Interaction and Intelligent Technologies Laboratory (CIIT Lab), and a researcher at the ICAR-CNR Institute in Palermo (Italy). Previously he was post-doc and tenure-track researcher in Artificial Intelligence and Cognitive Systems at the Department of Computer Science of the University of Turin (2012-2023).
He is currently an elected Member of the Scientific Committee of the Italian Association for Artificial Intelligence (AI*IA), an ACM Distinguished Speaker on the topics of cognitively inspired AI, a member of the IEEE Technical Committee on Cognitive Robotics and an Associate Editor for the journal Cognitive Systems Research (Elsevier). He regularly serves in the PC of the main AI and Cognitive Science conferences (IJCAI, ECAI, AAAI, ACL, AAMAS, IROS, COGSCI).
Previously, he has been Vice-President of the Italian Association of Cognitive Sciences (AISC, 2017-2022), the recipient of the "Outstanding BICA Research Award" from the Biologically Inspired Cognitive Architecture Society (USA) and Deputy-Editor in Chief of the Journal of Experimental and Theoretical Artificial Intelligence (JETAI, Taylor & Francis, 2018-2023).
He was also (2016-2017) Research Associate and Scientific Consultant at the MEPhI (National Research Nuclear University, Moscow, Russia) and a Visiting Researcher at the University of Haifa (Israel), Carnegie Mellon University (USA) and Lund University (Sweden).
His research interests are at the intersection of Artificial Intelligence, Cognitive Science and Human-Machine Interaction and focuses on the following areas: Knowledge Representation and Automated Reasoning, Commonsense Reasoning, Semantic/Language Technologies, Cognitive Systems and Architectures for interactive intelligent agents, Persuasive Technologies. On these topics he has published more than 100 papers in top-tier peer-reviewed international conferences and journals and the book "Cognitive Design for Artificial Minds" (Routledge/Taylor & Francis, 2021).
Tutorial 4: Neuroergonomics for Situation Awareness: Applying Wearable Neurotechnologies in Complex, Real-world Environments
Location: tbd
Abstract: This tutorial introduces recent advancements in neuroscience and neuroengineering that have significantly expanded the accessibility and utility of mobile, wearable neurotechnologies, enabling real-time recording and modulation of brain activity in natural, everyday environments. Neuroergonomics, an emerging interdisciplinary field, leverages these innovations to explore the intricate relationship between brain function and behavior in real-world contexts. Portable neuroimaging sensors and non-invasive neurostimulation techniques allow researchers to study perceptual, cognitive, and motor functions with unprecedented precision and ecological validity. These technological advances provide valuable insights that inform the design of next-generation systems and technologies, seamlessly integrating with human users by aligning with cognitive and physiological processes. The application of neuroergonomic principles has the potential to enhance how we communicate, learn, work, and engage with technology, leading to more intuitive human-system interactions and enhanced human performance and experience. Neuroergonomics could provide vital information on what individuals are paying attention to and responding to in real-life situations and how well individuals are coordinating when teaming in complex realistic and real-world situations.
Intended Audience: This tutorial is intended for researchers, engineers, practitioners, and students in cognitive science, neuroscience, neuroengineering, human factors, human-computer interaction, artificial intelligence, systems engineering, and related fields. Professionals involved in designing, developing, or evaluating systems that require an understanding of human cognitive and physiological states in real-world scenarios will particularly benefit from attending. No prior specialized knowledge of neuroimaging or neurostimulation techniques is required; however, familiarity with cognitive and computational approaches to human-system integration will be advantageous.
Hasan Ayaz, PhD, is a Provost Solutions Fellow and Associate Professor at Drexel University's School of Biomedical Engineering, Science, and Health Systems, bringing expertise at the intersection of neuroscience, engineering and biomedical technologies. He holds affiliations with the Department of Psychological and Brain Sciences, AJ Drexel Autism Institute, and Solutions Institute of Drexel University; affiliated faculty at Applied Biomedical Engineering at Whiting School of Engineering of Johns Hopkins University, and an Associate Fellow at the Center for Injury Research and Prevention of Children's Hospital of Philadelphia.
With a professional history spanning over two decades, Dr. Ayaz has dedicated his career to advancing the field of miniaturized continuous wave near-infrared spectroscopy systems tailored for neuroimaging. His work has resulted in the development of multiple brain monitoring systems, data collection, processing, and analysis methods widely employed globally in clinical and field research across academic institutions, government agencies, and corporate laboratories. Notably, he spearheaded the software development of the novel medical device, Infrascanner, the world's first mobile optical-brain-monitoring tool for traumatic brain injury. This handheld instrument, utilizing near-infrared technology, has proven instrumental in the early detection of hematomas in head trauma patients and is currently deployed in 42 countries across 6 continents, serving both civilian and military hospitals.
Dr. Ayaz's research is centered around Neuroergonomics, the study of brain function in everyday life tasks. His studies include both method development and their applications at the intersection of cognitive neuroscience and biomedical engineering, focusing on unraveling the complexities of brain functioning through mobile neuroimaging. His focus is on human-machine systems, particularly the role of human attention, memory, and vigilance in automated and robotic systems. He has conducted studies using information processing, functional neuroimaging (fNIRS, EEG, and fMRI), and brain stimulation methods (tDCS and rTMS), both in healthy young adults and specialized experts.
His work encompasses diverse real-world and realistic environments and spans the entire lifespan, addressing conditions from health to clinical, through collaborative efforts with domain experts and clinical partners with over 400 publications. His research group designs and develops next-generation brain imaging technologies and neuro/physiological data analytic approaches with applications ranging from aerospace to healthcare. His contributions include organizing and chairing international conferences on this subject, specifically the International Neuroergonomics Conferences, and co-founding Field Chief Editor of the journal "Frontiers of Neuroergonomics," dedicated to advancing mobile neurotechnology methods, interdisciplinary applications, and a home for Neuroergonomics research.
He received a B.S. in Electrical and Electronics Engineering from Boğaziçi University, Istanbul, Türkiye, and a Ph.D. in Biomedical Engineering (Neural Engineering) from Drexel University, Philadelphia, PA, USA.
