Over the last several years, end-to-end neural conversational agents have vastly improved in their ability to carry a chit-chat conversation with humans. However, these models are often trained on large datasets from the internet, and as a result, may learn undesirable behaviors from this data, such as toxic or otherwise harmful language.  In this talk, I will discuss the problem landscape for safety for E2E convAI, including recent and related work. I will highlight tensions between values, potential positive impact, and potential harms, and describe a possible path for moving forward.

* Applying Strategic Reasoning for Accountability Ascription in Multiagent Teams, Vahid Yazdanpanah, Sebastian Stein, Enrico Gerding and Nicholas R. Jennings.
* Impossibility of Unambiguous Communication as a Source of Failure in AI Systems, William Howe and Roman Yampolskiy.
> Debate Panel - Session Discussants: Seán Ó hÉigeartaigh, Gabriel Pedroza

* Uncontrollability of Artificial Intelligence, Roman Yampolskiy.
* Domain Shifts in Reinforcement Learning: Identifying Disturbances in Environments, Tom Haider, Felippe Schmoeller Roza, Dirk Eilers, Karsten Roscher and Stephan Günnemann.
* Chess as a Testing Grounds for the Oracle Approach to AI Safety, James Miller, Roman Yampolskiy, Olle Häggström and Stuart Armstrong.
* Socio-technical co-Design for accountable autonomous software, Ayan Banerjee, Imane Lamrani, Katina Michael, Diana Bowman and Sandeep Gupta.

The field of Artificial Intelligence (AI) is going through a period of great expectations, introducing a certain level of anxiety in research, business and also policy. This anxiety is further energised by an AI race narrative that makes people believe they might be missing out. Whether real or not, a belief in this narrative may be detrimental as some stake-holders will feel obliged to cut corners on safety precautions, or ignore societal consequences just to "win". In our recent works, we examine this problem theoretically, resorting to a novel innovation dilemma where technologists can choose a safe (SAFE) vs risk-taking (UNSAFE) course of development. Companies are held to race towards the deployment of some AI-based product in a domain X. They can either carefully consider all data and AI pitfalls along the way (the SAFE ones) or else take undue risks by skipping recommendable testing so as to speed up the processing involved (the UNSAFE ones). Overall, SAFE are costlier strategies and take more time to implement than UNSAFE ones, therefore permitting UNSAFE strategists to claim significant further benefits from reaching technological supremacy first. We show that the range of risk probabilities where the social dilemma arises depends on many factors, the most important among them are the time-scale to reach supremacy in a given domain (i.e. short-term vs long-term AI) and the speed gain by ignoring safety measures. Moreover, given the more complex nature of this scenario, we show that incentives such as reward and punishment (for example, for the purpose of technology regulation) are much more challenging to supply correctly than in case of cooperation dilemmas such as the Prisoner's Dilemma and the Public Good Games.   These results are directly relevant for the design of governance and regulatory policies that aim to ensure an ethical and responsible AI technology development process.

* Assessing the Reliability of Deep Learning Classifiers Through Robustness Evaluation and Operational Profiles, Xingyu Zhao, Wei Huang, Alec Banks, Victoria Cox, David Flynn, Sven Schewe and Xiaowei Huang [Best Paper Award].
* Towards robust perception using topological invariants, Romie Banerjee, Feng Liu and Pei Ke.
* Measuring Ensemble Diversity and Its Effects on Model Robustness, Lena Heidemann, Adrian Schwaiger and Karsten Roscher
> Debate Panel - Session Discussants: Fabio Arnez, José Hernández-Orallo

Assuring the safety of autonomous driving is a complex endeavour. It is not only a technical difficult and resource intensive task but autonomous vehicles and their wider sociotechnical context demonstrate characteristics of complex systems in the stricter sense of the term. That is, they exhibit emergent behaviour, coupled feedback, non-linearity and semi-permeable system boundaries. These drivers of complexity are further exacerbated by the introduction of AI and machine learning techniques. All these factors severely limit our ability to apply traditional safety measures both at design and operation-time.
 
In this presentation, I present how considering AI-based autonomous vehicles as a complex system could lead us towards better arguments for their overall safety. In doing so, I address the issue from two different perspectives. Firstly by considering the topic of safety within the wider system context including technical, management, and regulatory considerations. I then discuss how these viewpoints lead to specific requirements on AI components within the system. Residual inadequacies of machine learning techniques are an inevitable side effect of the technology. I explain how an understanding of root causes of such insufficiencies as well as the effectiveness of measures during design and operation is key to the construction of a convincing safety assurance argument of the system. I will finish the talk with a summary of our current standardisation initiatives in this area as well as directions for future research.

*  Deep neural network loses attention to adversarial images, Shashank Kotyan and Danilo Vasconcellos Vargas.
* An Adversarial Attacker for Neural Networks in Regression Problems, Kavya Gupta, Jean-Christophe Pesquet, Beatrice Pesquet-Popescu, Fateh Kaakai and Fragkiskos Malliaros.
* Coyote: A Dataset of Challenging Scenarios in Visual Perception for Autonomous Vehicles, Suruchi Gupta, Ihsan Ullah and Michael Madden.
> Debate Panel - Session Discussants: Mauricio Castillo-Effen, José Hernández-Orallo

* Epistemic Defenses against Scientific and Empirical Adversarial AI Attacks, Nadisha-Marie Aliman and Leon Kester.
* On the Differences between Human and Machine Intelligence, Roman Yampolskiy.
* A Mixed Integer Programming Approach for Verifying Properties of Binarized Neural Networks, Christopher Lazarus and Mykel Kochenderfer. 

There is a huge effort towards establishing the methodologies for engineering AI-based systems for safety-critical applications. The automated driving community is highlighting the importance of simulation for virtual development. The correct operation of the systems relies on the correct operation of the tools that are used to create them. Accordingly, the safety of AI-based systems relies on the simulations that are used in their development. We need methods to qualify simulations to be used in the development of safety-critical AI-based systems. Tool qualification requirements have already been established for various safety-critical domains. However, the methods and guidelines for applying these requirements in the simulation engineering life cycle are still missing. This talk proposes a simulation qualification approach, particularly for aviation applications, based on the IEEE Recommended Practice for Distributed Simulation Engineering and Execution Process (DSEEP) and DO-330/ED-215 Software Tool Qualification Considerations.

* Building a safety argument for hardware-fault tolerance in convolutional neural networks using activation range supervision, Florian Geissler, Syed Qutub, Sayanta Roychowdhury, Ali Asgari, Yang Peng, Akash Dhamasia, Ralf Graefe, Karthik Pattabiraman and Michael Paulitsch.
* Artificial Intelligence for Future Skies: On-going standardization activities to build the next certification/approval framework for airborne and ground aeronautic products, Christophe Gabreau, Béatrice Pesquet-Popescu, Fateh Kaakai and Baptiste Lefevre.
* [No recording as per Speaker request] Using Complementary Risk Acceptance Criteria to Structure Assurance Cases for Safety-Critical AI Components, Michael Klaes, Rasmus Adler, Lisa Jöckel, Janek Groß and Jan Reich.
> Debate Panel - Session Discussants: John McDermid, Huascar Espinoza