Projects

My research interests focus on engineering intelligent and autonomous systems for practical applications and lie at the intersection between different areas: AI, cognitive robotics and intelligent control.

My goal is to create robust, intelligent systems that have an impact on the real world and, to this end, I collaborate extensively with industry and end users. I have built technology for use in real scenarios in several fields: space mission operations, surveillance missions using both aerial and underwater vehicles, disaster response and assistive technology. In all these areas, I develop systems capable of adapting their behaviours to environments that are complex, rapidly changing, and uncertain.

Current Projects @ Royal Holloway

Connect-R: Providing Structure in Unstructured Extreme Environments

(£6m, funded by Innovate UK and in collaboration with Barrnon, Ross Robotics, Tharsus, Jigsaw Structures, UK Atomic Energy Authority and University of Edinburgh)

There are many hostile working environments that require sophisticated tasks to be performed, such as the building of structures and deployment of tools where there is significant risk to the health and safety of any manual workers involved, high cost of deployment and significant timescales for completion. Examples are Nuclear Decommissioning, Oil and Gas, Mining and Space systems. Common to many of these environments is the extreme difficulty of effective deployment of the sophisticated kind of equipment that replaces human beings. These environments present the following challenges:

  • Hazardous working environments requiring protective equipment and limited time windows for operation
  • Limited access through which to deploy the systems
  • Unstable structures present that prevent occupation
  • Lifting of heavy objects (~50kg) that require mechanical assistance
  • Processing of large volumes of liquids (1000s litres)
The Connect-R team propose to develop an industrial-scale, self-building, autonomous, modular robotic solution to provide robotic access to work-sites in these hazardous environments.





MIMRee: Multi-Platform Inspection Maintenance & Repair In Extreme Environment

(£4.2m, funded by Innovate UK and in collaboration with Thales UK, Plant Integrity, Wootzano, ORE Catapult, Royal College of Art, University of Bristol and University of Manchester)

The MIMRee project will introduce a step-change in the Operations and Maintenance of offshore wind farms by removing humans from the loop during the inspection, maintenance and repair (IMR) of offshore wind turbine blades. The aim is to significantly reduce the costs and turbine downtime associated with these tasks and reduce the H&S risks of using rope access technicians. In this project, the multi autonomous platform approach will be demonstrated for a use case in offshore renewables; however, the developed autonomous vehicle surface vessel hub, Human-Machine Interface, robotic teaming and communications, and automated mission planning will also have applications in the offshore Oil & Gas, Search and Rescue and Defense sectors. The key objectives are:

  • Remove the need to send humans offshore to carry out wind turbine blade IMR tasks
  • Remove the need to shut wind turbines down to carry out blade inspections
  • Reduce the risk of using autonomous vehicles offshore to carry out asset IMR tasks
  • Safely demonstrate a fully autonomous approach to blade IMR tasks in real-world operating conditions
  • Establish the business case for using autonomous vehicles for blade IMR
  • Develop a roadmap for transferring the MIMRee system to other relevant industries








Prometheus: Reconfigurable Robotic Platform(s) with Advanced Sensing for Confined Spaces

(£2.2m, funded by Innovate UK and in collaboration with Thales UK, Headlight AI, Callen-Lenz, Network Rail, University of Bristol and University of Manchester)

The Prometheus project will develop a fully autonomous robot capable of geo-technical surveys in unknown voids for use in the mining, water infrastructure monitoring and offshore industries. This robot will be able to be automatically deployed and recovered through a standard restricted access bore of 150mm diameter, significantly increasing potential use cases over existing systems. The joint requirements of fully autonomous operation beyond visual line of sight (BVLOS) combined with deployment through a limited access 150-diameter borehole will be demonstrated both in a university lab environment and at key milestone demonstrations in conjunction with Network Rail. Further applications are within the water industry with aging water infrastructure. Prometheus will be an excellent illustration of robotics, autonomy and AI in extreme environments with widespread applications. The final system will demonstrate a step change in autonomous capability, highly flexible operation and deployment, meeting a real and existing industrial need for rapid inspection of areas that are difficult to access and complex to navigate.




Shared Autonomy via Robust Task Planning and Argumentation (SHARPA)

(funded by EPSRC and in collaboration with MIT)

The overarching objective of this project is to endow autonomous systems with advanced decision-making capabilities and collaboration skills. We aim to build artificial systems that, in real-world environments, are capable of reasoning about high-level goals specified by human operators and formulating, in collaboration with them, a course of actions to successfully achieve such goals. Strategic reasoning and fluid teaming are fundamental skills of cognitive systems: they are needed in a variety of situations, from day-to-day tasks such as assisting humans in household chores, to extreme missions, such as space exploration. The techniques that we propose are general and can be used to support both robotic systems and software agents. We choose disaster response operations where unmanned aerial vehicles (UAVs) assist emergency responders as our demonstration arena. In this domain, in fact, it is crucial for the UAVs to think strategically to pursue goals efficiently and to act in concert with the human operators who are ultimately in charge of critical decisions.

Related Publications:


Extracting Mutual Exclusion Invariants from Lifted Temporal Planning Domains
Sara Bernardini, Fabio Fagnani and David E. Smith
Artificial Intelligence Journal (AIJ) (5-year impact factor: 5.786), Volume 258, Pages 1-65 , May 2018, Elsevier.

Extracting Mutual Exclusion Invariants from Lifted Temporal Planning Domains
Sara Bernardini, Fabio Fagnani and David E. Smith
Proceedings of the Twenty Eight International Conference on Automated Planning and Scheduling (ICAPS-18). Delft, The Netherlands, June 2018.


Autonomous Search-and-Tracking via Multiple Coordinated Drones

(funded by the UK Industrial Strategy Uplift Fund and in collaboration with University of Toronto)

Search and tracking is the problem of locating a moving target and following it to its destination. In this project, we consider a scenario in which the target moves across a large geographical area by following a road network and the search is performed by a team of coordinated unmanned aerial vehicles (UAVs). We formulate search and tracking as a combinatorial optimisation problem and explore different optimisation techniques to solve it, from Greedy Algorithms to AI Planning and Constraint Programming (CP). We also study the formal properties of the objective function, which has lead us into an investigation of submodularity and how this feature can be exploited to improve our algorithms. Via an extensive experimental evaluation, we study the scalability of the different techniques and identifies the conditions under which one approach becomes preferable to the others.

Related Publications:


Autonomous Target Search with Multiple Coordinated UAVs
Chiara Piacentini
, Sara Bernardini and Chris Beck
Submitted to Journal of Artificial Intelligence Research (JAIR).


Past Projects @ Royal Holloway

Automated Plan-Based Policy-Learning for Surveillance Problems

(funded by EPSRC and in collaboration with BAE Systems and King's College London)

In this project, we investigate the use of automated planning technology to elicit high-level intelligent behaviour from autonomous unmanned aerial vehicles (UAVs) engaged in surveillance applications. We focus on search-and-tracking, which is the problem of searching for a mobile target and tracking it once it is found. Since search-and-tracking platforms face many sources of uncertainty and operational constraints, progress in the field has been restricted to simple and unrealistic scenarios. During the course of this project, we have devised a new approach to search-and-tracking that allows us to successfully address large-scale and complex search-and-tracking missions in which the target exhibits an intentional and evasive behaviour. We demonstrated our approach by using two different classes of vehicles, a fixed-wing UAV deployed in simulation and the “Parrot AR.Drone2.0” quadcopter deployed in a physical environment. Our experiments show that our unique way of leveraging deterministic planning to solve the search problem pays off when we tackle complex and realistic missions.

Related Publications:


Combining Temporal Planning with Probabilistic Reasoning for Autonomous Surveillance Missions
Sara Bernardini, Maria Fox and Derek Long
Autonomous Robots (AURO) (5-year impact factor: 4.095) , Vol. 41, No. 1, 01.2017, p. 181-203, Springer.

Boosting Search Guidance in Problems with Semantic Attachments
Sara Bernardini, Maria Fox, Derek Long and Chiara Piacentini

Proceedings of the Twenty Seventh International Conference on Automated Planning and Scheduling (ICAPS-17). Pittsburgh, PA, USA, June 2017.

Deterministic vs Probabilistic Methods for Searching for an Evasive Target
Sara Bernardini, Maria Fox, Derek Long and Chiara Piacentini
Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence (AAAI-17) (Acceptance rate: 25%). San Francisco, CA, February 2017.

Leveraging Probabilistic Reasoning in Deterministic Planning for Large-Scale Autonomous Search-and-Tracking
Sara Bernardini, Maria Fox, Derek Long and Chiara Piacentini
Proceedings of the Twenty Sixth International Conference on Automated Planning and Scheduling (ICAPS-16) (Acceptance rate: 35%). London, UK, June 2016.

In Search of a Lost Target via Monte Carlo Simulation and Automated Planning
Sara Bernardini, Maria Fox and Derek Long
Proceedings of the Sixth Italian Workshop on Automated Planning and Scheduling (IPS- 15). Ferrara, Italy, September 2015.

Planning the Behaviour of Low-Cost Quadcopters for Surveillance Missions
Sara Bernardini, Maria Fox and Derek Long
Proceedings of the Twenty Four International Conference on Automated Planning and Scheduling (ICAPS-14). Portsmouth, NH, USA, June 2014.

Planning the Behaviour of Low-Cost Quadcopters for Surveillance Missions
Sara Bernardini, Maria Fox and Derek Long
Proceedings of the Fifth Italian Workshop on Automated Planning and Scheduling (IPS-13). Torino, Italy, December 2013.

Autonomous Search and Tracking via Temporal Planning
Sara Bernardini, Maria Fox, Derek Long and John Bookless
Proceedings of the Twenty Third International Conference on Automated Planning and Scheduling (ICAPS-13). Rome, Italy, June 2013.

Automated Planning and Policy Learning for Surveillance Missions
Sara Bernardini, Maria Fox and Derek Long
Proceedings of the Eight International Workshop on Planning and Scheduling for Space (IWPSS-13). Moffett Field, CA, USA, March 2013.

Past Projects @ King's College London

Planning Distributed Search Operations

(funded by DSTL (ASUR programme) and in collaboration with Seebyte)

There are multiple contexts in which several diverse vehicles need to be deployed underwater to carry out search for targets (e.g. search and rescue, search and detect or search and track missions). Efficient use of assets in these missions requires planned coordination, sometimes using heterogeneous assets in a cooperative task, as well as sharing responsibilities and coverage of search areas. This project explores the problem of coordinating search underwater when facing intermittent communications. In this context, adaptive planning involves each asset of the search deciding its own activity in the face of prior commitments (both to other agents and from other agents) that underpin the cooperative achievement of the mission goals. We provide robust strategies for replanning, relying on the shared commitments that remain a priority for the agents, which are partners in activities that require coordination or synchronised behaviours.

Related Publications:


Planning Autonomous Underwater Reconnaissance Operations
Sara Bernardini, Maria Fox, Derek Long and Bram Ridder
Proceedings of the Tenth Scheduling and Planning Applications woRKshop (SPARK) -- Twenty Sixth International Conference on Automated Planning and Scheduling (ICAPS-16). London, UK, June 2016.

Past Projects @ UCL Knowledge Lab


Intelligent virtual environment for improving the social communication skills of children with autism

(jointly funded by EPSRC and ESRC)

ECHOES is a serious game built for helping young children with Autism Spectrum Conditions (ASCs) acquire social communication skills. In ECHOES, children interact with an intelligent virtual character in socially realistic situations through a 42 inch multitouch LCD display with eye-gaze tracking. The character, which can act credibly both as a peer and as a tutor, inhabits a three-dimensional sensory garden populated by interactive objects that can change their shape and function when the agent or the child touch them. The interaction between the child and the character is structured around a number of different learning activities intended for real-world use in schools as part of the children's everyday activities. The character's decision making is driven by an autonomous agent based on automated planning techniques.

Related Publications:


Blending Human and Artificial Intelligence to Support Autistic Children’s Social Skills
Kaska Porayska-Pomsta, Alyssa Alcorn, Katerina Avramides, Sara Bernardini, et al.
ACM Transactions on Computer-Human Interaction. In press.

Blending Human and Artificial Intelligence to Support Autistic Children’s Social Skills
Kaska Porayska-Pomsta, Alyssa Alcorn, Katerina Avramides, Sara Bernardini, et al.
Proceedings of the ACM CHI Conference on Human Factors in Computing Systems Glasgow, UK, May 2019.

ECHOES: an Intelligent Serious Game for Fostering Social Communication in Children with Autism
Sara Bernardini, Kaska Porayska-Pomsta, Tim Smith
Information Sciences (5-year impact factor: 4.732) , Elsevier, Volume 264, 2013, pp.41-60.

Planning-Based Social Partners for Children with Autism
Sara Bernardini and Kaska Porayska-Pomsta
Proceedings of the Twenty Third International Conference on Automated Planning and Scheduling (ICAPS-13). Rome, Italy, June 2013.

Designing an Intelligent Virtual Agent for Social Communication in Autism
Sara Bernardini, Kaska Porayska-Pomsta and Harini Sampath
Proceedings of the Ninth Annual AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment (AIIDE-13). Boston, MA, USA, October 2013.

Building an Intelligent, Authorable Serious Game for Autistic Children and Their Carers
Kaska Porayska-Pomsta, Keith Anderson, Sara Bernardini, Karen Guldberg, Tim Smith, Lila Kossivaki, Scott Hodgins and Ian Lowe
Proceedings of the Tenth International Conference on Advances in Computer Entertainment Technology (ACE-13). Enschede, the Netherlands, November 2013.

Building Autonomous Social Partners for Autistic Children
Sara Bernardini, Kaska Porayska-Pomsta, Tim Smith and Katerina Avramides
Proceedings of the Twelfth International Conference on Intelligent Virtual Agents (IVA 2012). Santa Cruz, CA, USA, September 2012.

State-of-the-art in TEL to support social communication skill development in children with autism: A multidisciplinary review
Katerina Avramides, Sara Bernardini, Mary Ellen Foster, Chris Frauenberger, Lila Kossyvaki and Marilena Mademtzi
International Journal of Technology Enhanced Learning. 4(5-6), 2012, pp. 359-372.

Developing Technology for Autism: an Interdisciplinary Approach
Kaska Porayska-Pomsta, Chris Frauenberger, Hellen Pain, Thusha Rajendran, Tim Smith, Rachel Menzies, Mary Ellen Foster, Alyssa Alcorn, Sam Wass, Sara Bernardini, Katerina Avramides, Wendy Keay-Bright, Jingying Chen, Annalu Waller, Karen Guldberg, Judith Good and Oliver Lemon
Journal of Personal and Ubiquitous Computing (5-year impact factor: 2.512), Springer, 16 (2), 2012, pp. 117-127.

Social Communication between Virtual Characters and Children with Autism
Alyssa Alcorn, Helen Pain, Thusha Rajendran, Tim Smith, Oliver Lemon, Kaska Porayska-Pomsta, Mary Ellen Foster, Katerina Avramides, Chris Frauenberger, and Sara Bernardini
Proceedings of the Fiftieths International Conference on Artificial Intelligence in Education (AIED-11). Auckland, New Zealand, June 2011.

Supporting Children's Social Communication Skills through Interactive Narratives with Virtual Characters
Mary Ellen Foster, Katerina Avramides, Sara Bernardini, Jingying Chen, Chris Frauenberger, Oliver Lemon and Kaska Porayska-Pomsta
Proceedings of ACM Multimedia 2010 (ACM-10). Firenze, Italy, October 2010.

ECHOES: Technology-Enhanced Learning for Exploring and Improving Social Interaction Skills
Katerina Avramides, Sara Bernardini, Jingying Chen, Chris Frauenberger, Mary Ellen Foster, Oliver Lemon and Kaska Porayska-Pomsta
Proceedings of the Forth International Conference on Cognitive Systems (CogSys-2010). Zurich, Switzerland, January 2010.

Embodiment as a means for Scaffolding Young Children's Social Skill Acquisition
Kaska Porayska-Pomsta, Sara Bernardini and Thusha Rajendran
Proceedings of the Workshop on Children and Embodied Interaction: Seeking Common Ground, Eighth International Conference on Interaction Design and Children (IDC-09). Como, Italy, June 2009.


Training young Adult’s Regulation of emotions and Development of social Interaction Skills

(unded by the European Union – FP7 ICT 2011-7)

TARDIS aims to build a serious-game simulation platform for young people (aged 18-25) at risk of exclusion to explore, practice and improve their social skills. TARDIS facilitates the interaction through the use of a virtual agent that acts as recruiters in job interviews scenarios. The agent is designed to deliver realistic socio-emotional interactions and acts as a credible, yet tireless interlocutor. TARDIS exploits the unique affordances of digital technology by creating an environment in which the quality and the quantity of emotional display by the agents can be modulated to scaffold the young trainees through a diverse range of possible interview situations. The scenarios are co-designed with experienced practitioners in several European countries in order to ensure their relevance to the different individuals across a number of cultural contexts.

Related Publications:


The TARDIS framework: Intelligent virtual agents for social coaching in job interviews
K. Anderson, E. André, T. Baur, S. Bernardini, M. Chollet, E. Chryssafidou, I. Damian, C. Ennis, A. Egges, P. Gebhard, H. Jones, M. Ochs, C. Pelachaud, K. Porayska-Pomsta, P. Rizzo, and N. Sabouret
Proceedings of the Tenth International Conference on Advances in Computer Entertainment Technology (ACE-13). Enschede, the Netherlands, November 2013.

Modelling Users' Affect in Job Interviews: Technological Demo
Kaska Porayska-Pomsta, Keith Anderson, Ionut Damian, Tobias Baur, Elisabeth Andre, Sara Bernardini and Paola Rizzo
Proceedings of the Twenty First Conference on User Modeling, Adaptation and Personalization (UMAP-13) - System Demonstration Track. Rome, Italy, June 2013. ("Most Participative Demo" award)

Past Projects @ NASA Ames Research Center

EUROPA: Extensible Universal Remote Operations Planning Architecture

EUROPA is a framework to model and tackle problems in planning, scheduling and constraint programming. It includes a plan database, a problem solver and a tool box. The plan database provides support for the storage and manipulation of plans as they are initialised and refined. It integrates a rich representation for actions, states, objects and constraints with powerful algorithms for automated reasoning, constraint propagation and plan manipulation. The problem solver is used to automatically find and fix flaws in the plan database. It can be configured to plan, schedule or both and can be customised to integrate specialised heuristics. The tool box includes a debugger for instrumentation and visualisation of applications and a very high-level, declarative modelling language, NDDL, for describing problem domains and partial-plans.


Related Publications:


Extracting Mutual Exclusion Invariants from Lifted Temporal Planning Domains
Sara Bernardini, Fabio Fagnani and David E. Smith
Artificial Intelligence Journal (AIJ) (5-year impact factor: 5.786), Volume 258, Pages 1-65 , May 2018, Elsevier.

Automatic Synthesis of Temporal Invariants
Sara Bernardini and David E. Smith
Proceedings of the Ninth Symposium on Abstraction, Reformulation and Approximation (SARA-11). Parador de Cardona, Spain, July 2011.

Finding Mutual Exclusion Invariants in Temporal Planning Domains
Sara Bernardini and David E. Smith
Proceedings of the Seventh International Workshop on Planning and Scheduling for Space (IWPSS-11). Darmstadt, Germany, June 2011.

Finding Mutual Exclusion Invariants in Temporal Planning Domains
Sara Bernardini and David E. Smith
Proceedings of the Workshop on Knowledge Engineering for Planning and Scheduling (KEPS), Twenty First International Conference on Automated Planning and Scheduling (ICAPS-11). Freiburg, Germany, June 2011.

Towards Search Control via Dependency Graphs in EUROPA2
Sara Bernardini and David E. Smith
Proceedings of the Workshop on Heuristics for Domain Independent Planning (HDIP), Nineteenth International Conference on Automated Planning and Scheduling (ICAPS-2009). Thessaloniki, Greece, September 2009.

Translating PDDL2.2 into a Constraint-based Variable/Value Language
Sara Bernardini and David E. Smith
Proceedings of the Workshop on Knowledge Engineering for Planning and Scheduling (KEPS), Eighteenth International Conference on Automated Planning and Scheduling (ICAPS-08). Sydney, Australia, September 2008.

Constraint-based Temporal Planning: Domain Modelling and Search Control
Sara Bernardini
Doctoral Consortium, Eighteenth International Conference on Automated Planning and Scheduling (ICAPS-08). Sydney, Australia, September 2008.

Automatically Generated Heuristic Guidance for EUROPA2
Sara Bernardini and David E. Smith
Proceedings of the Ninth International Symposium on Artificial Intelligence, Robotics, and Automation for Space (iSAIRAS-08). Hollywood, USA, February 2008.

Developing Domain-Independent Search Control for EUROPA2
Sara Bernardini and David E. Smith
Proceedings of the Workshop on Heuristics for Domain Independent Planning: Progress, Ideas, Limitations, Challenges, (HDIP), Seventieth International Conference on Automated Planning and Scheduling (ICAPS-07). Providence, RI, USA, September 2007.


IDEA: Intelligent Distributed Execution Architecture

The Intelligent Distributed Execution Architecture (IDEA) is a real-time architecture that exploits AI planning as the core reasoning engine for interacting autonomous agents. Rather than enforcing separate deliberation and execution layers, IDEA unifies them under a single planning technology in so enabling a level of consistency, coordination and flexibility that is not possible in classical, three-layer approaches. The deliberative and reactive planners reason about and act according to a single representation of the past, present and future domain state. A single model unifies the entire domain description: the controlled subsystem and internal control logic, the coordination of control layers (e.g. how to interleave plan execution and deliberation) and the interactions with other agents. IDEA is also unified under the tenets of real-time control. Reactions to events are synchronised to a global clock, and must complete within a minimum time quantum.

Related Publications:


Model-Unified Planning and Execution for Distributed Autonomous System Control
Pascal Aschwanden, Vijay Baskaran, Sara Bernardini, Chuck Fry, Maria D. Moreno, Nicola Muscettola, Chris Plaunt, David Rijsman and Paul Tompkins
Proceedings of the AAAI Fall Symposium on Spacecraft Autonomy: Using AI to Expand Human Space Exploration. Arlington, VI, USA, October 2006.