Abstract

Perception, which involves organization, identification, and interpretation of sensory streams, has been a long-standing problem in robotics, and has been rapidly promoted by modern deep learning techniques. Traditional research in this field generally lies in single-robot scenarios, such as object detection, tracking, and semantic/panoptic segmentation. However, single-robot perception suffers from long-range and occlusion issues due to the limited sensing capability and dense traffic situations, and the imperfect perception could severely degrade the later planning and control modules.

Collaborative perception has been proposed to fundamentally solve the aforementioned problem, yet it is still faced with challenges including lack of real-world dataset, extra computational burden, high communication bandwidth, and subpar performance in adversarial scenarios. To tackle these challenging issues and to promote more research in collaborative perception and learning, this workshop aims to stimulate discussion on techniques that will enable better multi-agent autonomous systems with an emphasis on robust collaborative perception and learning methods, perception-based multi-robot planning and control, cooperative and competitive multi-agent systems, and safety-critical connected autonomous driving.

In line with the ICRA 2023 Making Robots for Humans theme, this workshop will provide a venue for academics and industry practitioners to create a vision for connected robots to promote the safety and intelligence for humans. The half-day workshop will feature presentations by distinguished speakers as well as interactive activities in the form of poster sessions and panel discussions.

Invited Speakers

Mac Schwager

Stanford

Multi-robot systems, distributed estimation

Giuseppe Loianno

NYU

Multi-robot perception, swarm robotics

Yu Wang

Tsinghua University

Multi-agent exploration, efficient DL

Peter Stone

UT Austin

Machine learning, multiagent systems, and robotics

Fei Miao

UConn

Connected and autonomous vehicles (CAVs)

Bolei Zhou

UCLA

Interpretable human-AI interaction

Invited Speakers

Mac Schwager

Stanford

Multi-robot systems, distributed estimation

Giuseppe Loianno

NYU

Multi-robot perception, swarm robotics

Yu Wang

Tsinghua University

Multi-agent exploration, efficient DL

Peter Stone

UT Austin

Machine learning, multiagent systems, and robotics

Fei Miao

UConn

Connected and autonomous vehicles (CAVs)

Bolei Zhou

UCLA

Interpretable human-AI interaction

Extended Abstract Submission

We invite researchers working on related topics to submit abstracts or extended abstracts (no longer than 4 pages in ICRA paper format, including references; you may add appendix after references) that can contribute to this workshop. The accepted extended abstracts will be publicly available on this workshop website until the end of ICRA'23.

Note: we DO allow previously published papers to be presented in this workshop, because the accepted extended abstracts in this workshop will NOT be published in the ICRA'23 proceeding.

Desired Works could:

  • identify novel collaborative perception algorithms for outdoor or indoor robotics,
  • discuss multi-agent systems in the context of applications in autonomous driving, human-robot interaction, or unmanned aerial vehicles,
  • demonstrate multi-robot communication efficiency,
  • describe novel perception-based multi-robot planning methods such as collaborative visual navigation or exploration,
  • review and benchmark various methods proposed by different communities (e.g., robotics, computer vision, transportation) with the ultimate goal to enhance the mutual understanding of challenges and opportunities related to this workshop.

Important Dates


Extended Abstract Submission (send to coperception.icra2023@gmail.com):

May 7, 2023, 11:59PM PDT.

Extended Abstract Acceptance:

May 14, 2023, 11:59PM PDT.

Final Version Submission:

May 21, 2023, 11:59PM PDT.



Best Paper Awards (Sponsored by IEEE RAS TC for Computer & Robot Vision)


First Prize

$150

Second Prize

$100

Third Prize

$50

Topics of Interest

  • Collaborative perception (detection, segmentation, tracking, motion forecasting, etc.)
  • Communication-efficient collaborative perception
  • Robust collaborative perception (latency / pose errors)
  • Collaborative embodied AI
  • Representation learning in multi-agent systems
  • Adversarial learning in multi-agent perception
  • Vehicle-to-vehicle (V2V) and vehicle-to-infrastructure (V2I)
  • Connected and autonomous vehicles (CAVs)
  • Intelligent transportation systems
  • Smart cities
  • Multi-robot systems and swarm systems
  • Multi-robot exploration and mapping
  • Distributed optimization
  • Efficient large-scale collaborative learning
  • Edge AI and federated learning
  • Cooperative and competitive multi-agent systems
  • Simulation for multi-agent learning
  • Dataset and benchmarking for collaborative perception and learning

Program (Friday 2nd June 2023)

Time in London Talk in South Gallery Room 25
08:45 – 08:50    Welcome / Introductions   
08:50 – 09:15    Distributed Models and Representations for Robot Collective Intelligence, Giuseppe Loianno
  • Abstract: In this talk, I will discuss recent techniques to address fast distributed multi-vehicle perception, planning, and control problems to increase situational of each member of the team as well as robustness, adaptation, and resilience with respect to robots’ and/or sensors’ failures and interaction with humans with limited of absence of communication. Specifically, by combining learning-based and physics-based techniques it is possible to achieve collaborative and shared autonomy tasks that address a wide range of problems such as search and rescue, monitoring, construction, and transportation.
09:15 – 09:40    Learning and Control for Safety, Efficiency, and Resiliency of Cyber-Physical Systems, Fei Miao
  • Abstract: The rapid evolution of ubiquitous sensing, communication, and computation technologies has contributed to the revolution of cyber-physical systems (CPS). Learning-based methodologies are integrated to the control of physical systems and provide tremendous opportunities for AI-enabled CPS. However, existing networked CPS decision-making frameworks lack understanding of the tridirectional relationship among communication, learning and control. It remains challenging to leverage the communication capability for the learning and control methodology design of CPS, to improve the safety, efficiency, and robustness of the system. In the talk, we will present our research contributions on learning and control with information sharing for networked CPS. We design the first uncertainty quantification method for collaborative perception of connected autonomous vehicles (CAVs) and show the accuracy improvement and uncertainty reduction performance of our method. To utilize the information shared among agents, we then develop a safe and scalable deep multi-agent reinforcement learning (MARL) algorithms to improve system safety and efficiency. We validate the benefits of communication in MARL especially for CAVs under challenging mixed traffic scenarios. To motivate agents to communicate and coordinate, we design a novel stable and efficient Shapley value-based reward reallocation scheme for MARL. Finally, considering the complicated system dynamics and state information uncertainties from sensors and learning-based perception of networked CPS, we present our contribution to robust MARL methods, including formal analysis on the solution concept of MARL under state uncertainties and state perturbations.
09:40 – 10:05    MetaDriverse: Simulating Digital Twins of Real-World Traffic Scenarios for AI Safety, Bolei Zhou
  • Abstract: Autonomous driving (AD) powered by AI is an emerging technology that revolutionizes mobility and transportation. However, it remains difficult to ensure the AI safety when driving in a wide range of complex real-world situations. To tackle this, driving simulation platform becomes a stepping stone for evaluating AD systems. In this case, diverse and realistic traffic scenarios that reflect the real-world complexity are crucial for evaluating the AI safety in simulation. I will introduce our effort of building the MetaDrivese platform to manage and simulate more than 1 million different traffic scenarios. This platform can import HD maps and replay real-world vehicle trajectories as well as learning to generate novel ones. It can substantially improve the realism and diversity of the traffic scenarios in simulation as well as thorougly evaluating the decision-making and AI safety of AD systems. Recent progress of MetaDriverse platform is available at https://metadriverse.github.io/.
10:05 – 10:35    Panel Discussion
  • If you have any topic related to coperception you’d like to propose to discuss, please send your proposed question directly to coperception.icra2023@gmail.com.
10:45 – 11:10    Coopernaut: End-to-End Driving with Cooperative Perception for Networked Vehicles, Peter Stone
  • Abstract: Optical sensors and learning algorithms for autonomous vehicles have dramatically advanced in the past few years. Nonetheless, the reliability of today's autonomous vehicles is hindered by the limited line-of-sight sensing capability and the brittleness of data-driven methods in handling extreme situations. With recent developments of telecommunication technologies, cooperative perception with vehicle-to-vehicle communications has become a promising paradigm to enhance autonomous driving in dangerous or emergency situations. We introduce COOPERNAUT, an end-to-end learning model that uses cross-vehicle perception for vision-based cooperative driving. Our model encodes LiDAR information into compact point-based representations that can be transmitted as messages between vehicles via realistic wireless channels. To evaluate our model, we develop AUTOCASTSIM, a network-augmented driving simulation framework with example accident-prone scenarios. Our experiments on AUTOCASTSIM suggest that our cooperative perception driving models lead to a 40% improvement in average success rate over egocentric driving models in these challenging driving situations and a 5x smaller bandwidth requirement than prior work V2VNet.
11:10 – 11:35    Applications of Distributed Optimization in Multi-robot Systems, Mac Schwager
  • Abstract: Distributed optimization is are an expressive and powerful optimization paradigm that allows for principled translation of single robot tasks to the multi-robot domain. This talk centers on how teams of connected robots can leverage distributed optimization algorithms to perform collaborative estimation and learning.
11:35 – 12:00    Collaborative multi-robot exploration: fundamental technology and applications, Yu Wang
  • Abstract: With the advancement of individual agent capabilities, collaboration between multiple agents has become possible. Compared to single-agent intelligence, multi-agent collaborative intelligence has a larger perception range of the environment, stronger action capabilities, and can further improve system efficiency. However, multi-agent systems face challenges such as resource limitations in communication, perception, data, and computation. To address these challenges, research needs to be conducted in communication systems, computation systems, collaborative perception and decision-making algorithms. This report focuses on the collaborative multi-robot exploration under resource-limited conditions and showcases the research team's achievements in collaborative mapping and exploration systems in unknown environments with communication limitations, as well as adaptive multi-task decision-making algorithms.
12:00 – 12:30    Presentations of Workshop Papers
  • Jan Blumenkamp, Qingbiao Li, Binyu Wang, Zhe Liu, and Amanda Prorok (University of Cambridge). See What the Robot Can’t See: Learning Cooperative Perception for Visual Navigation. [PDF]
  • Nathaniel Moore Glaser and Zsolt Kira (Georgia Tech). We Need to Talk: Identifying and Overcoming Communication-Critical Scenarios for Self-Driving. [PDF]
  • Arash Asgharivaskasi and Nikolay Atanasov (UCSD). Distributed Optimization with Consensus Constraint for Multi-Robot Semantic Octree Mapping. [PDF]
  • Giuliano Albanese, Arka Mitra, Jan-Nico Zaech, Yupeng Zhao, Ajad Chhatkuli, and Luc Van Gool (ETH Zurich and KU Leuven). Optimizing Long-Term Player Tracking and Identification in NAO Robot Soccer by fusing Game-state and External Video. [PDF]
  • Rui Song, Lingjuan Lyu, Wei Jiang, Andreas Festag, and Alois Knoll (Fraunhofer Institute for Transportation and Infrastructure Systems IVI, TUM, Sony AI, DFKI, and Technische Hochschule Ingolstadt). V2X-Boosted Federated Learning for Cooperative Intelligent Transportation Systems with Contextual Client Selection. [PDF]
  • Chelsea Zou, Kishan Chandan, Yan Ding, and Shiqi Zhang (Binghamton University). ARDIE: AR, Dialogue, and Eye Gaze Policies for Human-Robot Collaboration. [PDF]
  • Sebin Lee, Woobin Im, and Sung-Eui Yoon (KAIST). Multi-Resolution Distillation for Self-Supervised Monocular Depth. [PDF]

Organizers

Student Organizers

Acknowledgement

  • IEEE RAS TC for Computer & Robot Vision
  • IEEE RAS TC on Multi-Robot Systems
  • IEEE RAS Autonomous Ground Vehicles and Intelligent Transportation Systems TC