We introduce \(\mathcal{L}_1\)-MBRL, a control-theoretic augmentation scheme for Model-Based Reinforcement Learning (MBRL) algorithms. Unlike model-free approaches, MBRL algorithms learn a model of the transition function using data and use it to design a control input. Our approach generates an approximate control-affine model of the learned transition function according to the switching law. Using the approximate model, control input produced by the underlying MBRL is perturbed by the \(\mathcal{L}_1\) adaptive control, which is designed to enhance the robustness of the system against uncertainties. Importantly, this approach is agnostic to the choice of MBRL algorithm, which enables the utilization of the scheme in various MBRL algorithms. Our method exhibits superior performance and sample efficiency on multiple MuJoCo environments, both with and without system noise, as demonstrated through numerical simulations.

We consider Markov chain approximation for optimal control of diffusion processes under infinite horizon discounted cost optimality and apply the simulation-based Empirical Value Iteration to estimate the value function of each approximating chain. We follow a nested multi-grid discretization of the state space to establish weak convergence of the value function sequence to the value function of the original controlled diffusion. We illustrate the convergence performance of the model on the popular Benes' bang-bang control problem [Beneš (1974)].

Autonomous navigation of Unmanned Aerial Vehicles (UAVs) has real-life applications in remote sensing, wildlife surveillance, search and rescue operations. A popular training paradigm to learn optimal actions for navigating such complex, dynamic, and uncertain environments is Reinforcement Learning (RL), where the optimal decisions are learnt over time through a reward-feedback received from the environment. However, manually constructing a feedback function that can help guide the UAV to accomplish the desired objective is often very hard. Preference-based Reinforcement Learning (PbRL) is an emerging sub-field of RL where the manual construction of reward function is replaced with human feedback. In this setting, a human is presented with a pair of trajectories followed by the RL agent to elicit the subject’s preference for one over the other. A PbRL algorithm would then compute an optimal sequence of actions using just the set of preferences collected over different trajectories. In this work, we consider PbRL for UAV navigation and follow an ensemble approach to enhance navigation performance. We demonstrate the efficacy of the proposed algorithm through experiments on a range of complex environments and tasks. Ours is the first work that uses human preferences to solve the UAV navigation problem to the best of our knowledge.

In this work, we consider the problem of fully autonomous navigation of a vehicle to reach its desired goal in a constrained environment, specifically when the vehicle dynamics are unknown. To this end, we design a modular framework comprised of (i) LiDAR-based Hector SLAM for building a map of the environment, detecting obstacles, and tracking vehicle's conformance to any given trajectory, (ii) motion primitives-based kinodynamic Rapidly-exploring Random Trees (RRTs) for building an obstacle-free trajectory between a source and destination vehicle configuration, and finally, (iii) a Proportional Integral Derivative (PID) controller for trajectory tracking and disturbance rejection. We demonstrate the execution of our framework on an actual laboratory vehicle to perform complex maneuvers such as parallel parking, perpendicular parking, and reversing motion in a constrained environment under diverse weather and illumination conditions, such as sunny days, rainy days, and night-time.

We present an end-to-end air traffic management framework for Unmanned Aircraft Systems (UAS) operations that is both scalable and robust. Our approach involves defining congestion in the airspace and developing a congestion-based cost map that can mitigate potential congestion while adhering to regulations and guidelines. Each UAS operation leverages a recursively updated cost map in solving the path planning problem, providing scalability of the framework. Additionally, we introduce a time-critical controller to enhance the robustness of mission execution. Empirical evidence confirms the feasibility and effectiveness of our method, achieving significant reductions in both cumulative and maximum levels of airspace congestion.

We propose an optimization-based integrated perception and planning framework for autonomous vehicle navigation that achieves real-time state estimation and path planning with high accuracy and robustness. Our Simultaneous Localization And Mapping (SLAM) module is based on ErrorState Extended Kalman Filter (ES-EKF) for LiDAR-Inertial sensor fusion. The SLAM system generates a cost map using Euclidean Distance Transform (EDT) that directly encodes environmental constraints as a cost map. A non-linear trajectory optimization problem is formulated with the cost function and solved in real-time using the direct collocation approach. Our results on the KITTI dataset demonstrate the effectiveness of our framework.

Visual-inertial SLAM is essential for robot navigation in GPS-denied environments, e.g. indoor, underground. Conventionally, the performance of visual-inertial SLAM is evaluated with open-loop analysis, with a focus on the drift level of SLAM systems. In this paper, we raise the question on the importance of visual estimation latency in closed-loop navigation tasks, such as accurate trajectory tracking. To understand the impact of both drift and latency on visualinertial SLAM systems, a closed-loop benchmarking simulation is conducted, where a robot is commanded to follow a desired trajectory using the feedback from visual-inertial estimation. By extensively evaluating the trajectory tracking performance of representative state-of-the-art visual-inertial SLAM systems, we reveal the importance of latency reduction in visual estimation module of these systems. The findings suggest directions of future improvements for visual-inertial SLAM.