We develop online methods to generate safe and smooth trajectories for aerial navigation through unknown, complex, and possibly dynamic environments. We use convex optimization tools to ensure both collision avoidance and dynamic feasibility.

Micro aerial vehicles (MAVs), especially quadrotors, have drawn increasing attention in recent years thanks to their superior mobility in complex environments that are inaccessible or dangerous for human or other ground vehicles. In autonomous navigation missions, quadrotors should be able to online generate and execute smooth and safe trajectories from a start position to a target position, while avoiding unexpected obstacles. The generated trajectories should have the guarantee of safety and smoothness considering the dynamic ability of the quadrotor.

In this project, some novel methods are developed to generate safe and smooth trajectories in cluttered environments. Based on good localization and mapping techniques, a flight corridor with safety guarantee is obtained in the cluttered environments first, following an optimization-based algorithm to assign a global optimal trajectory within the flight corridor entirely. Our works are implemented onboard a quadrotor and are suitable for fast online re-planning, making them able to work in unknown dynamic environments with unexpected obstacles.

Our algorithms can be widely used on various types of mapping modules, such as laser-based octomap and point clouds or monocular dense mapping. Both simulation results and indoor and outdoor autonomous flights in unknown cluttered environments show the good performance of our methods.

 

Gradient-based online safe trajectory generation for quadrotor flight in complex environments

By Fei GAO

We propose a trajectory generation framework for quadrotor autonomous navigation in unknown 3-D complex environments using gradient information. We decouple the trajectory generation problem as front-end path searching and back-end trajectory refinement. Based on the map that is incrementally built onboard, we adopt a sampling- based informed path searching method to find a safe path passing through obstacles. We convert the path consists of line segments to an initial safe trajectory. An optimization- based method which minimizes the penalty of collision cost, smoothness and dynamical feasibility is used to refine the trajectory. Our method shows the ability to online gener- ate smooth and dynamical feasible trajectories with safety guarantee. We integrate the state estimation, dense mapping and motion planning module into a customized light-weight quadrotor platform. We validate our proposed method by presenting fully autonomous navigation in unknown cluttered indoor and outdoor environments.

 

Tracking a moving target in cluttered environments using a quadrotor

By Jing CHEN

We address the challenging problem of tracking a moving target in cluttered environments using a quadrotor. Our online trajectory planning method generates smooth, dynamically feasible, and collision-free polynomial trajectories that follow a visually-tracked moving target. As visual observations of the target are obtained, the target trajectory can be estimated and used to predict the target motion for a short time horizon. We propose a formulation to embed both limited horizon tracking error and quadrotor control costs in the cost function for a quadratic programming (QP), while encoding both collision avoidance and dynamical feasibility as linear inequality constraints for the QP. Our method generates tracking trajectories in the order of milliseconds and is therefore suitable for online target tracking with a limited sensing range. We implement our approach on-board a quadrotor testbed equipped with cameras, a laser range finder, an IMU, and onboard computing. Statistical analysis, simulation, and real-world experiments are conducted to demonstrate the effectiveness of our approach.

 

Online quadrotor trajectory generation and autonomous navigation on point clouds

By Fei GAO

We present a framework for online generation of safe trajectories directly on point cloud for autonomous quadrotor flight. Considering a quadrotor operating in unknown environments, we use a 3-D laser range finder for state estimation and simultaneously build a point cloud map of the environment. Based on the incrementally built point cloud map, we utilize the property of the fast nearest neighbor search in KD-tree and adopt the sampling-based path finding method to generate a flight corridor with safety guarantee in 3-D space. A trajectory generation method formulated in quadratically constrained quadratic programming (QCQP) is then used to generate trajectories that constrained entirely within the corridor. Our method runs onboard within 100 milliseconds, making it suitable for online re-planning. We integrate the proposed planning method with laser-based state estimation and mapping modules, and demonstrate the autonomous quadrotor flight in unknown indoor and outdoor environments.

 

Online generation of collision-free trajectories for quadrotor flight in unknown cluttered environments

By Jing CHEN

We present an online method for generating collision-free trajectories for autonomous quadrotor flight through cluttered environments. We consider the real-world scenario that the quadrotor aerial robot is equipped with limited sensing and operates in initially unknown environments. During flight, an octree-based environment representation is incrementally built using onboard sensors. Utilizing efficient operations in the octree data structure, we are able to generate free-space flight corridors consisting of large overlapping 3-D grids in an online fashion. A novel optimization-based method then generates smooth trajectories that both are bounded entirely within the safe flight corridor and satisfy higher order dynamical constraints. Our method computes valid trajectories within fractions of a second on a moderately fast computer, thus permitting online re-generation of trajectories for reaction to new obstacles. We build a complete quadrotor testbed with onboard sensing, state estimation, mapping, and control, and integrate the proposed method to show online navigation through complex unknown environments.

 

Improving octree-based occupancy maps using environment sparsity with application to aerial robot navigation

By Jing CHEN

We present an improved octree-based mapping framework for autonomous navigation of mobile robots. Octree is best known for its memory efficiency for representing large-scale environments. However, existing implementations, including the state-of-the-art OctoMap [1], are computationally too expensive for online applications that require frequent map updates and inquiries. Utilizing the sparse nature of the environment, we propose a ray tracing method with early termination for efficient probabilistic map update. We also propose a divide-and-conquer volume occupancy inquiry method which serves as the core operation for generation of free-space configurations for optimization-based trajectory generation. We experimentally demonstrate that our method maintains the same storage advantage of the original OctoMap, but being computationally more efficient for map update and occupancy inquiry. Finally, by integrating the proposed map structure in a complete navigation pipeline, we show autonomous quadrotor flight through complex environments.

 

Quadrotor trajectory generation in dynamic environments using semi-definite relaxation on nonconvex QCQP

By Fei GAO

We present an optimization-based framework for generating quadrotor trajectories which are free of collision in dynamic environments with both static and moving obstacles. Using the finite-horizon motion prediction of moving obstacles, our method is able to generate safe and smooth trajectories with minimum control efforts. Our method optimizes trajectories globally for all observed moving and static obstacles, such that the avoidance behavior is most unnoticeable. This method first utilizes semi-definite relaxation on a quadratically constrained quadratic programming (QCQP) problem to eliminate the nonconvex constraints in the moving obstacle avoidance problem. A feasible and reasonably good solution to the original nonconvex problem is obtained using a randomization method and convex linear restriction. We detail the trajectory generation formulation and the solving procedure of the nonconvex quadratic program. Our approach is validated by both simulation and experimental results.

 

We develop real-time methods for generating dense maps for large-scale autonomous navigation of aerial robots. We investigate into monocular and multi-camera dense mapping methods with special attention on the tight integration between maps and motion planning modules.

Without any prior knowledge of the environment, our dense mapping module utilizes a inverse depth labeling method to extract a 3D cost volume through temporal aggregation on synchronized camera poses. After semi-global optimization and post-processing, a dense depth image is calculated and fed into our uncertainty-aware truncated signed distance function (TSDF) fusion approach, from which a live dense 3D map is produced.
 

Autonomous aerial navigation using monocular visual-inertial fusion

By Yi LIN

We present a real-time monocular visual-inertial dense mapping and autonomous navigation system. The whole system is implemented on a tight size and light weight quadrotor where all modules are processing onboard and in real time. By properly coordinating three major system modules: state estimation, dense mapping and trajectory planning, we validate our system in both cluttered indoor and outdoor environments via multiple autonomous flight experiments. A tightly-coupled monocular visual-inertial state estimator is develop for providing high-accuracy odometry, which is used for both feedback control and dense mapping. Our estimator supports on-the-fly initialization, and is able to online estimate vehicle velocity, metric scale, and IMU biases.
Without any prior knowledge of the environment, our dense mapping module utilizes a plane-sweeping-based method to extract a 3D cost volume through temporal aggregation on synchronized camera poses. After semi-global optimization and post-processing, a dense depth image is calculated and fed into our uncertainty-aware TSDF fusion approach, from which a live dense 3D map is produced. Using this map, our planning module firstly generates an initial collision-free trajectory based on our sampling-based path searching method. A gradient-based optimization method is then applied to ensure trajectory smoothness and dynamic feasibility. Following the trend of rapid increases in mobile computing power, we believe our minimum sensing sensor setup suggests a feasible solution to fully autonomous miniaturized aerial robots.

 

High-precision online markerless stereo extrinsic calibration

By Yonggen LING

Stereo cameras and dense stereo matching algorithms are core components for many robotic applications due to their abilities to directly obtain dense depth measurements and their robustness against changes in lighting conditions. However, the performance of dense depth estimation relies heavily on accurate stereo extrinsic calibration. In this work, we present a real-time markerless approach for obtaining high-precision stereo extrinsic calibration using a novel 5-DOF (degrees-of-freedom) and nonlinear optimization on a manifold, which captures the observability property of vision-only stereo calibration. Our method minimizes epipolar errors between spatial per-frame sparse natural features. It does not require temporal feature correspondences, making it not only invariant to dynamic scenes and illumination changes, but also able to run significantly faster than standard bundle adjustment-based approaches. We introduce a principled method to determine if the calibration converges to the required level of accuracy, and show through online experiments that our approach achieves a level of accuracy that is comparable to offline markerbased calibration methods. Our method refines stereo extrinsic to the accuracy that is sufficient for block matching-based dense disparity computation. It provides a cost-effective way to improve the reliability of stereo vision systems for long-term autonomy.

 

Real-time monocular dense mapping on aerial robots using visual-inertial fusion

By Zhenfei YANG

In this work, we present a solution to real-time monocular dense mapping. A tightly-coupled visual-inertial localization module is designed to provide metric and high-accuracy odometry. A motion stereo algorithm is proposed to take the video input from one camera to produce local depth measurements with semi-global regularization. The local measurements are then integrated into a global map for noise filtering and map refinement. The global map obtained is able to support navigation and obstacle avoidance for aerial robots through our indoor and outdoor experimental verification. Our system runs at 10Hz on an Nvidia Jetson TX1 by properly distributing computation to CPU and GPU. Through onboard experiments, we demonstrate its ability to close the perception-action loop for autonomous aerial robots. We release our implementation as open-source software.

 

Building maps for autonomous navigation using sparse visual SLAM features

By Yonggen LING

Autonomous navigation, which consists of a systematic integration of localization, mapping, motion planning and control, is the core capability of mobile robotic systems. However, most research considers only isolated technical modules. There exist significant gaps between maps generated by SLAM algorithms and maps required for motion planning. Our work presents a complete online system that consists in three modules: incremental SLAM, real-time dense mapping, and free space extraction. The obtained free-space volume (i.e. a tessellation of tetrahedra) can be served as regular geometric constraints for motion planning. Our system runs in real-time thanks to the engineering decisions proposed to increase the system efficiency. We conduct extensive experiments on the KITTI dataset to demonstrate the run-time performance. Qualitative and quantitative results on mapping accuracy are also shown. For the benefit of the community, we make the source code public.

 

Author: Yi LIN

Visual-based simultaneous localization and mapping (SLAM) technology has been well developed these years. Both feature-based methods and direct methods show impressive performance. However in extreme environments such as low-light environment, long-time exposure with aggressive motions always cause serious motion blur. In almost all visual odometry systems, blurry images drastically impede the feature detecting and feature matching. Also, frame-to-frame corresponding pixel intensity invariance assumption in photometric-based method will be affected. Adopting a blur kernel to deblur the raw images from the cameras as the pre-procedure is a popular way to deal with this problem. Instead of using a blur kernel, directly estimating the blurry images is another novel way. Notice the fact that motion blur is caused by camera motion during the exposure period, we model each pixel intensity engendering. A blur-aware motion estimation method we proposed to estimate the trajectory from an initial unblur frame to blur frame by knowing the depth map from stereo cameras.