MINOS: Multimodal Indoor Simulator for Navigation in Complex Environments

Introduction

Skillful mobile operation in three-dimensional environments has long been posited as an essential milestone on the road to general intelligence (Moravec, 1984). Despite extensive research, navigation remains a challenging problem. Classical approaches, based on simultaneous localization and mapping (Durrant-Whyte and Bailey, 2006), are sensitive to noisy sensory input and changes in the environment. Recent deep-learning-based methods are potentially more robust, but require extensive training and have only been demonstrated to perform well in simple three-dimensional mazes (Mnih et al., 2016).

A key bottleneck for developing and benchmarking approaches to sensorimotor control is the logistical difficulty of operating a mobile agent in the physical world. The physical world is constrained to operate in real time; poor performance can cause breakage that requires repairing or replacing the physical system; and the system may need to be supervised by a human during the learning process. Moreover, in order to ensure proper generalization, a control system must be evaluated in a wide variety of environments.

Due to these limitations, sensorimotor control models are often developed and benchmarked in simulation (Gupta et al., 2017; Zhu et al., 2017). Once a promising model has been developed and validated, it can be transferred to the physical world (Pomerleau, 1988; Sadeghi and Levine, 2017). The transfer is more likely to succeed if the simulator provides a wide range of large and realistic three-dimensional environments.

In this paper, we present MINOS (Multimodal Indoor Simulator) – a simulation framework for indoor environments that is designed to support the development and validation of multisensory models for navigation. MINOS has been designed with several desiderata in mind. First, the simulator provides access to a large number of realistic environments: the SUNCG dataset of more than $45@000$ three-dimensional models of furnished houses (Song et al., 2017) and the Matterport3D dataset of reconstructed indoor scenes (Chang et al., 2017). Second, the simulator supports flexible multimodal sensing, including vision, depth, surface normals, touch (contact forces), and semantic segmentation. The number of sensors, their positions, and their parameters can be easily specified by the client. Third, our simulation framework allows for procedural reconfiguration of the environments by programmatic modification of scene composition and appearance. Finally, the rendering framework is specifically set up to provide high frame-rates – hundreds of frames per second on a typical workstation – to support approaches that consume millions of simulation steps during training.

We use MINOS to set up a benchmark for indoor navigation algorithms. First, we establish fixed train/validation/test splits of varying complexity on both SUNCG and Matterport3D. This allows for controlled investigation of the generalization of learning-based methods. Second, we set up three goal-directed navigation tasks: PointGoal, ObjectGoal, and RoomGoal. The first involves a purely spatial goal specification, while the latter two specify an object type or room type as the goal. In the PointGoal task, the agent is provided with a vector pointing towards the goal; in the physical world this signal may be provided by an indoor GPS system. The semantic goal tasks provide the agent with semantic information regarding the goal: a room type (kitchen, bedroom, etc.) or an object type (television, mug, etc.). This type of command may be provided by a human user interacting with the agent.

Using the presented benchmark, we conduct a controlled study of approaches to sensorimotor learning. We evaluate several deep reinforcement learning algorithms that navigate towards distal goals using different combinations of sensory modalities. We find that complex realistic environments present a significant challenge for existing algorithms. For example, in furnished medium-scale Matterport3D scenes, the most successful methods complete the PointGoal task in at most 20% of trials. The performance on the RoomGoal task is even worse: even in small Matterport3D scenes, the best methods complete the task in only 14% of trials.

Experiments with varying sensory modalities demonstrate that depth and touch are particularly powerful and can be individually more effective than vision for learning to navigate indoor scenes. Combinations of sensory modalities are more effective still, especially in cluttered environments.

These experiments illustrate the utility of the presented simulation framework for sensorimotor learning research. To support further research in this direction, MINOS is released open-source to the research community at http://minosworld.org.

Related Work

Simulation is an established approach to developing, training, and benchmarking sensorimotor control models. The Arcade Learning Environment (Bellemare et al., 2013) simulates two-dimensional Atari games and has been instrumental in the recent surge of interest in deep reinforcement learning (Mnih et al., 2015, 2016). The ELF platform (Tian et al., 2017) allows for efficient simulation of 2D real-time strategy games. Project Malmo enables simulated agents to interface with the game Minecraft (Johnson et al., 2016). The TORCS (Wymann et al., 2014) and CARLA (Dosovitskiy et al., 2017) simulators have been used to study autonomous driving policies. UAV control has been studied using AirSim (Shah et al., 2017) and UE4Sim (Müller et al., 2017). The Gazebo simulator (Koenig and Howard, 2014) has been used extensively in robotics research.

VizDoom (Kempka et al., 2016) and DeepMind Lab (Beattie et al., 2016) simulate stylized immersive three-dimensional labyrinths. These come close to indoor navigation, but lack realism in terms of layout and appearance, as well as the presence of objects in the scene. Our work is distinguished from these in its focus on realistic indoor environments. This allows for development and validation of sensorimotor control models that operate in realistic, cluttered indoor scenes.

Akin to our work, the AI2-THOR project focuses on realistic indoor environments (Zhu et al., 2017). The goals of AI2-THOR and our work are aligned, but MINOS is distinguished in a number of ways. First, we leverage large datasets, including thousands of furnished houses, with realistic interconnected layouts of up to dozens of rooms each, as opposed to 32 single-room environments provided by THOR. Second, we focus on flexibility of the agent’s sensor suite, both in terms of the available sensors (vision, depth, surface normals, segmentation, touch) and their number and parameters. Third, mindful of the data-hungry nature of many deep RL algorithms, MINOS was developed to run at hundreds of simulation steps per second (rendering tens of millions of frames per day) on typical workstations.

The development of MINOS was initiated in Fall 2016 and the core functionality was completed by June 2017, at which time a version of this paper was submitted for conference publication. Since then, a number of independent efforts have investigated navigation in indoor environments using the SUNCG dataset (Wu et al., 2017; Brodeur et al., 2017; Das et al., 2017) and the Matterport3D dataset (Anderson et al., 2017). MINOS is distinguished from these in several ways. First, our simulation framework provides a flexible user API, allowing for (a) environment configuration via object addition/removal and material variation, and (b) fully parameterized placement and specification of multimodal sensor suites with an arbitrary number of sensors. Second, in addition to the simulator itself, we provide a set of specific benchmark tasks for navigation algorithms. Third, MINOS supports navigation with both continuous and discrete state spaces in both SUNCG and Matterport3D environments.

We leverage the simulator to study the performance of learning-based navigation agents in cluttered indoor environments. Recent work on visual navigation used an actor-critic model that discretizes the agent and state space (Zhu et al., 2017) and explored explicit map representations for planning (Gupta et al., 2017). Other work uses auxiliary tasks or secondary prediction targets to assist learning (Jaderberg et al., 2017; Mirowski et al., 2017). Direct prediction of future measurements or rewards also appears effective for sensorimotor learning in immersive environments (Dosovitskiy and Koltun, 2017). All these methods have been developed in different environments, which lack in either realism or scale. Our work provides a fair comparison of a representative set of state-of-the-art deep navigation models in large, complex, and diverse indoor environments.

Simulation Framework

MINOS is a flexible, efficient, and customizable framework for simulation of large-scale indoor environments. Figure 1 provides an overview of the framework. We now describe the components of the system in detail.

The MINOS simulator API is designed to be flexible and easy to use. A generic dataset layer allows the framework to source environments from the dataset pool. A flexible configuration API supports:

Environment configuration by selecting subsets of environments with filters predicated on properties of the house and the objects present, and programmatically creating variations of the original scenes by re-texturing object surfaces in a semantically consistent fashion, as well as removing or replacing furniture;

Agent control configuration by choosing between discrete and continuous navigation with a parameterized agent physical model;

Generic sensor specification allowing for arbitrary configurations of sensors with custom type, position, orientation, resolution, and encoding. Multiple concurrent sensor streams of each type are supported.

The simulator is implemented in a server-client paradigm. The WebGL-based server is focused on efficiency, and its instances can be deployed in parallel to servers on any OS. We offer two client APIs: a Python wrapper designed to support efficient RL, and a web client that is particularly useful for interactive exploration and crowdsourced data collection. Both the Python and web client APIs communicate with backend instances through a WebSocket layer, allowing for distributed training.

2 Environments

MINOS supports navigation in arbitrary environments. At the time of writing, the simulator provides immediate support for two datasets: the SUNCG dataset of synthetic furnished houses (Song et al., 2017) and the Matterport3D dataset of reconstructed real buildings (Chang et al., 2017). Example scenes are shown in Figure 2.

The SUNCG dataset provides approximately $45@000$ houses with more than 750K rooms of different types. These models support long-range navigation across layouts that are complex both on the inter-room (interconnected floor plans that require traversing from room to room to reach a goal) and the intra-room scale (rooms are densely furnished and navigation requires maneuvering among the furniture).

The Matterport3D dataset consists of 90 multi-floor residences with approximately $2@000$ annotated room regions. These residences are more realistic than the synthetic SUNCG houses, matching the appearance and composition of real environments more closely. We use Matterport3D as a challenging testbed for RL navigation methods.

3 Agent

The agent is represented by a cylinder proxy geometry with parameterized height, radius and offset from the ground. We define a set of control commands that inject linear or angular acceleration: step forward, step back, turn left, turn right, look up, look down, strafe left, and strafe right. The web client maps these commands to interactive keyboard control, whereas the RL API receives a set of string identifiers (one for each command) to be applied in a given time step. Each command is parameterized to allow for scaling of the applied acceleration.

The dynamics of the agent is further parameterized by mass, maximum linear and angular speeds, and coefficient of friction. These parameters along with the simulation time-step duration can be set to implement continuous navigation agents, or to effectively discretize motion. For convenient and reproducible experimentation, we provide two pre-configured agents: a discrete controls agent (effectively a discrete space gridworld agent) and a continuous controls agent.

4 Multimodal sensory inputs

Multimodal perception is crucial for development of sensorimotor skills in animals (Smith and Gasser, 2005) and in artificial systems (Mirowski et al., 2017). To support research on multimodal sensorimotor control, we provide a flexible generic sensory input specification API allowing for any number of sensory inputs in a variety of modalities:

Vision: implemented in WebGL through a real-time rasterization rendering engine. Supports RGB and grayscale output in arbitrary resolutions.

Depth: extracted from the rasterization depth buffer. Supports byte or short quantized values, or floating point range in meters, and noise model specification.

Surface normals: per-pixel normals computed from the 3D mesh of the environment.

Contact forces: collision detection of agent proxy geometry against 3D object meshes. Provides collision impulse response forces at the positions of specified contact sensors.

Semantic segmentation: per-pixel category labeling corresponding to fine-grained SUNCG and Matterport3D category hierarchies, as well as per-instance labeling (instance segmentation).

Measurements: agent velocity and acceleration, distance and direction to specified navigation target (Euclidean distance and distance along shortest path), and normalized episode time (fraction of episode time elapsed). These measurements can be used for debugging, visualization, or as modality-agnostic training inputs that can indicate progress towards the goal.

5 Customization

MINOS also provides an API for introducing controlled variation in the environments and for defining a variety of navigation goals and corresponding tasks:

Material variation: textures and colors can be sampled in a semantically consistent way (i.e., respecting the observation frequencies of given material textures and colors for each object instance in the dataset). The variation can be set to respect the training/validation/test splits so as to ensure that material configurations for particular objects are not shared between splits. This functionality allows for significant augmentation of synthetic 3D environments. Such randomized retexturing has been used in the work of Dosovitskiy and Koltun (2017) and Sadeghi and Levine (2017), and was shown to significantly aid generalization.

Object clutter variation: sets of specified categories of objects can be removed from each environment (e.g., all chairs and all tables).

Navigation goal specification: goals can be specified as arbitrary points in space (randomly sampled or manually placed), with threshold distances for success. Instances of an object category or a room category can also be specified as goals. More specifically, any instance of a category, a randomly selected instance, or the closest instance to the agent can be defined as the goal.

Task specification: the task to be performed by the agent is specified through an arbitrary Python function that computes reward signals and episode success or failure given the agent’s current and past observations, measurements, and state. For our experiments we implement the navigate to X task as a distance check between the current agent position and the closest point in the goal region (which is a point, an object, or a room).

Methods

We use MINOS to benchmark a set of recent navigation algorithms. We assume that an agent interacts with the environment over discrete time steps in an episodic setup. Each episode of interaction with the environment ends after a maximum number of time steps $T$. At each time step $t$, the agent receives an observation $\mathbf{o}_{t}$ and a scalar reward $r_{t}$ from the environment. The observation $\mathbf{o}_{t}=\left\langle\mathbf{s}_{t}^{1},\ldots,\mathbf{s}_{t}^{M}\right\rangle$ is a tuple consisting of $M$ raw sensory inputs $\mathbf{s}_{t}^{1},\ldots,\mathbf{s}_{t}^{M}$ coming from different modalities. Based on the observation, the agent takes an action $a_{t}$ from a discrete action set $\mathcal{A}$ (we discretize the continuous action space provided by the simulator).

We study four end-to-end navigation algorithms. The first three are based on asynchronous advantage actor-critic (A3C) (Mnih et al., 2016). The fourth is Direct Future Prediction (Dosovitskiy and Koltun, 2017), which has shown good performance in a maze navigation task. Note that since we consider agents acting in the continuous state space, we do not include the method of Gupta et al. (2017), which assumes a discrete gridworld-like environment. We now describe the methods in more detail.

Feedforward A3C is the most basic version of the asynchronous advantage actor-critic algorithm, where a feedforward convolutional network is used as a function approximator. The agent is trained to estimate two quantities. The first is the value function: the expected discounted sum of rewards from the current moment until the end of the episode. The second is the policy: a distribution over the set of actions, indicating the degree of benefit expected from each action. The value function is trained via the multi-step Bellman equation – a recurrent relation stating that the expected cumulative reward can be approximated as a sum of several rewards plus an agent’s estimate at a future time step. The policy is trained to maximize the probability of actions leading to larger-than-average rewards and to minimize the probability of actions leading to smaller-than-average rewards. This is achieved via policy gradient with the value function serving as a baseline. Further details are provided by Mnih et al. (2016).

LSTM A3C is an A3C agent in which the feedforward network is augmented by long short-term memory (LSTM) units, trained via backpropagation through time. The vanilla A3C agent can only behave reactively based on the current observation; such an agent is unable to build an internal representation of the environment or execute temporally extended action sequences. LSTM provides the agent with a simple memory, which can potentially help to alleviate these shortcomings of the feedforward agent.

UNREAL is a version of LSTM A3C that is augmented with auxiliary unsupervised tasks. These extra tasks provide additional training signals to the network, leading to improved convergence and stability. The auxiliary tasks include: value function replay, reward prediction, and “pixel control”. Further details are provided by Jaderberg et al. (2017).

DFP is the Direct Future Prediction algorithm (Dosovitskiy and Koltun, 2017). It differs from the aforementioned methods in that it does not explicitly aim to maximize future rewards. Instead it predicts future measurements – a set of low-dimensional sensory inputs. The actions are then selected so as to maximize an objective function that is defined in terms of these measurements. The method can be seen as Monte Carlo reinforcement learning with a decomposed reward. Further details are provided by Dosovitskiy and Koltun (2017).

Experiments

We use MINOS to evaluate the methods summarized in Section 4. We compare the algorithms on goal-directed navigation, with goals specified either by their location relative to the agent, or by their semantic meaning. We evaluate the effectiveness of different combinations of sensory modalities for navigation and measure the effect of environmental complexity on the methods’ performance. In contrast with many previous works, we do not evaluate the agents in the same environment they were trained in. Rather, we study generalization to previously unseen environments. Finally, we perform our experiments in both SUNCG environments and Matterport3D environments to investigate how algorithm performance is impacted by the domain difference between synthetic and reconstructed scenes.

Our goal-directed navigation tasks are set up as a sequence of trials. In each trial, the agent is initialized at a random location in an indoor environment and has to reach a goal location. We experiment with three ways of specifying goals: by their position relative to the agent (PointGoal), or by the semantic category of the target object (ObjectGoal) or target room region (RoomGoal). In this work, we specify the spatial goal using Euclidean distance and normalized direction to a randomly chosen point. The room goal is specified as one of 9 distinct room classes (kitchen, bedroom, living room, toilet, bathroom, dining room, office, hallway, and miscellaneous), and the object goal is specified as an object class (we use doors as navigation targets in the reported experiments).

1.2 Environments

We establish specific subsets of environments for benchmarking indoor navigation. These were selected manually by verifying the realism and traversability of each environment floorplan.

For the SUNCG dataset, we create two variants of each house with different complexity: an empty variant in which the scenes are emptied of furniture and include only architectural elements such as walls, ceilings, floors, doors, and windows, and a furnished variant in which the scenes contain their full content except people and plants.

1.3 Agent

1.4 Training details

We use an epsilon-greedy random exploration schedule, starting with a fully random policy and decaying to approximately 10% probability of random actions by the end of training. The navigation goal is chosen at random for each episode. The DFP agents are trained against a future prediction loss with measurements at time $t$: ${\mathbf{m}_{t}=\left\langle d_{t},x_{t},z_{t},t\right\rangle}$, where $d_{t}$ is the Euclidean distance to the goal and $(x_{t},z_{t})$ is the normalized 2D direction to the goal for the PointGoal task, or the dot product of a one-hot representation of the room category in the RoomGoal task with the goal room category. On-policy actions are chosen by selecting the action that minimizes an objective with a linear combination of the predicted distance $d_{t}$ and the normalized time $t$ with equal weight. The temporal offsets $\{\tau_{1},\ldots,\tau_{n}\}$ are set to $1$, $2$, $4$, $8$, $16$, and $32$ steps. The A3C FF, A3C-LSTM and UNREAL agents are all trained under a reward function computing the difference in Euclidean distance to the goal $d_{t}$ and normalized time $t$ for each time step (matching the objective used for the DFP agent). We use the same training hyperparameters as reported by Jaderberg et al. (2017), but with only four asynchronous threads. For the RoomGoal task, we provide the one-hot goal room category vector as an additional input concatenated with the agent state.

2 Results

At test time, agents are tested for $10$ episodes per scene, in a fixed permuted order of scenes with a set of pre-sampled starting configurations selected to span a range of distances from the goal. Agent performance is evaluated by the overall episode success rate (fraction of episodes ending with the agent arriving at the goal), reported as percentage averaged over all testing episodes.

Table 2 shows the performance of the various navigation agents on goal-directed navigation with variable goal specification (PointGoal or RoomGoal), environment complexity (size, presence of furniture), and environment realism (synthetic or reconstructed). We now analyze these results.

On most tasks, the UNREAL agent performs best, followed by DFP and A3C-LSTM. A3C-FF failed to learn any meaningful policy, and performs worse than a random agent in some cases, which could be due to hyperparameter selection. Despite the lack of memory, DFP outperforms A3C-LSTM and UNREAL for the smaller and less cluttered SUNCG environments in point navigation. In more challenging setups using larger Matterport3D environments and for the semantic room navigation task, UNREAL significantly outperforms the other approaches. This is likely due to the combination of memory and supervision through auxiliary learning.

The performance of all methods declines significantly in large and cluttered environments. The best-performing agent in the PointGoal task is successful in about $80\%$ of trials in the simplest two-room empty SUNCG environments. However, in the most complex PointGoal setup – Matterport 3D houses with up to 24 rooms – all agents have success rate of $20\%$ or lower. These results indicate that even in the simplest scenario the performance is not perfect, and that existing RL methods fail in large, cluttered, realistic environments.

The RoomGoal task is more difficult for all algorithms than PointGoal. This is likely due to the sparsity of the RoomGoal reward signal, which only indicates whether the agent is in a room with a type matching the goal room type.

3 Navigation with multimodal sensory input

Table 3 reports the performance of each agent after training on several sets of novel environments with different complexity: empty single-room SUNCG environments, all empty SUNCG houses, furnished single-room SUNCG environments, and all furnished SUNCG houses. In the simplest setting of empty single-room environments, all agents do well. In particular, the combination of measurements and contact force input performs best, likely due to the simplicity and sufficiency of the goal direction and contact signals for navigating towards a single target door (usually unobstructed in this trivial setting). For empty multi-room houses (up to ten rooms each), the depth modality performs particularly well, whereas the vision modality does not increase performance significantly. This is likely due to the fact that in the absence of clutter, the additional benefit of visual input is limited. The full multimodal agent outperforms the ablated versions in this setting. The full multimodal agent likewise performs best in the furnished settings. Among individual modalities in the furnished settings, depth confers the strongest advantage.

Discussion

We presented a multimodal simulation platform that is designed to support the development of multisensory models for goal-directed navigation in indoor environments. Our simulator provides a suite of sensory input modules that can be flexibly combined. By leveraging two large-scale datasets of indoor environments and augmenting the data through controlled variation of appearance and clutter, we provide orders of magnitude more indoor environments for training and testing than previously available. Our experiments demonstrate that current deep reinforcement learning approaches fail in large, realistic indoor environments, and that multimodality is beneficial in learning to act in cluttered indoor scenes.

References