Fabian Flohr

Anticipating future situations from streaming sensor data is a key perception challenge for mobile robotics and automated vehicles. We address the problem of predicting the path of objects with multiple dynamic modes. The dynamics of such targets can be described by a Switching Linear Dynamical System (SLDS). However, predictions from this probabilistic model cannot anticipate when a change in dynamic mode will occur. We propose to extract various types of cues with computer vision to provide… Mehr anzeigen

Anticipating future situations from streaming sensor data is a key perception challenge for mobile robotics and automated vehicles. We address the problem of predicting the path of objects with multiple dynamic modes. The dynamics of such targets can be described by a Switching Linear Dynamical System (SLDS). However, predictions from this probabilistic model cannot anticipate when a change in dynamic mode will occur. We propose to extract various types of cues with computer vision to provide context on the target’s behavior, and incorporate these in a Dynamic Bayesian Network (DBN). The DBN extends the SLDS by conditioning the mode transition probabilities on additional context states. We describe efficient online inference in this DBN for probabilistic path prediction, accounting for uncertainty in both measurements and target behavior. Our approach is illustrated on two scenarios in the … Weniger anzeigen

Big data has had a great share in the success of deep learning in computer vision. Recent works suggest that there is significant further potential to increase object detection performance by utilizing even bigger datasets. In this paper, we introduce the EuroCity Persons dataset, which provides a large number of highly diverse, accurate and detailed annotations of pedestrians, cyclists and other riders in urban traffic scenes. The images for this dataset were collected on-board a moving… Mehr anzeigen

Big data has had a great share in the success of deep learning in computer vision. Recent works suggest that there is significant further potential to increase object detection performance by utilizing even bigger datasets. In this paper, we introduce the EuroCity Persons dataset, which provides a large number of highly diverse, accurate and detailed annotations of pedestrians, cyclists and other riders in urban traffic scenes. The images for this dataset were collected on-board a moving vehicle in 31 cities of 12 European countries. With over 238200 person instances manually labeled in over 47300 images, EuroCity Persons is nearly one order of magnitude larger than person datasets used previously for benchmarking. The dataset furthermore contains a large number of person orientation annotations (over 211200). We optimize four state-of-the-art deep learning approaches (Faster R-CNN, R-FCN, SSD and YOLOv3) to serve as baselines for the new object detection benchmark. In experiments with previous datasets we analyze the generalization capabilities of these detectors when trained with the new dataset. We furthermore study the effect of the training set size, the dataset diversity (day-vs. night-time, geographical region), the dataset detail (ie availability of object orientation information) and the annotation quality on the detector performance. Finally, we analyze error sources and discuss the road ahead. Weniger anzeigen

This thesis addresses the detection, segmentation and orientation estimation of persons in visual data. In particular, the aim of this work is to establish an accurate machine representation of the Vulnerable Road Users (VRU, eg pedestrians, cyclists) by using image-based cues to support context-aware automated driving.

Object tracking is the task of estimating over time the state of a single or multiple objects based on noisy measurements received from one or several sensors. The field of object tracking spans over several application domains ranging from military radar systems and sensor fusion approaches, to today's computer vision tracking methods employed in consumer electronics and surveillance systems. It also plays a substantial role in autonomous driving. In recent years, the use of deep neural… Mehr anzeigen

Object tracking is the task of estimating over time the state of a single or multiple objects based on noisy measurements received from one or several sensors. The field of object tracking spans over several application domains ranging from military radar systems and sensor fusion approaches, to today's computer vision tracking methods employed in consumer electronics and surveillance systems. It also plays a substantial role in autonomous driving. In recent years, the use of deep neural networks has spiked in various fields, due to their impressive performance in detection and classification tasks. This aspect also makes these methods applicable to object tracking. Therefore, the aim of this survey is to give the reader a brief yet comprehensive start into the widespread field of object tracking with a focus on the latest deep-learning based extensions and approaches. At first, traditional non-deep tracking systems are … Weniger anzeigen

Accidents involving Vulnerable Road Users (VRU) are still a very significant issue for road safety. According to the World Health Organisation, pedestrian and cyclist deaths account for more than 25% of all road traffic deaths worldwide [1]. Autonomous Emergency Braking Systems have the potential to improve safety for these VRU groups.
The PROSPECT project (Proactive Safety for Pedestrians and Cyclists) aims to significantly improve the effectiveness of active VRU safety systems compared to… Mehr anzeigen

Accidents involving Vulnerable Road Users (VRU) are still a very significant issue for road safety. According to the World Health Organisation, pedestrian and cyclist deaths account for more than 25% of all road traffic deaths worldwide [1]. Autonomous Emergency Braking Systems have the potential to improve safety for these VRU groups.
The PROSPECT project (Proactive Safety for Pedestrians and Cyclists) aims to significantly improve the effectiveness of active VRU safety systems compared to those currently on the market by expanding the scope of scenarios addressed by the systems and improving the overall system performance. The project pursues an integrated approach: Newest available accident data combined with naturalistic observations and HMI guidelines represent key inputs for the system specifications, which form the basis for the system development. For system development, two main aspects are considered: advanced sensor processing with situation analysis, and intervention strategies including braking and steering. All these concepts are implemented in several vehicle prototypes. Special emphasis is put on balancing system performance in critical scenarios and avoiding undesired system activations. Weniger anzeigen

Significant progress has been achieved over the past decade on vision-based pedestrian detection; this has led to active pedestrian safety systems being deployed in most mid- to high-range cars on the market. Comparatively little effort has been spent on vision-based cyclist detection, especially when it concerns quantitative performance analysis on large datasets. We present a large-scale experimental study on cyclist detection where we examine the currently most promising object detection… Mehr anzeigen

Significant progress has been achieved over the past decade on vision-based pedestrian detection; this has led to active pedestrian safety systems being deployed in most mid- to high-range cars on the market. Comparatively little effort has been spent on vision-based cyclist detection, especially when it concerns quantitative performance analysis on large datasets. We present a large-scale experimental study on cyclist detection where we examine the currently most promising object detection methods; we consider Aggregated Channel Features, Deformable Part Models and Region-based Convolutional Neural Networks. We also introduce a new method called Stereo-Proposal based Fast R-CNN (SP-FRCN) to detect cyclists based on stereo proposals and Fast R-CNN (FRCN) framework. Experiments are performed on a dataset containing 22161 annotated cyclist instances in over 30000 images, recorded from a moving vehicle in the urban traffic of Beijing. Results indicate that all the three solution families can reach top performance around 0.89 average precision on the easy case, but the performance drops gradually with the difficulty increasing. The dataset including rich annotations, stereo images and evaluation scripts (termed “Tsinghua-Daimler Cyclist Benchmark”) is made public to the scientific community, to serve as a common point of reference for future research. Weniger anzeigen

Significant progress has been achieved over the past decade on vision-based pedestrian detection; this has led to active pedestrian safety systems being deployed in most mid- to high-range cars on the market. Comparatively little effort has been spent on vision-based cyclist detection, especially when it concerns quantitative performance analysis on large datasets. We present a large-scale experimental study on cyclist detection where we examine the currently most promising object detection… Mehr anzeigen

Significant progress has been achieved over the past decade on vision-based pedestrian detection; this has led to active pedestrian safety systems being deployed in most mid- to high-range cars on the market. Comparatively little effort has been spent on vision-based cyclist detection, especially when it concerns quantitative performance analysis on large datasets. We present a large-scale experimental study on cyclist detection where we examine the currently most promising object detection methods; we consider Aggregated Channel Features, Deformable Part Models and Region-based Convolutional Neural Networks. We also introduce a new method called Stereo-Proposal based Fast R-CNN (SP-FRCN) to detect cyclists based on stereo proposals and Fast R-CNN (FRCN) framework. Experiments are performed on a dataset containing 22161 annotated cyclist instances in over 30000 images, recorded from … Weniger anzeigen

Extensive research interest has been focused on protecting vulnerable road users in recent years, particularly pedestrians and cyclists, due to their attributes of vulnerability. However, comparatively little effort has been spent on detecting pedestrian and cyclist together, particularly when it concerns quantitative performance analysis on large datasets. In this paper, we present a unified framework for concurrent pedestrian and cyclist detection, which includes a novel detection proposal… Mehr anzeigen

Extensive research interest has been focused on protecting vulnerable road users in recent years, particularly pedestrians and cyclists, due to their attributes of vulnerability. However, comparatively little effort has been spent on detecting pedestrian and cyclist together, particularly when it concerns quantitative performance analysis on large datasets. In this paper, we present a unified framework for concurrent pedestrian and cyclist detection, which includes a novel detection proposal method (termed UB-MPR) to output a set of object candidates, a discriminative deep model based on Fast R-CNN for classification and localization, and a specific postprocessing step to further improve detection performance. Experiments are performed on a new pedestrian and cyclist dataset containing 30 490 annotated pedestrian and 26 771 cyclist instances in over 50 000 images, recorded from a moving vehicle in the urban traffic … Weniger anzeigen

Extensive research interest has been focused on protecting vulnerable road users in recent years, particularly pedestrians and cyclists, due to their attributes of vulnerability. However, comparatively little effort has been spent on detecting pedestrian and cyclist together, particularly when it concerns quantitative performance analysis on large datasets. In this paper, we present a unified framework for concurrent pedestrian and cyclist detection, which includes a novel detection proposal… Mehr anzeigen

Extensive research interest has been focused on protecting vulnerable road users in recent years, particularly pedestrians and cyclists, due to their attributes of vulnerability. However, comparatively little effort has been spent on detecting pedestrian and cyclist together, particularly when it concerns quantitative performance analysis on large datasets. In this paper, we present a unified framework for concurrent pedestrian and cyclist detection, which includes a novel detection proposal method (termed UB-MPR) to output a set of object candidates, a discriminative deep model based on Fast R-CNN for classification and localization, and a specific postprocessing step to further improve detection performance. Experiments are performed on a new pedestrian and cyclist dataset containing 30 490 annotated pedestrian and 26 771 cyclist instances in over 50 000 images, recorded from a moving vehicle in the urban traffic of Beijing. Experimental results indicate that the proposed method outperforms other state-of-the-art methods significantly. Weniger anzeigen

We present a novel approach for vehicle-pedestrian collision risk analysis that incorporates mutual situational awareness, a degree of potential motion coupling and the spatial layout of the environment. The approach uses a Dynamic Bayesian Network (DBN) for modeling the individual object paths; collision risk is subsequently computed by an intersection operation. More specifically, the proposed DBN consists of two subgraphs for modeling pedestrian and vehicle path, respectively. They consist… Mehr anzeigen

We present a novel approach for vehicle-pedestrian collision risk analysis that incorporates mutual situational awareness, a degree of potential motion coupling and the spatial layout of the environment. The approach uses a Dynamic Bayesian Network (DBN) for modeling the individual object paths; collision risk is subsequently computed by an intersection operation. More specifically, the proposed DBN consists of two subgraphs for modeling pedestrian and vehicle path, respectively. They consist of latent states on top of Switching Linear Dynamical Systems (SLDSs) to anticipate changes in object dynamics. The pedestrian and vehicle-related sub-graphs contain latent states to model whether the pedestrian has seen the oncoming vehicle, and conversely, whether the driver has seen the pedestrian (associated measurements involve the respective head orientations). The pedestrian-related sub-graph furthermore contains a latent state modeling whether the pedestrian is at the curbside or not. Finally, a latent state is shared by the two sub-graphs, which models the potential motion coupling (i.e. at full awareness of the other traffic participant). We consider the scenario of a crossing pedestrian, who might stop or continue walking at the curb, in combination with an approaching vehicle, that might stop or continue driving. In experiments we illustrate that with the proposed approach, a more anticipatory driver warning and/or vehicle control strategy can be implemented. Weniger anzeigen

This paper presents a novel approach for joint object detection and orientation estimation in a single deep convolutional neural network utilizing proposals calculated from 3D data. For orientation estimation, we extend a R-CNN like architecture by several carefully designed layers. Two new object proposal methods are introduced, to make use of stereo as well as lidar data. Our experiments on the KITTI dataset show that by combining proposals of both domains, high recall can be achieved while… Mehr anzeigen

This paper presents a novel approach for joint object detection and orientation estimation in a single deep convolutional neural network utilizing proposals calculated from 3D data. For orientation estimation, we extend a R-CNN like architecture by several carefully designed layers. Two new object proposal methods are introduced, to make use of stereo as well as lidar data. Our experiments on the KITTI dataset show that by combining proposals of both domains, high recall can be achieved while keeping the number of proposals low. Furthermore, our method for joint detection and orientation estimation outperforms state of the art approaches for cyclists on the easy test scenario of the KITTI test dataset. Weniger anzeigen

We present a probabilistic framework for the joint estimation of pedestrian head and body orientation from a mobile stereo vision platform. For both head and body parts, we convert the responses of a set of orientation-specific detectors into a (continuous) probability density function. The parts are localized by means of a pictorial structure approach, which balances part-based detector responses with spatial constraints. Head and body orientations are estimated jointly to account for… Mehr anzeigen

We present a probabilistic framework for the joint estimation of pedestrian head and body orientation from a mobile stereo vision platform. For both head and body parts, we convert the responses of a set of orientation-specific detectors into a (continuous) probability density function. The parts are localized by means of a pictorial structure approach, which balances part-based detector responses with spatial constraints. Head and body orientations are estimated jointly to account for anatomical constraints. The joint single-frame orientation estimates are integrated over time by particle filtering. The experiments involved data from a vehicle-mounted stereo vision camera in a realistic traffic setting; 65 pedestrian tracks were supplied by a state-of-the-art pedestrian tracker. We show that the proposed joint probabilistic orientation estimation framework reduces the mean absolute head and body orientation error up to 15° … Weniger anzeigen

We present a novel Dynamic Bayesian Network for pedestrian path prediction in the intelligent vehicle domain. The model incorporates the pedestrian situational awareness, situation criticality and spatial layout of the environment as latent states on top of a Switching Linear Dynamical System (SLDS) to anticipate changes in the pedestrian dynamics. Using computer vision, situational awareness is assessed by the pedestrian head orientation, situation criticality by the distance between vehicle… Mehr anzeigen

We present a novel Dynamic Bayesian Network for pedestrian path prediction in the intelligent vehicle domain. The model incorporates the pedestrian situational awareness, situation criticality and spatial layout of the environment as latent states on top of a Switching Linear Dynamical System (SLDS) to anticipate changes in the pedestrian dynamics. Using computer vision, situational awareness is assessed by the pedestrian head orientation, situation criticality by the distance between vehicle and pedestrian at the expected point of closest approach, and spatial layout by the distance of the pedestrian to the curbside. Our particular scenario is that of a crossing pedestrian, who might stop or continue walking at the curb. In experiments using stereo vision data obtained from a vehicle, we demonstrate that the proposed approach results in more accurate path prediction than only SLDS, at the relevant short time horizon (1 s), and slightly outperforms a computationally more demanding state-of-the-art method. Weniger anzeigen

We present an approach for the joint probabilistic estimation of pedestrian head and body orientation in the context of intelligent vehicles. For both, head and body, we convert the output of a set of orientation-specific detectors into a full (continuous) probability density function. The parts are localized with a pictorial structure approach which balances part-based detector output with spatial constraints. Head and body orientation estimates are furthermore coupled probabilistically to… Mehr anzeigen

We present an approach for the joint probabilistic estimation of pedestrian head and body orientation in the context of intelligent vehicles. For both, head and body, we convert the output of a set of orientation-specific detectors into a full (continuous) probability density function. The parts are localized with a pictorial structure approach which balances part-based detector output with spatial constraints. Head and body orientation estimates are furthermore coupled probabilistically to account for anatomical constraints. Finally, the coupled single-frame orientation estimates are integrated over time by particle filtering. The experiments involve 37 pedestrian tracks obtained from an external stereo vision-based pedestrian detector in realistic traffic settings. We show that the proposed joint probabilistic orientation estimation approach reduces the mean head and body orientation error by 10 degrees and more. Weniger anzeigen

This paper presents an iterative, EM-like framework for accurate pedestrian segmentation, combining generative shape models and multiple data cues. In the E-step, shape priors are introduced in the unary terms of a Conditional Random Field (CRF) formulation, joining other data terms derived from color, texture and disparity cues. In the M-step, the resulting segmentation is used to adapt an Active Shape Model (ASM), after which the EM process alternates.
Experiments on the public Penn-Fudan… Mehr anzeigen

This paper presents an iterative, EM-like framework for accurate pedestrian segmentation, combining generative shape models and multiple data cues. In the E-step, shape priors are introduced in the unary terms of a Conditional Random Field (CRF) formulation, joining other data terms derived from color, texture and disparity cues. In the M-step, the resulting segmentation is used to adapt an Active Shape Model (ASM), after which the EM process alternates.
Experiments on the public Penn-Fudan pedestrian dataset suggest that our method outperforms the state-of-the-art. We further provide results on a new Daimler pedestrian dataset, captured from on-board a vehicle, which includes disparity data. This dataset is made public to facilitate benchmarking. Weniger anzeigen

In this article, we present an evaluation of several multi-target tracking methods based on simulated scenarios in the maritime domain. In particular, we consider variations of the Joint Integrated Probabilistic Data Association (JIPDA) algorithm, namely the Linear Multi-Target IPDA (LMIPDA), Linear Joint IPDA (LJIPDA), and Markov Chain Monte Carlo Data Association (MCMCDA). The algorithms are compared with respect to an extension of the Optimal Subpattern Assignment (OSPA) metric, the… Mehr anzeigen

In this article, we present an evaluation of several multi-target tracking methods based on simulated scenarios in the maritime domain. In particular, we consider variations of the Joint Integrated Probabilistic Data Association (JIPDA) algorithm, namely the Linear Multi-Target IPDA (LMIPDA), Linear Joint IPDA (LJIPDA), and Markov Chain Monte Carlo Data Association (MCMCDA). The algorithms are compared with respect to an extension of the Optimal Subpattern Assignment (OSPA) metric, the Hellinger distance and further performance measures. As no single algorithm is equally well fitted to all tested scenarios, our results show which algorithms fits best for specific scenarios. Weniger anzeigen

View my publications on Google Scholar.