SlideShare a Scribd company logo

Intelligent Traffic light detection for individuals with CVD

•
•
S
Swaroop Aradhya M C

This is a technical seminar ppt on mobile standards based traffic light detection which can be used as an assistive device in vehicles for individuals with Color vision deficiency Source : “Mobile Standards-Based Traffic Light Detection in Assistive Devices for Individuals with Color-Vision Deficiency” An IEEE Transaction on Intelligent Transport Systems 2014

Related slideshows

Deep learning for object detection
Deep learning for object detectionDeep learning for object detection
Deep learning for object detection
 
Moving object detection in video surveillance
Moving object detection in video surveillanceMoving object detection in video surveillance
Moving object detection in video surveillance
 
Automatic number plate recognition (anpr)
Automatic number plate recognition (anpr)Automatic number plate recognition (anpr)
Automatic number plate recognition (anpr)
 
1 of 19
BANGALORE INSTITUTE OF TECHNOLOGY DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING SEMINAR TITLE: MOBILE STANDARDS – BASED TRAFFIC LIGHT DETECTION IN ASSISTIVE DEVICES FOR INDIVIDUALS WITH COLOR – VISION DEFICIENCY BY SWAROOP ARADHYA 8TH SEMESTER B.E
PROBLEM STATEMENT To design a mobile standard based traffic light detection system as an assistive device in vehicle for the individuals with color vision deficiency using image processing and artificial intelligence
LITERATURE SURVEY • Akhan Almagambetov, Member, IEEE, Senem Velipasalar, Senior Member, IEEE, and Assel Baitassova “Mobile Standards-Based Traffic Light Detection in Assistive Devices for Individuals with Color-Vision Deficiency” An IEEE Transaction on Intelligent Transport Systems 2014 • A. Vu, A. Ramanandan, A. Chen, J. Farrell, and M. Barth, “Real-time computer vision/DGPS-aided inertial navigation system for lane-level vehicle navigation” IEEETrans.Intel.Transp.Syst., vol.13,no.2,pp.899– 913, Jun. 2012 • N. Fairfield and C. Urmson, “Traffic light mapping and detection,” in Proc. IEEE ICRA, 2011, pp. 5421–5426 • J. Levinson, J. Askeland, J. Dolson, and S. Thrun, “Traffic light mapping, localization, and state detection for autonomous vehicles,” in Proc. IEEE ICRA, 2011, pp. 5784– 5791 • Y. Shen, U. Ozguner, K. Redmill, and J. Liu, “A robust video based traffic light detection algorithm for intelligent vehicles,” in Proc. IEEE Intel. Veh. Symp., 2009, pp. 521–526
EXISTING SYSTEM Current traffic light detection methods are divided into the following three categories. GPS or Map based Systems • These systems require the integration of GPS with an onboard vehicle camera. Coupled with a map, the GPS receiver pinpoints the location of the vehicle in relation to traffic intersections. • These methods, while robust, are very limited in their scope of use. • These methods involve a significant man-hours investment and courses have to be predefined on a map and pre-driven. • Any deviation from a set pre driven course would result in the failure of this system. • The appropriate regions of interest have to be marked by hand at each relevant intersection.
EXISTING SYSTEM Image based Methods • These methods only use the image sensor for the detection of traffic lights. • Traffic light positions are determined either by observing the traffic signal head installations and localizing the light positions with respect to the signal head or the positions are inferred from detecting the colored lights themselves. • These types of algorithms are very efficient, although current state-of-the-art algorithms are not very robust. Uses no fail-safe or correction mechanisms, and is prone to failures (undetected lights) Infrastructure Modification • Infrastructure modifications are proposed by a number of vehicle manufacturers to enable communication between the vehicle and the surrounding infrastructure. • This, however, requires an overhaul of the current Intelligent Transportation Systems (ITS) and in- vehicle technologies and is unlikely to be implemented by even the most progressive municipalities.
PROPOSED SYSTEM • This proposed system has presented a robust method for detecting the status of the traffic lights from visual information only (versus relying on traffic light localization via GPS, radar, lidar, or map-based prior knowledge) • The first to approach the problem from the standpoint of multiple light weight agents that are used to decompose a scene, each one integrating fail-safe mechanisms designed to prevent erroneous detection. • The presented system can be easily ported over for use on a low-power embedded smart camera as a stand-alone windshield-mounted driver-assistance device. (a) Hardware setup, (b) (c) View
Algorithm Flowchart
SYSTEM DESIGN Initial TCSH Light Identification • Standards-Based Soft Filtering: Standards-based color thresholds (UV components) are used for initial soft filtering of the image. • In order to speed up the computation time, the light color is not determined at this point of the algorithm. • Regions that fall within the specified soft thresholds are used regardless of their colors and are labeled “potential matches”. • The YUV colorspace separates the luminance (Y) and color data (UV) to describe a wide range of colors by adding an unconstrained brightness component
SYSTEM DESIGN Morphological Operations: • In order to “cleanup” the image after the soft filtering step, mathematical morphology operations are applied for noise elimination. • After the mathematical morphology operations are run on each individual “blob” that matched the relaxed thresholds, Its properties are stored in a data structure shown below Morphological operations performed on an actual light capture. (a) RGB. (b) BW. (c) Final
SYSTEM DESIGN Accurate Color Detection: • Once all of the detected blobs have been stored, each blob is analyzed, and a color is assigned to the blob through the ColorLabel variable. • Using V and H channel histogram, accurate color in the traffic light is detected. • After the color of the blob is reliably determined, it is stored into the ColorLabel field of the blob data structure. V-channel histogram for red and green H-channel histogram for yellow and red
Light Assertion and Filtering Relative orientation • Relative orientation of a blob determines the rotation of a particular elliptical shape with respect to the major axis . • The relative orientation of each blob is calculated when the blob data structure is first created, and it serves as the first filtering step that separates valid traffic signal lights from those that need further checks or that are invalid. Eccentricity: Eccentricity is the ratio of the length between the two foci to the length of the major axis. Hence, perfect circle would have an eccentricity value of “0”, a line segment would have = 1 • Hence, lights that have a horizontal relative orientation, are in the eccentricity range of 0.25 <<0.75, and do not exceed the empirically determined vertical: horizontal ratio of 1:5 are asserted as valid. a) Relative Orientation b) Size Ratio Finding Eccentricity
Algorithm for asserting validity of detected light blobs
Lane Detection and Road Edge Detection • In order to detect lane markers and road edges, a linear Hough transform is used Luminance Edge Based Detection Color Edge Detection Super Imposed Bit Images Hough Transform Candidate Filtering For left and Right marker Detection Processing steps for (a) and (b) color edge detection and the (c) combined results for luminance- and color-based edge detection. (a) Original capture. (b) Color edges. (c) Luminance + color edge Super imposed binary edge detection
SYSTEM DESIGN Final TCSH Voting and Detection • After the filtering process is complete, the algorithm generates “Primary Status” (1S) and (if applicable) “Secondary Status” (2S) for each of the traffic lights. Arrows are classified as 2S, which is only generated when 1S is not reported. • Lane Detection Based Voting A = 0.30α + 0.70β α = distance from horizontal center of road β = distance from top of frame
SYSTEM DESIGN • Once a valid traffic light for the intersection is found, it is tracked until it leaves the field of view using a Linear Kalman filter (LKF). • The LKF can reliably track and predict the future position of the light at time t + 1, preventing the loss of lights due to inconsistent detection from frame to frame. • The Kalman filter tracks each of the trackers until the RemovalFlag is set to “1,” after which the tracker is discarded and the allocated memory is freed. • The RemovalFlag is set to 1 under two conditions: 1) when the traffic light cannot be reliably tracked for a frame rate equivalent of two consecutive seconds; 2) traffic light reaches one of the tracker destruction zones at the perimeter of the frame. • Using this information, it is possible to predict the next color of the light in the sequence. • This approach dramatically reduces the number of errors from incorrect color detection, in addition to serving as the final filtering step of the algorithm
Audio Signal Output for Individuals with CVD • This system can be integrated with a wide variety of alert signal mechanisms • The system simply outputs the necessary flags, which can be then transmitted to alert device with three set tone system OUTPUT OF THE TRAFFIC LIGHT DETECTION
ADVANTAGES COMPARISON WITH OTHER METHODS • Fully compliant standards-based (ITS and BSI) color limits: • Detection of arrows. • Detection of defective, non-standard lights, or incorrectly installed HVS • No requirement for a GPS system • No requirement for pre driving the course • Context-aware detection and next-state prediction • Fixed-point operations for increased efficiency
CONCLUSION • This system has presented a robust method for detecting the status of the traffic lights from visual information only. • It is the first work to use official ITE (U.Ss.) and BSI (EU) traffic light standards to define strict color thresholds for traffic light detection. • The presented system can be easily ported over for use on a low-power embedded smart camera as a stand-alone windshield mounted driver- assistance device. • This system was tested with over 50 h of real video over 2000 intersections, achieving 97.5% solid light detection accuracy. • Unlike other systems, the proposed system is able to detect faulty, arrow, and HVS lights. • Traffic light status can be reliably detected at 400 ft away from an intersection.
Thank You

More Related Content

Intelligent Traffic light detection for individuals with CVD

  • 1. BANGALORE INSTITUTE OF TECHNOLOGY DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING SEMINAR TITLE: MOBILE STANDARDS – BASED TRAFFIC LIGHT DETECTION IN ASSISTIVE DEVICES FOR INDIVIDUALS WITH COLOR – VISION DEFICIENCY BY SWAROOP ARADHYA 8TH SEMESTER B.E
  • 2. PROBLEM STATEMENT To design a mobile standard based traffic light detection system as an assistive device in vehicle for the individuals with color vision deficiency using image processing and artificial intelligence
  • 3. LITERATURE SURVEY • Akhan Almagambetov, Member, IEEE, Senem Velipasalar, Senior Member, IEEE, and Assel Baitassova “Mobile Standards-Based Traffic Light Detection in Assistive Devices for Individuals with Color-Vision Deficiency” An IEEE Transaction on Intelligent Transport Systems 2014 • A. Vu, A. Ramanandan, A. Chen, J. Farrell, and M. Barth, “Real-time computer vision/DGPS-aided inertial navigation system for lane-level vehicle navigation” IEEETrans.Intel.Transp.Syst., vol.13,no.2,pp.899– 913, Jun. 2012 • N. Fairfield and C. Urmson, “Traffic light mapping and detection,” in Proc. IEEE ICRA, 2011, pp. 5421–5426 • J. Levinson, J. Askeland, J. Dolson, and S. Thrun, “Traffic light mapping, localization, and state detection for autonomous vehicles,” in Proc. IEEE ICRA, 2011, pp. 5784– 5791 • Y. Shen, U. Ozguner, K. Redmill, and J. Liu, “A robust video based traffic light detection algorithm for intelligent vehicles,” in Proc. IEEE Intel. Veh. Symp., 2009, pp. 521–526
  • 4. EXISTING SYSTEM Current traffic light detection methods are divided into the following three categories. GPS or Map based Systems • These systems require the integration of GPS with an onboard vehicle camera. Coupled with a map, the GPS receiver pinpoints the location of the vehicle in relation to traffic intersections. • These methods, while robust, are very limited in their scope of use. • These methods involve a significant man-hours investment and courses have to be predefined on a map and pre-driven. • Any deviation from a set pre driven course would result in the failure of this system. • The appropriate regions of interest have to be marked by hand at each relevant intersection.
  • 5. EXISTING SYSTEM Image based Methods • These methods only use the image sensor for the detection of traffic lights. • Traffic light positions are determined either by observing the traffic signal head installations and localizing the light positions with respect to the signal head or the positions are inferred from detecting the colored lights themselves. • These types of algorithms are very efficient, although current state-of-the-art algorithms are not very robust. Uses no fail-safe or correction mechanisms, and is prone to failures (undetected lights) Infrastructure Modification • Infrastructure modifications are proposed by a number of vehicle manufacturers to enable communication between the vehicle and the surrounding infrastructure. • This, however, requires an overhaul of the current Intelligent Transportation Systems (ITS) and in- vehicle technologies and is unlikely to be implemented by even the most progressive municipalities.
  • 6. PROPOSED SYSTEM • This proposed system has presented a robust method for detecting the status of the traffic lights from visual information only (versus relying on traffic light localization via GPS, radar, lidar, or map-based prior knowledge) • The first to approach the problem from the standpoint of multiple light weight agents that are used to decompose a scene, each one integrating fail-safe mechanisms designed to prevent erroneous detection. • The presented system can be easily ported over for use on a low-power embedded smart camera as a stand-alone windshield-mounted driver-assistance device. (a) Hardware setup, (b) (c) View
  • 8. SYSTEM DESIGN Initial TCSH Light Identification • Standards-Based Soft Filtering: Standards-based color thresholds (UV components) are used for initial soft filtering of the image. • In order to speed up the computation time, the light color is not determined at this point of the algorithm. • Regions that fall within the specified soft thresholds are used regardless of their colors and are labeled “potential matches”. • The YUV colorspace separates the luminance (Y) and color data (UV) to describe a wide range of colors by adding an unconstrained brightness component
  • 9. SYSTEM DESIGN Morphological Operations: • In order to “cleanup” the image after the soft filtering step, mathematical morphology operations are applied for noise elimination. • After the mathematical morphology operations are run on each individual “blob” that matched the relaxed thresholds, Its properties are stored in a data structure shown below Morphological operations performed on an actual light capture. (a) RGB. (b) BW. (c) Final
  • 10. SYSTEM DESIGN Accurate Color Detection: • Once all of the detected blobs have been stored, each blob is analyzed, and a color is assigned to the blob through the ColorLabel variable. • Using V and H channel histogram, accurate color in the traffic light is detected. • After the color of the blob is reliably determined, it is stored into the ColorLabel field of the blob data structure. V-channel histogram for red and green H-channel histogram for yellow and red
  • 11. Light Assertion and Filtering Relative orientation • Relative orientation of a blob determines the rotation of a particular elliptical shape with respect to the major axis . • The relative orientation of each blob is calculated when the blob data structure is first created, and it serves as the first filtering step that separates valid traffic signal lights from those that need further checks or that are invalid. Eccentricity: Eccentricity is the ratio of the length between the two foci to the length of the major axis. Hence, perfect circle would have an eccentricity value of “0”, a line segment would have = 1 • Hence, lights that have a horizontal relative orientation, are in the eccentricity range of 0.25 <<0.75, and do not exceed the empirically determined vertical: horizontal ratio of 1:5 are asserted as valid. a) Relative Orientation b) Size Ratio Finding Eccentricity
  • 12. Algorithm for asserting validity of detected light blobs
  • 13. Lane Detection and Road Edge Detection • In order to detect lane markers and road edges, a linear Hough transform is used Luminance Edge Based Detection Color Edge Detection Super Imposed Bit Images Hough Transform Candidate Filtering For left and Right marker Detection Processing steps for (a) and (b) color edge detection and the (c) combined results for luminance- and color-based edge detection. (a) Original capture. (b) Color edges. (c) Luminance + color edge Super imposed binary edge detection
  • 14. SYSTEM DESIGN Final TCSH Voting and Detection • After the filtering process is complete, the algorithm generates “Primary Status” (1S) and (if applicable) “Secondary Status” (2S) for each of the traffic lights. Arrows are classified as 2S, which is only generated when 1S is not reported. • Lane Detection Based Voting A = 0.30α + 0.70β α = distance from horizontal center of road β = distance from top of frame
  • 15. SYSTEM DESIGN • Once a valid traffic light for the intersection is found, it is tracked until it leaves the field of view using a Linear Kalman filter (LKF). • The LKF can reliably track and predict the future position of the light at time t + 1, preventing the loss of lights due to inconsistent detection from frame to frame. • The Kalman filter tracks each of the trackers until the RemovalFlag is set to “1,” after which the tracker is discarded and the allocated memory is freed. • The RemovalFlag is set to 1 under two conditions: 1) when the traffic light cannot be reliably tracked for a frame rate equivalent of two consecutive seconds; 2) traffic light reaches one of the tracker destruction zones at the perimeter of the frame. • Using this information, it is possible to predict the next color of the light in the sequence. • This approach dramatically reduces the number of errors from incorrect color detection, in addition to serving as the final filtering step of the algorithm
  • 16. Audio Signal Output for Individuals with CVD • This system can be integrated with a wide variety of alert signal mechanisms • The system simply outputs the necessary flags, which can be then transmitted to alert device with three set tone system OUTPUT OF THE TRAFFIC LIGHT DETECTION
  • 17. ADVANTAGES COMPARISON WITH OTHER METHODS • Fully compliant standards-based (ITS and BSI) color limits: • Detection of arrows. • Detection of defective, non-standard lights, or incorrectly installed HVS • No requirement for a GPS system • No requirement for pre driving the course • Context-aware detection and next-state prediction • Fixed-point operations for increased efficiency
  • 18. CONCLUSION • This system has presented a robust method for detecting the status of the traffic lights from visual information only. • It is the first work to use official ITE (U.Ss.) and BSI (EU) traffic light standards to define strict color thresholds for traffic light detection. • The presented system can be easily ported over for use on a low-power embedded smart camera as a stand-alone windshield mounted driver- assistance device. • This system was tested with over 50 h of real video over 2000 intersections, achieving 97.5% solid light detection accuracy. • Unlike other systems, the proposed system is able to detect faulty, arrow, and HVS lights. • Traffic light status can be reliably detected at 400 ft away from an intersection.