Maxwell Zhou

Recreating a realistic world environment (plus “runaway chickens” noisy scenarios) is a very powerful tool for what-if analysis. If you have the good model, of course. And for populations, we have it #datagenesis

7

My paper on the Cruise robotaxi pedestrian dragging mishap has been updated and is now available in its final form. I'll be presenting this paper at Safecomp in September, but you can see the preprint right now. Thanks to the SafeComp reviewers for their review comments! If you haven't had a chance to look at this paper before, the short takeaway is that there is a lot buried in the external reports that is not obvious from a casual reading. My paper is the result of carefully assembling the pieces to tell a more direct story of the events that unfolded. Title: Anatomy of a Robotaxi Crash: Lessons from the Cruise Pedestrian Dragging Mishap (Safecomp 2024 practical experience report) Abstract: An October 2023 crash between a GM Cruise robotaxi and a pedestrian in San Francisco resulted not only in a severe injury, but also dramatic upheaval at that company that will likely have lasting effects throughout the industry. Is-sues stem not just from the loss events themselves, but also from how Cruise mishandled dealing with their robotaxi dragging a pedestrian under the vehicle after the initial post-crash stop. External investigation reports provide raw material describing the incident and critique the company's response from a regulatory point of view, but exclude safety engineering recommendations from scope. We highlight specific facts and relationships among events by tying together different pieces of the external report material. We then explore safety lessons that might be learned related to: recognizing and responding to nearby mishaps, building an accurate world model of a post-collision scenario, the in-adequacy of a so-called "minimal risk condition" strategy in complex situations, poor organizational discipline in responding to a mishap, overly aggressive post-collision automation choices that made a bad situation worse, and a reluctance to admit to a mishap causing much worse organizational harm down-stream. https://lnkd.in/e5h84-up

104

9 Comments

Its sometimes hard to believe that we live in an era where driverless vehicles aren't just a vision anymore but a reality. Waymo driverless cabs for example, are transforming the San Francisco streets forever! It was a wonderful experience working on an application of autonomous vehicles as part of my final project in the course CS267: Applications of Parallel Computing at the University of California, Berkeley. Our team, comprising Terence N., Zeid Solh, and myself, focused on parallelizing stereo vision for driverless applications. Stereo vision is crucial for self-driving cars as it builds a 3D understanding of the environment by using two cameras. However, real-time processing of these images is computationally expensive, posing a significant challenge for autonomous vehicles. We tackled this by exploring various parallel computing techniques to improve the speed of calculating the Disparity Map algorithm. We used popular techniques such as #SIMD (Single Instruction, Multiple Data), #OpenMP (multi-threading), and CUDA (NVIDIA GPU acceleration), and benchmarked the performance. By distributing the computational load across multiple cores and processors, we achieved substantial performance improvements, enabling real-time processing of high-resolution images and videos. The results were promising! Utilizing CUDA, we saw significant speedups, demonstrating the potential for efficient, real-time depth perception in autonomous systems. This project signified the importance of parallel computing in advancing autonomous driving technologies. I'm incredibly proud of our team's hard work and the insights we gained through this project. Check out our GitHub repo: https://lnkd.in/gYHrEU9z for more details on our implementation! #UCberkeley #CS267 #ParallelComputing #StereoVision #AutonomousDriving #DeepLearning #ComputerVision

50

3 Comments

Following up on my previous post, here is our video demonstration. If you have any questions, please feel free to contact me. 😁 Source code available at: https://lnkd.in/gZedX2uW #robotics #research #slam #localization #mapping #vision #roboticperception

95

4 Comments

One of the best things about being in San Francisco 🌉 is the ability to drop in on great talks. Last week I had a chance to see NVIDIA's CEO Jensen Huang (JH) being interviewed by Stripe's CEO Patrick Collison. Jensen is among the longest-serving founder-CEOs in Silicon Valley at the moment (30+ years). With that comes a special clarity of mind and some hot takes. 🌶👇 🗣 Jensen doesn't believe in 'praise in public and criticism in private'. Instead, he reasons that most information in the organization should not be privy to certain people. By giving feedback in public, you allow everyone to learn. 'What is better than learning from your own mistakes? It's learning from someone else's mistakes!!' 🔥 JH also doesn't like to fire people. He would rather "torture them into greatness". He invests in people and he doesn't like to write off his investments. 👨‍💻 Asked whether he likes his job every day, he answers "no". The perfect job doesn't give you happiness 100% of the time. "No great things were done, that were easy all the way." ... "Even if I come home from a day where I did things I didn't enjoy, it could still be have been a great day." 👩‍👧‍👦 Jensen famously has 60 direct reports. "This allowed us to cut out probably 7 levels of management". His subordinates don't necessarily have such large teams. "It doesn't scale down, less senior people need more guidance". Looking at the numbers, this approach allowed NVidia to reach ~2T market cap with ~28.000 people vs. Microsoft's 3T with ~220.000 people and Apple's 2.6T with 160.000 people. ❔ Jensen answers almost every question in the same way: "Let's break it down, let's reason about it". This is a great response on so many levels. First, it allows him to not be put on the spot. Second, it allows him to teach his workforce how he reasons through a problem so that they can emulate it and create alignment. Third, it gives people a chance to point out flaws in his reasoning. He admits he is not right all the time. ON AI 👨‍🔧 "You will not lose your job to AI, your company will go out of business because another company used AI." "If you're not using AI you're doing it wrong." ⏱ The extent to which AI will enter many processes is largely underestimated. Right now we are focused on language - sounds in a time sequence. "Any time sequence can be tokenized." This will include robotic movement. 🥛 JH also expects that we won't be satisfied with foundation models "You're going to want to tune it into perfection because you care so deeply. Say if it's already at 99% you'll want to get it to say 99.3%" This last piece is also why we created Query Vary (YC W22). AI will never feel truly intelligent until it completely understands your data and context. It needs to be taught, regardless of how powerful the foundation model is. Finally, besides insights, it was remarkable how funny Jensen was, cracking a joke at roughly a 1 per 3 min rate. It was a very entertaining talk. #ai #sf

61

8 Comments

Waymo's LiDAR Technology: A Comprehensive Look at Its Safety and Efficiency As a Waymo enthusiast, my friends often ask me about the safety of autonomous vehicles. Recently, concerns have resurfaced about the potential risks LiDAR might pose to passengers and pedestrians. In the Waymo Jaguar I-Pace, there are four LiDAR sensors strategically placed for comprehensive 360° coverage: one at the front, two on the sides, and one at the rear. These LiDAR sensors are part of a larger sensor suite that includes six radar units and fourteen cameras, ensuring strong detection capabilities at various ranges and speeds. While the total number of sensors in the Jaguar I-Pace is 32, the four LiDAR units themselves have remarkably low energy output. To put this into perspective, even if a vehicle were equipped with 1,000 LiDAR sensors, their total energy output would be equivalent to that of a single car headlight. This comparison shows not only the efficiency of Waymo's LiDAR technology but also its inherent safety. LiDAR sensors operate in the infrared spectrum and are classified as Class-1 lasers according to FDA standards, the safest category. This classification is shared with everyday devices like CD players and LED lights, ensuring they are harmless to human eyes and skin under normal usage conditions. This safety feature is important, given the proximity of these sensors to passengers and pedestrians. Waymo also collaborates with global standards bodies to ensure their LiDAR systems meet rigorous safety criteria under various environmental conditions. For individuals outside the vehicle, the risk of exposure to harmful LiDAR emissions is negligible due to low energy levels and rapid dissipation over distance. The advanced sensors not only ensure robust detection and navigation capabilities but also support Waymo's high-resolution mapping for autonomous navigation—underscoring their commitment to innovation and public safety. If you notice any inaccuracies in this post or have additional insights, please provide citations in the comments below and I will update my post accordingly. Thank you for helping enhance the accuracy and depth of this discussion. #Waymo #LiDAR #AutonomousVehicleSafety

5

🔥🚀 Exciting News from MIT: Introducing FlowMap for Deep Learning-based Structure from Motion! 🔥🚀 MIT unveils FlowMap, a groundbreaking self-supervised, end-to-end differentiable SfM method delivering COLMAP-level accuracy for 360° scenes. FlowMap revolutionizes the process by precisely solving camera poses, intrinsics, and dense depth of video sequences without relying on unique image features or manual matching. This innovation outperforms prior gradient-descent based methods and rivals COLMAP, marking a significant advancement in SfM technology. 👉 Explore the project page, paper, and code: https://lnkd.in/gyyinuTu #Reconstruction #SfM #AI #DeepLearning #Innovation

7

A few takeaways from SuperAI conference day 2: On open models: - Emad Mostaque emphasized the safety of open models and how releasing open source pushes safety standards and avoids single points of failure. Small edge models can suffice for many use cases. - Vipul Ved Prakash noted increased investment in inference efficiency, enabling more cost-efficient production apps. - Sean (Xiang) Ren highlighted that value comes from applications (the full software and data stack), not just the models themselves. On AI venture trends: - Harshjit Sethi predicts the rise of regional foundation model players as governments push for sovereignty and foreign companies won't collect local datasets for smaller markets. - Vishal Harnal sees AI becoming table stakes for software companies, as opposed to a distinct vertical. He uses AI personally with diverse applications from deal analysis to generating hip-hop album art for birthday party guests. - Prayank Swaroop observes the paradigm of using specialized models for different tasks emerging. He advises focusing on unique datasets. - Geetali Kumar is most interested in AI use cases in healthcare. Looking ahead, Girish Dilip Patil from AWS identified 3 key trends: natively multimodal models, autonomous agents, and small on-device models for privacy-sensitive use cases ( 👋 Opilot). SuperAI also had great food and drinks, cool cyberpunk exhibits, EDM music, top-notch people-watching, some unhinged speakers, and of course it was a great opportunity to meet interesting people. Overall I had a great time, thanks Prahlad Jaya for the ticket!

26

AI/ML+Bio is moving so fast that DiffDock is already outdated 😅 , thanks to an update called DiffDock-L 😎 , which is coming soon to 310 copilot. In the meantime, you can run the "classic" version.

30

This is #icra2024 Week, and I want to tell you about a very cool paper I spotted from Waymo's research team: Stateful Tracking with Transformers for Autonomous Driving! It's a 3D Multi-Object Tracker... but with Transformers rather than Kalman Filters. It's very "edgy", and I'm doing a complete breakdown in this video. Dragomir Anguelov (Drago) Qinru Li How did I do? Did I miss anything? #deeplearning #computervision #lidar #selfdrivingcars #robotics

119

6 Comments

✨ NVIDIA #CorrDiff: Resolving Extreme Weather Events With Generative #AI 🌡🧠 🚀 NVIDIA's Earth-2 Inference service is revolutionizing extreme weather forecasting by enabling users to combine global medium-range AI weather prediction models like NVIDIA’s FourCastNet with generative AI super-resolution / downscaling models like NVIDIA’s CorrDiff. 💡 This innovative approach enables the rapid exploration of numerous high-resolution weather forecast scenarios, providing decision-makers with a detailed understanding of a storm's potential impacts. 🎯 By empowering proactive strategies, this groundbreaking technology aims to minimize the loss of life and property damage caused by extreme weather events. 📽 Video Outline: ✅ As the earth’s climate changes, AI-powered weather forecasting is allowing us to more accurately predict and track severe storms like Super-Typhoon Chanthu which caused widespread damage in Taiwan, the Philippines, China, and Japan in 2021. ✅ Current AI forecast models can accurately predict the track of storms, but they are limited to 25-kilometer resolution, which can miss important details. NVIDIA's CorrDiff is a revolutionary new generative model trained on high-resolution radar-assimilated WRF weather forecasts and ERA5 reanalysis data. ✅ Using CorrDiff, extreme events like Chanthu can be super-resolved from 25km to 2km resolution, with 1,000 times the speed and 3,000 times the energy efficiency of conventional weather models. 🎥 Witness the impressive weather modelling by watching the video #Nvidia #CorrDiff #Weather #Resolving #Modelling #AI #Climate 👉 Follow me Levent Coskun to stay up to date on #robotics #AI #hightech and #innovation. To get notifications about my posts please activate the notification bell 🔔 in my profile

51

27 Comments

The focus on improving hand movements for better driving shows a practical approach to refining its capabilities and ensuring safety in real-world applications. As Musashi progresses through its testing phases and enhancements, it could potentially revolutionize the way we think about automated driving systems and open up new possibilities for the future of robotics and transportation. #AI #robotics #futurism #socialsaturday #letstalkfuture #ChangeLives #lederskap #ledelse #suksess #entreprenørskap #presentasjon #vokalisme #karriere #freementorship #læringskultur #YourJobOurWork #TechnicalProfessionals #Karriereutvikling #Rekruttering #Utenlandske #investeringer #Investeringsmuligheter #prosjektledelse #gratiskurs #upskill #boostyourcareer #digitaltransformasjon #teknologi #fremtiden #artificialintelligence #teknologi #fremtiden #ChatGPT #LighthouseCommunications #OpenAi #Shakespeare #innovasjon #bærekraft #projectmanagement #agile #business #learning #tech #prosjektledelse #learning #ai #midt #generativeai #midjourney 

5

🚕~ One slide in the deck of my talk about AI-controlled Humanoid Robots at Robotics Summit & Expo was triggering a particular amount of positive feedback: The dismantling of the Autonomy Levels used for autonomous cars by functionality and safety. Often referred to L1-L5, these actually describe both dimensions, but don't distinguish them - which is why, in my observations, they are often misunderstood: L2 doesn't mean that it can do less than >L3, it just means that it can't do that on its own. This is because the levels are now officially awarded according to the safety aspect, not the functional aspect. Tesla has an AI driver that can drive for you, A to B, quite reliably. Mercedes-Benz AG has driving automation that drives for you in the traffic jam (or up to 60kph/37mph). Now the other aspect is the safety certification of these. Mercedes let’s you focus on other tasks when in automatic driving, it's certified to do that reliabily. Tesla’s FSD, you have to watch at all times, because it’s not certified to be able to drive alone, even that it can functionally do much more. AI-controlled robots have the same challenge like Tesla's AI driver, when it comes to behavioral safety: With end-to-end neural control, there is no way to certify safety based on the classical methods of functional safety with full traceability from requirement to verification of the implementation. It can only be certified with statistics, which is very hard. At Synapticon, where we're leading the market with functional safety for motion control, we are working on bringing robot safety to the next level: Enabling safe, autonomous, intelligent robots that humans can rely on won't harm them, on a motion AND behavioral level. Stay tuned for POSITRON AI SAFETY updates 🦾 Let me know in the comments if you want a copy of the complete deck of my talk. I'll also present it next week on the Humanoid Summit Shanghai as well as on the Konradin Verlag RobotX Forum on 20th June - here added by explanations on how the robotic software stack is currently transformed by AI. #robotaxi #humanoids #robotics #safety #aisafety

49

6 Comments

Waymo searching for another head-on collision. This is yet another avoidable example of our lack of intelligent infrastructure. It is time to accelerate the Path to Commerce. Trying to deploy autonomous systems without INTELLIGENT INFRASTRUCTURE is like trying to build the internet without networks. The fundamental challenges of mobility are optimization, orchestration, reducing emissions, and addressing Vision Zero. The root challenges can only be addressed with INTELLIGENT INFRASTRUCTURE. PINNs, E-GPS, Sensors, Edge Compute, Active Digital Twins, Data Exchanges, . . . The Federal Infrastructure Bank and infrastructure investors are mobilizing $billions in private capital to build the Roads of the 21st Century. Intelligent Infrastructure Economic Zones will revitalize infrastructure in the United States. https://lnkd.in/gWnJD877 INTELLIGENT INFRASTRUCTURE (the foundation of ARPA-I) goes well beyond transportation and will be the brains of our economy. Building the Infrastructure for the Next 100 Years. Autonomy Institute https://lnkd.in/gd4tNKyi Other issues caused by the lack of Intelligent Infrastructure: https://lnkd.in/gfjBJJ-a https://lnkd.in/gvVkQRwF https://lnkd.in/gvfVaZam #infrastructure #future #transportation #arpa-i #arpax #jobs #bank #digital #innovation #data #people #visionzero #continuumofcapital #dataexchange #partnerships #supplychains #digitaltwin #privacy #investment #p3programs

35

18 Comments

Learn how to squeeze the maximum performance out of OpenCV’s Deep Neural Network (DNN) module using Intel’s OpenVINO. https://lnkd.in/eq4TjmGR

1

New research on holographic AR glasses by Stanford Computational Imaging Lab. Interesting to see how image quality compares to conventional optics. "Current waveguide designs require projection optics with a thickness proportional to the focal length of the projection lens, introducing optical bulk, and they are limited to displaying two-dimensional (2D) images at a fixed distance to the user. These limitations result in reduced perceptual realism and visual discomfort due to the vergence–accommodation conflict."

28

1 Comment

😸How jerry can catch the tom in a better way ?🐭 In GAN , Discriminator - Jerry Generator - Tom In 2017, a paper released for the stabilisation of GAN and in this video we will understand what the big changes that was introduced. https://lnkd.in/gp_yRRak

7