Understanding AI Safety & Decision-Making with Professor Kochenderfer

We have all heard about autonomous vehicles, especially self-driving cars. But did you know that many of the same principles apply to self-driving airplanes? As the era of autonomous transportation becomes a reality, critical questions about safety and decision-making arise: How do algorithms account for randomness in autonomous systems like cars and aircraft? How do these systems ensure safety during unpredictable circumstances? How do imperfect sensors impact overall vehicle safety? How is future uncertainty modeled? And, importantly, how can artificial intelligence (AI) be trained to handle rare, high-stakes scenarios? To explore these questions, I interviewed Mykel Kochenderfer, an Associate Professor of Aeronautics and Astronautics and Associate Professor, by courtesy, of Computer Science at Stanford University and Director of the Stanford Intelligent Systems Laboratory (SISL).

Professor Kochenderfer’s work centers on creating advanced algorithms and analytical methods for decision-making in dynamic, uncertain environments. His team focuses on high-stakes systems like air traffic control, unmanned aircraft, and automated vehicles, where safety and efficiency are paramount. By leveraging probabilistic models and optimization techniques, they aim to design robust systems capable of adapting to real-world variability.

Modeling Randomness In Autonomous Systems

When asked about randomness in autonomous systems, Professor Kochenderfer highlighted the inherent variability of real-world environments. “The systems we build, whether for aircraft or cars, need to interact with the real world,” he explained. “And the real world has a tremendous amount of variability. There are other drivers on the road and pedestrians, and there is this inherent randomness. People don’t always walk straight. Cars don’t always follow the speed limit.”

To account for this, the team uses probabilistic models that assign different weights to possible outcomes, optimizing decision-making strategies based on these probabilities. For example, “Most of the time, aircraft fly straight, but sometimes they turn left or right. It’s important to weigh the different possible futures appropriately,” he noted. Their methodologies optimize objectives such as reaching a destination safely while minimizing passenger discomfort.

Data collection is a critical part of this process. For aircraft, radar tracks from the Federal Aviation Administration can be used to build statistical models of behavior during encounters. For driving, publicly available datasets from organizations like Waymo serve as valuable resources for modeling naturalistic behaviors.

The Role Of Imperfect Sensors

Imperfect sensors are another challenge in autonomous systems. “When you have imperfect sensors, your understanding of the world will inherently be imperfect,” Professor Kochenderfer explained. This makes it difficult to predict future events and complicates decision-making.

To address these limitations, decision-making strategies must be more robust and conservative, accounting for sensor noise and occlusions. “We strive to plan in a way that acknowledges these limitations,” he said. “While it’s challenging to create a system that is 100% safe, we work to ensure a very high level of safety by identifying vulnerabilities and characterizing expected failure rates.”

Modeling Future Uncertainty

A central question in the design of autonomous systems is how to model future uncertainty. Professor Kochenderfer described the use of probability distributions parameterized to reflect observed data. Metrics like log-likelihood are used to measure how well these models capture uncertainty. An additional validation method involves simulating trajectories that are indistinguishable from real-world data, akin to a variation of the Turing test in AI. “Simulations that look realistic to human experts can help establish confidence that our models are appropriate,” he said.

Incorporating Human Experience Into AI Training

Human expertise remains essential in designing AI systems capable of handling rare or edge-case scenarios. According to Professor Kochenderfer, “Building these systems requires both data and expert judgment.” By automating the optimization of decision-making strategies and validating them against human experiences, discrepancies between optimized system behaviors and human judgment can be analyzed to refine models.

Edge cases pose a significant challenge because it is impossible to validate every scenario against human judgment. The design process involves prioritizing human effort to ensure safety in critical situations. This iterative approach balances automation and human input to create more reliable systems.

Balancing Safety And Operational Efficiency

The balance between safety and operational efficiency is delicate. Overly cautious systems that frequently apply hard brakes, for example, could frustrate users or even cause secondary accidents. Conversely, insufficient caution can compromise safety. “Getting that balance right is really tricky,” Professor Kochenderfer said.

To address this complexity, his team has developed tools that help designers weigh numerous metrics within safety and efficiency categories. These tools aim to simplify the process of creating systems that are both effective and practical for real-world use.

Regulatory And Ethical Considerations

The role of government policy is critical in fostering safe and responsible innovation. “The Department of Transportation, for example, has a challenging job,” Professor Kochenderfer remarked. “Their top priority is safety, and they’ve achieved an incredible safety record in aviation. But they also recognize the potential of emerging technologies to bring additional safety benefits.”

Balancing innovation with regulation is no easy task, especially for emerging technologies like AI. Policymakers must encourage advancements while preventing premature deployment. Professor Kochenderfer emphasized the importance of a measured approach, learning from past mistakes and adopting incremental steps to build trust in autonomous systems.

Inspiring The Next Generation

As the interview concluded, Professor Kochenderfer shared advice for young students interested in the intersection of transportation and technology. “Develop good study habits and cultivate an interest in mathematics, statistics, and optimization,” he said. “The underlying mathematics of AI is unbelievably fun and creative. Additionally, learning how to work effectively in teams is crucial, as these technologies require collaboration on a massive scale.”

Conclusion

The development of autonomous systems presents a unique set of challenges and opportunities. By leveraging probabilistic modeling, robust algorithms, and a blend of human expertise and data-driven optimization, researchers like Professor Kochenderfer are paving the way for safer, more efficient transportation systems. As we stand on the brink of a transformative era in mobility, their work underscores the importance of innovation grounded in safety and rigorous validation.