Same Silicon, More Intelligence for Automated Driving

Our co-inventor agents worked with engineers at a Detroit-based automaker to speed up their behavior prediction model, up to 14.6x for certain variations, recovering compute budget that can be reinvested in more intelligence on the same silicon. Maximizing efficiency like this serves their mission to democratize hands-free driving, bringing automated technology to their most affordable EV.

An advanced driver-assistance system (ADAS) on a $30,000 vehicle has a fixed compute budget shared across every model and software component, so nothing gets more without something else getting less. The goal is to maximize intelligence within that package. True to the bitter lesson, scale produces smarter models, and automated driving technology is no exception. The engineering problem then becomes how to fit larger models within these constraints.

The AI model

The ADAS technology consists of a stack of models working in sequence. Sensors feed perception, which builds a picture of what surrounds the car; prediction anticipates how that picture will change; planning decides what the car should do about it. Driving is impossible without the behavior prediction step in the middle, since every choice the planner makes depends on anticipating where everything else will be a few seconds in the future.

Behavior prediction is also uniquely challenging, because motion forecasting lives at the intersection of physics, road geometry, and multi-agent interaction. Vehicles move through Cartesian space, drive along curvilinear road structures, and respond to other agents in ways that are naturally graph-like.

The automaker uses a Transformer-based encoder-decoder model to run behavior prediction in production. The encoder takes map data and tracked objects and produces a vector embedding of the scene. The decoder takes that embedding and predicts where other vehicles will be.

Our co-inventor agents and the automaker’s engineers worked hand-in-hand to improve the intelligence-latency tradeoff of kernels for the encoder-decoder, achieving up to 5.6x on encoders, and 14.6x on decoders. The co-inventors achieved these speedups by searching over and combining various strategies, such as kernel fusion and adaptive data processing. By executing this invention process quickly and rigorously, our co-inventors helped the automaker iterate faster and fit more intelligence into the same latency budget.

Working with co-inventor agents

Full-stack systems engineering is one of the frontier applications of AI. Making a model faster in production means reasoning across the architecture, the compiler, the runtime, and the silicon at the same time.

Our co-inventors automate the innovation process, working with human teams to help people find better solutions faster and having a direct impact in real production use cases. The automaker’s engineers set the target, the constraints, and the acceptance criteria, then review what comes back. In between, the co-inventors run the search and optimization process in hours to days.

Working with our co-inventors gave the automaker’s team more leverage: the compute-optimization work happened in parallel, rather than pulling attention away from on-road model quality. By acting as a force multiplier on the systems side, we enabled their team to stay focused on the domain problems they know best while still moving faster toward a more capable deployment.

Automating innovation on edge AI chips

Our co-inventors ran experiments inside a sandbox that emulates a part of the automaker’s production environment hardware (including the ARM CPU) and software (including the PyTorch and ONNX inference stack). We found that this was sufficient to make the results translate directly into better performance on their production car.

Working with edge AI presented very different technical boundary conditions than our work on datacenter optimization. Every chip and use case presents its own dialect and idiosyncrasies, and edge optimization sits further out of distribution; far more work has been published about optimizing server GPUs than embedded ARM chips. Our co-inventors are flexible and able to adapt to new environments and considerations.

Integration work spanned the full depth of the inference stack, including the Open Neural Network Exchange (ONNX) Runtime that executes the model, the custom C++ operators it dispatches to, and the ARM NEON instructions underneath those. Within that environment, the co-inventors reasoned about whether a given kernel is memory-bound or compute-bound, proposed candidate implementations, profiled them, and refined them.

Two techniques show up repeatedly: sparsity exploitation, which skips the large portions of the input that carry no signal, and kernel fusion, which collapses a sequence of operations so intermediate results don’t have to be written out to memory and read back.

Both are well understood in principle but tedious to implement and verify case by case. Our co-inventors used a rigorous process to find efficient implementations and navigate subtleties when implementing kernels on edge devices.

Verifying results

For the optimized kernels to be useful in production, every change had to verifiably preserve the behavior of the automaker’s baseline model within a strict tolerance.

The stringent verification process enforced by our co-inventors is especially critical for automated driving technology, because the code is intended for real deployed settings where model behavior needs to remain consistent under production conditions.

Every candidate kernel was checked end to end against reference outputs from the automaker’s test data. The optimized encoder-decoder outputs matched the reference to within millimeters.

Correctness is a first-class citizen for our co-inventors, enabling them to detect subtle numerical inconsistencies in candidate kernels and keep the final ones well inside tolerance on real data, which is the same principle behind our work on verified C-to-Rust translations.

Looking forward

The co-invention loop that produced these kernels is not specific to this chipset or this model, and the same framework can be applied to a variety of hardware and architecture targets.

End-to-end inference optimization sits at the cutting edge of AI and pays off immediately, in latency recovered and capability shipped. Our co-inventors demonstrate that frontier agents can produce verified, deployable code inside real edge constraints.