Skip to content

Part XVIII: Company deep dives

Why this matters at staff level. Staff interviews at these companies are not abstract. The system-design prompt is their own product ("design the perception stack for a rider-only robotaxi", "design the RLHF data pipeline for our next model"), the ML-depth questions follow the choices their engineering blogs already describe, and the strongest signal is a candidate who can say why the company chose what it chose, what it rejected, and how it measures the result. This part personalises Part XVII to sixteen companies using only their public write-ups.

How to use the deep dives

Each company page has the same six sections, so you can read one in forty minutes the night before a loop:

  1. The business in one paragraph and the ML problems that define the company (problem → why it is hard → public evidence).
  2. The stack as publicly described: a mermaid diagram assembled from talks, papers and blog posts, with what is known separated from what is inferred.
  3. Deep dives: three to six mini-chapters on paradigms the company owns, each with the problem, the approach (linked to the technique chapter that derives the math), the trade-off they chose versus the alternative, the sources, and a boxed "How to say it in the interview" script.
  4. Likely interview questions (10–15) phrased as their business problem, each with a staff-level answer sketch and a script that cites the public source behind the rationale.
  5. What to bring from your background: how large-scale perception / OCR / detection plus ML-systems experience maps onto their problems.
  6. Sources: verified links grouped by type. Where a link could not be verified we give the title, venue and year instead.

Two rules the pages follow, and that you should follow in the room: never attribute a design choice to a company without a public source, and mark inferences as inferences ("the AI Day talk implies…"). Interviewers at these companies know what is public and what is not; confident claims about their internals are a red flag.

The scripts use a fixed shape borrowed from Hello Interview: decision → rejected alternative → trade-off → how I would evaluate it, in the first person, six to twelve sentences, with a named paper, talk or blog post carrying the rationale.

Companies × paradigms

Companies × paradigms heatmap

The heatmap is an editorial weighting of what each company's public engineering write-ups emphasise (••• defines the company, •• important, • present). Read down a column to see who else interviews for the paradigm you are strongest in; read across a row to see which chapters a given loop will draw on.

Company Perception / autonomy Ranking / recs Fraud / anomaly LLM / post-training Systems / infra Data engines
Tesla ••• • ••• •••
Waymo ••• •• •• •••
Zoox, Nuro, Aurora ••• • •• ••
NVIDIA ••• • ••• ••• ••
Meta • ••• •• ••• ••• ••
Google & YouTube • ••• •• ••• ••• ••
TikTok / ByteDance • ••• •• •• ••• ••
Netflix & Spotify ••• • • •• •
Pinterest •• ••• • • •• •
Uber & DoorDash • ••• ••• • •• •
Airbnb • ••• •• • •• •
Amazon •• ••• ••• ••• ••• ••
Apple ••• •• • ••• ••• •
Stripe & fintech • ••• • •• •
OpenAI, Anthropic & DeepMind • ••• ••• •••
Scale AI & data engines •• ••• • •••

Which Part XVII chapters to master for each company

The system-design round is graded on the framework: requirements, trade-offs, a committed decision, and the reasoning. The table names the two or three Part XVII chapters whose structure you should be able to reproduce cold for each company, plus the technique parts they lean on.

Company Part XVII chapters to master Technique parts that carry the depth round
Tesla AV perception, ML platform & monitoring XI Perception & autonomy, X Weak supervision & auto-labelling, XIV Systems, XII Imitation learning
Waymo AV perception, Evaluation (treat as a design chapter), ML platform XI Sensor fusion, XI Prediction & planning, XI World models, VIII VLM architecture, XIII Uncertainty & reliability
Zoox, Nuro, Aurora AV perception, Forecasting & ETA (for fleet ops) XI Sensor fusion, XI Tracking, XII Imitation learning, XV Safety & failure modes
NVIDIA AV perception, LLM assistant with RAG (for inference-serving roles), ML platform XIV Hardware, memory & roofline, XIV Inference systems, VI Quantization, IX Diffusion, XI World models
Meta, Google/YouTube, TikTok Feed ranking, Ads CTR, Content moderation II Classical ML, XIII Retrieval, XIV Systems, VII Post-training
Netflix, Spotify, Pinterest Feed ranking, Visual search, Notifications & experimentation VIII CLIP & contrastive, XIII Retrieval, I Statistics
Uber, DoorDash, Airbnb Forecasting & ETA, Search ranking, Fraud II Trees & ensembles, XIII Uncertainty, XVII Experimentation
Amazon Search ranking, Fraud, OCR & document understanding, LLM assistant IV Detection, XIII Retrieval, XIV Inference systems
Apple Visual search, OCR & document understanding, LLM assistant VI Quantization, VI Fine-tuning & LoRA, IV Vision
Stripe & fintech Fraud & anomaly, ML platform II Classical ML, XIII Uncertainty, I Statistics
OpenAI, Anthropic, DeepMind LLM assistant with RAG, ML platform, Content moderation (as safety classifiers) VI LLM training, VII Post-training, XIV Systems, XV Interpretability & safety, XIII Evaluation
Scale AI & data engines ML platform, AV perception (the labelling half), LLM assistant (the eval half) X Weak supervision & auto-labelling, VII Reward models, XIII Evaluation

The pages

Autonomy, robotics, hardware and frontier labs

  • Tesla (Autopilot / FSD / Optimus). vision-only multi-camera BEV → occupancy → end-to-end FSD V12; the data engine; Dojo; Optimus.
  • Waymo. lidar-centric fusion, Wayformer / MotionLM prediction, Waymax and closed-loop evaluation, EMMA, the safety framework and rider-only metrics.
  • Zoox, Nuro, Aurora & robotaxi peers. a purpose-built bidirectional vehicle, a licensable AI-first driver, and a verifiable-AI trucking stack.
  • NVIDIA (DRIVE, Cosmos, Isaac). DRIVE platform, Hydra-MDP, Cosmos world foundation models, GR00T, TensorRT-LLM / Dynamo, training infra.
  • OpenAI, Anthropic & DeepMind. pretraining and scaling, post-training (RLHF / Constitutional AI / RLVR / reasoning), evals and safety frameworks, inference, multimodal, agents.
  • Scale AI & data engines. the data-engine business, RLHF data, SEAL evaluations, AV labelling and auto-labelling, label cost/quality trade-offs.

Consumer, marketplace and fintech (written by the other author of this part)

Two figures that recur across the autonomy pages

The AV data-engine flywheel

The flywheel above is the shape behind every AV page: fleet → triggers → upload → offline auto-labelling → human QA on the hard slice → training → evaluation → OTA. Look at the two dashed chords: evaluation feeds backwards into both triggers and training, which is why a company's evaluation stack determines its iteration speed.

Publicly announced architecture shifts at Tesla and Waymo

The timeline shows that the two best-documented AV programmes moved in the same direction (more learned structure, fewer hand-written interfaces) from opposite starting points: Tesla from cameras and fleet scale, Waymo from lidar and a safety case. In an interview, being able to place a question on this timeline ("that was the occupancy era; V12 changed the interface") is itself a signal.