References¶

Consolidated bibliography. Each chapter carries its own reference list; this page collects the primary sources that recur across the book.

This page is generated from the References section of every chapter by scripts/build_references.py. It lists 1005 distinct sources with the chapters that cite each one. A source appears under "Cited by title" when no URL for it could be verified, in which case search the exact title and venue.

arXiv preprints and papers¶

492 sources.

Team OLMo, "2 OLMo 2 Furious", 2024 (arXiv:2501.00656).
Cited in: part01, part06, part06
X. Weng et al., "3D Multi-Object Tracking: A Baseline and New Evaluation Metrics", IROS 2020 (arXiv:1907.03961).
Cited in: part01
T. Dettmers, M. Lewis, S. Shleifer & L. Zettlemoyer, "8-bit Optimizers via Block-wise Quantization", ICLR 2022 (arXiv:2110.02861).
Cited in: part01
Hendrycks & Gimpel, "A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks", ICLR 2017, arXiv:1610.02136.
Cited in: part13
Liu, Z. et al. (2022). A ConvNet for the 2020s (ConvNeXt). arXiv:2201.03545 · Liu, Z. et al. (2021). Swin Transformer. arXiv:2103.14030
Cited in: part03, part04, part04, part08
Azar et al. (2023). A General Theoretical Paradigm to Understand Learning from Human Preferences (IPO). arXiv 2310.12036.
Cited in: part07
Angelopoulos & Bates, "A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantification", 2021, arXiv:2107.07511.
Cited in: part13
V. Dumoulin and F. Visin, "A guide to convolution arithmetic for deep learning", 2016, arXiv:1603.07285.
Cited in: part04
Markov, T. et al. "A Holistic Approach to Undesired Content Detection in the Real World." AAAI 2023 (arXiv:2208.03274).
Cited in: part17
Chen, Kornblith, Norouzi, Hinton, A Simple Framework for Contrastive Learning of Visual Representations (SimCLR), ICML 2020. arXiv:2002.05709
Cited in: part10
Lee et al., "A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks", NeurIPS 2018, arXiv:1807.03888.
Cited in: part13
Karras, Laine, Aila, A Style-Based Generator Architecture for Generative Adversarial Networks, CVPR 2019. arXiv:1812.04948
Cited in: part09
D. Russo, B. Van Roy, A. Kazerouni, I. Osband & Z. Wen, "A Tutorial on Thompson Sampling", Foundations and Trends in ML, 2018 (arXiv:1707.02038).
Cited in: part01
Lundberg, S., Lee, S.-I. "A Unified Approach to Interpreting Model Predictions" (SHAP). NeurIPS 2017 (arXiv:1705.07874).
Cited in: part17
Black et al., π₀: A Vision-Language-Action Flow Model for General Robot Control, Physical Intelligence, 2024. arXiv:2410.24164.
Cited in: part08
Xinshuo Weng et al. "AB3DMOT: A Baseline for 3D Multi-Object Tracking and New Evaluation Metrics." IROS 2020. arXiv:2008.08063
Cited in: part11
Guo, R. et al. "Accelerating Large-Scale Inference with Anisotropic Vector Quantization" (ScaNN). ICML 2020 (arXiv:1908.10396).
Cited in: part17
P. Goyal et al., "Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour", 2017 (arXiv:1706.02677).
Cited in: part01, part03, part03, part04
Zhai, J. et al. "Actions Speak Louder than Words: Trillion-Parameter Sequential Transducers for Generative Recommendations." ICML 2024 (arXiv:2402.17152).
Cited in: part17
Sener, Savarese, Active Learning for Convolutional Neural Networks: A Core-Set Approach, ICLR 2018. arXiv:1708.00489
Cited in: part10
N. Shazeer & M. Stern, "Adafactor: Adaptive Learning Rates with Sublinear Memory Cost", ICML 2018 (arXiv:1804.04235).
Cited in: part01
D. Kingma & J. Ba, "Adam: A Method for Stochastic Optimization", ICLR 2015 (arXiv:1412.6980).
Cited in: part01
Miller, "Adding Error Bars to Evals: A Statistical Approach to Language Model Evaluations", Anthropic, 2024, arXiv:2411.00640.
Cited in: part13
Ktena, S. I. et al. "Addressing Delayed Feedback for Continuous Training with Neural Networks in CTR prediction." RecSys 2019 (arXiv:1907.06558).
Cited in: part17
Brown et al., "Adversarial Patch", 2017, arXiv:1712.09665.
Cited in: part13
S. Xie et al., "Aggregated Residual Transformations for Deep Neural Networks", CVPR 2017, arXiv:1611.05431 (ResNeXt).
Cited in: part04
Johari, R., Pekelis, L., Walsh, D. J. "Always Valid Inference: Bringing Sequential Analysis to A/B Testing." 2015 (arXiv:1512.04922).
Cited in: part17
B. Singh and L. Davis, "An Analysis of Scale Invariance in Object Detection: SNIP", CVPR 2018, arXiv:1711.08189.
Cited in: part04
S. McCandlish, J. Kaplan, S. Amodei et al., "An Empirical Model of Large-Batch Training", 2018 (arXiv:1812.06162).
Cited in: part01
Shi, B., Bai, X., Yao, C. "An End-to-End Trainable Neural Network for Image-based Sequence Recognition and Its Application to Scene Text Recognition" (CRNN). IEEE TPAMI 2017 (arXiv:1507.05717).
Cited in: part17
Dosovitskiy et al., An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale, ICLR 2021. arXiv:2010.11929.
Cited in: part08
Haldar, M. et al. "Applying Deep Learning to Airbnb Search." KDD 2019 (arXiv:1810.09591).
Cited in: part02, part17
A. Vaswani et al., "Attention Is All You Need", NeurIPS 2017 (arXiv:1706.03762).
Cited in: part01, part01, part03, part03
Jain, S., Wallace, B. C. Attention is not Explanation. NAACL 2019. arXiv:1902.10186.
Cited in: part15
Wiegreffe, S., Pinter, Y. Attention is not not Explanation. EMNLP 2019. arXiv:1908.04626.
Cited in: part15
Borsos et al., AudioLM: a Language Modeling Approach to Audio Generation, IEEE/ACM TASLP 2023. arXiv:2209.03143.
Cited in: part08
D. Kingma & M. Welling, "Auto-Encoding Variational Bayes", ICLR 2014 (arXiv:1312.6114).
Cited in: part01, part01, part02, part02, part09
Baydin, A. G., Pearlmutter, B. A., Radul, A. A., Siskind, J. M. (2018). Automatic differentiation in machine learning: a survey. JMLR. arXiv:1502.05767
Cited in: part01, part03, part03
Wang et al. Auxiliary-Loss-Free Load Balancing Strategy for Mixture-of-Experts. 2024. arXiv:2408.15664
Cited in: part06
Lin et al. AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration. MLSys 2024. arXiv:2306.00978
Cited in: part06
He, T. et al. (2018). Bag of Tricks for Image Classification with Convolutional Neural Networks. arXiv:1812.01187
Cited in: part03
Ioffe, S., Szegedy, C. (2015). Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. arXiv:1502.03167
Cited in: part03
Ioffe, S. (2017). Batch Renormalization. arXiv:1702.03275
Cited in: part03
Houlsby et al., "Bayesian Active Learning for Classification and Preference Learning" (BALD), 2011, arXiv:1112.5745.
Cited in: part13
Bao, Dong, Piao, Wei, BEiT: BERT Pre-Training of Image Transformers, ICLR 2022. arXiv:2106.08254
Cited in: part10
Yao et al., "τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains", 2024, arXiv:2406.12045.
Cited in: part13
Hendrycks & Dietterich, "Benchmarking Neural Network Robustness to Common Corruptions and Perturbations", ICLR 2019, arXiv:1903.12261.
Cited in: part13
Yinhao Li et al. "BEVDepth: Acquisition of Reliable Depth for Multi-view 3D Object Detection." AAAI 2023. arXiv:2206.10092
Cited in: part11
Zhiqi Li et al. "BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformers." ECCV 2022. arXiv:2203.17270
Cited in: part08, part11, part11, part17
Zhijian Liu et al. "BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representation." ICRA 2023. arXiv:2205.13542
Cited in: part11, part11, part17, part17
Sardana et al. Beyond Chinchilla-Optimal: Accounting for Inference in Language Model Scaling Laws. ICML 2024. arXiv:2401.00448
Cited in: part06
Beal, J. et al. "Billion-scale pretraining with vision transformers for multi-task visual representations." WACV 2022 (arXiv:2108.05887).
Cited in: part17
Johnson, J., Douze, M., Jégou, H. "Billion-scale similarity search with GPUs." 2017. arXiv:1702.08734; FAISS wiki The index factory
Cited in: part02, part02, part17
Li et al., BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models, ICML 2023. arXiv:2301.12597.
Cited in: part08
Grill et al., Bootstrap Your Own Latent (BYOL), NeurIPS 2020. arXiv:2006.07733
Cited in: part10
H. Kervadec et al., "Boundary loss for highly unbalanced segmentation", MIDL 2019, arXiv:1812.07032.
Cited in: part04
S. Zhang et al., "Bridging the Gap Between Anchor-based and Anchor-free Detection via Adaptive Training Sample Selection", CVPR 2020, arXiv:1912.02424 (ATSS).
Cited in: part04
Yifu Zhang et al. "ByteTrack: Multi-Object Tracking by Associating Every Detection Box." ECCV 2022. arXiv:2110.06864
Cited in: part11
Ovadia et al., "Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift", NeurIPS 2019, arXiv:1906.02530.
Cited in: part13
Prokhorenkova, L. et al. "CatBoost: unbiased boosting with categorical features." NeurIPS 2018. arXiv:1706.09516
Cited in: part02
Yin, T., Zhou, X., Krähenbühl, P. "Center-based 3D Object Detection and Tracking" (CenterPoint). CVPR 2021 (arXiv:2006.11275).
Cited in: part04, part17
Wei et al. (2022). Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. arXiv 2201.11903.
Cited in: part07
Chameleon Team (Meta FAIR), Chameleon: Mixed-Modal Early-Fusion Foundation Models, 2024. arXiv:2405.09818.
Cited in: part08
Chiang et al., "Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference", 2024, arXiv:2403.04132.
Cited in: part13
Besiroglu et al. Chinchilla Scaling: A replication attempt. 2024. arXiv:2404.10102
Cited in: part06
Romano, Sesia & Candès, "Classification with Valid and Adaptive Coverage" (APS), NeurIPS 2020, arXiv:2006.02544.
Cited in: part13
Ho, Salimans, Classifier-Free Diffusion Guidance, 2022. arXiv:2207.12598
Cited in: part09
Luo et al., CLIP4Clip: An Empirical Study of CLIP for End to End Video Clip Retrieval, Neurocomputing 2022. arXiv:2104.08860.
Cited in: part08
Hessel et al., "CLIPScore: A Reference-free Evaluation Metric for Image Captioning", EMNLP 2021, arXiv:2104.08718.
Cited in: part13
Han et al., Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labels, NeurIPS 2018. arXiv:1804.06872
Cited in: part10
Rozière et al. Code Llama: Open Foundation Models for Code. 2023. arXiv:2308.12950
Cited in: part06
Khattab, O., Zaharia, M. "ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT." SIGIR 2020 (arXiv:2004.12832).
Cited in: part17
Lin et al., Common Diffusion Noise Schedules and Sample Steps are Flawed, WACV 2024. arXiv:2305.08891
Cited in: part09
Shi, H.-J. M. et al. "Compositional Embeddings Using Complementary Partitions for Memory-Efficient Recommendation Systems." KDD 2020 (arXiv:1909.02107).
Cited in: part17
Amodei et al. (2016). Concrete Problems in AI Safety (reward hacking taxonomy). arXiv 1606.06565.
Cited in: part07
Meng et al., Conditional DETR for Fast Training Convergence, ICCV 2021. arXiv:2108.06152.
Cited in: part08
Tibshirani et al., "Conformal Prediction Under Covariate Shift", NeurIPS 2019, arXiv:1904.06019.
Cited in: part13
Romano, Patterson & Candès, "Conformalized Quantile Regression", NeurIPS 2019, arXiv:1905.03222.
Cited in: part13
Song, Dhariwal, Chen, Sutskever, Consistency Models, ICML 2023. arXiv:2303.01469
Cited in: part09, part09
Bai et al. (2022). Constitutional AI: Harmlessness from AI Feedback. arXiv:2212.08073.
Cited in: part07, part07, part07, part07
NVIDIA. "Cosmos World Foundation Model Platform for Physical AI." 2025. arXiv:2501.03575
Cited in: part11
Yun, S. et al. (2019). CutMix. arXiv:1905.04899
Cited in: part03
Liu et al., DAB-DETR: Dynamic Anchor Boxes are Better Queries for DETR, ICLR 2022. arXiv:2201.12329.
Cited in: part08
Yu et al. (2025). DAPO: An Open-Source LLM Reinforcement Learning System at Scale. arXiv 2503.14476.
Cited in: part07
Ratner, De Sa, Wu, Selsam, Ré, Data Programming: Creating Large Training Sets, Quickly, NeurIPS 2016. arXiv:1605.07723
Cited in: part10
Gadre et al., DataComp: In search of the next generation of multimodal datasets, NeurIPS 2023. arXiv:2304.14108.
Cited in: part08
Wang, R. et al. "DCN V2: Improved Deep & Cross Network and Practical Lessons for Web-scale Learning to Rank Systems." WWW 2021 (arXiv:2008.13535).
Cited in: part17
Loshchilov, I., Hutter, F. (2019). Decoupled Weight Decay Regularization (AdamW). ICLR. arXiv:1711.05101
Cited in: part01, part03
Lee et al. Deduplicating Training Data Makes Language Models Better. ACL 2022. arXiv:2107.06499
Cited in: part06
Ash, Zhang, Krishnamurthy, Langford, Agarwal, Deep Batch Active Learning by Diverse, Uncertain Gradient Lower Bounds (BADGE), ICLR 2020. arXiv:1906.03671
Cited in: part10
Wang, R. et al. "Deep & Cross Network for Ad Click Predictions." ADKDD 2017 (arXiv:1708.05123).
Cited in: part17
Nakkiran, P. et al. (2020). Deep Double Descent. ICLR. arXiv:1912.02292
Cited in: part03
Fort, Hu & Lakshminarayanan, "Deep Ensembles: A Loss Landscape Perspective", 2019, arXiv:1912.02757.
Cited in: part13
Zhou, G. et al. "Deep Interest Network for Click-Through Rate Prediction." KDD 2018 (arXiv:1706.06978); "Deep Interest Evolution Network." AAAI 2019 (arXiv:1809.03672).
Cited in: part17
Naumov, M. et al. "Deep Learning Recommendation Model for Personalization and Recommendation Systems." 2019 (arXiv:1906.00091).
Cited in: part17
Abadi, M. et al. Deep Learning with Differential Privacy. CCS 2016. arXiv:1607.00133.
Cited in: part15
Huang, G., Sun, Y., Liu, Z., Sedra, D., Weinberger, K. (2016). Deep Networks with Stochastic Depth. arXiv:1603.09382
Cited in: part03
He, K., Zhang, X., Ren, S., Sun, J. (2015). Deep Residual Learning for Image Recognition. arXiv:1512.03385. the residual gradient identity of §6.
Cited in: part03, part04
M. Zaheer et al., "Deep Sets", NeurIPS 2017, arXiv:1703.06114.
Cited in: part04
Salinas, D., Flunkert, V., Gasthaus, J. "DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networks." 2017 (arXiv:1704.04110); published in the International Journal of Forecasting, 2020.
Cited in: part17
Guo, H. et al. "DeepFM: A Factorization-Machine based Neural Network for CTR Prediction." IJCAI 2017 (arXiv:1703.04247).
Cited in: part17
DeepSeek-AI (2025). DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning. arXiv 2501.12948.
Cited in: part07, part07
DeepSeek-AI. DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model. 2024. arXiv:2405.04434
Cited in: part06, part06
DeepSeek-AI, "DeepSeek-V3 Technical Report", 2024 (arXiv:2412.19437).
Cited in: part01, part06, part06, part06, part06, part06 and 1 more
Shao et al. (2024). DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models (introduces GRPO). arXiv 2402.03300.
Cited in: part07
Dai et al. DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models. 2024. arXiv:2401.06066
Cited in: part06
Zhu et al., Deformable DETR: Deformable Transformers for End-to-End Object Detection, ICLR 2021. arXiv:2010.04159.
Cited in: part08
Xu et al., Demystifying CLIP Data (MetaCLIP), ICLR 2024. arXiv:2309.16671.
Cited in: part08
Song, Meng, Ermon, Denoising Diffusion Implicit Models, ICLR 2021. arXiv:2010.02502
Cited in: part09
Ho, Jain, Abbeel, Denoising Diffusion Probabilistic Models, NeurIPS 2020. arXiv:2006.11239
Cited in: part09
Karpukhin, V. et al. "Dense Passage Retrieval for Open-Domain Question Answering." EMNLP 2020 (arXiv:2004.04906).
Cited in: part17
G. Huang et al., "Densely Connected Convolutional Networks", CVPR 2017, arXiv:1608.06993 (DenseNet).
Cited in: part04
Lihe Yang et al. "Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data." CVPR 2024. arXiv:2401.10891
Cited in: part11
I. Radosavovic et al., "Designing Network Design Spaces", CVPR 2020, arXiv:2003.13678 (RegNet).
Cited in: part04
Lipton, Wang & Smola, "Detecting and Correcting for Label Shift with Black Box Predictors" (BBSE), ICML 2018, arXiv:1802.03916.
Cited in: part13
Guo, W. et al. "DeText: A Deep Text Ranking Framework with BERT." CIKM 2020 (arXiv:2008.02460).
Cited in: part17
Wang et al., DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D Queries, CoRL 2021. arXiv:2110.06922.
Cited in: part08
Zhao et al., DETRs Beat YOLOs on Real-time Object Detection (RT-DETR), CVPR 2024. arXiv:2304.08069.
Cited in: part08
Dhariwal, Nichol, Diffusion Models Beat GANs on Image Synthesis, NeurIPS 2021. arXiv:2105.05233
Cited in: part09
C. Godard et al., "Digging Into Self-Supervised Monocular Depth Estimation", ICCV 2019, arXiv:1806.01260 (Monodepth2).
Cited in: part04
Zhang et al., DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection, ICLR 2023. arXiv:2203.03605.
Cited in: part08
Maxime Oquab et al. "DINOv2: Learning Robust Visual Features without Supervision." TMLR 2024. arXiv:2304.07193
Cited in: part08, part10, part11
Guo et al. (2024). Direct Language Model Alignment from Online AI Feedback (online DPO). arXiv 2402.04792.
Cited in: part07
Rafailov et al. (2023). Direct Preference Optimization: Your Language Model is Secretly a Reward Model. arXiv 2305.18290; NeurIPS 2023.
Cited in: part07
Z. Zheng et al., "Distance-IoU Loss", AAAI 2020, arXiv:1911.08287 (DIoU and CIoU).
Cited in: part04
G. Hinton, O. Vinyals & J. Dean, "Distilling the Knowledge in a Neural Network", 2015 (arXiv:1503.02531).
Cited in: part01
Li et al., DN-DETR: Accelerate DETR Training by Introducing Query DeNoising, CVPR 2022. arXiv:2203.01305.
Cited in: part08
Soldaini et al. Dolma: an Open Corpus of Three Trillion Tokens. 2024. arXiv:2402.00159
Cited in: part06, part06
Gururangan et al. Don't Stop Pretraining: Adapt Language Models to Domains and Tasks. ACL 2020. arXiv:2004.10964
Cited in: part06
Liu et al. DoRA: Weight-Decomposed Low-Rank Adaptation. ICML 2024. arXiv:2402.09353
Cited in: part06
Xie et al. DoReMi: Optimizing Data Mixtures Speeds Up Language Model Pretraining. NeurIPS 2023. arXiv:2305.10429
Cited in: part06
Gal & Ghahramani, "Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning", ICML 2016, arXiv:1506.02142.
Cited in: part01, part03, part13
Malkov, Y., Yashunin, D. "Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphs." 2016 (arXiv:1603.09320).
Cited in: part02, part17
Kwon et al. Efficient Memory Management for Large Language Model Serving with PagedAttention. SOSP 2023. arXiv:2309.06180
Cited in: part06
Xiao et al. Efficient Streaming Language Models with Attention Sinks. ICLR 2024. arXiv:2309.17453
Cited in: part06
M. Tan, R. Pang, Q. Le, "EfficientDet: Scalable and Efficient Object Detection", CVPR 2020, arXiv:1911.09070.
Cited in: part04
Gu, Goel, Ré. Efficiently Modeling Long Sequences with Structured State Spaces. ICLR 2022. arXiv:2111.00396 (S4)
Cited in: part06
Pope et al. Efficiently Scaling Transformer Inference. MLSys 2023. arXiv:2211.05102 (decode memory-boundedness analysis)
Cited in: part06, part14, part14
M. Tan and Q. Le, "EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks", ICML 2019, arXiv:1905.11946.
Cited in: part04
Huang, J.-T. et al. "Embedding-based Retrieval in Facebook Search." KDD 2020 (arXiv:2006.11632).
Cited in: part17, part17
Caron et al., Emerging Properties in Self-Supervised Vision Transformers (DINO), ICCV 2021. arXiv:2104.14294
Cited in: part10
Jyh-Jing Hwang et al. "EMMA: End-to-End Multimodal Model for Autonomous Driving." 2024. arXiv:2410.23262
Cited in: part11
A. Kendall et al., "End-to-End Learning of Geometry and Context for Deep Stereo Regression", ICCV 2017, arXiv:1703.04309 (GC-Net, soft-argmin).
Cited in: part04
N. Carion et al., "End-to-End Object Detection with Transformers", ECCV 2020, arXiv:2005.12872 (DETR).
Cited in: part04, part08
Liu et al., "Energy-based Out-of-distribution Detection", NeurIPS 2020, arXiv:2010.03759.
Cited in: part13
Ma, X. et al. "Entire Space Multi-Task Model: An Effective Approach for Estimating Post-Click Conversion Rate." SIGIR 2018 (arXiv:1804.07931).
Cited in: part17
R. Ge, F. Huang, C. Jin & Y. Yuan, "Escaping From Saddle Points, Online Stochastic Gradient for Tensor Decomposition", COLT 2015 (arXiv:1503.02101).
Cited in: part01
Bengio, Léonard, Courville. Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation. 2013. arXiv:1308.3432 (STE)
Cited in: part06
Wager, S., Athey, S. "Estimation and Inference of Heterogeneous Treatment Effects using Random Forests." JASA 2018 (arXiv:1510.04342).
Cited in: part17
Chen et al., "Evaluating Large Language Models Trained on Code", 2021, arXiv:2107.03374 (HumanEval, pass@k).
Cited in: part13
Li et al., Evaluating Object Hallucination in Large Vision-Language Models (POPE), EMNLP 2023. arXiv:2305.10355.
Cited in: part08
Saxe, A. M., McClelland, J. L., Ganguli, S. (2014). Exact solutions to the nonlinear dynamics of learning in deep linear neural networks. ICLR. arXiv:1312.6120
Cited in: part03
Goodfellow, Shlens & Szegedy, "Explaining and Harnessing Adversarial Examples", ICLR 2015, arXiv:1412.6572.
Cited in: part13
Li et al., Exploring Plain Vision Transformer Backbones for Object Detection (ViTDet), ECCV 2022. arXiv:2203.16527.
Cited in: part08
Chen, He, Exploring Simple Siamese Representation Learning (SimSiam), CVPR 2021. arXiv:2011.10566
Cited in: part10
Raffel, C. et al. (2020). Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer (T5). arXiv:1910.10683
Cited in: part03
Chen et al. Extending Context Window of Large Language Models via Positional Interpolation. 2023. arXiv:2306.15595
Cited in: part06
Rabanser, Günnemann & Lipton, "Failing Loudly: An Empirical Study of Methods for Detecting Dataset Shift", NeurIPS 2019, arXiv:1810.11953.
Cited in: part13
Geyik, S. C., Ambler, S., Kenthapadi, K. "Fairness-Aware Ranking in Search & Recommendation Systems with Application to LinkedIn Talent Search." KDD 2019 (arXiv:1905.01989).
Cited in: part17
A. Lavin and S. Gray, "Fast Algorithms for Convolutional Neural Networks", CVPR 2016, arXiv:1509.09308.
Cited in: part04
Shazeer. Fast Transformer Decoding: One Write-Head is All You Need. 2019. arXiv:1911.02150 (MQA)
Cited in: part06
S. Ren, K. He, R. Girshick, J. Sun, "Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks", NeurIPS 2015, arXiv:1506.01497.
Cited in: part04
Z. Tian et al., "FCOS: Fully Convolutional One-Stage Object Detection", ICCV 2019, arXiv:1904.01355.
Cited in: part04
G. Yang & E. Hu, "Feature Learning in Infinite-Width Neural Networks", ICML 2021 (arXiv:2011.14522); G. Yang et al., "Tensor Programs V: Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfer", 2022 (arXiv:2203.03466).
Cited in: part01
T.-Y. Lin et al., "Feature Pyramid Networks for Object Detection", CVPR 2017, arXiv:1612.03144.
Cited in: part04, part04
Halko, N., Martinsson, P.-G., Tropp, J. "Finding Structure with Randomness." SIAM Review, 2011. arXiv:0909.4061 (for the SVD at scale; see dimensionality reduction)
Cited in: part01, part02, part02
Ziegler et al. (2019). Fine-Tuning Language Models from Human Preferences. arXiv 1909.08593 (the origin of the per-token KL reward and the adaptive controller).
Cited in: part07
Sohn et al., FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence, NeurIPS 2020. arXiv:2001.07685
Cited in: part10
Zhang, H., Dauphin, Y. N., Ma, T. (2019). Fixup Initialization: Residual Learning Without Normalization. ICLR. arXiv:1901.09321
Cited in: part03
Alayrac et al., Flamingo: a Visual Language Model for Few-Shot Learning, NeurIPS 2022. arXiv:2204.14198.
Cited in: part08
T. Dao, "FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning", 2023 (arXiv:2307.08691).
Cited in: part01, part06
Shah et al. FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precision. 2024. arXiv:2407.08608
Cited in: part06
T. Dao, D. Fu, S. Ermon, A. Rudra & C. Ré, "FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness", NeurIPS 2022 (arXiv:2205.14135); T. Dao, "FlashAttention-2", 2023 (arXiv:2307.08691).
Cited in: part01, part01, part06
Zichen Yu et al. "FlashOcc: Fast and Memory-Efficient Occupancy Prediction via Channel-to-Height Plugin." 2023. arXiv:2311.12058
Cited in: part11
Beyer et al., FlexiViT: One Model for All Patch Sizes, CVPR 2023. arXiv:2212.08013.
Cited in: part08
Bin Xiao et al. "Florence-2: Advancing a Unified Representation for a Variety of Vision Tasks." CVPR 2024. arXiv:2311.06242
Cited in: part11
Lipman, Chen, Ben-Hamu, Nickel, Le, Flow Matching for Generative Modeling, ICLR 2023. arXiv:2210.02747
Cited in: part09
Liu, Gong, Liu, Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow, ICLR 2023. arXiv:2209.03003
Cited in: part09
T.-Y. Lin et al., "Focal Loss for Dense Object Detection", ICCV 2017, arXiv:1708.02002 (RetinaNet).
Cited in: part04
Micikevicius, P. et al. FP8 Formats for Deep Learning. 2022. arXiv:2209.05433.
Cited in: part06, part14
Bain et al., Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrieval (WebVid), ICCV 2021. arXiv:2104.00650.
Cited in: part08
J. Long, E. Shelhamer, T. Darrell, "Fully Convolutional Networks for Semantic Segmentation", CVPR 2015, arXiv:1411.4038.
Cited in: part04
Anthony Hu et al. "GAIA-1: A Generative World Model for Autonomous Driving." 2023. arXiv:2309.17080
Cited in: part11
Heusel et al., "GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium", NeurIPS 2017, arXiv:1706.08500 (FID).
Cited in: part09, part13
Hendrycks, D., Gimpel, K. (2016). Gaussian Error Linear Units (GELUs). arXiv:1606.08415
Cited in: part03
Gemini Team, Google, Gemini: A Family of Highly Capable Multimodal Models, 2023. arXiv:2312.11805.
Cited in: part08
Gemma Team. Gemma 2: Improving Open Language Models at a Practical Size. 2024. arXiv:2408.00118 (interleaved local/global attention)
Cited in: part01, part06
Rezatofighi et al., Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regression, CVPR 2019. arXiv:1902.09630.
Cited in: part04, part08
Athey, S., Tibshirani, J., Wager, S. "Generalized Random Forests." Annals of Statistics 2019 (arXiv:1610.01271).
Cited in: part17
Razavi, van den Oord, Vinyals, Generating Diverse High-Fidelity Images with VQ-VAE-2, NeurIPS 2019. arXiv:1906.00446
Cited in: part09
I. Goodfellow et al., "Generative Adversarial Nets", NeurIPS 2014 (arXiv:1406.2661).
Cited in: part01, part09
Song, Ermon, Generative Modeling by Estimating Gradients of the Data Distribution (NCSN), NeurIPS 2019. arXiv:1907.05600
Cited in: part09
Sun et al., Generative Pretraining in Multimodality (Emu), ICLR 2024. arXiv:2307.05222.
Cited in: part08
Jake Bruce et al. "Genie: Generative Interactive Environments." ICML 2024. arXiv:2402.15391
Cited in: part11
C. Szegedy et al., "Going Deeper with Convolutions", CVPR 2015, arXiv:1409.4842, and "Rethinking the Inception Architecture", CVPR 2016, arXiv:1512.00567.
Cited in: part04
Black, S. et al. (2022). GPT-NeoX-20B. arXiv:2204.06745 · Chowdhery, A. et al. (2022). PaLM. arXiv:2204.02311
Cited in: part03
Frantar et al. GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers. ICLR 2023. arXiv:2210.17323
Cited in: part06
J. Ainslie et al., "GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints", EMNLP 2023 (arXiv:2305.13245).
Cited in: part01, part06
J. Cohen et al., "Gradient Descent on Neural Networks Typically Occurs at the Edge of Stability", ICLR 2021 (arXiv:2103.00065).
Cited in: part01
Ying, R. et al. "Graph Convolutional Neural Networks for Web-Scale Recommender Systems." KDD 2018 (arXiv:1806.01973).
Cited in: part17
Liunian Harold Li et al. "Grounded Language-Image Pre-training." CVPR 2022. arXiv:2112.03857
Cited in: part11
Shilong Liu et al. "Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection." ECCV 2024. arXiv:2303.05499
Cited in: part08, part11, part17
Wu, Y., He, K. (2018). Group Normalization. arXiv:1803.08494
Cited in: part03
Lepikhin et al. GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding. ICLR 2021. arXiv:2006.16668
Cited in: part06
Y. Xu et al., "GSPMD: General and Scalable Parallelization for ML Computation Graphs", 2021 (arXiv:2105.04663).
Cited in: part01
H. Li, Z. Xu, G. Taylor, C. Studer, T. Goldstein, NeurIPS 2018, arXiv:1712.09913, on visualising the loss surface of deep nets with and without skip connections.
Cited in: part04
He, K., Zhang, X., Ren, S., Sun, J. (2015), ICCV. PReLU and the He/Kaiming initialisation for rectifier networks. arXiv:1502.01852
Cited in: part03, part03
Ramesh et al., Hierarchical Text-Conditional Image Generation with CLIP Latents (unCLIP, DALL-E 2), 2022. arXiv:2204.06125.
Cited in: part08, part09
Schulman et al. (2015). High-Dimensional Continuous Control Using Generalized Advantage Estimation. arXiv 1506.02438.
Cited in: part07
Défossez et al., High Fidelity Neural Audio Compression (EnCodec), TMLR 2023. arXiv:2210.13438.
Cited in: part08
Rombach, Blattmann, Lorenz, Esser, Ommer, High-Resolution Image Synthesis with Latent Diffusion Models, CVPR 2022. arXiv:2112.10752
Cited in: part08, part09, part09
Liang et al., "Holistic Evaluation of Language Models" (HELM), TMLR 2023, arXiv:2211.09110.
Cited in: part13
Jonathon Luiten et al. "HOTA: A Higher Order Metric for Evaluating Multi-Object Tracking." IJCV 2021. arXiv:2009.07736
Cited in: part11
Santurkar, S., Tsipras, D., Ilyas, A., Madry, A. (2018). How Does Batch Normalization Help Optimization? arXiv:1805.11604
Cited in: part03
Miech et al., HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clips, ICCV 2019. arXiv:1906.03327.
Cited in: part08
Zhenxin Li et al. "Hydra-MDP: End-to-end Multimodal Planning with Multi-target Hydra-Distillation." 2024. arXiv:2406.06978
Cited in: part11
Zhou et al., iBOT: Image BERT Pre-Training with Online Tokenizer, ICLR 2022. arXiv:2111.07832
Cited in: part10
Y. Dauphin et al., "Identifying and attacking the saddle point problem in high-dimensional non-convex optimization", NeurIPS 2014 (arXiv:1406.2572).
Cited in: part01
K. He et al., "Identity Mappings in Deep Residual Networks", ECCV 2016, arXiv:1603.05027 (pre-activation).
Cited in: part04
Zhou et al. (2023). IFEval: Instruction-Following Evaluation for Large Language Models. arXiv 2311.07911.
Cited in: part07
Sautier, C. et al. "Image-to-Lidar Self-Supervised Distillation for Autonomous Driving Data" (SLidR). CVPR 2022 (arXiv:2203.16258).
Cited in: part17
Girdhar et al., ImageBind: One Embedding Space To Bind Them All, CVPR 2023. arXiv:2305.05665.
Cited in: part08, part08
Liu et al., Improved Baselines with Visual Instruction Tuning (LLaVA-1.5), CVPR 2024. arXiv:2310.03744.
Cited in: part08
Nichol, Dhariwal, Improved Denoising Diffusion Probabilistic Models, ICML 2021. arXiv:2102.09672
Cited in: part09
Kynkäänniemi et al., "Improved Precision and Recall Metric for Assessing Generative Models", NeurIPS 2019, arXiv:1904.06991.
Cited in: part13
Salimans et al., "Improved Techniques for Training GANs", NeurIPS 2016, arXiv:1606.03498 (Inception Score).
Cited in: part13
Gulrajani et al., Improved Training of Wasserstein GANs, NeurIPS 2017. arXiv:1704.00028
Cited in: part09
Haldar, M. et al. "Improving Deep Learning for Airbnb Search." KDD 2020 (arXiv:2002.05515).
Cited in: part17
Hamilton, W. L., Ying, R., Leskovec, J. "Inductive Representation Learning on Large Graphs" (GraphSAGE). NeurIPS 2017 (arXiv:1706.02216).
Cited in: part17
Wu et al. (2024). Inference Scaling Laws: An Empirical Analysis of Compute-Optimal Inference. arXiv 2408.00724.
Cited in: part07
Chen et al., InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks, CVPR 2024. arXiv:2312.14238.
Cited in: part08
Bertasius, Wang, Torresani, Is Space-Time Attention All You Need for Video Understanding? (TimeSformer), ICML 2021. arXiv:2102.05095.
Cited in: part08
Pang et al. (2024). Iterative Reasoning Preference Optimization (DPO + NLL). arXiv 2404.19733.
Cited in: part07
Lieber et al. Jamba: A Hybrid Transformer-Mamba Language Model. 2024. arXiv:2403.19887
Cited in: part06
Zheng et al., "Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena", NeurIPS 2023, arXiv:2306.05685.
Cited in: part07, part07, part13, part17
Liu et al. KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cache. ICML 2024. arXiv:2402.02750
Cited in: part06
Ethayarajh et al. (2024). KTO: Model Alignment as Prospect Theoretic Optimization. arXiv 2402.01306.
Cited in: part07
Schuhmann et al., LAION-5B: An open large-scale dataset for training next generation image-text models, NeurIPS 2022. arXiv:2210.08402.
Cited in: part08
G. Delétang et al., "Language Modeling Is Compression", ICLR 2024 (arXiv:2309.10668).
Cited in: part01
Brown et al. Language Models are Few-Shot Learners. NeurIPS 2020. arXiv:2005.14165 (13-gram decontamination)
Cited in: part01, part06
Ettinger, S. et al. "Large Scale Interactive Motion Forecasting for Autonomous Driving: The Waymo Open Motion Dataset." ICCV 2021 (arXiv:2104.10133).
Cited in: part17
Ba, J. L., Kiros, J. R., Hinton, G. E. (2016). Layer Normalization. arXiv:1607.06450
Cited in: part03
Xu, Y. et al. "LayoutLM: Pre-training of Text and Layout for Document Image Understanding." KDD 2020 (arXiv:1912.13318).
Cited in: part17
Huang, Y. et al. "LayoutLMv3: Pre-training for Document AI with Unified Text and Image Masking." ACM Multimedia 2022 (arXiv:2204.08387).
Cited in: part17
Zhai, A. et al. "Learning a Unified Embedding for Visual Search at Pinterest." KDD 2019 (arXiv:1908.01707).
Cited in: part02, part08, part17
Danijar Hafner et al. "Learning Latent Dynamics for Planning from Pixels." ICML 2019. arXiv:1811.04551
Cited in: part11
Abdool, M. et al. "Learning To Rank Diversely At Airbnb." CIKM 2023 (arXiv:2210.07774).
Cited in: part17
Stiennon et al. (2020). Learning to summarize from human feedback (the \(\log N - (N-1)/N\) result appears here). arXiv 2009.01325.
Cited in: part07, part07, part07
Radford et al., Learning Transferable Visual Models From Natural Language Supervision (CLIP), ICML 2021. arXiv:2103.00020.
Cited in: part01, part08, part17
Lightman et al. (2023). Let's Verify Step by Step (PRM800K, process supervision). arXiv 2305.20050.
Cited in: part07, part07
Jonah Philion and Sanja Fidler. "Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3D." ECCV 2020. arXiv:2008.05711
Cited in: part11, part17
Touvron, H. et al. (2023). LLaMA. arXiv:2302.13971 · Llama 2. arXiv:2307.09288 · Grattafiori, A. et al. (2024). The Llama 3 Herd of Models. arXiv:2407.21783
Cited in: part03, part03, part05
H. Touvron et al., "Llama 2: Open Foundation and Fine-Tuned Chat Models", 2023 (arXiv:2307.09288); "LLaMA: Open and Efficient Foundation Language Models", 2023 (arXiv:2302.13971).
Cited in: part01, part07, part07, part07, part07
Inan, H. et al. "Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations." 2023 (arXiv:2312.06674).
Cited in: part17, part17
Dettmers et al. LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale. NeurIPS 2022. arXiv:2208.07339
Cited in: part06
Beltagy et al. Longformer: The Long-Document Transformer. 2020. arXiv:2004.05150
Cited in: part06
Biderman et al. LoRA Learns Less and Forgets Less. TMLR 2024. arXiv:2405.09673
Cited in: part06
E. Hu et al., "LoRA: Low-Rank Adaptation of Large Language Models", ICLR 2022 (arXiv:2106.09685).
Cited in: part01, part06
Liu, N. F. et al. "Lost in the Middle: How Language Models Use Long Contexts." TACL 2024 (arXiv:2307.03172).
Cited in: part06, part17
R. Zhang, "Making Convolutional Networks Shift-Invariant Again", ICML 2019, arXiv:1904.11486.
Cited in: part04
Gu, Dao. Mamba: Linear-Time Sequence Modeling with Selective State Spaces. 2023. arXiv:2312.00752
Cited in: part06
Liao, B. et al. "MapTR: Structured Modeling and Learning for Online Vectorized HD Map Construction." ICLR 2023, spotlight (arXiv:2208.14437).
Cited in: part17
K. He, G. Gkioxari, P. Dollár, R. Girshick, "Mask R-CNN", ICCV 2017, arXiv:1703.06870, where ROIAlign is introduced.
Cited in: part04, part04, part04
B. Cheng et al., "Masked-attention Mask Transformer for Universal Image Segmentation", CVPR 2022, arXiv:2112.01527 (Mask2Former).
Cited in: part04, part08
He et al., Masked Autoencoders Are Scalable Vision Learners, CVPR 2022. arXiv:2111.06377.
Cited in: part08, part10
Feichtenhofer et al., Masked Autoencoders As Spatiotemporal Learners, NeurIPS 2022. arXiv:2205.09113.
Cited in: part08
Danijar Hafner et al. "Mastering Diverse Domains through World Models." 2023. arXiv:2301.04104
Cited in: part11
Wang et al. (2023). Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotations. arXiv 2312.08935.
Cited in: part07
Kusupati et al., "Matryoshka Representation Learning",
Cited in: part13
Tarvainen, Valpola, Mean teachers are better role models, NeurIPS 2017. arXiv:1703.01780
Cited in: part10
Nixon et al., "Measuring Calibration in Deep Learning", CVPR Workshops 2019, arXiv:1904.01685 (adaptive binning, ECE pitfalls).
Cited in: part13
Shoeybi, M. et al. Megatron-LM. 2019. arXiv:1909.08053.
Cited in: part14
Peng, C. et al. (2017). MegDet: A Large Mini-Batch Object Detector. arXiv:1711.07240
Cited in: part03
Künzel, S. R. et al. "Metalearners for estimating heterogeneous treatment effects using machine learning." PNAS 2019 (arXiv:1706.03461).
Cited in: part17
Lin et al., "Microsoft COCO: Common Objects in Context", ECCV 2014, arXiv:1405.0312.
Cited in: part13
Liang et al., Mind the Gap: Understanding the Modality Gap in Multi-modal Contrastive Representation Learning, NeurIPS 2022. arXiv:2203.02053.
Cited in: part08
Hu et al. MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategies. 2024. arXiv:2404.06395 (WSD schedule and scaling with it)
Cited in: part01, part06, part06
Y. Gu et al., "MiniLLM: Knowledge Distillation of Large Language Models", ICLR 2024 (arXiv:2306.08543); R. Agarwal et al., "On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes", ICLR 2024 (arXiv:2306.13649).
Cited in: part01
Jiang et al. Mistral 7B. 2023. arXiv:2310.06825
Cited in: part06
Leng et al., Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decoding, CVPR 2024. arXiv:2311.16922.
Cited in: part08
P. Micikevicius et al., "Mixed Precision Training", ICLR 2018 (arXiv:1710.03740).
Cited in: part01, part14
Berthelot et al., MixMatch: A Holistic Approach to Semi-Supervised Learning, NeurIPS 2019. arXiv:1905.02249
Cited in: part10
Jiang et al. Mixtral of Experts. 2024. arXiv:2401.04088; Mistral blog
Cited in: part06
Zhang, H., Cisse, M., Dauphin, Y. N., Lopez-Paz, D. (2018). mixup: Beyond Empirical Risk Minimization. ICLR. arXiv:1710.09412
Cited in: part03
A. Howard et al., "MobileNets", arXiv:1704.04861; M. Sandler et al., "MobileNetV2", arXiv:1801.04381; A. Howard et al., "Searching for MobileNetV3", arXiv:1905.02244.
Cited in: part04, part04, part20
Li et al., MoDAR: Using Motion Forecasting for 3D Object Detection in Point Cloud Sequences, CVPR 2023. arXiv:2306.03206
Cited in: part10
Mitchell, M. et al. Model Cards for Model Reporting. FAT* 2019. arXiv:1810.03993.
Cited in: part15, part15
He, Fan, Wu, Xie, Girshick, Momentum Contrast for Unsupervised Visual Representation Learning (MoCo), CVPR 2020. arXiv:1911.05722
Cited in: part10
Liu, Z. et al. "Monolith: Real Time Recommendation System With Collisionless Embedding Table." 2022 (arXiv:2209.07663).
Cited in: part17
Shaoshuai Shi et al. "Motion Transformer with Global Intention Localization and Local Movement Refinement." NeurIPS 2022. arXiv:2209.13508
Cited in: part11
Ari Seff et al. "MotionLM: Multi-Agent Motion Forecasting as Language Modeling." ICCV 2023. arXiv:2309.16534
Cited in: part11
Fangao Zeng et al. "MOTR: End-to-End Multiple-Object Tracking with Transformer." ECCV 2022. arXiv:2105.03247
Cited in: part11
Polyak et al., Movie Gen: A Cast of Media Foundation Models, Meta, 2024. arXiv:2410.13720
Cited in: part09, part09
F. Yu and V. Koltun, "Multi-Scale Context Aggregation by Dilated Convolutions", ICLR 2016, arXiv:1511.07122.
Cited in: part04
Daniel Dauner et al. "NAVSIM: Data-Driven Non-Reactive Autonomous Vehicle Simulation and Benchmarking." NeurIPS 2024. arXiv:2406.15349
Cited in: part11, part11
van den Oord, Vinyals, Kavukcuoglu, Neural Discrete Representation Learning (VQ-VAE), NeurIPS 2017. arXiv:1711.00937.
Cited in: part08, part09
Greshake, K. et al. "Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection." AISec 2023 (arXiv:2302.12173).
Cited in: part17
Blecher et al., Nougat: Neural Optical Understanding for Academic Documents, 2023. arXiv:2308.13418.
Cited in: part08
Caesar, H. et al. "nuScenes: A Multimodal Dataset for Autonomous Driving." CVPR 2020 (arXiv:1903.11027). nuScenes metrics: centre-distance matching, mAP, TP error metrics and NDS.
Cited in: part04, part11, part11, part11, part17, part17
Athalye et al., "Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples", ICML 2018, arXiv:1802.00420; and "Synthesizing Robust Adversarial Examples", ICML 2018, arXiv:1707.07397.
Cited in: part13
Xiaoyu Tian et al. "Occ3D: A Large-Scale 3D Occupancy Prediction Benchmark for Autonomous Driving." NeurIPS 2023 Datasets and Benchmarks. arXiv:2304.14365
Cited in: part11
Mescheder, L. et al. "Occupancy Networks: Learning 3D Reconstruction in Function Space." CVPR 2019 (arXiv:1812.03828).
Cited in: part17
Kim, G. et al. "OCR-free Document Understanding Transformer" (Donut). ECCV 2022 (arXiv:2111.15664).
Cited in: part08, part17
Qi et al., Offboard 3D Object Detection from Point Cloud Sequences (3D Auto Labeling), CVPR 2021. arXiv:2103.05073
Cited in: part10, part17
Agarwal et al., OmniSearchSage: Multi-Task Multi-Entity Embeddings for Pinterest Search, WWW 2024 companion. arXiv:2404.16260.
Cited in: part08
Guo et al., "On Calibration of Modern Neural Networks", ICML 2017, arXiv:1706.04599.
Cited in: part01, part02, part02, part13
Xiong, R. et al. (2020). On Layer Normalization in the Transformer Architecture. arXiv:2002.04745. why pre-LN needs less warmup; see chapter 5.
Cited in: part03, part03
R. Pascanu, T. Mikolov & Y. Bengio, "On the difficulty of training Recurrent Neural Networks", ICML 2013 (arXiv:1211.5063).
Cited in: part01
Seitzer et al., "On the Pitfalls of Heteroscedastic Uncertainty Estimation with Probabilistic Neural Networks" (\(\beta\)-NLL), ICLR 2022, arXiv:2203.09168.
Cited in: part13
L. Liu et al., "On the Variance of the Adaptive Learning Rate and Beyond", ICLR 2020 (arXiv:1908.03265) (RAdam).
Cited in: part01
B. Poole et al., "On Variational Bounds of Mutual Information", ICML 2019 (arXiv:1905.06922).
Cited in: part01
Fan, L. et al. "Once Detected, Never Lost: Surpassing Human Performance in Offline LiDAR Based 3D Object Detection" (CTRL). ICCV 2023, oral (arXiv:2304.12315).
Cited in: part17
Milakov, Gimelshein. Online normalizer calculation for softmax. 2018. arXiv:1805.02867
Cited in: part06
Open X-Embodiment Collaboration, Open X-Embodiment: Robotic Learning Datasets and RT-X Models, ICRA 2024. arXiv:2310.08864.
Cited in: part08
Kim et al., OpenVLA: An Open-Source Vision-Language-Action Model, CoRL 2024. arXiv:2406.09246.
Cited in: part08
Zhang, S. et al. OPT: Open Pre-trained Transformer Language Models. 2022. arXiv:2205.01068,
Cited in: part14, part14
Frantar, Alistarh. Optimal Brain Compression: A Framework for Accurate Post-Training Quantization and Pruning. NeurIPS 2022. arXiv:2208.11580 (OBQ)
Cited in: part06
Hong, Lee and Thorne (2024). ORPO: Monolithic Preference Optimization without Reference Model. arXiv 2403.07691.
Cited in: part07
Lucas Beyer et al. "PaliGemma: A versatile 3B VLM for transfer." 2024. arXiv:2407.07726
Cited in: part08, part08, part08, part11
A. Chowdhery et al., "PaLM: Scaling Language Modeling with Pathways", 2022 (arXiv:2204.02311).
Cited in: part01, part03, part14, part14, part14
Kirillov et al., "Panoptic Segmentation", CVPR 2019, arXiv:1801.00868 (PQ = SQ × RQ).
Cited in: part04, part13
Houlsby et al. Parameter-Efficient Transfer Learning for NLP. ICML 2019. arXiv:1902.00751 (bottleneck adapters)
Cited in: part06
Nogueira, R., Cho, K. "Passage Re-ranking with BERT." 2019 (arXiv:1901.04085).
Cited in: part17, part17
Dehghani et al., Patch n' Pack: NaViT, a Vision Transformer for any Aspect Ratio and Resolution, NeurIPS 2023. arXiv:2307.06304.
Cited in: part08
B. Cheng et al., "Per-Pixel Classification is Not All You Need for Semantic Segmentation", NeurIPS 2021, arXiv:2107.06278 (MaskFormer).
Cited in: part04
Jaegle et al., Perceiver IO: A General Architecture for Structured Inputs and Outputs, ICLR 2022. arXiv:2107.14795.
Cited in: part08
Yingfei Liu et al. "PETR: Position Embedding Transformation for Multi-View 3D Object Detection." ECCV 2022. arXiv:2203.05625
Cited in: part11
Saharia et al., Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding (Imagen), NeurIPS 2022. arXiv:2205.11487
Cited in: part09
Z. Allen-Zhu & Y. Li, "Physics of Language Models: Part 3.3, Knowledge Capacity Scaling Laws", 2024 (arXiv:2404.05405).
Cited in: part01
Pancha, N. et al. "PinnerFormer: Sequence Modeling for User Representation at Pinterest." KDD 2022 (arXiv:2205.04507).
Cited in: part17
Ting Chen et al. "Pix2seq: A Language Modeling Framework for Object Detection." ICLR 2022. arXiv:2109.10852
Cited in: part11
Yihan Hu et al. "Planning-oriented Autonomous Driving." CVPR 2023. arXiv:2212.10156
Cited in: part11
C. Qi et al., "PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space", NeurIPS 2017, arXiv:1706.02413.
Cited in: part04
C. Qi et al., "PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation", CVPR 2017, arXiv:1612.00593.
Cited in: part04
Sourabh Vora et al. "PointPainting: Sequential Fusion for 3D Object Detection." CVPR 2020. arXiv:1911.10150
Cited in: part11, part17
Lang, A. H. et al. "PointPillars: Fast Encoders for Object Detection from Point Clouds." CVPR 2019 (arXiv:1812.05784).
Cited in: part04, part17, part17
A. Kirillov et al., "PointRend: Image Segmentation as Rendering", CVPR 2020, arXiv:1912.08193.
Cited in: part04
Li, Liang. Prefix-Tuning: Optimizing Continuous Prompts for Generation. ACL 2021. arXiv:2101.00190
Cited in: part06
Salimans, Ho, Progressive Distillation for Fast Sampling of Diffusion Models, ICLR 2022. arXiv:2202.00512
Cited in: part09, part09
J. Schulman et al., "Proximal Policy Optimization Algorithms", 2017 (arXiv:1707.06347).
Cited in: part01, part07
Chen et al. Punica: Multi-Tenant LoRA Serving. MLSys 2024. arXiv:2310.18547
Cited in: part06
J.-R. Chang and Y.-S. Chen, "Pyramid Stereo Matching Network", CVPR 2018, arXiv:1803.08669 (PSMNet).
Cited in: part04
Wang et al., Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions, ICCV 2021. arXiv:2102.12122.
Cited in: part08
A. Paszke et al., "PyTorch: An Imperative Style, High-Performance Deep Learning Library", NeurIPS 2019 (arXiv:1912.01703).
Cited in: part01, part01
Dettmers et al. QLoRA: Efficient Finetuning of Quantized LLMs. NeurIPS 2023. arXiv:2305.14314
Cited in: part06, part06
Bai et al., Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond, 2023. arXiv:2308.12966.
Cited in: part08
Wang et al., Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution, 2024. arXiv:2409.12191.
Cited in: part08, part08
Z. Teed and J. Deng, "RAFT: Recurrent All-Pairs Field Transforms for Optical Flow", ECCV 2020, arXiv:2003.12039.
Cited in: part04
Cubuk, E. D., Zoph, B., Shlens, J., Le, Q. V. (2019). RandAugment. arXiv:1909.13719
Cited in: part03
Liao, M. et al. "Real-time Scene Text Detection with Differentiable Binarization." AAAI 2020 (arXiv:1911.08947).
Cited in: part17
Belkin, M., Hsu, D., Ma, S., Mandal, S. (2019). Reconciling modern machine learning practice and the bias-variance trade-off. PNAS 116(32). arXiv:1812.11118
Cited in: part03
van den Oord, Li, Vinyals, Representation Learning with Contrastive Predictive Coding (InfoNCE and the MI bound), 2018. arXiv:1807.03748.
Cited in: part01, part08
Cherti et al., Reproducible scaling laws for contrastive language-image learning (OpenCLIP), CVPR 2023. arXiv:2212.07143.
Cited in: part08
A. Veit, M. Wilber, S. Belongie, "Residual Networks Behave Like Ensembles of Relatively Shallow Networks", NeurIPS 2016, arXiv:1605.06431.
Cited in: part04
R. Wightman, H. Touvron, H. Jégou, "ResNet strikes back: An improved training procedure in timm", 2021, arXiv:2110.00476.
Cited in: part04
L.-C. Chen et al., "Rethinking Atrous Convolution for Semantic Image Segmentation", 2017, arXiv:1706.05587 (DeepLabv3), and "Encoder-Decoder with Atrous Separable Convolution", ECCV 2018, arXiv:1802.02611 (v3+).
Cited in: part04
Szegedy, C. et al. (2016). Rethinking the Inception Architecture for Computer Vision. arXiv:1512.00567
Cited in: part03
Lewis, P. et al. "Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks." NeurIPS 2020 (arXiv:2005.11401).
Cited in: part17
Lopez-Paz & Oquab, "Revisiting Classifier Two-Sample Tests", ICLR 2017, arXiv:1610.06545.
Cited in: part13
Coste et al. (2023). Reward Model Ensembles Help Mitigate Overoptimization. arXiv 2310.02743.
Cited in: part07
Lambert et al. (2024). RewardBench: Evaluating Reward Models for Language Modeling. arXiv 2403.13787.
Cited in: part07
Lee et al. (2023). RLAIF: Scaling Reinforcement Learning from Human Feedback with AI Feedback. arXiv 2309.00267.
Cited in: part07
Eykholt et al., "Robust Physical-World Attacks on Deep Learning Visual Classification", CVPR 2018, arXiv:1707.08945.
Cited in: part13
Radford et al., Robust Speech Recognition via Large-Scale Weak Supervision (Whisper), ICML 2023. arXiv:2212.04356.
Cited in: part08
Tsipras et al., "Robustness May Be at Odds with Accuracy", ICLR 2019, arXiv:1805.12152.
Cited in: part13
Zhang, B., Sennrich, R. (2019). Root Mean Square Layer Normalization. arXiv:1910.07467
Cited in: part03
P. Sun et al., "RSN: Range Sparse Net for Efficient, Accurate LiDAR 3D Object Detection", CVPR 2021, arXiv:2106.13365 (Waymo).
Cited in: part04, part04
Brohan et al., RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control, CoRL 2023. arXiv:2307.15818.
Cited in: part08
Hsieh et al. RULER: What's the Real Context Size of Your Long-Context Language Models?. 2024. arXiv:2404.06654
Cited in: part06
Sheng, Y. et al. S-LoRA: Serving Thousands of Concurrent LoRA Adapters. 2023. arXiv:2311.03285.
Cited in: part06, part14
Nikhila Ravi et al. "SAM 2: Segment Anything in Images and Videos." 2024. arXiv:2408.00714
Cited in: part11
Adebayo, J. et al. Sanity Checks for Saliency Maps. NeurIPS 2018. arXiv:1810.03292.
Cited in: part15
Sun, P. et al. "Scalability in Perception for Autonomous Driving: Waymo Open Dataset." CVPR 2020 (arXiv:1912.04838); dataset and metric definitions at waymo.com/open. Waymo Open Dataset metrics: 3D IoU matching, AP and APH, difficulty levels L1 and L2.
Cited in: part04, part04, part17, part17
Peebles and Xie, Scalable Diffusion Models with Transformers (DiT), ICCV 2023. arXiv:2212.09748.
Cited in: part08, part09
Muennighoff et al. Scaling Data-Constrained Language Models. NeurIPS 2023. arXiv:2305.16264
Cited in: part06, part06
Rae et al. Scaling Language Models: Methods, Analysis & Insights from Training Gopher. 2021. arXiv:2112.11446 (the "Gopher rules")
Cited in: part06
Kaplan et al. Scaling Laws for Neural Language Models. 2020. arXiv:2001.08361
Cited in: part06
Gao, L., Schulman, J. and Hilton, J. (2022). Scaling Laws for Reward Model Overoptimization. arXiv 2210.10760; ICML 2023.
Cited in: part07, part07
Snell et al. (2024). Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters. arXiv 2408.03314.
Cited in: part07
Esser et al., Scaling Rectified Flow Transformers for High-Resolution Image Synthesis (SD3), ICML 2024. arXiv:2403.03206
Cited in: part09, part09
Jia et al., Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision (ALIGN), ICML 2021. arXiv:2102.05918.
Cited in: part08
Dehghani et al., Scaling Vision Transformers to 22 Billion Parameters, ICML 2023. arXiv:2302.05442.
Cited in: part08
Song et al., Score-Based Generative Modeling through Stochastic Differential Equations (the probability-flow ODE), ICLR 2021. arXiv:2011.13456
Cited in: part09, part09
Pi, Q. et al. "Search-based User Interest Modeling with Lifelong Sequential Behavior Data for Click-Through Rate Prediction." CIKM 2020 (arXiv:2006.05639).
Cited in: part17
Ramachandran, P., Zoph, B., Le, Q. V. (2017). Searching for Activation Functions. arXiv:1710.05941
Cited in: part03
E. Xie et al., "SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers", NeurIPS 2021, arXiv:2105.15203.
Cited in: part04
Liu, Y. et al. "Segment Any Point Cloud Sequences by Distilling Vision Foundation Models" (Seal). NeurIPS 2023 (arXiv:2306.09347).
Cited in: part17
Alexander Kirillov et al. "Segment Anything." ICCV 2023. arXiv:2304.02643
Cited in: part04, part08, part08, part10, part11
Wang et al. (2022). Self-Consistency Improves Chain of Thought Reasoning in Language Models. arXiv 2203.11171.
Cited in: part07
Wang et al. (2022). Self-Instruct: Aligning Language Models with Self-Generated Instructions. arXiv 2212.10560.
Cited in: part07
Madaan et al. (2023). Self-Refine: Iterative Refinement with Self-Feedback. arXiv 2303.17651.
Cited in: part07
Assran et al., Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture (I-JEPA), CVPR 2023. arXiv:2301.08243
Cited in: part10
Goyal et al., Self-supervised Pretraining of Visual Features in the Wild (SEER), 2021. arXiv:2103.01988
Cited in: part10
Xie, Luong, Hovy, Le, Self-training with Noisy Student improves ImageNet classification, CVPR 2020. arXiv:1911.04252
Cited in: part10, part10
Nigam, P. et al. "Semantic Product Search." KDD 2019 (arXiv:1907.00937).
Cited in: part17
Abbas, A. et al. "SemDeDup: Data-efficient learning at web-scale through semantic deduplication." 2023. arXiv:2303.09540
Cited in: part02
I. Loshchilov & F. Hutter, "SGDR: Stochastic Gradient Descent with Warm Restarts", ICLR 2017 (arXiv:1608.03983).
Cited in: part01
Zheng et al. SGLang: Efficient Execution of Structured Language Model Programs. 2023. arXiv:2312.07104 (RadixAttention prefix caching)
Cited in: part06
V. Gupta, T. Koren & Y. Singer, "Shampoo: Preconditioned Stochastic Tensor Optimization", ICML 2018 (arXiv:1802.09568).
Cited in: part01
Zhai et al., Sigmoid Loss for Language Image Pre-Training (SigLIP), ICCV 2023. arXiv:2303.15343.
Cited in: part08
Copet et al., Simple and Controllable Music Generation (MusicGen), NeurIPS 2023. arXiv:2306.05284.
Cited in: part08
Lakshminarayanan, Pritzel & Blundell, "Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles", NeurIPS 2017, arXiv:1612.01474.
Cited in: part13
Ibrahim et al. Simple and Scalable Strategies to Continually Pre-train Large Language Models. TMLR 2024. arXiv:2403.08763 (LR re-warming and replay for CPT)
Cited in: part06
A. Bewley et al., "Simple Online and Realtime Tracking", ICIP 2016 (arXiv:1602.00763); N. Wojke et al., ICIP 2017 (arXiv:1703.07402); X. Weng et al., IROS 2020 (arXiv:1907.03961).
Cited in: part01, part01, part11
Nicolai Wojke, Alex Bewley and Dietrich Paulus. "Simple Online and Realtime Tracking with a Deep Association Metric." ICIP 2017. arXiv:1703.07402
Cited in: part01, part11
Minderer et al., Simple Open-Vocabulary Object Detection with Vision Transformers (OWL-ViT), ECCV 2022. arXiv:2205.06230.
Cited in: part08, part11
Meng, Xia and Chen (2024). SimPO: Simple Preference Optimization with a Reference-Free Reward. arXiv 2405.14734.
Cited in: part07
Smilkov, D. et al. SmoothGrad: removing noise by adding noise. 2017. arXiv:1706.03825.
Cited in: part15
Xiao et al. SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models. ICML 2023. arXiv:2211.10438
Cited in: part06
Ratner, Bach, Ehrenberg, Fries, Wu, Ré, Snorkel: Rapid Training Data Creation with Weak Supervision, VLDB 2018. arXiv:1711.10160
Cited in: part10
N. Bodla et al., "Soft-NMS: Improving Object Detection With One Line of Code", ICCV 2017, arXiv:1704.04503.
Cited in: part04
Zeghidour et al., SoundStream: An End-to-End Neural Audio Codec, IEEE/ACM TASLP 2021. arXiv:2107.03312.
Cited in: part08
T. Miyato et al., "Spectral Normalization for Generative Adversarial Networks", ICLR 2018 (arXiv:1802.05957).
Cited in: part01, part09
J. Hu, L. Shen, G. Sun, "Squeeze-and-Excitation Networks", CVPR 2018, arXiv:1709.01507.
Cited in: part04
W. Liu et al., "SSD: Single Shot MultiBox Detector", ECCV 2016, arXiv:1512.02325.
Cited in: part04
B. Graham and L. van der Maaten, "Submanifold Sparse Convolutional Networks", 2017, arXiv:1706.01307.
Cited in: part04
Jimenez et al., "SWE-bench: Can Language Models Resolve Real-World GitHub Issues?", ICLR 2024, arXiv:2310.06770; OpenAI, "Introducing SWE-bench Verified", 2024.
Cited in: part13
P. Sun et al., "SWFormer: Sparse Window Transformer for 3D Object Detection in Point Clouds", ECCV 2022, arXiv:2210.07372.
Cited in: part04
Liu et al., Swin Transformer: Hierarchical Vision Transformer using Shifted Windows, ICCV 2021. arXiv:2103.14030.
Cited in: part08
Fedus, Zoph, Shazeer. Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity. JMLR 2022. arXiv:2101.03961
Cited in: part06
Lambert et al. (2024). Tülu 3: Pushing Frontiers in Open Language Model Post-Training. arXiv 2411.15124.
Cited in: part07, part07
Agrawal et al. Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve. OSDI 2024. arXiv:2403.02310 (chunked prefill)
Cited in: part06
Esser, Rombach, Ommer, Taming Transformers for High-Resolution Image Synthesis (VQGAN), CVPR 2021. arXiv:2012.09841.
Cited in: part08, part09, part09
Yang, G. et al. (2022). Tensor Programs V: Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfer. arXiv:2203.03466
Cited in: part03
Douze et al., "The Faiss library", 2024, arXiv:2401.08281.
Cited in: part13
Dubey et al., The Llama 3 Herd of Models, 2024. arXiv:2407.21783. Meta AI blog, Llama 3.2: Revolutionizing edge AI and vision with open, customizable models, September 2024.
Cited in: part06, part06, part06, part06, part06, part06 and 9 more
M. Berman, A. Triki, M. Blaschko, "The Lovász-Softmax Loss", CVPR 2018, arXiv:1705.08790.
Cited in: part04
Huang et al. (2024). The N+ Implementation Details of RLHF with PPO: A Case Study on TL;DR Summarization. arXiv 2403.17031.
Cited in: part07
Gao et al. The Pile: An 800GB Dataset of Diverse Text for Language Modeling. 2020. arXiv:2101.00027 (bits-per-byte reporting)
Cited in: part01, part06
Lester, Al-Rfou, Constant. The Power of Scale for Parameter-Efficient Prompt Tuning. EMNLP 2021. arXiv:2104.08691
Cited in: part06
Zhang et al., "The Unreasonable Effectiveness of Deep Features as a Perceptual Metric" (LPIPS), CVPR 2018, arXiv:1801.03924.
Cited in: part13
Beirami et al. (2024). Theoretical guarantees on the best-of-n alignment policy. arXiv 2401.01879.
Cited in: part07
Puy, G. et al. "Three Pillars improving Vision Foundation Model Distillation for Lidar" (ScaLR). CVPR 2024 (arXiv:2310.17504).
Cited in: part17
Bolya et al., Token Merging: Your ViT But Faster, ICLR 2023. arXiv:2210.09461.
Cited in: part08
Chorowski, J., Jaitly, N. (2017). Towards better decoding and language model integration in sequence to sequence models. arXiv:1612.02695
Cited in: part03
Madry et al., "Towards Deep Learning Models Resistant to Adversarial Attacks", ICLR 2018, arXiv:1706.06083.
Cited in: part13
Hoffmann et al. Training Compute-Optimal Large Language Models. NeurIPS 2022. arXiv:2203.15556
Cited in: part06
Touvron et al., Training data-efficient image transformers and distillation through attention (DeiT), ICML 2021. arXiv:2012.12877.
Cited in: part03, part08
Chen, T., Xu, B., Zhang, C., Guestrin, C. (2016). Training Deep Nets with Sublinear Memory Cost. arXiv:1604.06174
Cited in: part03, part03, part14
Ouyang et al. (2022). Training language models to follow instructions with human feedback (InstructGPT). arXiv:2203.02155.
Cited in: part01, part07, part07, part07
Cobbe et al. (2021). Training Verifiers to Solve Math Word Problems (GSM8K, outcome verifiers). arXiv 2110.14168.
Cited in: part07, part07
Tsai, Y.-H. H. et al. "Transformer Dissection: A Unified Understanding of Transformer's Attention via the Lens of Kernel." EMNLP 2019. arXiv:1908.11775
Cited in: part02
Xuyang Bai et al. "TransFusion: Robust LiDAR-Camera Fusion for 3D Object Detection with Transformers." CVPR 2022. arXiv:2203.11496
Cited in: part11
Yao et al. (2023). Tree of Thoughts: Deliberate Problem Solving with Large Language Models. arXiv 2305.10601.
Cited in: part07
Li, M. et al. "TrOCR: Transformer-based Optical Character Recognition with Pre-trained Models." 2021 (arXiv:2109.10282).
Cited in: part17
Schulman et al. (2015). Trust Region Policy Optimization. arXiv 1502.05477.
Cited in: part07
Chang, J. et al. "TWIN: TWo-stage Interest Network for Lifelong User Behavior Modeling in CTR Prediction at Kuaishou." KDD 2023 (arXiv:2302.02352).
Cited in: part17
O. Ronneberger, P. Fischer, T. Brox, "U-Net: Convolutional Networks for Biomedical Image Segmentation", MICCAI 2015, arXiv:1505.04597.
Cited in: part04
McInnes, L., Healy, J., Melville, J. "UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction." 2018. arXiv:1802.03426
Cited in: part02
Joachims, T., Swaminathan, A., Schnabel, T. "Unbiased Learning-to-Rank with Biased Feedback." WSDM 2017 (arXiv:1608.04468).
Cited in: part17, part17, part17
Angelopoulos et al., "Uncertainty Sets for Image Classifiers using Conformal Prediction" (RAPS), ICLR 2021, arXiv:2009.14193.
Cited in: part13
Zhang, C. et al. (2017). Understanding deep learning requires rethinking generalization. arXiv:1611.03530
Cited in: part03
Liu et al. (2025). Understanding R1-Zero-Like Training: A Critical Perspective (Dr. GRPO). arXiv 2503.20783.
Cited in: part07
W. Luo et al., "Understanding the Effective Receptive Field in Deep Convolutional Neural Networks", NeurIPS 2016, arXiv:1701.04128.
Cited in: part04
Jha, R. et al. "Unified Embedding Based Personalized Retrieval in Etsy Search." 2023 (arXiv:2306.11424).
Cited in: part17
Lu et al., Unified-IO 2: Scaling Autoregressive Multimodal Models with Vision, Language, Audio, and Action, CVPR 2024. arXiv:2312.17172.
Cited in: part08
Razin et al. (2024). Unintentional Unalignment: Likelihood Displacement in Direct Preference Optimization. arXiv 2410.08847.
Cited in: part07
Caron et al., Unsupervised Learning of Visual Features by Contrasting Cluster Assignments (SwAV), NeurIPS 2020. arXiv:2006.09882
Cited in: part10
C. Godard, O. Mac Aodha, G. Brostow, "Unsupervised Monocular Depth Estimation with Left-Right Consistency", CVPR 2017, arXiv:1609.03677.
Cited in: part04
Gidaris, Singh, Komodakis, Unsupervised Representation Learning by Predicting Image Rotations, ICLR 2018. arXiv:1803.07728
Cited in: part10
Bo Jiang et al. "VAD: Vectorized Scene Representation for Efficient Autonomous Driving." ICCV 2023. arXiv:2303.12077
Cited in: part11
Jiyang Gao et al. "VectorNet: Encoding HD Maps and Agent Dynamics from Vectorized Representation." CVPR 2020. arXiv:2005.04259
Cited in: part11
Kumar, Liang & Ma, "Verified Uncertainty Calibration", NeurIPS 2019, arXiv:1909.10155.
Cited in: part13
K. Simonyan and A. Zisserman, "Very Deep Convolutional Networks for Large-Scale Image Recognition", ICLR 2015, arXiv:1409.1556 (VGG).
Cited in: part04
Zhang, Li, Bing, Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understanding, EMNLP 2023 demo. arXiv:2306.02858.
Cited in: part08
Xu et al., VideoCLIP: Contrastive Pre-training for Zero-shot Video-Text Understanding, EMNLP 2021. arXiv:2109.14084.
Cited in: part08
Tong et al., VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Training, NeurIPS 2022. arXiv:2203.12602.
Cited in: part08
Darcet et al., Vision Transformers Need Registers, ICLR 2024. arXiv:2309.16588.
Cited in: part08
Zhai, A. et al. "Visual Discovery at Pinterest." WWW 2017 (arXiv:1702.04680).
Cited in: part17
Liu et al., Visual Instruction Tuning (LLaVA), NeurIPS 2023. arXiv:2304.08485.
Cited in: part08
Yang, F. et al. "Visual Search at eBay." KDD 2017 (arXiv:1706.03154).
Cited in: part17
Jing, Y. et al. "Visual Search at Pinterest." KDD 2015 (arXiv:1505.07647).
Cited in: part02, part17
Arnab et al., ViViT: A Video Vision Transformer, ICCV 2021. arXiv:2103.15691.
Cited in: part08
Arjovsky, Chintala, Bottou, Wasserstein GAN, ICML 2017. arXiv:1701.07875
Cited in: part09
Nigamaa Nayakanti et al. "Wayformer: Motion Forecasting via Simple and Efficient Attention Networks." ICRA 2023. arXiv:2207.05844
Cited in: part11
Gulino, C. et al. "Waymax: An Accelerated, Data-Driven Simulator for Large-Scale Autonomous Driving Research." NeurIPS 2023 Datasets and Benchmarks (arXiv:2310.08710).
Cited in: part17
Zhou et al., "WebArena: A Realistic Web Environment for Building Autonomous Agents", ICLR 2024, arXiv:2307.13854.
Cited in: part13
Laurençon et al., What matters when building vision-language models? (Idefics2), NeurIPS 2024. arXiv:2405.02246.
Cited in: part08
Kendall & Gal, "What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?", NeurIPS 2017, arXiv:1703.04977.
Cited in: part13
Yuksekgonul et al., When and why vision-language models behave like bags-of-words, and what to do about it? (ARO), ICLR 2023. arXiv:2210.01936.
Cited in: part08
Müller, R., Kornblith, S., Hinton, G. "When Does Label Smoothing Help?" NeurIPS 2019. arXiv:1906.02629
Cited in: part02, part03
Grinsztajn, L., Oyallon, E., Varoquaux, G. "Why do tree-based models still outperform deep learning on tabular data?" NeurIPS 2022. arXiv:2207.08815
Cited in: part02
Hein, Andriushchenko & Bitterwolf, "Why ReLU networks yield high-confidence predictions far away from the training data and how to mitigate the problem", CVPR 2019, arXiv:1812.05720.
Cited in: part13
Cheng, H.-T. et al. "Wide & Deep Learning for Recommender Systems." DLRS 2016 (arXiv:1606.07792).
Cited in: part02, part17
Xu et al. (2023). WizardLM: Empowering Large Language Models to Follow Complex Instructions (Evol-Instruct). arXiv 2304.12244.
Cited in: part07
Chen, T., Guestrin, C. "XGBoost: A Scalable Tree Boosting System." KDD 2016. arXiv:1603.02754; tutorial: Introduction to Boosted Trees
Cited in: part02, part17
Peng et al. YaRN: Efficient Context Window Extension of Large Language Models. ICLR 2024. arXiv:2309.00071
Cited in: part06
Tianheng Cheng et al. "YOLO-World: Real-Time Open-Vocabulary Object Detection." CVPR 2024. arXiv:2401.17270
Cited in: part11
Z. Ge et al., "YOLOX: Exceeding YOLO Series in 2021", 2021, arXiv:2107.08430 (SimOTA).
Cited in: part04
J. Redmon et al., "You Only Look Once", CVPR 2016, arXiv:1506.02640; A. Bochkovskiy et al., "YOLOv4", 2020, arXiv:2004.10934.
Cited in: part04
Tunstall et al. (2023). Zephyr: Direct Distillation of LM Alignment. arXiv 2310.16944.
Cited in: part07
Qi, P. et al. Zero Bubble Pipeline Parallelism. ICLR 2024. arXiv:2401.10241.
Cited in: part14

Company engineering and research blogs¶

72 sources.

Elhage, N. et al. A Mathematical Framework for Transformer Circuits. Transformer Circuits
Cited in: part15
McMahan, H. B. et al. "Ad Click Prediction: a View from the Trenches." KDD 2013 (research.google).
Cited in: part02, part02, part17
Amazon Web Services. "Amazon Fraud Detector" developer documentation (docs.aws.amazon.com).
Cited in: part17
Amazon Web Services. "Amazon Textract" developer documentation (forms, tables, identity documents, invoices and receipts, confidence scores, Augmented A2I human review) (docs.aws.amazon.com, analysing documents).
Cited in: part17
Anthropic: Careers (interview process and AI-usage guidance).
Cited in: prefac
Apple. Live Text user-guide documentation (support.apple.com) and the Vision framework's VNRecognizeTextRequest (developer.apple.com).
Cited in: part17
Apple, ARKit developer documentation, on world tracking and its quality states. developer.apple.com/augmented-reality/arkit · ARWorldTrackingConfiguration
Cited in: part04
Netflix Technology Blog. "Artwork Personalization at Netflix." December 2017 (netflixtechblog.com).
Cited in: part01, part17
Apple, AVFoundation and ARKit developer documentation, on depth data and scene depth. AVDepthData · ARDepthData · WWDC 2022, "Discover advancements in iOS camera capture: Depth, focus, and multitasking"
Cited in: part04
Microsoft. "Azure AI Document Intelligence" documentation, including the page on interpreting model accuracy and confidence scores (learn.microsoft.com).
Cited in: part17
Covington, P., Adams, J., Sargin, E. "Deep Neural Networks for YouTube Recommendations." RecSys 2016, on offline metrics used for triage with live A/B as the final arbiter (research.google).
Cited in: part03, part03, part17, part17
Uber Engineering. "DeepETA: How Uber Predicts Arrival Times Using Deep Learning." February 2022 (uber.com).
Cited in: part02, part17
NVIDIA, DeepStream SDK documentation (docs.nvidia.com) and TensorRT Developer Guide (docs.nvidia.com), on segmentation deployment.
Cited in: part04, part04
Apple Machine Learning Research, "Deploying Transformers on the Apple Neural Engine", 2022, and Apple's Core ML Tools guide, for the on-device constraints in the production note above. machinelearning.apple.com · apple.github.io/coremltools
Cited in: part04
Meta AI, Detectron2, open-source detection platform, 2019. ai.meta.com · github.com/facebookresearch/detectron2
Cited in: part04
Chamandy, N. "Experimentation in a Ridesharing Marketplace." Lyft Engineering, 2016 (eng.lyft.com).
Cited in: part17
Anyscale. Fine-Tuning Llama-2: A Comprehensive Case Study for Tailoring Models to Unique Applications. 2023. anyscale.com
Cited in: part06
Anyscale. Fine-Tuning LLMs: LoRA or Full-Parameter? An in-depth Analysis with Llama 2. 2023. anyscale.com
Cited in: part06
Burges, C. J. C. "From RankNet to LambdaRank to LambdaMART: An Overview." Microsoft Research Technical Report MSR-TR-2010-82, 2010 (microsoft.com).
Cited in: part17
Google. "Go beyond the search box: Introducing multisearch" (blog.google) and the Search blog post on visual search with Lens and multisearch (blog.google).
Cited in: part17
OpenAI, GPT-4V(ision) System Card, September 2023.
Cited in: part08
Meta AI. "Harmful content can evolve quickly. Our new AI system adapts to tackle it" (Few-Shot Learner), December 2021 (ai.meta.com).
Cited in: part17
Meta Engineering. "How machine learning powers Facebook's News Feed ranking algorithm." January 2021 (engineering.fb.com).
Cited in: part17
Stripe. Radar documentation (docs.stripe.com, risk evaluation) and the Stripe engineering post "How we built it: Stripe Radar" (stripe.com).
Cited in: part17
DoorDash Engineering. "Improving ETA Prediction Accuracy for Long-tail Events." April 2021 (doordash.engineering); "Improving ETAs with multi-task models, deep learning, and probabilistic forecasts" (careersatdoordash.com); "Switchback Tests and Randomized Experimentation Under Network Effects at DoorDash" (careersatdoordash.com).
Cited in: part17
Betker et al., Improving Image Generation with Better Captions (DALL-E 3), OpenAI, 2023. PDF
Cited in: part09
Olsson, C. et al. In-context Learning and Induction Heads. Transformer Circuits Thread, 2022.
Cited in: part15
Netflix Technology Blog, "Innovating Faster on Personalization Algorithms at Netflix Using Interleaving", 2017.
Cited in: part13
Anthropic. "Introducing Contextual Retrieval." Anthropic engineering blog, September 2024 (anthropic.com).
Cited in: part17
Meta Engineering. "Introducing FBLearner Flow: Facebook's AI backbone." 2016 (engineering.fb.com).
Cited in: part17
Netflix Technology Blog, "It's All A/Bout Testing: The Netflix Experimentation Platform", April 2016 (techblog.netflix.com); "Decision Making at Netflix", 2021, the opening post of the experimentation series (netflixtechblog.com).
Cited in: part01
Radford, A. et al. (2019). Language Models are Unsupervised Multitask Learners. cdn.openai.com
Cited in: part03
Wayve. "LINGO-1: Exploring Natural Language for Autonomous Driving." wayve.ai
Cited in: part11
Anil, C. et al. Many-shot Jailbreaking. Anthropic, 2024.
Cited in: part15
Uber Engineering Blog, "Meet Michelangelo: Uber's Machine Learning Platform", 2017, and "Scaling Machine Learning at Uber with Michelangelo", 2018.
Cited in: part13
Uber Engineering. "Meet Michelangelo: Uber's Machine Learning Platform" (uber.com); "Palette Meta Store Journey" (uber.com) and the recorded talk "Michelangelo Palette: A Feature Engineering Platform at Uber" (infoq.com); "From Predictive to Generative: How Michelangelo Accelerates Uber's AI Journey" (uber.com).
Cited in: part17, part17
Microsoft. PhotoDNA documentation (microsoft.com); NCMEC and IWF hash-list programmes.
Cited in: part17
Ma, J. et al. "Modeling Task Relationships in Multi-task Learning with Multi-gate Mixture-of-Experts." KDD 2018 (research.google).
Cited in: part17
LinkedIn Engineering. "Musings on Building a Generative AI Product," 2024, on intent routing, retrieval over internal data, evaluation and streaming latency (linkedin.com).
Cited in: part17
NVIDIA. DRIVE platform developer documentation (developer.nvidia.com) and the DRIVE perception pages on obstacle, path and wait-condition DNNs (developer.nvidia.com).
Cited in: part17
NVIDIA. H100 datasheet,
Cited in: part14, part14, part14
NVIDIA. NVIDIA H100 Tensor Core GPU Architecture
Cited in: part14
Aurora. "Online to Offline: Turning real-world experience into virtual tests." aurora.tech
Cited in: part11
Netflix Technology Blog. "Open-Sourcing Metaflow, a Human-Centric Framework for Data Science." 2019 (netflixtechblog.com).
Cited in: part17
OpenAI: Interview guide.
Cited in: prefac
DoorDash Engineering. "Path to high-quality LLM-based Dasher support automation," on RAG, a response guardrail and LLM-based conversation-quality evaluation (careersatdoordash.com).
Cited in: part17
Bishop, C. Pattern Recognition and Machine Learning, ch. 6–7 (kernels, GPs, SVMs). Free PDF
Cited in: part02, part02, part02
He, X. et al. "Practical Lessons from Predicting Clicks on Ads at Facebook." ADKDD 2014 (ai.meta.com).
Cited in: part02, part02, part17
Zhao, Z. et al. "Recommending What Video to Watch Next: A Multitask Ranking System." RecSys 2019 (research.google).
Cited in: part17
Wayve. "Scaling GAIA-1: 9-billion parameter generative world model for autonomous driving." wayve.ai
Cited in: part11
Meta Engineering. "Scaling the Instagram Explore recommendations system." August 2023 (engineering.fb.com).
Cited in: part17
Google Research, "Semantic Image Segmentation with DeepLab in TensorFlow", 2018, the open-source release with a MobileNetV2 backbone for mobile and an Xception backbone for the server. research.google · Google Open Source Blog
Cited in: part04
Platt, J. "Sequential Minimal Optimization: A Fast Algorithm for Training Support Vector Machines." MSR-TR-98-14, 1998. microsoft.com
Cited in: part02
Waymo, "Sharing our safety framework for fully autonomous operations", 2020, and the Waymo safety pages; Sun et al., "Scalability in Perception for Autonomous Driving: Waymo Open Dataset", CVPR 2020, arXiv:1912.04838.
Cited in: part13
Waymo. Safety report and safety-framework publications (waymo.com/safety, "Sharing our safety framework for fully autonomous operations", October 2020, waymo.com/blog); the Waymo Open Dataset site (waymo.com/open).
Cited in: part17
OpenAI, "Spinning Up in Deep RL", for the PPO and GAE implementations in W7 and C7: spinningup.openai.com.
Cited in: part16
Spotify. Annoy. github.com/spotify/annoy
Cited in: part02
DoorDash Engineering. "Switchback Tests and Randomized Experimentation Under Network Effects at DoorDash" (careersatdoordash.com).
Cited in: part17
Nuro. "Technology." nuro.ai
Cited in: part11
NVIDIA, TensorRT Developer Guide, on the EfficientNMS and batched NMS plugins. docs.nvidia.com
Cited in: part04, part04
Baylor, D. et al. "TFX: A TensorFlow-Based Production-Scale Machine Learning Platform." KDD 2017 (research.google).
Cited in: part17
Breck, E. et al. "The ML Test Score: A Rubric for ML Production Readiness and Technical Debt Reduction." IEEE Big Data 2017 (research.google).
Cited in: part13, part17, part17
Meta AI. "The shift to generalized AI to better identify violating content," November 2021, which describes Whole Post Integrity Embeddings (WPIE) (ai.meta.com).
Cited in: part17
Elhage, N. et al. Toy Models of Superposition. Transformer Circuits Thread, 2022.
Cited in: part15
DeepMind and Google. "Traffic prediction with advanced Graph Neural Networks" (graph neural networks over supersegments for Google Maps ETAs), 2020 (deepmind.google).
Cited in: part17
Google (Nayak, P.). "Understanding searches better than ever before." The Keyword blog, October 2019 (blog.google).
Cited in: part17
Pinterest Engineering, Unifying visual embeddings for visual search at Pinterest, engineering blog, 2019.
Cited in: part08
Pinterest Engineering. "User state-based notification volume optimization," 2020 (medium.com/pinterest-engineering); "NEP: Notification System and Relevance" (medium.com/pinterest-engineering).
Cited in: part17
OpenAI. "Using GPT-4 for content moderation", August 2023 (openai.com); OpenAI moderation endpoint documentation.
Cited in: part17
OpenAI, Video generation models as world simulators (Sora technical report), February 2024.
Cited in: part08
Aurora. "Virtual Testing: The Invisible Accelerator." aurora.tech
Cited in: part11
NVIDIA, VPI (Vision Programming Interface) documentation, on hardware-accelerated optical flow and lens distortion correction. docs.nvidia.com/vpi · dense optical flow · pyramidal LK optical flow
Cited in: part04

Documentation and standards¶

15 sources.

Karpathy, A. (2019). A Recipe for Training Neural Networks. karpathy.github.io. the "overfit one batch first" discipline used in §6.
Cited in: part03, part16
PyTorch. Autograd mechanics. docs.pytorch.org/docs/stable/notes/autograd.html
Cited in: part03, part03
PyTorch documentation, "Broadcasting semantics", "Tensor Views" and torch.nn.functional.scaled_dot_product_attention.
Cited in: part01
Google Cloud. "Document AI" documentation (specialised processors, form parser, custom extractor training, human-in-the-loop review) (overview, processor list).
Cited in: part17
Feast (docs.feast.dev) and Tecton (docs.tecton.ai) documentation, the open-source and managed feature stores.
Cited in: part17
Liu et al., LLaVA-NeXT: Improved reasoning, OCR, and world knowledge, LLaVA blog, January 2024.
Cited in: part08
US Department of Transportation, Federal Highway Administration. "Manual on Uniform Traffic Control Devices" (MUTCD), crosswalk marking patterns (mutcd.fhwa.dot.gov).
Cited in: part17
K. Jordan et al., "Muon: An optimizer for hidden layers in neural networks", 2024. kellerjordan.github.io · github.com/KellerJordan/Muon
Cited in: part01
Stanford CS231n course notes, "Neural Networks Part 3: Learning and Evaluation" (gradient checks section). cs231n.github.io
Cited in: part01
PyTorch. Overview of the PyTorch autograd engine (blog). pytorch.org/blog
Cited in: part03
ISO 26262:2018, Road vehicles: Functional safety;
Cited in: part15
NVIDIA. TensorRT-LLM documentation and
Cited in: part14
JAX. The Autodiff Cookbook. docs.jax.dev
Cited in: part03
PyTorch. torch.nn.SyncBatchNorm. docs.pytorch.org
Cited in: part03
PyTorch. torch.utils.checkpoint. docs.pytorch.org/docs/main/checkpoint.html
Cited in: part03, part03

Code repositories¶

10 sources.

NVIDIA, CUDA-PointPillars sample (github.com/NVIDIA-AI-IOT/CUDA-PointPillars) and TensorRT Developer Guide (docs.nvidia.com), on deploying 3-D detectors.
Cited in: part04
FAISS wiki: Faiss indexes, Guidelines to choose an index
Cited in: part02
Sebastian Raschka, "LLMs-from-scratch", for the same material at a slower pace: github.com/rasbt/LLMs-from-scratch.
Cited in: part16
Andrej Karpathy, "micrograd", for the scalar autograd engine in E3: github.com/karpathy/micrograd.
Cited in: part03, part03, part16
Karpathy, A. nanoGPT. github.com/karpathy/nanoGPT
Cited in: part03
Andrej Karpathy, "nanoGPT", for the GPT structure in E1: github.com/karpathy/nanoGPT.
Cited in: part16
OpenAI. tiktoken. GitHub
Cited in: part05
Wightman, R. timm. layers/drop.py (DropPath reference implementation). github.com/huggingface/pytorch-image-models
Cited in: part03
Jonathon Luiten and Arne Hoffhues. "TrackEval." GitHub
Cited in: part11
Cole Gulino et al. "Waymax: An Accelerated, Data-Driven Simulator for Large-Scale Autonomous Driving Research." NeurIPS 2023. GitHub
Cited in: part11

Other primary sources¶

100 sources.

Daly, J. T. A higher order estimate of the optimum checkpoint interval for restart dumps.
Cited in: part14
Gretton et al., "A Kernel Two-Sample Test", JMLR 2012.
Cited in: part13
Neal, R., Hinton, G. "A View of the EM Algorithm that Justifies Incremental, Sparse, and Other Variants." Learning in Graphical Models, 1998. cs.toronto.edu
Cited in: part02
Meta Business Help Center. "About Ad Auctions" (the "total value" description of the auction) (facebook.com/business/help).
Cited in: part17
About Amazon, Your complete guide to the Amazon interview process.
Cited in: part20, part20
Tesla (2021). AI Day 2021. Full presentation
Cited in: part05
Amazon Jobs: Leadership Principles,
Cited in: part20
Amazon Jobs: The interview loop; Leadership Principles; Applied Scientist interview prep.
Cited in: part20, prefac
Cybenko, G. (1989). Approximation by superpositions of a sigmoidal function. Mathematics of Control, Signals and Systems 2, 303–314. Springer
Cited in: part03
Airbnb Engineering. "Architecting a Machine Learning System for Risk" (medium.com/airbnb-engineering); "Fighting Financial Fraud with Targeted Friction" (medium.com/airbnb-engineering); "Graph Machine Learning at Airbnb" (medium.com/airbnb-engineering).
Cited in: part17
NumPy documentation, "Broadcasting" and the ndarray reference on internal memory layout and strides.
Cited in: part01
Agarwal, D. et al. "Budget pacing for targeted online advertisements at LinkedIn." KDD 2014 (dl.acm.org).
Cited in: part17
S. Agarwal et al., "Bundle Adjustment in the Large", ECCV 2010, and the Ceres Solver documentation on bundle adjustment and the Schur-complement solvers. ceres-solver.org
Cited in: part04
Airbnb Engineering. "Chronon, Airbnb's ML Feature Platform, Is Now Open Source." 2024, the project was originally named Zipline (medium.com/airbnb-engineering).
Cited in: part17
YouTube. "Community Guidelines enforcement" transparency reports (transparencyreport.google.com).
Cited in: part17
Meta. "Community Standards Enforcement Report" and its published methodology (prevalence, content actioned, proactive rate, appeals and restorations) (transparency.meta.com).
Cited in: part17
Ng, A., Ma, T. CS229 Lecture Notes, part on generative learning algorithms (GDA, Naive Bayes). cs229.stanford.edu
Cited in: part02
Järvelin & Kekäläinen, "Cumulated gain-based evaluation of IR techniques", ACM TOIS, 2002.
Cited in: part13
Breck, E. et al. "Data Validation for Machine Learning." MLSys 2019 (proceedings.mlsys.org).
Cited in: part17
A. Odena, V. Dumoulin, C. Olah, "Deconvolution and Checkerboard Artifacts", Distill, 2016. distill.pub
Cited in: part04
Zong, B. et al. "Deep Autoencoding Gaussian Mixture Model for Unsupervised Anomaly Detection." ICLR 2018. openreview.net
Cited in: part02
ICAO. "Doc 9303, Machine Readable Travel Documents" (MRZ format and check digits) (icao.int).
Cited in: part17
Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., Salakhutdinov, R. (2014). Dropout: A Simple Way to Prevent Neural Networks from Overfitting. JMLR 15(56):1929–1958. jmlr.org
Cited in: part03
A. Rogozhnikov, "Einops: Clear and Reliable Tensor Manipulations with Einstein-like Notation", ICLR 2022. OpenReview · ICLR page · einops.rocks
Cited in: part01
Nix & Weigend, "Estimating the mean and variance of the target probability distribution", ICNN 1994 (the original heteroscedastic NLL head).
Cited in: part13
Griewank, A., Walther, A. (2008). Evaluating Derivatives: Principles and Techniques of Algorithmic Differentiation, 2nd ed., SIAM. Google Books
Cited in: part03
"Experiments at Airbnb", Airbnb Engineering, 2014 (medium.com/airbnb-engineering); M. Shen, "Selection Bias in Online Experimentation", Airbnb Engineering, 2018 (medium.com/airbnb-engineering).
Cited in: part01
Rendle, S. "Factorization Machines." ICDM 2010 (dl.acm.org).
Cited in: part17
Hyndman, R. J., Athanasopoulos, G. "Forecasting: Principles and Practice," 3rd edition. Online textbook covering hierarchical forecasting, backtesting and evaluation (otexts.com/fpp3).
Cited in: part17
Google Careers: How we hire; Interviewing at Google: best practices, advice, and tips.
Cited in: part20, prefac
Y. LeCun et al., "Gradient-Based Learning Applied to Document Recognition", Proc. IEEE 86(11), 1998 (LeNet-5). leon.bottou.org
Cited in: part04
Friedman, J. "Greedy Function Approximation: A Gradient Boosting Machine." Annals of Statistics 29(5), 2001. Project Euclid
Cited in: part02
Hello Interview, The system design interview: what is expected at each level,
Cited in: part20
Sculley, D. et al. "Hidden Technical Debt in Machine Learning Systems." NeurIPS 2015 (papers.nips.cc).
Cited in: part13, part17, part17, part17
GitHub Engineering. "How GitHub Copilot is getting better at understanding your code" (github.blog) and "Inside GitHub: Working with the LLMs behind GitHub Copilot" (github.blog), February 2024.
Cited in: part17
PayPal Technology Blog. "How PayPal Uses Real-time Graph Database and Graph Analysis to Fight Fraud" (medium.com/paypal-tech).
Cited in: part17
TikTok Newsroom. "How TikTok recommends videos #ForYou." June 2020 (newsroom.tiktok.com).
Cited in: part17
Wattenberg, M., Viégas, F., Johnson, I. "How to Use t-SNE Effectively." Distill, 2016. distill.pub
Cited in: part02
Drapeau, R. "How we built it: Stripe Radar." Stripe, 2023. stripe.dev
Cited in: part02, part02, part02
Karatzas et al., "ICDAR 2015 competition on Robust Reading", ICDAR 2015 (end-to-end text spotting protocol).
Cited in: part13
A. Krizhevsky, I. Sutskever, G. Hinton, "ImageNet Classification with Deep Convolutional Neural Networks", NeurIPS 2012 (AlexNet). papers.nips.cc
Cited in: part04
Shimodaira, "Improving predictive inference under covariate shift by weighting the log-likelihood function", JSPI, 2000.
Cited in: part13
Deng, A., Xu, Y., Kohavi, R., Walker, T. "Improving the Sensitivity of Online Controlled Experiments by Utilizing Pre-Experiment Data" (CUPED). WSDM 2013 (dl.acm.org).
Cited in: part17
nostalgebraist. interpreting GPT: the logit lens. LessWrong, 2020.
Cited in: part15
Manning, Raghavan & Schütze, Introduction to Information Retrieval, CUP 2008 (ch. 8, evaluation).
Cited in: part13
G. Strang, Introduction to Linear Algebra, 5th ed., Wellesley-Cambridge Press, 2016, and the lectures for MIT 18.06 Linear Algebra on OpenCourseWare.
Cited in: part01
Chip Huyen, Introduction to Machine Learning Interviews (free, online): huyenchip.com/ml-interviews-book.
Cited in: prefac
Vig, J. et al. Investigating Gender Bias in Language Models Using Causal Mediation Analysis.
Cited in: part15
Arthur, D., Vassilvitskii, S. "k-means++: The Advantages of Careful Seeding." SODA 2007. PDF
Cited in: part02
Ashok Elluswamy. "Keynote, CVPR 2022 Workshop on Autonomous Driving." YouTube
Cited in: part11, part11
Ashok Elluswamy. "Keynote, CVPR 2023 Workshop on Autonomous Driving." YouTube
Cited in: part11
Chapelle, Joachims, Radlinski & Yue, "Large-scale validation and analysis of interleaved search evaluation", ACM TOIS, 2012.
Cited in: part13
Burges, C. J. C. et al. "Learning to Rank using Gradient Descent." ICML 2005 (icml.cc).
Cited in: part17
Lloyd, S. "Least Squares Quantization in PCM." IEEE Trans. Information Theory 28(2), 1982. DOI
Cited in: part02
Ke, G. et al. "LightGBM: A Highly Efficient Gradient Boosting Decision Tree." NeurIPS 2017. proceedings.neurips.cc; Features docs
Cited in: part02
Dempster, A., Laird, N., Rubin, D. "Maximum Likelihood from Incomplete Data via the EM Algorithm." JRSS-B 39(1), 1977. academic.oup.com
Cited in: part02
Meta Careers: Preparing for your full loop interview (ML); Hiring process.
Cited in: prefac
Bishop, C. "Mixture Density Networks." Aston University NCRG/94/004, 1994. publications.aston.ac.uk
Cited in: part02
Hello Interview, ML System Design in a Hurry: introduction, delivery framework, evaluation; and the blog post The system design interview: what is expected at each level.
Cited in: part17, prefac
Chapelle, O. "Modeling Delayed Feedback in Display Advertising." KDD 2014 (dl.acm.org).
Cited in: part17
Hornik, K., Stinchcombe, M., White, H. (1989). Multilayer feedforward networks are universal approximators. Neural Networks 2(5), 359–366. ScienceDirect
Cited in: part03
Goh et al., Multimodal Neurons in Artificial Neural Networks, Distill, 2021.
Cited in: part08
Wang, B. et al. Neural Cleanse: Identifying and Mitigating Backdoor Attacks in Neural Networks.
Cited in: part15
Zhao, B., Narita, K., Orten, B., Egan, J. "Notification Volume Control and Optimization System at Pinterest." KDD 2018 (dl.acm.org).
Cited in: part17
Naeini, Cooper & Hauskrecht, "Obtaining Well Calibrated Probabilities Using Bayesian Binning", AAAI 2015 (ECE).
Cited in: part13
Broder. On the resemblance and containment of documents. Compression and Complexity of Sequences, 1997 (MinHash). ACM DL
Cited in: part06
Meta Newsroom. "Open-Sourcing Photo- and Video-Matching Technology to Make the Internet Safer" (PDQ and TMK+PDQF), August 2019 (about.fb.com).
Cited in: part17
Yu, G.-I. et al. Orca: A Distributed Serving System for Transformer-Based Generative Models.
Cited in: part14
Wang, X. et al. "Position Bias Estimation for Unbiased Learning to Rank in Personal Search." WSDM 2018 (dl.acm.org).
Cited in: part17
Platt, J. "Probabilistic Outputs for Support Vector Machines and Comparisons to Regularized Likelihood Methods." Advances in Large Margin Classifiers, MIT Press, 1999. Stable implementation: Lin, Lin, Weng, "A Note on Platt's Probabilistic Outputs for SVMs", Machine Learning 2007. PDF
Cited in: part02, part02
Jégou, H., Douze, M., Schmid, C. "Product Quantization for Nearest Neighbor Search." IEEE TPAMI 33(1), 2011. ACM DL
Cited in: part02, part02
Tang, H. et al. "Progressive Layered Extraction (PLE): A Novel Multi-Task Learning (MTL) Model for Personalized Recommendations." RecSys 2020 (dl.acm.org).
Cited in: part17
Willison, S. Prompt injection post series and
Cited in: part15
Rahimi, A., Recht, B. "Random Features for Large-Scale Kernel Machines." NeurIPS 2007 (random Fourier features; see Exercise 3). proceedings.neurips.cc
Cited in: part02
Breiman, L. "Random Forests." Machine Learning 45, 2001. Springer
Cited in: part02
Bradley & Terry, "Rank Analysis of Incomplete Block Designs: I. The Method of Paired Comparisons", Biometrika, 1952; Hunter, "MM algorithms for generalized Bradley-Terry models", Annals of Statistics, 2004.
Cited in: part13
Tibshirani, R. "Regression Shrinkage and Selection via the Lasso." JRSS-B 58(1), 1996. DOI
Cited in: part02
Friedman, J., Hastie, T., Tibshirani, R. "Regularization Paths for Generalized Linear Models via Coordinate Descent." J. Stat. Software 33(1), 2010. jstatsoft.org
Cited in: part02
Zinkevich, M. "Rules of Machine Learning: Best Practices for ML Engineering." Google Developers guide (2017) (developers.google.com).
Cited in: part13, part17, part17
Yi, X. et al. "Sampling-Bias-Corrected Neural Modeling for Large Corpus Item Recommendations." RecSys 2019 (dl.acm.org).
Cited in: part17
Google. Secure AI Framework (SAIF) and the
Cited in: part15
Reynolds, D., Quatieri, T., Dunn, R. "Speaker Verification Using Adapted Gaussian Mixture Models." Digital Signal Processing 10, 2000. sciencedirect.com
Cited in: part02, part02
Cortes, C., Vapnik, V. "Support-Vector Networks." Machine Learning 20, 1995. Springer
Cited in: part02
Tesla. AI Day 2021 (recorded presentation) and AI Day 2022 (recorded presentation): multi-camera BEV fusion, HydraNet multi-task architecture, video/temporal module, occupancy networks, auto-labelling, the data engine, shadow mode.
Cited in: part17
Tesla. AI Day presentations. YouTube playlist
Cited in: part11
Hastie, Tibshirani, Friedman. The Elements of Statistical Learning, ch. 6 (kernel smoothing), ch. 12 (SVMs). Free PDF
Cited in: part02, part02, part02, part02
Dowson & Landau, "The Fréchet distance between multivariate normal distributions", Journal of Multivariate Analysis, 1982.
Cited in: part13
K. Petersen & M. Pedersen, The Matrix Cookbook, Technical University of Denmark, November 2012. DTU publication database
Cited in: part01, part01
Everingham et al., "The PASCAL Visual Object Classes (VOC) Challenge", IJCV 2010 (11-point and all-point AP).
Cited in: part13
Saito & Rehmsmeier, "The Precision-Recall Plot Is More Informative than the ROC Plot When Evaluating Binary Classifiers on Imbalanced Datasets", PLOS ONE, 2015.
Cited in: part13
Davis & Goadrich, "The Relationship Between Precision-Recall and ROC Curves", ICML 2006.
Cited in: part13
Zadrozny, B., Elkan, C. "Transforming Classifier Scores into Accurate Multiclass Probability Estimates." KDD 2002 (isotonic calibration). ACM DL
Cited in: part02
Kohavi, Tang & Xu, Trustworthy Online Controlled Experiments, CUP 2020; companion site experimentguide.com.
Cited in: part13
Glorot, X., Bengio, Y. (2010). Understanding the difficulty of training deep feedforward neural networks. AISTATS. proceedings.mlr.press/v9/glorot10a
Cited in: part03
Brier, "Verification of forecasts expressed in terms of probability", Monthly Weather Review, 1950.
Cited in: part13
Vienna Convention on Road Signs and Signals
Cited in: part20
Zhang, Y. et al. "Visual Search at Alibaba." KDD 2018 (dl.acm.org).
Cited in: part17
van der Maaten, L., Hinton, G. "Visualizing Data using t-SNE." JMLR 9, 2008. jmlr.org
Cited in: part02
Sculley, D. "Web-scale k-means clustering." WWW 2010. ACM DL
Cited in: part02
Beyer, K., Goldstein, J., Ramakrishnan, R., Shaft, U. "When Is 'Nearest Neighbor' Meaningful?" ICDT 1999. Springer; minds.wisconsin.edu
Cited in: part02

Cited by title¶

316 sources.

Liu, Mao, Wu, Feichtenhofer, Darrell, Xie. "A ConvNet for the 2020s", CVPR 2022.
Cited in: part19
Bellemare, Dabney & Munos, "A Distributional Perspective on Reinforcement Learning", ICML
Cited in: part12
Z. Zhang, "A Flexible New Technique for Camera Calibration", IEEE TPAMI 22(11), 2000.
Cited in: part04
Mirhoseini et al., "A graph placement methodology for fast chip design", Nature 594
Cited in: part12
C. Shannon, "A Mathematical Theory of Communication", Bell System Technical Journal, 1948.
Cited in: part01
Y. Nesterov, "A method for solving the convex programming problem with convergence rate \(O(1/k^2)\)", Soviet Mathematics Doklady, 1983.
Cited in: part01
R. Kalman, "A New Approach to Linear Filtering and Prediction Problems", Journal of Basic Engineering, 1960.
Cited in: part01
Nagel et al., A White Paper on Neural Network Quantization,
Cited in: part20
Chen, C. et al. Accelerating Large Language Model Decoding with Speculative Sampling. 2023.
Cited in: part14
J. Duchi, E. Hazan & Y. Singer, "Adaptive Subgradient Methods for Online Learning and Stochastic Optimization", JMLR 12, 2011 (AdaGrad).
Cited in: part01
Liang, M. et al. "AIDE: An Automatic Data Engine for Object Detection in Autonomous Driving." CVPR 2024.
Cited in: part17
Ainslie et al. "GQA: Training Generalized Multi-Query Transformer Models from
Cited in: part19
Ainslie, J. et al. (2023). GQA: Training Generalized Multi-Query Transformer Models from
Cited in: part05*
Vovk, Gammerman & Shafer, Algorithmic Learning in a Random World, Springer 2005.
Cited in: part13
L. Wasserman, All of Statistics, Springer, 2004.
Cited in: part01
Google Research (2019). An All-Neural On-Device Speech Recognizer.
Cited in: part05
D. Nistér, "An Efficient Solution to the Five-Point Relative Pose Problem", IEEE TPAMI 26(6), 2004.
Cited in: part04
O. Chapelle & L. Li, "An Empirical Evaluation of Thompson Sampling", NeurIPS 2011.
Cited in: part01
Dosovitskiy et al. "An Image is Worth 16x16 Words", ICLR 2021.
Cited in: part19
B. Efron & R. Tibshirani, An Introduction to the Bootstrap, Chapman & Hall, 1993.
Cited in: part01, part13
B. Lucas and T. Kanade, "An Iterative Image Registration Technique with an Application to Stereo Vision", IJCAI 1981.
Cited in: part04
Anthropic, ["Introducing Contextual
Cited in: part13
Schulman, J. (2020). Approximating KL Divergence (the \(k_1\), \(k_2\), \(k_3\) estimators), blog post.
Cited in: part07
Michel, Levy, Neubig. "Are Sixteen Heads Really Better than One?", NeurIPS 2019.
Cited in: part19
Netflix Technology Blog, "Artwork Personalization at Netflix", 2017.
Cited in: part12
Asai et al., ["Self-RAG: Learning to Retrieve, Generate, and Critique through
Cited in: part13
Lai & Robbins, "Asymptotically efficient adaptive allocation rules", Advances in Applied
Cited in: part12*
Vaswani et al. "Attention Is All You Need", NeurIPS 2017.
Cited in: part05, part05, part05, part05, part19
PyTorch, "Autograd mechanics", official documentation, on in-place operations and the version counter.
Cited in: part16
Sundararajan, M., Taly, A., Yan, Q. Axiomatic Attribution for Deep Networks. ICML 2017.
Cited in: part15
Bachrach et al., ["Speeding up the Xbox recommender system using a Euclidean
Cited in: part13
Bahdanau, D., Cho, K. & Bengio, Y. (2015). Neural Machine Translation by Jointly Learning to
Cited in: part05*
Bahdanau, D., Cho, K. & Bengio, Y. (2015). Neural Machine Translation by Jointly Learning to Align
Cited in: part05*
Bansal, Krizhevsky & Ogale, "ChauffeurNet: Learning to Drive by Imitating the Best and
Cited in: part12
Bengio, S., Vinyals, O., Jaitly, N. & Shazeer, N. (2015). Scheduled Sampling for Sequence
Cited in: part05*
Higgins et al., beta-VAE: Learning Basic Visual Concepts with a Constrained Variational Framework, ICLR 2017. (OpenReview; link not verified from this environment, search the title.)
Cited in: part09
Book cross-references: ads calibration and down-sampling, uncertainty & reliability, ML platform & point-in-time features.
Cited in: part17
Book cross-references: CLIP & contrastive learning, retrieval & RAG, OCR & document understanding, content moderation.
Cited in: part17
Book cross-references: detection, VLM architecture, quantization, uncertainty & reliability, weak supervision & auto-labeling, AV perception (teacher-student), LLM assistant with RAG (document ingestion).
Cited in: part17
Book cross-references: distributed training, training systems, inference systems, evaluation, LLM assistant with RAG, fraud & anomaly detection, forecasting & ETA.
Cited in: part17
Book cross-references: feed ranking (multi-task, sequence features), uncertainty & reliability (calibration).
Cited in: part17
Book cross-references: forecasting & ETA (switchbacks in marketplaces), feed ranking (long-term holdouts), evaluation.
Cited in: part17
Book cross-references: fraud & anomaly detection, LLM assistant with RAG, visual search, safety & failure modes.
Cited in: part17
Book cross-references: multi-camera & BEV, sensor fusion, tracking, occupancy & temporal perception, perception foundation models, self-supervised learning, weak supervision & auto-labeling, quantization, AV perception system design.
Cited in: part17, part17
Book cross-references: notifications, uplift & experimentation (switchbacks), ML platform & point-in-time features, uncertainty & reliability.
Cited in: part17
Book cross-references: Part XIII evaluation, Part XIV inference systems, Part XIV roofline.
Cited in: part17
Book cross-references: retrieval & RAG, evaluation, uncertainty & reliability, inference systems, fine-tuning & LoRA, SFT, agents & tool use, content moderation.
Cited in: part17, part17
Bricken, T. et al. [Towards Monosemanticity: Decomposing Language Models With Dictionary
Cited in: part15*
NumPy, "Broadcasting", NumPy user guide. The three-rule statement is the one to memorise.
Cited in: part16, part16
PyTorch, "Broadcasting semantics" and "torch.gather", official documentation.
Cited in: part16
Brohan et al., "RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic
Cited in: part12
B. Triggs et al., "Bundle Adjustment: A Modern Synthesis", Vision Algorithms workshop, 1999.
Cited in: part04
Wei et al. "Chain-of-Thought Prompting Elicits Reasoning in Large Language Models",
Cited in: part19
Chen, Beutel, Covington, Jain, Belletti & Chi, "Top-K Off-Policy Correction for a
Cited in: part12
Chen, Kornblith, Norouzi, Hinton. "A Simple Framework for Contrastive Learning of
Cited in: part19
Chen, S. et al. (2023). Extending Context Window of Large Language Models via Positional
Cited in: part05*
Cho, K. et al. (2014). Learning Phrase Representations using RNN Encoder-Decoder for Statistical
Cited in: part05*
R. Szeliski, Computer Vision: Algorithms and Applications, 2nd ed., chapters on image processing and pyramids.
Cited in: part04
Graves, A. et al. "Connectionist Temporal Classification: Labelling Unsegmented Sequence Data with Recurrent Neural Networks." ICML 2006.
Cited in: part17
Kumar et al., "Conservative Q-Learning for Offline Reinforcement Learning", NeurIPS 2020.
Cited in: part12
Lillicrap et al., "Continuous control with deep reinforcement learning" (DDPG), ICLR 2016.
Cited in: part12
S. Boyd & L. Vandenberghe, Convex Optimization, Cambridge University Press, 2004 (chapter 5 on duality and KKT).
Cited in: part01, part01
Cormack, Clarke & Buettcher, ["Reciprocal Rank Fusion outperforms Condorcet and
Cited in: part13
Dal Pozzolo, A. et al. "Credit Card Fraud Detection: A Realistic Modeling and a Novel Learning Strategy." IEEE Transactions on Neural Networks and Learning Systems, 2018.
Cited in: part17
Järvelin, K., Kekäläinen, J. "Cumulated Gain-Based Evaluation of IR Techniques." ACM TOIS 2002.
Cited in: part17
Dao, Fu, Ermon, Rudra, Ré. "FlashAttention: Fast and Memory-Efficient Exact Attention
Cited in: part19
Dao, T. et al. (2022). FlashAttention: Fast and Memory-Efficient Exact Attention with
Cited in: part05*
Loshchilov, Hutter. "Decoupled Weight Decay Regularization", ICLR 2019.
Cited in: part19
Lee, K. et al. Deduplicating Training Data Makes Language Models Better. ACL 2022.
Cited in: part15
van Hasselt et al., "Deep Reinforcement Learning and the Deadly Triad", 2018.
Cited in: part12
van Hasselt, Guez & Silver, "Deep Reinforcement Learning with Double Q-learning", AAAI
Cited in: part12
Evans & Gao, "DeepMind AI reduces Google data centre cooling bill by 40%", 2016.
Cited in: part12
DeepSeek-AI. "DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via
Cited in: part19
DeepSeek-AI. "DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts
Cited in: part19
Hewitt, J., Liang, P. Designing and Interpreting Probes with Control Tasks. EMNLP 2019.
Cited in: part15
Dettmers, Pagnoni, Holtzman, Zettlemoyer. "QLoRA: Efficient Finetuning of Quantized
Cited in: part19
Devlin, J. et al. (2019). BERT: Pre-training of Deep Bidirectional Transformers for Language
Cited in: part05*
Devlin, J. et al. (2019). BERT (WordPiece in practice).
Cited in: part05
Chi et al., "Diffusion Policy: Visuomotor Policy Learning via Action Diffusion", RSS 2023.
Cited in: part12
Lan, K. K. G., DeMets, D. L. "Discrete Sequential Boundaries for Clinical Trials." Biometrika 1983.
Cited in: part17
A. Oppenheim and R. Schafer, Discrete-Time Signal Processing, for the sampling theorem and the DFT convolution theorem.
Cited in: part04
van Hasselt, "Double Q-learning", NIPS 2010. The tabular version of the estimator used by
Cited in: part12
Wang et al., "Dueling Network Architectures for Deep Reinforcement Learning", ICML 2016.
Cited in: part12
Edge et al., ["From Local to Global: A Graph RAG Approach to Query-Focused
Cited in: part13
Ross & Bagnell, "Efficient Reductions for Imitation Learning", AISTATS 2010. The
Cited in: part12
Xiao, G. et al. (2023). Efficient Streaming Language Models with Attention Sinks.
Cited in: part05
Tim Rocktäschel, "Einsum is All You Need" (2018), for the einsum notation used in drill 28.
Cited in: part16
T. Cover & J. Thomas, Elements of Information Theory, 2nd ed., Wiley, 2006.
Cited in: part01
Carion et al. "End-to-End Object Detection with Transformers", ECCV 2020.
Cited in: part19
V. Lepetit, F. Moreno-Noguer, P. Fua, "EPnP: An Accurate O(n) Solution to the PnP Problem", IJCV 81(2), 2009.
Cited in: part04
Es et al., ["RAGAS: Automated Evaluation of Retrieval Augmented
Cited in: part13
Raffel, C. et al. (2020). Exploring the Limits of Transfer Learning (T5 relative bias).
Cited in: part05
Carlini, N. et al. Extracting Training Data from Large Language Models. USENIX Security 2021.
Cited in: part15
Singh, A., Joachims, T. "Fairness of Exposure in Rankings." KDD 2018.
Cited in: part17
Shazeer, N. (2019). Fast Transformer Decoding: One Write-Head is All You Need.
Cited in: part05, part19
Weinberger, K. et al. "Feature Hashing for Large Scale Multitask Learning." ICML 2009.
Cited in: part17
Auer, Cesa-Bianchi & Fischer, "Finite-time Analysis of the Multiarmed Bandit Problem",
Cited in: part12
Dao, T. FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning.
Cited in: part14
Dao, T. et al. FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness.
Cited in: part14
Lin, Goyal, Girshick, He, Dollár. "Focal Loss for Dense Object Detection", ICCV 2017.
Cited in: part19, part19
Formal, Piwowarski & Clinchant, ["SPLADE: Sparse Lexical and Expansion Model for
Cited in: part13
Fujimoto, van Hoof & Meger, "Addressing Function Approximation Error in Actor-Critic
Cited in: part12
OpenAI, "Function calling" and "Structured model outputs".
Cited in: part12
Gao et al., ["Precise Zero-Shot Dense Retrieval without Relevance
Cited in: part13
Hendrycks, D. & Gimpel, K. (2016). Gaussian Error Linear Units (GELUs).
Cited in: part05
C. Rasmussen & C. Williams, Gaussian Processes for Machine Learning, MIT Press, 2006 (chapter 2).
Cited in: part01
Ge, He, Ke & Sun, ["Optimized Product Quantization for Approximate Nearest Neighbor
Cited in: part13
Ho & Ermon, "Generative Adversarial Imitation Learning", NIPS 2016.
Cited in: part12
Shazeer, N. (2020). GLU Variants Improve Transformer.
Cited in: part05, part19
Gollapudi et al., ["Filtered-DiskANN: Graph Algorithms for Approximate Nearest Neighbor
Cited in: part13
J. Shi and C. Tomasi, "Good Features to Track", CVPR 1994.
Cited in: part04
Wu, Y. et al. (2016). Google's Neural Machine Translation System (wordpiece).
Cited in: part05, part05, part05
Black, S. et al. (2022). GPT-NeoX-20B (the rotate-half RoPE convention).
Cited in: part05
Black, S. et al. (2022). GPT-NeoX-20B: An Open-Source Autoregressive Language Model.
Cited in: part05
Ugander, J. et al. "Graph Cluster Randomization: Network Exposure to Multiple Universes." KDD 2013.
Cited in: part17
Friedman. "Greedy Function Approximation: A Gradient Boosting Machine", Annals of
Cited in: part19
Greshake, K. et al. Not what you've signed up for: Compromising Real-World LLM-Integrated
Cited in: part15*
Gu, T., Dolan-Gavitt, B., Garg, S. BadNets: Identifying Vulnerabilities in the Machine Learning
Cited in: part15*
Guo et al., ["Accelerating Large-Scale Inference with Anisotropic Vector
Cited in: part13
Han, Yao, Yu, Niu, Xu, Hu, Tsang, Sugiyama. "Co-teaching: Robust Training of Deep
Cited in: part19
He, Chen, Xie, Li, Dollár, Girshick. "Masked Autoencoders Are Scalable Vision
Cited in: part19
He, Zhang, Ren, Sun. ICCV 2015: the PReLU and He-initialization paper.
Cited in: part19
Figure, "Helix: A Vision-Language-Action Model for Generalist Humanoid Control", 2025.
Cited in: part12
K. Chellapilla, S. Puri, P. Simard, "High Performance Convolutional Neural Networks for Document Processing", 2006, the original im2col-as-GEMM formulation.
Cited in: part04
Sergeev, A., Del Balso, M. Horovod: fast and easy distributed deep learning in TensorFlow.
Cited in: part14
Santurkar, Tsipras, Ilyas, Madry. "How Does Batch Normalization Help Optimization?",
Cited in: part19
Huang, Dossa, Raffin, Kanervisto & Wang, "The 37 Implementation Details of Proximal Policy
Cited in: part12
Huang et al., ["Embedding-based Retrieval in Facebook
Cited in: part13
Huang, Y. et al. GPipe: Efficient Training of Giant Neural Networks using Pipeline
Cited in: part14*
Hugging Face, TRL documentation, DPOTrainer (loss variants: sigmoid, ipo, kto_pair, simpo via loss_type).
Cited in: part07
Hugging Face, TRL documentation, PPOTrainer.
Cited in: part07
Hugging Face, TRL documentation, SFTTrainer (assistant-only loss and packing options).
Cited in: part07
Mnih et al., "Human-level control through deep reinforcement learning", Nature 518
Cited in: part12
A. Deng, Y. Xu, R. Kohavi & T. Walker, "Improving the Sensitivity of Online Controlled Experiments by Utilizing Pre-Experiment Data", WSDM 2013.
Cited in: part01
Jouppi, N. et al. In-Datacenter Performance Analysis of a Tensor Processing Unit. ISCA 2017.
Cited in: part14
R. Hartley, "In Defense of the Eight-Point Algorithm", IEEE TPAMI 19(6), 1997.
Cited in: part04
D. MacKay, Information Theory, Inference, and Learning Algorithms, Cambridge University Press, 2003.
Cited in: part01
Edelman, B., Ostrovsky, M., Schwarz, M. "Internet Advertising and the Generalized Second-Price Auction." American Economic Review 2007.
Cited in: part17
Liu, F. T., Ting, K. M., Zhou, Z.-H. "Isolation Forest." ICDM 2008.
Cited in: part17
Jégou, Douze & Schmid, ["Product Quantization for Nearest Neighbor
Cited in: part13
Jacob et al., Quantization and Training of Neural Networks for Efficient
Cited in: part20*
Jacobs, S. A. et al. DeepSpeed Ulysses: System Optimizations for Enabling Training of
Cited in: part14*
Johnson, Douze & Jégou, ["Billion-scale similarity search with
Cited in: part13
Jouppi, N. et al. TPU v4: An Optically Reconfigurable Supercomputer for Machine Learning with
Cited in: part14*
Karpukhin et al., ["Dense Passage Retrieval for Open-Domain Question
Cited in: part13
Keskar et al. "On Large-Batch Training for Deep Learning: Generalization Gap and
Cited in: part19
Khattab & Zaharia, ["ColBERT: Efficient and Effective Passage Search via Contextualized
Cited in: part13
Krishnamoorthi, Quantizing deep convolutional networks for efficient inference: a
Cited in: part20*
Kudo, T. (2018). Subword Regularization: Improving Neural Network Translation Models with
Cited in: part05*
Kudo, T. & Richardson, J. (2018). SentencePiece: A simple and language independent subword
Cited in: part05*
Kumar, Zhou, Tucker & Levine, "Conservative Q-Learning for Offline Reinforcement
Cited in: part12
Kwon et al. "Efficient Memory Management for Large Language Model Serving with
Cited in: part19
Kwon, W. et al. (2023). Efficient Memory Management for Large Language Model Serving with
Cited in: part05*
Kwon, W. et al. Efficient Memory Management for Large Language Model Serving with
Cited in: part14*
Petrov, A. et al. (2023). Language Model Tokenizers Introduce Unfairness Between Languages.
Cited in: part05
Brown, T. et al. (2020). Language Models are Few-Shot Learners (GPT-3).
Cited in: part05
Radford, A. et al. (2019). Language Models are Unsupervised Multitask Learners (byte-level BPE).
Cited in: part05, part05
D. Blei, A. Ng & M. Jordan, "Latent Dirichlet Allocation", JMLR 3, 2003.
Cited in: part01
Zhou, B. et al. Learning Deep Features for Discriminative Localization. CVPR 2016.
Cited in: part15
Cho, K. et al. (2014). Learning Phrase Representations using RNN Encoder-Decoder.
Cited in: part05
D. Rumelhart, G. Hinton & R. Williams, "Learning representations by back-propagating errors", Nature 323, 1986.
Cited in: part01
Burges, C. J. C., Ragno, R., Le, Q. V. "Learning to Rank with Nonsmooth Cost Functions." NeurIPS 2006.
Cited in: part17
OpenAI (2024). Learning to reason with LLMs; OpenAI (2024). o1 System Card.
Cited in: part07, part07
T. Tieleman & G. Hinton, "Lecture 6.5, RMSProp", Coursera: Neural Networks for Machine Learning, 2012.
Cited in: part01
Leviathan, Kalman, Matias. "Fast Inference from Transformers via Speculative
Cited in: part19, part19
Leviathan, Y., Kalman, M., Matias, Y. Fast Inference from Transformers via Speculative
Cited in: part14*
Lewis et al., ["Retrieval-Augmented Generation for Knowledge-Intensive NLP
Cited in: part13
Li, Chen, Hu, Yang. "Understanding the Disharmony between Dropout and Batch
Cited in: part19
Li, Chu, Langford & Schapire, "A Contextual-Bandit Approach to Personalized News Article
Cited in: part12
Chan, W., Jaitly, N., Le, Q. & Vinyals, O. (2015). Listen, Attend and Spell.
Cited in: part05
Liu et al., ["Lost in the Middle: How Language Models Use Long
Cited in: part13
Liu, H., Zaharia, M., Abbeel, P. Ring Attention with Blockwise Transformers for
Cited in: part14*
Inan, H. et al. Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations. 2023.
Cited in: part15
Touvron, H. et al. (2023). LLaMA: Open and Efficient Foundation Language Models.
Cited in: part05, part05
Meng, K. et al. Locating and Editing Factual Associations in GPT. NeurIPS 2022.
Cited in: part15
Hochreiter, S. & Schmidhuber, J. (1997). Long Short-Term Memory. Neural Computation 9(8),
Cited in: part05
Hu et al. "LoRA: Low-Rank Adaptation of Large Language Models", ICLR 2022.
Cited in: part19
Liu et al. "Lost in the Middle: How Language Models Use Long Contexts", TACL 2024.
Cited in: part19
Luong, M.-T., Pham, H. & Manning, C. D. (2015). Effective Approaches to Attention-based Neural
Cited in: part05*
Malkov & Yashunin, ["Efficient and robust approximate nearest neighbor search using
Cited in: part13
Silver et al., "Mastering the game of Go without human knowledge", Nature 550 (2017).
Cited in: part12
Y. Koren, R. Bell & C. Volinsky, "Matrix Factorization Techniques for Recommender Systems", IEEE Computer, 2009.
Cited in: part01
Ziebart, Maas, Bagnell & Dey, "Maximum Entropy Inverse Reinforcement Learning", AAAI 2008.
Cited in: part12
Dawid, Skene, Maximum Likelihood Estimation of Observer Error-Rates Using the EM Algorithm, Applied Statistics, 1979. (The origin of the annotator model; search the title.)
Cited in: part10
McInerney et al., "Explore, Exploit, Explain: Personalizing Explainable Recommendations
Cited in: part12
Shokri, R. et al. Membership Inference Attacks Against Machine Learning Models. IEEE S&P 2017.
Cited in: part15
Carlini, N. et al. Membership Inference Attacks From First Principles. IEEE S&P 2022.
Cited in: part15
R. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision, 2nd ed., Cambridge, 2004. The standard reference for every derivation in this chapter.
Cited in: part04
Nakkiran, Kaplun, Bansal, Yang, Barak, Sutskever. "Deep Double Descent: Where Bigger
Cited in: part19
Narayanan, D. et al. Efficient Large-Scale Language Model Training on GPU Clusters Using
Cited in: part14*
Ng, Harada & Russell, "Policy invariance under reward transformations: theory and
Cited in: part12
NLLB Team (2022). No Language Left Behind: Scaling Human-Centered Machine Translation.
Cited in: part05
Northcutt, Jiang, Chuang. "Confident Learning: Estimating Uncertainty in Dataset
Cited in: part19, part19
Kostrikov, Nair & Levine, "Offline Reinforcement Learning with Implicit Q-Learning", ICLR
Cited in: part12
Guo, Pleiss, Sun, Weinberger. "On Calibration of Modern Neural Networks", ICML 2017.
Cited in: part19
Xiong, R. et al. (2020). On Layer Normalization in the Transformer Architecture.
Cited in: part05, part19
R. Kohavi, A. Deng, B. Frasca, T. Walker, Y. Xu & N. Pohlmann, "Online Controlled Experiments at Large Scale", KDD 2013.
Cited in: part01
Ootomo et al., ["CAGRA: Highly Parallel Graph Construction and Approximate Nearest
Cited in: part13
Open X-Embodiment Collaboration, "Open X-Embodiment: Robotic Learning Datasets and RT-X
Cited in: part12
OpenAI, "OpenAI Five" and "Dota 2 with Large Scale Deep Reinforcement Learning".
Cited in: part12
Wickramasuriya, S. L., Athanasopoulos, G., Hyndman, R. J. "Optimal Forecast Reconciliation for Hierarchical and Grouped Time Series Through Trace Minimization." JASA 2019.
Cited in: part17
Chowdhery, A. et al. (2022). PaLM: Scaling Language Modeling with Pathways.
Cited in: part05
Pascanu, R., Mikolov, T. & Bengio, Y. (2013). On the difficulty of training Recurrent Neural
Cited in: part05*
Patel et al., ["ACORN: Performant and Predicate-Agnostic Search Over Vector Embeddings
Cited in: part13
C. Bishop, Pattern Recognition and Machine Learning, Springer, 2006 (chapters 1–2; §2.3 for Gaussian conditioning).
Cited in: part01
Peng, Quesnelle, Fan, Shippole. "YaRN: Efficient Context Window Extension of Large
Cited in: part19
N. Snavely, S. Seitz, R. Szeliski, "Photo Tourism: Exploring Photo Collections in 3D", SIGGRAPH 2006.
Cited in: part04
Mnih et al., "Playing Atari with Deep Reinforcement Learning", NIPS Deep Learning
Cited in: part12
Press, O., Smith, N. A. & Lewis, M. (2022). Train Short, Test Long: Attention with Linear Biases
Cited in: part05*
Press, Smith, Lewis. "Train Short, Test Long: Attention with Linear Biases Enables
Cited in: part19
Schaul, Quan, Antonoglou & Silver, "Prioritized Experience Replay", ICLR 2016.
Cited in: part12
Yeom, S. et al. Privacy Risk in Machine Learning: Analyzing the Connection to Overfitting.
Cited in: part15
K. Murphy, Probabilistic Machine Learning: An Introduction, MIT Press, 2022.
Cited in: part01
Jégou, H., Douze, M., Schmid, C. "Product Quantization for Nearest Neighbor Search." IEEE TPAMI 2011.
Cited in: part17
Schulman, Wolski, Dhariwal, Radford & Klimov, "Proximal Policy Optimization Algorithms",
Cited in: part12
Lee, Pseudo-Label: The Simple and Efficient Semi-Supervised Learning Method for Deep Neural Networks, ICML Workshop on Challenges in Representation Learning, 2013. (Workshop paper, no arXiv identifier; search the title.)
Cited in: part10
Li, S. et al. PyTorch Distributed: Experiences on Accelerating Data Parallel Training.
Cited in: part14
Zhao, Y. et al. PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallel. VLDB
Cited in: part14
Justin Johnson, "PyTorch tutorial" (CS231n / EECS 498), for the NumPy-to-Torch translation that this chapter's structure follows.
Cited in: part16
Carlini, N. et al. Quantifying Memorization Across Neural Language Models. ICLR 2023.
Cited in: part15
Radford et al. "Learning Transferable Visual Models From Natural Language
Cited in: part19
Rafailov et al. "Direct Preference Optimization: Your Language Model is Secretly a
Cited in: part19
Raffel, C. et al. (2020). Exploring the Limits of Transfer Learning with a Unified Text-to-Text
Cited in: part05*
Hessel et al., "Rainbow: Combining Improvements in Deep Reinforcement Learning", AAAI
Cited in: part12
Rajbhandari, S. et al. ZeRO: Memory Optimizations Toward Training Trillion Parameter
Cited in: part14*
M. Fischler and R. Bolles, "Random Sample Consensus", Communications of the ACM 24(6), 1981.
Cited in: part04
Bradley, R. A. and Terry, M. E. (1952). Rank Analysis of Incomplete Block Designs: I. The Method of Paired Comparisons. Biometrika 39(3/4).
Cited in: part07
Yao et al., "ReAct: Synergizing Reasoning and Acting in Language Models", ICLR 2023.
Cited in: part12
Radcliffe, N. J., Surry, P. D. "Real-World Uplift Modelling with Significance-Based Uplift Trees." Stochastic Solutions white paper, 2011.
Cited in: part17
Google Research (2020). Recent Advances in Google Translate.
Cited in: part05
Cormack, G. V., Clarke, C. L. A., Buettcher, S. "Reciprocal Rank Fusion Outperforms Condorcet and Individual Rank Learning Methods." SIGIR 2009.
Cited in: part17
Korthikanti, V. et al. Reducing Activation Recomputation in Large Transformer Models.
Cited in: part14, part14
Koenker, R., Bassett, G. "Regression Quantiles." Econometrica 1978.
Cited in: part17
Zou and Hastie. "Regularization and variable selection via the elastic net", Journal
Cited in: part19
Sutton & Barto, Reinforcement Learning: An Introduction, 2nd ed., chapters 2 (bandits),
Cited in: part12, part12
Liu, Y. et al. (2019). RoBERTa: A Robustly Optimized BERT Pretraining Approach.
Cited in: part05
Robertson & Zaragoza, ["The Probabilistic Relevance Framework: BM25 and
Cited in: part13
Eykholt, K. et al. Robust Physical-World Attacks on Deep Learning Visual Classification.
Cited in: part15
Su, J. et al. (2021). RoFormer: Enhanced Transformer with Rotary Position Embedding.
Cited in: part05, part19
Zhang, B. & Sennrich, R. (2019). Root Mean Square Layer Normalization.
Cited in: part05, part19
Ross, Gordon & Bagnell, "A Reduction of Imitation Learning and Structured Prediction to
Cited in: part12
Brohan et al., "RT-1: Robotics Transformer for Real-World Control at Scale", 2022.
Cited in: part12
Kaplan, J. et al. (2020). Scaling Laws for Neural Language Models.
Cited in: part05, part19
Schaeffer, Miranda, Koyejo. "Are Emergent Abilities of Large Language Models a
Cited in: part19
Schulman, Moritz, Levine, Jordan & Abbeel, "High-Dimensional Continuous Control Using
Cited in: part12
Y. Yan, Y. Mao, B. Li, "SECOND: Sparsely Embedded Convolutional Detection", Sensors 18(10), 2018.
Cited in: part04
Kirillov et al. "Segment Anything", ICCV 2023, for the model-in-the-loop data engine
Cited in: part19, part19
Selvaraju, R. R. et al. Grad-CAM: Visual Explanations from Deep Networks via Gradient-based
Cited in: part15*
Grandvalet, Bengio, Semi-supervised Learning by Entropy Minimization, NeurIPS 2004. (Search the title; the proceedings version is the primary source.)
Cited in: part10
Sennrich, R., Haddow, B. & Birch, A. (2016). Neural Machine Translation of Rare Words with
Cited in: part05*
Zheng, L. et al. SGLang: Efficient Execution of Structured Language Model Programs.
Cited in: part14
Shoeybi, M. et al. Megatron-LM: Training Multi-Billion Parameter Language Models Using
Cited in: part14*
Shrivastava & Li, ["Asymmetric LSH (ALSH) for Sublinear Time Maximum Inner Product
Cited in: part13
Simonyan, K., Vedaldi, A., Zisserman, A. Deep Inside Convolutional Networks: Visualising Image
Cited in: part15*
Xu, J. et al. "Smart Pacing for Effective Online Ad Campaign Optimization." KDD 2015.
Cited in: part17
Haarnoja, Zhou, Abbeel & Levine, "Soft Actor-Critic", ICML 2018.
Cited in: part12
Rumbelow, J. & Watkins, M. (2023). SolidGoldMagikarp (plus, prompt generation).
Cited in: part05
OpenAI, "Solving Rubik's Cube with a robot hand", 2019.
Cited in: part12
Tran, B., Li, J., Madry, A. Spectral Signatures in Backdoor Attacks. NeurIPS 2018.
Cited in: part15
OpenAI, Spinning Up,
Cited in: part12
OpenAI, Spinning Up in Deep RL, Part 1 (key concepts) and
Cited in: part12
Patel, P. et al. Splitwise: Efficient Generative LLM Inference Using Phase Splitting.
Cited in: part14
Tramèr, F. et al. Stealing Machine Learning Models via Prediction APIs. USENIX Security 2016.
Cited in: part15
He, Y. et al. (2018). Streaming End-to-End Speech Recognition for Mobile Devices.
Cited in: part05
Subramanya et al., ["DiskANN: Fast Accurate Billion-point Nearest Neighbor Search on a
Cited in: part13
Sutskever, I., Vinyals, O. & Le, Q. V. (2014). Sequence to Sequence Learning with Neural
Cited in: part05, part05*
Sutton & Barto, 2nd ed., chapter 11 for the triad and Baird's counterexample.
Cited in: part12
Sutton & Barto, 2nd ed., chapter 17.4 on temporal abstraction and options, for the
Cited in: part12
Sutton, McAllester, Singh & Mansour, "Policy Gradient Methods for Reinforcement Learning
Cited in: part12
Jimenez et al., "SWE-bench: Can Language Models Resolve Real-World GitHub Issues?", ICLR
Cited in: part12
Liu, Z. et al. (2021). Swin Transformer: Hierarchical Vision Transformer using Shifted Windows.
Cited in: part05, part19
Fedus, Zoph, Shazeer. "Switch Transformers", JMLR 2022.
Cited in: part19
Agrawal, A. et al. Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve.
Cited in: part14
Templeton, A. et al. [Scaling Monosemanticity: Extracting Interpretable Features from Claude 3
Cited in: part15*
Alexander Rush, "Tensor Considered Harmful" (2019), on named tensors and why shape bugs survive testing.
Cited in: part16
Tesla, AI Day 2021 and 2022 presentations (data engine, auto-labelling, vision-only perception).
Cited in: part13
Tesla AI Day 2021 recorded presentation, on HydraNets and multi-camera perception. No canonical URL for the recording could be confirmed; treat the HydraNet description as reported from the talk rather than published.
Cited in: part04
Tesla AI Day 2021 (recorded presentation), on the vision stack, raw photon counts and multi-camera fusion. No canonical URL for the recording could be confirmed; the claims above are reported from the talk, not from a published paper.
Cited in: part04
Tesla AI Day 2021, recorded public talk, multi-camera Transformer fusion segment.
Cited in: part08
C. Eckart & G. Young, "The approximation of one matrix by another of lower rank", Psychometrika, 1936.
Cited in: part01
The CARL framework, this book, for the delivery mechanics this
Cited in: part20
Holtzman, A. et al. (2019). The Curious Case of Neural Text Degeneration.
Cited in: part05, part05
T. Hastie, R. Tibshirani & J. Friedman, The Elements of Statistical Learning, 2nd ed., Springer, 2009 (chapter 7 on bias–variance and model assessment).
Cited in: part01
Penedo et al. The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scale. 2024.
Cited in: part06
Kuhn, The Hungarian method for the assignment problem, Naval Research Logistics Quarterly, 1955. Munkres, Algorithms for the assignment and transportation problems, J. SIAM, 1957.
Cited in: part08
P. Burt and E. Adelson, "The Laplacian Pyramid as a Compact Image Code", IEEE Trans. Communications 31(4), 1983.
Cited in: part04
Grattafiori, A. et al. (Llama Team, Meta). The Llama 3 Herd of Models. 2024.
Cited in: part05, part05, part14
Saito, T., Rehmsmeier, M. "The Precision-Recall Plot Is More Informative than the ROC Plot When Evaluating Binary Classifiers on Imbalanced Datasets." PLoS ONE, 2015.
Cited in: part17
Karpathy, A. (2015). The Unreasonable Effectiveness of Recurrent Neural Networks.
Cited in: part05
This book: quantization,
Cited in: part20
This book: what staff-level signal looks like for the
Cited in: part20
Tillet, P., Kung, H. T., Cox, D. [Triton: An Intermediate Language and Compiler for Tiled Neural
Cited in: part14*
Anthropic, "Tool use with Claude".
Cited in: part12
Chen et al., "Top-K Off-Policy Correction for a REINFORCE Recommender System", WSDM 2019.
Cited in: part12
PyTorch, "torch.Tensor.view" and "Tensor Views", official documentation. The stride discussion there is the shortest correct account of contiguity.
Cited in: part16
Hoffmann et al. "Training Compute-Optimal Large Language Models", NeurIPS 2022.
Cited in: part19
Li et al., TrOCR: Transformer-based Optical Character Recognition with Pre-trained Models,
Cited in: part20
Schulman, Levine, Moritz, Jordan & Abbeel, "Trust Region Policy Optimization", ICML 2015.
Cited in: part12
Kohavi, R., Tang, D., Xu, Y. "Trustworthy Online Controlled Experiments: A Practical Guide to A/B Testing." Cambridge University Press, 2020.
Cited in: part01, part17
Tsai, Y.-H. H. et al. (2019). Transformer Dissection: A Unified Understanding of Transformer's
Cited in: part05*
Google (2019). Understanding searches better than ever before.
Cited in: part05
Zou, A. et al. Universal and Transferable Adversarial Attacks on Aligned Language Models. 2023.
Cited in: part15
Press, O. & Wolf, L. (2017). Using the Output Embedding to Improve Language Models.
Cited in: part05
Liu, Li, Wu, Lee. "Visual Instruction Tuning", NeurIPS 2023.
Cited in: part19
Weaviate, Milvus and Pinecone documentation pages on filtered vector search (titles:
Cited in: part13
Roman Solovyev, Weimin Wang and Tatiana Gabruseva. "Weighted Boxes Fusion: Ensembling boxes from different object detection models." Image and Vision Computing, 2021.
Cited in: part11
What staff-level signal looks like, this book, for the
Cited in: part20
Williams, S., Waterman, A., Patterson, D. [Roofline: An Insightful Visual Performance Model for
Cited in: part14*
Williams, "Simple statistical gradient-following algorithms for connectionist
Cited in: part12
Xiao, Tian, Chen, Han, Lewis. "Efficient Streaming Language Models with Attention
Cited in: part19
Peng, B. et al. (2023). YaRN: Efficient Context Window Extension of Large Language Models.
Cited in: part05
Yi et al., ["Sampling-Bias-Corrected Neural Modeling for Large Corpus Item
Cited in: part13
Rajbhandari, S. et al. ZeRO: Memory Optimizations Toward Training Trillion Parameter Models.
Cited in: part14
Zhang, Bengio, Hardt, Recht, Vinyals. "Understanding deep learning requires
Cited in: part19
Zhao, Kumar, Levine & Finn, "Learning Fine-Grained Bimanual Manipulation with Low-Cost
Cited in: part12
Zhong, Y. et al. DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large
Cited in: part14*