Joohyung Yun
Seminars & paper readings › Paper reading

Improving Language Models by Retrieving from Trillions of Tokens (RETRO)

Borgeaud et al., ICML 2022 · presented by Joohyung Yun · opens with DiskANN (Subramanya et al., NeurIPS 2019)
Paper readingDSLab NLP seminar · POSTECH · 2024.10.07

Token-level RAG built up on one diagram: why per-token retrieval is slow, then chunked retrieval, chunked cross-attention, and frozen BERT embedders, each introduced as the fix to a specific problem. Opens with a seven-slide recap of DiskANN.

slides.pdfpaperdiskann paper
Slide 1
Slide 2
Slide 3
Slide 4
Slide 5
Slide 6
Slide 7
Slide 8
Slide 9
Slide 10
Slide 11
Slide 12
Slide 13
Slide 14
Slide 15
Slide 16
Slide 17
Slide 18
Slide 19
Slide 20
Slide 21
Slide 22
Slide 23
Slide 24
Slide 25
Slide 26
Slide 27
Slide 28
Slide 29
Slide 30
Slide 31
Slide 32
Slide 33
Slide 34
Slide 35
1 / 35 ← → keys · swipe · click a thumbnail · → also plays each slide's build steps

← all seminars & paper readings