← Back to Directory
Reiner Pope

Reiner Pope

Builder

Sydney

@reinerpope

Staff Research Engineer at Google DeepMind. Key contributor to PaLM scaling and infrastructure for training trillion-parameter models.

Skills

distributed trainingTPUJAXlarge-scale MLsystems

Looking For

collaborator

Research Foundations (2)

Attention Is All You Need

NeurIPS 201720176,510 citations

Used by: PaLM Training Infrastructure

Aggregated Residual Transformations for Deep Neural Networks

CVPR 2017201710,500 citations

Used by: PaLM Training Infrastructure

Projects (1)

PaLM Training Infrastructure

Scaling infrastructure behind Google's PaLM model. Achieved efficient training of 540B parameter models across 6144 TPU v4 chips.

Generative AINLP

Built on research:

paperAttention Is All You Need(2017)
paperAggregated Residual Transformations for Deep Neural Networks(2017)

AI Suggested Researchers

Researchers whose work may be relevant to your projects (auto-detected)

Hannaneh Hajishirzi

Hannaneh Hajishirzi

University of Washington / Allen AI

Shared domains: Generative AI, NLP
69% matchAI suggested
Luke Zettlemoyer

Luke Zettlemoyer

University of Washington / Meta AI

Shared domains: Generative AI, NLP
65% matchAI suggested
Kyunghyun Cho

Kyunghyun Cho

New York University / Genentech

Shared domains: Generative AI, NLP
61% matchAI suggested
Aaron Courville

Aaron Courville

Mila / Université de Montréal

Shared domains: Generative AI, NLP
61% matchAI suggested

AI Suggested Papers

Papers that may have inspired your projects (auto-detected by domain & keyword analysis)

GitHub