8 Advanced parallelization - Deep Learning with JAX

Using easy-to-revise parallelism with xmap() · Compiling and automatically partitioning functions with pjit() · Using tensor sharding to achieve parallelization with XLA · Running code in multi-host configurations

Breaking Up with NumPy: Why JAX is Your New Favorite Tool

Tutorial 6 (JAX): Transformers and Multi-Head Attention — UvA DL

Applying sequence and parallel graph splits on a data-parallel

Energies, Free Full-Text

Introducing PyTorch Fully Sharded Data Parallel (FSDP) API

Tutorial 2 (JAX): Introduction to JAX+Flax — UvA DL Notebooks v1.2

Hardware for Deep Learning. Part 4: ASIC

Breaking Up with NumPy: Why JAX is Your New Favorite Tool

Why You Should (or Shouldn't) be Using Google's JAX in 2023

JAX: accelerated machine learning research via composable function