3D Parallelism

3D Parallelism　の参考になる横田先生の資料。

https://fugaku100kei.jp/events/colloquium/doc/20240130_10th_colloquium_slides.pdf

DeepSpeed

https://www.deepspeed.ai/assets/files/DeepSpeed_Overview_Japanese_2023Jun7th.pdf