> ML_LITERATURE // ZHENG-2022-ALPA-AUTOMATING-INTER-AND-INTRA-OPERATOR-PARALLELISM_v1.0
Alpa: Automating Inter- and Intra-Operator Parallelism for Distributed Deep Learning
Lianmin Zheng, Hao Zhang, Jiaxuan You, Chunan Shi, Zhihao Jia, Yangqing Jia, Ion Stoica, Joseph E. Gonzalez · USENIX Symposium on Operating Systems Design and Implementation (OSDI) (2022)
systems2022industry-standardnotAssessed
Principal Contribution
Automated compilation framework discovering optimal combinations of intra-operator and inter-operator pipeline parallelism using integer linear programming.
Operational Relevance
Directly guides deployment choices and architecture selection for task-text-generation.
Assumptions
- Standard empirical regularity and statistical stability hold across evaluation domains
Limitations
- Performance characteristics depend on domain distribution and compute allocation parameters
Connected Algorithms, Architectures & Tools
Related Algorithms:
Related Architectures:
Implementing Libraries: