Apache Submarine是一套開源的端到端機器學習平台，支援最主流的兩個機器學習框架TensorFlow、PyTorch，並允許資料科學家可以選擇在Kubernetes（K8s）或Apache Hadoop YARN上運行分散式機器學習工作負載（ML workload），然而機器學習軟體從開發到部署在生產環境中，其過程須經過反覆且耗時的實驗。
本論文研究提出一個方法，將Apache Submarine原本的機器學習工作負載分割成顆粒度較細的任務，並且讓用戶能編排多個任務組成機器學習管線。管線中的任務執行完成所產生的輸出檔案稱為構件（Artifact），若執行某一機器學習管線，且在某個任務失敗而停止，重新執行的管線可存取預先儲存在構件儲存體（Artifact store）中的構件，直接從失敗的任務開始繼續執行。另一種用例，在不同的管線中可以分享構件，而不用執行管線中已完成的任務。本論文研究分析對Submarine提供具有構件儲存體之細粒度機器學習管線，對機器學習系統效能層面所帶來的影響。

Apache Submarine is an open source end-to-end machine learning platform that supports the two most dominant machine learning frameworks, TensorFlow and PyTorch, and enables data scientists to run distributed machine learning workloads on either Kubernetes (K8s) or Apache Hadoop YARN. However, the process of developing machine learning software from development to deployment in production requires iterative and time-consuming experiments.
In this study, we propose an approach that splits the original machine learning workload of Apache Submarine into smaller tasks and enables users to schedule multiple tasks to form a machine learning pipeline. If a machine learning pipeline stops when a task fails, the re-executed pipeline can access the artifacts stored in the artifact store and resume directly from the failed task, or it can share artifacts in different pipelines without executing the completed tasks in the pipeline. This study investigates the impact of providing a fine-grained machine learning pipeline with an artifact store in Submarine on the system-wide performance level.

[1] Apache Submarine. [Online]. Available: https://submarine.apache.org
[2] M. Abadi et al., "TensorFlow: A System for Large-Scale Machine Learning," presented at the Proceedings of the 12th USENIX Conference on Operating Systems Design and Implementation, 2016.
[3] A. Paszke et al., "PyTorch: An Imperative Style, High-Performance Deep Learning Library," presented at the Proceedings of the 33rd International Conference on Neural Information Processing Systems, 2019.
[4] Kubernetes. [Online]. Available: https://kubernetes.io
[5] Apache Hadoop. [Online]. Available: https://hadoop.apache.org
[6] Algorithmia, "2020 State of Enterprise Machine Learning." [Online]. Available: https://algorithmia.com/state-of-ml
[7] D. Sculley et al., "Hidden Technical Debt in Machine Learning Systems," presented at the Proceedings of the 28th International Conference on Neural Information Processing Systems, 2015.
[8] MLOps: Continuous Delivery and Automation Pipelines in Machine Learning. [Online]. Available: https://cloud.google.com/architecture/mlops-continuous-delivery-and-automation-pipelines-in-machine-learning
[9] Jupyter Notebook. [Online]. Available: https://jupyter.org
[10] Google Colaboratory. [Online]. Available: https://colab.research.google.com
[11] Apache Software Foundation. [Online]. Available: https://www.apache.org
[12] Amazon Elastic Kubernetes Service. [Online]. Available: https://aws.amazon.com/eks
[13] Google Kubernetes Engine. [Online]. Available: https://cloud.google.com/kubernetes-engine
[14] Azure Kubernetes Service. [Online]. Available: https://azure.microsoft.com/en-us/services/kubernetes-service
[15] Docker. [Online]. Available: https://www.docker.com
[16] Kubeflow. [Online]. Available: https://www.kubeflow.org
[17] Operator Pattern. [Online]. Available: https://kubernetes.io/docs/concepts/extend-kubernetes/operator
[18] TonY. [Online]. Available: https://github.com/linkedin/TonY
[19] D. Xin, H. Miao, A. Parameswaran, and N. Polyzotis, "Production Machine Learning Pipelines: Empirical Analysis and Optimization Opportunities," presented at the Proceedings of the 2021 International Conference on Management of Data, 2021.
[20] Argo Workflows. [Online]. Available: https://argoproj.github.io/argo-workflows
[21] Cloud Native Computing Foundation (CNCF). [Online]. Available: https://www.cncf.io
[22] MinIO. [Online]. Available: https://min.io
[23] Amazon S3. [Online]. Available: https://aws.amazon.com/s3
[24] MySQL. [Online]. Available: https://www.mysql.com
[25] Classify Text with BERT. [Online]. Available: https://www.tensorflow.org/text/tutorials/classify_text_with_bert
[26] Open-Sourcing Metaflow, A Human-Centric Framework for Data Science. [Online]. Available: https://netflixtechblog.com/open-sourcing-metaflow-a-human-centric-framework-for-data-science-fa72e04a5d9
[27] J. Ni, J. Li, and J. McAuley, "Justifying Recommendations using Distantly-Labeled Reviews and Fine-Grained Aspects," presented at the Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, 2019.
[28] Hyper-Scale Machine Learning with MinIO and TensorFlow. [Online]. Available: https://blog.min.io/hyper-scale-machine-learning-with-minio-and-tensorflow
[29] A. L. Maas, R. E. Daly, P. T. Pham, D. Huang, A. Y. Ng, and C. Potts, "Learning Word Vectors for Sentiment Analysis," presented at the Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, 2011.
[30] T.-Y. Lin et al., "Microsoft COCO: Common Objects in Context," presented at the Proceedings of the 13th European Conference on Computer Vision, 2014.
[31] Kubeflow Pipelines. [Online]. Available: https://www.kubeflow.org/docs/components/pipelines/overview/pipelines-overview
[32] D. Baylor et al., "TFX: A TensorFlow-Based Production-Scale Machine Learning Platform," presented at the Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2017.
[33] ML Metadata. [Online]. Available: https://github.com/google/ml-metadata
[34] Determined AI. [Online]. Available: https://www.determined.ai
[35] Flyte. [Online]. Available: https://flyte.org
[36] Amazon SageMaker. [Online]. Available: https://aws.amazon.com/tw/sagemaker
[37] Azure Machine Learning. [Online]. Available: https://azure.microsoft.com/zh-tw/services/machine-learning
[38] Google Vertex AI. [Online]. Available: https://cloud.google.com/vertex-ai
[39] T. Chen and C. Guestrin, "XGBoost: A Scalable Tree Boosting System," presented at the Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2016.
[40] KFServing. [Online]. Available: https://www.kubeflow.org/docs/components/kfserving
[41] Apache Airflow. [Online]. Available: https://airflow.apache.org
[42] Apache Beam. [Online]. Available: https://beam.apache.org