PandaLLM
latest
  • Quick Start
  • Train your LLM
  • Inference
  • Deploy your LLM
  • Pipeline Parallelism
  • 快速开始
  • 训练您的大模型(LLM)
  • 部署你的大模型( LLM )
  • 流水线并行
PandaLLM
  • Welcome to PandaLLMOps! 欢迎来到 PandLLMOps !
  • Edit on GitHub

Welcome to PandaLLMOps! 欢迎来到 PandLLMOps !

PandaLLMOps is an open-sourced Python framework for large language models (LLMs) training and inference.

PandaLLMOps 是一个用于大型语言模型(LLMs)训练和推理的开源Python框架。

Note

This tutorial is under active development. 该教程持续更新中

Contents

  • Quick Start
    • Installation
    • Quick Deployment
    • Quick Train
  • Train your LLM
    • Preliminary about Hydra
    • Data preparation
    • Models
    • Optimization
      • General settings
      • Low-rank adaptation (LoRA)
      • Quantization-aware training
  • Inference
    • HuggingFace Transformers’ Model Parallel
    • Deepspeed Inference
    • Tensor Parallel
  • Deploy your LLM
    • Coming Soon …
  • Pipeline Parallelism
    • Preliminary
    • Core Code Snippets
      • Model Implementation
      • Model initialization
      • Data Fetch Design of DeepSpeed and CPU Memory Reduction
    • Know Problems and Possible Solutions
      • BF16 Support
      • Torch Compile
    • Reference & Acknowledgement
  • 快速开始
    • 安装
    • 快速部署
    • 快速训练
  • 训练您的大模型(LLM)
    • 关于 Hydra 的初步介绍
    • 数据准备
    • 模型
    • 优化
      • 通用设置
      • 低秩适应 (LoRA)
      • Quantization-aware training
  • 部署你的大模型( LLM )
    • Coming Soon …
  • 流水线并行
    • 初步
    • 核心代码片段
      • 模型实现
      • 模型参数初始化
      • DeepSpeed和CPU内存减少的数据获取设计
    • 已知问题和可能的解决方案
      • BF16支持
      • Torch Compile
    • 参考和致谢
Next

© Copyright 2021, Graziella. Revision df709f81.

Built with Sphinx using a theme provided by Read the Docs.