Quick Start

PandaLLM Github

We will take deployment and training of Panda-13B as an example.

Installation

  1. Download our code from github

$ git clone https://github.com/dandelionsllm/pandallm

  1. Install the requirements in a new environment

$ conda create -n pandallm python=3.10
$ conda activate pandallm
(pandallm) $ pip install -r requirements.txt
(pandallm) $ mkdir pretrained_model

Quick Deployment

  1. Download LlaMA-13B from Huggingface.

  2. Download our model form Huggingface. Since the model file is too large for git clone, you may manually download the model files from here.

(pandallm) $ mkdir delta-models
(pandallm) $ cd delta-models
(pandallm) $ git clone https://huggingface.co/chitanda/llama-panda-13b-zh-wudao-chat-delta

  1. Move the downloaded files to the corresponding directory.

(pandallm) $ cd ..
(pandallm) $ mv delta-models/ ./

  1. Convert "delta-model" to a pretrained model. Replace ${PATH_TO_YOUR_MODEL} with your desired model path, where your model will be saved there.

(pandallm) $ python apply_delta.py --base_model ${PATH_TO_YOUR_MODEL} --target_model ./pretrained_model/panda-13B --delta_model ./delta-models/llama-panda-13b-zh-wudao-chat-delta/checkpoint-3000-delta

  1. Run the following command to deploy the chatbot.

(pandallm) $ python run_chat.py --model_path ./pretrained_model/panda-13B --query "write a peom"

Quick Train

Before you can directly train the model with the following commands, make sure you have finish the installation.

  1. Prepare the training data. You can download the training data from here. Please put the data folders at ./dataset.

  2. Run the following command to train the model:

(pandallm) $ PAD_TOKEN="</s>" deepspeed --include localhost:0,1,2,3,4,5,6,7  trainer_base_ds_mul.py -cp conf/llama/zh/ -cn llama_13b_zh_instruct_sft_combine_v1_0_ds

If you have less than \(8\) GPUs, you can change the --include parameter to the GPUs you have, e.g. "--include localhost:0,1,2,3" if you have \(4\) GPUS on one server.