DeepSpeed Llama-2-7B Reward Model Finetuning

Model description

LLaMA2 is a large language model released by Meta in 2023, with parameters ranging from 7B to 70B. Compared to LLaMA, the training corpus of LLaMA2 is 40% longer, and the context length has been upgraded from 2048 to 4096, allowing for understanding and generating longer texts.

Step 1: Installation

cd deepsparkhub/nlp/llm/llama2-7b_reward_sft/deepspeed
pip install -r requirements.txt
pip install -e .

Step 2: Preparing datasets

Prepare datasets and pretrained model weight

# Install lfs
wget https://packagecloud.io/github/git-lfs/packages/el/7/git-lfs-2.13.2-1.el7.x86_64.rpm/download -O lfs.rpm
rpm -ivh lfs.rpm
rm -y lfs.rpm
git lfs install

# Get Dahoas/rm-static
mkdir -p datasets/Dahoas && cd datasets/Dahoas
git clone https://huggingface.co/datasets/Dahoas/rm-static

# Get Llama-2-7b-hf pretraining weights 
- [Huggingface](https://huggingface.co/meta-llama/Llama-2-7b-hf)
- [aifasthub](https://aifasthub.com/models/NousResearch/Llama-2-7b-hf)
 
mv Llama-2-7b-hf/ datasets/

Step 3: Training

Fine-tuning

cd training/step2_reward_model_finetuning/training_scripts/llama2/
bash ./run_llama2_7b.sh

Results

GPUs	Epochs	FPS	ACC
BI-V100 x8	1	AvgSamplesPerSec: 1.948	0.6821

Reference

[DeepSpeed-Chat] (https://github.com/microsoft/DeepSpeedExamples/tree/master/applications/DeepSpeed-Chat)

DeepSpark / DeepSparkHub

DeepSpeed Llama-2-7B Reward Model Finetuning

Model description

Step 1: Installation

Step 2: Preparing datasets

Step 3: Training

Results

Reference

简介

发行版 (6)

贡献者

近期动态

DeepSpark / DeepSparkHub .gitee-modal { width: 500px !important; }

DeepSpeed Llama-2-7B Reward Model Finetuning

Model description

Step 1: Installation

Step 2: Preparing datasets

Step 3: Training

Results

Reference

简介

发行版 (6)

开源评估指数源自 OSS-Compass 评估体系，评估体系围绕以下三个维度对项目展开评估：

贡献者

近期动态

搜索帮助

DeepSpark / DeepSparkHub