knox / LLaMA-Factory-8.sh
0 likes
0 forks
1 files
Last active
| 1 | # replace the url according to your CANN version and devices |
| 2 | # install CANN Toolkit |
| 3 | wget https://ascend-repo.obs.cn-east-2.myhuaweicloud.com/Milan-ASL/Milan-ASL%20V100R001C17SPC701/Ascend-cann-toolkit_8.0.RC1.alpha001_linux-"$(uname -i)".run |
| 4 | bash Ascend-cann-toolkit_8.0.RC1.alpha001_linux-"$(uname -i)".run --install |
| 5 | |
| 6 | # install CANN Kernels |
| 7 | wget https://ascend-repo.obs.cn-east-2.myhuaweicloud.com/Milan-ASL/Milan-ASL%20V100R001C17SPC701/Ascend-cann-kernels-910b_8.0.RC1.alpha001_linux.run |
| 8 | bash Ascend-cann-kernels-910b_8.0.RC1.alpha001_linux.run --install |
| 9 | |
| 10 | # set env variables |
knox / LLaMA-Factory-7.sh
0 likes
0 forks
1 files
Last active
| 1 | pip install https://github.com/jllllll/bitsandbytes-windows-webui/releases/download/wheels/bitsandbytes-0.41.2.post2-py3-none-win_amd64.whl |
knox / LLaMA-Factory-6.sh
0 likes
0 forks
1 files
Last active
| 1 | git clone --depth 1 https://github.com/hiyouga/LLaMA-Factory.git |
| 2 | cd LLaMA-Factory |
| 3 | pip install -e ".[torch,metrics]" |
knox / LLaMA-Factory-5.md
0 likes
0 forks
1 files
Last active
| Method | Bits | 7B | 13B | 30B | 70B | 110B | 8x7B | 8x22B |
|---|---|---|---|---|---|---|---|---|
| Full | AMP | 120GB | 240GB | 600GB | 1200GB | 2000GB | 900GB | 2400GB |
| Full | 16 | 60GB | 120GB | 300GB | 600GB | 900GB | 400GB | 1200GB |
| Freeze | 16 | 20GB | 40GB | 80GB | 200GB | 360GB | 160GB | 400GB |
| LoRA/GaLore/BAdam | 16 | 16GB | 32GB | 64GB | 160GB | 240GB | 120GB | 320GB |
| QLoRA | 8 | 10GB | 20GB | 40GB | 80GB | 140GB | 60GB | 160GB |
| QLoRA | 4 | 6GB | 12GB | 24GB | 48GB | 72GB | 30GB | 96GB |
| QLoRA | 2 | 4GB | 8GB | 16GB | 24GB | 48GB | 18GB | 48GB |
knox / LLaMA-Factory-4.md
0 likes
0 forks
1 files
Last active
| Mandatory | Minimum | Recommend |
|---|---|---|
| python | 3.8 | 3.11 |
| torch | 1.13.1 | 2.4.0 |
| transformers | 4.41.2 | 4.43.4 |
| datasets | 2.16.0 | 2.20.0 |
| accelerate | 0.30.1 | 0.32.0 |
| peft | 0.11.1 | 0.12.0 |
| trl | 0.8.6 | 0.9.6 |
knox / LLaMA-Factory-3.sh
0 likes
0 forks
1 files
Last active
| 1 | pip install --upgrade huggingface_hub |
| 2 | huggingface-cli login |
knox / LLaMA-Factory-2.md
0 likes
0 forks
1 files
Last active
| Approach | Full-tuning | Freeze-tuning | LoRA | QLoRA |
|---|---|---|---|---|
| Pre-Training | ✅ | ✅ | ✅ | ✅ |
| Supervised Fine-Tuning | ✅ | ✅ | ✅ | ✅ |
| Reward Modeling | ✅ | ✅ | ✅ | ✅ |
| PPO Training | ✅ | ✅ | ✅ | ✅ |
| DPO Training | ✅ | ✅ | ✅ | ✅ |
| KTO Training | ✅ | ✅ | ✅ | ✅ |
| ORPO Training | ✅ | ✅ | ✅ | ✅ |
| SimPO Training | ✅ | ✅ | ✅ | ✅ |
knox / LLaMA-Factory-1.md
0 likes
0 forks
1 files
Last active
| Model | Model size | Template |
|---|---|---|
| Baichuan 2 | 7B/13B | baichuan2 |
| BLOOM/BLOOMZ | 560M/1.1B/1.7B/3B/7.1B/176B | - |
| ChatGLM3 | 6B | chatglm3 |
| Command R | 35B/104B | cohere |
| DeepSeek (Code/MoE) | 7B/16B/67B/236B | deepseek |
| Falcon | 7B/11B/40B/180B | falcon |
| Gemma/Gemma 2/CodeGemma | 2B/7B/9B/27B | gemma |
| GLM-4 | 9B | glm4 |
knox / Deep-Live-Cam-27.sh
0 likes
0 forks
1 files
Last active
| 1 | sudo apt-get -y install cuda-toolkit-11-8 |
knox / Deep-Live-Cam-26.sh
0 likes
0 forks
1 files
Last active
| 1 | /usr/local/cuda/bin/nvcc --version |