knox / LLaMA-Factory-13.sh
0 likes
0 forks
1 files
Last active
| 1 | cd docker/docker-npu/ |
| 2 | docker compose up -d |
| 3 | docker compose exec llamafactory bash |
knox / LLaMA-Factory-12.sh
0 likes
0 forks
1 files
Last active
| 1 | cd docker/docker-cuda/ |
| 2 | docker compose up -d |
| 3 | docker compose exec llamafactory bash |
knox / LLaMA-Factory-11.sh
0 likes
0 forks
1 files
Last active
| 1 | llamafactory-cli webui |
knox / LLaMA-Factory-10.sh
0 likes
0 forks
1 files
Last active
| 1 | llamafactory-cli train examples/train_lora/llama3_lora_sft.yaml |
| 2 | llamafactory-cli chat examples/inference/llama3_lora_sft.yaml |
| 3 | llamafactory-cli export examples/merge_lora/llama3_lora_sft.yaml |
knox / LLaMA-Factory-9.md
0 likes
0 forks
1 files
Last active
| Requirement | Minimum | Recommend |
|---|---|---|
| CANN | 8.0.RC1 | 8.0.RC1 |
| torch | 2.1.0 | 2.1.0 |
| torch-npu | 2.1.0 | 2.1.0.post3 |
| deepspeed | 0.13.2 | 0.13.2 |
knox / LLaMA-Factory-8.sh
0 likes
0 forks
1 files
Last active
| 1 | # replace the url according to your CANN version and devices |
| 2 | # install CANN Toolkit |
| 3 | wget https://ascend-repo.obs.cn-east-2.myhuaweicloud.com/Milan-ASL/Milan-ASL%20V100R001C17SPC701/Ascend-cann-toolkit_8.0.RC1.alpha001_linux-"$(uname -i)".run |
| 4 | bash Ascend-cann-toolkit_8.0.RC1.alpha001_linux-"$(uname -i)".run --install |
| 5 | |
| 6 | # install CANN Kernels |
| 7 | wget https://ascend-repo.obs.cn-east-2.myhuaweicloud.com/Milan-ASL/Milan-ASL%20V100R001C17SPC701/Ascend-cann-kernels-910b_8.0.RC1.alpha001_linux.run |
| 8 | bash Ascend-cann-kernels-910b_8.0.RC1.alpha001_linux.run --install |
| 9 | |
| 10 | # set env variables |
knox / LLaMA-Factory-7.sh
0 likes
0 forks
1 files
Last active
| 1 | pip install https://github.com/jllllll/bitsandbytes-windows-webui/releases/download/wheels/bitsandbytes-0.41.2.post2-py3-none-win_amd64.whl |
knox / LLaMA-Factory-6.sh
0 likes
0 forks
1 files
Last active
| 1 | git clone --depth 1 https://github.com/hiyouga/LLaMA-Factory.git |
| 2 | cd LLaMA-Factory |
| 3 | pip install -e ".[torch,metrics]" |
knox / LLaMA-Factory-5.md
0 likes
0 forks
1 files
Last active
| Method | Bits | 7B | 13B | 30B | 70B | 110B | 8x7B | 8x22B |
|---|---|---|---|---|---|---|---|---|
| Full | AMP | 120GB | 240GB | 600GB | 1200GB | 2000GB | 900GB | 2400GB |
| Full | 16 | 60GB | 120GB | 300GB | 600GB | 900GB | 400GB | 1200GB |
| Freeze | 16 | 20GB | 40GB | 80GB | 200GB | 360GB | 160GB | 400GB |
| LoRA/GaLore/BAdam | 16 | 16GB | 32GB | 64GB | 160GB | 240GB | 120GB | 320GB |
| QLoRA | 8 | 10GB | 20GB | 40GB | 80GB | 140GB | 60GB | 160GB |
| QLoRA | 4 | 6GB | 12GB | 24GB | 48GB | 72GB | 30GB | 96GB |
| QLoRA | 2 | 4GB | 8GB | 16GB | 24GB | 48GB | 18GB | 48GB |