Kimirock/PaddleNLP
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
6,040 Commits 70 Branches 54 Tags
develop
Commit Graph

6040 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Liujie0926
3f87dac9b7 Delete .github/workflows/distribute-v100.yml 2026-05-23 10:54:28 +08:00
Liujie0926
be4f19628f Delete .github/workflows/distribute-a100.yml 2026-05-23 10:53:53 +08:00
xingmingyyj
587a5cdf97 fix load ema and model_meta path (#11203) 2025-12-17 17:19:22 +08:00
liufengwei0103
32b9a2e257 decouple sharding io and non zcc (#11201) 2025-12-16 19:12:37 +08:00
liufengwei0103
51f73a5104 disable only sharding opt in save stage (#11200) 2025-12-16 10:36:29 +08:00
baoqiwen
2ac85997c2 Fix loss for Align reduce precision with PyTorch 2.9.1. (#11199) 2025-12-15 15:45:57 +08:00
xingmingyyj
dcbdeaa819 change_save_meta_in_zcc_crossponding_to_paddle and fix flags (#11198) 
* change_save_meta_in_zcc_crossponding_to_paddle

* delete

* fix

* change_save_meta_in_zcc_crossponding_to_paddle and fix args

* fix deadcode

---------

Co-authored-by: liufengwei02 <liufengwei02@baidu.com>
 2025-12-13 17:03:13 +08:00
baoqiwen
7ab35ce94e Align reduce precision with PyTorch 2.9.1, disable precision checks. (#11195) 2025-12-12 17:16:21 +08:00
xingmingyyj
bc08aeec91 ban ckpt_converter test (#11194) 2025-12-12 14:15:15 +08:00
liufengwei0103
f8d0829e02 fix zcc error in eb5 (#11193) 2025-12-12 11:08:47 +08:00
xingmingyyj
b0276d735a 【FlexCheckpoint】adapter flex checkpoint (#11188) 
* adapter flex checkpoint

* opt bf16 save

* fix multi moe_sharding_group load safetensors

* fix
 2025-12-08 23:07:15 +08:00
liufengwei0103
fa007c8f85 cp add_fc_into_zcc to nlp (#11187) 2025-12-05 21:56:54 +08:00
xingmingyyj
c8f141476c add config memory_growth_threshold for save hf (#11183) 2025-11-28 19:37:10 +08:00
xingmingyyj
e4d28de5a8 【FlexCheckpoint】support save hf (#11180) 
* support save hf

* fix
 2025-11-26 23:28:16 +08:00
sneaxiy
53b85cbb34 fix EMA (#11177) 2025-11-18 12:10:34 +08:00
zhengshengning
9904cfc78e fix_int64_to_int (#11175) 2025-11-14 14:03:12 +08:00
sneaxiy
c07e0c8d3a Add Non ZCC EMA callback (#11173) 2025-11-13 10:12:55 +08:00
Wennie396
7c82a262f2 fix opt chunk offload (#11167) 2025-11-12 21:06:30 +08:00
Weiguo Zhu
61856820c3 fix model save for moe_sharding (#11162) 2025-11-12 18:50:34 +08:00
Zhuoqun Li
4eef1079f5 [Embedding] Expand training and shortgpt_prune code to support more model (#11168) 
* final update

* expand training and shortgpt_prune code to support more model
 2025-11-12 18:49:39 +08:00
zhengshengning
d38f3d696e big tensor: tokens_unzip_gather (#11172) 2025-11-12 11:18:15 +08:00
wanghuancoder
e68b135bfa fix eb5 big tensor bug (#11169) 
* fix eb5 big tensor bug

* refine

* refine
 2025-11-11 15:31:33 +08:00
Shing
6ac0402875 BugFix: qwen pp cannot send bool when using fuse loss (#11156) 
* feat(model): add support for Qwen model in RL PipelineParallel

* feat(model): add support for QwenMoe model in RL PipelineParallel

* feat(model): add support for QwenMoe model in RL PipelineParallel

* BugFix: qwen model sequence parallel can not get batch size

* BugFix: qwen pp cannot send bool when using fuse loss
 2025-11-07 20:39:41 +08:00
sneaxiy
6ede865d0f fix zcc ema GPU alloc bug (#11165) 2025-11-07 17:03:52 +08:00
Siyu Zhao
29f97bb461 TARE提交 (#11151) 
Co-authored-by: zsy <zsy03260058@163.com>
 2025-11-05 14:04:57 +08:00
a31413510
10014d8f0a Update run_finetune.py (#11159) 2025-11-03 21:51:38 +08:00
hsz06
e73341efa4 [codestyle] Modular convert tools (#11129) 
* new file:   paddlenlp/transformers/convert/doc/README.md
	new file:   paddlenlp/transformers/convert/doc/llama_analysis.md
	new file:   paddlenlp/transformers/convert/doc/process.png
	new file:   paddlenlp/transformers/convert/doc/qwen2_analysis.md
	new file:   "paddlenlp/transformers/convert/doc/\351\241\271\347\233\256\346\212\245\345\221\212\357\274\232\351\243\236\346\241\250PaddleNLP-\345\211\215\346\262\277\346\250\241\345\236\213\346\250\241\345\235\227\345\214\226\350\256\276\350\256\241.md"
	new file:   paddlenlp/transformers/convert/main.py
	new file:   paddlenlp/transformers/convert/test_ability/test_base/test_modeling.py
	new file:   paddlenlp/transformers/convert/test_ability/test_paralle/compare_torch_with_paddle.py
	new file:   paddlenlp/transformers/convert/until/collect_import_modeling.py
	new file:   paddlenlp/transformers/convert/until/rename_identifiers.py
	new file:   paddlenlp/transformers/convert/until/rewrite_child_classes.py
	modified:   paddlenlp/transformers/qwen2/configuration.py
	new file:   paddlenlp/transformers/qwen2/modeling_qwen2.py
	new file:   paddlenlp/transformers/qwen2/modular_qwen2.py

* modified:   paddlenlp/transformers/qwen2/modeling_qwen2.py
	deleted:    paddlenlp/transformers/convert/doc/README.md
	deleted:    paddlenlp/transformers/convert/doc/llama_analysis.md
	deleted:    paddlenlp/transformers/convert/doc/process.png
	deleted:    paddlenlp/transformers/convert/doc/qwen2_analysis.md
	deleted:    "paddlenlp/transformers/convert/doc/\351\241\271\347\233\256\346\212\245\345\221\212\357\274\232\351\243\236\346\241\250PaddleNLP-\345\211\215\346\262\277\346\250\241\345\236\213\346\250\241\345\235\227\345\214\226\350\256\276\350\256\241.md"
	deleted:    paddlenlp/transformers/convert/main.py
	deleted:    paddlenlp/transformers/convert/test_ability/test_base/test_modeling.py
	deleted:    paddlenlp/transformers/convert/test_ability/test_paralle/compare_torch_with_paddle.py
	deleted:    paddlenlp/transformers/convert/until/collect_import_modeling.py
	deleted:    paddlenlp/transformers/convert/until/rename_identifiers.py
	deleted:    paddlenlp/transformers/convert/until/rewrite_child_classes.py
	deleted:    paddlenlp/transformers/qwen2/__init__.py
	deleted:    paddlenlp/transformers/qwen2/configuration.py
	deleted:    paddlenlp/transformers/qwen2/modeling.py
	deleted:    paddlenlp/transformers/qwen2/modeling_pp.py
	deleted:    paddlenlp/transformers/qwen2/modeling_qwen2.py
	deleted:    paddlenlp/transformers/qwen2/modular_qwen2.py
	deleted:    paddlenlp/transformers/qwen2/tokenizer.py
	deleted:    paddlenlp/transformers/qwen2/tokenizer_fast.py
	scripts/codestyle/convert/

* renamed:    paddlenlp/transformers/convert/doc/README.md -> scripts/codestyle/convert/doc/README.md
	renamed:    paddlenlp/transformers/convert/doc/llama_analysis.md -> scripts/codestyle/convert/doc/llama_analysis.md
	renamed:    paddlenlp/transformers/convert/doc/process.png -> scripts/codestyle/convert/doc/process.png
	renamed:    paddlenlp/transformers/convert/doc/qwen2_analysis.md -> scripts/codestyle/convert/doc/qwen2_analysis.md
	renamed:    "paddlenlp/transformers/convert/doc/\351\241\271\347\233\256\346\212\245\345\221\212\357\274\232\351\243\236\346\241\250PaddleNLP-\345\211\215\346\262\277\346\250\241\345\236\213\346\250\241\345\235\227\345\214\226\350\256\276\350\256\241.md" -> "scripts/codestyle/convert/doc/\351\241\271\347\233\256\346\212\245\345\221\212\357\274\232\351\243\236\346\241\250PaddleNLP-\345\211\215\346\262\277\346\250\241\345\236\213\346\250\241\345\235\227\345\214\226\350\256\276\350\256\241.md"
	renamed:    paddlenlp/transformers/convert/main.py -> scripts/codestyle/convert/main.py
	renamed:    paddlenlp/transformers/convert/test_ability/test_base/test_modeling.py -> scripts/codestyle/convert/test_ability/test_base/test_modeling.py
	renamed:    paddlenlp/transformers/convert/test_ability/test_paralle/compare_torch_with_paddle.py -> scripts/codestyle/convert/test_ability/test_paralle/compare_torch_with_paddle.py
	renamed:    paddlenlp/transformers/convert/until/collect_import_modeling.py -> scripts/codestyle/convert/until/collect_import_modeling.py
	renamed:    paddlenlp/transformers/convert/until/rename_identifiers.py -> scripts/codestyle/convert/until/rename_identifiers.py
	renamed:    paddlenlp/transformers/convert/until/rewrite_child_classes.py -> scripts/codestyle/convert/until/rewrite_child_classes.py
	renamed:    paddlenlp/transformers/qwen2/modeling_qwen2.py -> scripts/codestyle/qwen2/modeling_qwen2.py
	renamed:    paddlenlp/transformers/qwen2/modular_qwen2.py -> scripts/codestyle/qwen2/modular_qwen2.py

* modified:   scripts/codestyle/qwen2/modeling_qwen2.py

* modified:   paddlenlp/transformers/qwen2/configuration.py
	new file:   scripts/codestyle/convert/example/qwen2/__init__.py
	new file:   scripts/codestyle/convert/example/qwen2/configuration.py
	new file:   scripts/codestyle/convert/example/qwen2/modeling.py
	new file:   scripts/codestyle/convert/example/qwen2/modeling_pp.py
	renamed:    scripts/codestyle/qwen2/modeling_qwen2.py -> scripts/codestyle/convert/example/qwen2/modeling_qwen2.py
	renamed:    scripts/codestyle/qwen2/modular_qwen2.py -> scripts/codestyle/convert/example/qwen2/modular_qwen2.py
	new file:   scripts/codestyle/convert/example/qwen2/tokenizer.py
	new file:   scripts/codestyle/convert/example/qwen2/tokenizer_fast.py

* modified:   paddlenlp/transformers/qwen2/configuration.py

* modified:   scripts/codestyle/convert/doc/README.md
	deleted:    scripts/codestyle/convert/doc/process.png
 2025-10-30 15:44:26 +08:00
aiyinyuedejustin
1bcf877c7f LoRA: add lora target modules support for qwen2 and qwen3 dense model when using fused keys (#11153) 2025-10-27 14:58:28 +08:00
Shing
c621f0cfa3 BugFix: qwen model sequence parallel can not get batch size (#11147) 
* feat(model): add support for Qwen model in RL PipelineParallel

* feat(model): add support for QwenMoe model in RL PipelineParallel

* feat(model): add support for QwenMoe model in RL PipelineParallel

* BugFix: qwen model sequence parallel can not get batch size
 2025-10-24 16:57:09 +08:00
Gu Shiwei
3f7eafc774 paddleformers PR check (#11150) 
* paddleformers PR check

* echo line

* -x
 2025-10-23 17:09:34 +08:00
Pioneer-wxh
51787ba815 commit dislora (#11122) 
* commit dislora

* add DisLoRATrainer

* “readme文件修改”

* Delete docs/zh/llm/benchmark/rl/README.md
 2025-10-22 14:52:10 +08:00
Shing
a4d16faf33 BugFix: Support for Qwen Model PipelineParallel Training in RLInfra (#11143) 
* feat(model): add support for Qwen model in RL PipelineParallel

* feat(model): add support for QwenMoe model in RL PipelineParallel

* feat(model): add support for QwenMoe model in RL PipelineParallel
 2025-10-22 10:58:50 +08:00
Xuxinyi
7d30769994 [AutoParallel]update code (#11145) 
* update_code

* update_code
 2025-10-21 20:52:46 +08:00
yujun
2f78db69b0 Upgrad swandlab code (#11105) 
* Upgrad swandlab code

* add SWANLAB_EXP_NAME env
 2025-10-16 20:43:18 +08:00
Shing
5dd2855a52 feat(model): add support for Qwen model in RL PipelineParallel (#11135) 2025-10-11 10:35:33 +08:00
ZhenxingLi
119ed11904 [Auto-Paralllel] fix intermediate api use1 (#11125) 
* fix intermediate api use

* update
 2025-09-30 15:39:42 +08:00
zhongtianyun
3c51eda5d6 [Embedding] Fix readme and env for contrastive_training (#11108) 
modified:   csrc/gpu/unittest/test_get_padding_offset_v2.py
	modified:   paddlenlp/experimental/transformers/llama/modeling.py
	modified:   slm/pipelines/examples/contrastive_training/README.md
	modified:   slm/pipelines/examples/contrastive_training/evaluation/eval_mteb.py
	modified:   slm/pipelines/examples/contrastive_training/evaluation/eval_mteb.sh
	modified:   slm/pipelines/examples/contrastive_training/evaluation/modelling_quant.py
	modified:   slm/pipelines/examples/contrastive_training/shortgpt_prune.py
 2025-09-29 10:53:53 +08:00
Liujie0926
937b766f21 [CI]Update loss (#11115) 
Update loss
 2025-09-25 15:21:34 +08:00
Tianyu Zheng
3432f69e07 Pr adapt flex checkpoint (#11065) 
* adapt_flex_checkpoint

* Consistently use save_checkpoint_format and load_checkpoint_format

* fix the ckpt_format adapt

* delete the duplicate code

* delete safetensor
 2025-09-25 14:16:35 +08:00
Liujie0926
586f5fb544 [CI]Update loss for Paddle#75367 (#11106) 
* update loss_base

* update

* skip llama_dygraph_auto_bs8_fp16_DP2-MP2-PP2_hybrid_pp run dygraph auto hybrid pp in align mode

* fix

* fix

* fix
 2025-09-24 16:21:02 +08:00
zhangyuqin1998
af07de0b01 [Distributed] refine training_pipeline_step to avoid memory leaks (#11103) 2025-09-23 14:59:25 +08:00
Gu Shiwei
b30ad6ad2b fix approval bypass (#11104) 
* fix

* fix

* fix

* fix

* fix
 2025-09-23 10:21:16 +08:00
co63oc
615a7c96ed fix typos (#11041) 2025-09-22 17:51:50 +08:00
Tianyu Zheng
03bc1a0063 [AutoParallel] GradientClipByGlobalNorm: Add align mode branch in autotrainer (#10960) 
* Add test branch for align mode

* fix test bug

* rerun CI

* add the warning to warn the max_grad_norm is not 0.0
 2025-09-22 16:30:31 +08:00
Liujie0926
b6f214efae [CI] Update loss_check (#11090) 
* fix loss_check

* fix
 2025-09-15 15:44:58 +08:00
Wennie396
e28c816070 Add chunk offload optimizer (#11084) 
* add chunk offload optimizer

* fix get offload_opt_buffer_size arg
 2025-09-15 11:19:03 +08:00
Zhuoqun Li
efd7d26434 [Embedding] Add qwen3 and update readme (#11034) 
* add qwen3 and update readme

* update readme

* update readme

* update readme
 2025-09-11 10:20:34 +08:00
sneaxiy
148206ea20 Make sharding_first by default (#11059) 
* make sharding_first by default

* update run_ci.sh

* Update V100 baseline

* Update A100 baselines

* Revert "Update A100 baselines"

This reverts commit c682b2752a83b0e532a15a6b6f90ce08e386e447.

* Revert "Update V100 baseline"

This reverts commit 1ca17b7963d1912bc98c27a5ec1a4bfe0bf8abda.

* fix topo in CI cases

* fix JSON format error
 2025-09-08 10:49:10 +08:00
Gu Shiwei
462597fc1a add bypass (#11077) 
* add bypass

* add bypass
 2025-09-08 10:23:53 +08:00
Ferrebo
85295b6955 [fea] support dp-moe for zcc and global_expert_id (#11050) 
* support dp-moe for zcc (#10539)

* [fix] zcc ema under non-pp when `acc=1`

* zcc-ema fix load-state-dict when dp-moe

* [BugFix] shard-reshard下dp间is_matched行为不一致，以dp0为准 (#9404)

---------

Co-authored-by: Ferrebo <kebo01@baidu.com>

* [mod] Sharding IO: Added DP-Meta gather for MoE

---------

Co-authored-by: Meiyim <chenxuyi@baidu.com>
 2025-09-08 09:56:41 +08:00