Kimirock/PaddleNLP

Files

History

zhongtianyun 3c51eda5d6 [Embedding] Fix readme and env for contrastive_training (#11108 )

modified:   csrc/gpu/unittest/test_get_padding_offset_v2.py
	modified:   paddlenlp/experimental/transformers/llama/modeling.py
	modified:   slm/pipelines/examples/contrastive_training/README.md
	modified:   slm/pipelines/examples/contrastive_training/evaluation/eval_mteb.py
	modified:   slm/pipelines/examples/contrastive_training/evaluation/eval_mteb.sh
	modified:   slm/pipelines/examples/contrastive_training/evaluation/modelling_quant.py
	modified:   slm/pipelines/examples/contrastive_training/shortgpt_prune.py

2025-09-29 10:53:53 +08:00

..

[Inference] fix cpu install (#10218 )

2025-03-20 16:12:18 +08:00

[Embedding] Fix readme and env for contrastive_training (#11108 )

2025-09-29 10:53:53 +08:00

[NPU] support npu llama2-13B export & inference (#8442 )

2024-05-20 22:15:10 +08:00

[Custom Devices] feat(sdaa): support sdaa backend infer (#9570 )

2024-12-16 15:11:42 +08:00

[fp8] Support deep gemm in paddle (#9955 )

2025-03-11 10:51:11 +08:00

[paddlenlp_ops] update whl (#10576 )

2025-05-14 15:17:37 +08:00

Remove some bug config for block gemm (#10002 )

2025-03-06 17:14:37 +08:00

[XPU] support absorb mla, separate the model networking from GPU (#10353 )

2025-04-23 20:47:12 +08:00

README.md

[DSK] DeepSeek Support FP8 (#9956 )

2025-03-05 19:05:51 +08:00

requirements.txt

supports llama-dybatch-V1 (#6676 )

2023-08-22 10:06:50 +08:00

setup_cuda.py

[paddlenlp_ops] Fix custom all reduce and cpp_extensions.cu when sm70/75 (#10554 )

2025-05-09 10:59:19 +08:00

setup_hip.py

del update_inputs.cu (#10022 )

2025-03-07 00:54:58 -08:00

README.md

PaddleNLP 大模型高性能自定义推理算子

此文档介绍如何编译安装 PaddleNLP 大模型高性能自定义推理算子的安装教程。

使用这些高性能算子，可以大幅提升大模型推理速度。大模型推理相关教程详见此处。

安装 C++ 依赖

pip install -r requirements.txt

编译 Cuda 算子

python setup_cuda.py install

FP8 GEMM 自动调优

确保 cutlass 库已经安装，然后执行以下命令进行自动调优。

对于89架构的 GPU，CUDA 版本至少为12.4
对于90架构的 GPU，CUDA 版本至少为12.0

cd tools
sh tune_fp8_gemm.sh