rm print (#3929)

* support hf hub cache

* merge conflict

Co-authored-by: 骑马小猫 <1435130236@qq.com>

README_cn.md

@@ -30,6 +30,10 @@
 **PaddleNLP**是一款**简单易用**且**功能强大**的自然语言处理开发库。聚合业界**优质预训练模型**并提供**开箱即用**的开发体验，覆盖NLP多场景的模型库搭配**产业实践范例**可满足开发者**灵活定制**的需求。
 
 ## News 📢
+
+* 🔥 **2022.11.17 发布 [PaddleNLP v2.4.3](https://github.com/PaddlePaddle/PaddleNLP/releases/tag/v2.4.3)**
+  * 💪 框架升级：🏆 [**小样本 Prompt API**](./docs/advanced_guide/prompt.md) 升级，提示定义更加灵活，支撑 [FewCLUE AutoPrompt 方案](https://mp.weixin.qq.com/s/_JPiAzFA1f0BZ0igdv-EKA)；🕸 [**Trainer API**](./docs/trainer.md) 升级，新增sharding、bf16训练，新增Seq2seqTrainer、IterableDataset支持。
+  * 🔨 产业应用：🏃[**通用信息抽取 UIE 能力升级**](./model_zoo/uie)，支持量化训练及 INT8 精度推理，进一步提升 UIE 推理速度。
 * 🔥 **2022.10.27 发布 [PaddleNLP v2.4.2](https://github.com/PaddlePaddle/PaddleNLP/releases/tag/v2.4.2)**
   * NLG能力扩充：新增📄[**基于Pegasus的中文文本摘要方案**](https://github.com/PaddlePaddle/PaddleNLP/tree/develop/applications/text_summarization/pegasus)，效果领先；新增❓[**问题生成解决方案**](./examples/question_generation)，提供基于业界领先模型UNIMO-Text和大规模多领域问题生成数据集训练的通用问题生成预训练模型。均支持Taskflow一键调用，支持FasterGeneration高性能推理，训练推理部署全流程打通。
   * 发布 🖼[**PPDiffusers**](https://github.com/PaddlePaddle/PaddleNLP/tree/develop/ppdiffusers)：支持跨模态（如图像与语音）训练和推理的扩散模型（Diffusion Model）工具箱，可快速体验、二次开发 **Stable Diffusion**，持续支持更多模型。
@@ -39,7 +43,7 @@
 
 * 🔥 **2022.9.6 发布 [PaddleNLP v2.4](https://github.com/PaddlePaddle/PaddleNLP/releases/tag/v2.4.0)**
   * 💎 NLP工具：**[NLP 流水线系统 Pipelines](./pipelines)** 发布，支持快速搭建搜索引擎、问答系统，可扩展支持各类NLP系统，让解决 NLP 任务像搭积木一样便捷、灵活、高效！
-  * 💢 产业应用：新增 **[文本分类全流程应用方案](./applications/text_classification)** ，覆盖多分类、多标签、层次分类各类场景，支持 **小样本学习** 和 **TrustAI** 可信计算模型训练与调优；[**通用信息抽取 UIE 能力升级**](./model_zoo/uie)，发布 **UIE-M**，支持中英文混合抽取，新增**UIE 数据蒸馏**方案，打破 UIE 推理瓶颈，推理速度提升 100 倍以上；
+  * 🔨 产业应用：新增 **[文本分类全流程应用方案](./applications/text_classification)** ，覆盖多分类、多标签、层次分类各类场景，支持 **小样本学习** 和 **TrustAI** 可信计算模型训练与调优；[**通用信息抽取 UIE 能力升级**](./model_zoo/uie)，发布 **UIE-M**，支持中英文混合抽取，新增**UIE 数据蒸馏**方案，打破 UIE 推理瓶颈，推理速度提升 100 倍以上；
   * 🍭 AIGC 内容生成：新增代码生成 SOTA 模型[**CodeGen**](./examples/code_generation/codegen)，支持多种编程语言代码生成；集成[**文图生成潮流模型**](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/docs/model_zoo/taskflow.md#%E6%96%87%E5%9B%BE%E7%94%9F%E6%88%90) DALL·E Mini、Disco Diffusion、Stable Diffusion，更多趣玩模型等你来玩；新增[**中文文本摘要应用**](./applications/text_summarization)，基于大规模语料的中文摘要模型首次发布，可支持 Taskflow 一键调用和定制训练；
   * 💪 框架升级：[**模型自动压缩 API**](./docs/compression.md) 发布，自动对模型进行裁减和量化，大幅降低模型压缩技术使用门槛；[**小样本 Prompt**](./applications/text_classification/multi_class/few-shot)能力发布，集成 PET、P-Tuning、RGL 等经典算法。
 
@@ -234,17 +238,17 @@ PaddleNLP针对信息抽取、语义检索、智能问答、情感分析等高
 
 ### 高性能分布式训练与推理
 
-#### ⚡ FasterTokenizer：高性能文本处理库
+#### ⚡ FastTokenizer：高性能文本处理库
 
 <div align="center">
     <img src="https://user-images.githubusercontent.com/11793384/168407921-b4395b1d-44bd-41a0-8c58-923ba2b703ef.png" width="400">
 </div>
 
 ```python
-AutoTokenizer.from_pretrained("ernie-3.0-medium-zh", use_faster=True)
+AutoTokenizer.from_pretrained("ernie-3.0-medium-zh", use_fast=True)
 ```
 
-为了实现更极致的模型部署性能，安装FastTokenizers后只需在`AutoTokenizer` API上打开 `use_faster=True`选项，即可调用C++实现的高性能分词算子，轻松获得超Python百余倍的文本处理加速，更多使用说明可参考[FasterTokenizer文档](./faster_tokenizer)。
+为了实现更极致的模型部署性能，安装FastTokenizers后只需在`AutoTokenizer` API上打开 `use_fast=True`选项，即可调用C++实现的高性能分词算子，轻松获得超Python百余倍的文本处理加速，更多使用说明可参考[FastTokenizer文档](./fast_tokenizer)。
 
 #### ⚡️ FasterGeneration：高性能生成加速库
 
@@ -275,7 +279,7 @@ outputs, _ = model.generate(
 
 ### 环境依赖
 
-- python >= 3.6
+- python >= 3.7
 - paddlepaddle >= 2.2
 
 ### pip安装

README_en.md

@@ -30,15 +30,18 @@
 
 ## News 📢
 
+* 🔥 **2022.11.17 [PaddleNLP v2.4.3](https://github.com/PaddlePaddle/PaddleNLP/releases/tag/v2.4.3) Released!**
+  * 💪 Framework upgrade：🏆 Upgrade [**Prompt API**](./docs/advanced_guide/prompt.md), supporting more flexible prompt definitions and winning the 1st place in [FewCLUE](https://mp.weixin.qq.com/s/_JPiAzFA1f0BZ0igdv-EKA); 🕸 Upgrade [**Trainer API**](./docs/trainer.md), supporting Seq2seqTrainer, IterableDataset as well as bf16 and sharding strategies.
+  * 🔨 Industrial application：🏃 Upgrade for [**Universal Information Extraction**](./model_zoo/uie). Support **quantization aware training** and INT8 precision inference for inference performance boost.
 * 🔥 **2022.10.27 [PaddleNLP v2.4.2](https://github.com/PaddlePaddle/PaddleNLP/releases/tag/v2.4.2) Released!**
-  * NLG Upgrade: 📄 Release [**Solution of Text Summarization**](https://github.com/PaddlePaddle/PaddleNLP/tree/develop/applications/text_summarization/pegasus) based on Pegasus;❓ Release [**Solution of Problem Generation**](./examples/question_generation), providing **general problem generation pre-trained model** based on Baidu's UNIMO Text and large-scale multi domain problem generation dataset。Supporting high-performance inference ability based on FasterGeneration , and covering the whole process of training , inference and deployment.
+  * NLG Upgrade: 📄 Release [**Solution of Text Summarization**](https://github.com/PaddlePaddle/PaddleNLP/tree/develop/applications/text_summarization/pegasus) based on Pegasus;❓ Release [**Solution of Problem Generation**](./examples/question_generation), providing **general problem generation pre-trained model** based on Baidu's UNIMO Text and large-scale multi domain problem generation dataset. Supporting high-performance inference ability based on FasterGeneration , and covering the whole process of training , inference and deployment.
 * 🔥 **2022.10.14 [PaddleNLP v2.4.1](https://github.com/PaddlePaddle/PaddleNLP/releases/tag/v2.4.1) Released!**
   * 🧾 Release multilingual/cross-lingual pre-trained models [**ERNIE-Layout**](./model_zoo/ernie-layout/) which achieves new SOTA results in 11 downstream tasks. **DocPrompt** 🔖 based on ERNIE-Layout is also released which has the ability for multilingual document information extraction and question ansering.
 * 🔥 **2022.9.6 [PaddleNLPv2.4](https://github.com/PaddlePaddle/PaddleNLP/releases/tag/v2.4.0) Released!**
 
   * 💎 NLP Tool：**[Pipelines](./pipelines)** released. Supports for fast construction of search engine and question answering systems, and it is expandable to all kinds of NLP systems. Building end-to-end pipelines for NLP tasks like playing Lego!
 
-  * 💢 Industrial application：Release **[Complete Solution of Text Classification](./applications/text_classification)** covering various scenarios of text classification: multi-class, multi-label and hierarchical, it also supports for **few-shot learning** and the training and optimization of **TrustAI**. Upgrade for [**Universal Information Extraction**](./model_zoo/uie) and release **UIE-M**, support both Chinese and English information extraction in a single model; release the data distillation solution for UIE to break the bottleneck of time-consuming of inference.
+  * 🔨 Industrial application：Release **[Complete Solution of Text Classification](./applications/text_classification)** covering various scenarios of text classification: multi-class, multi-label and hierarchical, it also supports for **few-shot learning** and the training and optimization of **TrustAI**. Upgrade for [**Universal Information Extraction**](./model_zoo/uie) and release **UIE-M**, support both Chinese and English information extraction in a single model; release the data distillation solution for UIE to break the bottleneck of time-consuming of inference.
 
   * 🍭 AIGC: Release code generation SOTA model [**CodeGen**](./examples/code_generation/codegen), supports for multiple programming languages code generation. Integrate [**Text to Image Model**](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/docs/model_zoo/taskflow.md#%E6%96%87%E5%9B%BE%E7%94%9F%E6%88%90) DALL·E Mini, Disco Diffusion, Stable Diffusion, let's play and have some fun! Release [**Chinese Text Summarization Application**](./applications/text_summarization), first release of chinese text summarization model pretrained on a large scale of corpus, it can be use via Taskflow API and support for finetuning on your own data.
 
@@ -221,17 +224,17 @@ For more details please refer to [Speech Command Analysis](./applications/speech
 
 ### High Performance Distributed Training and Inference
 
-#### ⚡ FasterTokenizer: High Performance Text Preprocessing Library
+#### ⚡ FastTokenizer: High Performance Text Preprocessing Library
 
 <div align="center">
     <img src="https://user-images.githubusercontent.com/11793384/168407921-b4395b1d-44bd-41a0-8c58-923ba2b703ef.png" width="400">
 </div>
 
 ```python
-AutoTokenizer.from_pretrained("ernie-3.0-medium-zh", use_faster=True)
+AutoTokenizer.from_pretrained("ernie-3.0-medium-zh", use_fast=True)
 ```
 
-Set `use_faster=True` to use C++ Tokenizer kernel to achieve 100x faster on text pre-processing. For more usage please refer to [FasterTokenizer](./faster_tokenizer).
+Set `use_fast=True` to use C++ Tokenizer kernel to achieve 100x faster on text pre-processing. For more usage please refer to [FastTokenizer](./fast_tokenizer).
 
 #### ⚡ FasterGeneration: High Perforance Generation Library
 
@@ -263,7 +266,7 @@ For more super large-scale model pre-training details please refer to [GPT-3](./
 
 ### Prerequisites
 
-* python >= 3.6
+* python >= 3.7
 * paddlepaddle >= 2.2
 
 More information about PaddlePaddle installation please refer to [PaddlePaddle's Website](https://www.paddlepaddle.org.cn/install/quick?docurl=/documentation/docs/zh/install/conda/linux-conda.html).

applications/neural_search/README.md

@@ -16,7 +16,12 @@
 车头如何放置车牌    后牌照怎么装
 ```
 
-语义检索系统的关键就在于，采用语义而非关键词方式进行召回，达到更精准、更广泛得召回相似结果的目的。想快速体验搜索的效果，请参考[Pipelines的实现](https://github.com/PaddlePaddle/PaddleNLP/tree/develop/pipelines)
+语义检索系统的关键就在于，采用语义而非关键词方式进行召回，达到更精准、更广泛得召回相似结果的目的。想快速体验搜索的效果，请参考[Pipelines的语义检索实现](https://github.com/PaddlePaddle/PaddleNLP/tree/develop/pipelines/examples/semantic-search)
+
+<div align="center">
+    <img src="https://user-images.githubusercontent.com/12107462/190302765-663ba441-9dd3-470a-8fee-f7a6f81da615.gif" width="500px">
+</div>
+
 
 ## 2. 产品功能介绍
 
@@ -28,6 +33,7 @@
     + 手把手搭建起检索系统
     + 无需标注数据也能构建检索系统
     + 提供 训练、预测、ANN 引擎一站式能力
+    + Pipelines 快速实现语义检索系统
 
 + 效果好
     + 针对多种数据场景的专业方案
@@ -308,7 +314,7 @@ pip install -r requirements.txt
 
 ## 4. Neural Search 快速体验实践
 
-PaddleNLP已经基于ERNIE 1.0训练了一个基线模型，如果想快速搭建Neural Search的完整系统。
+PaddleNLP已经基于ERNIE 1.0训练了一个基线模型，如果想快速搭建Neural Search的完整系统，有两种方法，第一种是请参考下面的实现，包含了服务化的完整流程，另一种是使用Pipelines加载，Pipelines已经支持Neural Search训练的模型的载入，可以使用Pipelines的快速的基于Neural Search模型实现检索系统，详情请参考文档[Pipelines-Neural-Search](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/pipelines/examples/semantic-search/Neural_Search.md)。
 
 ### 4.1. 召回
 

applications/neural_search/recall/in_batch_negative/README.md

@@ -229,6 +229,9 @@ python -u -m paddle.distributed.launch --gpus "0,1,2,3" \
 * `recall_num`: 对 1 个文本召回的相似文本数量
 * `similar_text_pair_file`: 由相似文本对构成的评估集
 * `corpus_file`: 召回库数据 corpus_file
+* `use_recompute`: 使用Recompute策略，用于节省显存，是一种以时间换空间的技术
+* `use_gradient_cache`: 使用Gradient Cache策略，用于节省显存，是一种以时间换空间的技术
+* `chunk_numbers`: 使用Gradient Cache策略的参数，表示的是同一个批次的样本分几次执行
 
 也可以使用bash脚本：
 

applications/neural_search/recall/in_batch_negative/batch_negative/model.py

@@ -60,14 +60,14 @@ class SemanticIndexBatchNeg(SemanticIndexBase):
                                    title_cls_embedding,
                                    transpose_y=True)
 
-        # substract margin from all positive samples cosine_sim()
+        # Substract margin from all positive samples cosine_sim()
         margin_diag = paddle.full(shape=[query_cls_embedding.shape[0]],
                                   fill_value=self.margin,
                                   dtype=paddle.get_default_dtype())
 
         cosine_sim = cosine_sim - paddle.diag(margin_diag)
 
-        # scale cosine to ease training converge
+        # Scale cosine to ease training converge
         cosine_sim *= self.sacle
 
         labels = paddle.arange(0, query_cls_embedding.shape[0], dtype='int64')
@@ -76,3 +76,56 @@ class SemanticIndexBatchNeg(SemanticIndexBase):
         loss = F.cross_entropy(input=cosine_sim, label=labels)
 
         return loss
+
+
+class SemanticIndexCacheNeg(SemanticIndexBase):
+
+    def __init__(self,
+                 pretrained_model,
+                 dropout=None,
+                 margin=0.3,
+                 scale=30,
+                 output_emb_size=None):
+        super().__init__(pretrained_model, dropout, output_emb_size)
+        self.margin = margin
+        # Used scaling cosine similarity to ease converge
+        self.sacle = scale
+
+    def forward(self,
+                query_input_ids,
+                title_input_ids,
+                query_token_type_ids=None,
+                query_position_ids=None,
+                query_attention_mask=None,
+                title_token_type_ids=None,
+                title_position_ids=None,
+                title_attention_mask=None):
+
+        query_cls_embedding = self.get_pooled_embedding(query_input_ids,
+                                                        query_token_type_ids,
+                                                        query_position_ids,
+                                                        query_attention_mask)
+
+        title_cls_embedding = self.get_pooled_embedding(title_input_ids,
+                                                        title_token_type_ids,
+                                                        title_position_ids,
+                                                        title_attention_mask)
+
+        cosine_sim = paddle.matmul(query_cls_embedding,
+                                   title_cls_embedding,
+                                   transpose_y=True)
+
+        # Substract margin from all positive samples cosine_sim()
+        margin_diag = paddle.full(shape=[query_cls_embedding.shape[0]],
+                                  fill_value=self.margin,
+                                  dtype=cosine_sim.dtype)
+
+        cosine_sim = cosine_sim - paddle.diag(margin_diag)
+
+        # Scale cosine to ease training converge
+        cosine_sim *= self.sacle
+
+        labels = paddle.arange(0, query_cls_embedding.shape[0], dtype='int64')
+        labels = paddle.reshape(labels, shape=[-1, 1])
+
+        return [cosine_sim, labels, query_cls_embedding, title_cls_embedding]

applications/neural_search/recall/in_batch_negative/scripts/run_build_index.sh

@@ -1,6 +1,20 @@
+# Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
+# 
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+# 
+#     http://www.apache.org/licenses/LICENSE-2.0
+# 
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
 # GPU version
 root_dir="checkpoints/inbatch" 
-python -u -m paddle.distributed.launch --gpus "3" --log_dir "recall_log/" \
+python -u -m paddle.distributed.launch --gpus "0" --log_dir "recall_log/" \
         recall.py \
         --device gpu \
         --recall_result_dir "recall_result_dir" \
@@ -11,7 +25,7 @@ python -u -m paddle.distributed.launch --gpus "3" --log_dir "recall_log/" \
         --hnsw_ef 100 \
         --batch_size 64 \
         --output_emb_size 256\
-        --max_seq_length 60 \
+        --max_seq_length 64 \
         --recall_num 50 \
         --similar_text_pair "recall/dev.csv" \
         --corpus_file "recall/corpus.csv" 

applications/neural_search/recall/in_batch_negative/train_batch_neg.py

@@ -17,14 +17,17 @@ import random
 import time
 import numpy as np
 import paddle
+import paddle.nn.functional as F
 from functools import partial
+
 from paddlenlp.utils.log import logger
 from paddlenlp.data import Tuple, Pad
 from paddlenlp.datasets import load_dataset, MapDataset
 from paddlenlp.transformers import AutoModel, AutoTokenizer
 from paddlenlp.transformers import LinearDecayWithWarmup
+
 from base_model import SemanticIndexBase
-from batch_negative.model import SemanticIndexBatchNeg
+from batch_negative.model import SemanticIndexBatchNeg, SemanticIndexCacheNeg
 from data import read_text_pair, convert_example, create_dataloader, gen_id2corpus, gen_text_file
 from ann_util import build_index
 
@@ -89,6 +92,15 @@ parser.add_argument("--evaluate_result", type=str, default='evaluate_result.txt'
                     help="evaluate_result")
 parser.add_argument('--evaluate', action='store_true',
                     help='whether evaluate while training')
+parser.add_argument("--use_amp", action="store_true", help="Whether to use AMP.")
+parser.add_argument("--amp_loss_scale", default=32768, type=float,help="The value of scale_loss for fp16. This is only used for AMP training.")
+parser.add_argument("--use_recompute",
+                        action='store_true',
+                        help="Using the recompute to scale up the batch size and save the memory.")
+parser.add_argument("--use_gradient_cache",
+                        action='store_true',
+                        help="Using the gradient cache to scale up the batch size and save the memory.")
+parser.add_argument("--chunk_numbers",type=int,default=50,help="The number of the chunks for model")
 args = parser.parse_args()
 # yapf: enable
 
@@ -161,6 +173,179 @@ def evaluate(model, corpus_data_loader, query_data_loader, recall_result_file,
     return float(recall_N[1])
 
 
+def train(train_data_loader, model, optimizer, lr_scheduler, rank,
+          corpus_data_loader, query_data_loader, recall_result_file, text_list,
+          id2corpus, tokenizer):
+    global_step = 0
+    best_recall = 0.0
+    tic_train = time.time()
+    for epoch in range(1, args.epochs + 1):
+        for step, batch in enumerate(train_data_loader, start=1):
+            query_input_ids, query_token_type_ids, title_input_ids, title_token_type_ids = batch
+
+            loss = model(query_input_ids=query_input_ids,
+                         title_input_ids=title_input_ids,
+                         query_token_type_ids=query_token_type_ids,
+                         title_token_type_ids=title_token_type_ids)
+
+            global_step += 1
+            if global_step % args.log_steps == 0 and rank == 0:
+                print(
+                    "global step %d, epoch: %d, batch: %d, loss: %.5f, speed: %.2f step/s"
+                    % (global_step, epoch, step, loss, args.log_steps /
+                       (time.time() - tic_train)))
+                tic_train = time.time()
+            loss.backward()
+            optimizer.step()
+            lr_scheduler.step()
+            optimizer.clear_grad()
+            if not args.evaluate:
+                if global_step % args.save_steps == 0 and rank == 0:
+                    save_dir = os.path.join(args.save_dir,
+                                            "model_%d" % global_step)
+                    if not os.path.exists(save_dir):
+                        os.makedirs(save_dir)
+                    save_param_path = os.path.join(save_dir,
+                                                   'model_state.pdparams')
+                    paddle.save(model.state_dict(), save_param_path)
+                    tokenizer.save_pretrained(save_dir)
+        if args.evaluate and rank == 0:
+            print("evaluating")
+            recall_5 = evaluate(model, corpus_data_loader, query_data_loader,
+                                recall_result_file, text_list, id2corpus)
+            if recall_5 > best_recall:
+                best_recall = recall_5
+
+                save_dir = os.path.join(args.save_dir, "model_best")
+                if not os.path.exists(save_dir):
+                    os.makedirs(save_dir)
+                save_param_path = os.path.join(save_dir, 'model_state.pdparams')
+                paddle.save(model.state_dict(), save_param_path)
+                tokenizer.save_pretrained(save_dir)
+                with open(os.path.join(save_dir, "train_result.txt"),
+                          'a',
+                          encoding='utf-8') as fp:
+                    fp.write('epoch=%d, global_step: %d, recall: %s\n' %
+                             (epoch, global_step, recall_5))
+
+
+def gradient_cache_train(train_data_loader, model, optimizer, lr_scheduler,
+                         rank, tokenizer):
+
+    if args.use_amp:
+        scaler = paddle.amp.GradScaler(init_loss_scaling=args.amp_loss_scale)
+
+    if args.batch_size % args.chunk_numbers == 0:
+        chunk_numbers = args.chunk_numbers
+    else:
+        raise Exception(
+            f" Batch_size {args.batch_size} must divides chunk_numbers {args.chunk_numbers} without producing a remainder "
+        )
+
+    def split(inputs, chunk_numbers, axis=0):
+        if inputs.shape[0] % chunk_numbers == 0:
+            return paddle.split(inputs, chunk_numbers, axis=0)
+        else:
+            return paddle.split(inputs, inputs.shape[0], axis=0)
+
+    global_step = 0
+    tic_train = time.time()
+    for epoch in range(1, args.epochs + 1):
+        for step, batch in enumerate(train_data_loader, start=1):
+            # Separate large batches into several sub batches
+            chunked_x = [split(t, chunk_numbers, axis=0) for t in batch]
+            sub_batchs = [list(s) for s in zip(*chunked_x)]
+
+            all_grads = []
+            all_CUDA_rnd_state = []
+            all_query = []
+            all_title = []
+
+            for sub_batch in sub_batchs:
+                all_reps = []
+                all_labels = []
+                sub_query_input_ids, sub_query_token_type_ids, sub_title_input_ids, sub_title_token_type_ids = sub_batch
+                with paddle.amp.auto_cast(
+                        args.use_amp,
+                        custom_white_list=["layer_norm", "softmax", "gelu"]):
+
+                    with paddle.no_grad():
+                        sub_CUDA_rnd_state = paddle.framework.random.get_cuda_rng_state(
+                        )
+                        all_CUDA_rnd_state.append(sub_CUDA_rnd_state)
+                        sub_cosine_sim, sub_label, query_embedding, title_embedding = model(
+                            query_input_ids=sub_query_input_ids,
+                            title_input_ids=sub_title_input_ids,
+                            query_token_type_ids=sub_query_token_type_ids,
+                            title_token_type_ids=sub_title_token_type_ids)
+                        all_reps.append(sub_cosine_sim)
+                        all_labels.append(sub_label)
+                        all_title.append(title_embedding)
+                        all_query.append(query_embedding)
+
+                model_reps = paddle.concat(all_reps, axis=0)
+                model_title = paddle.concat(all_title)
+                model_query = paddle.concat(all_query)
+
+                model_title = model_title.detach()
+                model_query = model_query.detach()
+
+                model_query.stop_gradient = False
+                model_title.stop_gradient = False
+                model_reps.stop_gradient = False
+
+                model_label = paddle.concat(all_labels, axis=0)
+                loss = F.cross_entropy(input=model_reps, label=model_label)
+                loss.backward()
+                # Store gradients
+                all_grads.append(model_reps.grad)
+
+            for sub_batch, CUDA_state, grad in zip(sub_batchs,
+                                                   all_CUDA_rnd_state,
+                                                   all_grads):
+
+                sub_query_input_ids, sub_query_token_type_ids, sub_title_input_ids, sub_title_token_type_ids = sub_batch
+                paddle.framework.random.set_cuda_rng_state(CUDA_state)
+                # Recompute the forward propogation
+                sub_cosine_sim, sub_label, query_embedding, title_embedding = model(
+                    query_input_ids=sub_query_input_ids,
+                    title_input_ids=sub_title_input_ids,
+                    query_token_type_ids=sub_query_token_type_ids,
+                    title_token_type_ids=sub_title_token_type_ids)
+                # Chain rule
+                surrogate = paddle.dot(sub_cosine_sim, grad)
+                # Backward propogation
+                if args.use_amp:
+                    scaled = scaler.scale(surrogate)
+                    scaled.backward()
+                else:
+                    surrogate.backward()
+            # Update model parameters
+            if args.use_amp:
+                scaler.minimize(optimizer, scaled)
+            else:
+                optimizer.step()
+
+            global_step += 1
+            if global_step % args.log_steps == 0 and rank == 0:
+                print(
+                    "global step %d, epoch: %d, batch: %d, loss: %.5f, speed: %.2f step/s"
+                    % (global_step, epoch, step, loss, args.log_steps /
+                       (time.time() - tic_train)))
+                tic_train = time.time()
+
+            lr_scheduler.step()
+            optimizer.clear_grad()
+
+            if global_step % args.save_steps == 0 and rank == 0:
+                save_dir = os.path.join(args.save_dir, "model_%d" % global_step)
+                if not os.path.exists(save_dir):
+                    os.makedirs(save_dir)
+                save_param_path = os.path.join(save_dir, 'model_state.pdparams')
+                paddle.save(model.state_dict(), save_param_path)
+                tokenizer.save_pretrained(save_dir)
+
+
 def do_train():
     paddle.set_device(args.device)
     rank = paddle.distributed.get_rank()
@@ -173,7 +358,8 @@ def do_train():
                             data_path=args.train_set_file,
                             lazy=False)
 
-    pretrained_model = AutoModel.from_pretrained(args.model_name_or_path)
+    pretrained_model = AutoModel.from_pretrained(
+        args.model_name_or_path, enable_recompute=args.use_recompute)
 
     tokenizer = AutoTokenizer.from_pretrained(args.model_name_or_path)
 
@@ -197,11 +383,16 @@ def do_train():
                                           batch_size=args.batch_size,
                                           batchify_fn=batchify_fn,
                                           trans_fn=trans_func)
-
-    model = SemanticIndexBatchNeg(pretrained_model,
-                                  margin=args.margin,
-                                  scale=args.scale,
-                                  output_emb_size=args.output_emb_size)
+    if (args.use_gradient_cache):
+        model = SemanticIndexCacheNeg(pretrained_model,
+                                      margin=args.margin,
+                                      scale=args.scale,
+                                      output_emb_size=args.output_emb_size)
+    else:
+        model = SemanticIndexBatchNeg(pretrained_model,
+                                      margin=args.margin,
+                                      scale=args.scale,
+                                      output_emb_size=args.output_emb_size)
 
     if args.init_from_ckpt and os.path.isfile(args.init_from_ckpt):
         state_dict = paddle.load(args.init_from_ckpt)
@@ -262,57 +453,13 @@ def do_train():
         weight_decay=args.weight_decay,
         apply_decay_param_fun=lambda x: x in decay_params)
 
-    global_step = 0
-    best_recall = 0.0
-    tic_train = time.time()
-    for epoch in range(1, args.epochs + 1):
-        for step, batch in enumerate(train_data_loader, start=1):
-            query_input_ids, query_token_type_ids, title_input_ids, title_token_type_ids = batch
-
-            loss = model(query_input_ids=query_input_ids,
-                         title_input_ids=title_input_ids,
-                         query_token_type_ids=query_token_type_ids,
-                         title_token_type_ids=title_token_type_ids)
-
-            global_step += 1
-            if global_step % args.log_steps == 0 and rank == 0:
-                print(
-                    "global step %d, epoch: %d, batch: %d, loss: %.5f, speed: %.2f step/s"
-                    % (global_step, epoch, step, loss, 10 /
-                       (time.time() - tic_train)))
-                tic_train = time.time()
-            loss.backward()
-            optimizer.step()
-            lr_scheduler.step()
-            optimizer.clear_grad()
-            if not args.evaluate:
-                if global_step % args.save_steps == 0 and rank == 0:
-                    save_dir = os.path.join(args.save_dir,
-                                            "model_%d" % global_step)
-                    if not os.path.exists(save_dir):
-                        os.makedirs(save_dir)
-                    save_param_path = os.path.join(save_dir,
-                                                   'model_state.pdparams')
-                    paddle.save(model.state_dict(), save_param_path)
-                    tokenizer.save_pretrained(save_dir)
-        if args.evaluate and rank == 0:
-            print("evaluating")
-            recall_5 = evaluate(model, corpus_data_loader, query_data_loader,
-                                recall_result_file, text_list, id2corpus)
-            if recall_5 > best_recall:
-                best_recall = recall_5
-
-                save_dir = os.path.join(args.save_dir, "model_best")
-                if not os.path.exists(save_dir):
-                    os.makedirs(save_dir)
-                save_param_path = os.path.join(save_dir, 'model_state.pdparams')
-                paddle.save(model.state_dict(), save_param_path)
-                tokenizer.save_pretrained(save_dir)
-                with open(os.path.join(save_dir, "train_result.txt"),
-                          'a',
-                          encoding='utf-8') as fp:
-                    fp.write('epoch=%d, global_step: %d, recall: %s\n' %
-                             (epoch, global_step, recall_5))
+    if (args.use_gradient_cache):
+        gradient_cache_train(train_data_loader, model, optimizer, lr_scheduler,
+                             rank, tokenizer)
+    else:
+        train(train_data_loader, model, optimizer, lr_scheduler, rank,
+              corpus_data_loader, query_data_loader, recall_result_file,
+              text_list, id2corpus, tokenizer)
 
 
 if __name__ == "__main__":

applications/text_classification/README.md

@@ -115,6 +115,7 @@
 
 【快速开始】
 - 快速开启多分类任务参见 👉 [语义索引-多分类指南](https://github.com/PaddlePaddle/PaddleNLP/tree/develop/applications/text_classification/multi_class/retrieval_based#readme)
+- 快速开启多标签分类任务参见 👉 [语义索引-多标签分类指南](https://github.com/PaddlePaddle/PaddleNLP/tree/develop/applications/text_classification/multi_label/retrieval_based#readme)
 - 快速开启层次分类任务参见 👉 [语义索引-层次分类指南](https://github.com/PaddlePaddle/PaddleNLP/tree/develop/applications/text_classification/hierarchical/retrieval_based#readme)
 
 

applications/text_classification/doccano.py

@@ -115,14 +115,15 @@ def do_convert():
                     level_labels = label.split(args.separator)
                     for i in range(len(level_labels)):
                         l = args.separator.join(level_labels[:i + 1])
-                        label_dict.append(l)
+                        if l not in label_dict:
+                            label_dict.append(l)
                         if l not in label_list:
                             label_list.append(l)
                 if args.dirty:
                     text = ' '.join(text.strip().split('\t')[:-1])
                 else:
                     text = ' '.join(text.strip().split('\t'))
-                example = text + '\t' + ','.join(label_list) + '\n'
+                example = text + '\t' + ','.join(label_dict) + '\n'
             examples.append(example)
 
     if not args.dirty and not args.valid:

applications/text_classification/hierarchical/analysis/README.md

@@ -121,7 +121,7 @@ Text	Label	Prediction
 ### 单词级别可解释性分析
 本项目开源模型的词级别可解释性分析Notebook，提供LIME、Integrated Gradient、GradShap 三种分析方法，支持分析微调后模型的预测结果，开发者可以通过更改**数据目录**和**模型目录**在自己的任务中使用Jupyter Notebook进行数据分析。
 
-运行 [word_interpret.ipynb](./word_interpreter.ipynb) 代码，即可分析影响样本预测结果的关键词以及可视化所有词对预测结果的贡献情况，颜色越深代表这个词对预测结果影响越大：
+运行 [word_interpret.ipynb](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/applications/text_classification/hierarchical/analysis/README.md) 代码，即可分析影响样本预测结果的关键词以及可视化所有词对预测结果的贡献情况，颜色越深代表这个词对预测结果影响越大：
 <div align="center">
     <img src="https://user-images.githubusercontent.com/63761690/195334753-78cc2dc8-a5ba-4460-9fde-3b1bb704c053.png" width="1000">
 </div>

applications/text_classification/hierarchical/analysis/word_interpret.ipynb

@@ -112,6 +112,7 @@
     "            items = line.strip().split('\\t')\n",
     "            if items[0] == 'Text':\n",
     "                continue\n",
+    "            items[0] = items[0][:MAX_LENGTH-2]\n",
     "            if len(items) == 3:\n",
     "                yield {'text': items[0], 'label': items[1], 'predict': items[2]}\n",
     "            elif len(items) == 2:\n",

applications/text_classification/hierarchical/deploy/paddle_serving/README.md

@@ -54,10 +54,10 @@ pip install paddle-serving-server-gpu==0.8.3.post112 -i https://pypi.tuna.tsingh
 - 默认开启国内清华镜像源来加速下载，如果您使用 HTTP 代理可以关闭(-i https://pypi.tuna.tsinghua.edu.cn/simple)
 - 更多wheel包请参考[serving官网文档](https://github.com/PaddlePaddle/Serving/blob/develop/doc/Latest_Packages_CN.md)
 
-### 安装FasterTokenizer文本处理加速库（可选）
-推荐安装faster_tokenizer可以得到更极致的文本处理效率，进一步提升服务性能。
+### 安装FastTokenizer文本处理加速库（可选）
+推荐安装fast_tokenizer可以得到更极致的文本处理效率，进一步提升服务性能。
 ```shell
-pip install faster_tokenizer
+pip install fast-tokenizer-python
 ```
 
 
@@ -147,7 +147,7 @@ I0727 06:50:34.988327 43126 analysis_predictor.cc:1007] ======= optimize end ===
 I0727 06:50:34.992336 43126 naive_executor.cc:102] ---  skip [feed], feed -> token_type_ids
 I0727 06:50:34.992357 43126 naive_executor.cc:102] ---  skip [feed], feed -> input_ids
 I0727 06:50:34.993671 43126 naive_executor.cc:102] ---  skip [linear_75.tmp_1], fetch -> fetch
-[2022-07-27 06:50:35,954] [ WARNING] - Can't find the faster_tokenizers package, please ensure install faster_tokenizers correctly. You can install faster_tokenizers by `pip install faster_tokenizers`(Currently only work for linux platform).
+[2022-07-27 06:50:35,954] [ WARNING] - Can't find the fast_tokenizer package, please ensure install fast_tokenizer correctly. You can install fast_tokenizer by `pip install fast-tokenizer-python`.
 [2022-07-27 06:50:35,954] [    INFO] - We are using <class 'paddlenlp.transformers.ernie.tokenizer.ErnieTokenizer'> to load 'ernie-3.0-medium-zh'.
 [2022-07-27 06:50:35,954] [    INFO] - Already cached /root/.paddlenlp/models/ernie-3.0-medium-zh/ernie_3.0_medium_zh_vocab.txt
 [OP Object] init success

applications/text_classification/hierarchical/deploy/paddle_serving/service.py

@@ -49,7 +49,7 @@ class Op(Op):
 
     def init_op(self):
         self.tokenizer = AutoTokenizer.from_pretrained(args.model_name,
-                                                       use_faster=True)
+                                                       use_fast=True)
         # Output nodes may differ from model to model
         # You can see the output node name in the conf.prototxt file of serving_server
         self.fetch_names = [

applications/text_classification/hierarchical/deploy/predictor/README.md

@@ -11,7 +11,7 @@
 
 如果基于GPU部署，请先确保机器已正确安装NVIDIA相关驱动和基础软件，确保CUDA >= 11.2，CuDNN >= 8.2，并使用以下命令安装所需依赖:
 ```shell
-python -m pip install onnxruntime-gpu onnx onnxconverter-common psutil
+python -m pip install onnxruntime-gpu onnx onnxconverter-common==1.9.0 psutil
 ```
 
 如果基于CPU部署，请使用如下命令安装所需依赖:
@@ -19,10 +19,10 @@ python -m pip install onnxruntime-gpu onnx onnxconverter-common psutil
 python -m pip install onnxruntime psutil
 ```
 
-安装FasterTokenizer文本处理加速库（可选）
-推荐安装faster_tokenizer可以得到更极致的文本处理效率，进一步提升服务性能。
+安装FastTokenizer文本处理加速库（可选）
+推荐安装fast_tokenizer可以得到更极致的文本处理效率，进一步提升服务性能。
 ```shell
-pip install faster_tokenizer
+pip install fast-tokenizer-python
 ```
 
 ## 基于GPU部署推理样例

applications/text_classification/hierarchical/deploy/predictor/predictor.py

@@ -123,7 +123,7 @@ class Predictor(object):
 
     def __init__(self, args, label_list):
         self.tokenizer = AutoTokenizer.from_pretrained(args.model_name_or_path,
-                                                       use_faster=True)
+                                                       use_fast=True)
         self.label_list = label_list
         self.batch_size = args.batch_size
         self.max_seq_length = args.max_seq_length

applications/text_classification/hierarchical/deploy/simple_serving/README.md

@@ -0,0 +1,39 @@
+# 基于PaddleNLP SimpleServing 的服务化部署
+
+## 目录
+- [环境准备](#环境准备)
+- [Server启动服务](#Server服务启动)
+- [其他参数设置](#其他参数设置)
+
+## 环境准备
+使用有SimpleServing功能的PaddleNLP版本
+```shell
+pip install paddlenlp --upgrade
+```
+## Server服务启动
+### 分类任务启动
+#### 启动 分类 Server 服务
+```bash
+paddlenlp server server:app --host 0.0.0.0 --port 8189
+```
+
+#### 分类任务发送服务
+```bash
+python client.py
+```
+
+
+## 其他参数设置
+可以在client端设置 `max_seq_len`, `batch_size`, `prob_limit` 参数
+```python
+    data = {
+        'data': {
+            'text': texts,
+        },
+        'parameters': {
+            'max_seq_len': args.max_seq_len,
+            'batch_size': args.batch_size,
+            'prob_limit': args.prob_limit
+        }
+    }
+```

applications/text_classification/hierarchical/deploy/simple_serving/client.py

@@ -0,0 +1,46 @@
+# Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import argparse
+import requests
+import json
+
+# yapf: disable
+parser = argparse.ArgumentParser()
+parser.add_argument("--max_seq_len", default=128, type=int, help="The maximum total input sequence length after tokenization.")
+parser.add_argument("--batch_size", default=1, type=int, help="Batch size per GPU/CPU for predicting.")
+parser.add_argument("--prob_limit", default=0.5, type=float, help="The limitation of probability for the label.")
+args = parser.parse_args()
+# yapf: enable
+
+url = "http://0.0.0.0:8189/models/cls_hierarchical"
+headers = {"Content-Type": "application/json"}
+
+if __name__ == "__main__":
+    texts = [
+        '请问木竭胶囊能同高血压药、氨糖同时服吗？', '低压100*高压140*头涨，想吃点降压药。谢谢！', '脑穿通畸形易发人群有哪些',
+        '幼儿乱吃丙硫氧嘧啶片怎么办，我也不知道她吃了几片', '如果是可以降血糖的话,血糖值7点多的大概需要吃几个疗程?'
+    ]
+    data = {
+        'data': {
+            'text': texts,
+        },
+        'parameters': {
+            'max_seq_len': args.max_seq_len,
+            'batch_size': args.batch_size,
+            'prob_limit': args.prob_limit
+        }
+    }
+    r = requests.post(url=url, headers=headers, data=json.dumps(data))
+    print(r.text)

applications/text_classification/hierarchical/deploy/simple_serving/server.py

@@ -0,0 +1,23 @@
+# Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from paddlenlp import SimpleServer
+from paddlenlp.server import CustomModelHandler, MultiLabelClassificationPostHandler
+
+app = SimpleServer()
+app.register('cls_hierarchical',
+             model_path="../../export",
+             tokenizer_name='ernie-3.0-medium-zh',
+             model_handler=CustomModelHandler,
+             post_handler=MultiLabelClassificationPostHandler)

applications/text_classification/hierarchical/deploy/triton_serving/README.md

@@ -48,13 +48,13 @@ python3 -m pip install paddlepaddle-gpu paddlenlp -i https://mirror.baidu.com/py
 3. 更多关于PaddleNLP安装的详细教程请查看[Installation](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/docs/get_started/installation.rst)。
 
 
-### 安装FasterTokenizers文本处理加速库（可选）
+### 安装FastTokenizers文本处理加速库（可选）
 
-推荐安装faster_tokenizers可以得到更极致的文本处理效率，进一步提升服务性能。
+推荐安装fast_tokenizer可以得到更极致的文本处理效率，进一步提升服务性能。
 
-在容器内安装 faster_tokenizers
+在容器内安装 fast_tokenizer
 ```shell
-python3 -m pip install faster_tokenizers
+python3 -m pip install fast-tokenizer-python
 ```
 
 
@@ -105,7 +105,7 @@ models
     └── config.pbtxt
 ```
 
-模型配置文件config.pbtxt配置细节请参见[Triton Server Model Configuration](https://github.com/triton-inference-server/server/blob/main/docs/model_configuration.md)
+模型配置文件config.pbtxt配置细节请参见[Triton Server Model Configuration](https://github.com/triton-inference-server/server/blob/main/docs/user_guide/model_configuration.md)
 
 ## 部署模型
 

applications/text_classification/hierarchical/deploy/triton_serving/models/tokenizer/1/model.py

@@ -48,7 +48,7 @@ class TritonPythonModel(object):
           * model_name: Model name
         """
         self.tokenizer = AutoTokenizer.from_pretrained("ernie-3.0-medium-zh",
-                                                       use_faster=True)
+                                                       use_fast=True)
         # You must parse model_config. JSON string is not parsed here
         self.model_config = json.loads(args['model_config'])
         print("model_config:", self.model_config)

applications/text_classification/hierarchical/few-shot/README.md

@@ -65,9 +65,9 @@
 
    内存: 630 GB
 
-3. PaddlePaddle 版本：2.3.1
+3. PaddlePaddle 版本：2.4rc
 
-4. PaddleNLP 版本：2.3.5 (develop)
+4. PaddleNLP 版本：2.4.3
 
 5. 评估设置
 
@@ -77,6 +77,7 @@
   - 微调
 
     ```
+    cd ../
     python train.py --dataset_dir "./data/" --save_dir "./checkpoints" --max_seq_length 128 --model_name "ernie-3.0-base-zh" --batch_size 8 --learning_rate 3e-5 --epochs 100 --logging_steps 5 --early_stop --early_stop_num 20
     ```
 
@@ -91,7 +92,7 @@
    | model_name | 训练方式 | Micro F1分数 | Macro F1分数 |
    | ---------- | ------- | ----------- | ----------- |
    | ernie-3.0-base-zh | 微调学习 | 0.7172 | 0.3821 |
-   | ernie-3.0-base-zh | 提示学习 | 0.8855 | 0.8443 |
+   | ernie-3.0-base-zh | 提示学习 | 0.8945 | 0.8516 |
 
 
 <a name="定制训练"></a>
@@ -102,10 +103,10 @@
 <a name="运行环境"></a>
 ### 3.1 运行环境
 
-- python >= 3.6
-- paddlepaddle > 2.3（2.4版本发布前推荐安装[develop版本](https://www.paddlepaddle.org.cn/install/quick?docurl=/documentation/docs/zh/develop/install/pip/linux-pip.html)）
-- paddlenlp >= 2.3.5
-- paddle2onnx >= 1.0.0rc3
+- python >= 3.7
+- paddlepaddle >= 2.4rc
+- paddlenlp >= 2.4.3
+- paddle2onnx >= 1.0.3
 
 <a name="代码结构"></a>
 ### 3.2 代码结构
@@ -222,12 +223,12 @@ python train.py \
 --do_export \
 --num_train_epochs 100 \
 --logging_steps 5 \
+--save_total_limit 1 \
 --per_device_eval_batch_size 32 \
 --per_device_train_batch_size 8 \
 --metric_for_best_model macro_f1_score \
 --load_best_model_at_end \
---evaluation_strategy epoch \
---save_strategy epoch
+--eval_steps 100
 ```
 **多卡训练**
 
@@ -247,12 +248,12 @@ python -u -m paddle.distributed.launch --gpus 0,1,2,3 train.py \
 --do_export \
 --num_train_epochs 100 \
 --logging_steps 5 \
+--save_total_limit 1 \
 --per_device_eval_batch_size 32 \
 --per_device_train_batch_size 8 \
 --metric_for_best_model macro_f1_score \
 --load_best_model_at_end \
---evaluation_strategy epoch \
---save_strategy epoch
+--eval_steps 100
 ```
 
 可配置参数说明：
@@ -273,6 +274,7 @@ python -u -m paddle.distributed.launch --gpus 0,1,2,3 train.py \
 - `do_export`: 是否在运行结束时将模型导出为静态图，保存路径为`output_dir/export`。
 - `num_train_epochs`: 训练的最大轮数。
 - `max_steps`: 训练的最大步数。此设置将会覆盖`num_train_epochs`。
+- `save_total_limit`: 模型检查点保存数量。
 - `device`: 使用的设备，默认为`gpu`。
 - `eval_steps`: 评估模型的间隔步数。
 - `logging_steps`: 打印日志的间隔步数。
@@ -352,9 +354,9 @@ python infer.py --model_path_prefix checkpoints/export/model --data_dir ./data -
 可配置参数说明：
 
 - `model_path_prefix`: 导出的静态图模型路径及文件前缀。
-- `model_name_or_path`: 内置预训练模型名，或者模型参数配置目录路径，用于加载tokenizer。默认为`ernie-3.0-base-zh`。
+- `model_name`: 内置预训练模型名，用于加载tokenizer。默认为`ernie-3.0-base-zh`。
 - `data_dir`: 待推理数据所在路径，数据应存放在该目录下的`data.txt`文件。
-- `max_seq_length`: 最大句子长度，超过该长度的文本将被截断，不足的以Pad补全。提示文本不会被截断。
+- `max_length`: 最大句子长度，超过该长度的文本将被截断，不足的以Pad补全。提示文本不会被截断。
 - `batch_size`: 每次预测的样本数量。
 - `device`: 选择推理设备，包括`cpu`和`gpu`。默认为`gpu`。
 - `device_id`: 指定GPU设备ID。

applications/text_classification/hierarchical/few-shot/infer.py

@@ -14,23 +14,24 @@
 
 import os
 import six
+import json
 import psutil
 import argparse
 
 import numpy as np
 
 from paddlenlp.utils.log import logger
-from paddlenlp.prompt import AutoTemplate, Verbalizer, InputExample
-from paddlenlp.transformers import AutoTokenizer
+from paddlenlp.prompt import AutoTemplate, PromptDataCollatorWithPadding
+from paddlenlp.transformers import AutoTokenizer, AutoModelForMaskedLM
 import paddle2onnx
 import onnxruntime as ort
 
 # yapf: disable
 parser = argparse.ArgumentParser()
 parser.add_argument("--model_path_prefix", type=str, required=True, help="The path prefix of inference model to be used.")
-parser.add_argument("--model_name_or_path", default="ernie-3.0-base-zh", type=str, help="The directory or name of model.")
+parser.add_argument("--model_name", default="ernie-3.0-base-zh", type=str, help="The name of pretrained model.")
 parser.add_argument("--data_dir", default=None, type=str, help="The path to the prediction data, including label.txt and data.txt.")
-parser.add_argument("--max_seq_length", default=128, type=int, help="The maximum total input sequence length after tokenization.")
+parser.add_argument("--max_length", default=128, type=int, help="The maximum total input sequence length after tokenization.")
 parser.add_argument("--use_fp16", action='store_true', help="Whether to use fp16 inference, only takes effect when deploying on gpu.")
 parser.add_argument("--batch_size", default=200, type=int, help="Batch size per GPU/CPU for predicting.")
 parser.add_argument("--num_threads", default=psutil.cpu_count(logical=False), type=int, help="num_threads for cpu.")
@@ -103,12 +104,6 @@ class InferBackend(object):
                                                   'device_id':
                                                   device_id
                                               }])
-        self.input_handles = [
-            self.predictor.get_inputs()[0].name,
-            self.predictor.get_inputs()[1].name,
-            self.predictor.get_inputs()[2].name
-        ]
-
         if device == "gpu":
             try:
                 assert 'CUDAExecutionProvider' in self.predictor.get_providers()
@@ -122,27 +117,52 @@ class InferBackend(object):
         logger.info(">>> [InferBackend] Engine Created ...")
 
     def infer(self, input_dict: dict):
-        input_dict = {
-            k: v
-            for k, v in input_dict.items() if k in self.input_handles
-        }
         result = self.predictor.run(None, input_dict)
         return result
 
 
 class HierachicalPredictor(object):
 
-    def __init__(self, args, label_list):
-        self._label_list = label_list
-        self._tokenizer = AutoTokenizer.from_pretrained(args.model_name_or_path)
-        self._max_seq_length = args.max_seq_length
-        self._batch_size = args.batch_size
-        self.inference_backend = InferBackend(args.model_path_prefix,
-                                              args.device, args.device_id,
-                                              args.use_fp16, args.num_threads)
-        self._template = AutoTemplate.load_from(
-            os.path.dirname(args.model_path_prefix), self._tokenizer,
-            args.max_seq_length)
+    def __init__(self, args):
+        self.args = args
+        self.tokenizer = AutoTokenizer.from_pretrained(args.model_name)
+        self.model = AutoModelForMaskedLM.from_pretrained(args.model_name)
+        self.template, self.labels, self.input_handles = self.post_init()
+        self.collate_fn = PromptDataCollatorWithPadding(
+            self.tokenizer,
+            padding=True,
+            return_tensors="np",
+            return_attention_mask=True)
+
+        self.inference_backend = InferBackend(self.args.model_path_prefix,
+                                              self.args.device,
+                                              self.args.device_id,
+                                              self.args.use_fp16,
+                                              self.args.num_threads)
+
+    def post_init(self):
+        export_path = os.path.dirname(self.args.model_path_prefix)
+        template_path = os.path.join(export_path, "template_config.json")
+        with open(template_path, "r") as fp:
+            prompt = json.load(fp)
+            template = AutoTemplate.create_from(prompt, self.tokenizer,
+                                                self.args.max_length,
+                                                self.model)
+        keywords = template.extract_template_keywords(template.prompt)
+        inputs = [
+            "input_ids", "token_type_ids", "position_ids", "attention_mask",
+            "masked_positions"
+        ]
+        if "soft" in keywords:
+            inputs.append("soft_token_ids")
+        if "encoder" in keywords:
+            inputs.append("encoder_ids")
+        verbalizer_path = os.path.join(export_path, "verbalizer_config.json")
+        with open(verbalizer_path, "r") as fp:
+            label_words = json.load(fp)
+            labels = sorted(list(label_words.keys()))
+
+        return template, labels, inputs
 
     def predict(self, input_data: list):
         encoded_inputs = self.preprocess(input_data)
@@ -155,14 +175,27 @@ class HierachicalPredictor(object):
         infer_data = self.inference_backend.infer(input_dict)
         return infer_data
 
-    def infer_batch(self, encoded_inputs):
-        num_sample = len(encoded_inputs["input_ids"])
+    def infer_batch(self, inputs):
+        num_sample = len(inputs)
         infer_data = None
         num_infer_data = None
-        for idx in range(0, num_sample, self._batch_size):
-            l, r = idx, idx + self._batch_size
-            keys = encoded_inputs.keys()
-            input_dict = {k: encoded_inputs[k][l:r] for k in keys}
+        for index in range(0, num_sample, self.args.batch_size):
+            left, right = index, index + self.args.batch_size
+            batch_dict = self.collate_fn(inputs[left:right])
+            input_dict = {}
+            for key in self.input_handles:
+                value = batch_dict[key]
+                if key == "attention_mask":
+                    if value.ndim == 2:
+                        value = (1 - value[:, np.newaxis, np.newaxis, :]) * -1e4
+                    elif value.ndim != 4:
+                        raise ValueError(
+                            "Expect attention mask with ndim=2 or 4, but get ndim={}"
+                            .format(value.ndim))
+                    value = value.astype("float32")
+                else:
+                    value = value.astype("int64")
+                input_dict[key] = value
             results = self._infer(input_dict)
             if infer_data is None:
                 infer_data = [[x] for x in results]
@@ -175,16 +208,8 @@ class HierachicalPredictor(object):
         return infer_data
 
     def preprocess(self, input_data: list):
-        text = [InputExample(text_a=x) for x in input_data]
-        inputs = [self._template.wrap_one_example(x) for x in text]
-        inputs = {
-            "input_ids":
-            np.array([x["input_ids"] for x in inputs], dtype="int64"),
-            "mask_ids":
-            np.array([x["mask_ids"] for x in inputs], dtype="int64"),
-            "soft_token_ids":
-            np.array([x["soft_token_ids"] for x in inputs], dtype="int64")
-        }
+        text = [{"text_a": x} for x in input_data]
+        inputs = [self.template(x) for x in text]
         return inputs
 
     @staticmethod
@@ -197,7 +222,7 @@ class HierachicalPredictor(object):
         label_ids = np.argwhere(probs > threshold)
         labels = [[] for _ in range(probs.shape[0])]
         for idx, label_id in label_ids:
-            labels[idx].append(self._label_list[label_id])
+            labels[idx].append(self.labels[label_id])
         return {"label": labels}
 
     def printer(self, result, input_data):
@@ -212,12 +237,10 @@ if __name__ == "__main__":
     for arg_name, arg_value in vars(args).items():
         logger.info("{:20}: {}".format(arg_name, arg_value))
 
-    export_path = os.path.dirname(args.model_path_prefix)
-    labels, _ = Verbalizer.load_from(export_path)
+    predictor = HierachicalPredictor(args)
 
     text_dir = os.path.join(args.data_dir, "data.txt")
     with open(text_dir, "r", encoding="utf-8") as f:
         text_list = [x.strip() for x in f.readlines()]
 
-    predictor = HierachicalPredictor(args, labels)
     predictor.predict(text_list)

applications/text_classification/hierarchical/few-shot/requirements_cpu.txt

@@ -0,0 +1,5 @@
+psutil
+paddlepaddle>=2.4rc
+paddlenlp>=2.4.3
+paddle2onnx>=1.0.3
+onnxruntime

applications/text_classification/hierarchical/few-shot/requirements_gpu.txt

@@ -0,0 +1,7 @@
+psutil
+paddlepaddle-gpu>=2.4rc
+paddlenlp>=2.4.3
+paddle2onnx>=1.0.3
+onnxruntime-gpu
+onnx
+onnxconverter-common

applications/text_classification/hierarchical/few-shot/train.py

@@ -15,6 +15,7 @@
 from dataclasses import dataclass, field
 import os
 import sys
+from collections import defaultdict
 
 import paddle
 import paddle.nn.functional as F
@@ -41,9 +42,6 @@ from metric import MetricReport
 class DataArguments:
     data_dir: str = field(default="./data", metadata={"help": "The dataset dictionary includes train.txt, dev.txt, test.txt, label.txt and data.txt (optional) files."})
     prompt: str = field(default=None, metadata={"help": "The input prompt for tuning."})
-    soft_encoder: str = field(default="lstm", metadata={"help": "The encoder type of soft template, `lstm`, `mlp` or None."})
-    encoder_hidden_size: int = field(default=200, metadata={"help": "The dimension of soft embeddings."})
-
 
 @dataclass
 class ModelArguments:
@@ -67,17 +65,20 @@ def main():
     tokenizer = AutoTokenizer.from_pretrained(model_args.model_name_or_path)
 
     # Define the template for preprocess and the verbalizer for postprocess.
-    template = AutoTemplate.create_from(
-        data_args.prompt,
-        tokenizer,
-        training_args.max_seq_length,
-        model=model,
-        prompt_encoder=data_args.soft_encoder,
-        encoder_hidden_size=data_args.encoder_hidden_size)
-    logger.info("Using template: {}".format(template.template))
+    template = AutoTemplate.create_from(data_args.prompt,
+                                        tokenizer,
+                                        training_args.max_seq_length,
+                                        model=model)
+    logger.info("Using template: {}".format(template.prompt))
 
     label_file = os.path.join(data_args.data_dir, "label.txt")
-    verbalizer = SoftVerbalizer.from_file(tokenizer, model, label_file)
+    with open(label_file, "r", encoding="utf-8") as fp:
+        label_words = defaultdict(list)
+        for line in fp:
+            data = line.strip().split("==")
+            word = data[1] if len(data) > 1 else data[0].split("##")[-1]
+            label_words[data[0]].append(word)
+    verbalizer = SoftVerbalizer(label_words, tokenizer, model)
 
     # Load the few-shot datasets.
     train_ds, dev_ds, test_ds = load_local_dataset(
@@ -139,11 +140,24 @@ def main():
 
     # Export static model.
     if training_args.do_export:
+        template = prompt_model.template
+        template_keywords = template.extract_template_keywords(template.prompt)
         input_spec = [
-            InputSpec(shape=[None, None], dtype="int64"),  # input_ids
-            InputSpec(shape=[None, None], dtype="int64"),  # mask_ids
-            InputSpec(shape=[None, None], dtype="int64"),  # soft_token_ids
+            InputSpec(shape=[None, None], dtype="int64"),  # input_ids,
+            InputSpec(shape=[None, None], dtype="int64"),  # token_type_ids
+            InputSpec(shape=[None, None], dtype="int64"),  # position_ids
+            InputSpec(shape=[None, None, None, None],
+                      dtype="float32")  # attention_mask
         ]
+        if "mask" in template_keywords:
+            input_spec.append(InputSpec(shape=[None],
+                                        dtype="int64"))  # masked_positions
+        if "soft" in template_keywords:
+            input_spec.append(InputSpec(shape=[None, None],
+                                        dtype="int64"))  # soft_token_ids
+        if "encoder" in template_keywords:
+            input_spec.append(InputSpec(shape=[None, None],
+                                        dtype="int64"))  # encoder_ids
         export_path = os.path.join(training_args.output_dir, 'export')
         trainer.export_model(export_path,
                              input_spec=input_spec,

applications/text_classification/hierarchical/few-shot/utils.py

@@ -15,7 +15,6 @@
 import os
 
 from paddlenlp.datasets import load_dataset
-from paddlenlp.prompt import InputExample
 
 
 def load_local_dataset(data_path, splits, label_list):
@@ -39,14 +38,14 @@ def load_local_dataset(data_path, splits, label_list):
             for idx, line in enumerate(fp):
                 data = line.strip().split("\t")
                 if len(data) == 1:
-                    yield InputExample(text_a=data[0])
+                    yield {"text_a": data[0]}
                 else:
                     text, label = data
                     label = label.strip().split(",")
                     label = [
                         float(1) if x in label else float(0) for x in label_list
                     ]
-                    yield InputExample(text_a=text, labels=label)
+                    yield {"text_a": text, "labels": label}
 
     split_map = {"train": "train.txt", "dev": "dev.txt", "test": "test.txt"}
     datasets = []

applications/text_classification/multi_class/README.md

@@ -188,67 +188,93 @@ data.txt(待预测数据文件)，需要预测标签的文本数据。
 #### 2.4.1 预训练模型微调
 
 
-使用CPU/GPU训练，默认为GPU训练，使用CPU训练只需将设备参数配置改为`--device "cpu"`：
+使用CPU/GPU训练，默认为GPU训练，使用CPU训练只需将设备参数配置改为`--device cpu`：
 ```shell
 python train.py \
-    --dataset_dir "data" \
-    --device "gpu" \
+    --model_name_or_path ernie-3.0-medium-zh \
+    --data_dir ./data/ \
+    --output_dir checkpoint \
+    --device gpu \
+    --learning_rate 3e-5 \
+    --num_train_epochs 100 \
+    --early_stopping_patience 4 \
     --max_seq_length 128 \
-    --model_name "ernie-3.0-medium-zh" \
-    --batch_size 32 \
-    --early_stop \
-    --epochs 100
+    --per_device_eval_batch_size 32 \
+    --per_device_train_batch_size 32 \
+    --num_train_epochs 100 \
+    --do_train \
+    --do_eval \
+    --metric_for_best_model accuracy \
+    --load_best_model_at_end \
+    --evaluation_strategy epoch \
+    --save_strategy epoch \
+    --save_total_limit 1
 ```
 
 如果在CPU环境下训练，可以指定`nproc_per_node`参数进行多核训练：
 ```shell
-python -m paddle.distributed.launch --nproc_per_node 8 --backend "gloo" train.py \
-    --dataset_dir "data" \
-    --device "cpu" \
+python -m paddle.distributed.launch --nproc_per_node 8 --backend gloo train.py \
+    --model_name_or_path ernie-3.0-medium-zh \
+    --data_dir ./data/ \
+    --output_dir checkpoint \
+    --device cpu \
+    --learning_rate 3e-5 \
+    --num_train_epochs 100 \
     --max_seq_length 128 \
-    --model_name "ernie-3.0-medium-zh" \
-    --batch_size 32 \
-    --early_stop \
-    --epochs 100
+    --per_device_eval_batch_size 32 \
+    --per_device_train_batch_size 32 \
+    --num_train_epochs 100 \
+    --early_stopping_patience 4 \
+    --do_train \
+    --do_eval \
+    --metric_for_best_model accuracy \
+    --load_best_model_at_end \
+    --evaluation_strategy epoch \
+    --save_strategy epoch \
+    --save_total_limit 1
 ```
 
-如果在GPU环境中使用，可以指定`gpus`参数进行单卡/多卡训练。使用多卡训练可以指定多个GPU卡号，例如 --gpus "0,1"。如果设备只有一个GPU卡号默认为0，可使用`nvidia-smi`命令查看GPU使用情况:
+如果在GPU环境中使用，可以指定`gpus`参数进行单卡/多卡训练。使用多卡训练可以指定多个GPU卡号，例如 --gpus 0,1。如果设备只有一个GPU卡号默认为0，可使用`nvidia-smi`命令查看GPU使用情况:
 
 ```shell
 unset CUDA_VISIBLE_DEVICES
-python -m paddle.distributed.launch --gpus "0" train.py \
-    --dataset_dir "data" \
-    --device "gpu" \
+python -m paddle.distributed.launch --gpus 0,1 train.py \
+    --data_dir ./data/ \
+    --output_dir checkpoint \
+    --device cpu \
+    --learning_rate 3e-5 \
+    --num_train_epochs 100 \
     --max_seq_length 128 \
-    --model_name "ernie-3.0-medium-zh" \
-    --batch_size 32 \
-    --early_stop \
-    --epochs 100
+    --per_device_eval_batch_size 32 \
+    --per_device_train_batch_size 32 \
+    --num_train_epochs 100 \
+    --early_stopping_patience 4 \
+    --do_train \
+    --do_eval \
+    --metric_for_best_model accuracy \
+    --load_best_model_at_end \
+    --evaluation_strategy epoch \
+    --save_strategy epoch \
+    --save_total_limit 1
 ```
 
-可支持配置的参数：
+主要的配置的参数为：
+- `model_name_or_path`: 内置模型名，或者模型参数配置目录路径。默认为`ernie-3.0-base-zh`。
+- `data_dir`: 训练数据集路径，数据格式要求详见[数据标注](#数据标注)。
+- `output_dir`: 模型参数、训练日志和静态图导出的保存目录。
+- `max_seq_length`: 最大句子长度，超过该长度的文本将被截断，不足的以Pad补全。提示文本不会被截断。
+- `num_train_epochs`: 训练轮次，使用早停法时可以选择100
+- `early_stopping_patience`: 在设定的早停训练轮次内，模型在开发集上表现不再上升，训练终止；默认为4。
+- `learning_rate`: 预训练语言模型参数基础学习率大小，将与learning rate scheduler产生的值相乘作为当前学习率。
+- `do_train`: 是否进行训练。
+- `do_eval`: 是否进行评估。
+- `device`: 使用的设备，默认为`gpu`。
+- `per_device_train_batch_size`: 每次训练每张卡上的样本数量。可根据实际GPU显存适当调小/调大此配置。
+- `per_device_eval_batch_size`: 每次评估每张卡上的样本数量。可根据实际GPU显存适当调小/调大此配置。
 
-* `device`: 选用什么设备进行训练，选择cpu、gpu、xpu、npu。如使用gpu训练，可使用参数--gpus指定GPU卡号；默认为"gpu"。
-* `dataset_dir`：必须，本地数据集路径，数据集路径中应包含train.txt，dev.txt和label.txt文件;默认为None。
-* `save_dir`：保存训练模型的目录；默认保存在当前目录checkpoint文件夹下。
-* `max_seq_length`：分词器tokenizer使用的最大序列长度，ERNIE模型最大不能超过2048。请根据文本长度选择，通常推荐128、256或512，若出现显存不足，请适当调低这一参数；默认为128。
-* `model_name`：选择预训练模型,可选"ernie-1.0-large-zh-cw","ernie-3.0-xbase-zh", "ernie-3.0-base-zh", "ernie-3.0-medium-zh", "ernie-3.0-micro-zh", "ernie-3.0-mini-zh", "ernie-3.0-nano-zh", "ernie-2.0-base-en", "ernie-2.0-large-en","ernie-m-base","ernie-m-large"；默认为"ernie-3.0-medium-zh"。
-* `batch_size`：批处理大小，请结合显存情况进行调整，若出现显存不足，请适当调低这一参数；默认为32。
-* `learning_rate`：训练最大学习率；默认为3e-5。
-* `epochs`: 训练轮次，使用早停法时可以选择100；默认为10。
-* `early_stop`：选择是否使用早停法(EarlyStopping)，模型在开发集经过一定epoch后精度表现不再上升，训练终止；默认为False。
-* `early_stop_nums`：在设定的早停训练轮次内，模型在开发集上表现不再上升，训练终止；默认为4。
-* `logging_steps`: 训练过程中日志打印的间隔steps数，默认5。
-* `weight_decay`：控制正则项力度的参数，用于防止过拟合，默认为0.0。
-* `warmup`：是否使用学习率warmup策略，使用时应设置适当的训练轮次（epochs）；默认为False。
-* `warmup_steps`：学习率warmup策略的比例数，如果设为1000，则学习率会在1000steps数从0慢慢增长到learning_rate, 而后再缓慢衰减；默认为0。
-* `init_from_ckpt`: 模型初始checkpoint参数地址，默认None。
-* `seed`：随机种子，默认为3。
-* `train_file`：本地数据集中训练集文件名；默认为"train.txt"。
-* `dev_file`：本地数据集中开发集文件名；默认为"dev.txt"。
-* `label_file`：本地数据集中标签集文件名；默认为"label.txt"。
+训练脚本支持所有`TraingArguments`的参数，更多参数介绍可参考[TrainingArguments 参数介绍](https://paddlenlp.readthedocs.io/zh/latest/trainer.html#trainingarguments)。
 
-程序运行时将会自动进行训练，评估。同时训练过程中会自动保存开发集上最佳模型在指定的 `save_dir` 中，保存模型文件结构如下所示：
+程序运行时将会自动进行训练，评估。同时训练过程中会自动保存开发集上最佳模型在指定的 `output_dir` 中，保存模型文件结构如下所示：
 
 ```text
 checkpoint/
@@ -260,8 +286,8 @@ checkpoint/
 
 **NOTE:**
 
-* 如需恢复模型训练，则可以设置 `init_from_ckpt` ， 如 `init_from_ckpt=checkpoint/model_state.pdparams` 。
-* 如需训练英文文本分类任务，只需更换预训练模型参数 `model_name` 。英文训练任务推荐使用"ernie-2.0-base-en"、"ernie-2.0-large-en"。
+* 如需恢复模型训练，则可以设置 `resume_from_checkpoint` ， 如 `resume_from_checkpoint=./checkpoints/checkpoint-217` 。
+* 如需训练英文文本分类任务，只需更换预训练模型参数 `model_name_or_path` 。英文训练任务推荐使用"ernie-2.0-base-en"、"ernie-2.0-large-en"。
 * 英文和中文以外语言的文本分类任务，推荐使用基于96种语言（涵盖法语、日语、韩语、德语、西班牙语等几乎所有常见语言）进行预训练的多语言预训练模型"ernie-m-base"、"ernie-m-large"，详情请参见[ERNIE-M论文](https://arxiv.org/pdf/2012.15674.pdf)。
 
 #### 2.4.2 训练评估与模型优化

applications/text_classification/multi_class/analysis/README.md

@@ -119,7 +119,7 @@ Text	Label	Prediction
 ### 单词级别可解释性分析
 本项目开源模型的词级别可解释性分析Notebook，提供LIME、Integrated Gradient、GradShap 三种分析方法，支持分析微调后模型的预测结果，开发者可以通过更改**数据目录**和**模型目录**在自己的任务中使用Jupyter Notebook进行数据分析。
 
-运行 [word_interpret.ipynb](./word_interpreter.ipynb) 代码，即可分析影响样本预测结果的关键词以及可视化所有词对预测结果的贡献情况，颜色越深代表这个词对预测结果影响越大：
+运行 [word_interpret.ipynb](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/applications/text_classification/multi_class/analysis/README.md) 代码，即可分析影响样本预测结果的关键词以及可视化所有词对预测结果的贡献情况，颜色越深代表这个词对预测结果影响越大：
 <div align="center">
     <img src="https://user-images.githubusercontent.com/63761690/192739675-63145d59-23c6-416f-bf71-998fd4995254.png" width="1000">
 </div>

applications/text_classification/multi_class/analysis/evaluate.py

@@ -156,16 +156,17 @@ def evaluate():
     logger.info("Dev dataset size: {}".format(len(dev_ds)))
     logger.info("Accuracy in dev dataset: {:.2f}%".format(report['accuracy'] *
                                                           100))
-    logger.info("Top-2 accuracy in dev dataset: {:.2f}%".format(
-        top_k_accuracy_score(y_true=labels,
-                             y_score=probs,
-                             k=2,
-                             labels=list(range(len(label_list)))) * 100))
-    logger.info("Top-3 accuracy in dev dataset: {:.2f}%".format(
-        top_k_accuracy_score(y_true=labels,
-                             y_score=probs,
-                             k=3,
-                             labels=list(range(len(label_list)))) * 100))
+    if len(labels) > 2:
+        logger.info("Top-2 accuracy in dev dataset: {:.2f}%".format(
+            top_k_accuracy_score(y_true=labels,
+                                 y_score=probs,
+                                 k=2,
+                                 labels=list(range(len(label_list)))) * 100))
+        logger.info("Top-3 accuracy in dev dataset: {:.2f}%".format(
+            top_k_accuracy_score(y_true=labels,
+                                 y_score=probs,
+                                 k=3,
+                                 labels=list(range(len(label_list)))) * 100))
 
     for i, l in enumerate(label_list):
         logger.info("Class name: {}".format(l))

applications/text_classification/multi_class/analysis/word_interpret.ipynb

@@ -112,6 +112,7 @@
     "            items = line.strip().split('\\t')\n",
     "            if items[0] == 'Text':\n",
     "                continue\n",
+    "            items[0] = items[0][:MAX_LENGTH-2]\n",
     "            if len(items) == 3:\n",
     "                yield {'text': items[0], 'label': items[1], 'predict': items[2]}\n",
     "            elif len(items) == 2:\n",

applications/text_classification/multi_class/deploy/paddle_serving/README.md

@@ -53,10 +53,10 @@ pip install paddle-serving-server-gpu==0.8.3.post112 -i https://pypi.tuna.tsingh
 - 更多wheel包请参考[serving官网文档](https://github.com/PaddlePaddle/Serving/blob/develop/doc/Latest_Packages_CN.md)
 
 
-### 安装FasterTokenizer文本处理加速库（可选）
-推荐安装faster_tokenizer可以得到更极致的文本处理效率，进一步提升服务性能。
+### 安装FastTokenizer文本处理加速库（可选）
+推荐安装fast_tokenizer可以得到更极致的文本处理效率，进一步提升服务性能。
 ```shell
-pip install faster_tokenizer
+pip install fast-tokenizer-python
 ```
 
 
@@ -143,7 +143,7 @@ I0628 09:12:30.739985 74305 analysis_predictor.cc:1007] ======= optimize end ===
 I0628 09:12:30.776288 74305 naive_executor.cc:102] ---  skip [feed], feed -> token_type_ids
 I0628 09:12:30.779004 74305 naive_executor.cc:102] ---  skip [feed], feed -> input_ids
 I0628 09:12:30.787542 74305 naive_executor.cc:102] ---  skip [linear_147.tmp_1], fetch -> fetch
-[2022-06-28 09:12:32,879] [ WARNING] - Can't find the faster_tokenizers package, please ensure install faster_tokenizers correctly. You can install faster_tokenizers by `pip install faster_tokenizers`(Currently only work for linux platform).
+[2022-06-28 09:12:32,879] [ WARNING] - Can't find the fast_tokenizer package, please ensure install fast_tokenizer correctly. You can install fast_tokenizer by `pip install fast-tokenizer-python`.
 [2022-06-28 09:12:32,880] [    INFO] - We are using <class 'paddlenlp.transformers.ernie.tokenizer.ErnieTokenizer'> to load 'ernie-3.0-medium-zh'.
 [2022-06-28 09:12:32,880] [    INFO] - Already cached /root/.paddlenlp/models/ernie-3.0-medium-zh/ernie_3.0_base_zh_vocab.txt
 [OP Object] init success

applications/text_classification/multi_class/deploy/paddle_serving/service.py

@@ -49,7 +49,7 @@ class Op(Op):
 
     def init_op(self):
         self.tokenizer = AutoTokenizer.from_pretrained(args.model_name,
-                                                       use_faster=True)
+                                                       use_fast=True)
         # Output nodes may differ from model to model
         # You can see the output node name in the conf.prototxt file of serving_server
         self.fetch_names = [

applications/text_classification/multi_class/deploy/predictor/README.md

@@ -12,7 +12,7 @@
 
 如果基于GPU部署，请先确保机器已正确安装NVIDIA相关驱动和基础软件，确保CUDA >= 11.2，CuDNN >= 8.2，并使用以下命令安装所需依赖:
 ```
-python -m pip install onnxruntime-gpu onnx onnxconverter-common
+python -m pip install onnxruntime-gpu onnx onnxconverter-common==1.9.0
 ```
 
 如果基于CPU部署，请使用如下命令安装所需依赖:
@@ -20,10 +20,10 @@ python -m pip install onnxruntime-gpu onnx onnxconverter-common
 python -m pip install onnxruntime
 ```
 
-安装FasterTokenizer文本处理加速库（可选）
-推荐安装faster_tokenizer可以得到更极致的文本处理效率，进一步提升服务性能。
+安装FastTokenizer文本处理加速库（可选）
+推荐安装fast_tokenizer可以得到更极致的文本处理效率，进一步提升服务性能。
 ```shell
-pip install faster_tokenizer
+pip install fast-tokenizer-python
 ```
 ## 基于GPU部署推理样例
 

applications/text_classification/multi_class/deploy/predictor/predictor.py

@@ -117,7 +117,7 @@ class Predictor(object):
 
     def __init__(self, args, label_list):
         self.tokenizer = AutoTokenizer.from_pretrained(args.model_name_or_path,
-                                                       use_faster=True)
+                                                       use_fast=True)
         self.label_list = label_list
         self.batch_size = args.batch_size
         self.max_seq_length = args.max_seq_length

applications/text_classification/multi_class/deploy/simple_serving/README.md

@@ -0,0 +1,39 @@
+# 基于PaddleNLP SimpleServing 的服务化部署
+
+## 目录
+- [环境准备](#环境准备)
+- [Server启动服务](#Server服务启动)
+- [其他参数设置](#其他参数设置)
+
+## 环境准备
+使用有SimpleServing功能的PaddleNLP版本
+```shell
+pip install paddlenlp >= 2.4.4
+```
+## Server服务启动
+### 分类任务启动
+#### 启动分类 Server 服务
+```bash
+paddlenlp server server:app --host 0.0.0.0 --port 8189
+```
+
+#### 启动分类 Client 服务
+```bash
+python client.py
+```
+
+
+## 其他参数设置
+可以在client端设置 `max_seq_len`, `batch_size` 参数
+```python
+    data = {
+        'data': {
+            'text': texts,
+            'text_pair': text_pairs if len(text_pairs) > 0 else None
+        },
+        'parameters': {
+            'max_seq_len': args.max_seq_len,
+            'batch_size': args.batch_size
+        }
+    }
+```

applications/text_classification/multi_class/deploy/simple_serving/client.py

@@ -0,0 +1,45 @@
+# Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import argparse
+import requests
+import json
+
+# yapf: disable
+parser = argparse.ArgumentParser()
+parser.add_argument("--max_seq_len", default=128, type=int, help="The maximum total input sequence length after tokenization.")
+parser.add_argument("--batch_size", default=1, type=int, help="Batch size per GPU/CPU for predicting.")
+args = parser.parse_args()
+# yapf: enable
+
+url = "http://0.0.0.0:8189/models/cls_multi_class"
+headers = {"Content-Type": "application/json"}
+
+if __name__ == "__main__":
+    texts = [
+        '黑苦荞茶的功效与作用及食用方法', '交界痣会凸起吗', '检查是否能怀孕挂什么科', '鱼油怎么吃咬破吃还是直接咽下去',
+        '幼儿挑食的生理原因是'
+    ]
+    data = {
+        'data': {
+            'text': texts,
+        },
+        'parameters': {
+            'max_seq_len': args.max_seq_len,
+            'batch_size': args.batch_size
+        }
+    }
+    r = requests.post(url=url, headers=headers, data=json.dumps(data))
+    result_json = json.loads(r.text)
+    print(result_json)

applications/text_classification/multi_class/deploy/simple_serving/server.py

@@ -0,0 +1,23 @@
+# Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from paddlenlp import SimpleServer
+from paddlenlp.server import CustomModelHandler, MultiClassificationPostHandler
+
+app = SimpleServer()
+app.register('cls_multi_class',
+             model_path="../../export",
+             tokenizer_name='ernie-3.0-medium-zh',
+             model_handler=CustomModelHandler,
+             post_handler=MultiClassificationPostHandler)

applications/text_classification/multi_class/deploy/triton_serving/README.md

@@ -48,13 +48,13 @@ python3 -m pip install paddlepaddle-gpu paddlenlp -i https://mirror.baidu.com/py
 3. 更多关于PaddleNLP安装的详细教程请查看[Installation](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/docs/get_started/installation.rst)。
 
 
-### 安装FasterTokenizers文本处理加速库（可选）
+### 安装FastTokenizer文本处理加速库（可选）
 
-部署环境是Linux，推荐安装faster_tokenizers可以得到更极致的文本处理效率，进一步提升服务性能。
+部署环境是Linux，推荐安装fast_tokenizer可以得到更极致的文本处理效率，进一步提升服务性能。
 
-在容器内安装 faster_tokenizers
+在容器内安装 fast_tokenizer
 ```shell
-python3 -m pip install faster_tokenizers
+python3 -m pip install fast-tokenizer-python
 ```
 
 
@@ -102,7 +102,7 @@ models
     └── config.pbtxt
 ```
 
-模型配置文件config.pbtxt配置细节请参见[Triton Server Model Configuration](https://github.com/triton-inference-server/server/blob/main/docs/model_configuration.md)
+模型配置文件config.pbtxt配置细节请参见[Triton Server Model Configuration](https://github.com/triton-inference-server/server/blob/main/docs/user_guide/model_configuration.md)
 
 ## 部署模型
 

applications/text_classification/multi_class/deploy/triton_serving/models/tokenizer/1/model.py

@@ -48,7 +48,7 @@ class TritonPythonModel(object):
           * model_name: Model name
         """
         self.tokenizer = AutoTokenizer.from_pretrained("ernie-3.0-medium-zh",
-                                                       use_faster=True)
+                                                       use_fast=True)
         # You must parse model_config. JSON string is not parsed here
         self.model_config = json.loads(args['model_config'])
         print("model_config:", self.model_config)

applications/text_classification/multi_class/few-shot/README.md

@@ -65,9 +65,9 @@
 
    内存: 630 GB
 
-3. PaddlePaddle 版本：2.3.1
+3. PaddlePaddle 版本：2.4rc
 
-4. PaddleNLP 版本：2.3.5 (develop)
+4. PaddleNLP 版本：2.4.3
 
 5. 评估设置
 
@@ -76,13 +76,14 @@
 - 微调
 
 ```
+cd ../
 python train.py --dataset_dir "./data/" --save_dir "./checkpoints" --max_seq_length 128 --model_name "ernie-3.0-base-zh" --batch_size 8 --learning_rate 3e-5 --epochs 100 --logging_steps 5 --early_stop
 ```
 
 - 提示学习
 
 ```
-python train.py --data_dir ./data/ --output_dir ./checkpoints/ --prompt "这条新闻写的是" --model_name_or_path ernie-3.0-base-zh --max_seq_length 128  --learning_rate 3e-5 --ppt_learning_rate 3e-4 --do_train --do_eval --num_train_epochs 100 --logging_steps 5 --per_device_eval_batch_size 32 --per_device_train_batch_size 8 --do_predict --metric_for_best_model accuracy --load_best_model_at_end --evaluation_strategy epoch --save_strategy epoch
+python train.py --data_dir ./data/ --output_dir ./checkpoints/ --prompt "这条新闻写的是" --model_name_or_path ernie-3.0-base-zh --max_seq_length 128  --learning_rate 3e-5 --ppt_learning_rate 3e-4 --do_train --do_eval --num_train_epochs 100 --logging_steps 5 --per_device_eval_batch_size 32 --per_device_train_batch_size 8 --do_predict --metric_for_best_model accuracy --load_best_model_at_end --evaluation_strategy epoch --save_strategy epoch --save_total_limit 1
 ```
 
 6. 精度评价指标：Accuracy
@@ -102,10 +103,10 @@ python train.py --data_dir ./data/ --output_dir ./checkpoints/ --prompt "这条
 <a name="运行环境"></a>
 ### 3.1 运行环境
 
-- python >= 3.6
-- paddlepaddle > 2.3 （2.4版本发布前推荐安装[develop版本](https://www.paddlepaddle.org.cn/install/quick?docurl=/documentation/docs/zh/develop/install/pip/linux-pip.html)）
-- paddlenlp >= 2.3.5
-- paddle2onnx >= 1.0.0rc3
+- python >= 3.7
+- paddlepaddle >= 2.4rc
+- paddlenlp >= 2.4.3
+- paddle2onnx >= 1.0.3
 
 <a name="代码结构"></a>
 ### 3.2 代码结构
@@ -204,14 +205,15 @@ python train.py \
 --output_dir ./checkpoints/ \
 --prompt "这条新闻标题的主题是" \
 --max_seq_length 128  \
---learning_rate 3e-5 \
---ppt_learning_rate 3e-4 \
+--learning_rate 3e-6 \
+--ppt_learning_rate 3e-5 \
 --do_train \
 --do_eval \
 --use_rdrop \
 --max_steps 1000 \
 --eval_steps 10 \
 --logging_steps 5 \
+--save_total_limit 1 \
 --load_best_model_at_end True \
 --per_device_eval_batch_size 32 \
 --per_device_train_batch_size 8 \
@@ -227,8 +229,8 @@ python -u -m paddle.distributed.launch --gpus 0,1,2,3 train.py \
 --output_dir ./checkpoints/ \
 --prompt "这条新闻标题的主题是" \
 --max_seq_length 128  \
---learning_rate 3e-5 \
---ppt_learning_rate 3e-4 \
+--learning_rate 3e-6 \
+--ppt_learning_rate 3e-5 \
 --do_train \
 --do_eval \
 --use_rdrop \
@@ -236,6 +238,7 @@ python -u -m paddle.distributed.launch --gpus 0,1,2,3 train.py \
 --max_steps 1000 \
 --eval_steps 10 \
 --logging_steps 5 \
+--save_total_limit 1 \
 --load_best_model_at_end True \
 --per_device_eval_batch_size 32 \
 --per_device_train_batch_size 8 \
@@ -260,6 +263,7 @@ python -u -m paddle.distributed.launch --gpus 0,1,2,3 train.py \
 - `do_predict`: 是否进行预测。
 - `do_export`: 是否在运行结束时将模型导出为静态图，保存路径为`output_dir/export`。
 - `max_steps`: 训练的最大步数。此设置将会覆盖`num_train_epochs`。
+- `save_total_limit`: 模型检查点保存数量。
 - `eval_steps`: 评估模型的间隔步数。
 - `device`: 使用的设备，默认为`gpu`。
 - `logging_steps`: 打印日志的间隔步数。
@@ -335,9 +339,9 @@ python infer.py --model_path_prefix checkpoints/export/model --data_dir ./data -
 可配置参数说明：
 
 - `model_path_prefix`: 导出的静态图模型路径及文件前缀。
-- `model_name_or_path`: 内置预训练模型名，或者模型参数配置目录路径，用于加载tokenizer。默认为`ernie-3.0-base-zh`。
+- `model_name`: 内置预训练模型名，用于加载tokenizer。默认为`ernie-3.0-base-zh`。
 - `data_dir`: 待推理数据所在路径，数据应存放在该目录下的`data.txt`文件。
-- `max_seq_length`: 最大句子长度，超过该长度的文本将被截断，不足的以Pad补全。提示文本不会被截断。
+- `max_length`: 最大句子长度，超过该长度的文本将被截断，不足的以Pad补全。提示文本不会被截断。
 - `batch_size`: 每次预测的样本数量。
 - `device`: 选择推理设备，包括`cpu`和`gpu`。默认为`gpu`。
 - `device_id`: 指定GPU设备ID。

applications/text_classification/multi_class/few-shot/infer.py

@@ -14,23 +14,24 @@
 
 import os
 import six
+import json
 import psutil
 import argparse
 
 import numpy as np
 
 from paddlenlp.utils.log import logger
-from paddlenlp.prompt import AutoTemplate, Verbalizer, InputExample
-from paddlenlp.transformers import AutoTokenizer
+from paddlenlp.prompt import AutoTemplate, PromptDataCollatorWithPadding
+from paddlenlp.transformers import AutoTokenizer, AutoModelForMaskedLM
 import paddle2onnx
 import onnxruntime as ort
 
 # yapf: disable
 parser = argparse.ArgumentParser()
 parser.add_argument("--model_path_prefix", type=str, required=True, help="The path prefix of inference model to be used.")
-parser.add_argument("--model_name_or_path", default="ernie-3.0-base-zh", type=str, help="The directory or name of model.")
+parser.add_argument("--model_name", default="ernie-3.0-base-zh", type=str, help="The name of pretrained model.")
 parser.add_argument("--data_dir", default=None, type=str, help="The path to the prediction data, including label.txt and data.txt.")
-parser.add_argument("--max_seq_length", default=128, type=int, help="The maximum total input sequence length after tokenization.")
+parser.add_argument("--max_length", default=128, type=int, help="The maximum total input sequence length after tokenization.")
 parser.add_argument("--use_fp16", action='store_true', help="Whether to use fp16 inference, only takes effect when deploying on gpu.")
 parser.add_argument("--batch_size", default=200, type=int, help="Batch size per GPU/CPU for predicting.")
 parser.add_argument("--num_threads", default=psutil.cpu_count(logical=False), type=int, help="num_threads for cpu.")
@@ -103,11 +104,6 @@ class InferBackend(object):
                                                   'device_id':
                                                   device_id
                                               }])
-        self.input_handles = [
-            self.predictor.get_inputs()[0].name,
-            self.predictor.get_inputs()[1].name,
-            self.predictor.get_inputs()[2].name
-        ]
 
         if device == "gpu":
             try:
@@ -122,27 +118,53 @@ class InferBackend(object):
         logger.info(">>> [InferBackend] Engine Created ...")
 
     def infer(self, input_dict: dict):
-        input_dict = {
-            k: v
-            for k, v in input_dict.items() if k in self.input_handles
-        }
         result = self.predictor.run(None, input_dict)
         return result
 
 
 class MultiClassPredictor(object):
 
-    def __init__(self, args, label_list):
-        self._label_list = label_list
-        self._tokenizer = AutoTokenizer.from_pretrained(args.model_name_or_path)
-        self._max_seq_length = args.max_seq_length
-        self._batch_size = args.batch_size
-        self.inference_backend = InferBackend(args.model_path_prefix,
-                                              args.device, args.device_id,
-                                              args.use_fp16, args.num_threads)
-        self._template = AutoTemplate.load_from(
-            os.path.dirname(args.model_path_prefix), self._tokenizer,
-            args.max_seq_length)
+    def __init__(self, args):
+        self.args = args
+        self.tokenizer = AutoTokenizer.from_pretrained(args.model_name)
+        self.model = AutoModelForMaskedLM.from_pretrained(args.model_name)
+        self.template, self.labels, self.input_handles = self.post_init()
+        self.collate_fn = PromptDataCollatorWithPadding(
+            self.tokenizer,
+            padding=True,
+            return_tensors="np",
+            return_attention_mask=True)
+
+        self.inference_backend = InferBackend(self.args.model_path_prefix,
+                                              self.args.device,
+                                              self.args.device_id,
+                                              self.args.use_fp16,
+                                              self.args.num_threads)
+
+    def post_init(self):
+        export_path = os.path.dirname(self.args.model_path_prefix)
+        template_path = os.path.join(export_path, "template_config.json")
+        with open(template_path, "r") as fp:
+            prompt = json.load(fp)
+            template = AutoTemplate.create_from(prompt, self.tokenizer,
+                                                self.args.max_length,
+                                                self.model)
+        keywords = template.extract_template_keywords(template.prompt)
+        inputs = [
+            "input_ids", "token_type_ids", "position_ids", "attention_mask"
+        ]
+        if "mask" in keywords:
+            inputs.append("masked_positions")
+        if "soft" in keywords:
+            inputs.append("soft_token_ids")
+        if "encoder" in keywords:
+            inputs.append("encoder_ids")
+        verbalizer_path = os.path.join(export_path, "verbalizer_config.json")
+        with open(verbalizer_path, "r") as fp:
+            label_words = json.load(fp)
+            labels = sorted(list(label_words.keys()))
+
+        return template, labels, inputs
 
     def predict(self, input_data: list):
         encoded_inputs = self.preprocess(input_data)
@@ -155,14 +177,27 @@ class MultiClassPredictor(object):
         infer_data = self.inference_backend.infer(input_dict)
         return infer_data
 
-    def infer_batch(self, encoded_inputs):
-        num_sample = len(encoded_inputs["input_ids"])
+    def infer_batch(self, inputs):
+        num_sample = len(inputs)
         infer_data = None
         num_infer_data = None
-        for idx in range(0, num_sample, self._batch_size):
-            l, r = idx, idx + self._batch_size
-            keys = encoded_inputs.keys()
-            input_dict = {k: encoded_inputs[k][l:r] for k in keys}
+        for index in range(0, num_sample, self.args.batch_size):
+            left, right = index, index + self.args.batch_size
+            batch_dict = self.collate_fn(inputs[left:right])
+            input_dict = {}
+            for key in self.input_handles:
+                value = batch_dict[key]
+                if key == "attention_mask":
+                    if value.ndim == 2:
+                        value = (1 - value[:, np.newaxis, np.newaxis, :]) * -1e4
+                    elif value.ndim != 4:
+                        raise ValueError(
+                            "Expect attention mask with ndim=2 or 4, but get ndim={}"
+                            .format(value.ndim))
+                    value = value.astype("float32")
+                else:
+                    value = value.astype("int64")
+                input_dict[key] = value
             results = self._infer(input_dict)
             if infer_data is None:
                 infer_data = [[x] for x in results]
@@ -175,21 +210,13 @@ class MultiClassPredictor(object):
         return infer_data
 
     def preprocess(self, input_data: list):
-        text = [InputExample(text_a=x) for x in input_data]
-        inputs = [self._template.wrap_one_example(x) for x in text]
-        inputs = {
-            "input_ids":
-            np.array([x["input_ids"] for x in inputs], dtype="int64"),
-            "mask_ids":
-            np.array([x["mask_ids"] for x in inputs], dtype="int64"),
-            "soft_token_ids":
-            np.array([x["soft_token_ids"] for x in inputs], dtype="int64")
-        }
+        text = [{"text_a": x} for x in input_data]
+        inputs = [self.template(x) for x in text]
         return inputs
 
     def postprocess(self, infer_data):
         preds = np.argmax(infer_data[0], axis=-1)
-        labels = [self._label_list[x] for x in preds]
+        labels = [self.labels[x] for x in preds]
         return {"label": labels}
 
     def printer(self, result, input_data):
@@ -204,12 +231,10 @@ if __name__ == "__main__":
     for arg_name, arg_value in vars(args).items():
         logger.info("{:20}: {}".format(arg_name, arg_value))
 
-    export_path = os.path.dirname(args.model_path_prefix)
-    labels, _ = Verbalizer.load_from(export_path)
+    predictor = MultiClassPredictor(args)
 
     text_dir = os.path.join(args.data_dir, "data.txt")
     with open(text_dir, "r", encoding="utf-8") as f:
         text_list = [x.strip() for x in f.readlines()]
 
-    predictor = MultiClassPredictor(args, labels)
     predictor.predict(text_list)

applications/text_classification/multi_class/few-shot/requirements_cpu.txt

@@ -0,0 +1,5 @@
+psutil
+paddlepaddle>=2.4rc
+paddlenlp>=2.4.3
+paddle2onnx>=1.0.3
+onnxruntime

applications/text_classification/multi_class/few-shot/requirements_gpu.txt

@@ -0,0 +1,7 @@
+psutil
+paddlepaddle-gpu>=2.4rc
+paddlenlp>=2.4.3
+paddle2onnx>=1.0.3
+onnxruntime-gpu
+onnx
+onnxconverter-common

applications/text_classification/multi_class/few-shot/train.py

@@ -14,6 +14,9 @@
 
 from dataclasses import dataclass, field
 import os
+from collections import defaultdict
+
+import numpy as np
 
 import paddle
 from paddle.static import InputSpec
@@ -37,8 +40,6 @@ from utils import load_local_dataset
 class DataArguments:
     data_dir: str = field(default="./data/", metadata={"help": "Path to a dataset which includes train.txt, dev.txt, test.txt, label.txt and data.txt (optional)."})
     prompt: str = field(default=None, metadata={"help": "The input prompt for tuning."})
-    soft_encoder: str = field(default="lstm", metadata={"help": "The encoder type of soft template, `lstm`, `mlp` or None."})
-    encoder_hidden_size: int = field(default=200, metadata={"help": "The dimension of soft embeddings."})
 
 
 @dataclass
@@ -63,17 +64,20 @@ def main():
     tokenizer = AutoTokenizer.from_pretrained(model_args.model_name_or_path)
 
     # Define the template for preprocess and the verbalizer for postprocess.
-    template = AutoTemplate.create_from(
-        data_args.prompt,
-        tokenizer,
-        training_args.max_seq_length,
-        model=model,
-        prompt_encoder=data_args.soft_encoder,
-        encoder_hidden_size=data_args.encoder_hidden_size)
-    logger.info("Using template: {}".format(template.template))
+    template = AutoTemplate.create_from(data_args.prompt,
+                                        tokenizer,
+                                        training_args.max_seq_length,
+                                        model=model)
+    logger.info("Using template: {}".format(template.prompt))
 
     label_file = os.path.join(data_args.data_dir, "label.txt")
-    verbalizer = SoftVerbalizer.from_file(tokenizer, model, label_file)
+    with open(label_file, "r", encoding="utf-8") as fp:
+        label_words = defaultdict(list)
+        for line in fp:
+            data = line.strip().split("==")
+            word = data[1] if len(data) > 1 else data[0].split("##")[-1]
+            label_words[data[0]].append(word)
+    verbalizer = SoftVerbalizer(label_words, tokenizer, model)
 
     # Load the few-shot datasets.
     train_ds, dev_ds, test_ds = load_local_dataset(
@@ -133,11 +137,24 @@ def main():
 
     # Export static model.
     if training_args.do_export:
+        template = prompt_model.template
+        template_keywords = template.extract_template_keywords(template.prompt)
         input_spec = [
-            InputSpec(shape=[None, None], dtype="int64"),  # input_ids
-            InputSpec(shape=[None, None], dtype="int64"),  # mask_ids
-            InputSpec(shape=[None, None], dtype="int64"),  # soft_token_ids
+            InputSpec(shape=[None, None], dtype="int64"),  # input_ids,
+            InputSpec(shape=[None, None], dtype="int64"),  # token_type_ids
+            InputSpec(shape=[None, None], dtype="int64"),  # position_ids
+            InputSpec(shape=[None, None, None, None],
+                      dtype="float32")  # attention_mask
         ]
+        if "mask" in template_keywords:
+            input_spec.append(InputSpec(shape=[None],
+                                        dtype="int64"))  # masked_positions
+        if "soft" in template_keywords:
+            input_spec.append(InputSpec(shape=[None, None],
+                                        dtype="int64"))  # soft_token_ids
+        if "encoder" in template_keywords:
+            input_spec.append(InputSpec(shape=[None, None],
+                                        dtype="int64"))  # encoder_ids
         export_path = os.path.join(training_args.output_dir, 'export')
         trainer.export_model(export_path,
                              input_spec=input_spec,

applications/text_classification/multi_class/few-shot/utils.py

@@ -15,7 +15,6 @@
 import os
 
 from paddlenlp.datasets import load_dataset
-from paddlenlp.prompt import InputExample
 
 
 def load_local_dataset(data_path, splits, label_list):
@@ -38,10 +37,10 @@ def load_local_dataset(data_path, splits, label_list):
             for idx, line in enumerate(fp):
                 data = line.strip().split("\t")
                 if len(data) == 1:
-                    yield InputExample(text_a=data[0])
+                    yield {"text_a": data[0]}
                 else:
                     text, label = data
-                    yield InputExample(text_a=text, labels=label_list[label])
+                    yield {"text_a": text, "labels": label_list[label]}
 
     assert isinstance(splits, list) and len(splits) > 0
 

applications/text_classification/multi_class/train.py

@@ -12,206 +12,128 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-import re
-import json
 import functools
-import random
-import time
 import os
-import argparse
-
-import numpy as np
+from dataclasses import dataclass, field
 
 import paddle
-import paddle.nn.functional as F
 from paddle.metric import Accuracy
-from paddle.io import DataLoader, BatchSampler, DistributedBatchSampler
 from paddlenlp.data import DataCollatorWithPadding
 from paddlenlp.datasets import load_dataset
-from paddlenlp.transformers import AutoModelForSequenceClassification, AutoTokenizer, LinearDecayWithWarmup
-from paddlenlp.utils.log import logger
+from paddlenlp.trainer import PdArgumentParser, EarlyStoppingCallback, TrainingArguments, Trainer
+from paddlenlp.transformers import AutoModelForSequenceClassification, AutoTokenizer
+
+from utils import preprocess_function, read_local_dataset
+
+SUPPORTED_MODELS = [
+    "ernie-1.0-large-zh-cw", "ernie-3.0-xbase-zh", "ernie-3.0-base-zh",
+    "ernie-3.0-medium-zh", "ernie-3.0-micro-zh", "ernie-3.0-mini-zh",
+    "ernie-3.0-nano-zh", "ernie-2.0-base-en", "ernie-2.0-large-en",
+    "ernie-m-base", "ernie-m-large"
+]
 
-from utils import evaluate, preprocess_function, read_local_dataset
 
 # yapf: disable
-parser = argparse.ArgumentParser()
-parser.add_argument('--device', default="gpu", help="Select which device to train model, defaults to gpu.")
-parser.add_argument("--dataset_dir", required=True, default=None, type=str, help="Local dataset directory should include train.txt, dev.txt and label.txt")
-parser.add_argument("--save_dir", default="./checkpoint", type=str, help="The output directory where the model checkpoints will be written.")
-parser.add_argument("--max_seq_length", default=128, type=int, help="The maximum total input sequence length after tokenization. Sequences longer than this will be truncated, sequences shorter will be padded.")
-parser.add_argument('--model_name', default="ernie-3.0-medium-zh", help="Select model to train, defaults to ernie-3.0-medium-zh.",
-                    choices=["ernie-1.0-large-zh-cw","ernie-3.0-xbase-zh", "ernie-3.0-base-zh", "ernie-3.0-medium-zh", "ernie-3.0-micro-zh", "ernie-3.0-mini-zh", "ernie-3.0-nano-zh", "ernie-2.0-base-en", "ernie-2.0-large-en","ernie-m-base","ernie-m-large"])
-parser.add_argument("--batch_size", default=32, type=int, help="Batch size per GPU/CPU for training.")
-parser.add_argument("--learning_rate", default=3e-5, type=float, help="The initial learning rate for Adam.")
-parser.add_argument("--epochs", default=10, type=int, help="Total number of training epochs to perform.")
-parser.add_argument('--early_stop', action='store_true', help='Epoch before early stop.')
-parser.add_argument('--early_stop_nums', type=int, default=3, help='Number of epoch before early stop.')
-parser.add_argument("--logging_steps", default=5, type=int, help="The interval steps to logging.")
-parser.add_argument("--weight_decay", default=0.0, type=float, help="Weight decay if we apply some.")
-parser.add_argument('--warmup', action='store_true', help="whether use warmup strategy")
-parser.add_argument("--warmup_steps", default=0, type=int, help="Linear warmup steps over the training process.")
-parser.add_argument("--init_from_ckpt", type=str, default=None, help="The path of checkpoint to be loaded.")
-parser.add_argument("--seed", type=int, default=3, help="random seed for initialization")
-parser.add_argument("--train_file", type=str, default="train.txt", help="Train dataset file name")
-parser.add_argument("--dev_file", type=str, default="dev.txt", help="Dev dataset file name")
-parser.add_argument("--label_file", type=str, default="label.txt", help="Label file name")
-args = parser.parse_args()
+@dataclass
+class DataArguments:
+    data_dir: str = field(default="./data/", metadata={"help": "Path to a dataset which includes train.txt, dev.txt, test.txt, label.txt and data.txt (optional)."})
+    max_seq_length: int = field(default=128, metadata={"help": "Maximum number of tokens for the model"})
+    early_stopping_patience: int = field(default=4, metadata={"help": "Stop training when the specified metric worsens for early_stopping_patience evaluation calls"})
+
+@dataclass
+class ModelArguments:
+    model_name_or_path: str = field(default="ernie-3.0-base-zh", metadata={"help": "Build-in pretrained model name or the path to local model."})
 # yapf: enable
 
 
-def set_seed(seed):
-    """
-    Sets random seed
-    """
-    random.seed(seed)
-    np.random.seed(seed)
-    paddle.seed(seed)
-    os.environ['PYTHONHASHSEED'] = str(seed)
-
-
-def args_saving():
-    argsDict = args.__dict__
-    with open(os.path.join(args.save_dir, 'setting.txt'), 'w') as f:
-        f.writelines('------------------ start ------------------' + '\n')
-        for eachArg, value in argsDict.items():
-            f.writelines(eachArg + ' : ' + str(value) + '\n')
-        f.writelines('------------------- end -------------------')
-
-
-def train():
+def main():
     """
     Training a binary or multi classification model
     """
 
-    if not os.path.exists(args.save_dir):
-        os.makedirs(args.save_dir)
-    args_saving()
-    set_seed(args.seed)
-    paddle.set_device(args.device)
+    parser = PdArgumentParser(
+        (ModelArguments, DataArguments, TrainingArguments))
+    model_args, data_args, training_args = parser.parse_args_into_dataclasses()
+    training_args.print_config(model_args, "Model")
+    training_args.print_config(data_args, "Data")
+    print(not os.path.isdir(model_args.model_name_or_path))
+    print(model_args.model_name_or_path not in SUPPORTED_MODELS)
+    if not os.path.isdir(
+            model_args.model_name_or_path
+    ) and model_args.model_name_or_path not in SUPPORTED_MODELS:
+        raise ValueError(
+            f"{model_args.model_name_or_path} is not a supported model type. Either use a local model path or select a model from {SUPPORTED_MODELS}"
+        )
 
-    rank = paddle.distributed.get_rank()
-    if paddle.distributed.get_world_size() > 1:
-        paddle.distributed.init_parallel_env()
+    paddle.set_device(training_args.device)
 
     # load and preprocess dataset
-
     label_list = {}
-    with open(os.path.join(args.dataset_dir, args.label_file),
+    with open(os.path.join(data_args.data_dir, "label.txt"),
               'r',
               encoding='utf-8') as f:
         for i, line in enumerate(f):
             l = line.strip()
             label_list[l] = i
+
     train_ds = load_dataset(read_local_dataset,
-                            path=os.path.join(args.dataset_dir,
-                                              args.train_file),
+                            path=os.path.join(data_args.data_dir, "train.txt"),
                             label_list=label_list,
                             lazy=False)
     dev_ds = load_dataset(read_local_dataset,
-                          path=os.path.join(args.dataset_dir, args.dev_file),
+                          path=os.path.join(data_args.data_dir, "dev.txt"),
                           label_list=label_list,
                           lazy=False)
 
-    tokenizer = AutoTokenizer.from_pretrained(args.model_name)
+    tokenizer = AutoTokenizer.from_pretrained(model_args.model_name_or_path)
     trans_func = functools.partial(preprocess_function,
                                    tokenizer=tokenizer,
-                                   max_seq_length=args.max_seq_length)
+                                   max_seq_length=data_args.max_seq_length)
     train_ds = train_ds.map(trans_func)
     dev_ds = dev_ds.map(trans_func)
-    # batchify dataset
-    collate_fn = DataCollatorWithPadding(tokenizer)
-    if paddle.distributed.get_world_size() > 1:
-        train_batch_sampler = DistributedBatchSampler(
-            train_ds, batch_size=args.batch_size, shuffle=True)
-    else:
-        train_batch_sampler = BatchSampler(train_ds,
-                                           batch_size=args.batch_size,
-                                           shuffle=True)
-    dev_batch_sampler = BatchSampler(dev_ds,
-                                     batch_size=args.batch_size,
-                                     shuffle=False)
-    train_data_loader = DataLoader(dataset=train_ds,
-                                   batch_sampler=train_batch_sampler,
-                                   collate_fn=collate_fn)
-    dev_data_loader = DataLoader(dataset=dev_ds,
-                                 batch_sampler=dev_batch_sampler,
-                                 collate_fn=collate_fn)
 
-    # define model
+    # Define model
     model = AutoModelForSequenceClassification.from_pretrained(
-        args.model_name, num_classes=len(label_list))
-    if args.init_from_ckpt and os.path.isfile(args.init_from_ckpt):
-        state_dict = paddle.load(args.init_from_ckpt)
-        model.set_dict(state_dict)
-    model = paddle.DataParallel(model)
+        model_args.model_name_or_path, num_classes=len(label_list))
 
-    num_training_steps = len(train_data_loader) * args.epochs
-    lr_scheduler = LinearDecayWithWarmup(args.learning_rate, num_training_steps,
-                                         args.warmup_steps)
+    # Define the metric function.
+    def compute_metrics(eval_preds):
+        metric = Accuracy()
+        correct = metric.compute(paddle.to_tensor(eval_preds.predictions),
+                                 paddle.to_tensor(eval_preds.label_ids))
+        metric.update(correct)
+        acc = metric.accumulate()
+        return {'accuracy': acc}
 
-    # Generate parameter names needed to perform weight decay.
-    # All bias and LayerNorm parameters are excluded.
-    decay_params = [
-        p.name for n, p in model.named_parameters()
-        if not any(nd in n for nd in ["bias", "norm"])
+    # Define the early-stopping callback.
+    callbacks = [
+        EarlyStoppingCallback(
+            early_stopping_patience=data_args.early_stopping_patience,
+            early_stopping_threshold=0.)
     ]
-    optimizer = paddle.optimizer.AdamW(
-        learning_rate=lr_scheduler,
-        parameters=model.parameters(),
-        weight_decay=args.weight_decay,
-        apply_decay_param_fun=lambda x: x in decay_params)
 
+    # Define loss function
     criterion = paddle.nn.loss.CrossEntropyLoss()
-    metric = Accuracy()
 
-    global_step = 0
-    best_acc = 0
-    early_stop_count = 0
-    tic_train = time.time()
+    # Define Trainer
+    trainer = Trainer(model=model,
+                      tokenizer=tokenizer,
+                      args=training_args,
+                      criterion=criterion,
+                      train_dataset=train_ds,
+                      eval_dataset=dev_ds,
+                      callbacks=callbacks,
+                      data_collator=DataCollatorWithPadding(tokenizer),
+                      compute_metrics=compute_metrics)
 
-    for epoch in range(1, args.epochs + 1):
-
-        if args.early_stop and early_stop_count >= args.early_stop_nums:
-            logger.info("Early stop!")
-            break
-
-        for step, batch in enumerate(train_data_loader, start=1):
-            labels = batch.pop("labels")
-            logits = model(**batch)
-            loss = criterion(logits, labels)
-
-            loss.backward()
-            optimizer.step()
-            if args.warmup:
-                lr_scheduler.step()
-            optimizer.clear_grad()
-
-            global_step += 1
-            if global_step % args.logging_steps == 0 and rank == 0:
-                logger.info(
-                    "global step %d, epoch: %d, batch: %d, loss: %.5f, speed: %.2f step/s"
-                    % (global_step, epoch, step, loss, args.logging_steps /
-                       (time.time() - tic_train)))
-                tic_train = time.time()
-
-        early_stop_count += 1
-        acc = evaluate(model, criterion, metric, dev_data_loader)
-        save_best_path = args.save_dir
-        if not os.path.exists(save_best_path):
-            os.makedirs(save_best_path)
-
-        # save models
-        if acc > best_acc:
-            early_stop_count = 0
-            best_acc = acc
-            model._layers.save_pretrained(save_best_path)
-            tokenizer.save_pretrained(save_best_path)
-        logger.info("Current best accuracy: %.5f" % (best_acc))
-
-    logger.info("Final best accuracy: %.5f" % (best_acc))
-    logger.info("Save best accuracy text classification model in %s" %
-                (args.save_dir))
+    if training_args.do_train:
+        train_result = trainer.train(resume_from_checkpoint=None)
+        metrics = train_result.metrics
+        trainer.save_model()
+        trainer.log_metrics("train", metrics)
+        trainer.save_metrics("train", metrics)
+        trainer.save_state()
 
 
 if __name__ == "__main__":
-    train()
+    main()

applications/text_classification/multi_class/utils.py

@@ -14,40 +14,6 @@
 
 import numpy as np
 
-import paddle
-import paddle.nn.functional as F
-from paddlenlp.utils.log import logger
-
-
-@paddle.no_grad()
-def evaluate(model, criterion, metric, data_loader):
-    """
-    Given a dataset, it evaluates model and computes the metric.
-    Args:
-        model(obj:`paddle.nn.Layer`): A model to classify texts.
-        criterion(obj:`paddle.nn.Layer`): It can compute the loss.
-        metric(obj:`paddle.metric.Metric`): The evaluation metric.
-        data_loader(obj:`paddle.io.DataLoader`): The dataset loader which generates batches.
-    """
-
-    model.eval()
-    metric.reset()
-    losses = []
-    for batch in data_loader:
-        labels = batch.pop("labels")
-        logits = model(**batch)
-        loss = criterion(logits, labels)
-        losses.append(loss.numpy())
-        correct = metric.compute(logits, labels)
-        metric.update(correct)
-
-    acc = metric.accumulate()
-    logger.info("eval loss: %.5f, acc: %.5f" % (np.mean(losses), acc))
-    model.train()
-    metric.reset()
-
-    return acc
-
 
 def preprocess_function(examples, tokenizer, max_seq_length, is_test=False):
     """

applications/text_classification/multi_label/README.md

@@ -36,7 +36,7 @@
 对于大多数多标签分类任务，我们推荐使用预训练模型微调作为首选的文本分类方案，多标签分类项目中还提供 提示学习(小样本)和语义索引的两种全流程文本分类方案满足不同开发者需求，更多技术细节请参见[文本分类技术特色介绍](../README.md)。
 
 - 【标注成本高、标注样本较少的小样本场景】 👉 [提示学习多标签分类方案](./few-shot#readme)
-
+- 【标签类别不固定场景、标签类别众多】 👉 [语义索引多分类方案](./retrieval_based#readme)
 <a name="快速开始"></a>
 
 ## 2. 快速开始

applications/text_classification/multi_label/analysis/README.md

@@ -116,7 +116,7 @@ Text	Label	Prediction
 ### 单词级别可解释性分析
 本项目开源模型的词级别可解释性分析Notebook，提供LIME、Integrated Gradient、GradShap 三种分析方法，支持分析微调后模型的预测结果，开发者可以通过更改**数据目录**和**模型目录**在自己的任务中使用Jupyter Notebook进行数据分析。
 
-运行 [word_interpret.ipynb](./word_interpreter.ipynb) 代码，即可分析影响样本预测结果的关键词以及可视化所有词对预测结果的贡献情况，颜色越深代表这个词对预测结果影响越大：
+运行 [word_interpret.ipynb](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/applications/text_classification/multi_label/analysis/README.md) 代码，即可分析影响样本预测结果的关键词以及可视化所有词对预测结果的贡献情况，颜色越深代表这个词对预测结果影响越大：
 <div align="center">
     <img src="https://user-images.githubusercontent.com/63761690/192739675-63145d59-23c6-416f-bf71-998fd4995254.png" width="1000">
 </div>

applications/text_classification/multi_label/deploy/paddle_serving/README.md

@@ -51,10 +51,10 @@ pip install paddle-serving-server-gpu==0.8.3.post112 -i https://pypi.tuna.tsingh
 - 默认开启国内清华镜像源来加速下载，如果您使用 HTTP 代理可以关闭(-i https://pypi.tuna.tsinghua.edu.cn/simple)
 - 更多wheel包请参考[serving官网文档](https://github.com/PaddlePaddle/Serving/blob/develop/doc/Latest_Packages_CN.md)
 
-### 安装FasterTokenizer文本处理加速库（可选）
-推荐安装faster_tokenizer可以得到更极致的文本处理效率，进一步提升服务性能。
+### 安装FastTokenizer文本处理加速库（可选）
+推荐安装fast_tokenizer可以得到更极致的文本处理效率，进一步提升服务性能。
 ```shell
-pip install faster_tokenizer
+pip install fast-tokenizer-python
 ```
 
 
@@ -142,7 +142,7 @@ I0625 16:44:36.563802 40218 analysis_predictor.cc:1007] ======= optimize end ===
 I0625 16:44:36.571702 40218 naive_executor.cc:102] ---  skip [feed], feed -> token_type_ids
 I0625 16:44:36.571728 40218 naive_executor.cc:102] ---  skip [feed], feed -> input_ids
 I0625 16:44:36.574352 40218 naive_executor.cc:102] ---  skip [linear_147.tmp_1], fetch -> fetch
-[2022-06-25 16:44:37,545] [ WARNING] - Can't find the faster_tokenizers package, please ensure install faster_tokenizers correctly. You can install faster_tokenizers by `pip install faster_tokenizers`(Currently only work for linux platform).
+[2022-06-25 16:44:37,545] [ WARNING] - Can't find the fast_tokenizer package, please ensure install fast_tokenizer correctly. You can install fast_tokenizer by `pip install fast-tokenizer-python`.
 [2022-06-25 16:44:37,546] [    INFO] - We are using <class 'paddlenlp.transformers.ernie.tokenizer.ErnieTokenizer'> to load 'ernie-3.0-medium-zh'.
 [2022-06-25 16:44:37,546] [    INFO] - Already cached /root/.paddlenlp/models/ernie-3.0-medium-zh/ernie_3.0_base_zh_vocab.txt
 [OP Object] init success

applications/text_classification/multi_label/deploy/paddle_serving/service.py

@@ -49,7 +49,7 @@ class Op(Op):
 
     def init_op(self):
         self.tokenizer = AutoTokenizer.from_pretrained(args.model_name,
-                                                       use_faster=True)
+                                                       use_fast=True)
         # Output nodes may differ from model to model
         # You can see the output node name in the conf.prototxt file of serving_server
         self.fetch_names = [

applications/text_classification/multi_label/deploy/predictor/README.md

@@ -12,7 +12,7 @@
 
 如果基于GPU部署，请先确保机器已正确安装NVIDIA相关驱动和基础软件，确保CUDA >= 11.2，CuDNN >= 8.2，并使用以下命令安装所需依赖:
 ```shell
-python -m pip install onnxruntime-gpu onnx onnxconverter-common
+python -m pip install onnxruntime-gpu onnx onnxconverter-common==1.9.0
 ```
 
 如果基于CPU部署，请使用如下命令安装所需依赖:
@@ -20,10 +20,10 @@ python -m pip install onnxruntime-gpu onnx onnxconverter-common
 python -m pip install onnxruntime
 ```
 
-安装FasterTokenizer文本处理加速库（可选）
-推荐安装faster_tokenizer可以得到更极致的文本处理效率，进一步提升服务性能。
+安装FastTokenizer文本处理加速库（可选）
+推荐安装fast_tokenizer可以得到更极致的文本处理效率，进一步提升服务性能。
 ```shell
-pip install faster_tokenizer
+pip install fast-tokenizer-python
 ```
 ## 基于GPU部署推理样例
 

applications/text_classification/multi_label/deploy/predictor/predictor.py

@@ -124,7 +124,7 @@ class Predictor(object):
 
     def __init__(self, args, label_list):
         self.tokenizer = AutoTokenizer.from_pretrained(args.model_name_or_path,
-                                                       use_faster=True)
+                                                       use_fast=True)
         self.label_list = label_list
         self.batch_size = args.batch_size
         self.max_seq_length = args.max_seq_length

applications/text_classification/multi_label/deploy/simple_serving/README.md

@@ -0,0 +1,38 @@
+# 基于PaddleNLP SimpleServing 的服务化部署
+
+## 目录
+- [环境准备](#环境准备)
+- [Server启动服务](#Server服务启动)
+- [其他参数设置](#其他参数设置)
+
+## 环境准备
+使用有SimpleServing功能的PaddleNLP版本
+```shell
+pip install paddlenlp --upgrade
+```
+## Server服务启动
+### 分类任务启动
+#### 启动 分类 Server 服务
+```bash
+paddlenlp server server:app --host 0.0.0.0 --port 8189
+```
+
+#### 分类任务发送服务
+```bash
+python client.py
+```
+
+## 其他参数设置
+可以在client端设置 `max_seq_len`, `batch_size`, `prob_limit` 参数
+```python
+    data = {
+        'data': {
+            'text': texts,
+        },
+        'parameters': {
+            'max_seq_len': args.max_seq_len,
+            'batch_size': args.batch_size,
+            'prob_limit': args.prob_limit
+        }
+    }
+```

applications/text_classification/multi_label/deploy/simple_serving/client.py

@@ -0,0 +1,47 @@
+# Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import argparse
+import requests
+import json
+
+# yapf: disable
+parser = argparse.ArgumentParser()
+parser.add_argument("--max_seq_len", default=128, type=int, help="The maximum total input sequence length after tokenization.")
+parser.add_argument("--batch_size", default=1, type=int, help="Batch size per GPU/CPU for predicting.")
+parser.add_argument("--prob_limit", default=0.5, type=float, help="The limitation of probability for the label.")
+args = parser.parse_args()
+# yapf: enable
+
+url = "http://0.0.0.0:8189/models/cls_multi_label"
+headers = {"Content-Type": "application/json"}
+
+if __name__ == "__main__":
+    texts = [
+        '原、被告另购置橱柜、碗架、电磁炉、电饭锅各一个归原告王某某所有。',
+        '于是原告到儿子就读的幼儿园进行探望，被告碰见后对原告破口大骂，还不让儿子叫原告妈妈，而叫被告现在的妻子做妈妈。',
+        '由我全额出资购买的联想台式电脑，我均依次放弃。'
+    ]
+    data = {
+        'data': {
+            'text': texts,
+        },
+        'parameters': {
+            'max_seq_len': args.max_seq_len,
+            'batch_size': args.batch_size,
+            'prob_limit': args.prob_limit
+        }
+    }
+    r = requests.post(url=url, headers=headers, data=json.dumps(data))
+    print(json.loads(r.text))

applications/text_classification/multi_label/deploy/simple_serving/server.py

@@ -0,0 +1,23 @@
+# Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from paddlenlp import SimpleServer
+from paddlenlp.server import CustomModelHandler, MultiLabelClassificationPostHandler
+
+app = SimpleServer()
+app.register('cls_multi_label',
+             model_path="../../export",
+             tokenizer_name='ernie-3.0-medium-zh',
+             model_handler=CustomModelHandler,
+             post_handler=MultiLabelClassificationPostHandler)

applications/text_classification/multi_label/deploy/triton_serving/README.md

@@ -48,13 +48,13 @@ python3 -m pip install paddlepaddle-gpu paddlenlp -i https://mirror.baidu.com/py
 3. 更多关于PaddleNLP安装的详细教程请查看[Installation](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/docs/get_started/installation.rst)。
 
 
-### 安装FasterTokenizers文本处理加速库（可选）
+### 安装FastTokenizer文本处理加速库（可选）
 
-推荐安装faster_tokenizers可以得到更极致的文本处理效率，进一步提升服务性能。
+推荐安装fast_tokenizer可以得到更极致的文本处理效率，进一步提升服务性能。
 
-在容器内安装 faster_tokenizers
+在容器内安装 fast_tokenizer
 ```shell
-python3 -m pip install faster_tokenizers
+python3 -m pip install fast-tokenizer-python
 ```
 
 
@@ -104,7 +104,7 @@ models
     └── config.pbtxt
 ```
 
-模型配置文件config.pbtxt配置细节请参见[Triton Server Model Configuration](https://github.com/triton-inference-server/server/blob/main/docs/model_configuration.md)
+模型配置文件config.pbtxt配置细节请参见[Triton Server Model Configuration](https://github.com/triton-inference-server/server/blob/main/docs/user_guide/model_configuration.md)
 
 ## 部署模型
 

applications/text_classification/multi_label/deploy/triton_serving/models/tokenizer/1/model.py

@@ -48,7 +48,7 @@ class TritonPythonModel(object):
           * model_name: Model name
         """
         self.tokenizer = AutoTokenizer.from_pretrained("ernie-3.0-medium-zh",
-                                                       use_faster=True)
+                                                       use_fast=True)
         # You must parse model_config. JSON string is not parsed here
         self.model_config = json.loads(args['model_config'])
         print("model_config:", self.model_config)

applications/text_classification/multi_label/few-shot/README.md

@@ -70,9 +70,9 @@
 
    内存: 630 GB
 
-3. PaddlePaddle 版本：2.3.1
+3. PaddlePaddle 版本：2.4rc
 
-4. PaddleNLP 版本：2.3.5 (develop)
+4. PaddleNLP 版本：2.4.3
 
 5. 评估设置
 
@@ -82,13 +82,14 @@
   - 微调
 
     ```
+    cd ../
     python train.py --dataset_dir "./data/" --save_dir "./checkpoints" --max_seq_length 128 --model_name "ernie-3.0-base-zh" --batch_size 8 --learning_rate 3e-5 --epochs 100 --logging_steps 5 --early_stop --early_stop_num 10
     ```
 
   - 提示学习
 
     ```
-    python train.py --data_dir ./data/ --output_dir ./checkpoints/ --prompt "这句话包含的要素有" --model_name_or_path ernie-3.0-base-zh --max_seq_length 128  --learning_rate 3e-5 --ppt_learning_rate 3e-4 --do_train --do_eval --num_train_epochs 100 --logging_steps 5 --per_device_eval_batch_size 32 --per_device_train_batch_size 8 --do_predict --metric_for_best_model macro_f1_score --load_best_model_at_end --evaluation_strategy epoch --save_strategy epoch
+    python train.py --data_dir ./data/ --output_dir ./checkpoints/ --prompt "这句话包含的要素有" --model_name_or_path ernie-3.0-base-zh --max_seq_length 128  --learning_rate 3e-5 --ppt_learning_rate 3e-4 --do_train --do_eval --num_train_epochs 100 --logging_steps 5 --per_device_eval_batch_size 32 --per_device_train_batch_size 8 --do_predict --metric_for_best_model macro_f1_score --load_best_model_at_end --eval_steps 100 --save_total_limit 1
     ```
 
 6. 精度评价指标：Micro F1分数、Macro F1分数
@@ -96,7 +97,7 @@
    | model_name | 训练方式 | Micro F1分数 | Macro F1分数 |
    | ---------- | ------- | ----------- | ----------- |
    | ernie-3.0-base-zh | 微调学习 | 0.7419 | 0.5105 |
-   | ernie-3.0-base-zh | 提示学习 | 0.7838 | 0.6985 |
+   | ernie-3.0-base-zh | 提示学习 | 0.7839 | 0.6003 |
 
 <a name="定制训练"></a>
 ## 3.定制训练
@@ -106,10 +107,10 @@
 <a name="运行环境"></a>
 ### 3.1 运行环境
 
-- python >= 3.6
-- paddlepaddle > 2.3（2.4版本发布前推荐安装[develop版本](https://www.paddlepaddle.org.cn/install/quick?docurl=/documentation/docs/zh/develop/install/pip/linux-pip.html)）
-- paddlenlp >= 2.3.5
-- paddle2onnx >= 1.0.0rc3
+- python >= 3.7
+- paddlepaddle >= 2.4rc
+- paddlenlp >= 2.4.3
+- paddle2onnx >= 1.0.3
 
 <a name="代码结构"></a>
 ### 3.2 代码结构
@@ -223,6 +224,7 @@ python train.py \
 --do_export \
 --num_train_epochs 100 \
 --logging_steps 5 \
+--save_total_limit 1 \
 --per_device_eval_batch_size 32 \
 --per_device_train_batch_size 8 \
 --metric_for_best_model macro_f1_score \
@@ -248,6 +250,7 @@ python -u -m paddle.distributed.launch --gpus 0,1,2,3 train.py \
 --do_export \
 --num_train_epochs 100 \
 --logging_steps 5 \
+--save_total_limit 1 \
 --per_device_eval_batch_size 32 \
 --per_device_train_batch_size 8 \
 --metric_for_best_model macro_f1_score \
@@ -274,6 +277,7 @@ python -u -m paddle.distributed.launch --gpus 0,1,2,3 train.py \
 - `do_export`: 是否在运行结束时将模型导出为静态图，保存路径为`output_dir/export`。
 - `num_train_epochs`: 训练的最大轮数。
 - `max_steps`: 训练的最大步数。此设置将会覆盖`num_train_epochs`。
+- `save_total_limit`: 模型检查点保存数量。
 - `device`: 使用的设备，默认为`gpu`。
 - `eval_steps`: 评估模型的间隔步数。
 - `logging_steps`: 打印日志的间隔步数。
@@ -352,9 +356,9 @@ python infer.py --model_path_prefix checkpoints/export/model --data_dir ./data -
 可配置参数说明：
 
 - `model_path_prefix`: 导出的静态图模型路径及文件前缀。
-- `model_name_or_path`: 内置预训练模型名，或者模型参数配置目录路径，用于加载tokenizer。默认为`ernie-3.0-base-zh`。
+- `model_name`: 内置预训练模型名，用于加载tokenizer。默认为`ernie-3.0-base-zh`。
 - `data_dir`: 待推理数据所在路径，数据应存放在该目录下的`data.txt`文件。
-- `max_seq_length`: 最大句子长度，超过该长度的文本将被截断，不足的以Pad补全。提示文本不会被截断。
+- `max_length`: 最大句子长度，超过该长度的文本将被截断，不足的以Pad补全。提示文本不会被截断。
 - `batch_size`: 每次预测的样本数量。
 - `device`: 选择推理设备，包括`cpu`和`gpu`。默认为`gpu`。
 - `device_id`: 指定GPU设备ID。

applications/text_classification/multi_label/few-shot/infer.py

@@ -14,23 +14,24 @@
 
 import os
 import six
+import json
 import psutil
 import argparse
 
 import numpy as np
 
 from paddlenlp.utils.log import logger
-from paddlenlp.prompt import AutoTemplate, Verbalizer, InputExample
-from paddlenlp.transformers import AutoTokenizer
+from paddlenlp.prompt import AutoTemplate, PromptDataCollatorWithPadding
+from paddlenlp.transformers import AutoTokenizer, AutoModelForMaskedLM
 import paddle2onnx
 import onnxruntime as ort
 
 # yapf: disable
 parser = argparse.ArgumentParser()
 parser.add_argument("--model_path_prefix", type=str, required=True, help="The path prefix of inference model to be used.")
-parser.add_argument("--model_name_or_path", default="ernie-3.0-base-zh", type=str, help="The directory or name of model.")
+parser.add_argument("--model_name", default="ernie-3.0-base-zh", type=str, help="The name of pretrained model.")
 parser.add_argument("--data_dir", default=None, type=str, help="The path to the prediction data, including label.txt and data.txt.")
-parser.add_argument("--max_seq_length", default=128, type=int, help="The maximum total input sequence length after tokenization.")
+parser.add_argument("--max_length", default=128, type=int, help="The maximum total input sequence length after tokenization.")
 parser.add_argument("--use_fp16", action='store_true', help="Whether to use fp16 inference, only takes effect when deploying on gpu.")
 parser.add_argument("--batch_size", default=200, type=int, help="Batch size per GPU/CPU for predicting.")
 parser.add_argument("--num_threads", default=psutil.cpu_count(logical=False), type=int, help="num_threads for cpu.")
@@ -103,11 +104,6 @@ class InferBackend(object):
                                                   'device_id':
                                                   device_id
                                               }])
-        self.input_handles = [
-            self.predictor.get_inputs()[0].name,
-            self.predictor.get_inputs()[1].name,
-            self.predictor.get_inputs()[2].name
-        ]
 
         if device == "gpu":
             try:
@@ -122,27 +118,52 @@ class InferBackend(object):
         logger.info(">>> [InferBackend] Engine Created ...")
 
     def infer(self, input_dict: dict):
-        input_dict = {
-            k: v
-            for k, v in input_dict.items() if k in self.input_handles
-        }
         result = self.predictor.run(None, input_dict)
         return result
 
 
 class MultiLabelPredictor(object):
 
-    def __init__(self, args, label_list):
-        self._label_list = label_list
-        self._tokenizer = AutoTokenizer.from_pretrained(args.model_name_or_path)
-        self._max_seq_length = args.max_seq_length
-        self._batch_size = args.batch_size
-        self.inference_backend = InferBackend(args.model_path_prefix,
-                                              args.device, args.device_id,
-                                              args.use_fp16, args.num_threads)
-        self._template = AutoTemplate.load_from(
-            os.path.dirname(args.model_path_prefix), self._tokenizer,
-            args.max_seq_length)
+    def __init__(self, args):
+        self.args = args
+        self.tokenizer = AutoTokenizer.from_pretrained(args.model_name)
+        self.model = AutoModelForMaskedLM.from_pretrained(args.model_name)
+        self.template, self.labels, self.input_handles = self.post_init()
+        self.collate_fn = PromptDataCollatorWithPadding(
+            self.tokenizer,
+            padding=True,
+            return_tensors="np",
+            return_attention_mask=True)
+
+        self.inference_backend = InferBackend(self.args.model_path_prefix,
+                                              self.args.device,
+                                              self.args.device_id,
+                                              self.args.use_fp16,
+                                              self.args.num_threads)
+
+    def post_init(self):
+        export_path = os.path.dirname(self.args.model_path_prefix)
+        template_path = os.path.join(export_path, "template_config.json")
+        with open(template_path, "r") as fp:
+            prompt = json.load(fp)
+            template = AutoTemplate.create_from(prompt, self.tokenizer,
+                                                self.args.max_length,
+                                                self.model)
+        keywords = template.extract_template_keywords(template.prompt)
+        inputs = [
+            "input_ids", "token_type_ids", "position_ids", "attention_mask",
+            "masked_positions"
+        ]
+        if "soft" in keywords:
+            inputs.append("soft_token_ids")
+        if "encoder" in keywords:
+            inputs.append("encoder_ids")
+        verbalizer_path = os.path.join(export_path, "verbalizer_config.json")
+        with open(verbalizer_path, "r") as fp:
+            label_words = json.load(fp)
+            labels = sorted(list(label_words.keys()))
+
+        return template, labels, inputs
 
     def predict(self, input_data: list):
         encoded_inputs = self.preprocess(input_data)
@@ -155,14 +176,27 @@ class MultiLabelPredictor(object):
         infer_data = self.inference_backend.infer(input_dict)
         return infer_data
 
-    def infer_batch(self, encoded_inputs):
-        num_sample = len(encoded_inputs["input_ids"])
+    def infer_batch(self, inputs):
+        num_sample = len(inputs)
         infer_data = None
         num_infer_data = None
-        for idx in range(0, num_sample, self._batch_size):
-            l, r = idx, idx + self._batch_size
-            keys = encoded_inputs.keys()
-            input_dict = {k: encoded_inputs[k][l:r] for k in keys}
+        for index in range(0, num_sample, self.args.batch_size):
+            left, right = index, index + self.args.batch_size
+            batch_dict = self.collate_fn(inputs[left:right])
+            input_dict = {}
+            for key in self.input_handles:
+                value = batch_dict[key]
+                if key == "attention_mask":
+                    if value.ndim == 2:
+                        value = (1 - value[:, np.newaxis, np.newaxis, :]) * -1e4
+                    elif value.ndim != 4:
+                        raise ValueError(
+                            "Expect attention mask with ndim=2 or 4, but get ndim={}"
+                            .format(value.ndim))
+                    value = value.astype("float32")
+                else:
+                    value = value.astype("int64")
+                input_dict[key] = value
             results = self._infer(input_dict)
             if infer_data is None:
                 infer_data = [[x] for x in results]
@@ -175,16 +209,8 @@ class MultiLabelPredictor(object):
         return infer_data
 
     def preprocess(self, input_data: list):
-        text = [InputExample(text_a=x) for x in input_data]
-        inputs = [self._template.wrap_one_example(x) for x in text]
-        inputs = {
-            "input_ids":
-            np.array([x["input_ids"] for x in inputs], dtype="int64"),
-            "mask_ids":
-            np.array([x["mask_ids"] for x in inputs], dtype="int64"),
-            "soft_token_ids":
-            np.array([x["soft_token_ids"] for x in inputs], dtype="int64")
-        }
+        text = [{"text_a": x} for x in input_data]
+        inputs = [self.template(x) for x in text]
         return inputs
 
     @staticmethod
@@ -197,7 +223,7 @@ class MultiLabelPredictor(object):
         label_ids = np.argwhere(probs > threshold)
         labels = [[] for _ in range(probs.shape[0])]
         for idx, label_id in label_ids:
-            labels[idx].append(self._label_list[label_id])
+            labels[idx].append(self.labels[label_id])
         return {"label": labels}
 
     def printer(self, result, input_data):
@@ -212,12 +238,10 @@ if __name__ == "__main__":
     for arg_name, arg_value in vars(args).items():
         logger.info("{:20}: {}".format(arg_name, arg_value))
 
-    export_path = os.path.dirname(args.model_path_prefix)
-    labels, _ = Verbalizer.load_from(export_path)
+    predictor = MultiLabelPredictor(args)
 
     text_dir = os.path.join(args.data_dir, "data.txt")
     with open(text_dir, "r", encoding="utf-8") as f:
         text_list = [x.strip() for x in f.readlines()]
 
-    predictor = MultiLabelPredictor(args, labels)
     predictor.predict(text_list)

applications/text_classification/multi_label/few-shot/requirements_cpu.txt

@@ -0,0 +1,5 @@
+psutil
+paddlepaddle>=2.4rc
+paddlenlp>=2.4.3
+paddle2onnx>=1.0.3
+onnxruntime

applications/text_classification/multi_label/few-shot/requirements_gpu.txt

@@ -0,0 +1,7 @@
+psutil
+paddlepaddle-gpu>=2.4rc
+paddlenlp>=2.4.3
+paddle2onnx>=1.0.3
+onnxruntime-gpu
+onnx
+onnxconverter-common

applications/text_classification/multi_label/few-shot/train.py

@@ -15,6 +15,7 @@
 from dataclasses import dataclass, field
 import os
 import sys
+from collections import defaultdict
 
 import paddle
 import paddle.nn.functional as F
@@ -41,8 +42,6 @@ from metric import MetricReport
 class DataArguments:
     data_dir: str = field(default="./data", metadata={"help": "The dataset dictionary includes train.txt, dev.txt and label.txt files."})
     prompt: str = field(default=None, metadata={"help": "The input prompt for tuning."})
-    soft_encoder: str = field(default="lstm", metadata={"help": "The encoder type of soft template, `lstm`, `mlp` or None."})
-    encoder_hidden_size: int = field(default=200, metadata={"help": "The dimension of soft embeddings."})
 
 
 @dataclass
@@ -67,17 +66,20 @@ def main():
     tokenizer = AutoTokenizer.from_pretrained(model_args.model_name_or_path)
 
     # Define the template for preprocess and the verbalizer for postprocess.
-    template = AutoTemplate.create_from(
-        data_args.prompt,
-        tokenizer,
-        training_args.max_seq_length,
-        model=model,
-        prompt_encoder=data_args.soft_encoder,
-        encoder_hidden_size=data_args.encoder_hidden_size)
-    logger.info("Using template: {}".format(template.template))
+    template = AutoTemplate.create_from(data_args.prompt,
+                                        tokenizer,
+                                        training_args.max_seq_length,
+                                        model=model)
+    logger.info("Using template: {}".format(template.prompt))
 
     label_file = os.path.join(data_args.data_dir, "label.txt")
-    verbalizer = SoftVerbalizer.from_file(tokenizer, model, label_file)
+    with open(label_file, "r", encoding="utf-8") as fp:
+        label_words = defaultdict(list)
+        for line in fp:
+            data = line.strip().split("==")
+            word = data[1] if len(data) > 1 else data[0].split("##")[-1]
+            label_words[data[0]].append(word)
+    verbalizer = SoftVerbalizer(label_words, tokenizer, model)
 
     # Load the few-shot datasets.
     train_ds, dev_ds, test_ds = load_local_dataset(
@@ -139,11 +141,24 @@ def main():
 
     # Export static model.
     if training_args.do_export:
+        template = prompt_model.template
+        template_keywords = template.extract_template_keywords(template.prompt)
         input_spec = [
-            InputSpec(shape=[None, None], dtype="int64"),  # input_ids
-            InputSpec(shape=[None, None], dtype="int64"),  # mask_ids
-            InputSpec(shape=[None, None], dtype="int64"),  # soft_token_ids
+            InputSpec(shape=[None, None], dtype="int64"),  # input_ids,
+            InputSpec(shape=[None, None], dtype="int64"),  # token_type_ids
+            InputSpec(shape=[None, None], dtype="int64"),  # position_ids
+            InputSpec(shape=[None, None, None, None],
+                      dtype="float32")  # attention_mask
         ]
+        if "mask" in template_keywords:
+            input_spec.append(InputSpec(shape=[None],
+                                        dtype="int64"))  # masked_positions
+        if "soft" in template_keywords:
+            input_spec.append(InputSpec(shape=[None, None],
+                                        dtype="int64"))  # soft_token_ids
+        if "encoder" in template_keywords:
+            input_spec.append(InputSpec(shape=[None, None],
+                                        dtype="int64"))  # encoder_ids
         export_path = os.path.join(training_args.output_dir, 'export')
         trainer.export_model(export_path,
                              input_spec=input_spec,

applications/text_classification/multi_label/few-shot/utils.py

@@ -15,7 +15,6 @@
 import os
 
 from paddlenlp.datasets import load_dataset
-from paddlenlp.prompt import InputExample
 
 
 def load_local_dataset(data_path, splits, label_list):
@@ -39,14 +38,14 @@ def load_local_dataset(data_path, splits, label_list):
             for idx, line in enumerate(fp):
                 data = line.strip().split("\t")
                 if len(data) == 1:
-                    yield InputExample(text_a=data[0])
+                    yield {"text_a": data[0]}
                 else:
                     text, label = data
                     label = label.strip().split(",")
                     label = [
                         float(1) if x in label else float(0) for x in label_list
                     ]
-                    yield InputExample(text_a=text, labels=label)
+                    yield {"text_a": text, "labels": label}
 
     split_map = {"train": "train.txt", "dev": "dev.txt", "test": "test.txt"}
     datasets = []

applications/text_classification/multi_label/retrieval_based/README.md

@@ -0,0 +1,512 @@
+# 基于检索的多标签文本分类方法
+
+ **目录**
+
+* [1. 基于语义索引的多标签分类任务介绍](#基于语义索引的多标签分类任务介绍)
+* [2. 代码结构说明](#代码结构说明)
+* [3. 环境准备](#环境准备)
+* [4. 数据准备](#数据准备)
+* [5. 模型训练](#模型训练)
+* [6. 模型评估](#模型训练)
+* [7. 模型预测](#模型预测)
+* [8. 模型部署](#模型部署)
+* [9. 分类流程](#分类流程)
+
+<a name="基于语义索引的多标签分类任务介绍"></a>
+
+# 1.基于语义索引的多标签分类任务介绍
+
+以前的分类任务中，标签信息作为无实际意义，独立存在的one-hot编码形式存在，这种做法会潜在的丢失标签的语义信息，本方案把文本分类任务中的标签信息转换成含有语义信息的语义向量，将文本分类任务转换成向量检索和匹配的任务。这样做的好处是对于一些类别标签不是很固定的场景，或者需要经常有一些新增类别的需求的情况非常合适。另外，对于一些新的相关的分类任务，这种方法也不需要模型重新学习或者设计一种新的模型结构来适应新的任务。总的来说，这种基于检索的文本分类方法能够有很好的拓展性，能够利用标签里面包含的语义信息，不需要重新进行学习。这种方法可以应用到相似标签推荐，文本标签标注，金融风险事件分类，政务信访分类等领域。
+
+本方案是基于语义索引模型的分类，语义索引模型的目标是：给定输入文本，模型可以从海量候选召回库中**快速、准确**地召回一批语义相关文本。基于语义索引的多标签分类方法有两种，第一种方法是直接把标签变成召回库，即把输入文本和标签的文本进行匹配，第二种是利用召回的文本带有类别标签，把召回文本的类别标签作为给定输入文本的类别。本方案使用双塔模型，训练阶段引入In-batch Negatives  策略，使用hnswlib建立索引库，并把标签作为召回库，进行召回测试。最后利用召回的结果使用 Accuracy 指标来评估语义索引模型的分类的效果。
+
+**注意** 基于语义索引的文本分类的标签在预测过程中会抽取成向量，所以标签需要文本的形式，不能是ID形式的标签。
+
+<a name="代码结构说明"></a>
+
+## 2. 代码结构说明
+
+```
+|—— data.py # 数据读取、数据转换等预处理逻辑
+|—— base_model.py # 语义索引模型基类
+|—— train.py # In-batch Negatives 策略的训练主脚本
+|—— model.py # In-batch Negatives 策略核心网络结构
+
+|—— recall.py # 基于训练好的语义索引模型，从召回库中召回给定文本的相似文本
+|—— evaluate.py # 根据召回结果和评估集计算评估指标
+|—— metric.py # Macro F1和Micro F1评估指标
+|—— predict.py # 给定输入文件，计算文本 pair 的相似度
+|—— export_model.py # 动态图转换成静态图
+|—— export_to_serving.py # 静态图转 Serving
+|—— scripts
+    |—— export_model.sh  # 动态图转换成静态图脚本
+    |—— predict.sh  # 预测 bash 版本
+    |—— evaluate.sh # 评估 bash 版本
+    |—— run_build_index.sh # 构建索引 bash 版本
+    |—— train.sh  # 训练 bash 版本
+    |—— export_to_serving.sh  # Paddle Inference 转 Serving 的 bash 脚本
+    |—— run.sh # 构建Milvus向量的 bash 版本
+|—— utils
+    ├── config.py # Milvus 的配置文件
+    ├── feature_extract.py # 向量抽取文件
+    ├── milvus_util.py # Milvus 的配置文件
+|—— deploy
+    |—— python
+        |—— predict.py # PaddleInference
+        |—— deploy.sh # Paddle Inference 部署脚本
+        |—— rpc_client.py # Paddle Serving 的 Client 端
+        |—— web_service.py # Paddle Serving 的 Serving 端
+        |—— config_nlp.yml # Paddle Serving 的配置文件
+
+```
+
+<a name="环境准备"></a>
+
+## 3. 环境准备
+
+推荐使用GPU进行训练，在预测阶段使用CPU或者GPU均可。
+
+**环境依赖**
+* python >= 3.7
+* paddlepaddle >= 2.3.1
+* paddlenlp >= 2.3.4
+* hnswlib >= 0.5.2
+* visualdl >= 2.2.2
+
+```
+pip install -r requirements.txt
+```
+
+<a name="数据准备"></a>
+
+## 4. 数据准备
+
+训练需要准备指定格式的本地数据集,如果没有已标注的数据集，可以参考[文本分类任务doccano数据标注使用指南](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/applications/text_classification/doccano.md)进行文本分类数据标注。
+
+**指定格式本地数据集目录结构**
+
+```
+├── data # 数据集目录
+    ├── label.txt # 标签集
+    ├── dev.txt # 验证集
+    ├── test.txt # 测试集
+    ├── train.txt  # 训练集
+```
+
+**训练、开发、测试数据集**
+
+train.txt(训练数据集文件)， dev.txt(开发数据集文件)，test.txt(可选，测试数据集文件)，文件中文本与标签类别名用tab符`'\t'`分隔开，由于文本有多个标签，需要把文本拆分成多个单标签的形式，即多行文本标签对。训练集指用于训练模型的数据；开发集指用于评测模型表现的数据，可以根据模型在开发集上的精度调整训练参数和模型；测试集用于测试模型表现，没有测试集时可以使用开发集代替。
+
+- train.txt/test.txt 文件格式：
+```text
+<文本>'\t'<标签1>
+<文本>'\t'<标签2>
+...
+```
+- train.txt/test.txt 文件样例：
+```text
+茄子怎么做才好吃？怎么做才好吃？	生活
+茄子怎么做才好吃？怎么做才好吃？	美食/烹饪
+茄子怎么做才好吃？怎么做才好吃？	烹饪方法
+...
+```
+- dev.txt 文件格式：
+```
+<文本>'\t'<标签1>,<标签2>,<标签3>
+<文本>'\t'<标签2>,<标签5>
+```
+- dev.txt 文件样例：
+```text
+克隆人的思想和原体是一样的吗？	教育/科学,理工学科,生物学
+家用燃气灶哪个牌子好些?最好是高效节能的~最好是高效的~	生活,购物
+咨询下，黛莱美面膜怎么样，那里有咨询下，黛莱美怎么样，那里有	生活,美容/塑身,护肤
+...
+```
+
+**分类标签**
+
+label.txt(分类标签文件)记录数据集中所有标签集合，每一行为一个标签名。
+- label.txt 文件格式：
+```text
+<标签>
+<标签>
+...
+```
+- label.txt 文件样例：
+```text
+魔力宝贝
+完美游戏
+地球科学
+美食/烹饪
+肝胆外科
+动漫
+婚嫁
+历史话题
+新生儿
+...
+```
+
+<a name="模型训练"></a>
+
+## 5. 模型训练
+
+我们使用百科知识问答的数据来构建训练集，开发集。
+
+**训练集（train.txt）** 和 **开发集(dev.txt)** 格式一致，训练集30k条，开发集10k条，每行由文本的标题，内容和类别标签组成，以tab符分割，第一列是问题的标题和问题的描述拼接，剩下的列问题的类别。
+**召回库（label.txt）** 类别的数量是323类，召回标签库的构建有2种方式，第一种是把所有的类别标签当成召回库，第二种是把训练集当成召回集合，我们以第一种为例。
+
+数据集选择的是百科问答数据集的一个子集，问答数据集详情请参考[nlp_chinese_corpus](https://github.com/brightmart/nlp_chinese_corpus)
+
+- [baike_qa_category](https://paddlenlp.bj.bcebos.com/applications/baike_qa_multilabel.zip)
+
+```
+wget https://paddlenlp.bj.bcebos.com/applications/baike_qa_multilabel.zip
+unzip  baike_qa_category.zip
+```
+
+### 单机单卡训练/单机多卡训练
+
+这里采用单机多卡方式进行训练，通过如下命令，指定 GPU 0,1 卡;如果采用单机单卡训练，只需要把`--gpus`参数设置成单卡的卡号即可。
+
+如果使用CPU进行训练，则需要吧`--gpus`参数去除，然后吧`device`设置成cpu即可，详细请参考train.sh文件的训练设置
+
+然后运行下面的命令使用GPU训练，得到语义索引模型：
+
+```
+root_path=inbatch
+data_path=data
+python -u -m paddle.distributed.launch --gpus "0,1" \
+    train.py \
+    --device gpu \
+    --save_dir ./checkpoints/${root_path} \
+    --batch_size 24 \
+    --learning_rate 5E-5 \
+    --epochs 100 \
+    --output_emb_size 0 \
+    --save_steps 50 \
+    --max_seq_length 384 \
+    --warmup_proportion 0.0 \
+    --margin 0.2 \
+    --recall_result_dir "recall_result_dir" \
+    --recall_result_file "recall_result.txt" \
+    --train_set_file ${data_path}/train.txt \
+    --corpus_file ${data_path}/label.txt   \
+    --similar_text_pair_file ${data_path}/dev.txt \
+    --evaluate True
+```
+
+参数含义说明
+
+* `device`: 使用 cpu/gpu 进行训练
+* `save_dir`: 模型存储路径
+* `batch_size`: 训练的batch size的大小
+* `learning_rate`: 训练的学习率的大小
+* `epochs`: 训练的epoch数
+* `output_emb_size`: Transformer 顶层输出的文本向量维度
+* `save_steps`： 模型存储 checkpoint 的间隔 steps 个数
+* `max_seq_length`: 输入序列的最大长度
+* `margin`: 正样本相似度与负样本之间的目标 Gap
+* `train_set_file`: 训练集文件
+* `evaluate`: 是否开启边训练边评估模型训练效果，默认开启
+* `recall_result_dir`: 召回结果存储目录
+* `recall_result_file`: 召回结果的文件名
+* `hnsw_m`: hnsw 算法相关参数，保持默认即可
+* `hnsw_ef`: hnsw 算法相关参数，保持默认即可
+* `recall_num`: 对 1 个文本召回的相似文本数量
+* `similar_text_pair`: 由相似文本对构成的评估集
+* `corpus_file`: 召回库数据 corpus_file
+
+也可以使用bash脚本：
+
+```
+sh scripts/train.sh
+```
+
+<a name="模型评估"></a>
+
+## 6. 模型评估
+
+评估脚本命令如下：
+```
+python -u evaluate.py \
+        --similar_text_pair "data/dev.txt" \
+        --recall_result_file "./recall_result_dir/recall_result.txt" \
+        --label_path data/label.txt
+```
+也可以使用bash脚本：
+
+```
+sh scripts/evaluate.sh
+```
+会得到如下的输出结果：
+
+```
+Micro fl score: 99.30934877025769
+Macro f1 score: 78.20694877991563
+```
+
+<a name="模型预测"></a>
+
+## 7. 模型预测
+
+我们可以基于语义索引模型计算文本和标签的语义相似度。
+
+
+### 开始预测
+
+加载训练的语义索引模型，然后计算文本和标签的语义相似度:
+
+```
+root_dir="checkpoints/inbatch/model_best"
+python -u -m paddle.distributed.launch --gpus "0" \
+    predict.py \
+    --device gpu \
+    --params_path "${root_dir}/model_state.pdparams" \
+    --model_name_or_path rocketqa-zh-dureader-query-encoder \
+    --output_emb_size 0 \
+    --batch_size 128 \
+    --max_seq_length 384 \
+    --text_pair_file "data/test.txt"
+```
+
+参数含义说明
+* `device`: 使用 cpu/gpu 进行训练
+* `params_path`： 预训练模型的参数文件名
+* `output_emb_size`: Transformer 顶层输出的文本向量维度
+* `model_name_or_path`: 预训练模型，用于模型和`Tokenizer`的参数初始化。
+* `text_pair_file`: 由文本 Pair 构成的待预测数据集
+
+也可以运行下面的bash脚本：
+
+```
+sh scripts/predict.sh
+```
+predict.sh文件包含了cpu和gpu运行的脚本，默认是gpu运行的脚本
+
+产出如下结果
+```
+0.8841502070426941
+0.7834227681159973
+0.04591505229473114
+0.15116563439369202
+......
+```
+
+<a name="部署"></a>
+
+## 8. 模型部署
+
+模型部署分为：动转静导出，向量引擎，Paddle Inference推理， Paddle Serving服务化这几个部分。为了提升预测速度，通常需要把训练好的模型转换成静态图，然后就可以使用Paddle Inference静态图进行推理，向量引擎则是存放标签的向量的形式，方便快速检索，另外，Paddle Inference可以进一步对模型服务化，即使用Paddle Serving进行服务化，这样可以通过HTTP或者RPC的方式进行调用。Paddle Serving的服务化形式有Pipeline和C++两种形式，Pipeline灵活一点，方便进行修改，C++部署更麻烦一点，但C++的部署形式效率更高。
+
+### 动转静导出
+
+首先把动态图模型转换为静态图：
+
+```
+python export_model.py \
+                 --params_path checkpoints/inbatch/model_best/model_state.pdparams \
+                 --model_name_or_path rocketqa-zh-dureader-query-encoder \
+                 --output_path=./output
+```
+也可以运行下面的bash脚本：
+
+```
+sh scripts/export_model.sh
+```
+### Paddle Inference预测
+
+预测既可以抽取向量也可以计算两个文本的相似度。
+
+修改deploy/python/predict.py中的id2corpus和corpus_list的样本：
+
+```
+# 抽取向量
+id2corpus = {0: {"sentence": "快要到期的“我的资料”怎么续日期？"}}
+# 计算文本和类别的相似度
+corpus_list = [{
+        "sentence": "快要到期的“我的资料”怎么续日期？",
+        'label': '互联网'
+    }, {
+        "sentence": "快要到期的“我的资料”怎么续日期？",
+        'label': '游戏'
+    }]
+
+```
+
+然后使用PaddleInference
+
+```
+python deploy/python/predict.py --model_dir=./output
+```
+也可以运行下面的bash脚本：
+
+```
+sh deploy.sh
+```
+最终输出的是256维度的特征向量和句子对的预测概率：
+
+```
+(1, 768)
+[[ 0.01510728 -0.03822846 -0.0350773  -0.02304687  0.04219331 -0.04335611
+  -0.03983097  0.04164692  0.04074539 -0.02351343  0.04246496 -0.02563381
+   ....
+
+[0.8133385181427002, 0.1509452909231186]
+```
+
+### 向量引擎
+
+模型准备结束以后，开始搭建 Milvus 的向量检索引擎，用于文本语义向量的快速检索，本项目使用[Milvus](https://milvus.io/)开源工具进行向量检索，Milvus 的搭建教程请参考官方教程  [Milvus官方安装教程](https://milvus.io/cn/docs/v1.1.1/milvus_docker-cpu.md)本案例使用的是 Milvus 的1.1.1 CPU版本，建议使用官方的 Docker 安装方式，简单快捷。
+
+
+Milvus 搭建完系统以后就可以插入和检索向量了，首先生成 embedding 向量，每个样本生成768维度的向量：
+
+```
+CUDA_VISIBLE_DEVICES=0 python utils/feature_extract.py \
+        --data_name label \
+        --model_dir ./output \
+        --output_dir data \
+        --corpus_file "./data/label.txt"
+```
+其中 output 目录下存放的是召回的 Paddle Inference 静态图模型。
+
+修改 utils/config.py 的配置 ip 和端口，本项目使用的是8530端口，而 Milvus 默认的是19530，需要根据情况进行修改：
+```
+MILVUS_HOST='your milvus ip'
+MILVUS_PORT = 8530
+```
+
+然后向搭建好的 Milvus 系统插入向量：
+
+```
+python utils/vector_insert.py \
+                    --vector_path ./data/label_embedding.npy
+```
+也可以直接运行：
+
+```bash
+sh scripts/run.sh
+```
+
+### Paddle Serving部署
+
+Paddle Serving 的安装可以参考[Paddle Serving 安装文档](https://github.com/PaddlePaddle/Serving#installation)。需要在服务端和客户端安装相关的依赖，用pip安装Paddle Serving的依赖如下：
+
+```
+pip install paddle-serving-client==0.8.3 -i https://pypi.tuna.tsinghua.edu.cn/simple
+pip install paddle-serving-app==0.8.3 -i https://pypi.tuna.tsinghua.edu.cn/simple
+
+# 如果是CPU部署，只需要安装CPU Server
+pip install paddle-serving-server==0.8.3 -i https://pypi.tuna.tsinghua.edu.cn/simple
+
+# 如果是GPU Server，需要确认环境再选择执行哪一条，推荐使用CUDA 10.2的包
+# CUDA10.2 + Cudnn7 + TensorRT6（推荐）
+pip install paddle-serving-server-gpu==0.8.3.post102 -i https://pypi.tuna.tsinghua.edu.cn/simple
+# CUDA10.1 + TensorRT6
+pip install paddle-serving-server-gpu==0.8.3.post101 -i https://pypi.tuna.tsinghua.edu.cn/simple
+# CUDA11.2 + TensorRT8
+pip install paddle-serving-server-gpu==0.8.3.post112 -i https://pypi.tuna.tsinghua.edu.cn/simple
+```
+更详细的安装信息请参考[链接](https://github.com/PaddlePaddle/Serving/blob/v0.9.0/doc/Install_Linux_Env_CN.md)，安装完依赖后就可以执行下面的步骤。首先把生成的静态图模型导出为 Paddle Serving的格式，命令如下:
+
+```
+python export_to_serving.py \
+    --dirname "output" \
+    --model_filename "inference.get_pooled_embedding.pdmodel" \
+    --params_filename "inference.get_pooled_embedding.pdiparams" \
+    --server_path "./serving_server" \
+    --client_path "./serving_client" \
+    --fetch_alias_names "output_embedding"
+```
+
+参数含义说明
+* `dirname`: 需要转换的模型文件存储路径，Program 结构文件和参数文件均保存在此目录。
+* `model_filename`： 存储需要转换的模型 Inference Program 结构的文件名称。如果设置为 None ，则使用 `__model__` 作为默认的文件名
+* `params_filename`: 存储需要转换的模型所有参数的文件名称。当且仅当所有模型参数被保>存在一个单独的二进制文件中，它才需要被指定。如果模型参数是存储在各自分离的文件中，设置它的值为 None
+* `server_path`: 转换后的模型文件和配置文件的存储路径。默认值为 serving_server
+* `client_path`: 转换后的客户端配置文件存储路径。默认值为 serving_client
+* `fetch_alias_names`: 模型输出的别名设置，比如输入的 input_ids 等，都可以指定成其他名字，默认不指定
+* `feed_alias_names`: 模型输入的别名设置，比如输出 pooled_out 等，都可以重新指定成其他模型，默认不指定
+
+也可以运行下面的 bash 脚本：
+```
+sh scripts/export_to_serving.sh
+```
+
+Paddle Serving的部署采用Pipeline的方式，如果用户有对性能有更高的要求，可以采用C++的部署形式，请参考[Neural Search](https://github.com/PaddlePaddle/PaddleNLP/tree/develop/applications/neural_search/recall/in_batch_negative#c%E7%9A%84%E6%96%B9%E5%BC%8F)：
+
+#### Pipeline方式
+
+启动 Pipeline Server:
+
+```
+cd deploy/python/
+python web_service.py
+```
+
+启动客户端调用 Server, 使用 POST的方式：
+
+向服务端发送 POST 请求示例：
+
+```
+curl -X POST -k http://localhost:8090/ernie/prediction -d '{"key": ["0"], "value": ["{\"sentence\": \"中国农业大学怎么样？可以吗？\"}"]}'
+```
+
+也可以使用 rpc的方式：
+首先修改rpc_client.py中需要预测的样本：
+
+```
+list_data = [{
+    "sentence": "中国农业大学怎么样？可以吗？"
+}]
+```
+然后运行：
+
+```
+python rpc_client.py
+```
+模型的输出为：
+
+```
+PipelineClient::predict pack_data time:1658988633.3673246
+PipelineClient::predict before time:1658988633.3678396
+time to cost :0.014188766479492188 seconds
+['output_embedding']
+(1, 768)
+[[-0.06491912 -0.0133915   0.00937684  0.01285653 -0.02468005  0.03528611
+   0.0623698  -0.06062918  0.02238894 -0.05348937  0.02161925  0.04480227
+   ......
+```
+
+可以看到客户端发送了1条文本，返回这个 embedding 向量
+
+<a name="分类流程"></a>
+
+## 9. 分类流程
+
+为了演示基于检索的文本分类流程，我们使用下面的python脚本来完成整个流程，该分类系统使用了Client Server的模式，即抽取向量的模型部署在服务端，然后启动客户端（Client）端去访问，得到分类的结果。
+
+```
+python run_system.py
+```
+代码内置的测试用例为：
+
+```
+list_data = [{"sentence": "中国农业大学怎么样？可以吗？"}]
+```
+会输出如下的结果：
+
+```
+......
+PipelineClient::predict pack_data time:1658988661.507715
+PipelineClient::predict before time:1658988661.5081818
+Extract feature time to cost :0.02322244644165039 seconds
+Search milvus time cost is 0.06801486015319824 seconds
+{'sentence': '中国农业大学怎么样？可以吗？'} 教育/科学 0.6138255596160889
+{'sentence': '中国农业大学怎么样？可以吗？'} 院校信息 0.9188833236694336
+```
+输出的结果包括特征提取和检索的时间，还包含检索出来文本和对应的标签，通过设定阈值等方式可以得到最终的标签。
+
+## Reference
+
+[1] Vladimir Karpukhin, Barlas Oğuz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, Wen-tau Yih, Dense Passage Retrieval for Open-Domain Question Answering, Preprint 2020.

applications/text_classification/multi_label/retrieval_based/base_model.py

@@ -0,0 +1,180 @@
+# Copyright (c) 2021 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License"
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import abc
+import sys
+
+import numpy as np
+import paddle
+import paddle.nn as nn
+import paddle.nn.functional as F
+
+
+class SemanticIndexBase(nn.Layer):
+
+    def __init__(self, pretrained_model, dropout=None, output_emb_size=None):
+        super().__init__()
+        self.ptm = pretrained_model
+        self.dropout = nn.Dropout(dropout if dropout is not None else 0.1)
+
+        # if output_emb_size is not None, then add Linear layer to reduce embedding_size,
+        # we recommend set output_emb_size = 256 considering the trade-off beteween
+        # recall performance and efficiency
+
+        self.output_emb_size = output_emb_size
+        if output_emb_size > 0:
+            weight_attr = paddle.ParamAttr(
+                initializer=paddle.nn.initializer.TruncatedNormal(std=0.02))
+            self.emb_reduce_linear = paddle.nn.Linear(768,
+                                                      output_emb_size,
+                                                      weight_attr=weight_attr)
+
+    def get_pooled_embedding(self,
+                             input_ids,
+                             token_type_ids=None,
+                             position_ids=None,
+                             attention_mask=None):
+        _, cls_embedding = self.ptm(input_ids, token_type_ids, position_ids,
+                                    attention_mask)
+        if self.output_emb_size > 0:
+            cls_embedding = self.emb_reduce_linear(cls_embedding)
+        cls_embedding = self.dropout(cls_embedding)
+        cls_embedding = F.normalize(cls_embedding, p=2, axis=-1)
+        return cls_embedding
+
+    def get_semantic_embedding(self, data_loader):
+        self.eval()
+        with paddle.no_grad():
+            for batch_data in data_loader:
+                input_ids, token_type_ids = batch_data
+                text_embeddings = self.get_pooled_embedding(
+                    input_ids, token_type_ids=token_type_ids)
+                yield text_embeddings
+
+    def cosine_sim(self,
+                   query_input_ids,
+                   title_input_ids,
+                   query_token_type_ids=None,
+                   query_position_ids=None,
+                   query_attention_mask=None,
+                   title_token_type_ids=None,
+                   title_position_ids=None,
+                   title_attention_mask=None):
+
+        query_cls_embedding = self.get_pooled_embedding(query_input_ids,
+                                                        query_token_type_ids,
+                                                        query_position_ids,
+                                                        query_attention_mask)
+
+        title_cls_embedding = self.get_pooled_embedding(title_input_ids,
+                                                        title_token_type_ids,
+                                                        title_position_ids,
+                                                        title_attention_mask)
+
+        cosine_sim = paddle.sum(query_cls_embedding * title_cls_embedding,
+                                axis=-1)
+        return cosine_sim
+
+    @abc.abstractmethod
+    def forward(self):
+        pass
+
+
+class SemanticIndexBaseStatic(nn.Layer):
+
+    def __init__(self, pretrained_model, dropout=None, output_emb_size=None):
+        super().__init__()
+        self.ptm = pretrained_model
+        self.dropout = nn.Dropout(dropout if dropout is not None else 0.1)
+
+        # if output_emb_size is not None, then add Linear layer to reduce embedding_size,
+        # we recommend set output_emb_size = 256 considering the trade-off beteween
+        # recall performance and efficiency
+
+        self.output_emb_size = output_emb_size
+        if output_emb_size > 0:
+            weight_attr = paddle.ParamAttr(
+                initializer=paddle.nn.initializer.TruncatedNormal(std=0.02))
+            self.emb_reduce_linear = paddle.nn.Linear(768,
+                                                      output_emb_size,
+                                                      weight_attr=weight_attr)
+
+    @paddle.jit.to_static(input_spec=[
+        paddle.static.InputSpec(shape=[None, None], dtype='int64'),
+        paddle.static.InputSpec(shape=[None, None], dtype='int64')
+    ])
+    def get_pooled_embedding(self,
+                             input_ids,
+                             token_type_ids=None,
+                             position_ids=None,
+                             attention_mask=None):
+        _, cls_embedding = self.ptm(input_ids, token_type_ids, position_ids,
+                                    attention_mask)
+
+        if self.output_emb_size > 0:
+            cls_embedding = self.emb_reduce_linear(cls_embedding)
+        cls_embedding = self.dropout(cls_embedding)
+        cls_embedding = F.normalize(cls_embedding, p=2, axis=-1)
+
+        return cls_embedding
+
+    def get_semantic_embedding(self, data_loader):
+        self.eval()
+        with paddle.no_grad():
+            for batch_data in data_loader:
+                input_ids, token_type_ids = batch_data
+
+                text_embeddings = self.get_pooled_embedding(
+                    input_ids, token_type_ids=token_type_ids)
+
+                yield text_embeddings
+
+    def cosine_sim(self,
+                   query_input_ids,
+                   title_input_ids,
+                   query_token_type_ids=None,
+                   query_position_ids=None,
+                   query_attention_mask=None,
+                   title_token_type_ids=None,
+                   title_position_ids=None,
+                   title_attention_mask=None):
+
+        query_cls_embedding = self.get_pooled_embedding(query_input_ids,
+                                                        query_token_type_ids,
+                                                        query_position_ids,
+                                                        query_attention_mask)
+
+        title_cls_embedding = self.get_pooled_embedding(title_input_ids,
+                                                        title_token_type_ids,
+                                                        title_position_ids,
+                                                        title_attention_mask)
+
+        cosine_sim = paddle.sum(query_cls_embedding * title_cls_embedding,
+                                axis=-1)
+        return cosine_sim
+
+    def forward(self,
+                input_ids,
+                token_type_ids=None,
+                position_ids=None,
+                attention_mask=None):
+        _, cls_embedding = self.ptm(input_ids, token_type_ids, position_ids,
+                                    attention_mask)
+
+        if self.output_emb_size > 0:
+            cls_embedding = self.emb_reduce_linear(cls_embedding)
+        cls_embedding = self.dropout(cls_embedding)
+        cls_embedding = F.normalize(cls_embedding, p=2, axis=-1)
+
+        return cls_embedding

applications/text_classification/multi_label/retrieval_based/data.py

@@ -0,0 +1,273 @@
+# Copyright (c) 2021 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import os
+
+import hnswlib
+import numpy as np
+import paddle
+from paddlenlp.utils.log import logger
+
+
+def build_index(corpus_data_loader, model, output_emb_size, hnsw_max_elements,
+                hnsw_ef, hnsw_m):
+
+    index = hnswlib.Index(space='cosine',
+                          dim=output_emb_size if output_emb_size > 0 else 768)
+
+    # Initializing index
+    # max_elements - the maximum number of elements (capacity). Will throw an exception if exceeded
+    # during insertion of an element.
+    # The capacity can be increased by saving/loading the index, see below.
+    #
+    # ef_construction - controls index search speed/build speed tradeoff
+    #
+    # M - is tightly connected with internal dimensionality of the data. Strongly affects memory consumption (~M)
+    # Higher M leads to higher accuracy/run_time at fixed ef/efConstruction
+    index.init_index(max_elements=hnsw_max_elements,
+                     ef_construction=hnsw_ef,
+                     M=hnsw_m)
+
+    # Controlling the recall by setting ef:
+    # higher ef leads to better accuracy, but slower search
+    index.set_ef(hnsw_ef)
+
+    # Set number of threads used during batch search/construction
+    # By default using all available cores
+    index.set_num_threads(16)
+    logger.info("start build index..........")
+    all_embeddings = []
+    for text_embeddings in model.get_semantic_embedding(corpus_data_loader):
+        all_embeddings.append(text_embeddings.numpy())
+    all_embeddings = np.concatenate(all_embeddings, axis=0)
+    index.add_items(all_embeddings)
+    logger.info("Total index number:{}".format(index.get_current_count()))
+    return index
+
+
+def create_dataloader(dataset,
+                      mode='train',
+                      batch_size=1,
+                      batchify_fn=None,
+                      trans_fn=None):
+    if trans_fn:
+        dataset = dataset.map(trans_fn)
+    shuffle = True if mode == 'train' else False
+    if mode == 'train':
+        batch_sampler = paddle.io.DistributedBatchSampler(dataset,
+                                                          batch_size=batch_size,
+                                                          shuffle=shuffle)
+    else:
+        batch_sampler = paddle.io.BatchSampler(dataset,
+                                               batch_size=batch_size,
+                                               shuffle=shuffle)
+
+    return paddle.io.DataLoader(dataset=dataset,
+                                batch_sampler=batch_sampler,
+                                collate_fn=batchify_fn,
+                                return_list=True)
+
+
+def convert_example(example,
+                    tokenizer,
+                    max_seq_length=512,
+                    pad_to_max_seq_len=False):
+    """
+    Builds model inputs from a sequence.
+    A BERT sequence has the following format:
+    - single sequence: ``[CLS] X [SEP]``
+    Args:
+        example(obj:`list(str)`): The list of text to be converted to ids.
+        tokenizer(obj:`PretrainedTokenizer`): This tokenizer inherits from :class:`~paddlenlp.transformers.PretrainedTokenizer` 
+            which contains most of the methods. Users should refer to the superclass for more information regarding methods.
+        max_seq_len(obj:`int`): The maximum total input sequence length after tokenization. 
+            Sequences longer than this will be truncated, sequences shorter will be padded.
+        is_test(obj:`False`, defaults to `False`): Whether the example contains label or not.
+    Returns:
+        input_ids(obj:`list[int]`): The list of query token ids.
+        token_type_ids(obj: `list[int]`): List of query sequence pair mask.
+    """
+
+    result = []
+    for key, text in example.items():
+        encoded_inputs = tokenizer(text=text,
+                                   max_seq_len=max_seq_length,
+                                   pad_to_max_seq_len=pad_to_max_seq_len)
+        input_ids = encoded_inputs["input_ids"]
+        token_type_ids = encoded_inputs["token_type_ids"]
+        result += [input_ids, token_type_ids]
+    return result
+
+
+def convert_corpus_example(example,
+                           tokenizer,
+                           max_seq_length=512,
+                           pad_to_max_seq_len=False):
+    """
+    Builds model inputs from a sequence.
+
+    A BERT sequence has the following format:
+
+    - single sequence: ``[CLS] X [SEP]``
+
+    Args:
+        example(obj:`list(str)`): The list of text to be converted to ids.
+        tokenizer(obj:`PretrainedTokenizer`): This tokenizer inherits from :class:`~paddlenlp.transformers.PretrainedTokenizer` 
+            which contains most of the methods. Users should refer to the superclass for more information regarding methods.
+        max_seq_len(obj:`int`): The maximum total input sequence length after tokenization. 
+            Sequences longer than this will be truncated, sequences shorter will be padded.
+        is_test(obj:`False`, defaults to `False`): Whether the example contains label or not.
+
+    Returns:
+        input_ids(obj:`list[int]`): The list of query token ids.
+        token_type_ids(obj: `list[int]`): List of query sequence pair mask.
+    """
+    result = []
+    for k, v in example.items():
+        encoded_inputs = tokenizer(text=v,
+                                   max_seq_len=max_seq_length,
+                                   pad_to_max_seq_len=pad_to_max_seq_len)
+        input_ids = encoded_inputs["input_ids"]
+        token_type_ids = encoded_inputs["token_type_ids"]
+        result += [input_ids, token_type_ids]
+    return result
+
+
+def convert_label_example(example,
+                          tokenizer,
+                          max_seq_length=512,
+                          pad_to_max_seq_len=False):
+    """
+    Builds model inputs from a sequence.
+
+    A BERT sequence has the following format:
+
+    - single sequence: ``[CLS] X [SEP]``
+
+    Args:
+        example(obj:`list(str)`): The list of text to be converted to ids.
+        tokenizer(obj:`PretrainedTokenizer`): This tokenizer inherits from :class:`~paddlenlp.transformers.PretrainedTokenizer` 
+            which contains most of the methods. Users should refer to the superclass for more information regarding methods.
+        max_seq_len(obj:`int`): The maximum total input sequence length after tokenization. 
+            Sequences longer than this will be truncated, sequences shorter will be padded.
+        is_test(obj:`False`, defaults to `False`): Whether the example contains label or not.
+
+    Returns:
+        input_ids(obj:`list[int]`): The list of query token ids.
+        token_type_ids(obj: `list[int]`): List of query sequence pair mask.
+    """
+    result = []
+    for k, v in example.items():
+        encoded_inputs = tokenizer(text=v,
+                                   max_seq_len=max_seq_length,
+                                   pad_to_max_seq_len=pad_to_max_seq_len)
+        input_ids = encoded_inputs["input_ids"]
+        token_type_ids = encoded_inputs["token_type_ids"]
+        result += [input_ids, token_type_ids]
+    return result
+
+
+def read_text_pair(data_path):
+    """Reads data."""
+    with open(data_path, 'r', encoding='utf-8') as f:
+        for line in f:
+            data = line.rstrip().split("\t")
+            if (len(data) != 2):
+                print(data)
+                continue
+            yield {'sentence': data[0], 'label': data[1]}
+
+
+# ANN - active learning ------------------------------------------------------
+def get_latest_checkpoint(args):
+    """
+        Return: (latest_checkpint_path, global_step)
+    """
+    if not os.path.exists(args.save_dir):
+        return args.init_from_ckpt, 0
+
+    subdirectories = list(next(os.walk(args.save_dir))[1])
+
+    def valid_checkpoint(checkpoint):
+        chk_path = os.path.join(args.save_dir, checkpoint)
+        scheduler_path = os.path.join(chk_path, "model_state.pdparams")
+        succeed_flag_file = os.path.join(chk_path, "succeed_flag_file")
+        return os.path.exists(scheduler_path) and os.path.exists(
+            succeed_flag_file)
+
+    trained_steps = [int(s) for s in subdirectories if valid_checkpoint(s)]
+
+    if len(trained_steps) > 0:
+        return os.path.join(args.save_dir, str(max(trained_steps)),
+                            "model_state.pdparams"), max(trained_steps)
+
+    return args.init_from_ckpt, 0
+
+
+# ANN - active learning ------------------------------------------------------
+def get_latest_ann_data(ann_data_dir):
+    if not os.path.exists(ann_data_dir):
+        return None, -1
+
+    subdirectories = list(next(os.walk(ann_data_dir))[1])
+
+    def valid_checkpoint(step):
+        ann_data_file = os.path.join(ann_data_dir, step, "new_ann_data")
+        # succed_flag_file is an empty file that indicates ann data has been generated
+        succeed_flag_file = os.path.join(ann_data_dir, step,
+                                         "succeed_flag_file")
+        return os.path.exists(succeed_flag_file) and os.path.exists(
+            ann_data_file)
+
+    ann_data_steps = [int(s) for s in subdirectories if valid_checkpoint(s)]
+
+    if len(ann_data_steps) > 0:
+        latest_ann_data_file = os.path.join(ann_data_dir,
+                                            str(max(ann_data_steps)),
+                                            "new_ann_data")
+        logger.info(
+            "Using lateset ann_data_file:{}".format(latest_ann_data_file))
+        return latest_ann_data_file, max(ann_data_steps)
+
+    logger.info("no new ann_data, return (None, -1)")
+    return None, -1
+
+
+def label2ids(label_path):
+    label2id = {}
+    with open(label_path) as f:
+        for idx, label in enumerate(f.readlines()):
+            label = label.strip()
+            label2id[label] = idx
+    return label2id
+
+
+def gen_id2corpus(corpus_file):
+    id2corpus = {}
+    with open(corpus_file, 'r', encoding='utf-8') as f:
+        for idx, line in enumerate(f):
+            id2corpus[idx] = line.rstrip()
+    return id2corpus
+
+
+def gen_text_file(similar_text_pair_file):
+    text2similar_text = {}
+    texts = []
+    with open(similar_text_pair_file, 'r', encoding='utf-8') as f:
+        for idx, line in enumerate(f):
+            splited_line = line.rstrip().split("\t")
+            text, similar_text = splited_line[0], ','.join(splited_line[1:])
+            text2similar_text[text] = similar_text
+            texts.append({"text": text})
+    return texts, text2similar_text

applications/text_classification/multi_label/retrieval_based/deploy/python/config_nlp.yml

@@ -0,0 +1,34 @@
+# worker_num, 最大并发数。当build_dag_each_worker=True时, 框架会创建worker_num个进程，每个进程内构建grpcSever和DAG
+# 当build_dag_each_worker=False时，框架会设置主线程grpc线程池的max_workers=worker_num
+worker_num: 20
+# build_dag_each_worker, False，框架在进程内创建一条DAG；True，框架会每个进程内创建多个独立的DAG
+build_dag_each_worker: false
+
+dag:
+  # op资源类型, True, 为线程模型；False，为进程模型
+  is_thread_op: False
+  # 使用性能分析, True，生成Timeline性能数据，对性能有一定影响；False为不使用
+  tracer:
+    interval_s: 10
+# http端口, rpc_port和http_port不允许同时为空。当rpc_port可用且http_port为空时，不自动生成http_port
+http_port: 8090
+# rpc端口, rpc_port和http_port不允许同时为空。当rpc_port为空且http_port不为空时，会自动将rpc_port设置为http_port+1
+rpc_port: 8080
+op:
+  ernie:
+    # 并发数，is_thread_op=True时，为线程并发；否则为进程并发
+    concurrency: 1
+    # 当op配置没有server_endpoints时，从local_service_conf读取本地服务配置
+    local_service_conf:
+      # client类型，包括brpc, grpc和local_predictor.local_predictor不启动Serving服务，进程内预测
+      client_type: local_predictor
+      #ir_optim
+      ir_optim: True
+      # device_type, 0=cpu, 1=gpu, 2=tensorRT, 3=arm cpu, 4=kunlun xpu
+      device_type: 1
+      # 计算硬件ID，当devices为""或不写时为CPU预测；当devices为"0", "0,1,2"时为GPU预测，表示使用的GPU卡
+      devices: '0'
+      # Fetch结果列表，以client_config中fetch_var的alias_name为准, 如果没有设置则全部返回
+      fetch_list: ['output_embedding']
+      # 模型路径
+      model_config: ../../serving_server/

applications/text_classification/multi_label/retrieval_based/deploy/python/deploy.sh

@@ -1,15 +1,15 @@
 # Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
-#
+# 
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
 # You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
+# 
+#     http://www.apache.org/licenses/LICENSE-2.0
+# 
 # Unless required by applicable law or agreed to in writing, software
 # distributed under the License is distributed on an "AS IS" BASIS,
 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-from ..core_tokenizers.postprocessors import *
+python predict.py --model_dir=../../output

applications/text_classification/multi_label/retrieval_based/deploy/python/predict.py

@@ -0,0 +1,285 @@
+# Copyright (c) 2021 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import argparse
+import os
+import sys
+
+import numpy as np
+from scipy.special import softmax
+from scipy import spatial
+import paddle
+from paddle import inference
+from paddlenlp.transformers import AutoModel, AutoTokenizer
+from paddlenlp.data import Stack, Tuple, Pad
+from paddlenlp.datasets import load_dataset
+from paddlenlp.utils.log import logger
+
+sys.path.append('.')
+
+# yapf: disable
+parser = argparse.ArgumentParser()
+parser.add_argument("--model_dir", type=str, required=True,
+    help="The directory to static model.")
+
+parser.add_argument("--max_seq_length", default=128, type=int,
+    help="The maximum total input sequence length after tokenization. Sequences "
+    "longer than this will be truncated, sequences shorter will be padded.")
+parser.add_argument("--batch_size", default=15, type=int,
+    help="Batch size per GPU/CPU for training.")
+parser.add_argument('--device', choices=['cpu', 'gpu', 'xpu'], default="gpu",
+    help="Select which device to train model, defaults to gpu.")
+
+parser.add_argument('--use_tensorrt', default=False, type=eval, choices=[True, False],
+    help='Enable to use tensorrt to speed up.')
+parser.add_argument("--precision", default="fp32", type=str, choices=["fp32", "fp16", "int8"],
+    help='The tensorrt precision.')
+
+parser.add_argument('--cpu_threads', default=10, type=int,
+    help='Number of threads to predict when using cpu.')
+parser.add_argument('--enable_mkldnn', default=False, type=eval, choices=[True, False],
+    help='Enable to use mkldnn to speed up when using cpu.')
+
+parser.add_argument("--save_log_path", type=str, default="./log_output/",
+    help="The file path to save log.")
+args = parser.parse_args()
+# yapf: enable
+
+
+def convert_example(example,
+                    tokenizer,
+                    max_seq_length=512,
+                    pad_to_max_seq_len=False):
+    """
+    Builds model inputs from a sequence.
+    A BERT sequence has the following format:
+    - single sequence: ``[CLS] X [SEP]``
+    Args:
+        example(obj:`list(str)`): The list of text to be converted to ids.
+        tokenizer(obj:`PretrainedTokenizer`): This tokenizer inherits from :class:`~paddlenlp.transformers.PretrainedTokenizer` 
+            which contains most of the methods. Users should refer to the superclass for more information regarding methods.
+        max_seq_len(obj:`int`): The maximum total input sequence length after tokenization. 
+            Sequences longer than this will be truncated, sequences shorter will be padded.
+        is_test(obj:`False`, defaults to `False`): Whether the example contains label or not.
+    Returns:
+        input_ids(obj:`list[int]`): The list of query token ids.
+        token_type_ids(obj: `list[int]`): List of query sequence pair mask.
+    """
+
+    result = []
+    for key, text in example.items():
+        encoded_inputs = tokenizer(text=text,
+                                   max_seq_len=max_seq_length,
+                                   pad_to_max_seq_len=pad_to_max_seq_len)
+        input_ids = encoded_inputs["input_ids"]
+        token_type_ids = encoded_inputs["token_type_ids"]
+        result += [input_ids, token_type_ids]
+    return result
+
+
+def convert_query_example(example,
+                          tokenizer,
+                          max_seq_length=512,
+                          pad_to_max_seq_len=False):
+    """
+    Builds model inputs from a sequence.
+
+    A BERT sequence has the following format:
+
+    - single sequence: ``[CLS] X [SEP]``
+
+    Args:
+        example(obj:`list(str)`): The list of text to be converted to ids.
+        tokenizer(obj:`PretrainedTokenizer`): This tokenizer inherits from :class:`~paddlenlp.transformers.PretrainedTokenizer` 
+            which contains most of the methods. Users should refer to the superclass for more information regarding methods.
+        max_seq_len(obj:`int`): The maximum total input sequence length after tokenization. 
+            Sequences longer than this will be truncated, sequences shorter will be padded.
+        is_test(obj:`False`, defaults to `False`): Whether the example contains label or not.
+
+    Returns:
+        input_ids(obj:`list[int]`): The list of query token ids.
+        token_type_ids(obj: `list[int]`): List of query sequence pair mask.
+    """
+    result = []
+    encoded_inputs = tokenizer(text=example['sentence'],
+                               max_seq_len=max_seq_length,
+                               pad_to_max_seq_len=pad_to_max_seq_len,
+                               truncation_strategy="longest_first")
+    input_ids = encoded_inputs["input_ids"]
+    token_type_ids = encoded_inputs["token_type_ids"]
+    result += [input_ids, token_type_ids]
+    return result
+
+
+class Predictor(object):
+
+    def __init__(self,
+                 model_dir,
+                 device="gpu",
+                 max_seq_length=128,
+                 batch_size=32,
+                 use_tensorrt=False,
+                 precision="fp32",
+                 cpu_threads=10,
+                 enable_mkldnn=False):
+        self.max_seq_length = max_seq_length
+        self.batch_size = batch_size
+
+        model_file = model_dir + "/inference.get_pooled_embedding.pdmodel"
+        params_file = model_dir + "/inference.get_pooled_embedding.pdiparams"
+        if not os.path.exists(model_file):
+            raise ValueError("not find model file path {}".format(model_file))
+        if not os.path.exists(params_file):
+            raise ValueError("not find params file path {}".format(params_file))
+        config = paddle.inference.Config(model_file, params_file)
+
+        if device == "gpu":
+            # set GPU configs accordingly
+            # such as intialize the gpu memory, enable tensorrt
+            config.enable_use_gpu(100, 0)
+            precision_map = {
+                "fp16": inference.PrecisionType.Half,
+                "fp32": inference.PrecisionType.Float32,
+                "int8": inference.PrecisionType.Int8
+            }
+            precision_mode = precision_map[precision]
+
+            if args.use_tensorrt:
+                config.enable_tensorrt_engine(max_batch_size=batch_size,
+                                              min_subgraph_size=30,
+                                              precision_mode=precision_mode)
+        elif device == "cpu":
+            # set CPU configs accordingly,
+            # such as enable_mkldnn, set_cpu_math_library_num_threads
+            config.disable_gpu()
+            if args.enable_mkldnn:
+                # cache 10 different shapes for mkldnn to avoid memory leak
+                config.set_mkldnn_cache_capacity(10)
+                config.enable_mkldnn()
+            config.set_cpu_math_library_num_threads(args.cpu_threads)
+        elif device == "xpu":
+            # set XPU configs accordingly
+            config.enable_xpu(100)
+
+        config.switch_use_feed_fetch_ops(False)
+        self.predictor = paddle.inference.create_predictor(config)
+        self.input_handles = [
+            self.predictor.get_input_handle(name)
+            for name in self.predictor.get_input_names()
+        ]
+        self.output_handle = self.predictor.get_output_handle(
+            self.predictor.get_output_names()[0])
+
+    def extract_embedding(self, data, tokenizer):
+        """
+        Predicts the data labels.
+
+        Args:
+            data (obj:`List(str)`): The batch data whose each element is a raw text.
+            tokenizer(obj:`PretrainedTokenizer`): This tokenizer inherits from :class:`~paddlenlp.transformers.PretrainedTokenizer` 
+                which contains most of the methods. Users should refer to the superclass for more information regarding methods.
+
+        Returns:
+            results(obj:`dict`): All the feature vectors.
+        """
+        examples = []
+        for idx, text in data.items():
+            print(text)
+            input_ids, segment_ids = convert_query_example(text, tokenizer)
+            examples.append((input_ids, segment_ids))
+
+        batchify_fn = lambda samples, fn=Tuple(
+            Pad(axis=0, pad_val=tokenizer.pad_token_id, dtype="int64"),  # input
+            Pad(axis=0, pad_val=tokenizer.pad_token_id, dtype="int64"
+                ),  # segment
+        ): fn(samples)
+
+        input_ids, segment_ids = batchify_fn(examples)
+        self.input_handles[0].copy_from_cpu(input_ids)
+        self.input_handles[1].copy_from_cpu(segment_ids)
+        self.predictor.run()
+        logits = self.output_handle.copy_to_cpu()
+        return logits
+
+    def predict(self, data, tokenizer):
+        """
+        Predicts the data labels.
+
+        Args:
+            data (obj:`List(str)`): The batch data whose each element is a raw text.
+            tokenizer(obj:`PretrainedTokenizer`): This tokenizer inherits from :class:`~paddlenlp.transformers.PretrainedTokenizer` 
+                which contains most of the methods. Users should refer to the superclass for more information regarding methods.
+
+        Returns:
+            results(obj:`dict`): All the predictions probs.
+        """
+
+        examples = []
+        for idx, text in enumerate(data):
+            input_ids, segment_ids, title_ids, title_segment_ids = convert_example(
+                text, tokenizer)
+
+            examples.append(
+                (input_ids, segment_ids, title_ids, title_segment_ids))
+
+        batchify_fn = lambda samples, fn=Tuple(
+            Pad(axis=0, pad_val=tokenizer.pad_token_id, dtype="int64"),  # input
+            Pad(axis=0, pad_val=tokenizer.pad_token_type_id, dtype="int64"
+                ),  # segment
+            Pad(axis=0, pad_val=tokenizer.pad_token_id, dtype="int64"),  # input
+            Pad(axis=0, pad_val=tokenizer.pad_token_type_id, dtype="int64"
+                ),  # segment
+        ): fn(samples)
+
+        query_ids, query_segment_ids, title_ids, title_segment_ids = batchify_fn(
+            examples)
+        self.input_handles[0].copy_from_cpu(query_ids)
+        self.input_handles[1].copy_from_cpu(query_segment_ids)
+        self.predictor.run()
+        query_logits = self.output_handle.copy_to_cpu()
+
+        self.input_handles[0].copy_from_cpu(title_ids)
+        self.input_handles[1].copy_from_cpu(title_segment_ids)
+        self.predictor.run()
+        title_logits = self.output_handle.copy_to_cpu()
+
+        result = [
+            float(1 - spatial.distance.cosine(arr1, arr2))
+            for arr1, arr2 in zip(query_logits, title_logits)
+        ]
+        return result
+
+
+if __name__ == "__main__":
+    # Define predictor to do prediction.
+    predictor = Predictor(args.model_dir, args.device, args.max_seq_length,
+                          args.batch_size, args.use_tensorrt, args.precision,
+                          args.cpu_threads, args.enable_mkldnn)
+
+    output_emb_size = 256
+    tokenizer = AutoTokenizer.from_pretrained(
+        "rocketqa-zh-dureader-query-encoder")
+    id2corpus = {0: {"sentence": "快要到期的“我的资料”怎么续日期？"}}
+    res = predictor.extract_embedding(id2corpus, tokenizer)
+    print(res.shape)
+    print(res)
+    corpus_list = [{
+        "sentence": "快要到期的“我的资料”怎么续日期？",
+        'label': '互联网'
+    }, {
+        "sentence": "快要到期的“我的资料”怎么续日期？",
+        'label': '游戏'
+    }]
+    res = predictor.predict(corpus_list, tokenizer)
+    print(res)

applications/text_classification/multi_label/retrieval_based/deploy/python/rpc_client.py

@@ -0,0 +1,35 @@
+# Copyright (c) 2021 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+import time
+
+import numpy as np
+from paddle_serving_server.pipeline import PipelineClient
+
+client = PipelineClient()
+client.connect(['127.0.0.1:8080'])
+
+list_data = [{"sentence": "青岛有什么好一点的国际青旅推荐？离海近一点 外国人多一点 氛围好点的"}]
+feed = {}
+for i, item in enumerate(list_data):
+    feed[str(i)] = str(item)
+
+print(feed)
+start_time = time.time()
+ret = client.predict(feed_dict=feed)
+end_time = time.time()
+print("time to cost :{} seconds".format(end_time - start_time))
+result = np.array(eval(ret.value[0]))
+print(ret.key)
+print(result.shape)
+print(result)

applications/text_classification/multi_label/retrieval_based/deploy/python/web_service.py

@@ -0,0 +1,81 @@
+# Copyright (c) 2021 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import numpy as np
+import sys
+
+from paddle_serving_server.web_service import WebService, Op
+
+
+def convert_example(example,
+                    tokenizer,
+                    max_seq_length=512,
+                    pad_to_max_seq_len=False):
+    result = []
+    for text in example:
+        encoded_inputs = tokenizer(text=text['sentence'],
+                                   max_seq_len=max_seq_length,
+                                   pad_to_max_seq_len=pad_to_max_seq_len)
+        input_ids = encoded_inputs["input_ids"]
+        token_type_ids = encoded_inputs["token_type_ids"]
+        result += [input_ids, token_type_ids]
+    return result
+
+
+class ErnieOp(Op):
+
+    def init_op(self):
+        from paddlenlp.transformers import AutoTokenizer
+        model_name_or_path = 'rocketqa-zh-dureader-query-encoder'
+        self.tokenizer = AutoTokenizer.from_pretrained(model_name_or_path)
+
+    def preprocess(self, input_dicts, data_id, log_id):
+        from paddlenlp.data import Stack, Tuple, Pad
+        (_, input_dict), = input_dicts.items()
+        print("input dict", input_dict)
+        batch_size = len(input_dict.keys())
+        examples = []
+        for i in range(batch_size):
+            example = eval(input_dict[str(i)])
+            input_ids, segment_ids = convert_example([example], self.tokenizer)
+            examples.append((input_ids, segment_ids))
+        batchify_fn = lambda samples, fn=Tuple(
+            Pad(axis=0, pad_val=self.tokenizer.pad_token_id, dtype="int64"
+                ),  # input
+            Pad(axis=0, pad_val=self.tokenizer.pad_token_type_id, dtype="int64"
+                ),  # segment
+        ): fn(samples)
+        input_ids, segment_ids = batchify_fn(examples)
+        feed_dict = {}
+        feed_dict['input_ids'] = input_ids
+        feed_dict['token_type_ids'] = segment_ids
+        return feed_dict, False, None, ""
+
+    def postprocess(self, input_dicts, fetch_dict, data_id, log_id):
+        new_dict = {}
+        new_dict["output_embedding"] = str(
+            fetch_dict["output_embedding"].tolist())
+        return new_dict, None, ""
+
+
+class ErnieService(WebService):
+
+    def get_pipeline_response(self, read_op):
+        ernie_op = ErnieOp(name="ernie", input_ops=[read_op])
+        return ernie_op
+
+
+ernie_service = ErnieService(name="ernie")
+ernie_service.prepare_pipeline_config("config_nlp.yml")
+ernie_service.run_service()

applications/text_classification/multi_label/retrieval_based/evaluate.py

@@ -0,0 +1,75 @@
+# Copyright (c) 2021 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import os
+import argparse
+import time
+
+import numpy as np
+from tqdm import tqdm
+
+from metric import MetricReport
+from data import label2ids
+
+# yapf: disable
+parser = argparse.ArgumentParser()
+parser.add_argument("--label_path", type=str,
+                    default='data/label.txt', help="The full path of label file")
+parser.add_argument("--recall_result_file", type=str,
+                    default='./recall_result_dir/recall_result.txt', help="The full path of recall result file")
+parser.add_argument("--similar_text_pair", default='data/dev.txt',
+                    help="The full path of similar pair file")
+
+parser.add_argument("--threshold", default=0.5, type=float,
+                    help="The threshold for selection the labels")
+
+args = parser.parse_args()
+# yapf: enable
+
+
+def evaluate(label2id):
+    metric = MetricReport()
+    text2similar = {}
+    # Encoding labels as one hot
+    with open(args.similar_text_pair, 'r', encoding='utf-8') as f:
+        for line in f:
+            text, similar_text = line.rstrip().rsplit("\t", 1)
+            text2similar[text] = np.zeros(len(label2id))
+            # One hot Encoding
+            for label in similar_text.strip().split(','):
+                text2similar[text][label2id[label]] = 1
+    pred_labels = {}
+    # Convert predicted labels into one hot encoding
+    with open(args.recall_result_file, 'r', encoding='utf-8') as f:
+        for index, line in enumerate(f):
+            text_arr = line.rstrip().split("\t")
+            text, labels, cosine_sim = text_arr
+            # One hot Encoding
+            if (text not in pred_labels):
+                pred_labels[text] = np.zeros(len(label2id))
+            if (float(cosine_sim) > args.threshold):
+                for label in labels.split(','):
+                    pred_labels[text][label2id[label]] = float(cosine_sim)
+
+        for text, probs in tqdm(pred_labels.items()):
+            metric.update(probs, text2similar[text])
+
+        micro_f1_score, macro_f1_score = metric.accumulate()
+        print("Micro fl score: {}".format(micro_f1_score * 100))
+        print("Macro f1 score: {}".format(macro_f1_score * 100))
+
+
+if __name__ == "__main__":
+    label2id = label2ids(args.label_path)
+    evaluate(label2id)

applications/text_classification/multi_label/retrieval_based/export_model.py

@@ -0,0 +1,65 @@
+# Copyright (c) 2021 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import argparse
+import os
+from functools import partial
+
+import numpy as np
+import paddle
+import paddle.nn.functional as F
+from paddlenlp.data import Stack, Tuple, Pad
+from paddlenlp.transformers import AutoModel, AutoTokenizer
+
+from base_model import SemanticIndexBase, SemanticIndexBaseStatic
+
+# yapf: disable
+parser = argparse.ArgumentParser()
+parser.add_argument("--params_path", type=str, required=True,
+                    default='./checkpoint/model_900/model_state.pdparams', help="The path to model parameters to be loaded.")
+parser.add_argument("--output_path", type=str, default='./output',
+                    help="The path of model parameter in static graph to be saved.")
+parser.add_argument("--output_emb_size", default=0,
+                    type=int, help="output_embedding_size")
+parser.add_argument("--model_name_or_path", default='rocketqa-zh-dureader-query-encoder',
+                    type=str, help='The pretrained model used for training')
+args = parser.parse_args()
+# yapf: enable
+
+if __name__ == "__main__":
+    pretrained_model = AutoModel.from_pretrained(args.model_name_or_path)
+    tokenizer = AutoTokenizer.from_pretrained(args.model_name_or_path)
+    model = SemanticIndexBaseStatic(pretrained_model,
+                                    output_emb_size=args.output_emb_size)
+
+    if args.params_path and os.path.isfile(args.params_path):
+        state_dict = paddle.load(args.params_path)
+        model.set_dict(state_dict)
+        print("Loaded parameters from %s" % args.params_path)
+    else:
+        raise ValueError(
+            "Please set --params_path with correct pretrained model file")
+    model.eval()
+    # Convert to static graph with specific input description
+    model = paddle.jit.to_static(
+        model,
+        input_spec=[
+            paddle.static.InputSpec(shape=[None, None],
+                                    dtype="int64"),  # input_ids
+            paddle.static.InputSpec(shape=[None, None],
+                                    dtype="int64")  # segment_ids
+        ])
+    # Save in static graph model.
+    save_path = os.path.join(args.output_path, "inference")
+    paddle.jit.save(model, save_path)

applications/text_classification/multi_label/retrieval_based/export_to_serving.py

@@ -0,0 +1,49 @@
+# Copyright (c) 2021 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import argparse
+
+import paddle_serving_client.io as serving_io
+
+# yapf: disable
+parser = argparse.ArgumentParser()
+parser.add_argument("--dirname", type=str, required=True,
+                    default='./output', help="Path of saved model files. Program file and parameter files are saved in this directory.")
+parser.add_argument("--model_filename", type=str, required=True,
+                    default='inference.get_pooled_embedding.pdmodel', help="The name of file to load the inference program. If it is None, the default filename __model__ will be used.")
+parser.add_argument("--params_filename", type=str, required=True,
+                    default='inference.get_pooled_embedding.pdiparams', help="The name of file to load all parameters. It is only used for the case that all parameters were saved in a single binary file. If parameters were saved in separate files, set it as None. Default: None.")
+parser.add_argument("--server_path", type=str, default='./serving_server',
+                    help="The path of server parameter in static graph to be saved.")
+parser.add_argument("--client_path", type=str, default='./serving_client',
+                    help="The path of client parameter in static graph to be saved.")
+parser.add_argument("--feed_alias_names", type=str, default=None,
+                    help='set alias names for feed vars, split by comma \',\', you should run --show_proto to check the number of feed vars')
+parser.add_argument("--fetch_alias_names", type=str, default=None,
+                    help='set alias names for feed vars, split by comma \',\', you should run --show_proto to check the number of fetch vars')
+parser.add_argument("--show_proto", type=bool, default=False,
+                    help='If yes, you can preview the proto and then determine your feed var alias name and fetch var alias name.')
+# yapf: enable
+
+if __name__ == "__main__":
+    args = parser.parse_args()
+    serving_io.inference_model_to_serving(
+        dirname=args.dirname,
+        serving_server=args.server_path,
+        serving_client=args.client_path,
+        model_filename=args.model_filename,
+        params_filename=args.params_filename,
+        show_proto=args.show_proto,
+        feed_alias_names=args.feed_alias_names,
+        fetch_alias_names=args.fetch_alias_names)

applications/text_classification/multi_label/retrieval_based/metric.py

@@ -0,0 +1,85 @@
+# Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import numpy as np
+from sklearn.metrics import f1_score, classification_report
+from paddle.metric import Metric
+from paddlenlp.utils.log import logger
+
+
+class MetricReport(Metric):
+    """
+    F1 score for multi-label text classification task.
+    """
+
+    def __init__(self, name='MetricReport', average='micro'):
+        super(MetricReport, self).__init__()
+        self.average = average
+        self._name = name
+        self.reset()
+
+    def reset(self):
+        """
+        Resets all of the metric state.
+        """
+        self.y_prob = None
+        self.y_true = None
+
+    def f1_score(self, y_prob):
+        """
+        Compute micro f1 score and macro f1 score
+        """
+        threshold = 0.5
+        self.y_pred = y_prob > threshold
+        micro_f1_score = f1_score(y_pred=self.y_pred,
+                                  y_true=self.y_true,
+                                  average='micro')
+        macro_f1_score = f1_score(y_pred=self.y_pred,
+                                  y_true=self.y_true,
+                                  average='macro')
+        return micro_f1_score, macro_f1_score
+
+    def update(self, probs, labels):
+        """
+        Update the probability and label
+        """
+        if self.y_prob is not None:
+            self.y_prob = np.append(self.y_prob, probs, axis=0)
+        else:
+            self.y_prob = probs
+        if self.y_true is not None:
+            self.y_true = np.append(self.y_true, labels, axis=0)
+        else:
+            self.y_true = labels
+
+    def accumulate(self):
+        """
+        Returns micro f1 score and macro f1 score
+        """
+        micro_f1_score, macro_f1_score = self.f1_score(y_prob=self.y_prob)
+        return micro_f1_score, macro_f1_score
+
+    def report(self):
+        """
+        Returns classification report
+        """
+        self.y_pred = self.y_prob > 0.5
+        logger.info("classification report:\n" +
+                    classification_report(self.y_true, self.y_pred, digits=4))
+
+    def name(self):
+        """
+        Returns metric name
+        """
+        return self._name

applications/text_classification/multi_label/retrieval_based/model.py

@@ -0,0 +1,78 @@
+# Copyright (c) 2021 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License"
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import sys
+
+import numpy as np
+import paddle
+import paddle.nn as nn
+import paddle.nn.functional as F
+
+from base_model import SemanticIndexBase
+
+
+class SemanticIndexBatchNeg(SemanticIndexBase):
+
+    def __init__(self,
+                 pretrained_model,
+                 dropout=None,
+                 margin=0.3,
+                 scale=30,
+                 output_emb_size=None):
+        super().__init__(pretrained_model, dropout, output_emb_size)
+
+        self.margin = margin
+        # Used scaling cosine similarity to ease converge
+        self.sacle = scale
+
+    def forward(self,
+                query_input_ids,
+                title_input_ids,
+                query_token_type_ids=None,
+                query_position_ids=None,
+                query_attention_mask=None,
+                title_token_type_ids=None,
+                title_position_ids=None,
+                title_attention_mask=None):
+
+        query_cls_embedding = self.get_pooled_embedding(query_input_ids,
+                                                        query_token_type_ids,
+                                                        query_position_ids,
+                                                        query_attention_mask)
+
+        title_cls_embedding = self.get_pooled_embedding(title_input_ids,
+                                                        title_token_type_ids,
+                                                        title_position_ids,
+                                                        title_attention_mask)
+
+        cosine_sim = paddle.matmul(query_cls_embedding,
+                                   title_cls_embedding,
+                                   transpose_y=True)
+
+        # Substract margin from all positive samples cosine_sim()
+        margin_diag = paddle.full(shape=[query_cls_embedding.shape[0]],
+                                  fill_value=self.margin,
+                                  dtype=paddle.get_default_dtype())
+
+        cosine_sim = cosine_sim - paddle.diag(margin_diag)
+
+        # Scale cosine to ease training converge
+        cosine_sim *= self.sacle
+
+        labels = paddle.arange(0, query_cls_embedding.shape[0], dtype='int64')
+        labels = paddle.reshape(labels, shape=[-1, 1])
+
+        loss = F.cross_entropy(input=cosine_sim, label=labels)
+
+        return loss

applications/text_classification/multi_label/retrieval_based/predict.py

@@ -0,0 +1,118 @@
+# Copyright (c) 2021 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from functools import partial
+import argparse
+import sys
+import os
+import random
+import time
+
+import numpy as np
+import paddle
+import paddle.nn.functional as F
+from paddlenlp.transformers import AutoModel, AutoTokenizer
+from paddlenlp.datasets import load_dataset
+from paddlenlp.data import Stack, Tuple, Pad
+
+from data import read_text_pair, convert_example, create_dataloader
+from base_model import SemanticIndexBase
+
+# yapf: disable
+parser = argparse.ArgumentParser()
+parser.add_argument("--text_pair_file", type=str,
+                    required=True, help="The full path of input file")
+parser.add_argument("--params_path", type=str, required=True,
+                    help="The path to model parameters to be loaded.")
+parser.add_argument("--max_seq_length", default=64, type=int, help="The maximum total input sequence length after tokenization. "
+                    "Sequences longer than this will be truncated, sequences shorter will be padded.")
+parser.add_argument("--batch_size", default=32, type=int,
+                    help="Batch size per GPU/CPU for training.")
+parser.add_argument("--output_emb_size", default=None,
+                    type=int, help="output_embedding_size")
+parser.add_argument('--device', choices=['cpu', 'gpu'], default="gpu",
+                    help="Select which device to train model, defaults to gpu.")
+parser.add_argument("--pad_to_max_seq_len", action="store_true",
+                    help="Whether to pad to max seq length.")
+parser.add_argument("--model_name_or_path",default='rocketqa-zh-dureader-query-encoder',type=str,help='The pretrained model used for training')
+args = parser.parse_args()
+# yapf: enable
+
+
+def predict(model, data_loader):
+    """
+    Predicts the data labels.
+
+    Args:
+        model (obj:`SemanticIndexBase`): A model to extract text embedding or calculate similarity of text pair.
+        data_loaer (obj:`List(Example)`): The processed data ids of text pair: [query_input_ids, query_token_type_ids, title_input_ids, title_token_type_ids]
+    Returns:
+        results(obj:`List`): cosine similarity of text pairs.
+    """
+    cosine_sims = []
+    model.eval()
+    with paddle.no_grad():
+        for batch_data in data_loader:
+            query_input_ids, query_token_type_ids, title_input_ids, title_token_type_ids = batch_data
+            batch_cosine_sim = model.cosine_sim(
+                query_input_ids=query_input_ids,
+                title_input_ids=title_input_ids,
+                query_token_type_ids=query_token_type_ids,
+                title_token_type_ids=title_token_type_ids).numpy()
+            cosine_sims.append(batch_cosine_sim)
+        cosine_sims = np.concatenate(cosine_sims, axis=0)
+        return cosine_sims
+
+
+if __name__ == "__main__":
+    paddle.set_device(args.device)
+
+    tokenizer = AutoTokenizer.from_pretrained(args.model_name_or_path)
+    trans_func = partial(convert_example,
+                         tokenizer=tokenizer,
+                         max_seq_length=args.max_seq_length,
+                         pad_to_max_seq_len=args.pad_to_max_seq_len)
+    batchify_fn = lambda samples, fn=Tuple(
+        Pad(axis=0, pad_val=tokenizer.pad_token_id, dtype="int64"
+            ),  # query_input
+        Pad(axis=0, pad_val=tokenizer.pad_token_type_id, dtype="int64"
+            ),  # query_segment
+        Pad(axis=0, pad_val=tokenizer.pad_token_id, dtype="int64"
+            ),  # title_input
+        Pad(axis=0, pad_val=tokenizer.pad_token_type_id, dtype="int64"
+            ),  # tilte_segment
+    ): [data for data in fn(samples)]
+    valid_ds = load_dataset(read_text_pair,
+                            data_path=args.text_pair_file,
+                            lazy=False)
+    valid_data_loader = create_dataloader(valid_ds,
+                                          mode='predict',
+                                          batch_size=args.batch_size,
+                                          batchify_fn=batchify_fn,
+                                          trans_fn=trans_func)
+    pretrained_model = AutoModel.from_pretrained(args.model_name_or_path)
+    model = SemanticIndexBase(pretrained_model,
+                              output_emb_size=args.output_emb_size)
+    if args.params_path and os.path.isfile(args.params_path):
+        state_dict = paddle.load(args.params_path)
+        model.set_dict(state_dict)
+        print("Loaded parameters from %s" % args.params_path)
+    else:
+        raise ValueError(
+            "Please set --params_path with correct pretrained model file")
+    cosin_sim = predict(model, valid_data_loader)
+    for idx, cosine in enumerate(cosin_sim):
+        print('{}'.format(cosine))
+        if (idx > 5):
+            break

applications/text_classification/multi_label/retrieval_based/recall.py

@@ -0,0 +1,135 @@
+# Copyright (c) 2021 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# coding=UTF-8
+
+from functools import partial
+import argparse
+import os
+import sys
+import random
+import time
+
+import numpy as np
+import paddle
+import paddle.nn.functional as F
+from paddlenlp.data import Stack, Tuple, Pad
+from paddlenlp.datasets import load_dataset, MapDataset
+from paddlenlp.utils.log import logger
+from paddlenlp.transformers import AutoModel, AutoTokenizer
+
+from base_model import SemanticIndexBase
+from data import convert_corpus_example, create_dataloader
+from data import gen_id2corpus, gen_text_file
+from data import build_index
+
+# yapf: disable
+parser = argparse.ArgumentParser()
+parser.add_argument("--corpus_file", type=str, required=True,
+                    help="The full path of input file")
+parser.add_argument("--similar_text_pair_file", type=str,
+                    required=True, help="The full path of similar text pair file")
+parser.add_argument("--recall_result_dir", type=str, default='recall_result',
+                    help="The full path of recall result file to save")
+parser.add_argument("--recall_result_file", type=str,
+                    default='recall_result_file', help="The file name of recall result")
+parser.add_argument("--params_path", type=str, required=True,
+                    help="The path to model parameters to be loaded.")
+parser.add_argument("--max_seq_length", default=64, type=int, help="The maximum total input sequence length after tokenization. "
+                    "Sequences longer than this will be truncated, sequences shorter will be padded.")
+parser.add_argument("--batch_size", default=32, type=int,
+                    help="Batch size per GPU/CPU for training.")
+parser.add_argument("--output_emb_size", default=None,
+                    type=int, help="output_embedding_size")
+parser.add_argument("--recall_num", default=10, type=int,
+                    help="Recall number for each query from Ann index.")
+parser.add_argument("--hnsw_m", default=100, type=int,
+                    help="Recall number for each query from Ann index.")
+parser.add_argument("--hnsw_ef", default=100, type=int,
+                    help="Recall number for each query from Ann index.")
+parser.add_argument("--hnsw_max_elements", default=1000000,
+                    type=int, help="Recall number for each query from Ann index.")
+parser.add_argument('--device', choices=['cpu', 'gpu'], default="gpu",
+                    help="Select which device to train model, defaults to gpu.")
+parser.add_argument("--model_name_or_path", default='rocketqa-zh-dureader-query-encoder', type=str, help='The pretrained model used for training')
+args = parser.parse_args()
+# yapf: enable
+
+if __name__ == "__main__":
+    paddle.set_device(args.device)
+    rank = paddle.distributed.get_rank()
+    if paddle.distributed.get_world_size() > 1:
+        paddle.distributed.init_parallel_env()
+    tokenizer = AutoTokenizer.from_pretrained(args.model_name_or_path)
+    trans_func = partial(convert_corpus_example,
+                         tokenizer=tokenizer,
+                         max_seq_length=args.max_seq_length)
+    batchify_fn = lambda samples, fn=Tuple(
+        Pad(axis=0, pad_val=tokenizer.pad_token_id, dtype="int64"
+            ),  # text_input
+        Pad(axis=0, pad_val=tokenizer.pad_token_type_id, dtype="int64"
+            ),  # text_segment
+    ): [data for data in fn(samples)]
+    pretrained_model = AutoModel.from_pretrained(args.model_name_or_path)
+    model = SemanticIndexBase(pretrained_model,
+                              output_emb_size=args.output_emb_size)
+    model = paddle.DataParallel(model)
+    # Load pretrained semantic model
+    if args.params_path and os.path.isfile(args.params_path):
+        state_dict = paddle.load(args.params_path)
+        model.set_dict(state_dict)
+        logger.info("Loaded parameters from %s" % args.params_path)
+    else:
+        raise ValueError(
+            "Please set --params_path with correct pretrained model file")
+    id2corpus = gen_id2corpus(args.corpus_file)
+    # conver_example function's input must be dict
+    corpus_list = [{idx: text} for idx, text in id2corpus.items()]
+    corpus_ds = MapDataset(corpus_list)
+    corpus_data_loader = create_dataloader(corpus_ds,
+                                           mode='predict',
+                                           batch_size=args.batch_size,
+                                           batchify_fn=batchify_fn,
+                                           trans_fn=trans_func)
+    # Need better way to get inner model of DataParallel
+    inner_model = model._layers
+    final_index = build_index(corpus_data_loader,
+                              inner_model,
+                              output_emb_size=args.output_emb_size,
+                              hnsw_max_elements=args.hnsw_max_elements,
+                              hnsw_ef=args.hnsw_ef,
+                              hnsw_m=args.hnsw_m)
+    text_list, text2similar_text = gen_text_file(args.similar_text_pair_file)
+    query_ds = MapDataset(text_list)
+    query_data_loader = create_dataloader(query_ds,
+                                          mode='predict',
+                                          batch_size=args.batch_size,
+                                          batchify_fn=batchify_fn,
+                                          trans_fn=trans_func)
+    query_embedding = inner_model.get_semantic_embedding(query_data_loader)
+    if not os.path.exists(args.recall_result_dir):
+        os.mkdir(args.recall_result_dir)
+    recall_result_file = os.path.join(args.recall_result_dir,
+                                      args.recall_result_file)
+    with open(recall_result_file, 'w', encoding='utf-8') as f:
+        for batch_index, batch_query_embedding in enumerate(query_embedding):
+            recalled_idx, cosine_sims = final_index.knn_query(
+                batch_query_embedding.numpy(), args.recall_num)
+            batch_size = len(cosine_sims)
+            for row_index in range(batch_size):
+                text_index = args.batch_size * batch_index + row_index
+                for idx, doc_idx in enumerate(recalled_idx[row_index]):
+                    f.write("{}\t{}\t{}\n".format(
+                        text_list[text_index]["text"], id2corpus[doc_idx],
+                        1.0 - cosine_sims[row_index][idx]))

applications/text_classification/multi_label/retrieval_based/requirements.txt

@@ -0,0 +1,5 @@
+pymilvus==1.1.2
+pandas==0.25.1 
+paddlenlp>=2.3.7    
+hnswlib>=0.5.2
+pybind11

applications/text_classification/multi_label/retrieval_based/run_system.py

@@ -0,0 +1,72 @@
+# Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from functools import partial
+import argparse
+import os
+import sys
+import random
+import time
+
+sys.path.append('utils')
+import numpy as np
+import pandas as pd
+from tqdm import tqdm
+from paddle_serving_server.pipeline import PipelineClient
+
+from data import gen_id2corpus
+from utils.milvus_util import RecallByMilvus
+from utils.config import collection_name, partition_tag
+
+
+def search_in_milvus(text_embedding, corpus_file, query_text):
+    client = RecallByMilvus()
+    start_time = time.time()
+    status, results = client.search(collection_name=collection_name,
+                                    vectors=text_embedding,
+                                    partition_tag=partition_tag)
+    end_time = time.time()
+    print('Search milvus time cost is {} seconds '.format(end_time -
+                                                          start_time))
+    id2corpus = gen_id2corpus(corpus_file)
+    list_data = []
+    for line in results:
+        for item in line:
+            idx = item.id
+            distance = item.distance
+            text = id2corpus[idx]
+            list_data.append([query_text, text, distance])
+    df = pd.DataFrame(list_data,
+                      columns=['query_text', 'label', 'innner_product'])
+    df = df.sort_values(by="innner_product")
+    for index, row in df.iterrows():
+        if (row['innner_product'] > 0.5):
+            print(row['query_text'], row['label'], row['innner_product'])
+
+
+if __name__ == "__main__":
+    client = PipelineClient()
+    client.connect(['127.0.0.1:8080'])
+    corpus_file = "data/label.txt"
+    list_data = [{"sentence": "中国农业大学怎么样？可以吗？"}]
+    feed = {}
+    for i, item in enumerate(list_data):
+        feed[str(i)] = str(item)
+    start_time = time.time()
+    ret = client.predict(feed_dict=feed)
+    end_time = time.time()
+    print("Extract feature time to cost :{} seconds".format(end_time -
+                                                            start_time))
+    result = np.array(eval(ret.value[0]))
+    search_in_milvus(result, corpus_file, list_data[0])

applications/text_classification/multi_label/retrieval_based/scripts/evaluate.sh

@@ -0,0 +1,18 @@
+# Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
+# 
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+# 
+#     http://www.apache.org/licenses/LICENSE-2.0
+# 
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+python -u evaluate.py \
+        --similar_text_pair "data/dev.txt" \
+        --recall_result_file "./recall_result_dir/recall_result.txt" \
+        --label_path data/label.txt

applications/text_classification/multi_label/retrieval_based/scripts/export_model.sh

@@ -0,0 +1,18 @@
+# Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
+# 
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+# 
+#     http://www.apache.org/licenses/LICENSE-2.0
+# 
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+python export_model.py \
+                 --params_path checkpoints/inbatch/model_best/model_state.pdparams \
+                 --model_name_or_path rocketqa-zh-dureader-query-encoder \
+                 --output_path=./output 

applications/text_classification/multi_label/retrieval_based/scripts/export_to_serving.sh

@@ -0,0 +1,21 @@
+# Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
+# 
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+# 
+#     http://www.apache.org/licenses/LICENSE-2.0
+# 
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+python export_to_serving.py \
+    --dirname "output" \
+    --model_filename "inference.get_pooled_embedding.pdmodel" \
+    --params_filename "inference.get_pooled_embedding.pdiparams" \
+    --server_path "serving_server" \
+    --client_path "serving_client" \
+    --fetch_alias_names "output_embedding"

applications/text_classification/multi_label/retrieval_based/scripts/predict.sh

@@ -0,0 +1,26 @@
+# Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
+# 
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+# 
+#     http://www.apache.org/licenses/LICENSE-2.0
+# 
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# gpu version
+root_dir="checkpoints/inbatch/model_best" 
+python -u -m paddle.distributed.launch --gpus "0" \
+    predict.py \
+    --device gpu \
+    --params_path "${root_dir}/model_state.pdparams" \
+    --model_name_or_path rocketqa-zh-dureader-query-encoder \
+    --output_emb_size 0 \
+    --batch_size 128 \
+    --max_seq_length 384 \
+    --text_pair_file "data/test.txt"
+

applications/text_classification/multi_label/retrieval_based/scripts/run.sh

@@ -0,0 +1,22 @@
+# Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
+# 
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+# 
+#     http://www.apache.org/licenses/LICENSE-2.0
+# 
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+CUDA_VISIBLE_DEVICES=0 python utils/feature_extract.py \
+        --data_name label \
+        --model_dir ./output \
+        --output_dir data \
+        --corpus_file "./data/label.txt" 
+
+python utils/vector_insert.py \
+                    --vector_path ./data/label_embedding.npy

applications/text_classification/multi_label/retrieval_based/scripts/run_build_index.sh

@@ -0,0 +1,30 @@
+# Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
+# 
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+# 
+#     http://www.apache.org/licenses/LICENSE-2.0
+# 
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+root_dir="checkpoints/inbatch" 
+python -u -m paddle.distributed.launch --gpus "0" --log_dir "recall_log/" \
+        recall.py \
+        --device gpu \
+        --recall_result_dir "recall_result_dir" \
+        --recall_result_file "recall_result.txt" \
+        --params_path "${root_dir}/model_best/model_state.pdparams" \
+        --model_name_or_path rocketqa-zh-dureader-query-encoder \
+        --hnsw_m 100 \
+        --hnsw_ef 100 \
+        --batch_size 64 \
+        --output_emb_size 0 \
+        --max_seq_length 384 \
+        --recall_num 5 \
+        --similar_text_pair "data/dev.txt" \
+        --corpus_file "data/label.txt" 

applications/text_classification/multi_label/retrieval_based/scripts/train.sh

@@ -0,0 +1,36 @@
+# Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
+# 
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+# 
+#     http://www.apache.org/licenses/LICENSE-2.0
+# 
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# GPU training
+root_path=inbatch
+data_path=data
+python -u -m paddle.distributed.launch --gpus "0,1" \
+    train.py \
+    --device gpu \
+    --save_dir ./checkpoints/${root_path} \
+    --batch_size 24 \
+    --learning_rate 5E-5 \
+    --epochs 100 \
+    --output_emb_size 0 \
+    --save_steps 50 \
+    --max_seq_length 384 \
+    --warmup_proportion 0.0 \
+    --margin 0.2 \
+    --recall_result_dir "recall_result_dir" \
+    --recall_result_file "recall_result.txt" \
+    --train_set_file ${data_path}/train.txt \
+    --corpus_file ${data_path}/label.txt   \
+    --similar_text_pair_file ${data_path}/dev.txt \
+    --evaluate True \
+    --model_name_or_path rocketqa-zh-dureader-query-encoder

applications/text_classification/multi_label/retrieval_based/train.py

@@ -0,0 +1,300 @@
+# Copyright (c) 2021 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+import argparse
+import os
+import random
+import time
+import numpy as np
+from functools import partial
+from collections import Counter
+
+import paddle
+import paddle.nn as nn
+from paddlenlp.utils.log import logger
+from paddlenlp.data import Tuple, Pad
+from paddlenlp.datasets import load_dataset, MapDataset
+from paddlenlp.transformers import AutoModel, AutoTokenizer
+from paddlenlp.transformers import LinearDecayWithWarmup
+
+from base_model import SemanticIndexBase
+from model import SemanticIndexBatchNeg
+from data import read_text_pair, convert_example, create_dataloader, gen_id2corpus, gen_text_file, convert_corpus_example
+from data import convert_label_example
+from data import build_index, label2ids
+from metric import MetricReport
+
+# yapf: disable
+parser = argparse.ArgumentParser()
+parser.add_argument("--save_dir", default='./checkpoint', type=str,
+                    help="The output directory where the model checkpoints will be written.")
+parser.add_argument("--max_seq_length", default=512, type=int,
+                    help="The maximum total input sequence length after tokenization. "
+                    "Sequences longer than this will be truncated, sequences shorter will be padded.")
+parser.add_argument("--batch_size", default=32, type=int,
+                    help="Batch size per GPU/CPU for training.")
+parser.add_argument("--output_emb_size", default=256,
+                    type=int, help="output_embedding_size")
+parser.add_argument("--learning_rate", default=5E-5, type=float,
+                    help="The initial learning rate for Adam.")
+parser.add_argument("--weight_decay", default=0.0, type=float,
+                    help="Weight decay if we apply some.")
+parser.add_argument("--epochs", default=10, type=int,
+                    help="Total number of training epochs to perform.")
+parser.add_argument("--warmup_proportion", default=0.0, type=float,
+                    help="Linear warmup proption over the training process.")
+parser.add_argument("--init_from_ckpt", type=str, default=None,
+                    help="The path of checkpoint to be loaded.")
+parser.add_argument("--seed", type=int, default=1000,
+                    help="random seed for initialization")
+parser.add_argument('--device', choices=['cpu', 'gpu'], default="cpu",
+                    help="Select which device to train model, defaults to gpu.")
+parser.add_argument('--save_steps', type=int, default=10000,
+                    help="Inteval steps to save checkpoint")
+parser.add_argument('--log_steps', type=int, default=10,
+                    help="Inteval steps to print log")
+parser.add_argument("--train_set_file", type=str,
+                    default='./data/train.txt',
+                    help="The full path of train_set_file.")
+parser.add_argument("--margin", default=0.2, type=float,
+                    help="Margin beteween pos_sample and neg_samples")
+parser.add_argument("--scale", default=30, type=int,
+                    help="Scale for pair-wise margin_rank_loss")
+parser.add_argument("--corpus_file", type=str, default='./data/label.txt',
+                    help="The full path of input file")
+parser.add_argument("--similar_text_pair_file", type=str,
+                    default='./data/dev.txt',
+                    help="The full path of similar text pair file")
+parser.add_argument("--recall_result_dir", type=str, default='./recall_result_dir',
+                    help="The full path of recall result file to save")
+parser.add_argument("--recall_result_file", type=str,
+                    default='recall_result_init.txt', help="The file name of recall result")
+parser.add_argument("--recall_num", default=50, type=int,
+                    help="Recall number for each query from Ann index.")
+parser.add_argument("--hnsw_m", default=100, type=int,
+                    help="Recall number for each query from Ann index.")
+parser.add_argument("--hnsw_ef", default=100, type=int,
+                    help="Recall number for each query from Ann index.")
+parser.add_argument("--hnsw_max_elements", default=1000000,
+                    type=int, help="Recall number for each query from Ann index.")
+parser.add_argument("--evaluate_result", type=str, default='evaluate_result.txt',
+                    help="evaluate_result")
+parser.add_argument('--evaluate', default=True, type=eval, choices=[True, False],
+                    help='whether evaluate while training')
+parser.add_argument("--model_name_or_path",default='rocketqa-zh-dureader-query-encoder',
+                    type=str,help='The pretrained model used for training')
+parser.add_argument("--threshold", default=0.5, type=float,
+                    help="The threshold for selection the labels")
+args = parser.parse_args()
+# yapf: enable
+
+
+def set_seed(seed):
+    """sets random seed"""
+    random.seed(seed)
+    np.random.seed(seed)
+    paddle.seed(seed)
+
+
+@paddle.no_grad()
+def evaluate(model, corpus_data_loader, query_data_loader, recall_result_file,
+             text_list, id2corpus, label2id):
+    metric = MetricReport()
+    # Load pretrained semantic model
+    inner_model = model._layers
+    final_index = build_index(corpus_data_loader,
+                              inner_model,
+                              output_emb_size=args.output_emb_size,
+                              hnsw_max_elements=args.hnsw_max_elements,
+                              hnsw_ef=args.hnsw_ef,
+                              hnsw_m=args.hnsw_m)
+    query_embedding = inner_model.get_semantic_embedding(query_data_loader)
+    with open(recall_result_file, 'w', encoding='utf-8') as f:
+        for batch_index, batch_query_embedding in enumerate(query_embedding):
+            recalled_idx, cosine_sims = final_index.knn_query(
+                batch_query_embedding.numpy(), args.recall_num)
+            batch_size = len(cosine_sims)
+            for row_index in range(batch_size):
+                text_index = args.batch_size * batch_index + row_index
+                for idx, doc_idx in enumerate(recalled_idx[row_index]):
+                    f.write("{}\t{}\t{}\n".format(
+                        text_list[text_index]["text"], id2corpus[doc_idx],
+                        1.0 - cosine_sims[row_index][idx]))
+    text2similar = {}
+    with open(args.similar_text_pair_file, 'r', encoding='utf-8') as f:
+        for line in f:
+            text_arr = line.rstrip().rsplit("\t")
+            text, similar_text = text_arr[0], text_arr[1]
+            text2similar[text] = np.zeros(len(label2id))
+            # One hot Encoding
+            for label in similar_text.strip().split(','):
+                text2similar[text][label2id[label]] = 1
+    # Convert predicted labels into one hot encoding
+    pred_labels = {}
+    with open(recall_result_file, 'r', encoding='utf-8') as f:
+        for index, line in enumerate(f):
+            text_arr = line.rstrip().split("\t")
+            text, labels, cosine_sim = text_arr
+            # One hot Encoding
+            if (text not in pred_labels):
+                pred_labels[text] = np.zeros(len(label2id))
+            if (float(cosine_sim) > args.threshold):
+                for label in labels.split(','):
+                    pred_labels[text][label2id[label]] = float(cosine_sim)
+
+        for text, probs in pred_labels.items():
+            metric.update(probs, text2similar[text])
+        micro_f1_score, macro_f1_score = metric.accumulate()
+    return macro_f1_score
+
+
+def do_train():
+    paddle.set_device(args.device)
+    rank = paddle.distributed.get_rank()
+    if paddle.distributed.get_world_size() > 1:
+        paddle.distributed.init_parallel_env()
+    set_seed(args.seed)
+    train_ds = load_dataset(read_text_pair,
+                            data_path=args.train_set_file,
+                            lazy=False)
+    pretrained_model = AutoModel.from_pretrained(args.model_name_or_path)
+    tokenizer = AutoTokenizer.from_pretrained(args.model_name_or_path)
+    trans_func = partial(convert_example,
+                         tokenizer=tokenizer,
+                         max_seq_length=args.max_seq_length)
+    batchify_fn = lambda samples, fn=Tuple(
+        Pad(axis=0, pad_val=tokenizer.pad_token_id, dtype='int64'
+            ),  # query_input
+        Pad(axis=0, pad_val=tokenizer.pad_token_type_id, dtype='int64'
+            ),  # query_segment
+        Pad(axis=0, pad_val=tokenizer.pad_token_id, dtype='int64'
+            ),  # title_input
+        Pad(axis=0, pad_val=tokenizer.pad_token_type_id, dtype='int64'
+            ),  # tilte_segment
+    ): [data for data in fn(samples)]
+    train_data_loader = create_dataloader(train_ds,
+                                          mode='train',
+                                          batch_size=args.batch_size,
+                                          batchify_fn=batchify_fn,
+                                          trans_fn=trans_func)
+    model = SemanticIndexBatchNeg(pretrained_model,
+                                  margin=args.margin,
+                                  scale=args.scale,
+                                  output_emb_size=args.output_emb_size)
+    if args.init_from_ckpt and os.path.isfile(args.init_from_ckpt):
+        state_dict = paddle.load(args.init_from_ckpt)
+        model.set_dict(state_dict)
+        print("warmup from:{}".format(args.init_from_ckpt))
+    model = paddle.DataParallel(model)
+    batchify_fn_dev = lambda samples, fn=Tuple(
+        Pad(axis=0, pad_val=tokenizer.pad_token_id, dtype='int64'
+            ),  # text_input
+        Pad(axis=0, pad_val=tokenizer.pad_token_type_id, dtype='int64'
+            ),  # text_segment
+    ): [data for data in fn(samples)]
+    if (args.evaluate):
+        eval_func = partial(convert_example,
+                            tokenizer=tokenizer,
+                            max_seq_length=args.max_seq_length)
+        id2corpus = gen_id2corpus(args.corpus_file)
+        label2id = label2ids(args.corpus_file)
+        # conver_example function's input must be dict
+        corpus_list = [{idx: text} for idx, text in id2corpus.items()]
+        corpus_ds = MapDataset(corpus_list)
+        corpus_data_loader = create_dataloader(corpus_ds,
+                                               mode='predict',
+                                               batch_size=args.batch_size,
+                                               batchify_fn=batchify_fn_dev,
+                                               trans_fn=eval_func)
+        query_func = partial(convert_example,
+                             tokenizer=tokenizer,
+                             max_seq_length=args.max_seq_length)
+        text_list, _ = gen_text_file(args.similar_text_pair_file)
+        query_ds = MapDataset(text_list)
+        query_data_loader = create_dataloader(query_ds,
+                                              mode='predict',
+                                              batch_size=args.batch_size,
+                                              batchify_fn=batchify_fn_dev,
+                                              trans_fn=query_func)
+        if not os.path.exists(args.recall_result_dir):
+            os.mkdir(args.recall_result_dir)
+        recall_result_file = os.path.join(args.recall_result_dir,
+                                          args.recall_result_file)
+    num_training_steps = len(train_data_loader) * args.epochs
+    lr_scheduler = LinearDecayWithWarmup(args.learning_rate, num_training_steps,
+                                         args.warmup_proportion)
+    # Generate parameter names needed to perform weight decay.
+    # All bias and LayerNorm parameters are excluded.
+    decay_params = [
+        p.name for n, p in model.named_parameters()
+        if not any(nd in n for nd in ["bias", "norm"])
+    ]
+    optimizer = paddle.optimizer.AdamW(
+        learning_rate=lr_scheduler,
+        parameters=model.parameters(),
+        weight_decay=args.weight_decay,
+        apply_decay_param_fun=lambda x: x in decay_params,
+        grad_clip=nn.ClipGradByNorm(clip_norm=1.0))
+    global_step = 0
+    best_score = 0.0
+    tic_train = time.time()
+    for epoch in range(1, args.epochs + 1):
+        for step, batch in enumerate(train_data_loader, start=1):
+            query_input_ids, query_token_type_ids, title_input_ids, title_token_type_ids = batch
+            loss = model(query_input_ids=query_input_ids,
+                         title_input_ids=title_input_ids,
+                         query_token_type_ids=query_token_type_ids,
+                         title_token_type_ids=title_token_type_ids)
+            global_step += 1
+            if global_step % args.log_steps == 0 and rank == 0:
+                print(
+                    "global step %d, epoch: %d, batch: %d, loss: %.5f, speed: %.2f step/s"
+                    % (global_step, epoch, step, loss, 10 /
+                       (time.time() - tic_train)))
+                tic_train = time.time()
+            loss.backward()
+            optimizer.step()
+            lr_scheduler.step()
+            optimizer.clear_grad()
+            if not args.evaluate and rank == 0:
+                if global_step % args.save_steps == 0 and rank == 0:
+                    save_dir = os.path.join(args.save_dir,
+                                            "model_%d" % global_step)
+                    if not os.path.exists(save_dir):
+                        os.makedirs(save_dir)
+                    save_param_path = os.path.join(save_dir,
+                                                   'model_state.pdparams')
+                    paddle.save(model.state_dict(), save_param_path)
+                    tokenizer.save_pretrained(save_dir)
+        if args.evaluate and rank == 0:
+            print("evaluating")
+            macro_f1_score = evaluate(model, corpus_data_loader,
+                                      query_data_loader, recall_result_file,
+                                      text_list, id2corpus, label2id)
+            if macro_f1_score > best_score:
+                best_score = macro_f1_score
+                save_dir = os.path.join(args.save_dir, "model_best")
+                if not os.path.exists(save_dir):
+                    os.makedirs(save_dir)
+                save_param_path = os.path.join(save_dir, 'model_state.pdparams')
+                paddle.save(model.state_dict(), save_param_path)
+                tokenizer.save_pretrained(save_dir)
+                with open(os.path.join(save_dir, "train_result.txt"),
+                          'a',
+                          encoding='utf-8') as fp:
+                    fp.write('epoch=%d, global_step: %d, Macro f1: %s\n' %
+                             (epoch, global_step, macro_f1_score))
+
+
+if __name__ == "__main__":
+    do_train()

applications/text_classification/multi_label/retrieval_based/utils/__init__.py

@@ -4,12 +4,10 @@
 # you may not use this file except in compliance with the License.
 # You may obtain a copy of the License at
 #
-# http://www.apache.org/licenses/LICENSE-2.0
+#     http://www.apache.org/licenses/LICENSE-2.0
 #
 # Unless required by applicable law or agreed to in writing, software
 # distributed under the License is distributed on an "AS IS" BASIS,
 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 # See the License for the specific language governing permissions and
 # limitations under the License.
-
-from ..core_tokenizers.models import *

applications/text_classification/multi_label/retrieval_based/utils/config.py

@@ -0,0 +1,35 @@
+# Copyright (c) 2021 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from milvus import MetricType, IndexType
+
+MILVUS_HOST = 'localhost'
+MILVUS_PORT = 8530
+
+output_emb_size = 0
+
+collection_param = {
+    'dimension': output_emb_size if output_emb_size > 0 else 768,
+    'index_file_size': 256,
+    'metric_type': MetricType.IP
+}
+
+index_type = IndexType.FLAT
+index_param = {'nlist': 1000}
+
+top_k = 20
+search_param = {'nprobe': 20}
+
+collection_name = 'multi_label'
+partition_tag = 'partition_2'

applications/text_classification/multi_label/retrieval_based/utils/feature_extract.py

@@ -0,0 +1,217 @@
+# Copyright (c) 2021 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import argparse
+import os
+import sys
+
+from tqdm import tqdm
+import numpy as np
+from scipy.special import softmax
+
+import paddle
+from paddle import inference
+import paddlenlp as ppnlp
+from paddlenlp.data import Stack, Tuple, Pad
+from paddlenlp.utils.log import logger
+
+# yapf: disable
+parser = argparse.ArgumentParser()
+parser.add_argument("--model_dir", type=str, required=True,
+    help="The directory to static model.")
+
+parser.add_argument("--corpus_file", type=str, required=True,
+    help="The corpus_file path.")
+
+parser.add_argument("--output_dir", type=str, required=True,
+    help="The ouput path.")
+
+parser.add_argument("--max_seq_length", default=64, type=int,
+    help="The maximum total input sequence length after tokenization. Sequences "
+    "longer than this will be truncated, sequences shorter will be padded.")
+parser.add_argument("--batch_size", default=32, type=int,
+    help="Batch size per GPU/CPU for training.")
+parser.add_argument('--device', choices=['cpu', 'gpu', 'xpu'], default="gpu",
+    help="Select which device to train model, defaults to gpu.")
+parser.add_argument("--data_name", type=str, required=True, help="The dataset name.")
+parser.add_argument('--use_tensorrt', default=False, type=eval, choices=[True, False],
+    help='Enable to use tensorrt to speed up.')
+parser.add_argument("--precision", default="fp32", type=str, choices=["fp32", "fp16", "int8"],
+    help='The tensorrt precision.')
+parser.add_argument("--model_name_or_path",default='rocketqa-zh-dureader-query-encoder',type=str,help='The pretrained model used for training')
+parser.add_argument('--cpu_threads', default=10, type=int,
+    help='Number of threads to predict when using cpu.')
+parser.add_argument('--enable_mkldnn', default=False, type=eval, choices=[True, False],
+    help='Enable to use mkldnn to speed up when using cpu.')
+
+args = parser.parse_args()
+# yapf: enable
+
+
+def convert_example(example,
+                    tokenizer,
+                    max_seq_length=512,
+                    pad_to_max_seq_len=False):
+    """
+    Builds model inputs from a sequence.
+        
+    A BERT sequence has the following format:
+    - single sequence: ``[CLS] X [SEP]``
+    Args:
+        example(obj:`list(str)`): The list of text to be converted to ids.
+        tokenizer(obj:`PretrainedTokenizer`): This tokenizer inherits from :class:`~paddlenlp.transformers.PretrainedTokenizer` 
+            which contains most of the methods. Users should refer to the superclass for more information regarding methods.
+        max_seq_len(obj:`int`): The maximum total input sequence length after tokenization. 
+            Sequences longer than this will be truncated, sequences shorter will be padded.
+        is_test(obj:`False`, defaults to `False`): Whether the example contains label or not.
+    Returns:
+        input_ids(obj:`list[int]`): The list of query token ids.
+        token_type_ids(obj: `list[int]`): List of query sequence pair mask.
+    """
+
+    result = []
+
+    encoded_inputs = tokenizer(text=example,
+                               max_seq_len=max_seq_length,
+                               pad_to_max_seq_len=pad_to_max_seq_len)
+    input_ids = encoded_inputs["input_ids"]
+    token_type_ids = encoded_inputs["token_type_ids"]
+    result += [input_ids, token_type_ids]
+    return result
+
+
+class Predictor(object):
+
+    def __init__(self,
+                 model_dir,
+                 device="gpu",
+                 max_seq_length=128,
+                 batch_size=32,
+                 use_tensorrt=False,
+                 precision="fp32",
+                 cpu_threads=10,
+                 enable_mkldnn=False):
+        self.max_seq_length = max_seq_length
+        self.batch_size = batch_size
+
+        model_file = model_dir + "/inference.get_pooled_embedding.pdmodel"
+        params_file = model_dir + "/inference.get_pooled_embedding.pdiparams"
+        if not os.path.exists(model_file):
+            raise ValueError("not find model file path {}".format(model_file))
+        if not os.path.exists(params_file):
+            raise ValueError("not find params file path {}".format(params_file))
+        config = paddle.inference.Config(model_file, params_file)
+
+        if device == "gpu":
+            # set GPU configs accordingly
+            # such as intialize the gpu memory, enable tensorrt
+            config.enable_use_gpu(100, 0)
+            precision_map = {
+                "fp16": inference.PrecisionType.Half,
+                "fp32": inference.PrecisionType.Float32,
+                "int8": inference.PrecisionType.Int8
+            }
+            precision_mode = precision_map[precision]
+
+            if args.use_tensorrt:
+                config.enable_tensorrt_engine(max_batch_size=batch_size,
+                                              min_subgraph_size=30,
+                                              precision_mode=precision_mode)
+        elif device == "cpu":
+            # set CPU configs accordingly,
+            # such as enable_mkldnn, set_cpu_math_library_num_threads
+            config.disable_gpu()
+            if args.enable_mkldnn:
+                # cache 10 different shapes for mkldnn to avoid memory leak
+                config.set_mkldnn_cache_capacity(10)
+                config.enable_mkldnn()
+            config.set_cpu_math_library_num_threads(args.cpu_threads)
+        elif device == "xpu":
+            # set XPU configs accordingly
+            config.enable_xpu(100)
+
+        config.switch_use_feed_fetch_ops(False)
+        self.predictor = paddle.inference.create_predictor(config)
+        self.input_handles = [
+            self.predictor.get_input_handle(name)
+            for name in self.predictor.get_input_names()
+        ]
+        self.output_handle = self.predictor.get_output_handle(
+            self.predictor.get_output_names()[0])
+
+    def predict(self, data, tokenizer, data_name):
+        """
+        Predicts the data labels.
+        Args:
+            data (obj:`List(str)`): The batch data whose each element is a raw text.
+            tokenizer(obj:`PretrainedTokenizer`): This tokenizer inherits from :class:`~paddlenlp.transformers.PretrainedTokenizer` 
+                which contains most of the methods. Users should refer to the superclass for more information regarding methods.
+        Returns:
+            results(obj:`dict`): All the predictions labels.
+        """
+
+        batchify_fn = lambda samples, fn=Tuple(
+            Pad(axis=0, pad_val=tokenizer.pad_token_id, dtype='int64'),  # input
+            Pad(axis=0, pad_val=tokenizer.pad_token_type_id, dtype='int64'
+                ),  # segment
+        ): fn(samples)
+
+        all_embeddings = []
+        examples = []
+        for idx, text in tqdm(data.items()):
+            input_ids, segment_ids = convert_example(
+                text,
+                tokenizer,
+                max_seq_length=self.max_seq_length,
+                pad_to_max_seq_len=True)
+            examples.append((input_ids, segment_ids))
+            if (len(examples) >= self.batch_size):
+                input_ids, segment_ids = batchify_fn(examples)
+                self.input_handles[0].copy_from_cpu(input_ids)
+                self.input_handles[1].copy_from_cpu(segment_ids)
+                self.predictor.run()
+                logits = self.output_handle.copy_to_cpu()
+                all_embeddings.append(logits)
+                examples = []
+
+        if (len(examples) > 0):
+            input_ids, segment_ids = batchify_fn(examples)
+            self.input_handles[0].copy_from_cpu(input_ids)
+            self.input_handles[1].copy_from_cpu(segment_ids)
+            self.predictor.run()
+            logits = self.output_handle.copy_to_cpu()
+            all_embeddings.append(logits)
+
+        all_embeddings = np.concatenate(all_embeddings, axis=0)
+        np.save('./{}/{}_embedding'.format(args.output_dir, data_name),
+                all_embeddings)
+
+
+def read_text(file_path):
+    file = open(file_path)
+    id2corpus = {}
+    for idx, line in enumerate(file.readlines()):
+        id2corpus[idx] = line.rstrip()
+    return id2corpus
+
+
+if __name__ == "__main__":
+    predictor = Predictor(args.model_dir, args.device, args.max_seq_length,
+                          args.batch_size, args.use_tensorrt, args.precision,
+                          args.cpu_threads, args.enable_mkldnn)
+    data_name = args.data_name
+    tokenizer = ppnlp.transformers.ErnieTokenizer.from_pretrained(
+        args.model_name_or_path)
+    id2corpus = read_text(args.corpus_file)
+    predictor.predict(id2corpus, tokenizer, data_name)

applications/text_classification/multi_label/retrieval_based/utils/milvus_util.py

@@ -0,0 +1,114 @@
+# Copyright (c) 2021 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from milvus import *
+
+from config import MILVUS_HOST, MILVUS_PORT, collection_param, index_type, index_param
+from config import top_k, search_param
+
+
+class VecToMilvus():
+
+    def __init__(self):
+        self.client = Milvus(host=MILVUS_HOST, port=MILVUS_PORT)
+
+    def has_collection(self, collection_name):
+        try:
+            status, ok = self.client.has_collection(collection_name)
+            return ok
+        except Exception as e:
+            print("Milvus has_table error:", e)
+
+    def creat_collection(self, collection_name):
+        try:
+            collection_param['collection_name'] = collection_name
+            status = self.client.create_collection(collection_param)
+            print(status)
+            return status
+        except Exception as e:
+            print("Milvus create collection error:", e)
+
+    def create_index(self, collection_name):
+        try:
+            status = self.client.create_index(collection_name, index_type,
+                                              index_param)
+            print(status)
+            return status
+        except Exception as e:
+            print("Milvus create index error:", e)
+
+    def has_partition(self, collection_name, partition_tag):
+        try:
+            status, ok = self.client.has_partition(collection_name,
+                                                   partition_tag)
+            return ok
+        except Exception as e:
+            print("Milvus has partition error: ", e)
+
+    def delete_partition(self, collection_name, partition_tag):
+        try:
+            status = self.client.drop_collection(collection_name)
+            return status
+        except Exception as e:
+            print("Milvus has partition error: ", e)
+
+    def create_partition(self, collection_name, partition_tag):
+        try:
+            status = self.client.create_partition(collection_name,
+                                                  partition_tag)
+            print('create partition {} successfully'.format(partition_tag))
+            return status
+        except Exception as e:
+            print('Milvus create partition error: ', e)
+
+    def insert(self, vectors, collection_name, ids=None, partition_tag=None):
+        try:
+            if not self.has_collection(collection_name):
+                self.creat_collection(collection_name)
+                self.create_index(collection_name)
+                print('collection info: {}'.format(
+                    self.client.get_collection_info(collection_name)[1]))
+            if (partition_tag is not None) and (not self.has_partition(
+                    collection_name, partition_tag)):
+                self.create_partition(collection_name, partition_tag)
+            status, ids = self.client.insert(collection_name=collection_name,
+                                             records=vectors,
+                                             ids=ids,
+                                             partition_tag=partition_tag)
+            self.client.flush([collection_name])
+            print(
+                'Insert {} entities, there are {} entities after insert data.'.
+                format(len(ids),
+                       self.client.count_entities(collection_name)[1]))
+            return status, ids
+        except Exception as e:
+            print("Milvus insert error:", e)
+
+
+class RecallByMilvus():
+
+    def __init__(self):
+        self.client = Milvus(host=MILVUS_HOST, port=MILVUS_PORT)
+
+    def search(self, vectors, collection_name, partition_tag=None):
+        try:
+            status, results = self.client.search(
+                collection_name=collection_name,
+                query_records=vectors,
+                top_k=top_k,
+                params=search_param,
+                partition_tag=partition_tag)
+            return status, results
+        except Exception as e:
+            print('Milvus recall error: ', e)

applications/text_classification/multi_label/retrieval_based/utils/vector_insert.py

@@ -0,0 +1,56 @@
+# Copyright (c) 2021 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import random
+import argparse
+
+from tqdm import tqdm
+import numpy as np
+
+from milvus_util import VecToMilvus
+from config import collection_name, partition_tag
+
+# yapf: disable
+parser = argparse.ArgumentParser()
+parser.add_argument("--vector_path", type=str, required=True,
+    help="feature file path.")
+
+args = parser.parse_args()
+# yapf: enable
+
+
+def vector_insert(file_path):
+    embeddings = np.load(file_path)
+    print(embeddings.shape)
+    embedding_ids = [i for i in range(embeddings.shape[0])]
+    print(len(embedding_ids))
+    client = VecToMilvus()
+
+    if (client.has_partition(collection_name, partition_tag)):
+        client.delete_partition(collection_name, partition_tag)
+    data_size = len(embedding_ids)
+    batch_size = 50000
+    for i in tqdm(range(0, data_size, batch_size)):
+        cur_end = i + batch_size
+        if (cur_end > data_size):
+            cur_end = data_size
+        batch_emb = embeddings[np.arange(i, cur_end)]
+        status, ids = client.insert(collection_name=collection_name,
+                                    vectors=batch_emb.tolist(),
+                                    ids=embedding_ids[i:i + batch_size],
+                                    partition_tag=partition_tag)
+
+
+if __name__ == "__main__":
+    vector_insert(args.vector_path)

docs/advanced_guide/prompt.md

@@ -22,11 +22,11 @@ Prompt API 提供了这类算法实现的基本模块，支持[PET](https://arxi
 * [如何定义模板](#如何定义模板)
     * [离散型模板](#离散型模板)
     * [连续型模板](#连续型模板)
+    * [前缀连续型模板](#前缀连续型模板)
     * [快速定义模板](#快速定义模板)
 * [如何定义标签词映射](#如何定义标签词映射)
-    * [单掩码映射](#单掩码映射)
-    * [多掩码映射](#多掩码映射)
-    * [标签词映射分类](#标签词映射分类)
+    * [离散型标签词映射](#离散型标签词映射)
+    * [连续型标签词映射](#连续型标签词映射)
 * [快速开始训练](#快速开始训练)
     * [数据准备](#数据准备)
     * [预训练参数准备](#预训练参数准备)
@@ -39,18 +39,28 @@ Prompt API 提供了这类算法实现的基本模块，支持[PET](https://arxi
 
 ## 如何定义模板
 
-**模板**（Template）的功能是在原有输入文本上增加提示语句，从而将原任务转化为 MLM 任务，可以分为离散型和连续型两种。Prompt API 中提供了统一的数据结构来构造不同类型的模板，输入相应格式的**字符串**，通过解析得到对应的输入模板，即字典构成的列表。
+**模板**（Template）的功能是在原有输入文本上增加提示语句，从而将原任务转化为 MLM 任务，可以分为离散型和连续型两种。Prompt API 中提供了统一的数据结构来构造不同类型的模板，输入相应格式的**字符串**，通过解析得到对应的输入模板。模板由不同字段构成，可任意组合。每个字段中的关键字定义了数据文本或者提示文本，即 `input_ids`，属性可定义该字段是否可截断，以及对应的 `position_ids`，`token_type_ids` 等。
 
 ### 离散型模板
 
 离散型模板 `ManualTemplate` 是直接将提示语句与原始输入文本拼接起来，二者的词向量矩阵共享，均为预训练模型学到的词向量矩阵。可用于实现 PET、RGL 等算法。
 
-**模板关键字**
+**模板关键字及属性**
 
-- ``text`` ：数据集中原始输入文本对应的关键字，包括`text_a`和`text_b`。[数据准备](#数据准备)中介绍了如何将自定义数据集转化为统一格式。
-- ``hard`` ：自定义的文本提示语句。
+- ``text`` ：数据集中原始输入文本对应的关键字，例如，`text_a`、`text_b` 和 `content`。
+- ``hard`` ：自定义的提示语句文本。
 - ``mask`` ：待预测词的占位符。
-- ``sep`` ：用于区分不同的句子。`sep`前后的句子对应不同的`token_type_id`。
+    - ``length`` ：定义 ``mask`` 的数量。
+- ``sep`` ：句间的标志符。不同句子的 `token_type_ids` 需使用 `token_type` 属性定义，默认相同。
+- ``options`` ：数据集字典或者文件中的候选标签序列。
+    - ``add_omask`` ：在每个标签前新增 `[O-MASK]` 字符，用于计算候选标签的预测值。支持实现 [UniMC](https://arxiv.org/pdf/2210.08590.pdf) 算法。
+    - ``add_prompt`` ：给每个标签拼接固定的提示文本，标签位置由 `[OPT]` 标记。支持实现 [EFL](https://arxiv.org/pdf/2104.14690.pdf) 算法。
+
+**模版通用属性**
+
+- `position`: 定义当前字段的起始 `position id`。
+- `token_type`: 定义当前字段及后续字段的 `token type id`。
+- `truncate`: 定义当提示和文本总长度超过最大长度时，当前字段是否可截断。可选 `True` 和 `False`。
 
 **模板定义**
 
@@ -64,16 +74,25 @@ Prompt API 提供了这类算法实现的基本模块，支持[PET](https://arxi
 “{'text': 'text_a'}”和“{'text': 'text_b'}”之间的逻辑关系是{'mask'}
 ```
 
+```
+{'options': './data/label.txt'}{'sep'}下边两句话间的逻辑关系是什么？{'text': 'text_a'}{'sep': None, 'token_type': 1}{'text': 'text_b'}
+```
+其中 `label.txt` 为候选标签的本地文件路径，每行一个候选标签，例如
+
+```
+中立
+蕴含
+矛盾
+```
+
 **样本示例**
 
 例如，对于自然语言推理任务，给定样本
 
 ```python
-from paddlenlp.prompt import InputExample
-sample = InputExample(uid=0,
-                      text_a="心里有些生畏,又不知畏惧什么",
-                      text_b="心里特别开心",
-                      labels="矛盾")
+sample = {
+    "text_a": "心里有些生畏,又不知畏惧什么", "text_b": "心里特别开心", "labels": "矛盾"
+}
 ```
 
 按照模板修改拼接后，最终输入模型的文本数据为
@@ -90,17 +109,17 @@ from paddlenlp.prompt import ManualTemplate
 from paddlenlp.transformers import AutoTokenizer
 
 tokenizer = AutoTokenizer.from_pretrained("ernie-3.0-base-zh")
-template = ManualTemplate(tokenizer=tokenizer,
-                          max_seq_length=512,
-                          template="“{'text': 'text_a'}”和“{'text': 'text_b'}”之间的逻辑关系是{'mask'}")
-input_dict = template.wrap_one_example(sample)
+template = ManualTemplate(prompt="“{'text': 'text_a'}”和“{'text': 'text_b'}”之间的逻辑关系是{'mask'}",
+                          tokenizer=tokenizer,
+                          max_length=512)
+input_dict = template(sample)
 ```
 
 其中初始化参数定义如下
 
+- ``prompt`` ：定义提示语句以及与输入文本组合方式的字符串。
 - ``tokenizer`` ：预训练模型的 tokenizer，用于文本编码。
-- ``max_seq_length`` ：定义输入模型文本的最大长度，包括提示部分。当输入长度超过最大长度时，只会截断`text`关键字对应的输入文本，提示部分不做处理。
-- ``template`` ：定义提示语句以及与输入文本组合方式的字符串。
+- ``max_length`` ：定义输入模型文本的最大长度，包括提示部分。
 
 **使用技巧**
 
@@ -115,24 +134,42 @@ input_dict = template.wrap_one_example(sample)
 
 **模板关键字**
 
-- ``text`` ：数据集中原始输入文本对应的关键字，包括`text_a`和`text_b`。[数据准备](#数据准备)中介绍了如何将自定义数据集转化为统一格式。
+- ``text`` ：数据集中原始输入文本对应的关键字，例如，`text_a`和`text_b`。
 - ``hard`` ：自定义的文本提示语句。
 - ``mask`` ：待预测词的占位符。
-- ``sep`` ：用于区分不同的句子。`sep`前后的句子对应不同的`token_type_id`。
-- ``soft`` 表示连续型提示。若值为 ``None`` ，则使用对应数量的随机初始化向量作为提示；若值为文本，则使用对应长度的连续性向量作为提示，并预训练词向量中文本对应的向量进行初始化。
+- ``sep`` ：句间的标志符。不同句子的 `token_type_ids` 需使用 `token_type` 属性定义，默认相同。
+- ``soft`` 表示连续型提示。若值为 ``None`` ，则随机初始化提示向量；若值为文本，则使用文本对应的预训练字向量初始化提示向量。
+    - ``length`` ：定义 ``soft token`` 的数量。若定义文本长度小于该值，超过部分随机初始化。
+    - ``encoder`` ：定义 `soft token` 的编码器类型，可选 `lstm`，`mlp`。默认为 `None`， 不使用编码器。
+    - ``hidden_size`` ：定义编码器的隐藏层维度。默认与预训练词向量维度相同。
+- ``options`` ：数据集字典或者文件中的候选标签序列。
+    - ``add_omask`` ：在每个标签前新增 `[O-MASK]` 字符，用于计算候选标签的预测值。支持实现 [UniMC](https://arxiv.org/pdf/2210.08590.pdf) 算法。
+    - ``add_prompt`` ：给每个标签拼接固定的提示文本，标签位置由 `[OPT]` 标记。支持实现 [EFL](https://arxiv.org/pdf/2104.14690.pdf) 算法。
+
+**模版通用属性**
+
+- `position`: 定义当前字段的起始 `position id`。
+- `token_type`: 定义当前字段及后续字段的 `token type id`。
+- `truncate`: 定义当提示和文本总长度超过最大长度时，当前字段是否可截断。可选 `True` 和 `False`。
 
 **模板定义**
 
 - 定义长度为 1 的连续型提示，随机初始化：
 
 ```python
-"{'soft': None}{'text': 'text_a'}{'sep'}{'text': 'text_b'}"
+"{'soft'}{'text': 'text_a'}{'sep': None, 'token_type': 1}{'text': 'text_b'}"
 ```
 
-- 定义长度为 10 的连续型提示，随机初始化，其中 ``duplicate`` 参数表示连续型提示的长度（仅在随机初始化时有效，即`soft`值为`None`）：
+- 定义长度为 10 的连续型提示，随机初始化，编码器为 `mlp`：
 
 ```python
-"{'text': 'text_a'}{'sep'}{'text': 'text_b'}{'soft': None, `duplicate`:10}{'mask'}"
+"{'text': 'text_a'}{'sep'}{'text': 'text_b'}{'soft': None, 'length':10, 'encoder': 'mlp'}{'mask'}"
+```
+
+- 定义长度为 15 的连续型提示，使用 `请判断` 初始化前三个 soft token，其余随机初始化，编码器为隐藏层维度为 100 的双层 LSTM：
+
+```python
+"{'text': 'text_a'}{'sep'}{'text': 'text_b'}{'soft': '请判断：', 'length': 15, 'encoder': 'lstm', 'hidden_size': 100}{'mask'}"
 ```
 
 - 定义长度为 15 的连续型提示，使用 `"请判断这两个句子间的逻辑关系："` 的预训练词向量逐一进行初始化：
@@ -156,38 +193,98 @@ from paddlenlp.transformers import AutoTokenizer, AutoModelForMaskedLM
 
 model = AutoModelForMaskedLM.from_pretrained("ernie-3.0-base-zh")
 tokenizer = AutoTokenizer.from_pretrained("ernie-3.0-base-zh")
-template = SoftTemplate(tokenizer=tokenizer,
-                        max_seq_length=512,
-                        model=model,
-                        template="{'text': 'text_a'}{'sep'}{'text': 'text_b'}{'soft': '请判断这两个句子间的逻辑关系：'}{'mask'}",
-                        prompt_encoder='lstm',
-                        encoder_hidden_size=200)
+template = SoftTemplate(prompt="{'text': 'text_a'}{'sep'}{'text': 'text_b'}{'soft': '请判断这两个句子间的逻辑关系：'}{'mask'}",
+                        tokenizer=tokenizer,
+                        max_length=512,
+                        word_embeddings=model.get_input_embeddings())
 ```
 
 其中初始化参数定义如下
 
+- ``prompt`` ：定义连续型模板的提示语句、初始化以及与输入文本组合方式的字符串。
 - ``tokenizer`` ：预训练模型的 tokenizer，用于文本编码。
-- ``max_seq_length`` ：定义输入模型文本的最大长度，包括提示部分。当输入长度超过最大长度时，只会截断`text`关键字对应的输入文本，提示部分不做处理。
-- ``model`` : 预训练语言模型，为了取预训练词向量用于连续型提示向量初始化。
-- ``template`` ：定义连续型模板的提示语句、初始化以及与输入文本组合方式的字符串。
-- ``prompt_encoder`` : 连续型提示向量的编码器，可选 ``mlp`` 和 ``lstm``。默认为 ``None`` ，即无编码器，直接使用向量。
-- ``encoder_hidden_size`` : 连续型提示向量的维度。默认为 ``None`` ，即与预训练词向量维度相同。
+- ``max_seq_length`` ：定义输入模型文本的最大长度，包括提示部分。
+- ``word_embeddings`` ：预训练语言模型的词向量，用于连续型提示向量初始化。
+- ``soft_embeddings`` ：连续型提示向量矩阵，可用于不同模板间的连续型参数共享。设置后将覆盖默认连续型向量矩阵。
 
 **使用技巧**
 
 - 对于分类任务，推荐的连续型提示长度一般为10-20。
 - 对于随机初始化的连续性 prompt 向量，通常用比预训练模型微调更大的学习率来更新参数。
 - 与离散型模板相似，连续型模板对初始化参数也比较敏感。自定义提示语句作为连续性 prompt 向量的初始化参数通常比随机初始化效果好。
-- prompt_encoder 为已有论文中的策略，用于建模不同连续型提示向量之间的序列关系。在实际应用中推荐先去掉 prompt_encoder 调整向量初始化。
+- prompt_encoder 为已有论文中的策略，用于建模不同连续型提示向量之间的序列关系。
+
+
+### 前缀连续型模板
+
+`PrefixTemplate` 同样使用了连续型向量作为提示，与 `SoftTemplate` 的不同，该模版的提示向量不仅仅作用于输入层，每层都会有相应的提示向量。可用于实现 P-Tuning 等算法。
+
+**模板关键字**
+
+- ``text`` ：数据集中原始输入文本对应的关键字，例如，`text_a`和`text_b`。
+- ``hard`` ：自定义的文本提示语句。
+- ``mask`` ：待预测词的占位符。
+- ``sep`` ：句间的标志符。不同句子的 `token_type_ids` 需使用 `token_type` 属性定义，默认相同。
+- ``prefix`` 表示连续型提示，该字段**必须**位于模板首位。若值为 ``None`` ，则随机初始化提示向量；若值为文本，则使用文本对应的预训练字向量初始化提示向量。
+    - ``length`` ：定义 ``soft token`` 的数量。若定义文本长度小于该值，超过部分随机初始化。
+    - ``encoder`` ：定义 `soft token` 的编码器类型，可选 `lstm`，`mlp`。默认为 `None`， 不使用编码器。
+    - ``hidden_size`` ：定义编码器的隐藏层维度。默认与预训练词向量维度相同。
+- ``options`` ：数据集字典或者文件中的候选标签序列。
+    - ``add_omask`` ：在每个标签前新增 `[O-MASK]` 字符，用于计算候选标签的预测值。支持实现 [UniMC](https://arxiv.org/pdf/2210.08590.pdf) 算法。
+    - ``add_prompt`` ：给每个标签拼接固定的提示文本，标签位置由 `[OPT]` 标记。支持实现 [EFL](https://arxiv.org/pdf/2104.14690.pdf) 算法。
+
+**模版通用属性**
+
+- `position`: 定义当前字段的起始 `position id`。
+- `token_type`: 定义当前字段及后续字段的 `token type id`。
+- `truncate`: 定义当提示和文本总长度超过最大长度时，当前字段是否可截断。可选 `True` 和 `False`。
+
+**模板定义**
+
+- 定义长度为 15 的连续型提示，随机初始化：
+
+```python
+"{'prefix': '新闻类别', 'length': 10, 'encoder': 'lstm'}{'text': 'text_a'}"
+```
+
+- 定义混合模板，这里`prefix`关键字对应的提示和`hard`对应的提示对应两套不同的向量：
+
+```python
+"{'prefix': '自然语言推理任务：', 'encoder': 'mlp'}{'text': 'text_a'}{'sep'}{'text': 'text_b'}这两个句子间的逻辑关系是{'mask'}"
+```
+
+
+**调用 API**
+
+```python
+from paddlenlp.prompt import PrefixTemplate
+from paddlenlp.transformers import AutoTokenizer, AutoModelForMaskedLM
+
+model = AutoModelForMaskedLM.from_pretrained("ernie-3.0-base-zh")
+tokenizer = AutoTokenizer.from_pretrained("ernie-3.0-base-zh")
+template = PrefixTemplate(prompt="{'prefix': '任务描述'}{'text': 'text_a'}{'mask'}",
+                          tokenizer=tokenizer,
+                          max_length=512,
+                          model=model,
+                          prefix_dropout=0.1)
+```
+
+其中初始化参数定义如下
+
+- ``prompt`` ：定义连续型模板的提示语句、初始化以及与输入文本组合方式的字符串。
+- ``tokenizer`` ：预训练模型的 tokenizer，用于文本编码。
+- ``max_length`` ：定义输入模型文本的最大长度，包括提示部分。
+- ``model`` ：预训练语言模型，用于连续型提示向量初始化，以及根据模型结构生成每层对应的提示向量。
+- ``prefix_dropout`` ：连续型提示向量的丢弃概率，用于正则化。
 
 
 ### 快速定义模板
 
-PaddleNLP 提供了 ``AutoTemplate`` API 以便快速定义单句输入的手工初始化的连续型模板，同时支持直接按照模板类型自动切换离散型模板和离散型模板。
+PaddleNLP 提供了 ``AutoTemplate`` API 快速定义简化离散型模板，也可根据完整模板字符串自动切换 ManualTemplate、SoftTemplate 和 PrefixTemplate。
 
 **模板定义**
 
-- 只定义用于初始化连续型向量的文本提示，即可得到拼接到句尾的连续型模板输入。例如，
+- 快速定义离散型的文本提示。例如，
 
 ```python
 "这篇文章表达了怎样的情感？"
@@ -196,7 +293,7 @@ PaddleNLP 提供了 ``AutoTemplate`` API 以便快速定义单句输入的手工
 等价于
 
 ```python
-"{'text': 'text_a'}{'soft': '这篇文章表达了怎样的情感？'}{'mask'}"
+"{'text': 'text_a'}{'hard': '这篇文章表达了怎样的情感？'}{'mask'}"
 ```
 
 - 当输入为完整模板字符串时，解析得到的模板与[离散型模板](#离散型模板)和[连续型模板](#连续型模板)中描述的一致。
@@ -210,40 +307,37 @@ from paddlenlp.transformers import AutoTokenizer, AutoModelForMaskedLM
 model = AutoModelForMaskedLM.from_pretrained("ernie-3.0-base-zh")
 tokenizer = AutoTokenizer.from_pretrained("ernie-3.0-base-zh")
 # 离散型模板，返回值为 ManualTemplate 实例
-template = AutoTemplate.create_from(template="{'text': 'text_a'}和{'text': 'text_b'}之间的逻辑关系是{'mask'}",
+template = AutoTemplate.create_from(prompt="这个句子表达了怎样的情感？",
                                     tokenizer=tokenizer,
-                                    max_seq_length=512)
+                                    max_length=512)
+
+template = AutoTemplate.create_from(prompt="这个句子表达了怎样的情感？{'text': 'text_a'}{'mask'}",
+                                    tokenizer=tokenizer,
+                                    max_length=512)
 
 # 连续型模板，返回值为 SoftTemplate 实例
-template = AutoTemplate.create_from(template="{'text': 'text_a'}{'sep'}{'text': 'text_b'}{'soft': '请判断这两个句子间的逻辑关系：'}{'mask'}",
+template = AutoTemplate.create_from(prompt="{'text': 'text_a'}{'sep'}{'text': 'text_b'}{'soft': '请判断这两个句子间的逻辑关系：'}{'mask'}",
                                     tokenizer=tokenizer,
-                                    max_seq_length=512,
-                                    model=model,
-                                    prompt_encoder='lstm',
-                                    encoder_hidden_size=200)
+                                    max_length=512,
+                                    model=model)
 
-# 快速定义单句连续型模板，返回值为 SoftTemplate 实例
-template = AutoTemplate.create_from(template="这篇文章表达了怎样的情感？",
+# 前缀连续型模板，返回值为 PrefixTemplate 实例
+template = AutoTemplate.create_from(prompt="{'prefix': None, 'encoder': 'mlp', 'hidden_size': 50}{'text': 'text_a'}",
                                     tokenizer=tokenizer,
-                                    max_seq_length=512,
-                                    model=model,
-                                    prompt_encoder='lstm',
-                                    encoder_hidden_size=200)
+                                    max_length=512,
+                                    model=model)
 ```
 
 其中初始化参数定义如下
 
+- ``prompt`` ：定义离散型/连续型提示、初始化以及和输入文本的组合方式。
 - ``tokenizer`` ：预训练模型的 tokenizer，用于文本编码。
-- ``max_seq_length`` ：定义输入模型文本的最大长度，包括提示部分。当输入长度超过最大长度时，只会截断`text`关键字对应的输入文本，提示部分不做处理。
+- ``max_length`` ：定义输入模型文本的最大长度，包括提示部分。
 - ``model`` ：预训练语言模型，为了取预训练词向量用于连续型提示向量初始化。
-- ``template`` ：定义离散型/连续型提示、初始化以及和输入文本的组合方式。
-- ``prompt_encoder`` ：连续型提示向量的编码器，可选 ``mlp`` 和 ``lstm`` 。默认为 ``None`` ，即无编码器，直接使用向量。
-- ``encoder_hidden_size`` ：连续型提示向量的维度。默认为 ``None`` ，即与预训练词向量维度相同。
-
 
 ## 如何定义标签词映射
 
-**标签词映射**（Verbalizer）也是提示学习中可选的重要模块，用于建立预测词和标签之间的映射，将“预训练-微调”模式中预测标签的任务转换为预测模板中掩码位置的词语，从而将下游任务统一为预训练任务的形式。目前框架支持了离散型标签词映射和 [Word-level Adversarial ReProgramming (WARP)](https://aclanthology.org/2021.acl-long.381/) 方法。
+**标签词映射**（Verbalizer）也是提示学习中可选的重要模块，用于建立预测词和标签之间的映射，将“预训练-微调”模式中预测标签的任务转换为预测模板中掩码位置的词语，从而将下游任务统一为预训练任务的形式。目前框架支持了离散型标签词映射和连续型标签词映射 [Word-level Adversarial ReProgramming (WARP)](https://aclanthology.org/2021.acl-long.381/) 方法。
 
 
 例如，在情感二分类任务中，微调方法和提示学习的标签体系如下
@@ -259,9 +353,9 @@ template = AutoTemplate.create_from(template="这篇文章表达了怎样的情
 具体来说，对于模板 ``{'text':'text_a'}这句话表示我{'mask'}满意。`` ，我们使用映射 ``{'负向': '不', '正向': '很'}`` 将标签 ``负向`` 映射为 ``不`` ，将标签 ``正向`` 映射为 ``很`` 。也就是说，我们期望对于正向情感的文本，预测结果为 ``...这句话表示我很满意。`` ，对于负向情感的文本，预测结果为 ``...这句话表示我不满意。``
 
 
-### 单掩码映射
+### 离散型标签词映射
 
-``ManualVerbalizer`` 支持构造简单的单 ``{'mask'}`` 标签词映射，直接作用于 ``AutoMaskedLM`` 模型结构。当标签对应的预测词长度大于 ``1`` 时取均值。
+``ManualVerbalizer`` 支持构造 ``{'mask'}`` 对应的标签词映射，同一标签可对应多个不同长度的词，直接作用于 ``AutoMaskedLM`` 模型结构。当标签对应的预测词长度大于 ``1`` 时，默认取均值；当标签对应多个 `{'mask'}` 时，默认与单个 `{mask}` 效果等价。
 
 **调用 API**
 
@@ -271,41 +365,31 @@ from paddlenlp.transformers import AutoTokenizer
 
 tokenizer = AutoTokenizer.from_pretrained("ernie-3.0-base-zh")
 verbalizer = ManualVerbalizer(tokenizer=tokenizer,
-                              labels=['负向', '正向'],
-                              label_words={'负向': '不', '正向': '很'},
-                              prefix=None)
+                              label_words={'负向': '不', '正向': '很'})
 ```
 
 其中初始化参数定义如下
 
+- ``label_words`` : 原标签到预测词之间的映射字典。
 - ``tokenizer`` : 预训练模型的 tokenizer，用于预测词的编码。
-- ``labels`` : 数据集的原标签列表（可选）。
-- ``label_words`` : 原标签到预测词之间的映射字典。如果同时定义了 ``labels`` ，二者的标签集合需要相同。
-- ``prefix`` : 预测词解码前增加的前缀，用于 ``RoBERTa`` 等对前缀敏感的模型，例如 `roberta-large`， `good` 和 ` good` 经过 tokenize 会得到不同的 id。默认为 ``None`` ，无前缀。
 
-
-### 多掩码映射
-
-``MultiMaskVerbalizer`` 继承自 ``ManualVerbalizer`` ，支持多 ``{'mask'}`` 标签词映射。预测词长度需与 ``{'mask'}`` 长度一致。
+``MaskedLMVerbalizer`` 同样支持构造 ``{'mask'}`` 对应的标签词映射，映射词与模板中的 `{'mask'}` 逐字对应，因此，映射词长度应与 `{'mask'}` 数量保持一致。当定义的标签词映射中同一标签对应多个词时，仅有第一个映射词生效。在自定义的 `compute_metric` 函数中需先调用 `verbalizer.aggregate_multiple_mask` 将多 `{'mask'}` 合并后再计算评估函数，默认使用乘积的方式。
 
 **调用 API**
-
-```python
-from paddlenlp.prompt import MultiMaskVerbalizer
+from paddlenlp.prompt import MaskedLMVerbalizer
 from paddlenlp.transformers import AutoTokenizer
 
 tokenizer = AutoTokenizer.from_pretrained("ernie-3.0-base-zh")
-verbalizer = MultiMaskVerbalizer(tokenizer=tokenizer,
-                                 labels=['负向', '正向'],
-                                 label_words={'负向': '生气', '正向': '高兴'},
-                                 prefix=None)
+verbalizer = MaskedLMVerbalizer(tokenizer=tokenizer,
+                                label_words={'负向': '不', '正向': '很'})
 ```
 
+其中初始化参数定义如下
 
-其中初始化参数定义同[单掩码映射](#单掩码映射) 。
+- ``label_words`` : 原标签到预测词之间的映射字典。
+- ``tokenizer`` : 预训练模型的 tokenizer，用于预测词的编码。
 
-
-### 标签词映射分类
+### 连续型标签词映射
 
 标签词映射分类器 ``SoftVerbalizer`` 修改了原 ``AutoMaskedLM`` 的模型结构，将预训练模型最后一层“隐藏层-词表”替换为“隐藏层-标签”的映射。该层网络的初始化参数由标签词映射中的预测词词向量来决定，如果预测词长度大于 ``1`` ，则使用词向量均值进行初始化。当前支持的预训练模型包括 ``ErnieForMaskedLM`` 、 ``BertForMaskedLM`` 、 ``AlbertForMaskedLM`` 和 ``RobertaForMaskedLM`` 。可用于实现 WARP 算法。
 
@@ -318,15 +402,13 @@ from paddlenlp.transformers import AutoTokenizer, AutoModelForMaskedLM
 
 model = AutoModelForMaskedLM.from_pretrained("ernie-3.0-base-zh")
 tokenizer = AutoTokenizer.from_pretrained("ernie-3.0-base-zh")
-verbalizer = SoftVerbalizer(tokenizer=tokenizer,
-                            model=model,
-                            labels=['负向', '正向'],
-                            label_words={'负向': '生气', '正向': '高兴'},
-                            prefix=None)
+verbalizer = SoftVerbalizer(label_words={'负向': '生气', '正向': '高兴'},
+                            tokenizer=tokenizer,
+                            model=model)
 ```
 
-其中初始化参数定义同[单掩码映射](#单掩码映射) ，此外
-
+- ``label_words`` : 原标签到预测词之间的映射字典。
+- ``tokenizer`` : 预训练模型的 tokenizer，用于预测词的编码。
 - ``model`` ：预训练语言模型，用于取预训练词向量进行“隐藏层-标签”网络的修改和初始化。
 
 ## 快速开始训练
@@ -335,29 +417,24 @@ verbalizer = SoftVerbalizer(tokenizer=tokenizer,
 
 ### 数据准备
 
-Prompt 框架定义了统一的样本结构 ``InputExample`` 以便进行数据处理，数据集样本需要封装在 ``MapDataset`` 中。
+数据集封装为 ``MapDataset`` 类型。每条数据格式为字典结构，字典中关键字与模板中 `text` 定义的值相对应，统一使用 `labels` 关键字表示样本标签。
 
-例如，对于文本语义相似度 BUSTM 数据集中的原始样本
-
-```python
-data = [
-    {'id': 3, 'sentence1': '你晚上吃了什么', 'sentence2': '你晚上吃啥了', 'label': 1},
-    {'id': 4, 'sentence1': '我想打开滴滴叫的士', 'sentence2': '你叫小欧吗', 'label': 0},
-    {'id': 5, 'sentence1': '女孩子到底是不是你', 'sentence2': '你不是女孩子吗', 'label': 1}
-]
-```
-
-
-需要转换为统一格式
+例如，文本语义相似度 BUSTM 数据集中的数据样本
 
 ```python
 from paddlenlp.datasets import MapDataset
-from paddlenlp.prompt import InputExample
 
-data_ds = MapDataset([InputExample(uid=example["id"],
-                                   text_a=example["sentence1"],
-                                   text_b=example["sentence2"],
-                                   labels=example["label"]) for example in data])
+data_ds = MapDataset([
+    {'id': 3, 'sentence1': '你晚上吃了什么', 'sentence2': '你晚上吃啥了', 'label': 1},
+    {'id': 4, 'sentence1': '我想打开滴滴叫的士', 'sentence2': '你叫小欧吗', 'label': 0},
+    {'id': 5, 'sentence1': '女孩子到底是不是你', 'sentence2': '你不是女孩子吗', 'label': 1}
+])
+
+def convert_label_keyword(input_dict):
+    input_dict["labels"] = input_dict.pop("label")
+    return input_dict
+
+data_ds = data_ds.map(convert_label_keyword)
 ```
 
 ### 预训练参数准备
@@ -383,13 +460,13 @@ from paddlenlp.prompt import ManualVerbalizer
 from paddlenlp.prompt import PromptModelForSequenceClassification
 
 # 定义模板
-template = AutoTemplate.create_from(template="{'text': 'text_a'}和{'text': 'text_b'}说的是{'mask'}同的事情。",
+template = AutoTemplate.create_from(prompt="{'text': 'text_a'}和{'text': 'text_b'}说的是{'mask'}同的事情。",
                                     tokenizer=tokenizer,
-                                    max_seq_length=512)
+                                    max_length=512)
 
 # 定义标签词映射
-verbalizer = ManualVerbalizer(tokenizer=tokenizer,
-                              label_words={0: '不', 1: '相'})
+verbalizer = ManualVerbalizer(label_words={0: '不', 1: '相'},
+                              tokenizer=tokenizer)
 
 # 定义文本分类提示模型
 prompt_model = PromptModelForSequenceClassification(model,
@@ -404,8 +481,8 @@ prompt_model = PromptModelForSequenceClassification(model,
 - ``model`` : 预训练模型实例，支持 ``AutoModelForMaskedLM`` 和 ``AutoModelForSequenceClassification`` 。
 - ``template`` : 模板实例。
 - ``verbalizer`` : 标签词映射实例。当设为 ``None`` 时，不使用标签词映射，模型输出及损失值计算由 ``model`` 类型定义。
-- ``freeze_plm`` : 在训练时是否固定预训练模型参数。对于规模较小的预训练模型，推荐更新预训练模型参数。
-- ``freeze_dropout`` : 在训练时是否固定预训练模型参数并关闭 ``dropout`` 。 当 ``freeze_dropout=True`` ，``freeze_plm`` 也为 ``True`` 。
+- ``freeze_plm`` : 在训练时固定预训练模型参数，默认为 `False`。对于轻量级预训练模型，推荐使用默认值。
+- ``freeze_dropout`` : 在训练时固定预训练模型参数并关闭 ``dropout`` 。 当 ``freeze_dropout=True`` ，``freeze_plm`` 也为 ``True`` 。
 
 
 ### 使用PromptTrainer训练
@@ -497,6 +574,8 @@ if training_args.do_train:
 - WARP: Word-level Adversarial ReProgramming. [[PDF]](https://aclanthology.org/2021.acl-long.381/)
 - RGL: A Simple yet Effective Relation Graph Augmented Prompt-based Tuning Approach for Few-Shot Learning. [[PDF]](https://aclanthology.org/2022.findings-naacl.81/)
 - R-Drop: Regularized Dropout for Neural Networks. [[PDF]](https://arxiv.org/abs/2106.14448)
+- Openprompt: An open-source framework for prompt-learning. [[PDF]](https://arxiv.org/abs/2111.01998)
+
 
 ### 附录
 
@@ -506,9 +585,9 @@ if training_args.do_train:
 
 | 参数              |  类型  | 默认值   |   含义                                                   |
 | ---------------- | ------ | ------- | ------------------------------------------------------- |
-| max_seq_length   |  int   |  512    |    模型输入的最大长度，包括模板部分                          |
-| freeze_plm       |  bool  |  False  |    是否在训练时固定预训练模型的参数                          |
-| freeze_dropout   |  bool  |  False  |    是否在训练时固定预训练模型的参数，同时关闭 dropout         |
+| max_seq_length   |  int   |  512    |   模型输入的最大长度，包括模板部分                          |
+| freeze_plm       |  bool  |  False  |   是否在训练时固定预训练模型的参数                          |
+| freeze_dropout   |  bool  |  False  |   是否在训练时固定预训练模型的参数，同时关闭 dropout         |
 | use_rdrop        |  bool  |  False  |   是否使用 RDrop 策略，详见 [RDrop 论文](https://arxiv.org/abs/2106.14448) |
 | alpha_rdrop      |  float |  5.0    |   RDrop Loss 的权重                                      |
 | use_rgl          |  bool  |  False  |   是否使用 RGL 策略，详见 [RGL 论文](https://aclanthology.org/2022.findings-naacl.81/) |

docs/locale/en/LC_MESSAGES/source/paddlenlp.transformers.ernie.faster_tokenizer.po

@@ -17,7 +17,7 @@ msgstr ""
 "Content-Transfer-Encoding: 8bit\n"
 "Generated-By: Babel 2.10.1\n"
 
-#: ../source/paddlenlp.transformers.ernie.faster_tokenizer.rst:2
-msgid "faster\\_tokenizer"
+#: ../source/paddlenlp.transformers.ernie.fast_tokenizer.rst:2
+msgid "fast\\_tokenizer"
 msgstr ""
 

docs/requirements.txt

@@ -9,4 +9,5 @@ Markdown < 3.4
 sphinx-copybutton==0.3.1
 sphinx-markdown-tables==0.0.15
 
-paddlepaddle>=2.2.2
+# use paddlepaddle == 2.3.* according to: https://github.com/PaddlePaddle/Paddle/issues/48243
+paddlepaddle>=2.2.2,<2.4.0

docs/server.md

@@ -0,0 +1,183 @@
+# PaddleNLP SimpleSevring
+
+PaddleNLP SimpleServing 是基于 unicorn 封装的模型部署服务化工具，该服务化工具具备灵活、易用的特性，可以简易部署预训练模型和预训练模型工具Taskflow，PaddleNLP SimpleServing 具备以下两个特性：
+  - 易用：一行代码即可部署预训练模型和预训练工具Taskflow
+  - 灵活：Handler机制可以快速定制化服务化部署方式
+
+
+## Tasflow部署
+
+Taskflow 是 PaddleNLP 预训练模型工具，具备开箱即用的特性，同时 Taskflow 可以支持加载微调后的模型，基于 Taskflow 的服务化方式可以进一步降低使用者的部署难度。PaddleNLP SimpleServing 基于这样的设计需求，设计了一套基于Taskflow的快速部署方式。下面从 server 搭建，client 发送请求来详细介绍使用方式。
+
+### server 搭建
+
+下面是 Taskflow 搭建服务的简易代码
+
+```python
+schema = ['出发地', '目的地', '费用', '时间']
+uie = Taskflow("information_extraction", schema=schema)
+app = SimpleServer()
+app.register_taskflow('uie', uie)
+```
+这里主要是使用 `SimpleServer` 服务类来注册 Taskflow Server，下面我们具体介绍一下 `register_taskflow` 相关参数
+
+```text
+def register_taskflow(
+        task_name,
+    task,
+    taskflow_handler=None)
+
+task_name(str)：
+      服务化的名称，最终的服务化的URL: https://host:port/taskflow/{task_name}
+task(paddlenlp.Taskflow or list(paddlenlp.Taskflow)):
+      Taskflow的实例对象，将想要注册的Taskflow任务注册进去，可以是多个Taskflow实例来支持多卡服务化
+taskflow_handler(paddlenlp.server.BaseTaskflowHandler, 可选):
+      Taskflow句柄处理类，可以自定义处理类来定制化Taskflow服务，默认为None，是默认的TaskflowHandler
+```
+### 多卡服务化(可选)
+在机器环境里面如果有多卡，那就可以register taskflow服务化时，可以注册多个Taskflow实例，在服务化处理请求的过程中做了负载均衡，保证机器设备利用率充分利用，下面是具体的使用例子
+```python
+schema = ['出发地', '目的地', '费用', '时间']
+uie1 = Taskflow("information_extraction", schema=schema, device_id=0)
+uie2 = Taskflow("information_extraction", schema=schema, device_id=1)
+app = SimpleServer()
+app.register_taskflow('uie', [uie1, uie2])
+```
+### 启动服务化
+执行代码的即可启动服务
+```
+paddlenlp server server:app --host 0.0.0.0 --port 8189 --workers 1
+```
+服务化整体参数配置如下：
+```text
+--host: 启动服务化的IP地址，通常可以设置成 0.0.0.0
+--port：启动服务化的网络端口
+--workers: 接收服务化的进程数，默认为1
+--log_level：服务化输出日志的级别，默认为 info 级别
+--limit_concurrency：服务化能接受的并发数目，默认为None, 没有限制
+--timeout_keep_alive：保持服务化连接的时间，默认为15s
+--app_dir：服务化本地的路径，默认为服务化启动的位置
+--reload: 当 app_dir的服务化相关配置和代码发生变化时，是否重启server，默认为False
+```
+
+### client 发送
+```python
+import requests
+import json
+
+url = "http://0.0.0.0:8189/taskflow/uie"
+headers = {"Content-Type": "application/json"}
+texts = ['城市内交通费7月5日金额114广州至佛山', '5月9日交通费29元从北苑到望京搜后']
+data = {
+    'data':  texts
+}
+r = requests.post(url=url, headers=headers, data=json.dumps(data))
+datas = json.loads(r.text)
+print(datas)
+```
+通过上述代码配置即可发送POST请求，同时注意在`data`这个key填入相关请求
+
+## 预训练模型部署
+PaddleNLP SimpleServing 除了能支持Taskflow的服务化部署，也能支持预训练模型的部署，通过简单的配置即可加载预训练模型来进行服务化，同时在接口层面也能支持服务化的扩展，支持模型前后处理的定制化需求。
+
+## server 搭建
+
+下面是预训练模型的搭建的简易代码
+```python
+from paddlenlp import SimpleServer
+from paddlenlp.server import CustomModelHandler, MultiClassificationPostHandler
+
+app = SimpleServer()
+app.register('cls_multi_class',
+             model_path="./export",
+             tokenizer_name='ernie-3.0-medium-zh',
+             model_handler=CustomModelHandler,
+             post_handler=MultiClassificationPostHandler)
+```
+
+这里主要是使用 `SimpleServer` 服务类来注册 Transformers Server，下面我们具体介绍一下 `register` 相关参数
+
+```text
+def register(task_name,
+             model_path,
+             tokenizer_name,
+             model_handler,
+             post_handler,
+             precision='fp32',
+             device_id=0)
+task_name(str)：
+      服务化的名称，最终的服务化的URL: https://host:port/models/{task_name}
+model_path(str):
+      需要部署的模型路径，这里的路径必须是动转静后的模型路径
+model_handler(paddlenlp.server.BaseModelHandler):
+      模型前置处理以及模型预测的Handler类别名字，这里可以继承 BaseModelHandler 自定义处理逻辑
+ post_handler(paddlenlp.server.BasePostHandler):
+      模型后置处理的Handler类别名字，这里可以继承 BasePostHandler 自定义处理逻辑
+precision(str):
+      模型的预测精度，默认为fp32；可选fp16，fp16的支持需要以下条件 1) **硬件**： V100、T4、A10、A100/GA100、Jetson AGX Xavier 、3080、3080、2080、2090 等显卡 2）**CUDA环境**：确保 CUDA >= 11.2，cuDNN >= 8.1.1 3) **安装依赖**：安装 onnx、 onnxruntime-gpu
+device_id(int, list(int)):
+       GPU设备，device_id默认为0，同时如果有多张显卡，可以设置成list,例如[0, 1]就可以支持多卡服务化；CPU设备，不用设置。
+```
+- BaseModelHandler继承类：主要是 `CustomModelHandler`，该类的实现可以参考[链接](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/paddlenlp/server/handlers/custom_model_handler.py), 绝大多数语义理解模型均可使用该继承类
+- BasePostHandler继承类：主要是文本分类 `MultiClassificationPostHandler`、`MultiLabelClassificationPostHandler` 来支持多分类、多标签分类，实现代码部分可以参考[链接](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/paddlenlp/server/handlers/cls_post_handler.py)；`TokenClsModelHandler` 支持 序列标注任务，实现代码部分可以参考[链接](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/paddlenlp/server/handlers/token_model_handler.py)
+
+### 启动服务化
+执行代码的即可启动服务
+```
+paddlenlp server server:app --host 0.0.0.0 --port 8189 --workers 1
+```
+服务化整体参数配置如下：
+```text
+--host: 启动服务化的IP地址，通常可以设置成 0.0.0.0
+--port：启动服务化的网络端口，注意和已有网络端口冲突
+--workers: 接收服务化的进程数，默认为1
+--log_level：服务化输出日志的级别，默认为 info 级别
+--limit_concurrency：服务化能接受的并发数目，默认为None, 没有限制
+--timeout_keep_alive：保持服务化连接的时间，默认为15s
+--app_dir：服务化本地的路径，默认为服务化启动的位置
+--reload: 当 app_dir的服务化相关配置和代码发生变化时，是否重启server，默认为False
+```
+
+### 多卡服务化(可选)
+在机器环境里面如果有多卡，通过简单设置 device_id 即可实现多卡服务化，保证机器设备利用率充分利用，下面是具体的使用例子
+```python
+from paddlenlp import SimpleServer
+from paddlenlp.server import CustomModelHandler, MultiClassificationPostHandler
+
+app = SimpleServer()
+app.register('cls_multi_class',
+             model_path="../../export",
+             tokenizer_name='ernie-3.0-medium-zh',
+             model_handler=CustomModelHandler,
+             post_handler=MultiClassificationPostHandler，
+             device_id=[0,1]) # device_id是0,1 两张卡
+```
+### client 发送
+```python
+
+import requests
+import json
+
+texts = [
+        '黑苦荞茶的功效与作用及食用方法', '交界痣会凸起吗', '检查是否能怀孕挂什么科', '鱼油怎么吃咬破吃还是直接咽下去',
+        '幼儿挑食的生理原因是'
+    ]
+    data = {
+        'data': {
+            'text': texts,
+        },
+        'parameters': {
+            'max_seq_len': 128,
+            'batch_size': 2
+        }
+    }
+    r = requests.post(url=url, headers=headers, data=json.dumps(data))
+    result_json = json.loads(r.text)
+    print(result_json)
+```
+在Client发送请求的过程中可以一些参数来控制服务化处理逻辑，例如上面的 `max_seq_len`和 `batch_size` 均可以控制服务化处理时的序列长度和处理batch_size 。
+
+## 参考示例
+- [UIE 服务化部署](https://github.com/PaddlePaddle/PaddleNLP/tree/develop/model_zoo/uie/deploy/serving/simple_serving)
+- [文本分类服务化部署](https://github.com/PaddlePaddle/PaddleNLP/tree/develop/applications/text_classification/multi_class/deploy/simple_serving)
+- [预训练模型定制化post_handler](https://github.com/PaddlePaddle/PaddleNLP/tree/develop/model_zoo/ernie-health/cblue/deploy/serving/simple_serving)

docs/source/paddlenlp.transformers.bert.fast_tokenizer.rst

@@ -1,7 +1,7 @@
-faster\_tokenizer
+fast\_tokenizer
 ====================================================
 
-.. automodule:: paddlenlp.transformers.bert.faster_tokenizer
+.. automodule:: paddlenlp.transformers.bert.fast_tokenizer
    :members:
    :no-undoc-members:
    :show-inheritance: