Kimirock/PaddleNLP
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

[Deprecated] Delete few used model and tools. (#10279)

Browse Source 
* delete few used model and tools.

* fix
This commit is contained in:
Zhong Hui
2025-03-31 12:06:23 +08:00
committed by GitHub
parent c050c6a392
commit 88ca820e2b
141 changed files with 13 additions and 21504 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
docs/zh/index.rst
View File

@@ -89,7 +89,6 @@
:caption: PaddleNLP工具库
一键预测功能 <model_zoo/taskflow>
预训练词向量 <model_zoo/embeddings>
.. toctree::
:maxdepth: 1
@@ -159,7 +158,6 @@
paddlenlp.data <source/paddlenlp.data>
paddlenlp.datasets <source/paddlenlp.datasets>
paddlenlp.embeddings <source/paddlenlp.embeddings>
paddlenlp.layers <source/paddlenlp.layers>
paddlenlp.losses <source/paddlenlp.losses>
paddlenlp.metrics <source/paddlenlp.metrics>

4
docs/zh/locale/en/LC_MESSAGES/index.po
View File

@@ -133,10 +133,6 @@ msgstr ""
msgid "paddlenlp.datasets"
msgstr ""
#: ../index.rst:88
msgid "paddlenlp.embeddings"
msgstr ""
#: ../index.rst:88
msgid "paddlenlp.layers"
msgstr ""

219
docs/zh/locale/en/LC_MESSAGES/model_zoo/embeddings.po
View File

@@ -1,219 +0,0 @@
# SOME DESCRIPTIVE TITLE.
# Copyright (C) 2021, PaddleNLP
# This file is distributed under the same license as the PaddleNLP package.
# FIRST AUTHOR <EMAIL@ADDRESS>, 2022.
#
#, fuzzy
msgid ""
msgstr ""
"Project-Id-Version: PaddleNLP \n"
"Report-Msgid-Bugs-To: \n"
"POT-Creation-Date: 2022-03-18 21:31+0800\n"
"PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
"Last-Translator: FULL NAME <EMAIL@ADDRESS>\n"
"Language-Team: LANGUAGE <LL@li.org>\n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=utf-8\n"
"Content-Transfer-Encoding: 8bit\n"
"Generated-By: Babel 2.9.0\n"
#: ../model_zoo/embeddings.md:1
msgid "PaddleNLP Embedding API"
msgstr ""
#: ../model_zoo/embeddings.md:3 ../model_zoo/embeddings.md:24
msgid "介绍"
msgstr ""
#: ../model_zoo/embeddings.md:4 ../model_zoo/embeddings.md:28
msgid "用法"
msgstr ""
#: ../model_zoo/embeddings.md:5 ../model_zoo/embeddings.md:30
msgid "TokenEmbedding参数"
msgstr ""
#: ../model_zoo/embeddings.md:6 ../model_zoo/embeddings.md:69
msgid "初始化"
msgstr ""
#: ../model_zoo/embeddings.md:7 ../model_zoo/embeddings.md:101
msgid "查询embedding结果"
msgstr ""
#: ../model_zoo/embeddings.md:8 ../model_zoo/embeddings.md:115
msgid "可视化embedding结果"
msgstr ""
#: ../model_zoo/embeddings.md:9 ../model_zoo/embeddings.md:140
msgid "计算词向量cosine相似度"
msgstr ""
#: ../model_zoo/embeddings.md:10 ../model_zoo/embeddings.md:147
msgid "计算词向量内积"
msgstr ""
#: ../model_zoo/embeddings.md:11 ../model_zoo/embeddings.md:155
msgid "训练"
msgstr ""
#: ../model_zoo/embeddings.md:12 ../model_zoo/embeddings.md:171
msgid "切词"
msgstr ""
#: ../model_zoo/embeddings.md:13 ../model_zoo/embeddings.md:183
msgid "预训练模型"
msgstr ""
#: ../model_zoo/embeddings.md:14 ../model_zoo/embeddings.md:189
msgid "中文词向量"
msgstr ""
#: ../model_zoo/embeddings.md:15 ../model_zoo/embeddings.md:343
msgid "英文词向量"
msgstr ""
#: ../model_zoo/embeddings.md:16 ../model_zoo/embeddings.md:345
msgid "Word2Vec"
msgstr ""
#: ../model_zoo/embeddings.md:17 ../model_zoo/embeddings.md:362
msgid "GloVe"
msgstr ""
#: ../model_zoo/embeddings.md:18 ../model_zoo/embeddings.md:395
msgid "FastText"
msgstr ""
#: ../model_zoo/embeddings.md:19 ../model_zoo/embeddings.md:416
msgid "使用方式"
msgstr ""
#: ../model_zoo/embeddings.md:20 ../model_zoo/embeddings.md:427
msgid "模型信息"
msgstr ""
#: ../model_zoo/embeddings.md:21 ../model_zoo/embeddings.md:751
msgid "致谢"
msgstr ""
#: ../model_zoo/embeddings.md:22 ../model_zoo/embeddings.md:756
msgid "参考论文"
msgstr ""
#: ../model_zoo/embeddings.md:26
msgid "PaddleNLP提供多个开源的预训练词向量模型，用户仅需在使用paddlenlp.embeddings.TokenEmbedding时，指定预训练模型的名称，即可加载相对应的预训练模型。以下将介绍TokenEmbeddign详细用法，并列出PaddleNLP所支持的预训练Embedding模型。"
msgstr ""
#: ../model_zoo/embeddings.md:116
msgid "使用深度学习可视化工具VisualDL的High Dimensional组件可以对embedding结果进行可视化展示，便于对其直观分析，步骤如下："
msgstr ""
#: ../model_zoo/embeddings.md:128
msgid "执行完毕后会在当前路径下生成一个visualize目录，并将日志存放在其中，我们在命令行启动VisualDL即可进行查看，启动命令为："
msgstr ""
#: ../model_zoo/embeddings.md:132
msgid "启动后打开浏览器即可看到可视化结果"
msgstr ""
#: ../model_zoo/embeddings.md:138
msgid "使用VisualDL除可视化embedding结果外，还可以对标量、图片、音频等进行可视化，有效提升训练调参效率。关于VisualDL更多功能和详细介绍，可参考VisualDL使用文档。"
msgstr ""
#: ../model_zoo/embeddings.md:157
msgid ""
"以下为TokenEmbedding简单的组网使用方法。有关更多TokenEmbedding训练流程相关的使用方法，请参考Word "
"Embedding with PaddleNLP。"
msgstr ""
#: ../model_zoo/embeddings.md:185
msgid "以下将列举PaddleNLP支持的Embedding预训练模型。"
msgstr ""
#: ../model_zoo/embeddings.md:186
msgid "模型命名方式为：${训练模型}.${语料}.${词向量类型}.${co-occurrence type}.dim${维度}。"
msgstr ""
#: ../model_zoo/embeddings.md:187
msgid "模型有三种，分别是Word2Vec(w2v, skip-gram), GloVe(glove)和FastText(fasttext)。"
msgstr ""
#: ../model_zoo/embeddings.md:191
msgid "以下预训练词向量由Chinese-Word-Vectors提供。"
msgstr ""
#: ../model_zoo/embeddings.md:193
msgid "根据不同类型的上下文为每个语料训练多个目标词向量，第二列开始表示不同类型的上下文。以下为上下文类别："
msgstr ""
#: ../model_zoo/embeddings.md:195
msgid "Word表示训练时目标词预测的上下文是一个Word。"
msgstr ""
#: ../model_zoo/embeddings.md:196
msgid ""
"Word + "
"N-gram表示训练时目标词预测的上下文是一个Word或者Ngram，其中bigram表示2-grams，ngram.1-2表示1-gram或者2-grams。"
msgstr ""
#: ../model_zoo/embeddings.md:197
msgid ""
"Word + Character表示训练时目标词预测的上下文是一个Word或者Character，其中word-"
"character.char1-2表示上下文是1个或2个Character。"
msgstr ""
#: ../model_zoo/embeddings.md:198
msgid ""
"Word + Character + Ngram表示训练时目标词预测的上下文是一个Word、Character或者Ngram。bigram-"
"char表示上下文是2-grams或者1个Character。"
msgstr ""
#: ../model_zoo/embeddings.md:284
msgid "特别地，对于百度百科语料，在不同的 Co-occurrence类型下分别提供了目标词与上下文向量："
msgstr ""
#: ../model_zoo/embeddings.md:418
msgid ""
"以上所述的模型名称可直接以参数形式传入padddlenlp.embeddings.TokenEmbedding，加载相对应的模型。比如要加载语料为Wiki2017，通过FastText训练的预训练模型（fasttext"
".wiki-news.target.word-word.dim300.en），只需执行以下代码："
msgstr ""
#: ../model_zoo/embeddings.md:752
msgid "感谢 Chinese-Word-Vectors提供Word2Vec中文预训练词向量。"
msgstr ""
#: ../model_zoo/embeddings.md:753
msgid "感谢 GloVe Project提供的GloVe英文预训练词向量。"
msgstr ""
#: ../model_zoo/embeddings.md:754
msgid "感谢 FastText Project提供的英文预训练词向量。"
msgstr ""
#: ../model_zoo/embeddings.md:757
msgid ""
"Li, Shen, et al. \"Analogical reasoning on chinese morphological and "
"semantic relations.\" arXiv preprint arXiv:1805.06504 (2018)."
msgstr ""
#: ../model_zoo/embeddings.md:758
msgid ""
"Qiu, Yuanyuan, et al. \"Revisiting correlations between intrinsic and "
"extrinsic evaluations of word embeddings.\" Chinese Computational "
"Linguistics and Natural Language Processing Based on Naturally Annotated "
"Big Data. Springer, Cham, 2018. 209-221."
msgstr ""
#: ../model_zoo/embeddings.md:759
msgid ""
"Jeffrey Pennington, Richard Socher, and Christopher D. Manning. 2014. "
"GloVe: Global Vectors for Word Representation."
msgstr ""
#: ../model_zoo/embeddings.md:760
msgid ""
"T. Mikolov, E. Grave, P. Bojanowski, C. Puhrsch, A. Joulin. Advances in "
"Pre-Training Distributed Word Representations."
msgstr ""

23
docs/zh/locale/en/LC_MESSAGES/source/paddlenlp.embeddings.po
View File

@@ -1,23 +0,0 @@
# SOME DESCRIPTIVE TITLE.
# Copyright (C) 2021, PaddleNLP
# This file is distributed under the same license as the PaddleNLP package.
# FIRST AUTHOR <EMAIL@ADDRESS>, 2022.
#
#, fuzzy
msgid ""
msgstr ""
"Project-Id-Version: PaddleNLP \n"
"Report-Msgid-Bugs-To: \n"
"POT-Creation-Date: 2022-03-18 21:31+0800\n"
"PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
"Last-Translator: FULL NAME <EMAIL@ADDRESS>\n"
"Language-Team: LANGUAGE <LL@li.org>\n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=utf-8\n"
"Content-Transfer-Encoding: 8bit\n"
"Generated-By: Babel 2.9.0\n"
#: ../source/paddlenlp.embeddings.rst:2
msgid "paddlenlp.embeddings"
msgstr ""

200
docs/zh/locale/en/LC_MESSAGES/source/paddlenlp.embeddings.token_embedding.po
View File

@@ -1,200 +0,0 @@
# SOME DESCRIPTIVE TITLE.
# Copyright (C) 2021, PaddleNLP
# This file is distributed under the same license as the PaddleNLP package.
# FIRST AUTHOR <EMAIL@ADDRESS>, 2022.
#
#, fuzzy
msgid ""
msgstr ""
"Project-Id-Version: PaddleNLP \n"
"Report-Msgid-Bugs-To: \n"
"POT-Creation-Date: 2022-03-18 21:31+0800\n"
"PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
"Last-Translator: FULL NAME <EMAIL@ADDRESS>\n"
"Language-Team: LANGUAGE <LL@li.org>\n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=utf-8\n"
"Content-Transfer-Encoding: 8bit\n"
"Generated-By: Babel 2.9.0\n"
#: ../source/paddlenlp.embeddings.token_embedding.rst:2
msgid "token\\_embedding"
msgstr ""
#: of paddlenlp.embeddings.token_embedding.list_embedding_name:1
msgid "Lists all names of pretrained embedding models paddlenlp provides."
msgstr ""
#: of paddlenlp.embeddings.token_embedding.TokenEmbedding:1
msgid "基类：:class:`paddle.nn.layer.common.Embedding`"
msgstr ""
#: of paddlenlp.embeddings.token_embedding.TokenEmbedding:1
msgid ""
"A `TokenEmbedding` can load pre-trained embedding model which paddlenlp "
"provides by specifying embedding name. Furthermore, a `TokenEmbedding` "
"can load extended vocabulary by specifying extended_vocab_path."
msgstr ""
#: of paddlenlp.embeddings.token_embedding.TokenEmbedding
#: paddlenlp.embeddings.token_embedding.TokenEmbedding.cosine_sim
#: paddlenlp.embeddings.token_embedding.TokenEmbedding.dot
#: paddlenlp.embeddings.token_embedding.TokenEmbedding.get_idx_from_word
#: paddlenlp.embeddings.token_embedding.TokenEmbedding.get_idx_list_from_words
#: paddlenlp.embeddings.token_embedding.TokenEmbedding.search
#: paddlenlp.embeddings.token_embedding.TokenEmbedding.set_trainable
msgid "参数"
msgstr ""
#: of paddlenlp.embeddings.token_embedding.TokenEmbedding:5
msgid ""
"The pre-trained embedding model name. Use "
"`paddlenlp.embeddings.list_embedding_name()` to list the names of all "
"embedding models that we provide. Defaults to "
"`w2v.baidu_encyclopedia.target.word-word.dim300`."
msgstr ""
#: of paddlenlp.embeddings.token_embedding.TokenEmbedding:9
msgid "Specifies unknown token. Defaults to `[UNK]`."
msgstr ""
#: of paddlenlp.embeddings.token_embedding.TokenEmbedding:12
msgid ""
"To initialize the vector of unknown token. If it's none, use normal "
"distribution to initialize the vector of unknown token. Defaults to "
"`None`."
msgstr ""
#: of paddlenlp.embeddings.token_embedding.TokenEmbedding:16
msgid "The file path of extended vocabulary. Defaults to `None`."
msgstr ""
#: of paddlenlp.embeddings.token_embedding.TokenEmbedding:19
msgid "Whether the weight of embedding can be trained. Defaults to True."
msgstr ""
#: of paddlenlp.embeddings.token_embedding.TokenEmbedding:22
msgid ""
"Whether to keep the extended vocabulary only, will be effective only if "
"provides extended_vocab_path. Defaults to False."
msgstr ""
#: of paddlenlp.embeddings.token_embedding.TokenEmbedding.set_trainable:1
msgid "Whether or not to set the weights of token embedding to be trainable."
msgstr ""
#: of paddlenlp.embeddings.token_embedding.TokenEmbedding.set_trainable:3
msgid ""
"The weights can be trained if trainable is set to True, or the weights "
"are fixed if trainable is False."
msgstr ""
#: of paddlenlp.embeddings.token_embedding.TokenEmbedding.search:1
msgid "Gets the vectors of specifying words."
msgstr ""
#: of paddlenlp.embeddings.token_embedding.TokenEmbedding.search:3
msgid "The words which need to be searched."
msgstr ""
#: of paddlenlp.embeddings.token_embedding.TokenEmbedding.cosine_sim
#: paddlenlp.embeddings.token_embedding.TokenEmbedding.dot
#: paddlenlp.embeddings.token_embedding.TokenEmbedding.get_idx_from_word
#: paddlenlp.embeddings.token_embedding.TokenEmbedding.get_idx_list_from_words
#: paddlenlp.embeddings.token_embedding.TokenEmbedding.search
msgid "返回"
msgstr ""
#: of paddlenlp.embeddings.token_embedding.TokenEmbedding.search:6
msgid "The vectors of specifying words."
msgstr ""
#: of paddlenlp.embeddings.token_embedding.TokenEmbedding.cosine_sim
#: paddlenlp.embeddings.token_embedding.TokenEmbedding.dot
#: paddlenlp.embeddings.token_embedding.TokenEmbedding.get_idx_from_word
#: paddlenlp.embeddings.token_embedding.TokenEmbedding.get_idx_list_from_words
#: paddlenlp.embeddings.token_embedding.TokenEmbedding.search
msgid "返回类型"
msgstr ""
#: of paddlenlp.embeddings.token_embedding.TokenEmbedding.search:7
msgid "`numpy.array`"
msgstr ""
#: of paddlenlp.embeddings.token_embedding.TokenEmbedding.cosine_sim:13
#: paddlenlp.embeddings.token_embedding.TokenEmbedding.dot:14
#: paddlenlp.embeddings.token_embedding.TokenEmbedding.get_idx_list_from_words:10
#: paddlenlp.embeddings.token_embedding.TokenEmbedding.search:10
msgid "实际案例"
msgstr ""
#: of paddlenlp.embeddings.token_embedding.TokenEmbedding.get_idx_from_word:1
msgid "Gets the index of specifying word by searching word_to_idx dict."
msgstr ""
#: of paddlenlp.embeddings.token_embedding.TokenEmbedding.get_idx_from_word:3
msgid "The input token word which we want to get the token index converted from."
msgstr ""
#: of paddlenlp.embeddings.token_embedding.TokenEmbedding.get_idx_from_word:6
msgid "The index of specifying word."
msgstr ""
#: of paddlenlp.embeddings.token_embedding.TokenEmbedding.get_idx_from_word:7
msgid "`int`"
msgstr ""
#: of
#: paddlenlp.embeddings.token_embedding.TokenEmbedding.get_idx_list_from_words:1
msgid "Gets the index list of specifying words by searching word_to_idx dict."
msgstr ""
#: of
#: paddlenlp.embeddings.token_embedding.TokenEmbedding.get_idx_list_from_words:3
msgid ""
"The input token words which we want to get the token indices converted "
"from."
msgstr ""
#: of
#: paddlenlp.embeddings.token_embedding.TokenEmbedding.get_idx_list_from_words:6
msgid "The indexes list of specifying words."
msgstr ""
#: of
#: paddlenlp.embeddings.token_embedding.TokenEmbedding.get_idx_list_from_words:7
msgid "`list`"
msgstr ""
#: of paddlenlp.embeddings.token_embedding.TokenEmbedding.dot:1
msgid ""
"Calculates the dot product of 2 words. Dot product or scalar product is "
"an algebraic operation that takes two equal-length sequences of numbers "
"(usually coordinate vectors), and returns a single number."
msgstr ""
#: of paddlenlp.embeddings.token_embedding.TokenEmbedding.cosine_sim:4
#: paddlenlp.embeddings.token_embedding.TokenEmbedding.dot:5
msgid "The first word string."
msgstr ""
#: of paddlenlp.embeddings.token_embedding.TokenEmbedding.cosine_sim:6
#: paddlenlp.embeddings.token_embedding.TokenEmbedding.dot:7
msgid "The second word string."
msgstr ""
#: of paddlenlp.embeddings.token_embedding.TokenEmbedding.dot:10
msgid "The dot product of 2 words."
msgstr ""
#: of paddlenlp.embeddings.token_embedding.TokenEmbedding.cosine_sim:1
msgid ""
"Calculates the cosine similarity of 2 word vectors. Cosine similarity is "
"the cosine of the angle between two n-dimensional vectors in an "
"n-dimensional space."
msgstr ""
#: of paddlenlp.embeddings.token_embedding.TokenEmbedding.cosine_sim:9
msgid "The cosine similarity of 2 words."
msgstr ""

307
docs/zh/model_zoo/embeddings.md
View File

@@ -1,307 +0,0 @@
# PaddleNLP Embedding API
- [介绍](#介绍)
- [用法](#用法)
* [TokenEmbedding 参数](#TokenEmbedding 参数)
* [初始化](#初始化)
* [查询 embedding 结果](#查询 embedding 结果)
* [可视化 embedding 结果](#可视化 embedding 结果)
* [计算词向量 cosine 相似度](#计算词向量 cosine 相似度)
* [计算词向量内积](#计算词向量内积)
* [训练](#训练)
* [切词](#切词)
- [预训练模型](#预训练模型)
* [中文词向量](#中文词向量)
* [英文词向量](#英文词向量)
* [Word2Vec](#word2vec)
* [GloVe](#glove)
* [FastText](#fasttext)
* [使用方式](#使用方式)
* [模型信息](#模型信息)
- [致谢](#致谢)
- [参考论文](#参考论文)
## 介绍
PaddleNLP 提供多个开源的预训练词向量模型，用户仅需在使用`paddlenlp.embeddings.TokenEmbedding`时，指定预训练模型的名称，即可加载相对应的预训练模型。以下将介绍`TokenEmbedding`详细用法，并列出 PaddleNLP 所支持的预训练 Embedding 模型。
## 用法
### TokenEmbedding 参数
| 参数 | 类型 | 属性 |
| ------------ | ------------ | ------------ |
| embedding_name | **string** | 预训练 embedding 名称，可通过 paddlenlp.embeddings.list_embedding_name()或[Embedding 模型汇总](#中文词向量)查询。 |
| unknown_token | **string** | unknown token。 |
| unknown_token_vector | **list** 或者 **np.array** | 用来初始化 unknown token 对应的 vector。默认为 None（以正态分布方式初始化 vector）|
| extended_vocab_path | **string** | 扩展词表的文件名路径。词表格式为一行一个词。 |
| trainable | **bool** | 是否可训练。True 表示 Embedding 可以更新参数，False 为不可更新。 |
### 初始化
```python
import paddle
from paddlenlp.embeddings import TokenEmbedding, list_embedding_name
paddle.set_device("cpu")
# 查看预训练embedding名称：
print(list_embedding_name()) # ['w2v.baidu_encyclopedia.target.word-word.dim300']
# 初始化TokenEmbedding， 预训练embedding没下载时会自动下载并加载数据
token_embedding = TokenEmbedding(embedding_name="w2v.baidu_encyclopedia.target.word-word.dim300")
# 查看token_embedding详情
print(token_embedding)
Object type: <paddlenlp.embeddings.token_embedding.TokenEmbedding object at 0x7fda7eb5f290>
Unknown index: 635963
Unknown token: [UNK]
Padding index: 635964
Padding token: [PAD]
Parameter containing:
Tensor(shape=[635965, 300], dtype=float32, place=CPUPlace, stop_gradient=False,
[[-0.24200200, 0.13931701, 0.07378800, ..., 0.14103900, 0.05592300, -0.08004800],
[-0.08671700, 0.07770800, 0.09515300, ..., 0.11196400, 0.03082200, -0.12893000],
[-0.11436500, 0.12201900, 0.02833000, ..., 0.11068700, 0.03607300, -0.13763499],
...,
[ 0.02628800, -0.00008300, -0.00393500, ..., 0.00654000, 0.00024600, -0.00662600],
[-0.00924490, 0.00652097, 0.01049327, ..., -0.01796000, 0.03498908, -0.02209341],
[ 0. , 0. , 0. , ..., 0. , 0. , 0. ]])
```
### 查询 embedding 结果
```python
test_token_embedding = token_embedding.search("中国")
print(test_token_embedding)
[[ 0.260801 0.1047 0.129453 -0.257317 -0.16152 0.19567 -0.074868
0.361168 0.245882 -0.219141 -0.388083 0.235189 0.029316 0.154215
-0.354343 0.017746 0.009028 0.01197 -0.121429 0.096542 0.009255
...,
-0.260592 -0.019668 -0.063312 -0.094939 0.657352 0.247547 -0.161621
0.289043 -0.284084 0.205076 0.059885 0.055871 0.159309 0.062181
0.123634 0.282932 0.140399 -0.076253 -0.087103 0.07262 ]]
```
### 可视化 embedding 结果
使用深度学习可视化工具[VisualDL](https://github.com/PaddlePaddle/VisualDL)的 High Dimensional 组件可以对 embedding 结果进行可视化展示，便于对其直观分析，步骤如下：
```python
# 获取词表中前1000个单词
labels = token_embedding.vocab.to_tokens(list(range(0,1000)))
test_token_embedding = token_embedding.search(labels)
# 引入VisualDL的LogWriter记录日志
from visualdl import LogWriter
with LogWriter(logdir='./visualize') as writer:
writer.add_embeddings(tag='test', mat=test_token_embedding, metadata=labels)
```
执行完毕后会在当前路径下生成一个 visualize 目录，并将日志存放在其中，我们在命令行启动 VisualDL 即可进行查看，启动命令为：
```shell
visualdl --logdir ./visualize
```
启动后打开浏览器即可看到可视化结果
<p align="center">
<img src="https://user-images.githubusercontent.com/48054808/103188111-1b32ac00-4902-11eb-914e-c2368bdb8373.gif" width="80%"/>
</p>
使用 VisualDL 除可视化 embedding 结果外，还可以对标量、图片、音频等进行可视化，有效提升训练调参效率。关于 VisualDL 更多功能和详细介绍，可参考[VisualDL 使用文档](https://github.com/PaddlePaddle/VisualDL/tree/develop/docs)。
### 计算词向量 cosine 相似度
```python
score = token_embedding.cosine_sim("中国", "美国")
print(score) # 0.49586025
```
### 计算词向量内积
```python
score = token_embedding.dot("中国", "美国")
print(score) # 8.611071
```
### 训练
以下为`TokenEmbedding`简单的组网使用方法。有关更多`TokenEmbedding`训练流程相关的使用方法，请参考[Word Embedding with PaddleNLP](https://github.com/PaddlePaddle/PaddleNLP/tree/release/2.8/examples/word_embedding)。
```python
in_words = paddle.to_tensor([0, 2, 3])
input_embeddings = token_embedding(in_words)
linear = paddle.nn.Linear(token_embedding.embedding_dim, 20)
input_fc = linear(input_embeddings)
print(input_fc)
Tensor(shape=[3, 20], dtype=float32, place=CPUPlace, stop_gradient=False,
[[ 0. , 0. , 0. , ..., 0. , 0. , 0. ],
[-0.23473957, 0.17878169, 0.07215232, ..., 0.03698236, 0.14291850, 0.05136518],
[-0.42466098, 0.15017235, -0.04780108, ..., -0.04995505, 0.15847842, 0.00025209]])
```
### 切词
```python
from paddlenlp.data import JiebaTokenizer
tokenizer = JiebaTokenizer(vocab=token_embedding.vocab)
words = tokenizer.cut("中国人民")
print(words) # ['中国人', '民']
tokens = tokenizer.encode("中国人民")
print(tokens) # [12530, 1334]
```
## 预训练模型
以下将列举 PaddleNLP 支持的 Embedding 预训练模型。
- 模型命名方式为：\${训练模型}.\${语料}.\${词向量类型}.\${co-occurrence type}.dim\${维度}。
- 模型有三种，分别是 Word2Vec(w2v, skip-gram), GloVe(glove)和 FastText(fasttext)。
### 中文词向量
以下预训练词向量由[Chinese-Word-Vectors](https://github.com/Embedding/Chinese-Word-Vectors)提供。
根据不同类型的上下文为每个语料训练多个目标词向量，第二列开始表示不同类型的上下文。以下为上下文类别：
* Word 表示训练时目标词预测的上下文是一个 Word。
* Word + N-gram 表示训练时目标词预测的上下文是一个 Word 或者 Ngram，其中 bigram 表示2-grams，ngram.1-2表示1-gram 或者2-grams。
* Word + Character 表示训练时目标词预测的上下文是一个 Word 或者 Character，其中 word-character.char1-2表示上下文是1个或2个 Character。
* Word + Character + Ngram 表示训练时目标词预测的上下文是一个 Word、Character 或者 Ngram。bigram-char 表示上下文是2-grams 或者1个 Character。
| 语料 | Word | Word + N-gram | Word + Character | Word + Character + N-gram |
| ------------------------------------------- | ---- | ---- | ---- | ---- |
| Baidu Encyclopedia 百度百科 | w2v.baidu_encyclopedia.target.word-word.dim300 | w2v.baidu_encyclopedia.target.word-ngram.1-2.dim300 | w2v.baidu_encyclopedia.target.word-character.char1-2.dim300 | w2v.baidu_encyclopedia.target.bigram-char.dim300 |
| Wikipedia_zh 中文维基百科 | w2v.wiki.target.word-word.dim300 | w2v.wiki.target.word-bigram.dim300 | w2v.wiki.target.word-char.dim300 | w2v.wiki.target.bigram-char.dim300 |
| People's Daily News 人民日报 | w2v.people_daily.target.word-word.dim300 | w2v.people_daily.target.word-bigram.dim300 | w2v.people_daily.target.word-char.dim300 | w2v.people_daily.target.bigram-char.dim300 |
| Sogou News 搜狗新闻 | w2v.sogou.target.word-word.dim300 | w2v.sogou.target.word-bigram.dim300 | w2v.sogou.target.word-char.dim300 | w2v.sogou.target.bigram-char.dim300 |
| Financial News 金融新闻 | w2v.financial.target.word-word.dim300 | w2v.financial.target.word-bigram.dim300 | w2v.financial.target.word-char.dim300 | w2v.financial.target.bigram-char.dim300 |
| Zhihu_QA 知乎问答 | w2v.zhihu.target.word-word.dim300 | w2v.zhihu.target.word-bigram.dim300 | w2v.zhihu.target.word-char.dim300 | w2v.zhihu.target.bigram-char.dim300 |
| Weibo 微博 | w2v.weibo.target.word-word.dim300 | w2v.weibo.target.word-bigram.dim300 | w2v.weibo.target.word-char.dim300 | w2v.weibo.target.bigram-char.dim300 |
| Literature 文学作品 | w2v.literature.target.word-word.dim300 | w2v.literature.target.word-bigram.dim300 | w2v.literature.target.word-char.dim300 | w2v.literature.target.bigram-char.dim300 |
| Complete Library in Four Sections 四库全书 | w2v.sikuquanshu.target.word-word.dim300 | w2v.sikuquanshu.target.word-bigram.dim300 | 无 | 无 |
| Mixed-large 综合 | w2v.mixed-large.target.word-word.dim300 | 暂无 | w2v.mixed-large.target.word-word.dim300 | 暂无 |
特别地，对于百度百科语料，在不同的 Co-occurrence 类型下分别提供了目标词与上下文向量：
| Co-occurrence 类型 | 目标词向量 | 上下文词向量 |
| --------------------------- | ------ | ---- |
| Word → Word | w2v.baidu_encyclopedia.target.word-word.dim300 | w2v.baidu_encyclopedia.context.word-word.dim300 |
| Word → Ngram (1-2) | w2v.baidu_encyclopedia.target.word-ngram.1-2.dim300 | w2v.baidu_encyclopedia.context.word-ngram.1-2.dim300 |
| Word → Ngram (1-3) | w2v.baidu_encyclopedia.target.word-ngram.1-3.dim300 | w2v.baidu_encyclopedia.context.word-ngram.1-3.dim300 |
| Ngram (1-2) → Ngram (1-2)| w2v.baidu_encyclopedia.target.word-ngram.2-2.dim300 | w2v.baidu_encyclopedia.target.word-ngram.2-2.dim300 |
| Word → Character (1) | w2v.baidu_encyclopedia.target.word-character.char1-1.dim300 | w2v.baidu_encyclopedia.context.word-character.char1-1.dim300 |
| Word → Character (1-2) | w2v.baidu_encyclopedia.target.word-character.char1-2.dim300 | w2v.baidu_encyclopedia.context.word-character.char1-2.dim300 |
| Word → Character (1-4) | w2v.baidu_encyclopedia.target.word-character.char1-4.dim300 | w2v.baidu_encyclopedia.context.word-character.char1-4.dim300 |
| Word → Word (left/right) | w2v.baidu_encyclopedia.target.word-wordLR.dim300 | w2v.baidu_encyclopedia.context.word-wordLR.dim300 |
| Word → Word (distance) | w2v.baidu_encyclopedia.target.word-wordPosition.dim300 | w2v.baidu_encyclopedia.context.word-wordPosition.dim300 |
### 英文词向量
### Word2Vec
| 语料 | 名称 |
|------|------|
| Google News | w2v.google_news.target.word-word.dim300.en |
### GloVe
| 语料 | 25维 | 50维 | 100维 | 200维 | 300 维 |
| ----------------- | ------ | ------ | ------ | ------ | ------ |
| Wiki2014 + GigaWord | 无 | glove.wiki2014-gigaword.target.word-word.dim50.en | glove.wiki2014-gigaword.target.word-word.dim100.en | glove.wiki2014-gigaword.target.word-word.dim200.en | glove.wiki2014-gigaword.target.word-word.dim300.en |
| Twitter | glove.twitter.target.word-word.dim25.en | glove.twitter.target.word-word.dim50.en | glove.twitter.target.word-word.dim100.en | glove.twitter.target.word-word.dim200.en | 无 |
### FastText
| 语料 | 名称 |
|------|------|
| Wiki2017 | fasttext.wiki-news.target.word-word.dim300.en |
| Crawl | fasttext.crawl.target.word-word.dim300.en |
### 使用方式
以上所述的模型名称可直接以参数形式传入`padddlenlp.embeddings.TokenEmbedding`，加载相对应的模型。比如要加载语料为 Wiki2017，通过 FastText 训练的预训练模型（`fasttext.wiki-news.target.word-word.dim300.en`），只需执行以下代码：
```python
import paddle
from paddlenlp.embeddings import TokenEmbedding
token_embedding = TokenEmbedding(embedding_name="fasttext.wiki-news.target.word-word.dim300.en")
```
### 模型信息
| 模型 | 文件大小 | 词表大小 |
|-----|---------|---------|
| w2v.baidu_encyclopedia.target.word-word.dim300 | 678.21 MB | 635965 |
| w2v.baidu_encyclopedia.target.word-character.char1-1.dim300 | 679.15 MB | 636038 |
| w2v.baidu_encyclopedia.target.word-character.char1-2.dim300 | 679.30 MB | 636038 |
| w2v.baidu_encyclopedia.target.word-character.char1-4.dim300 | 679.51 MB | 636038 |
| w2v.baidu_encyclopedia.target.word-ngram.1-2.dim300 | 679.48 MB | 635977 |
| w2v.baidu_encyclopedia.target.word-ngram.1-3.dim300 | 671.27 MB | 628669 |
| w2v.baidu_encyclopedia.target.word-ngram.2-2.dim300 | 7.28 GB | 6969069 |
| w2v.baidu_encyclopedia.target.word-wordLR.dim300 | 678.22 MB | 635958 |
| w2v.baidu_encyclopedia.target.word-wordPosition.dim300 | 679.32 MB | 636038 |
| w2v.baidu_encyclopedia.target.bigram-char.dim300 | 679.29 MB | 635976 |
| w2v.baidu_encyclopedia.context.word-word.dim300 | 677.74 MB | 635952 |
| w2v.baidu_encyclopedia.context.word-character.char1-1.dim300 | 678.65 MB | 636200 |
| w2v.baidu_encyclopedia.context.word-character.char1-2.dim300 | 844.23 MB | 792631 |
| w2v.baidu_encyclopedia.context.word-character.char1-4.dim300 | 1.16 GB | 1117461 |
| w2v.baidu_encyclopedia.context.word-ngram.1-2.dim300 | 7.25 GB | 6967598 |
| w2v.baidu_encyclopedia.context.word-ngram.1-3.dim300 | 5.21 GB | 5000001 |
| w2v.baidu_encyclopedia.context.word-ngram.2-2.dim300 | 7.26 GB | 6968998 |
| w2v.baidu_encyclopedia.context.word-wordLR.dim300 | 1.32 GB | 1271031 |
| w2v.baidu_encyclopedia.context.word-wordPosition.dim300 | 6.47 GB | 6293920 |
| w2v.wiki.target.bigram-char.dim300 | 375.98 MB | 352274 |
| w2v.wiki.target.word-char.dim300 | 375.52 MB | 352223 |
| w2v.wiki.target.word-word.dim300 | 374.95 MB | 352219 |
| w2v.wiki.target.word-bigram.dim300 | 375.72 MB | 352219 |
| w2v.people_daily.target.bigram-char.dim300 | 379.96 MB | 356055 |
| w2v.people_daily.target.word-char.dim300 | 379.45 MB | 355998 |
| w2v.people_daily.target.word-word.dim300 | 378.93 MB | 355989 |
| w2v.people_daily.target.word-bigram.dim300 | 379.68 MB | 355991 |
| w2v.weibo.target.bigram-char.dim300 | 208.24 MB | 195199 |
| w2v.weibo.target.word-char.dim300 | 208.03 MB | 195204 |
| w2v.weibo.target.word-word.dim300 | 207.94 MB | 195204 |
| w2v.weibo.target.word-bigram.dim300 | 208.19 MB | 195204 |
| w2v.sogou.target.bigram-char.dim300 | 389.81 MB | 365112 |
| w2v.sogou.target.word-char.dim300 | 389.89 MB | 365078 |
| w2v.sogou.target.word-word.dim300 | 388.66 MB | 364992 |
| w2v.sogou.target.word-bigram.dim300 | 388.66 MB | 364994 |
| w2v.zhihu.target.bigram-char.dim300 | 277.35 MB | 259755 |
| w2v.zhihu.target.word-char.dim300 | 277.40 MB | 259940 |
| w2v.zhihu.target.word-word.dim300 | 276.98 MB | 259871 |
| w2v.zhihu.target.word-bigram.dim300 | 277.53 MB | 259885 |
| w2v.financial.target.bigram-char.dim300 | 499.52 MB | 467163 |
| w2v.financial.target.word-char.dim300 | 499.17 MB | 467343 |
| w2v.financial.target.word-word.dim300 | 498.94 MB | 467324 |
| w2v.financial.target.word-bigram.dim300 | 499.54 MB | 467331 |
| w2v.literature.target.bigram-char.dim300 | 200.69 MB | 187975 |
| w2v.literature.target.word-char.dim300 | 200.44 MB | 187980 |
| w2v.literature.target.word-word.dim300 | 200.28 MB | 187961 |
| w2v.literature.target.word-bigram.dim300 | 200.59 MB | 187962 |
| w2v.sikuquanshu.target.word-word.dim300 | 20.70 MB | 19529 |
| w2v.sikuquanshu.target.word-bigram.dim300 | 20.77 MB | 19529 |
| w2v.mixed-large.target.word-char.dim300 | 1.35 GB | 1292552 |
| w2v.mixed-large.target.word-word.dim300 | 1.35 GB | 1292483 |
| w2v.google_news.target.word-word.dim300.en | 1.61 GB | 3000000 |
| glove.wiki2014-gigaword.target.word-word.dim50.en | 73.45 MB | 400002 |
| glove.wiki2014-gigaword.target.word-word.dim100.en | 143.30 MB | 400002 |
| glove.wiki2014-gigaword.target.word-word.dim200.en | 282.97 MB | 400002 |
| glove.wiki2014-gigaword.target.word-word.dim300.en | 422.83 MB | 400002 |
| glove.twitter.target.word-word.dim25.en | 116.92 MB | 1193516 |
| glove.twitter.target.word-word.dim50.en | 221.64 MB | 1193516 |
| glove.twitter.target.word-word.dim100.en | 431.08 MB | 1193516 |
| glove.twitter.target.word-word.dim200.en | 848.56 MB | 1193516 |
| fasttext.wiki-news.target.word-word.dim300.en | 541.63 MB | 999996 |
| fasttext.crawl.target.word-word.dim300.en | 1.19 GB | 2000002 |
## 致谢
- 感谢 [Chinese-Word-Vectors](https://github.com/Embedding/Chinese-Word-Vectors)提供 Word2Vec 中文预训练词向量。
- 感谢 [GloVe Project](https://nlp.stanford.edu/projects/glove)提供的 GloVe 英文预训练词向量。
- 感谢 [FastText Project](https://fasttext.cc/docs/en/english-vectors.html)提供的英文预训练词向量。
## 参考论文
- Li, Shen, et al. "Analogical reasoning on chinese morphological and semantic relations." arXiv preprint arXiv:1805.06504 (2018).
- Qiu, Yuanyuan, et al. "Revisiting correlations between intrinsic and extrinsic evaluations of word embeddings." Chinese Computational Linguistics and Natural Language Processing Based on Naturally Annotated Big Data. Springer, Cham, 2018. 209-221.
- Jeffrey Pennington, Richard Socher, and Christopher D. Manning. 2014. GloVe: Global Vectors for Word Representation.
- T. Mikolov, E. Grave, P. Bojanowski, C. Puhrsch, A. Joulin. Advances in Pre-Training Distributed Word Representations.

14
docs/zh/source/paddlenlp.embeddings.rst
View File

@@ -1,14 +0,0 @@
paddlenlp.embeddings
============================
.. automodule:: paddlenlp.embeddings
:members:
:no-undoc-members:
:show-inheritance:
.. toctree::
:maxdepth: 4
paddlenlp.embeddings.constant
paddlenlp.embeddings.token_embedding

7
docs/zh/source/paddlenlp.embeddings.token_embedding.rst
View File

@@ -1,7 +0,0 @@
token\_embedding
============================================
.. automodule:: paddlenlp.embeddings.token_embedding
:members:
:no-undoc-members:
:show-inheritance:

1
docs/zh/source/paddlenlp.rst
View File

@@ -13,7 +13,6 @@ paddlenlp
paddlenlp.data
paddlenlp.dataaug
paddlenlp.datasets
paddlenlp.embeddings
paddlenlp.experimental
paddlenlp.layers
paddlenlp.losses

1
paddlenlp/__init__.py
View File

@@ -44,7 +44,6 @@ from . import (
data,
dataaug,
datasets,
embeddings,
experimental,
layers,
losses,

15
paddlenlp/embeddings/__init__.py
View File

@@ -1,15 +0,0 @@
# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from .token_embedding import *

98
paddlenlp/embeddings/constant.py
View File

@@ -1,98 +0,0 @@
# Copyright (c) 2020 PaddlePaddle Authors and Chinese-Word-Vectors Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
URL_ROOT = "https://bj.bcebos.com/paddlenlp"
EMBEDDING_URL_ROOT = URL_ROOT + "/models/embeddings"
PAD_TOKEN = "[PAD]"
UNK_TOKEN = "[UNK]"
EMBEDDING_NAME_LIST = [
# Word2Vec
# baidu_encyclopedia
"w2v.baidu_encyclopedia.target.word-word.dim300",
"w2v.baidu_encyclopedia.target.word-character.char1-1.dim300",
"w2v.baidu_encyclopedia.target.word-character.char1-2.dim300",
"w2v.baidu_encyclopedia.target.word-character.char1-4.dim300",
"w2v.baidu_encyclopedia.target.word-ngram.1-2.dim300",
"w2v.baidu_encyclopedia.target.word-ngram.1-3.dim300",
"w2v.baidu_encyclopedia.target.word-ngram.2-2.dim300",
"w2v.baidu_encyclopedia.target.word-wordLR.dim300",
"w2v.baidu_encyclopedia.target.word-wordPosition.dim300",
"w2v.baidu_encyclopedia.target.bigram-char.dim300",
"w2v.baidu_encyclopedia.context.word-word.dim300",
"w2v.baidu_encyclopedia.context.word-character.char1-1.dim300",
"w2v.baidu_encyclopedia.context.word-character.char1-2.dim300",
"w2v.baidu_encyclopedia.context.word-character.char1-4.dim300",
"w2v.baidu_encyclopedia.context.word-ngram.1-2.dim300",
"w2v.baidu_encyclopedia.context.word-ngram.1-3.dim300",
"w2v.baidu_encyclopedia.context.word-ngram.2-2.dim300",
"w2v.baidu_encyclopedia.context.word-wordLR.dim300",
"w2v.baidu_encyclopedia.context.word-wordPosition.dim300",
# wikipedia
"w2v.wiki.target.bigram-char.dim300",
"w2v.wiki.target.word-char.dim300",
"w2v.wiki.target.word-word.dim300",
"w2v.wiki.target.word-bigram.dim300",
# people_daily
"w2v.people_daily.target.bigram-char.dim300",
"w2v.people_daily.target.word-char.dim300",
"w2v.people_daily.target.word-word.dim300",
"w2v.people_daily.target.word-bigram.dim300",
# weibo
"w2v.weibo.target.bigram-char.dim300",
"w2v.weibo.target.word-char.dim300",
"w2v.weibo.target.word-word.dim300",
"w2v.weibo.target.word-bigram.dim300",
# sogou
"w2v.sogou.target.bigram-char.dim300",
"w2v.sogou.target.word-char.dim300",
"w2v.sogou.target.word-word.dim300",
"w2v.sogou.target.word-bigram.dim300",
# zhihu
"w2v.zhihu.target.bigram-char.dim300",
"w2v.zhihu.target.word-char.dim300",
"w2v.zhihu.target.word-word.dim300",
"w2v.zhihu.target.word-bigram.dim300",
# finacial
"w2v.financial.target.bigram-char.dim300",
"w2v.financial.target.word-char.dim300",
"w2v.financial.target.word-word.dim300",
"w2v.financial.target.word-bigram.dim300",
# literature
"w2v.literature.target.bigram-char.dim300",
"w2v.literature.target.word-char.dim300",
"w2v.literature.target.word-word.dim300",
"w2v.literature.target.word-bigram.dim300",
# siku
"w2v.sikuquanshu.target.word-word.dim300",
"w2v.sikuquanshu.target.word-bigram.dim300",
# Mix-large
"w2v.mixed-large.target.word-char.dim300",
"w2v.mixed-large.target.word-word.dim300",
# GOOGLE NEWS
"w2v.google_news.target.word-word.dim300.en",
# GloVe
"glove.wiki2014-gigaword.target.word-word.dim50.en",
"glove.wiki2014-gigaword.target.word-word.dim100.en",
"glove.wiki2014-gigaword.target.word-word.dim200.en",
"glove.wiki2014-gigaword.target.word-word.dim300.en",
"glove.twitter.target.word-word.dim25.en",
"glove.twitter.target.word-word.dim50.en",
"glove.twitter.target.word-word.dim100.en",
"glove.twitter.target.word-word.dim200.en",
# FastText
"fasttext.wiki-news.target.word-word.dim300.en",
"fasttext.crawl.target.word-word.dim300.en",
]

378
paddlenlp/embeddings/token_embedding.py
View File

@@ -1,378 +0,0 @@
# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import os.path as osp
import numpy as np
import paddle
import paddle.nn as nn
from paddle.utils.download import get_path_from_url
from paddlenlp.data import Vocab, get_idx_from_word
from paddlenlp.utils.env import MODEL_HOME, _get_sub_home
from paddlenlp.utils.log import logger
from .constant import EMBEDDING_NAME_LIST, EMBEDDING_URL_ROOT, PAD_TOKEN, UNK_TOKEN
EMBEDDING_HOME = _get_sub_home("embeddings", parent_home=MODEL_HOME)
__all__ = ["list_embedding_name", "TokenEmbedding"]
def list_embedding_name():
"""
Lists all names of pretrained embedding models paddlenlp provides.
"""
return list(EMBEDDING_NAME_LIST)
class TokenEmbedding(nn.Embedding):
"""
A `TokenEmbedding` can load pre-trained embedding model which paddlenlp provides by
specifying embedding name. Furthermore, a `TokenEmbedding` can load extended vocabulary
by specifying extended_vocab_path.
Args:
embedding_name (`str`, optional):
The pre-trained embedding model name. Use `paddlenlp.embeddings.list_embedding_name()` to
list the names of all embedding models that we provide.
Defaults to `w2v.baidu_encyclopedia.target.word-word.dim300`.
unknown_token (`str`, optional):
Specifies unknown token.
Defaults to `[UNK]`.
unknown_token_vector (`list`, optional):
To initialize the vector of unknown token. If it's none, use normal distribution to
initialize the vector of unknown token.
Defaults to `None`.
extended_vocab_path (`str`, optional):
The file path of extended vocabulary.
Defaults to `None`.
trainable (`bool`, optional):
Whether the weight of embedding can be trained.
Defaults to True.
keep_extended_vocab_only (`bool`, optional):
Whether to keep the extended vocabulary only, will be effective only if provides extended_vocab_path.
Defaults to False.
"""
def __init__(
self,
embedding_name=EMBEDDING_NAME_LIST[0],
unknown_token=UNK_TOKEN,
unknown_token_vector=None,
extended_vocab_path=None,
trainable=True,
keep_extended_vocab_only=False,
):
vector_path = osp.join(EMBEDDING_HOME, embedding_name + ".npz")
if not osp.exists(vector_path):
# download
url = EMBEDDING_URL_ROOT + "/" + embedding_name + ".tar.gz"
get_path_from_url(url, EMBEDDING_HOME)
logger.info("Loading token embedding...")
vector_np = np.load(vector_path)
self.embedding_dim = vector_np["embedding"].shape[1]
self.unknown_token = unknown_token
if unknown_token_vector is not None:
unk_vector = np.array(unknown_token_vector).astype(paddle.get_default_dtype())
else:
unk_vector = np.random.normal(scale=0.02, size=self.embedding_dim).astype(paddle.get_default_dtype())
pad_vector = np.array([0] * self.embedding_dim).astype(paddle.get_default_dtype())
if extended_vocab_path is not None:
embedding_table = self._extend_vocab(
extended_vocab_path, vector_np, pad_vector, unk_vector, keep_extended_vocab_only
)
trainable = True
else:
embedding_table = self._init_without_extend_vocab(vector_np, pad_vector, unk_vector)
self.vocab = Vocab.from_dict(self._word_to_idx, unk_token=unknown_token, pad_token=PAD_TOKEN)
self.num_embeddings = embedding_table.shape[0]
# import embedding
super(TokenEmbedding, self).__init__(
self.num_embeddings, self.embedding_dim, padding_idx=self._word_to_idx[PAD_TOKEN]
)
self.weight.set_value(embedding_table)
self.set_trainable(trainable)
logger.info("Finish loading embedding vector.")
s = "Token Embedding info:\
\nUnknown index: {}\
\nUnknown token: {}\
\nPadding index: {}\
\nPadding token: {}\
\nShape :{}".format(
self._word_to_idx[self.unknown_token],
self.unknown_token,
self._word_to_idx[PAD_TOKEN],
PAD_TOKEN,
self.weight.shape,
)
logger.info(s)
def _init_without_extend_vocab(self, vector_np, pad_vector, unk_vector):
"""
Constructs index to word list, word to index dict and embedding weight.
"""
self._idx_to_word = list(vector_np["vocab"])
self._idx_to_word.append(self.unknown_token)
self._idx_to_word.append(PAD_TOKEN)
self._word_to_idx = self._construct_word_to_idx(self._idx_to_word)
# insert unk, pad embedding
embedding_table = np.append(vector_np["embedding"], [unk_vector, pad_vector], axis=0)
return embedding_table
def _read_vocab_list_from_file(self, extended_vocab_path):
# load new vocab table from file
vocab_list = []
with open(extended_vocab_path, "r", encoding="utf-8") as f:
for line in f.readlines():
vocab = line.rstrip("\n").split("\t")[0]
vocab_list.append(vocab)
return vocab_list
def _extend_vocab(self, extended_vocab_path, vector_np, pad_vector, unk_vector, keep_extended_vocab_only):
"""
Constructs index to word list, word to index dict and embedding weight using
extended vocab.
"""
logger.info("Start extending vocab.")
extend_vocab_list = self._read_vocab_list_from_file(extended_vocab_path)
extend_vocab_set = set(extend_vocab_list)
# update idx_to_word
self._idx_to_word = extend_vocab_list
self._word_to_idx = self._construct_word_to_idx(self._idx_to_word)
# use the Xavier init the embedding
xavier_scale = np.sqrt(6.0 / float(len(self._idx_to_word) + self.embedding_dim))
embedding_table = np.random.uniform(
low=-1.0 * xavier_scale, high=xavier_scale, size=(len(self._idx_to_word), self.embedding_dim)
).astype(paddle.get_default_dtype())
pretrained_idx_to_word = list(vector_np["vocab"])
pretrained_word_to_idx = self._construct_word_to_idx(pretrained_idx_to_word)
pretrained_embedding_table = np.array(vector_np["embedding"])
pretrained_vocab_set = set(pretrained_idx_to_word)
extend_vocab_set = set(self._idx_to_word)
vocab_intersection = pretrained_vocab_set & extend_vocab_set
vocab_subtraction = pretrained_vocab_set - extend_vocab_set
# assignment from pretrained_vocab_embedding to extend_vocab_embedding
pretrained_vocab_intersect_index = [pretrained_word_to_idx[word] for word in vocab_intersection]
pretrained_vocab_subtract_index = [pretrained_word_to_idx[word] for word in vocab_subtraction]
extend_vocab_intersect_index = [self._word_to_idx[word] for word in vocab_intersection]
embedding_table[extend_vocab_intersect_index] = pretrained_embedding_table[pretrained_vocab_intersect_index]
if not keep_extended_vocab_only:
for idx in pretrained_vocab_subtract_index:
word = pretrained_idx_to_word[idx]
self._idx_to_word.append(word)
self._word_to_idx[word] = len(self._idx_to_word) - 1
embedding_table = np.append(
embedding_table, pretrained_embedding_table[pretrained_vocab_subtract_index], axis=0
)
if self.unknown_token not in extend_vocab_set:
self._idx_to_word.append(self.unknown_token)
self._word_to_idx[self.unknown_token] = len(self._idx_to_word) - 1
embedding_table = np.append(embedding_table, [unk_vector], axis=0)
else:
unk_idx = self._word_to_idx[self.unknown_token]
embedding_table[unk_idx] = unk_vector
if PAD_TOKEN not in extend_vocab_set:
self._idx_to_word.append(PAD_TOKEN)
self._word_to_idx[PAD_TOKEN] = len(self._idx_to_word) - 1
embedding_table = np.append(embedding_table, [pad_vector], axis=0)
else:
embedding_table[self._word_to_idx[PAD_TOKEN]] = pad_vector
logger.info("Finish extending vocab.")
return embedding_table
def set_trainable(self, trainable):
"""
Whether or not to set the weights of token embedding to be trainable.
Args:
trainable (`bool`):
The weights can be trained if trainable is set to True, or the weights are fixed if trainable is False.
"""
self.weight.stop_gradient = not trainable
def search(self, words):
"""
Gets the vectors of specifying words.
Args:
words (`list` or `str` or `int`): The words which need to be searched.
Returns:
`numpy.array`: The vectors of specifying words.
Examples:
.. code-block::
from paddlenlp.embeddings import TokenEmbedding
embed = TokenEmbedding()
vector = embed.search('Welcome to use PaddlePaddle and PaddleNLP!')
"""
idx_list = self.get_idx_list_from_words(words)
idx_tensor = paddle.to_tensor(idx_list)
return self(idx_tensor).cpu().numpy()
def get_idx_from_word(self, word):
"""
Gets the index of specifying word by searching word_to_idx dict.
Args:
word (`list` or `str` or `int`): The input token word which we want to get the token index converted from.
Returns:
`int`: The index of specifying word.
"""
return get_idx_from_word(word, self.vocab.token_to_idx, self.unknown_token)
def get_idx_list_from_words(self, words):
"""
Gets the index list of specifying words by searching word_to_idx dict.
Args:
words (`list` or `str` or `int`): The input token words which we want to get the token indices converted from.
Returns:
`list`: The indexes list of specifying words.
Examples:
.. code-block::
from paddlenlp.embeddings import TokenEmbedding
embed = TokenEmbedding()
index = embed.get_idx_from_word('Welcome to use PaddlePaddle and PaddleNLP!')
#635963
"""
if isinstance(words, str):
idx_list = [self.get_idx_from_word(words)]
elif isinstance(words, int):
idx_list = [words]
elif isinstance(words, list) or isinstance(words, tuple):
idx_list = [self.get_idx_from_word(word) if isinstance(word, str) else word for word in words]
else:
raise TypeError
return idx_list
def _dot_np(self, array_a, array_b):
return np.sum(array_a * array_b)
def _calc_word(self, word_a, word_b, calc_kernel):
embeddings = self.search([word_a, word_b])
embedding_a = embeddings[0]
embedding_b = embeddings[1]
return calc_kernel(embedding_a, embedding_b)
def dot(self, word_a, word_b):
"""
Calculates the dot product of 2 words. Dot product or scalar product is an
algebraic operation that takes two equal-length sequences of numbers (usually
coordinate vectors), and returns a single number.
Args:
word_a (`str`): The first word string.
word_b (`str`): The second word string.
Returns:
float: The dot product of 2 words.
Examples:
.. code-block::
from paddlenlp.embeddings import TokenEmbedding
embed = TokenEmbedding()
dot_product = embed.dot('PaddlePaddle', 'PaddleNLP!')
#0.11827179
"""
dot = self._dot_np
return self._calc_word(word_a, word_b, lambda x, y: dot(x, y))
def cosine_sim(self, word_a, word_b):
"""
Calculates the cosine similarity of 2 word vectors. Cosine similarity is the
cosine of the angle between two n-dimensional vectors in an n-dimensional space.
Args:
word_a (`str`): The first word string.
word_b (`str`): The second word string.
Returns:
float: The cosine similarity of 2 words.
Examples:
.. code-block::
from paddlenlp.embeddings import TokenEmbedding
embed = TokenEmbedding()
cosine_simi = embed.cosine_sim('PaddlePaddle', 'PaddleNLP!')
#0.99999994
"""
dot = self._dot_np
return self._calc_word(word_a, word_b, lambda x, y: dot(x, y) / (np.sqrt(dot(x, x)) * np.sqrt(dot(y, y))))
def _construct_word_to_idx(self, idx_to_word):
"""
Constructs word to index dict.
Args:
idx_to_word ('list'):
Returns:
`Dict`: The word to index dict constructed by idx_to_word.
"""
word_to_idx = {}
for i, word in enumerate(idx_to_word):
word_to_idx[word] = i
return word_to_idx
def __repr__(self):
"""
Returns:
`Str`: The token embedding information.
"""
info = "Object type: {}\
\nUnknown index: {}\
\nUnknown token: {}\
\nPadding index: {}\
\nPadding token: {}\
\n{}".format(
super(TokenEmbedding, self).__repr__(),
self._word_to_idx[self.unknown_token],
self.unknown_token,
self._word_to_idx[PAD_TOKEN],
PAD_TOKEN,
self.weight,
)
return info

9
paddlenlp/transformers/__init__.py
View File

@@ -140,6 +140,9 @@ from .dallebart.tokenizer import *
from .deberta.configuration import *
from .deberta.modeling import *
from .deberta.tokenizer import *
from .nezha.configuration import *
from .nezha.modeling import *
from .nezha.tokenizer import *
from .deberta_v2.configuration import *
from .deberta_v2.modeling import *
from .deberta_v2.tokenizer import *
@@ -188,9 +191,6 @@ from .fnet.tokenizer import *
from .funnel.configuration import *
from .funnel.modeling import *
from .funnel.tokenizer import *
from .gau_alpha.configuration import *
from .gau_alpha.modeling import *
from .gau_alpha.tokenizer import *
from .gemma import *
from .glm.configuration import *
from .glm.modeling import *
@@ -243,9 +243,6 @@ from .mpnet.modeling import *
from .mpnet.tokenizer import *
from .mt5.configuration import *
from .mt5.modeling import *
from .nezha.configuration import *
from .nezha.modeling import *
from .nezha.tokenizer import *
from .nv_embed.modeling import *
from .nystromformer.configuration import *
from .nystromformer.modeling import *

2
paddlenlp/transformers/auto/configuration.py
View File

@@ -71,7 +71,6 @@ CONFIG_MAPPING_NAMES = OrderedDict(
("ernie_vil", "ErnieViLConfig"),
("fnet", "FNetConfig"),
("funnel", "FunnelConfig"),
("gau_alpha", "GAUAlphaConfig"),
("gemma", "GemmaConfig"),
("glm", "GLMConfig"),
("gpt", "GPTConfig"),
@@ -163,7 +162,6 @@ MODEL_NAMES_MAPPING = OrderedDict(
("ernie_vil", "ErnieViL"),
("fnet", "FNet"),
("funnel", "Funnel"),
("gau_alpha", "GAUAlpha"),
("gemma", "Gemma"),
("glm", "GLM"),
("gpt", "GPT"),

1
paddlenlp/transformers/auto/modeling.py
View File

@@ -104,7 +104,6 @@ MAPPING_NAMES = OrderedDict(
("T5", "t5"),
("Bert", "bert"),
("Bart", "bart"),
("GAUAlpha", "gau_alpha"),
("CodeGen", "codegen"),
("CLIPVision", "clip"),
("CLIPText", "clip"),

1
paddlenlp/transformers/auto/tokenizer.py
View File

@@ -145,7 +145,6 @@ else:
"GPTTokenizerFast" if is_tokenizers_available() else None,
),
),
("gau_alpha", "GAUAlphaTokenizer"),
("artist", "ArtistTokenizer"),
("chineseclip", "ChineseCLIPTokenizer"),
("ernie_vil", "ErnieViLTokenizer"),

13
paddlenlp/transformers/gau_alpha/__init__.py
View File

@@ -1,13 +0,0 @@
# Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

161
paddlenlp/transformers/gau_alpha/configuration.py
View File

@@ -1,161 +0,0 @@
# Copyright (c) 2023 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from __future__ import annotations
from ..configuration_utils import PretrainedConfig
__all__ = ["GAUAlPHA_PRETRAINED_INIT_CONFIGURATION", "GAUAlphaConfig", "GAUAlPHA_PRETRAINED_RESOURCE_FILES_MAP"]
GAUAlPHA_PRETRAINED_INIT_CONFIGURATION = {
"chinese_GAU-alpha-char_L-24_H-768": {
"vocab_size": 12000,
"hidden_size": 768,
"intermediate_size": 1536,
"num_hidden_layers": 24,
"max_position_embeddings": 512,
"type_vocab_size": 2,
"attention_key_size": 128,
"norm_eps": 1e-12,
"pad_token_id": 0,
"hidden_dropout_prob": 0.1,
"attention_probs_dropout_prob": 0.1,
"hidden_act": "swish",
"use_bias": False,
"normalization": "softmax_plus",
"attention_scale": True,
},
}
GAUAlPHA_PRETRAINED_RESOURCE_FILES_MAP = {
"model_state": {
"chinese_GAU-alpha-char_L-24_H-768": "https://bj.bcebos.com/paddlenlp/models/transformers/gau_alpha/chinese_GAU-alpha-char_L-24_H-768/model_state.pdparams",
}
}
class GAUAlphaConfig(PretrainedConfig):
r"""
This is the configuration class to store the configuration of a [`GAUAlphaModel`]. It is used to
instantiate a GAUAlpha model according to the specified arguments, defining the model architecture. Instantiating a
configuration with the defaults will yield a similar configuration to that of the GAUAlpha
chinese_GAU-alpha-char_L-24_H-768 architecture.
Configuration objects inherit from [`PretrainedConfig`] and can be used to control the model outputs. Read the
documentation from [`PretrainedConfig`] for more information.
Args:
vocab_size (`int`, *optional*, defaults to 30522):
Vocabulary size of the GAUAlpha model. Defines the number of different tokens that can be represented by the
`inputs_ids` passed when calling [`GAUAlphaModel`].
hidden_size (`int`, *optional*, defaults to 768):
Dimensionality of the encoder layers and the pooler layer.
num_hidden_layers (`int`, *optional*, defaults to 12):
Number of hidden layers in the Transformer encoder.
num_attention_heads (`int`, *optional*, defaults to 12):
Number of attention heads for each attention layer in the Transformer encoder.
intermediate_size (`int`, *optional*, defaults to 3072):
Dimensionality of the "intermediate" (often named feed-forward) layer in the Transformer encoder.
hidden_act (`str` or `Callable`, *optional*, defaults to `"gelu"`):
The non-linear activation function (function or string) in the encoder and pooler. If string, `"gelu"`,
`"relu"`, `"silu"` and `"gelu_new"` are supported.
hidden_dropout_prob (`float`, *optional*, defaults to 0.1):
The dropout probability for all fully connected layers in the embeddings, encoder, and pooler.
attention_probs_dropout_prob (`float`, *optional*, defaults to 0.1):
The dropout ratio for the attention probabilities.
max_position_embeddings (`int`, *optional*, defaults to 512):
The maximum sequence length that this model might ever be used with. Typically set this to something large
just in case (e.g., 512 or 1024 or 2048).
type_vocab_size (`int`, *optional*, defaults to 2):
The vocabulary size of the `token_type_ids` passed when calling [`GAUAlphaModel`].
initializer_range (`float`, *optional*, defaults to 0.02):
The standard deviation of the truncated_normal_initializer for initializing all weight matrices.
layer_norm_eps (`float`, *optional*, defaults to 1e-12):
The epsilon used by the layer normalization layers.
position_embedding_type (`str`, *optional*, defaults to `"absolute"`):
Type of position embedding. Choose one of `"absolute"`, `"relative_key"`, `"relative_key_query"`. For
positional embeddings use `"absolute"`. For more information on `"relative_key"`, please refer to
[Self-Attention with Relative Position Representations (Shaw et al.)](https://arxiv.org/abs/1803.02155).
For more information on `"relative_key_query"`, please refer to *Method 4* in [Improve Transformer Models
with Better Relative Position Embeddings (Huang et al.)](https://arxiv.org/abs/2009.13658).
use_cache (`bool`, *optional*, defaults to `True`):
Whether or not the model should return the last key/values attentions (not used by all models). Only
relevant if `config.is_decoder=True`.
classifier_dropout (`float`, *optional*):
The dropout ratio for the classification head.
Examples:
```python
>>> from paddlenlp.transformers import GAUAlphaModel, GAUAlphaConfig
>>> # Initializing a GAUAlpha chinese_GAU-alpha-char_L-24_H-768style configuration
>>> configuration = GAUAlphaConfig()
>>> # Initializing a model from the style configuration
>>> model = GAUAlphaModel(configuration)
>>> # Accessing the model configuration
>>> configuration = model.config
```"""
model_type = "gau_alpha"
pretrained_init_configuration = GAUAlPHA_PRETRAINED_INIT_CONFIGURATION
def __init__(
self,
vocab_size: int = 30522,
hidden_size: int = 768,
num_hidden_layers: int = 12,
num_attention_heads: int = 12,
task_id=0,
intermediate_size: int = 3072,
hidden_act: str = "gelu",
hidden_dropout_prob: float = 0.1,
attention_probs_dropout_prob: float = 0.1,
max_position_embeddings: int = 512,
task_type_vocab_size: int = 3,
type_vocab_size: int = 16,
attention_key_size=128,
initializer_range: float = 0.02,
pad_token_id: int = 0,
pool_act: str = "tanh",
activation: str = "swish",
normalization: str = "softmax_plus",
fuse: bool = False,
layer_norm_eps=1e-12,
norm_eps=1e-12,
use_cache=False,
use_task_id=True,
use_bias=False,
attention_scale=True,
**kwargs
):
super().__init__(pad_token_id=pad_token_id, **kwargs)
self.vocab_size = vocab_size
self.hidden_size = hidden_size
self.num_hidden_layers = num_hidden_layers
self.num_attention_heads = num_attention_heads
self.task_id = task_id
self.intermediate_size = intermediate_size
self.hidden_act = hidden_act
self.hidden_dropout_prob = hidden_dropout_prob
self.attention_probs_dropout_prob = attention_probs_dropout_prob
self.max_position_embeddings = max_position_embeddings
self.task_type_vocab_size = task_type_vocab_size
self.type_vocab_size = type_vocab_size
self.initializer_range = initializer_range
self.pool_act = pool_act
self.fuse = fuse
self.layer_norm_eps = layer_norm_eps
self.norm_eps = norm_eps
self.use_cache = use_cache
self.use_task_id = use_task_id
self.use_bias = use_bias
self.activation = activation
self.attention_key_size = attention_key_size
self.normalization = normalization
self.attention_scale = attention_scale

810
paddlenlp/transformers/gau_alpha/modeling.py
View File

@@ -1,810 +0,0 @@
# Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import numpy as np
import paddle
import paddle.nn as nn
import paddle.nn.functional as F
from paddle.nn import Layer
from paddlenlp.utils.env import CONFIG_NAME
from .. import PretrainedModel, register_base_model
from ..albert.modeling import ACT2FN
from .configuration import (
GAUAlPHA_PRETRAINED_INIT_CONFIGURATION,
GAUAlPHA_PRETRAINED_RESOURCE_FILES_MAP,
GAUAlphaConfig,
)
__all__ = [
"GAUAlphaModel",
"GAUAlphaForMaskedLM",
"GAUAlphaPretrainedModel",
"GAUAlphaForSequenceClassification",
"GAUAlphaForTokenClassification",
"GAUAlphaForQuestionAnswering",
"GAUAlphaForMultipleChoice",
]
INF = 1e4
class Norm(Layer):
def __init__(self, epsilon=1e-12):
super().__init__()
self._epsilon = epsilon
def forward(self, x):
variance = paddle.mean(paddle.square(x), axis=-1, keepdim=True)
return x / paddle.sqrt(variance + self._epsilon)
def attention_normalize(a, mask=None, axis=-1, method="softmax"):
if method == "softmax":
return F.softmax(a, axis=axis)
else:
if mask is not None:
l = mask.sum(-1, keepdim=True)
else:
l = paddle.ones_like(a) * a.shape[-2]
if method == "squared_relu":
return F.relu(a) ** 2 / l
elif method == "softmax_plus":
scale = paddle.log(l) / np.log(512)
# mask: 1 for not padding, 0 for padding
# padding position's scale is 1
if mask is not None:
scale = scale * mask + 1 - mask
return F.softmax(a * scale, axis=axis)
return a
class ScaleOffset(Layer):
def __init__(
self,
hidden_size=768,
scale=True,
offset=True,
):
super().__init__()
self.scale = scale
self.offset = offset
if self.scale:
self.weight = self.create_parameter((hidden_size,), default_initializer=nn.initializer.Constant(1.0))
if self.offset:
self.bias = self.create_parameter((hidden_size,), is_bias=True)
def forward(self, inputs):
if self.scale:
inputs = inputs * self.weight
if self.offset:
inputs = inputs + self.bias
return inputs
class GatedAttentionUnit(Layer):
"""
https://github.com/ZhuiyiTechnology/GAU-alpha/blob/ea15e08a85d35652775c360218090cbaed98da18/models.py#L6-L85
"""
def __init__(self, config: GAUAlphaConfig):
super().__init__()
self.activation = ACT2FN[config.activation]
self.intermediate_size = config.intermediate_size
self.attention_key_size = config.attention_key_size
self.use_bias = config.use_bias
self.normalization = config.normalization
self.attention_scale = config.attention_scale
self.attention_dropout = config.attention_probs_dropout_prob
self.i_dense = nn.Linear(
config.hidden_size,
2 * config.intermediate_size + config.attention_key_size,
bias_attr=self.use_bias,
)
self.o_dense = nn.Linear(config.intermediate_size, config.hidden_size, bias_attr=self.use_bias)
self.q_scaleoffset = ScaleOffset(config.attention_key_size, offset=self.use_bias)
self.k_scaleoffset = ScaleOffset(config.attention_key_size, offset=self.use_bias)
self.rotary = RotaryPositionEmbedding(config)
def forward(self, hidden_states, attention_mask=None):
x = self.i_dense(hidden_states)
u, v, qk = paddle.split(
self.activation(x),
[self.intermediate_size, self.intermediate_size, self.attention_key_size],
axis=-1,
)
q, k = self.q_scaleoffset(qk), self.k_scaleoffset(qk)
# apply_rotary
q, k = self.rotary(q), self.rotary(k)
# Attention
a = paddle.matmul(q, k, transpose_y=True)
if self.attention_scale:
a = a / self.attention_key_size**0.5
if attention_mask is not None:
a = a * attention_mask + (attention_mask - 1) * INF
A = attention_normalize(a, attention_mask, axis=-1, method=self.normalization)
A = F.dropout(A, p=self.attention_dropout, training=self.training)
o = self.o_dense(u * paddle.matmul(A, v))
return o
class GAULayer(Layer):
def __init__(self, config: GAUAlphaConfig):
super().__init__()
self.gau = GatedAttentionUnit(config)
self.norm = Norm(config.norm_eps)
self.hidden_dropout = config.hidden_dropout_prob
def forward(self, hidden_states, attention_mask=None):
gau_output = self.gau(hidden_states, attention_mask=attention_mask)
# dropout and residual
o = F.dropout(gau_output[0], p=self.hidden_dropout, training=self.training)
o = self.norm(hidden_states + o)
return o
def initializer(tensor, num_hidden_layers=12, order=2, gain=1.0):
"""
https://github.com/bojone/bert4keras/blob/5572ed481a14f5a62be7107e3846c88a5d6b617d/bert4keras/models.py#L1226-L1235
"""
shape = tensor.shape
if shape[0] > 10000 or shape[0] < 10:
hidden_size = shape[1]
else:
hidden_size = shape[0]
gain *= num_hidden_layers ** (-1.0 / order)
std = 1.13684723 / hidden_size**0.5 * gain
return nn.initializer.TruncatedNormal(std=std)
class RotaryPositionEmbedding(Layer):
def __init__(self, config: GAUAlphaConfig):
super().__init__()
inv_freq = 1.0 / (
10000
** (
paddle.arange(0, config.attention_key_size, 2, dtype=paddle.get_default_dtype())
/ config.attention_key_size
)
)
t = paddle.arange(config.max_position_embeddings, dtype=paddle.get_default_dtype())
freqs = paddle.matmul(t.unsqueeze(1), inv_freq.unsqueeze(0))
self.register_buffer("sin", freqs.sin(), persistable=False)
self.register_buffer("cos", freqs.cos(), persistable=False)
def forward(self, x, offset=0):
# x shape [batch_size, seqlen, dim]
seqlen = x.shape[-2]
sin, cos = (
self.sin[offset : offset + seqlen, :],
self.cos[offset : offset + seqlen, :],
)
x1, x2 = x[..., 0::2], x[..., 1::2]
# [cos_nθ, -sin_nθ] [x1]
# [sin_nθ, cos_nθ] [x2]
# => [x1 * cos_nθ - x2 * sin_nθ, x1 * sin_nθ + x2 * cos_nθ]
return paddle.stack([x1 * cos - x2 * sin, x1 * sin + x2 * cos], axis=-1).flatten(-2, -1)
class GAUAlphaPretrainedModel(PretrainedModel):
"""
An abstract class for pretrained GAU-alpha models. It provides GAU-alpha related
`model_config_file`, `pretrained_init_configuration`, `resource_files_names`,
`pretrained_resource_files_map`, `base_model_prefix` for downloading and
loading pretrained models.
See :class:`~paddlenlp.transformers.model_utils.PretrainedModel` for more details.
"""
model_config_file = CONFIG_NAME
config_class = GAUAlphaConfig
resource_files_names = {"model_state": "model_state.pdparams"}
base_model_prefix = "gau_alpha"
pretrained_init_configuration = GAUAlPHA_PRETRAINED_INIT_CONFIGURATION
pretrained_resource_files_map = GAUAlPHA_PRETRAINED_RESOURCE_FILES_MAP
def _init_weights(self, layer):
"""Initialization hook"""
if isinstance(layer, (nn.Linear, nn.Embedding)):
# In the dygraph mode, use the `set_value` to reset the parameter directly,
# and reset the `state_dict` to update parameter in static mode.
if isinstance(layer.weight, paddle.Tensor):
initializer(layer.weight, self.config.num_hidden_layers, order=2, gain=1.0)
if isinstance(layer, nn.Linear):
use_bias = self.use_bias if hasattr(self, "use_bias") else self.gau_alpha.config["use_bias"]
if layer.bias is not None and not use_bias:
layer.bias = None
@register_base_model
class GAUAlphaModel(GAUAlphaPretrainedModel):
"""
The bare GAUAlpha Model transformer outputting raw hidden-states.
This model inherits from :class:`~paddlenlp.transformers.model_utils.PretrainedModel`.
Refer to the superclass documentation for the generic methods.
This model is also a Paddle `paddle.Layer <https://www.paddlepaddle.org.cn/documentation
/docs/zh/api/paddle/nn/Layer_cn.html>`__ subclass. Use it as a regular Paddle Layer
and refer to the Paddle documentation for all matter related to general usage and behavior.
Args:
vocab_size (int):
Vocabulary size of `inputs_ids` in `GAUAlphaModel`. Also is the vocab size of token embedding matrix.
Defines the number of different tokens that can be represented by the `inputs_ids` passed when calling `GAUAlphaModel`.
hidden_size (int, optional):
Dimensionality of the, encoder layers and pooler layer. Defaults to `768`.
intermediate_size (int, optional):
Dimensionality of the feed-forward (ff) layer in the encoder. Input tensors
to ff layers are firstly projected from `hidden_size` to `intermediate_size`,
and then projected back to `hidden_size`. Typically `intermediate_size` is larger than `hidden_size`.
Defaults to `3072`.
num_hidden_layers (int, optional):
Number of hidden layers in the gau_alpha encoder. Defaults to `12`.
max_position_embeddings (int, optional):
The maximum value of the dimensionality of position encoding, which dictates the maximum supported length of an input
sequence. Defaults to `512`.
type_vocab_size (int, optional):
The vocabulary size of `token_type_ids`.
Defaults to `2`.
attention_key_size (int, optional):
The dimensionality of the key used in the gau layer. Defaults to `128`.
norm_eps (float, optional):
The epsilon value used in the normalization layer.
Defaults to `1e-12`.
pad_token_id (int, optional):
The index of padding token in the token vocabulary.
Defaults to `0`.
hidden_dropout_prob (float, optional):
The dropout probability for all fully connected layers in the embeddings and encoder.
Defaults to `0.1`.
attention_probs_dropout_prob (float, optional):
The dropout probability used in gau in all encoder layers to drop some attention target.
Defaults to `0.1`.
hidden_act (str, optional):
The activation function used in gau layer. Defaults to `swish`.
use_bias (bool, optional):
Whether or not use bias.
Defaults to `False`.
normalization (str, optional):
The normalization method used in gau layer.
Defaults to `softmax_plus`.
attention_scale (bool, optional):
Whether or not to scale the attention scores.
Defaults to `True`.
"""
def __init__(self, config: GAUAlphaConfig):
super(GAUAlphaModel, self).__init__(config)
self.pad_token_id = config.pad_token_id
self.norm_eps = config.norm_eps
self.num_hidden_layers = config.num_hidden_layers
self.use_bias = config.use_bias
self.embeddings = GAUAlphaEmbeddings(config)
self.encoder = GAUAlphaEncoder(config)
def get_input_embeddings(self):
return self.embeddings.word_embeddings
def set_input_embeddings(self, value):
self.embeddings.word_embeddings = value
def forward(
self,
input_ids=None,
attention_mask=None,
token_type_ids=None,
):
r"""
The GAUAlphaModel forward method, overrides the `__call__()` special method.
Args:
input_ids (Tensor):
Indices of input sequence tokens in the vocabulary. They are
numerical representations of tokens that build the input sequence.
Its data type should be `int64` and it has a shape of [batch_size, sequence_length].
attention_mask (Tensor, optional):
Mask used in gau to avoid performing attention on to some unwanted positions,
usually the paddings or the subsequent positions.
Its data type can be int, float and bool.
When the data type is bool, the `masked` tokens have `False` values and the others have `True` values.
When the data type is int, the `masked` tokens have `0` values and the others have `1` values.
It is a tensor with shape broadcasted to `[batch_size, sequence_length, sequence_length]`.
Defaults to `None`, which means nothing needed to be prevented attention to.
token_type_ids (Tensor, optional):
Segment token indices to indicate different portions of the inputs.
Selected in the range ``[0, type_vocab_size - 1]``.
If `type_vocab_size` is 2, which means the inputs have two portions.
Indices can either be 0 or 1:
- 0 corresponds to a *sentence A* token,
- 1 corresponds to a *sentence B* token.
Its data type should be `int64` and it has a shape of [batch_size, sequence_length].
Defaults to `None`, which means we don't add segment embeddings.
Returns:
tuple: Returns `last_hidden_state` (Tensor)
Sequence of hidden-states at the last layer of the model.
It's data type should be float32 and its shape is [batch_size, sequence_length, hidden_size].
Example:
.. code-block::
import paddle
from paddlenlp.transformers import GAUAlphaModel, GAUAlphaTokenizer
tokenizer = GAUAlphaTokenizer.from_pretrained('chinese_GAU-alpha-char_L-24_H-768')
model = GAUAlphaModel.from_pretrained('chinese_GAU-alpha-char_L-24_H-768')
inputs = tokenizer("欢迎使用百度飞桨!")
inputs = {k:paddle.to_tensor([v], dtype="int64") for (k, v) in inputs.items()}
last_hidden_state = model(**inputs)
"""
if attention_mask is None:
attention_mask = input_ids != self.pad_token_id
if attention_mask.ndim == 2:
attention_mask = attention_mask.unsqueeze(1) # bs, 1, seqlen
attention_mask = attention_mask.astype(paddle.get_default_dtype())
attention_mask.stop_gradient = True
embedding_output = self.embeddings(
input_ids=input_ids,
token_type_ids=token_type_ids,
)
last_hidden_state = self.encoder(embedding_output, attention_mask=attention_mask)
return last_hidden_state
class GAUAlphaEmbeddings(Layer):
"""
Include embeddings from word and token_type embeddings
"""
def __init__(self, config: GAUAlphaConfig):
super(GAUAlphaEmbeddings, self).__init__()
self.word_embeddings = nn.Embedding(config.vocab_size, config.hidden_size)
self.token_type_embeddings = nn.Embedding(config.type_vocab_size, config.hidden_size)
self.norm = Norm(config.norm_eps)
self.dropout = nn.Dropout(config.hidden_dropout_prob)
def forward(self, input_ids, token_type_ids=None):
if token_type_ids is None:
token_type_ids = paddle.zeros_like(input_ids, dtype="int64")
input_embedings = self.word_embeddings(input_ids)
token_type_embeddings = self.token_type_embeddings(token_type_ids)
embeddings = input_embedings + token_type_embeddings
embeddings = self.norm(embeddings)
embeddings = self.dropout(embeddings)
return embeddings
class GAUAlphaEncoder(Layer):
def __init__(self, config: GAUAlphaConfig):
super().__init__()
self.layer = nn.LayerList([GAULayer(config) for _ in range(config.num_hidden_layers)])
def forward(self, hidden_states, attention_mask=None):
for layer_module in self.layer:
hidden_states = layer_module(
hidden_states,
attention_mask,
)
return hidden_states
class GAUAlphaForQuestionAnswering(GAUAlphaPretrainedModel):
"""
GAUAlpha with a linear layer on top of the hidden-states output to compute `span_start_logits`
and `span_end_logits`, designed for question-answering tasks like SQuAD.
Args:
gau_alpha (:class:`GAUAlphaModel`):
An instance of GAUAlphaModel.
dropout (float, optional):
The dropout probability for output of GAUAlpha.
If None, use the same value as `hidden_dropout_prob` of `GAUAlphaModel`
instance `gau_alpha`. Defaults to `None`.
"""
def __init__(self, config: GAUAlphaConfig):
super(GAUAlphaForQuestionAnswering, self).__init__(config)
self.gau_alpha = GAUAlphaModel(config)
self.dropout = nn.Dropout(
config.classifier_dropout if config.classifier_dropout is not None else config.hidden_dropout_prob
)
self.classifier = nn.Linear(config.hidden_size, 2)
def forward(self, input_ids, token_type_ids=None, attention_mask=None):
r"""
The GAUAlphaForQuestionAnswering forward method, overrides the __call__() special method.
Args:
input_ids (Tensor):
See :class:`GAUAlphaModel`.
token_type_ids (Tensor, optional):
See :class:`GAUAlphaModel`.
attention_mask (Tensor, optional):
See :class:`GAUAlphaModel`.
Returns:
tuple: Returns tuple (`start_logits`, `end_logits`).
With the fields:
- `start_logits` (Tensor):
A tensor of the input token classification logits, indicates the start position of the labelled span.
Its data type should be float32 and its shape is [batch_size, sequence_length].
- `end_logits` (Tensor):
A tensor of the input token classification logits, indicates the end position of the labelled span.
Its data type should be float32 and its shape is [batch_size, sequence_length].
Example:
.. code-block::
import paddle
from paddlenlp.transformers import GAUAlphaForQuestionAnswering, GAUAlphaTokenizer
tokenizer = GAUAlphaTokenizer.from_pretrained('chinese_GAU-alpha-char_L-24_H-768')
model = GAUAlphaForQuestionAnswering.from_pretrained('chinese_GAU-alpha-char_L-24_H-768')
inputs = tokenizer("欢迎使用百度飞桨!")
inputs = {k:paddle.to_tensor([v], dtype="int64") for (k, v) in inputs.items()}
outputs = model(**inputs)
start_logits = outputs[0]
end_logits = outputs[1]
"""
sequence_output = self.gau_alpha(input_ids, token_type_ids=token_type_ids, attention_mask=attention_mask)
logits = self.classifier(sequence_output)
start_logits, end_logits = paddle.unstack(logits, axis=-1)
return start_logits, end_logits
class GAUAlphaForSequenceClassification(GAUAlphaPretrainedModel):
"""
GAUAlpha Model with a linear layer on top of the output layer,
designed for sequence classification/regression tasks like GLUE tasks.
Args:
gau_alpha (`GAUAlphaModel`):
An instance of `paddlenlp.transformers.GAUAlphaModel`.
num_labels (int, optional):
The number of classes. Default to `2`.
dropout (float, optional):
The dropout probability for output of GAUAlpha.
If None, use the same value as `hidden_dropout_prob`
of `paddlenlp.transformers.GAUAlphaModel` instance. Defaults to `None`.
"""
def __init__(self, config: GAUAlphaConfig):
super(GAUAlphaForSequenceClassification, self).__init__(config)
self.num_labels = config.num_labels
self.gau_alpha = GAUAlphaModel(config)
self.dropout = nn.Dropout(
config.classifier_dropout if config.classifier_dropout is not None else config.hidden_dropout_prob
)
self.classifier = nn.Linear(config.hidden_size, config.num_labels)
def forward(self, input_ids, token_type_ids=None, attention_mask=None):
r"""
Args:
input_ids (Tensor):
See :class:`GAUAlphaModel`.
token_type_ids (Tensor, optional):
See :class:`GAUAlphaModel`.
attention_mask (Tensor, optional):
See :class:`GAUAlphaModel`.
Returns:
Tensor: Returns tensor `logits`, a tensor of the input text classification logits.
Shape as `[batch_size, num_labels]` and dtype as float32.
Example:
.. code-block::
import paddle
from paddlenlp.transformers import GAUAlphaForSequenceClassification, GAUAlphaTokenizer
tokenizer = GAUAlphaTokenizer.from_pretrained('chinese_GAU-alpha-char_L-24_H-768')
model = GAUAlphaForSequenceClassification.from_pretrained('chinese_GAU-alpha-char_L-24_H-768')
inputs = tokenizer("欢迎使用百度飞桨!")
inputs = {k:paddle.to_tensor([v], dtype="int64") for (k, v) in inputs.items()}
logits = model(**inputs)
"""
sequence_output = self.gau_alpha(input_ids, token_type_ids=token_type_ids, attention_mask=attention_mask)
pooled_output = sequence_output[:, 0]
pooled_output = self.dropout(pooled_output)
logits = self.classifier(pooled_output)
return logits
class GAUAlphaForTokenClassification(GAUAlphaPretrainedModel):
"""
GAUAlpha Model with a linear layer on top of the hidden-states output layer,
designed for token classification tasks like NER tasks.
Args:
gau_alpha (`GAUAlphaModel`):
An instance of `paddlenlp.transformers.GAUAlphaModel`.
num_labels (int, optional):
The number of classes. Default to `2`.
dropout (float, optional):
The dropout probability for output of GAUAlpha.
If None, use the same value as `hidden_dropout_prob`
of `paddlenlp.transformers.GAUAlphaModel` instance. Defaults to `None`.
"""
def __init__(self, config: GAUAlphaConfig):
super(GAUAlphaForTokenClassification, self).__init__(config)
self.num_labels = config.num_labels
self.gau_alpha = GAUAlphaModel(config) # allow gau_alpha to be config
self.dropout = nn.Dropout(
config.classifier_dropout if config.classifier_dropout is not None else config.hidden_dropout_prob
)
self.classifier = nn.Linear(config.hidden_size, config.num_labels)
def forward(self, input_ids, token_type_ids=None, attention_mask=None):
r"""
Args:
input_ids (Tensor):
See :class:`GAUAlphaModel`.
token_type_ids (Tensor, optional):
See :class:`GAUAlphaModel`.
attention_mask (Tensor, optional):
See :class:`GAUAlphaModel`.
Returns:
Tensor: Returns tensor `logits`, a tensor of the input token classification logits.
Shape as `[batch_size, sequence_length, num_labels]` and dtype as `float32`.
Example:
.. code-block::
import paddle
from paddlenlp.transformers import GAUAlphaForTokenClassification, GAUAlphaTokenizer
tokenizer = GAUAlphaTokenizer.from_pretrained('chinese_GAU-alpha-char_L-24_H-768')
model = GAUAlphaForTokenClassification.from_pretrained('chinese_GAU-alpha-char_L-24_H-768')
inputs = tokenizer("欢迎使用百度飞桨!")
inputs = {k:paddle.to_tensor([v], dtype="int64") for (k, v) in inputs.items()}
logits = model(**inputs)
"""
sequence_output = self.gau_alpha(input_ids, token_type_ids=token_type_ids, attention_mask=attention_mask)
sequence_output = self.dropout(sequence_output)
logits = self.classifier(sequence_output)
return logits
class GAUAlphaForMultipleChoice(GAUAlphaPretrainedModel):
"""
GAUAlpha Model with a linear layer on top of the hidden-states output layer,
designed for multiple choice tasks like RocStories/SWAG tasks.
Args:
gau_alpha (:class:`GAUAlphaModel`):
An instance of GAUAlphaModel.
num_choices (int, optional):
The number of choices. Defaults to `2`.
dropout (float, optional):
The dropout probability for output of GAUAlpha.
If None, use the same value as `hidden_dropout_prob` of `GAUAlphaModel`
instance `gau_alpha`. Defaults to None.
"""
def __init__(self, config: GAUAlphaConfig):
super(GAUAlphaForMultipleChoice, self).__init__(config)
self.num_choices = config.num_choices
self.gau_alpha = GAUAlphaModel(config)
self.dropout = nn.Dropout(
config.classifier_dropout if config.classifier_dropout is not None else config.hidden_dropout_prob
)
self.classifier = nn.Linear(config.hidden_size, 1)
def forward(self, input_ids, token_type_ids=None, attention_mask=None):
r"""
The GAUAlphaForMultipleChoice forward method, overrides the __call__() special method.
Args:
input_ids (Tensor):
See :class:`GAUAlphaModel` and shape as [batch_size, num_choice, sequence_length].
token_type_ids(Tensor, optional):
See :class:`GAUAlphaModel` and shape as [batch_size, num_choice, sequence_length].
attention_mask (list, optional):
See :class:`GAUAlphaModel` and shape as [batch_size, num_choice, sequence_length].
Returns:
Tensor: Returns tensor `reshaped_logits`, a tensor of the multiple choice classification logits.
Shape as `[batch_size, num_choice]` and dtype as `float32`.
Example:
.. code-block::
import paddle
from paddlenlp.transformers import GAUAlphaForMultipleChoice, GAUAlphaTokenizer
from paddlenlp.data import Pad
tokenizer = GAUAlphaTokenizer.from_pretrained('chinese_GAU-alpha-char_L-24_H-768')
model = GAUAlphaForMultipleChoice.from_pretrained('chinese_GAU-alpha-char_L-24_H-768', num_choices=2)
data = [
{
"question": "如何打开ipad屏幕？",
"answer1": "按音量按钮。",
"answer2": "按下锁定按钮。",
"label": 1,
},
{
"question": "如何缩进一些文本？",
"answer1": "在开始写之前留一些空格。",
"answer2": "按空格键。",
"label": 0,
},
]
text = []
text_pair = []
for d in data:
text.append(d["question"])
text_pair.append(d["answer1"])
text.append(d["question"])
text_pair.append(d["answer2"])
inputs = tokenizer(text, text_pair)
input_ids = Pad(axis=0, pad_val=tokenizer.pad_token_id)(inputs["input_ids"])
token_type_ids = Pad(axis=0, pad_val=tokenizer.pad_token_type_id)(inputs["token_type_ids"])
reshaped_logits = model(
input_ids=paddle.to_tensor(input_ids, dtype="int64"),
token_type_ids=paddle.to_tensor(token_type_ids, dtype="int64"),
)
print(reshaped_logits.shape)
# [2, 2]
"""
# input_ids: [bs, num_choice, seq_l]
input_ids = input_ids.reshape(shape=(-1, input_ids.shape[-1])) # flat_input_ids: [bs*num_choice,seq_l]
if token_type_ids is not None:
token_type_ids = token_type_ids.reshape(shape=(-1, token_type_ids.shape[-1]))
if attention_mask is not None:
attention_mask = attention_mask.reshape(shape=(-1, attention_mask.shape[-1]))
sequence_output = self.gau_alpha(input_ids, token_type_ids=token_type_ids, attention_mask=attention_mask)
pooled_output = sequence_output[:, 0]
pooled_output = self.dropout(pooled_output)
logits = self.classifier(pooled_output) # logits: (bs*num_choice,1)
reshaped_logits = logits.reshape(shape=(-1, self.num_choices)) # logits: (bs, num_choice)
return reshaped_logits
class GAUAlphaLMPredictionHead(Layer):
def __init__(
self,
config: GAUAlphaConfig,
embedding_weights=None,
):
super(GAUAlphaLMPredictionHead, self).__init__()
self.use_bias = config.use_bias
self.decoder_weight = (
self.create_parameter(shape=[config.vocab_size, config.hidden_size], dtype=self.transform.weight.dtype)
if embedding_weights is None
else embedding_weights
)
if self.use_bias:
self.decoder_bias = self.create_parameter(
shape=[config.vocab_size], dtype=self.decoder_weight.dtype, is_bias=True
)
def forward(self, hidden_states):
hidden_states = paddle.matmul(hidden_states, self.decoder_weight, transpose_y=True)
if self.use_bias:
hidden_states = hidden_states + self.decoder_bias
return hidden_states
class GAUAlphaForMaskedLM(GAUAlphaPretrainedModel):
"""
GAUAlpha Model with a `masked language modeling` head on top.
Args:
gau_alpha (:class:`GAUAlphaModel`):
An instance of :class:`GAUAlphaModel`.
"""
def __init__(
self,
config: GAUAlphaConfig,
):
super(GAUAlphaForMaskedLM, self).__init__(config)
self.gau_alpha = GAUAlphaModel(config)
self.cls = GAUAlphaLMPredictionHead(
config=config,
embedding_weights=self.gau_alpha.embeddings.word_embeddings.weight,
)
def forward(self, input_ids, token_type_ids=None, attention_mask=None):
r"""
Args:
input_ids (Tensor):
See :class:`GAUAlphaModel`.
token_type_ids (Tensor, optional):
See :class:`GAUAlphaModel`.
attention_mask (Tensor, optional):
See :class:`GAUAlphaModel`.
Returns:
Tensor: Returns tensor `prediction_scores`, The scores of masked token prediction.
Its data type should be float32 and shape is [batch_size, sequence_length, vocab_size].
Example:
.. code-block::
import paddle
from paddlenlp.transformers import GAUAlphaForMaskedLM, GAUAlphaTokenizer
tokenizer = GAUAlphaTokenizer.from_pretrained('chinese_GAU-alpha-char_L-24_H-768')
model = GAUAlphaForMaskedLM.from_pretrained('chinese_GAU-alpha-char_L-24_H-768')
inputs = tokenizer("欢迎使用百度飞桨!")
inputs = {k:paddle.to_tensor([v], dtype="int64") for (k, v) in inputs.items()}
logits = model(**inputs)
print(logits.shape)
# [1, 11, 12000]
"""
sequence_output = self.gau_alpha(input_ids, token_type_ids=token_type_ids, attention_mask=attention_mask)
prediction_scores = self.cls(sequence_output)
return prediction_scores

292
paddlenlp/transformers/gau_alpha/tokenizer.py
View File

@@ -1,292 +0,0 @@
# Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
# Copyright 2018 The Google AI Language Team Authors and The HuggingFace Inc. team.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import os
from ..bert.tokenizer import BasicTokenizer, WordpieceTokenizer
from ..tokenizer_utils import PretrainedTokenizer
__all__ = ["GAUAlphaTokenizer"]
PRETRAINED_POSITIONAL_EMBEDDINGS_SIZES = {"chinese_GAU-alpha-char_L-24_H-768": 512}
class GAUAlphaTokenizer(PretrainedTokenizer):
"""
Constructs a GAUAlpha tokenizer. It uses a basic tokenizer to do punctuation
splitting, lower casing and so on, and follows a WordPiece tokenizer to
tokenize as subwords.
This tokenizer inherits from :class:`~paddlenlp.transformers.tokenizer_utils.PretrainedTokenizer`
which contains most of the main methods. For more information regarding those methods,
please refer to this superclass.
Args:
vocab_file (str):
The vocabulary file path (ends with '.txt') required to instantiate
a `WordpieceTokenizer`.
do_lower_case (bool,optional):
Whether or not to lowercase the input when tokenizing.
If you use the GAUAlpha pretrained model, lower is set to
False when using the cased model, otherwise it is set to True.
Defaults to`True`.
unk_token (str,optional):
A special token representing the *unknown (out-of-vocabulary)* token.
An unknown token is set to be `unk_token` inorder to be converted to an ID.
Defaults to "[UNK]".
sep_token (str,optional):
A special token separating two different sentences in the same input.
Defaults to "[SEP]".
pad_token (str,optional):
A special token used to make arrays of tokens the same size for batching purposes.
Defaults to "[PAD]".
cls_token (str,optional):
A special token used for sequence classification. It is the last token
of the sequence when built with special tokens. Defaults to "[CLS]".
mask_token (str,optional):
A special token representing a masked token. This is the token used
in the masked language modeling task which the model tries to predict the original unmasked ones.
Defaults to "[MASK]".
Examples:
.. code-block::
from paddlenlp.transformers import GAUAlphaTokenizer
tokenizer = GAUAlphaTokenizer.from_pretrained('chinese_GAU-alpha-char_L-24_H-768')
tokens = tokenizer('欢迎使用百度飞桨')
'''
{'input_ids': [101, 3223, 6500, 421, 4179, 4331, 2008, 7263, 3055, 102],
'token_type_ids': [0, 0, 0, 0, 0, 0, 0, 0, 0, 0]}
'''
"""
resource_files_names = {"vocab_file": "vocab.txt"} # for save_pretrained
pretrained_resource_files_map = {
"vocab_file": {
"chinese_GAU-alpha-char_L-24_H-768": "https://bj.bcebos.com/paddlenlp/models/transformers/gau_alpha/chinese_GAU-alpha-char_L-24_H-768/vocab.txt",
}
}
pretrained_init_configuration = {
"chinese_GAU-alpha-char_L-24_H-768": {"do_lower_case": True},
}
padding_side = "right"
max_model_input_sizes = PRETRAINED_POSITIONAL_EMBEDDINGS_SIZES
def __init__(
self,
vocab_file,
do_lower_case=True,
unk_token="[UNK]",
sep_token="[SEP]",
pad_token="[PAD]",
cls_token="[CLS]",
mask_token="[MASK]",
**kwargs
):
if not os.path.isfile(vocab_file):
raise ValueError(
"Can't find a vocabulary file at path '{}'. To load the "
"vocabulary from a pretrained model please use "
"`tokenizer = GAUAlphaTokenizer.from_pretrained(PRETRAINED_MODEL_NAME)`".format(vocab_file)
)
self.do_lower_case = do_lower_case
self.vocab = self.load_vocabulary(vocab_file, unk_token=unk_token)
self.basic_tokenizer = BasicTokenizer(do_lower_case=do_lower_case)
self.wordpiece_tokenizer = WordpieceTokenizer(vocab=self.vocab, unk_token=unk_token)
@property
def vocab_size(self):
"""
Return the size of vocabulary.
Returns:
int: The size of vocabulary.
"""
return len(self.vocab)
def _tokenize(self, text):
"""
End-to-end tokenization for GAUAlpha models.
Args:
text (str): The text to be tokenized.
Returns:
list: A list of string representing converted tokens.
"""
split_tokens = []
for token in self.basic_tokenizer.tokenize(text):
for sub_token in self.wordpiece_tokenizer.tokenize(token):
split_tokens.append(sub_token)
return split_tokens
def convert_tokens_to_string(self, tokens):
"""
Converts a sequence of tokens (list of string) to a single string. Since
the usage of WordPiece introducing `##` to concat subwords, also removes
`##` when converting.
Args:
tokens (list): A list of string representing tokens to be converted.
Returns:
str: Converted string from tokens.
Examples:
.. code-block::
from paddlenlp.transformers import GAUAlphaTokenizer
tokenizer = GAUAlphaTokenizer.from_pretrained('chinese_GAU-alpha-char_L-24_H-768')
tokens = tokenizer.tokenize('欢迎使用百度飞桨!')
'''
['欢', '迎', '使', '用', '百', '度', '飞', '桨', '!']
'''
strings = tokenizer.convert_tokens_to_string(tokens)
'''
'欢 迎 使 用 百 度 飞 桨 !'
'''
"""
out_string = " ".join(tokens).replace(" ##", "").strip()
return out_string
def num_special_tokens_to_add(self, pair=False):
"""
Returns the number of added tokens when encoding a sequence with special tokens.
Args:
pair(bool):
Whether the input is a sequence pair or a single sequence.
Defaults to `False` and the input is a single sequence.
Returns:
int: Number of tokens added to sequences.
"""
token_ids_0 = []
token_ids_1 = []
return len(self.build_inputs_with_special_tokens(token_ids_0, token_ids_1 if pair else None))
def build_inputs_with_special_tokens(self, token_ids_0, token_ids_1=None):
"""
Build model inputs from a sequence or a pair of sequence for sequence classification tasks by concatenating and
adding special tokens.
A GAUAlpha sequence has the following format:
- single sequence: ``[CLS] X [SEP]``
- pair of sequences: ``[CLS] A [SEP] B [SEP]``
Args:
token_ids_0 (List[int]):
List of IDs to which the special tokens will be added.
token_ids_1 (List[int], optional):
Optional second list of IDs for sequence pairs. Defaults to None.
Returns:
List[int]: List of input_id with the appropriate special tokens.
"""
if token_ids_1 is None:
return [self.cls_token_id] + token_ids_0 + [self.sep_token_id]
_cls = [self.cls_token_id]
_sep = [self.sep_token_id]
return _cls + token_ids_0 + _sep + token_ids_1 + _sep
def build_offset_mapping_with_special_tokens(self, offset_mapping_0, offset_mapping_1=None):
"""
Build offset map from a pair of offset map by concatenating and adding offsets of special tokens.
A GAUAlpha offset_mapping has the following format:
- single sequence: ``(0,0) X (0,0)``
- pair of sequences: ``(0,0) A (0,0) B (0,0)``
Args:
offset_mapping_ids_0 (List[tuple]):
List of wordpiece offsets to which the special tokens will be added.
offset_mapping_ids_1 (List[tuple], optional):
Optional second list of wordpiece offsets for offset mapping pairs. Defaults to None.
Returns:
List[tuple]: A list of wordpiece offsets with the appropriate offsets of special tokens.
"""
if offset_mapping_1 is None:
return [(0, 0)] + offset_mapping_0 + [(0, 0)]
return [(0, 0)] + offset_mapping_0 + [(0, 0)] + offset_mapping_1 + [(0, 0)]
def create_token_type_ids_from_sequences(self, token_ids_0, token_ids_1=None):
"""
Create a mask from the two sequences passed to be used in a sequence-pair classification task.
A GAUAlpha sequence pair mask has the following format:
::
0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1
| first sequence | second sequence |
If `token_ids_1` is `None`, this method only returns the first portion of the mask (0s).
Args:
token_ids_0 (List[int]):
A list of `inputs_ids` for the first sequence.
token_ids_1 (List[int], optional):
Optional second list of IDs for sequence pairs. Defaults to None.
Returns:
List[int]: List of token_type_id according to the given sequence(s).
"""
_sep = [self.sep_token_id]
_cls = [self.cls_token_id]
if token_ids_1 is None:
return len(_cls + token_ids_0 + _sep) * [0]
return len(_cls + token_ids_0 + _sep) * [0] + len(token_ids_1 + _sep) * [1]
def get_special_tokens_mask(self, token_ids_0, token_ids_1=None, already_has_special_tokens=False):
"""
Retrieves sequence ids from a token list that has no special tokens added. This method is called when adding
special tokens using the tokenizer ``encode`` methods.
Args:
token_ids_0 (List[int]):
A list of `inputs_ids` for the first sequence.
token_ids_1 (List[int], optional):
Optional second list of IDs for sequence pairs. Defaults to None.
already_has_special_tokens (bool, optional): Whether or not the token list is already
formatted with special tokens for the model. Defaults to None.
Returns:
List[int]: The list of integers either be 0 or 1: 1 for a special token, 0 for a sequence token.
"""
if already_has_special_tokens:
if token_ids_1 is not None:
raise ValueError(
"You should not supply a second sequence if the provided sequence of "
"ids is already formatted with special tokens for the model."
)
return list(map(lambda x: 1 if x in [self.sep_token_id, self.cls_token_id] else 0, token_ids_0))
if token_ids_1 is not None:
return [1] + ([0] * len(token_ids_0)) + [1] + ([0] * len(token_ids_1)) + [1]
return [1] + ([0] * len(token_ids_0)) + [1]
def get_vocab(self):
return dict(self.vocab.token_to_idx, **self.added_tokens_encoder)

1
pyproject.toml
View File

@@ -16,7 +16,6 @@ testpaths = [
"tests/data",
"tests/dataaug",
"tests/datasets",
"tests/embeddings",
"tests/experimental",
"tests/generation",
"tests/layers",

2
slm/applications/question_answering/unsupervised_qa/README.md
View File

@@ -28,7 +28,7 @@
## 简介
问答（QA）系统中最关键的挑战之一是标记数据的稀缺性，这是因为对目标领域获取问答对或常见问答对（FAQ）的成本很高，需要消耗大量的人力和时间。由于上述制约，这导致检索式问答系统落地困难，解决此问题的一种方法是依据问题上下文或大量非结构化文本自动生成的 QA 问答对。
在此背景下，无监督检索式问答系统（即问答对自动生成智能检索式问答），基于 PaddleNLP[问题生成](../../../examples/question_generation/README.md)、[UIE](../../../model_zoo/uie/README.md)、[检索式问答](https://github.com/PaddlePaddle/PaddleNLP/blob/release/2.8/applications/question_answering/supervised_qa/faq_finance/README.md)，支持以非结构化文本形式为上下文自动生成 QA 问答对，生成的问答对语料可以通过无监督的方式构建检索式问答系统。
在此背景下，无监督检索式问答系统（即问答对自动生成智能检索式问答），基于 PaddleNLP[问题生成](https://github.com/PaddlePaddle/PaddleNLP/tree/release/2.8/examples/question_generation/README.md)、[UIE](../../../model_zoo/uie/README.md)、[检索式问答](https://github.com/PaddlePaddle/PaddleNLP/blob/release/2.8/applications/question_answering/supervised_qa/faq_finance/README.md)，支持以非结构化文本形式为上下文自动生成 QA 问答对，生成的问答对语料可以通过无监督的方式构建检索式问答系统。
若开发者已有 FAQ 语料，请参考[supervised_qa](https://github.com/PaddlePaddle/PaddleNLP/blob/release/2.8/applications/question_answering/supervised_qa)。

10
slm/examples/README.md
View File

@@ -12,21 +12,21 @@ PaddleNLP provides rich application examples covering mainstream NLP task to hel
| lexical_analysis | [词法分析 (Lexical Analysis)](./lexical_analysis/) |
| dependency_parsing | [句法依存分析 (Dependency Parsing)](https://github.com/PaddlePaddle/PaddleNLP/tree/release/2.8/examples/dependency_parsing/ddparser) |
| text_to_sql | [语义解析 (Semantic Parsing/Text to SQL)](https://github.com/PaddlePaddle/PaddleNLP/tree/release/2.8/examples/text_to_sql):star: |
| text_classification | [文本分类 (Text Classification)](./text_classification/) |
| text_classification | [文本分类 (Text Classification)](https://github.com/PaddlePaddle/PaddleNLP/tree/develop/slm/applications/text_classification) |
| text_matching | [文本匹配 (Text Matching)](./text_matching/) |
| text_generation | [文本生成 (Text Generation)](./text_generation/) |
| text_generation | [文本生成 (Text Generation)](https://github.com/PaddlePaddle/PaddleNLP/tree/release/2.8/examples/text_generation/) |
| text_summarization | [文本摘要 (Text Summarization)](./text_summarization/) |
| text_correction | [文本纠错 (Text Correction)](./text_correction/):star: |
| text_correction | [文本纠错 (Text Correction)](https://github.com/PaddlePaddle/PaddleNLP/tree/release/2.8/examples/text_correction/):star: |
| semantic_indexing | [语义索引 (Semantic Indexing)](./semantic_indexing/) |
| information_extraction | [信息抽取 (Information Extraction)](./information_extraction/) |
| question_generation | [问题生成 (Question Generation)](./question_generation/) |
| question_generation | [问题生成 (Question Generation)](https://github.com/PaddlePaddle/PaddleNLP/tree/release/2.8/examples/question_generation/) |
## NLP 系统应用 (NLP System Applications)
| 目录 Folder | 任务 Task |
|:------------------------------|------------------------------------------------------------------------------|
| sentiment_analysis | [情感分析 (Sentiment Analysis)](./sentiment_analysis/):star2: |
| dialogue | [通用对话 (General Dialogue System)](./dialogue/) |
| dialogue | [通用对话 (General Dialogue System)](https://github.com/PaddlePaddle/PaddleNLP/tree/release/2.8/examples/dialogue/) |
| machine_translation | [文本翻译 (Machine Translation)](./machine_translation/) |
| simultaneous_translation | [同声翻译 (Simultaneous Translation)](./simultaneous_translation/) |
| machine_reading_comprehension | [阅读理解 (Machine Reading Comprehension)](./machine_reading_comprehension/) |

241
slm/examples/RLHF/README.md
View File

@@ -1,241 +0,0 @@
# RLHF PPO
提供了基于强化学习 PPO 算法对 LLM 进行人类偏好对齐的代码及完整使用示例，支持**3D 分布式并行训练以及 rollout 阶段使用预测优化进行生成加速**。其中 PPO 代码实现细节参考了 [PKU-Alignment/safe-rlhf](https://github.com/PKU-Alignment/safe-rlhf)（PKU Beaver） 中的 PPO 实现，支持 reward normalization、pretraining loss 等常用的 PPO 稳定训练策略；示例使用 PKU-Alignment/safe-rlhf 提供的部分数据集和模型。后续将持续完善扩展，支持更好效果、更低成本、更高性能、更大规模的 RLHF 能力。
## 快速开始
项目整体组织结构如下：
```
.
├── reward_main.py # reward model训练脚本
├── reward_config.json # reward model训练配置文件
├── reward_trainer.py # reward训练执行器py脚本
├── ppo_main.py # RLHF训练脚本
├── ppo_config.json # RLHF训练配置文件
├── ppo_trainer.py # RLHF训练执行器py脚本
├── ppo_config.json # RLHF训练配置文件
├── trainer_utils.py # Trainer补丁及工具py脚本
├── infer_utils.py # 生成加速工具py脚本
├── data # 数据集相关目录
│ └── base.py # 数据集基类及工具py文件
│ └── alpaca.py # alpaca(raw)数据集py文件
│ └── safe_rlhf.py # safe_rlhf(raw)数据集py文件
│ └── preference.py # 偏好数据集py文件
│ └── prompt_only.py # prompt only数据集py文件
│ └── supervised.py # supervised数据集py文件
├── models # 模型相关目录
│ └── score_model_utils.py # score model基类及工具py文件
│ └── score_model.py # score model模型定义py文件
│ └── ppo_model_utils.py # PPO loss等模型策略py文件
│ └── pp_model_utils.py # 流水线并行补丁及工具py文件
│ └── model_pp.py # 流水线并行模型py文件
│ └── infer_model_utils.py # 预测加速模型补丁及工具py文件
└── README.md
```
### 环境准备
- Python >= 3.10
- PaddlePaddle >= 2.6.0
- PaddleNLP 最新版本
如需使用生成加速功能，需要安装 [paddlenlp_ops](https://github.com/PaddlePaddle/PaddleNLP/tree/develop/csrc) ，请使用 `git clone https://github.com/PaddlePaddle/PaddleNLP.git` 克隆 PaddleNLP 代码库并且将 PaddleNLP/llm 目录的路径加入 PYTHONPATH（后续将进行完善）。安装 paddlenlp_ops 后训练时将直接开启生成加速（开启流水线并行时不支持生成加速），否则使用原生动态图进行生成。
### 数据准备
PPO 训练包括 Supervised Fine-Tuning、Reward Model Fine-Tuning、RLHF 三个阶段（可见下文训练部分），会涉及到多个数据集，下面分别介绍并给出自定义数据的方法。
#### Supervised Fine-Tuning 数据
同[LLM 精调](https://github.com/PaddlePaddle/PaddleNLP/tree/develop/llm#2-%E7%B2%BE%E8%B0%83)，可以直接参考对应内容进行数据准备。
#### Reward Model Fine-Tuning 数据
Reward Model Fine-Tuning 阶段需要使用人类偏好数据。示例使用 PKU-Alignment/safe-rlhf 提供的 [PKU-Alignment/PKU-SafeRLHF-30K](https://huggingface.co/datasets/PKU-Alignment/PKU-SafeRLHF-30K) 数据集，下面是其中一条样本，这里使用其中的`prompt、response_0、response_1、better_response_id`字段来组织偏好数据（safe 字段，该数据集将 helpful 和 harmless 分开标注，示例这里使用其 helpful 标注）。
```json
{
"prompt": "Is it possible to purchase a complete chess set at a pawn shop?",
"response_0": "Yes, it is possible to purchase a complete chess set at a pawn shop.",
"response_1": "Yes, it is possible to purchase a complete chess set at a pawn shop. Pawn shops may carry chess sets that have been either inherited or previously borrowed. There is also a possibility of finding individual chess pieces such as rooks, knights, bishops, queens, and kings. If you cannot find a complete set, you may be able to purchase the various chess pieces separately.",
"is_response_0_safe": true,
"is_response_1_safe": true,
"better_response_id": 1,
"safer_response_id": 1
}
```
#### RLHF 数据
RLHF 阶段使用 prompt only 数据，另外可以可选的提供额外的监督数据用于构建 LM 损失约束 RLHF 训练。示例使用 [PKU-Alignment/PKU-SafeRLHF](https://huggingface.co/datasets/PKU-Alignment/PKU-SafeRLHF) 数据集（同样是人类偏好数据集，这里只使用其 prompt 字段并对 prompt 去重）。此外还使用了 [tatsu-lab/alpaca](https://huggingface.co/datasets/tatsu-lab/alpaca)的数据来构建额外的损失项。
以上示例数据集在训练时将自动下载缓存使用。
#### 自定义数据
数据定义围绕 `RawSample` 和 `RawDataset` 两个预置的类进行；`RawSample` 提供了数据样本级别接入协议规范，`RawDataset` 提供了数据集级别接入协议规范；按照下面介绍的规范接入，即可通过预置的 `SupervisedDataset`、`PreferenceDataset`、`PromptOnlyDataset` 三类 RLHF 训练所需数据接口来使用自定义数据。
自定义数据集需要：
- 继承 `RawDataset` ，并定义类属性 `NAME` 用于注册数据集。
- 实现 `__init__` 方法（加载数据），`__getitem__` 方法（根据 index 获取样本并转换为 `RawSample` 对象返回）、`__len__` 方法（数据集大小）。
示例如下：
```python
from datasets import load_dataset
from data import RawDataset, RawSample
class MyRawDataset(RawDataset):
NAME = 'my-dataset-name'
def __init__(self, path=None) -> None:
# Load a dataset from Hugging Face or any other data source
# self.data = load_dataset(path or 'my-organization/my-dataset')['train']
self.data = [{
'col1': 'question',
'col2': 'answer1',
'col3': 'answer2',
'col4': 1, # score of answer1
'col5': 2 # score of answer2
}] * 10 # dummy data for example
def __getitem__(self, index: int) -> RawSample:
data = self.data[index]
# Construct a `RawSample` dictionary from your custom dataset item
return RawSample(
input=data['col1'],
answer=data['col2'],
other_answer=data['col3'],
better=float(data['col4']) > float(data['col5']),
)
def __len__(self) -> int:
return len(self.data) # dataset size
```
其中 `RawSample` 是整个 RLHF 训练过程用到的几种数据类型的超集，如下所示，其可以桥接各训练阶段所需样本类型。在自定义数据时，对于 SFT 数据使用 `RawSample` 的 `(input, answer)` 字段；对于人类偏好数据使用 `RawSample` 的 `(input, answer, other_answer, better)` 字段；对于 prompt only 数据，使用 `RawSample` 的 `(input)`字段。
```python
class RawSample(TypedDict, total=False):
"""Raw sample type.
For SupervisedDataset, should provide (input, answer) or (dialogue).
For PreferenceDataset, should provide (input, answer, other_answer, better).
For SafetyPreferenceDataset, should provide (input, answer, other_answer, safer, is_safe, is_other_safe).
For PromptOnlyDataset, should provide (input).
When input is a list, it would be processed as a dialogue.
"""
# Texts
input: NotRequired[str | list[str]] # either `input` or `dialogue` should be provided
"""User input text."""
answer: NotRequired[str]
"""Assistant answer text."""
other_answer: NotRequired[str]
"""Other assistant answer text via resampling."""
dialogue: NotRequired[list[str]] # either `input` or `dialogue` should be provided
"""Dialogue history."""
# Flags
better: NotRequired[bool]
"""Whether ``answer`` is better than ``other_answer``."""
safer: NotRequired[bool]
"""Whether ``answer`` is safer than ``other_answer``."""
is_safe: NotRequired[bool]
"""Whether ``answer`` is safe."""
is_other_safe: NotRequired[bool]
"""Whether ``other_answer`` is safe."""
```
如此定义的数据集将可以通过预置接口根据 `NAME` 来使用，当前内置支持`"PKU-SafeRLHF/train", "PKU-SafeRLHF/test", "PKU-SafeRLHF-30K/train", "PKU-SafeRLHF-30K/test", "PKU-SafeRLHF-10K/train", "alpaca"` 几个数据集。另外还支持使用多个数据集并指定数据比例，我们可以按照需要为每个阶段训练准备多份数据集。示例如下：
```python
from paddlenlp.transformers import AutoTokenizer
from data import PreferenceDataset
tokenizer = AutoTokenizer.from_pretrained('facebook/llama-7b')
dataset = PreferenceDataset({
'alpaca': 0.75,
'my-dataset-name': 0.5
}, tokenizer)
```
### 训练
PPO 完整的训练过程包括以下 3 个阶段，如下图所示（来自[DeepSpeed-Chat](https://github.com/microsoft/DeepSpeedExamples/blob/master/applications/DeepSpeed-Chat)）：
<p align="center">
<img src="https://github.com/microsoft/DeepSpeedExamples/blob/master/applications/DeepSpeed-Chat/assets/image/ppo_trainer.png?raw=true" align="middle" width = "600" />
</p>
1. Supervised Fine-Tuning (SFT)
同[LLM 精调](https://github.com/PaddlePaddle/PaddleNLP/tree/develop/llm#2-%E7%B2%BE%E8%B0%83)，可以直接参考对应内容进行训练并使用其产出模型。
2. Reward Model Fine-Tuning
使用 `reward_main.py` 脚本根据 `reward_config.json` 训练奖励模型
```
python -u -m paddle.distributed.launch reward_main.py ./reward_config.json
```
`reward_config.json` 中的绝大部分参数释义同[LLM 精调](https://github.com/PaddlePaddle/PaddleNLP/tree/develop/llm#2-%E7%B2%BE%E8%B0%83)，不再赘述；稍有区别的是 `train_datasets`/`eval_datasets` 分别使用数据集定义注册时的`NAME`属性给出训练和验证集。另外对于奖励模型训练有以下特殊参数配置及释义（使用 PKU-Alignment/PKU-SafeRLHF 中的默认值）：
- `normalize_score_during_training`：是否在训练过程中对奖励进行 normalize，默认为 `False`。
- `normalizer_type`：使用 normalizer 时计算 mean、var 的方式，可选`"RunningMeanStd", "ExponentialMovingAverage"`。
- `normalizer_momentum`：使用 `ExponentialMovingAverage` normalizer 时指定的 momentum ，默认为 `0.9`。
- `loss_type`：使用 token 级或是 sequence 级 loss 进行奖励模型训练，可选`"token-wise", "sequence-wise"`，默认为 `"sequence-wise"`。
- `regularization`：奖励模型训练目标中对奖励的正则化系数，默认为 `0.001`。
3. RLHF：
RLHF 阶段需要 actor model、reference model、critic model、reward model 四个模型；actor-model/reference-model 使用 SFT 模型进行 initialize/frozen；critic-model/reward-model 使用 reward 模型进行 initialize/frozen (另外注意若 SFT 使用 LoRA 请先将 LoRA 权重合并）。这里使用 PKU-Alignment/PKU-SafeRLHF 提供的 SFT 模型（[PKU-Alignment/alpaca-7b-reproduced](https://huggingface.co/PKU-Alignment/alpaca-7b-reproduced)）和 reward 模型（[PKU-Alignment/beaver-7b-v1.0-reward](https://huggingface.co/PKU-Alignment/beaver-7b-v1.0-reward)，注意该模型只关注 helpful 未考量 harmless）作为示例，使用 `ppo_main.py` 脚本根据 `ppo_config.json` 进行 RLHF 训练。
```
# 类型提升 warning 暂时通过 loglevel 屏蔽，待后续修复
GLOG_minloglevel=2 python -u -m paddle.distributed.launch ppo_main.py ./ppo_config.json
```
`ppo_config.json` 中的绝大部分参数释义同[LLM 精调](https://github.com/PaddlePaddle/PaddleNLP/tree/develop/llm#2-%E7%B2%BE%E8%B0%83)，不再赘述，重点给出以下参数配置及释义（使用 PKU-Alignment/PKU-SafeRLHF 中的默认值）：
- `train_datasets`：使用数据集定义注册时的`NAME`属性给出训练集。
- `eval_datasets`：使用数据集定义注册时的`NAME`属性给出验证集。
- `ptx_datasets`：使用数据集定义注册时的`NAME`属性给出 ptx-loss 使用的数据集，未提供时将不使用 ptx-loss。
- `actor_model_name_or_path`：actor-model/reference-model 用来 initialize/frozen 的模型名称或目录。
- `reward_model_name_or_path`：reward-model 的模型名称或目录。
- `reward_critic_model_name_or_path`：critic-model 的模型名称或目录，未提供时将使用`reward_model_name_or_path`进行 critic-model 的初始化。
- `per_device_prompt_batch_size`：训练时 prompt only 数据集读取用于 rollout 生成的批次大小（每张卡）。
- `per_device_train_batch_size`：根据 prompt 进行生成及训练使用的批次大小（每张卡）。
- `num_return_sequences`：生成时每个 prompt 生成的回复个数，即 `GenerationConfig.num_return_sequences`，所有回复都将用来训练。
- `temperature`：生成采样时使用的 `temperature` ，即 `GenerationConfig.temperature`。
- `top_p`：生成采样时 top-p-filtering 阈值，即 `GenerationConfig.top_p`。
- `repetition_penalty`：生成采样时长度惩罚系数，即 `GenerationConfig.repetition_penalty`。
- `update_iters`：一次生成的数据被使用的次数。
- `kl_coeff`：对 reward 进行 KL-Penalty 的系数。
- `clip_range_score`：对 reward 进行裁剪的阈值。
- `clip_range_value`：critic model（value function）对当前 sequence 的新值与 Experience Buffer 中旧值的差距超过该范围将进行裁剪。
- `clip_range_ratio`：将当前 sequence 的新概率与 Experience Buffer 中旧概率比值裁剪到`(1-clip_range_ratio, 1+clip_range_ratio)`范围（PPO-Clip）。
- `ptx_coeff`： 预训练损失项 ptx-loss 的系数。
另外所有 [`TrainingArguments` 支持参数配置](https://paddlenlp.readthedocs.io/zh/latest/trainer.html#trainingarguments)将为 actor-model 和 critic-model 的训练复用（如`sharding_stage`），除单独提供了 `critic_learning_rate/critic_weight_decay/critic_lr_scheduler_type/critic_warmup_ratio/critic_recompute` 这些参数支持为 critic-model 训练单独指定相应配置。actor-model 和 critic-model 的 checkpoints 将分别保存在 `outpt_dir` 所指定目录的 policy 和 value 文件夹下。
此外为了支持更高性、更大规模的 RLHF 训练提供了以下特殊参数配置，可以按需使用：
- `use_fusemt`：安装 paddlenlp_ops 后将在 rollout 生成时开启生成加速（开启流水线并行时不支持生成加速），通过此设置可以禁用生成加速。
- `eval_mode`：支持为空或者设置为 "single"、"tensor_parallel"；通常可以在使用流水线并行训练时设置为"tensor_parallel"，以此在 rollout 生成阶段使用非流水线并行模型并进行生成加速。
- `offload_level`：支持设置为"freeze_model"、"optimizer"、"train_model"或者同时使用(空格分隔），分别指示 reward+reference 两个冻结模型、actor+critic 两个训练模型的优化器状态和模型参数的 offload/reload，用于在不同阶段 model/optimizer 使用结束后及时 offload 并在下次使用时 reload 相应参数权重以节省显存。
另外注意，在使用流水线并行时（pipeline_parallel_degree 大于1）建议将 `dataloader_drop_last` 设置为 true, 以此避免不同 batch size 带来的问题。
### 推理
训练完成后可以直接使用 `outpt_dir` 所指定目录中 policy 文件夹下的 checkpoints 按照[LLM 推理](https://github.com/PaddlePaddle/PaddleNLP/tree/develop/llm#4-%E6%8E%A8%E7%90%86)部分的介绍来进行推理，请参考相应部分内容。
## Acknowledge
我们借鉴了[PKU-Alignment/safe-rlhf](https://github.com/PKU-Alignment/safe-rlhf)（PKU Beaver）的优秀设计实现，在此对其作者表示感谢。
## 参考文献
- Zheng R, Dou S, Gao S, et al. Secrets of rlhf in large language models part i: Ppo[J]. arXiv preprint arXiv:2307.04964, 2023.
- Dai J, Pan X, Sun R, et al. Safe rlhf: Safe reinforcement learning from human feedback[J]. arXiv preprint arXiv:2310.12773, 2023.

403
slm/examples/RLHF/comm_utils.py
View File

@@ -1,403 +0,0 @@
# Copyright (c) 2023 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import sys
import paddle
import paddle.distributed as dist
from paddlenlp.trainer.plugins.unified_checkpoint import flatten_list
from paddlenlp.trainer.trainer import Trainer, logger
from paddlenlp.trainer.utils.helper import nested_broadcast_tensor_with_empty
from paddlenlp.utils.distributed import distributed_gather
global_dev_id = 0 if paddle.get_device() == "cpu" else int(paddle.get_device().split(":")[1])
def offload_tensor_to_cpu(tensors):
if isinstance(tensors, dict):
for _, v in tensors.items():
offload_tensor_to_cpu(v)
elif isinstance(tensors, paddle.Tensor):
if tensors.place.is_gpu_place():
cpu_tensor = tensors._copy_to(paddle.CUDAPinnedPlace(), False)
tensors.value().get_tensor()._share_data_with(cpu_tensor.value().get_tensor())
else:
logger.warning(f"Can't parse for type {type(tensors)}")
return tensors
def reload_tensor_to_gpu(tensors):
if isinstance(tensors, dict):
for _, v in tensors.items():
reload_tensor_to_gpu(v)
elif isinstance(tensors, paddle.Tensor):
if tensors._is_initialized() and not tensors.place.is_gpu_place():
gpu_tensor = tensors._copy_to(paddle.CUDAPlace(global_dev_id), False)
tensors.value().get_tensor()._share_data_with(gpu_tensor.value().get_tensor())
else:
logger.warning(f"Can't parse for type {type(tensors)}")
return tensors
def cleanup_tensor_space(tensors):
if isinstance(tensors, dict):
for _, v in tensors.items():
cleanup_tensor_space(v)
elif isinstance(tensors, paddle.Tensor):
tensors._clear_data()
else:
logger.warning(f"Can't parse for type {type(tensors)}")
return tensors
def data_group_split(tensors, group):
if group is None:
return tensors
if isinstance(tensors, (list, tuple)):
return type(tensors)(data_group_split(t, group) for t in tensors)
elif isinstance(tensors, dict):
new_dict = {}
for k, v in tensors.items():
new_dict[k] = data_group_split(v, group)
return new_dict
elif isinstance(tensors, paddle.Tensor):
return tensors.split(group.nranks)[group.rank]
else:
logger.warning(f"Can't parse for type {type(tensors)}")
return tensors
def data_group_merge(tensors, group):
if group is None:
return tensors
if isinstance(tensors, (list, tuple)):
return type(tensors)(data_group_merge(t, group) for t in tensors)
elif isinstance(tensors, dict):
new_dict = {}
for k, v in tensors.items():
new_dict[k] = data_group_merge(v, group)
return new_dict
elif isinstance(tensors, paddle.Tensor):
tensor_list = []
all_gather_nd(tensor_list, tensors, group=group, padded=True)
return paddle.concat(tensor_list)
else:
logger.warning(f"Can't parse for type {type(tensors)}")
return tensors
def group_rank_guard(group, rank=0):
def decorator(func):
def wrapper_func(*args, **kwargs):
if group.rank == rank:
ret = func(*args, **kwargs)
dist.barrier()
else:
ret = None
dist.barrier()
ret = nested_broadcast_tensor_with_empty(ret, group=group)
return ret
return wrapper_func
return decorator
def repad_rl_batches(batches, input_lengths):
if batches.get("position_ids", None) is not None:
v = batches["position_ids"]
for x in range(v.shape[0]):
v[x, input_lengths[x] :] = 1
batches["position_ids"] = v
for key in list(batches.keys()):
if batches[key].shape[0] != input_lengths.shape[0]:
batches[key] = batches[key].mean()
return batches
# https://stackoverflow.com/questions/12594148/skipping-execution-of-with-block
class SkipWithBlock(Exception):
pass
class SkipContextManager:
def __init__(self, skip):
self.skip = skip
def __enter__(self):
if self.skip:
sys.settrace(lambda *args, **keys: None)
frame = sys._getframe(1)
frame.f_trace = self.trace
def trace(self, frame, event, arg):
raise SkipWithBlock()
def __exit__(self, type, value, traceback):
if type is None:
return # No exception
if issubclass(type, SkipWithBlock):
return True # Suppress special SkipWithBlock exception
def all_gather_nd(tensor_list, tensor, group=None, padded=False):
"""
Gathers tensor arrays of different lengths in a list.
The length dimension is 0. This supports any number of extra dimensions in the tensors.
All the other dimensions should be equal between the tensors.
Args:
tensor (Tensor): Tensor to be broadcast from current process.
Returns:
(Tensor): output list of tensors that can be of different sizes
"""
if len(tensor.shape) == 0:
tensor = tensor.reshape([1])
dist.all_gather(tensor_list, tensor, group=group)
return tensor_list
world_size = group.nranks
local_size = paddle.to_tensor(tensor.shape, place=tensor.place)
all_sizes = [paddle.zeros_like(local_size) for _ in range(world_size)]
dist.all_gather(all_sizes, local_size, group=group)
# max_length = max(size[0] for size in all_sizes)
# length_diff = max_length.item() - local_size[0].item()
# if length_diff:
# pad_size = (length_diff, *tensor.size()[1:])
# padding = paddle.zeros(pad_size, place=tensor.place(), dtype=tensor.dtype)
# tensor = padle.concat((tensor, padding))
max_length = max(size[-1] for size in all_sizes)
length_diff = max_length.item() - local_size[-1].item()
if length_diff:
pad_size = (*tensor.shape[:-1], length_diff)
padding = paddle.zeros(pad_size, dtype=tensor.dtype)
tensor = paddle.concat([tensor, padding], axis=-1)
all_tensors_padded = []
dist.all_gather(all_tensors_padded, tensor, group=group)
# all_tensors = []
if padded:
tensor_list.extend(all_tensors_padded)
return all_tensors_padded
for tensor_, size in zip(all_tensors_padded, all_sizes):
tensor_list.append(tensor_[..., : size[-1]])
return tensor_list
def export_evaluate_model(self: Trainer, train_model, eval_model, **kwargs):
if eval_model is None:
return None
with_offload = kwargs.pop("with_offload", False)
train_tp_size = max(train_model.config.tensor_parallel_degree, 1)
eval_tp_size = max(eval_model.config.tensor_parallel_degree, 1)
eval_tp_rank = max(eval_model.config.tensor_parallel_rank, 0)
hcg = dist.fleet.get_hybrid_communicate_group()
tp_group = hcg.get_model_parallel_group()
pp_group = hcg.get_pipe_parallel_group()
sd_group = hcg.get_sharding_parallel_group()
dp_group = hcg.get_data_parallel_group()
global_rank = paddle.distributed.get_rank()
train_state_dict = train_model.state_dict()
eval_state_dict = eval_model.state_dict()
if dp_group.rank <= 0 and sd_group.rank <= 0:
train_pp_size = pp_group.nranks
if eval_tp_size > 1 and train_tp_size != eval_tp_size:
raise ValueError("Only support for the same tensor_parallel_degree for train and eval model for now.")
# 单卡情况
# tp->single
# tp+pp -> single
if eval_tp_size == 1:
if train_pp_size == 1 and train_tp_size > 1:
# tp ->single
logger.error("using tp to single eval model.")
# state = train_model.merge_tensor_parallel()
tp_actions = train_model.get_tensor_parallel_convert_actions(
train_model.config,
loaded_state_dict_keys=eval_state_dict.keys(),
is_split=False,
ignore_error=False,
)
is_dst = global_rank == 0
for key in eval_state_dict.keys():
tensor = train_state_dict[key]
if key in tp_actions:
ret = distributed_gather(tensor, dst=0, group=tp_group, offload=False)
action = tp_actions.pop(key)
tensor = action(ret) if is_dst else None
else:
tensor = tensor._copy_to(paddle.CPUPlace(), False) if is_dst else None
if tensor is not None:
eval_state_dict[key].set_value(tensor)
if not eval_state_dict[key]._is_initialized():
v = eval_state_dict[key]
t = paddle._C_ops.full_like(v, 0, v.dtype, paddle.CUDAPlace(global_dev_id))
v.get_tensor()._share_data_with(t.get_tensor())
if with_offload:
offload_tensor_to_cpu(train_state_dict[key])
else:
# single to single
# tp+pp -> single
raise ValueError("Not support yet.")
def create_send_recv_table(train_keys, eval_keys):
recv_table = []
send_table = []
if pp_group.rank == 0:
for key in eval_keys:
recv_table.append((key, global_rank))
for key in train_keys:
send_table.append((key, global_rank))
all_recv, all_send = [], []
paddle.distributed.all_gather_object(all_recv, [recv_table], group=pp_group)
paddle.distributed.all_gather_object(all_send, [send_table], group=pp_group)
all_recv = flatten_list(all_recv)
all_send = flatten_list(all_send)
send_dict = {}
for k, v in all_send:
send_dict[k] = v
table = []
for k, v in all_recv:
# key, send, recv
table.append([k, send_dict.pop(k), v])
assert len(send_dict) == 0, f"Some key can't be recv {send_dict.keys()}"
return table
# pp0tp0 -> pp0tp0
# pp0tp1 -> pp0tp1
# pp1tp0 -> pp0tp0
# pp1tp1 -> pp0tp1
# tp情况
# tp+pp->tp
self.timers and self.timers("export-merge-pp").start()
if eval_tp_size > 1 and train_pp_size > 1:
table = create_send_recv_table(train_state_dict.keys(), eval_state_dict.keys())
for key, src_rank, dst_rank in table:
# Init tensor for model is cleaned
if not eval_state_dict[key]._is_initialized():
v = eval_state_dict[key]
t = paddle._C_ops.full_like(v, 0, v.dtype, paddle.CUDAPlace(global_dev_id))
v.get_tensor()._share_data_with(t.get_tensor())
if src_rank == dst_rank and global_rank == src_rank:
eval_state_dict[key].copy_(train_state_dict[key], True)
else:
if global_rank == src_rank:
dist.stream.send(train_state_dict[key], dst=dst_rank)
if global_rank == dst_rank:
dist.stream.recv(eval_state_dict[key], src=src_rank)
# Offload train model if need
if global_rank == src_rank and with_offload:
offload_tensor_to_cpu(train_state_dict[key])
self.timers and self.timers("export-merge-pp").stop()
self.timers and self.timers("export-broadcast-pp").start()
if pp_group.nranks > 1:
paddle.distributed.parallel.sync_params_buffers(
eval_model, comm_group=pp_group, src_rank=pp_group.ranks[0], fuse_params=False
)
self.timers and self.timers("export-broadcast-pp").stop()
else:
# 其他 DP rank 的state dict, 适配 offload 和初始化
self.timers and self.timers("export-offload-and-init").start()
if with_offload:
for key in list(train_state_dict.keys()):
offload_tensor_to_cpu(train_state_dict[key])
for k, v in eval_state_dict.items():
if not v._is_initialized():
t = paddle._C_ops.full_like(v, 0, v.dtype, paddle.CUDAPlace(global_dev_id))
v.get_tensor()._share_data_with(t.get_tensor())
self.timers and self.timers("export-offload-and-init").stop()
paddle.distributed.barrier()
self.timers and self.timers("export-broadcast-sd-dp").start()
if eval_tp_size == 1:
for _, tensor in eval_state_dict.items():
paddle.distributed.broadcast(tensor, src=0, group=None, sync_op=True)
else:
if sd_group.nranks > 1:
if dp_group.rank <= 0:
paddle.distributed.parallel.sync_params_buffers(
eval_model, comm_group=sd_group, src_rank=sd_group.ranks[0], fuse_params=False
)
if dp_group.nranks > 1:
paddle.distributed.parallel.sync_params_buffers(
eval_model, comm_group=dp_group, src_rank=dp_group.ranks[0], fuse_params=False
)
self.timers and self.timers("export-broadcast-sd-dp").stop()
# paddle.save(eval_state_dict, f"./tmp/eval_{sd_group.rank}_tp_{eval_tp_rank}_pp_{pp_group.rank}.pdparams")
# paddle.save(train_state_dict, f"./tmp/train_{sd_group.rank}_tp_{tp_group.rank}_pp_{pp_group.rank}.pdparams")
# paddle.distributed.barrier()
# exit(-1)
old_dp_workers = self.args.world_size // (max(sd_group.nranks, 1) * max(dp_group.nranks, 1))
group_nums = self.args.logical_process_index // old_dp_workers * eval_tp_size + eval_tp_rank
if not hasattr(self, "_policy_model_eval_group") or self._policy_model_eval_group is None:
self._policy_model_eval_group = create_data_trans_group(global_rank, group_nums)
return None
def create_data_trans_group(global_rank, group_nums):
all_split_table = []
paddle.distributed.all_gather_object(all_split_table, [(global_rank, group_nums)])
all_split_table = flatten_list(all_split_table)
split_dict = {}
for k, v in all_split_table:
split_dict[k] = v
split_ranks = {}
for k, v in all_split_table:
if v in split_ranks:
split_ranks[v].append(k)
else:
split_ranks[v] = [k]
group = None
for k, ranks in split_ranks.items():
gp = paddle.distributed.new_group(ranks=ranks)
if global_rank in ranks:
group = gp
return group
Trainer.export_evaluate_model = export_evaluate_model

33
slm/examples/RLHF/data/__init__.py
View File

@@ -1,33 +0,0 @@
# Copyright (c) 2023 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from paddle.io import Dataset
from .alpaca import *
from .base import *
from .preference import *
from .prompt_only import *
from .safe_rlhf import *
from .supervised import *
class DummyDataset(Dataset):
def __init__(self, length: int) -> None:
self.length = length
def __len__(self) -> int:
return self.length
def __getitem__(self, index: int):
return {}

42
slm/examples/RLHF/data/alpaca.py
View File

@@ -1,42 +0,0 @@
# Copyright (c) 2023 PaddlePaddle Authors. All Rights Reserved.
# Copyright 2023 PKU-Alignment Team. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Stanford Alpaca dataset for supervised instruction fine-tuning."""
from __future__ import annotations
from datasets import load_dataset
from .base import RawDataset, RawSample
__all__ = ["AlpacaDataset"]
class AlpacaDataset(RawDataset):
NAME: str = "alpaca"
ALIASES: tuple[str, ...] = ("stanford-alpaca",)
def __init__(self, path: str | None = None) -> None:
self.data = load_dataset(path or "tatsu-lab/alpaca", split="train")
def __getitem__(self, index: int) -> RawSample:
data = self.data[index]
input = ( # pylint: disable=redefined-builtin
" ".join((data["instruction"], data["input"])) if data["input"] else data["instruction"]
)
answer = data["output"]
return RawSample(input=input, answer=answer)
def __len__(self) -> int:
return len(self.data)

520
slm/examples/RLHF/data/base.py
View File

@@ -1,520 +0,0 @@
# Copyright (c) 2023 PaddlePaddle Authors. All Rights Reserved.
# Copyright 2023 PKU-Alignment Team. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Base dataset class."""
from __future__ import annotations
import abc
import bisect
import copy
import os
import warnings
from fractions import Fraction
from typing import Any, Callable, ClassVar, Collection, Iterable, Iterator, List
from weakref import WeakValueDictionary
import numpy as np
import paddle
from paddle.io import Dataset, IterableDataset, Subset
from paddle.io.dataloader.collate import default_collate_fn
from tqdm import tqdm
from typing_extensions import NotRequired # Python 3.11+
from typing_extensions import TypedDict # Python 3.10+
from paddlenlp.transformers.tokenizer_utils import (
PaddingStrategy,
PretrainedTokenizerBase,
TruncationStrategy,
)
# from safe_rlhf.utils import is_main_process
__all__ = [
"RawDataset",
"RawSample",
"TokenizedDataset",
"CollatorBase",
"parse_dataset",
]
IGNORE_INDEX: int = -100
PROMPT_BEGIN: str = "BEGINNING OF CONVERSATION: "
PROMPT_USER: str = "USER: {input} "
PROMPT_ASSISTANT: str = "ASSISTANT:" # should not have a space at the end
PROMPT_INPUT: str = PROMPT_BEGIN + PROMPT_USER + PROMPT_ASSISTANT
def format_prompt(
input: str | list[str], # pylint: disable=redefined-builtin
eos_token: str,
) -> str:
if isinstance(input, str):
input = [input]
elif not isinstance(input, list):
raise ValueError(f"Unsupported type of `input`: {type(input)}. Expected: str or list[str].")
if len(input) % 2 != 1:
raise ValueError(
"The length of `input` must be odd, while `input` must end at the user question.",
)
buffer = [PROMPT_BEGIN]
for i, line in enumerate(input):
if i % 2 == 0:
# User input
buffer.extend((PROMPT_USER.format(input=line), PROMPT_ASSISTANT))
else:
# Assistant response
buffer.extend((line, eos_token))
return "".join(buffer)
def left_padding(sequences, padding_value=0):
arrs = [np.asarray(seq) for seq in sequences]
max_length = max([len(seq) for seq in sequences])
bs = len(sequences)
data = np.full([bs, max_length], padding_value, dtype=arrs[0].dtype)
for i, arr in enumerate(arrs):
data[i, -len(arr) :] = arr
return data
def right_padding(sequences, padding_value=0):
arrs = [np.asarray(seq) for seq in sequences]
max_length = max([len(seq) for seq in sequences])
bs = len(sequences)
data = np.full([bs, max_length], padding_value, dtype=arrs[0].dtype)
for i, arr in enumerate(arrs):
data[i, : len(arr)] = arr
return data
class ConcatDataset(Dataset):
r"""Dataset as a concatenation of multiple datasets.
This class is useful to assemble different existing datasets.
Args:
datasets (sequence): List of datasets to be concatenated
"""
datasets: List[Dataset]
cumulative_sizes: List[int]
@staticmethod
def cumsum(sequence):
r, s = [], 0
for e in sequence:
l = len(e)
r.append(l + s)
s += l
return r
def __init__(self, datasets: Iterable[Dataset]) -> None:
super(ConcatDataset, self).__init__()
self.datasets = list(datasets)
assert len(self.datasets) > 0, "datasets should not be an empty iterable" # type: ignore[arg-type]
for d in self.datasets:
assert not isinstance(d, IterableDataset), "ConcatDataset does not support IterableDataset"
self.cumulative_sizes = self.cumsum(self.datasets)
def __len__(self):
return self.cumulative_sizes[-1]
def __getitem__(self, idx):
if idx < 0:
if -idx > len(self):
raise ValueError("absolute value of index should not exceed dataset length")
idx = len(self) + idx
dataset_idx = bisect.bisect_right(self.cumulative_sizes, idx)
if dataset_idx == 0:
sample_idx = idx
else:
sample_idx = idx - self.cumulative_sizes[dataset_idx - 1]
return self.datasets[dataset_idx][sample_idx]
@property
def cummulative_sizes(self):
warnings.warn(
"cummulative_sizes attribute is renamed to " "cumulative_sizes", DeprecationWarning, stacklevel=2
)
return self.cumulative_sizes
def parse_dataset(string: str) -> tuple[str, dict[str, Any]]:
"""Parse dataset path and its proportion and optionally additional arguments from a string.
Args:
string (str): Dataset string in the format of ``dataset_name[:proportion[:dataset_path]]``.
"""
if string.count(":") > 2:
raise ValueError(
f"Invalid dataset string `{string}`, "
"should be in the format of `dataset_name[:proportion[:dataset_path]]`.",
)
name, colon, proportion = string.partition(":")
if not colon:
return name, {"proportion": 1.0}
proportion, colon, path = proportion.partition(":")
if "/" in proportion:
left, right = proportion.split("/")
left, right = left.strip(), right.strip()
if right == "-1":
proportion = Fraction(int(left), -1)
if proportion == 0:
proportion = 0.0
else:
proportion = float(left) / float(right)
elif proportion != "":
proportion = float(proportion)
else:
proportion = 1.0
if not colon:
return name, {"proportion": proportion}
if not path:
raise ValueError(f"Invalid dataset path `{path}`.")
path = os.path.expanduser(path)
return name, {"proportion": proportion, "path": path}
class RawSample(TypedDict, total=False):
"""Raw sample type.
For SupervisedDataset, should provide (input, answer) or (dialogue).
For PreferenceDataset, should provide (input, answer, other_answer, better).
For SafetyPreferenceDataset, should provide (input, answer, other_answer, safer, is_safe, is_other_safe).
For PromptOnlyDataset, should provide (input).
When input is a list, it would be processed as a dialogue.
"""
# Texts
input: NotRequired[str | list[str]] # either `input` or `dialogue` should be provided
"""User input text."""
answer: NotRequired[str]
"""Assistant answer text."""
other_answer: NotRequired[str]
"""Other assistant answer text via resampling."""
dialogue: NotRequired[list[str]] # either `input` or `dialogue` should be provided
"""Dialogue history."""
# Flags
better: NotRequired[bool]
"""Whether ``answer`` is better than ``other_answer``."""
safer: NotRequired[bool]
"""Whether ``answer`` is safer than ``other_answer``."""
is_safe: NotRequired[bool]
"""Whether ``answer`` is safe."""
is_other_safe: NotRequired[bool]
"""Whether ``other_answer`` is safe."""
class RawDataset(Dataset):
"""Dataset that provides raw text samples."""
NAME: ClassVar[str]
"""Name of the dataset."""
ALIASES: ClassVar[Collection[str]]
"""Name aliases of the dataset."""
__REGISTRY: ClassVar[WeakValueDictionary[str, RawDataset]] = WeakValueDictionary()
"""Registry of all subclasses."""
__ALIAS_NAME_MAPPING: ClassVar[dict[str, str]] = {}
"""Mapping from aliases to names."""
def __init_subclass__(cls) -> None:
super().__init_subclass__()
name = getattr(cls, "NAME", None)
if name is None:
return
if not isinstance(name, str):
raise ValueError("`NAME` should be a string.")
if not name:
raise ValueError("`NAME` should not be empty.")
if name in cls.__REGISTRY or name in cls.__ALIAS_NAME_MAPPING:
raise ValueError(f"Duplicate dataset name `{name}`.")
cls.__REGISTRY[name] = cls
aliases = set(cls.__dict__.get("ALIASES", ())) # do not inherit aliases from parent class
for alias in aliases:
if not isinstance(alias, str):
raise ValueError("`ALIASES` should be a set of strings.")
if not alias:
raise ValueError("`ALIASES` should not contain empty strings.")
if alias in cls.__REGISTRY or alias in cls.__ALIAS_NAME_MAPPING:
raise ValueError(f"Duplicate dataset alias `{alias}`.")
cls.__ALIAS_NAME_MAPPING.update(dict.fromkeys(aliases, name))
@staticmethod
def load(name: str, /, *args: Any, **kwargs: Any) -> RawDataset: # noqa: E999
"""Load a raw dataset by name."""
normalized_name = RawDataset.__ALIAS_NAME_MAPPING.get(name, name)
try:
cls = RawDataset.__REGISTRY[normalized_name]
except KeyError as ex:
raise ValueError(
f"Unknown dataset name `{name}`. "
"You should implement a raw dataset with that name and import it in your code. "
f'Available datasets are: {", ".join(sorted(RawDataset.__REGISTRY.keys()))}.',
) from ex
return cls(*args, **kwargs)
make = load # alias
@abc.abstractmethod
def __getitem__(self, index: int) -> RawSample:
"""Get a data sample by index."""
raise NotImplementedError
@abc.abstractmethod
def __len__(self) -> int:
"""Get the number of samples in the dataset."""
raise NotImplementedError
def __iter__(self) -> Iterator[RawSample]:
"""Iterate over the dataset."""
return (self[index] for index in range(len(self)))
def split_train_test(
self,
split_ratio: float = 0.2,
shuffle: bool = True,
seed: int = 42,
) -> tuple[Dataset[RawSample], Dataset[RawSample]]:
"""Split the dataset into train dataset and test dataset by split ratio."""
num_samples = len(self)
indices = list(range(num_samples))
num_test_samples = round(num_samples * split_ratio)
if shuffle:
rng = np.random.default_rng(seed)
rng.shuffle(indices)
train_dataset = Subset(self, indices[:-num_test_samples])
test_dataset = Subset(self, indices[-num_test_samples:])
return train_dataset, test_dataset
class TokenizedDataset(Dataset):
"""Dataset that provides tokenized samples."""
raw_datasets: list[RawDataset]
"""Raw datasets."""
tokenizer: PretrainedTokenizerBase
"""Tokenizer."""
rawdata: list[RawSample]
"""Merged raw data samples."""
data: list[dict[str, paddle.Tensor]]
"""Tokenized data samples."""
_SENTINEL: Any = object()
def __init__( # pylint: disable=too-many-branches
self,
dataset_names_and_attributes: dict[str, float | dict[str, Any]] | Iterable[tuple[str, float | dict[str, Any]]],
tokenizer: PretrainedTokenizerBase,
lazy_tokenization: bool = True,
seed: int = 42,
) -> None:
if not isinstance(dataset_names_and_attributes, dict):
dataset_names_and_attributes = tuple(dataset_names_and_attributes)
dataset_names = [name for name, _ in dataset_names_and_attributes]
if len(dataset_names) != len(set(dataset_names)):
raise ValueError(
f"Dataset names should be unique, but got {dataset_names}.",
)
super().__init__()
self.dataset_names_and_proportion: dict[str, float | Fraction] = {}
self.raw_datasets = []
for name, attributes in dict(dataset_names_and_attributes).items():
if isinstance(attributes, float):
kwargs = {"proportion": attributes}
elif isinstance(attributes, dict):
kwargs = dict(attributes) # copy
else:
raise TypeError(
f"Dataset `{name}` attributes should be a float or a dict, " f"got {type(attributes).__name__}.",
)
proportion = kwargs.pop("proportion", 1.0)
if isinstance(proportion, Fraction):
if not (proportion < 0 and proportion.denominator == 1):
raise ValueError(
f"Dataset `{name}` proportion should be a negative integer "
f"represents `num_samples / -1`, got {proportion}.",
)
else:
proportion = float(proportion)
if proportion < 0.0:
raise ValueError(
f"Dataset `{name}` proportion should be no less than 0.0, " f"got {proportion}.",
)
if proportion == 0.0:
continue
self.dataset_names_and_proportion[name] = proportion
self.raw_datasets.append(RawDataset.load(name, **kwargs))
self.tokenizer = tokenizer
self.seed = seed
merged_rawdata = self._merge_raw_datasets(seed=seed)
self.rawdata = [merged_rawdata[i] for i in range(len(merged_rawdata))]
if lazy_tokenization:
self.data = [self._SENTINEL for _ in range(len(self.rawdata))]
else:
self.data = list(
map(
self.preprocess,
tqdm(
self.rawdata,
desc="Preprocessing raw dataset...",
disable=False, # not is_main_process(),
),
),
)
def __getitem__(self, index: int) -> dict[str, paddle.Tensor]:
"""Get a tokenized data sample by index."""
data = self.data[index]
if data is self._SENTINEL:
raw_sample = self.rawdata[index]
data = self.preprocess(raw_sample)
self.data[index] = data
return data
def __len__(self) -> int:
"""Get the number of samples in the dataset."""
return len(self.data)
@abc.abstractmethod
def preprocess(self, raw_sample: RawSample) -> dict[str, paddle.Tensor]:
"""Pre-process a raw sample into a tokenized sample."""
raise NotImplementedError
def _merge_raw_datasets(self, seed: int | None = None) -> Dataset[RawSample]:
"""Merge multiple raw datasets into one dataset."""
if seed is None:
seed = self.seed
num_datasets = len(self.raw_datasets)
datasets = []
for raw_dataset, seed_seq in zip(
self.raw_datasets,
np.random.SeedSequence(seed).spawn(num_datasets),
):
num_raw_samples = len(raw_dataset)
proportion = self.dataset_names_and_proportion[raw_dataset.NAME]
if isinstance(proportion, Fraction):
num_selected_samples = abs(int(proportion))
else:
num_selected_samples = round(num_raw_samples * proportion)
indices = []
while num_selected_samples >= num_raw_samples > 0:
indices.extend(range(num_raw_samples))
num_selected_samples -= num_raw_samples
if num_selected_samples > 0:
rng = np.random.default_rng(seed_seq)
indices.extend(
rng.choice(
len(raw_dataset),
size=num_selected_samples,
replace=False,
).tolist(),
)
datasets.append(Subset(raw_dataset, indices))
if num_datasets == 1:
return datasets[0]
return ConcatDataset(datasets)
def tokenize(
self,
text: str,
add_special_tokens: bool = True,
padding: bool | str | PaddingStrategy = PaddingStrategy.DO_NOT_PAD,
truncation: bool | str | TruncationStrategy = TruncationStrategy.LONGEST_FIRST,
max_length: int | None = None,
) -> paddle.Tensor: # size = (L,)
"""Tokenize a text string into a tensor representation."""
if max_length is None:
max_length = self.tokenizer.model_max_length
return self.tokenizer(
text,
add_special_tokens=add_special_tokens,
padding=padding,
max_length=max_length,
truncation=truncation,
return_tensors="np",
)["input_ids"][0]
def get_collator(self) -> Callable[[list[dict[str, paddle.Tensor]]], dict[str, paddle.Tensor]]:
"""Get a collator function for the dataset."""
return default_collate_fn
def resize(self, size: int) -> None:
"""Resize the dataset to the given size."""
if size < 0:
raise ValueError(f"Invalid dataset size: {size}")
old_size = len(self)
if size == old_size:
return
if size > old_size:
num_replicates = (size + old_size - 1) // old_size
self.data = self.data * num_replicates
self.rawdata = self.rawdata * num_replicates
self.data = self.data[:size]
self.rawdata = self.rawdata[:size]
def split_train_test(
self,
split_ratio: float = 0.2,
shuffle: bool = True,
seed: int | None = None,
) -> tuple[Dataset[dict[str, paddle.Tensor]], Dataset[dict[str, paddle.Tensor]]]:
"""Split the dataset into train dataset and test dataset by split ratio."""
if seed is None:
seed = self.seed
num_samples = len(self)
indices = list(range(num_samples))
num_test_samples = round(num_samples * split_ratio)
if shuffle:
rng = np.random.default_rng(seed)
rng.shuffle(indices)
memo = {id(attr): attr for attr in self.__dict__.values()}
train_dataset = copy.deepcopy(self, memo=memo.copy())
test_dataset = copy.deepcopy(self, memo=memo.copy())
train_dataset.data = [self.data[i] for i in indices[:-num_test_samples]]
train_dataset.rawdata = [self.rawdata[i] for i in indices[:-num_test_samples]]
test_dataset.data = [self.data[i] for i in indices[-num_test_samples:]]
test_dataset.rawdata = [self.rawdata[i] for i in indices[-num_test_samples:]]
return train_dataset, test_dataset
class CollatorBase(metaclass=abc.ABCMeta):
pad_token_id: int # The id of the padding token for the tokenizer.
def __init__(self, pad_token_id: int) -> None:
"""Initialize a collator."""
self.pad_token_id = pad_token_id
@abc.abstractmethod
def __call__(self, samples: list[dict[str, paddle.Tensor]]) -> dict[str, paddle.Tensor]:
"""Collate a list of samples into a batch."""
raise NotImplementedError

113
slm/examples/RLHF/data/preference.py
View File

@@ -1,113 +0,0 @@
# Copyright (c) 2023 PaddlePaddle Authors. All Rights Reserved.
# Copyright 2023 PKU-Alignment Team. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Dataset class for preference training."""
from __future__ import annotations
from typing import Callable
import numpy as np
import paddle
from typing_extensions import TypedDict # Python 3.10+
from .base import (
CollatorBase,
RawSample,
TokenizedDataset,
format_prompt,
right_padding,
)
__all__ = [
"PreferenceDataset",
"PreferenceCollator",
"PreferenceSample",
"PreferenceBatch",
]
class PreferenceSample(TypedDict, total=True):
better_input_ids: paddle.Tensor # size = (L,)
worse_input_ids: paddle.Tensor # size = (L,)
class PreferenceBatch(TypedDict, total=True):
better_input_ids: paddle.Tensor # size = (B, L)
better_attention_mask: paddle.Tensor # size = (B, L)
worse_input_ids: paddle.Tensor # size = (B, L)
worse_attention_mask: paddle.Tensor # size = (B, L)
class PreferenceDataset(TokenizedDataset):
def preprocess(self, raw_sample: RawSample, index=0) -> PreferenceSample:
prompt = format_prompt(input=raw_sample["input"], eos_token=self.tokenizer.eos_token)
better_answer = raw_sample["answer"]
worse_answer = raw_sample["other_answer"]
better = raw_sample["better"]
if not better:
better_answer, worse_answer = worse_answer, better_answer
better_input_ids = self.tokenize(prompt + better_answer + self.tokenizer.eos_token)
worse_input_ids = self.tokenize(prompt + worse_answer + self.tokenizer.eos_token)
if (
better_input_ids.shape == worse_input_ids.shape
and np.all(np.equal(better_input_ids, worse_input_ids)).item()
):
# raise ValueError(
import warnings
warnings.warn(
"Two responses get the same `input_ids` after tokenization.\n\n"
f"Prompt: {prompt}\n\n"
f"Better answer: {better_answer}\n\n"
f"Worse answer: {worse_answer}",
)
return {
"better_input_ids": better_input_ids, # size = (L,)
"worse_input_ids": worse_input_ids, # size = (L,)
}
def get_collator(self) -> Callable[[list[dict[str, paddle.Tensor]]], dict[str, paddle.Tensor]]:
return PreferenceCollator(self.tokenizer.pad_token_id)
class PreferenceCollator(CollatorBase):
def __call__(self, samples: list[PreferenceSample]) -> PreferenceBatch:
input_ids = [sample["better_input_ids"] for sample in samples] + [
sample["worse_input_ids"] for sample in samples
] # size = (2 * B, L)
attention_mask = [np.ones(input_id.shape, dtype=bool) for input_id in input_ids] # size = (2 * B, L)
input_ids = right_padding(input_ids, padding_value=self.pad_token_id) # size = (2 * B, L)
attention_mask = right_padding(attention_mask, padding_value=0) # size = (2 * B, L)
bs = len(samples)
(better_input_ids, worse_input_ids,) = ( # size = (B, L) # size = (B, L)
input_ids[:bs],
input_ids[bs:],
)
(better_attention_mask, worse_attention_mask,) = ( # size = (B, L) # size = (B, L)
attention_mask[:bs],
attention_mask[bs:],
)
return {
"better_input_ids": better_input_ids, # size = (B, L)
"better_attention_mask": better_attention_mask, # size = (B, L)
"worse_input_ids": worse_input_ids, # size = (B, L)
"worse_attention_mask": worse_attention_mask, # size = (B, L)
}

78
slm/examples/RLHF/data/prompt_only.py
View File

@@ -1,78 +0,0 @@
# Copyright (c) 2023 PaddlePaddle Authors. All Rights Reserved.
# Copyright 2023 PKU-Alignment Team. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Dataset class for preference training."""
from __future__ import annotations
from typing import Callable, Hashable
import numpy as np
import paddle
from paddle.io import Dataset, Subset
from typing_extensions import TypedDict # Python 3.10+
from .base import CollatorBase, RawSample, TokenizedDataset, format_prompt, left_padding
__all__ = [
"PromptOnlyDataset",
"PromptOnlyCollator",
"PromptOnlySample",
"PromptOnlyBatch",
]
class PromptOnlySample(TypedDict, total=True):
input_ids: paddle.Tensor # size = (L,)
class PromptOnlyBatch(TypedDict, total=True):
input_ids: paddle.Tensor # size = (B, L)
attention_mask: paddle.Tensor # size = (B, L)
class PromptOnlyDataset(TokenizedDataset):
def preprocess(self, raw_sample: RawSample) -> PromptOnlySample:
prompt = format_prompt(input=raw_sample["input"], eos_token=self.tokenizer.eos_token)
input_ids = self.tokenize(prompt)
return {
"input_ids": input_ids, # size = (L,)
}
def get_collator(self) -> Callable[[list[dict[str, paddle.Tensor]]], dict[str, paddle.Tensor]]:
return PromptOnlyCollator(self.tokenizer.pad_token_id)
def _merge_raw_datasets(self, seed: int | None = None) -> Dataset[RawSample]:
"""Merge multiple raw datasets into one dataset and remove duplicates."""
def to_hashable(raw_sample: RawSample) -> Hashable:
input = raw_sample["input"] # pylint: disable=redefined-builtin
return input if isinstance(input, str) else tuple(input)
merged = super()._merge_raw_datasets(seed)
inputs = {to_hashable(merged[i]): i for i in range(len(merged))}
return Subset(merged, sorted(inputs.values()))
class PromptOnlyCollator(CollatorBase):
def __call__(self, samples: list[PromptOnlySample]) -> PromptOnlyBatch:
input_ids = [sample["input_ids"] for sample in samples]
attention_mask = [np.ones(input_id.shape, dtype=bool) for input_id in input_ids]
input_ids = left_padding(input_ids, padding_value=self.pad_token_id)
attention_mask = left_padding(attention_mask, padding_value=0)
return {
"input_ids": input_ids, # size = (B, L)
"attention_mask": attention_mask, # size = (B, L)
}

88
slm/examples/RLHF/data/safe_rlhf.py
View File

@@ -1,88 +0,0 @@
# Copyright (c) 2023 PaddlePaddle Authors. All Rights Reserved.
# Copyright 2023 PKU-Alignment Team. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Safe-RLHF preference datasets."""
from __future__ import annotations
from typing import ClassVar
from datasets import load_dataset
from .base import RawDataset, RawSample
__all__ = [
"SafeRLHFDataset",
"SafeRLHFTrainDataset",
"SafeRLHFTestDataset",
"SafeRLHF10KTrainDataset",
]
class SafeRLHFDataset(RawDataset):
SPLIT: ClassVar[str]
PATH: ClassVar[str]
def __init__(self, path: str | None = None) -> None:
self.data = load_dataset(path or self.PATH, split=self.SPLIT)
def __getitem__(self, index: int) -> RawSample:
data = self.data[index]
return RawSample(
input=data["prompt"],
answer=data["response_0"],
other_answer=data["response_1"],
better=int(data["better_response_id"]) == 0,
safer=int(data["safer_response_id"]) == 0,
is_safe=bool(data["is_response_0_safe"]),
is_other_safe=bool(data["is_response_1_safe"]),
)
def __len__(self) -> int:
return len(self.data)
class SafeRLHFTrainDataset(SafeRLHFDataset):
NAME: str = "PKU-SafeRLHF/train"
ALIASES: tuple[str, ...] = ("PKU-Alignment/PKU-SafeRLHF/train",)
PATH: str = "PKU-Alignment/PKU-SafeRLHF"
SPLIT: str = "train"
class SafeRLHFTestDataset(SafeRLHFDataset):
NAME: str = "PKU-SafeRLHF/test"
ALIASES: tuple[str, ...] = ("PKU-Alignment/PKU-SafeRLHF/test",)
PATH: str = "PKU-Alignment/PKU-SafeRLHF"
SPLIT: str = "test"
class SafeRLHF30KTrainDataset(SafeRLHFDataset):
NAME: str = "PKU-SafeRLHF-30K/train"
ALIASES: tuple[str, ...] = ("PKU-Alignment/PKU-SafeRLHF-30K/train",)
PATH: str = "PKU-Alignment/PKU-SafeRLHF-30K"
SPLIT: str = "train"
class SafeRLHF30KTestDataset(SafeRLHFDataset):
NAME: str = "PKU-SafeRLHF-30K/test"
ALIASES: tuple[str, ...] = ("PKU-Alignment/PKU-SafeRLHF-30K/test",)
PATH: str = "PKU-Alignment/PKU-SafeRLHF-30K"
SPLIT: str = "test"
class SafeRLHF10KTrainDataset(SafeRLHFDataset):
NAME: str = "PKU-SafeRLHF-10K/train"
ALIASES: tuple[str, ...] = ("PKU-Alignment/PKU-SafeRLHF-10K/train",)
PATH: str = "PKU-Alignment/PKU-SafeRLHF-10K"
SPLIT: str = "train"

128
slm/examples/RLHF/data/supervised.py
View File

@@ -1,128 +0,0 @@
# Copyright (c) 2023 PaddlePaddle Authors. All Rights Reserved.
# Copyright 2023 PKU-Alignment Team. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Dataset class for preference training."""
from __future__ import annotations
from typing import Callable
import paddle
from typing_extensions import TypedDict # Python 3.10+
from .base import (
IGNORE_INDEX,
PROMPT_ASSISTANT,
PROMPT_BEGIN,
PROMPT_USER,
CollatorBase,
RawSample,
TokenizedDataset,
format_prompt,
right_padding,
)
__all__ = [
"SupervisedDataset",
"SupervisedCollator",
"SupervisedSample",
"SupervisedBatch",
]
class SupervisedSample(TypedDict, total=True):
input_ids: paddle.Tensor # size = (L,)
labels: paddle.Tensor # size = (L,)
class SupervisedBatch(TypedDict, total=True):
input_ids: paddle.Tensor # size = (B, L)
labels: paddle.Tensor # size = (B, L)
attention_mask: paddle.Tensor # size = (B, L)
class SupervisedDataset(TokenizedDataset):
def preprocess(self, raw_sample: RawSample) -> SupervisedSample:
if raw_sample.get("input") is None and raw_sample.get("dialogue") is None:
raise ValueError("Either `input` or `dialogue` must be provided.")
if raw_sample.get("input") is not None and raw_sample.get("dialogue") is not None:
raise ValueError("At most one of `input` and `dialogue` can be provided.")
if raw_sample.get("input") is not None:
input = raw_sample["input"] # pylint: disable=redefined-builtin
if not isinstance(input, str):
raise ValueError(f"Unsupported type of `input`: {type(input)}. Expected: str.")
prompt = format_prompt(input=input, eos_token=self.tokenizer.eos_token)
answer = raw_sample["answer"]
text = prompt + answer + self.tokenizer.eos_token
input_ids = self.tokenize(text)
labels = input_ids.copy()
# Mask non-assistant input
labels[: len(self.tokenize(prompt))] = IGNORE_INDEX
return {"input_ids": input_ids, "labels": labels}
dialogue = raw_sample["dialogue"] # is not None
text = PROMPT_BEGIN
offsets = [0]
for i, line in enumerate(dialogue):
if i % 2 == 0:
# User input
text += PROMPT_USER.format(input=line) + PROMPT_ASSISTANT
else:
# Assistant input
text += line + self.tokenizer.eos_token
input_ids = self.tokenize(text)
offsets.append(len(input_ids))
labels = input_ids.copy()
# Mask non-assistant input
for begin, end in zip(offsets[::2], offsets[1::2]):
labels[begin:end] = IGNORE_INDEX
return {
"input_ids": input_ids, # size = (L,)
"labels": labels, # size = (L,)
}
def get_collator(
self, shift: bool = False
) -> Callable[[list[dict[str, paddle.Tensor]]], dict[str, paddle.Tensor]]:
return SupervisedCollator(self.tokenizer.pad_token_id, shift)
class SupervisedCollator(CollatorBase):
def __init__(self, pad_token_id: int, shift: bool = False) -> None:
"""Initialize a collator."""
super().__init__(pad_token_id)
self._shift = shift
def __call__(self, samples: list[SupervisedSample]) -> SupervisedBatch:
input_ids = right_padding(
[sample["input_ids"] for sample in samples],
padding_value=self.pad_token_id,
)
labels = right_padding(
[sample["labels"] for sample in samples],
padding_value=IGNORE_INDEX,
)
if self._shift:
input_ids = input_ids[:, :-1]
labels = labels[:, 1:]
attention_mask = input_ids != self.pad_token_id
return {
"input_ids": input_ids, # size = (B, L)
"labels": labels, # size = (B, L)
"attention_mask": attention_mask, # size = (B, L)
}

379
slm/examples/RLHF/infer_utils.py
View File

@@ -1,379 +0,0 @@
# Copyright (c) 2023 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from __future__ import annotations
import copy
import inspect
import types
from contextlib import contextmanager
import paddle
import paddle.distributed as dist
from comm_utils import cleanup_tensor_space, offload_tensor_to_cpu, reload_tensor_to_gpu
from paddle.utils import try_import
from trainer_utils import guard_set_args
import paddlenlp
from paddlenlp.trainer.trainer import Trainer, logger
from paddlenlp.transformers import PretrainedModel, PretrainedTokenizer
from paddlenlp.transformers.model_utils import dtype_guard
class Predictor:
def __init__(self, config, model: PretrainedModel = None, tokenizer: PretrainedTokenizer = None):
self.model_config = model.config
self.config = config
self.tokenizer = tokenizer
self.model = model
self.is_available = False
self._weights_mapping = None
# TODO(guosheng): Removde dependency on llm.Predictor
# 1. buffer_maker creates caches and other buffer inputs can be shared
# among multi time prediction. define caches and extra inputs creation
# method instead of using predictor.__init__
# 2. inputs_processer creates caches and other inputs can be shared among
# multi time prediction. define caches and extra inputs creation method
# instead of using predictor.__init__
from predictor import InferencePredictorMixin
self._buffer_maker = types.MethodType(InferencePredictorMixin.__init__, self)
self._inputs_processer = types.MethodType(InferencePredictorMixin._preprocess, self)
@staticmethod
def create_predictor(trainer):
from predictor import PdArgumentParser, PredictorArgument
from paddlenlp.trl.llm_utils import get_model_max_position_embeddings
# create infer model
# NOTE: infer model use static name param_attr to create and cannot be
# created multiple times.
def create_infer_model(model, dtype, set_state=False):
from models.infer_model_utils import patch_infer_generate
# apply patches to make FuseMT adapt
patch_infer_generate(
eos_token_id=trainer.tokenizer.eos_token_id, pad_token_id=trainer.tokenizer.pad_token_id
)
config = copy.deepcopy(model.config)
hcg = dist.fleet.get_hybrid_communicate_group() # may differ with training
config.tensor_parallel_degree = hcg.get_model_parallel_world_size()
config.tensor_parallel_rank = hcg.get_model_parallel_rank()
config.quant_type = None
config.single_card_ptq = True
infer_model_cls = getattr(paddlenlp.experimental.transformers, model.__class__.__name__ + "InferenceModel")
# ori_init_weights = infer_model_cls.init_weights
# infer_model_cls.init_weights = lambda self: None
with dtype_guard(dtype):
infer_model = infer_model_cls(config)
# infer_model_cls.init_weights = ori_init_weights
if set_state:
state_dict = {}
for k, v in model.state_dict().items():
# state_dict[k] = np.from_dlpack(paddle.utils.dlpack.to_dlpack(v))
state_dict[k] = v.numpy()
infer_model.set_state_dict(state_dict)
return infer_model
# to avoid oom, clear param of infer_model imediately
ori_creat_param = paddle.nn.Layer.create_parameter
def _create_param(self, *args, **kwargs):
param = ori_creat_param(self, *args, **kwargs)
param._clear_data()
# param._clear()
return param
paddle.nn.Layer.create_parameter = _create_param
# trainer might use an extra model instead of trainer.model for eval
eval_model = getattr(trainer, "_inner_eval_model", None)
infer_model = create_infer_model(trainer.model if eval_model is None else eval_model, dtype=trainer.amp_dtype)
paddle.nn.Layer.create_parameter = ori_creat_param
# for k, v in infer_model.state_dict().items():
# v._clear()
# create predictor
parser = PdArgumentParser((PredictorArgument,))
predictor_args = parser.parse_dict(
{
"src_length": get_model_max_position_embeddings( # can be changed dynamically by predictor.input_length
trainer.model.config if eval_model is None else eval_model.config
),
"max_length": trainer.args.max_length,
"dtype": trainer.amp_dtype,
"batch_size": trainer.args.per_device_train_batch_size,
# infer model do not support top_k, and differ with non-infer model
# generation which gets default top_K=50 using generation_config.top_k
"top_p": trainer.args.top_p,
"temperature": trainer.args.temperature,
"repetition_penalty": trainer.args.repetition_penalty,
}
)[0]
policy_predictor = Predictor(predictor_args, model=infer_model, tokenizer=trainer.tokenizer)
return policy_predictor
def _create_caches(self):
"""inputs can be reused among multiple predictions, such as cache"""
if hasattr(self, "cache_kvs_shape"): # has created cache
input_length = getattr(self, "input_length", 0)
# TODO(guosheng): better way to get history max cahce length, we can
# not get cahce length form cache tensor when not know cache layout
if input_length <= self.config.src_length: # reuse cahce
return
else: # create longer cache
self._clear_caches()
self.config.src_length = getattr(self, "input_length", self.config.src_length)
if not hasattr(self, "_buffer_attrs"):
pre_attrs = set(self.__dict__.keys())
self.cache_k_shapes, self.cache_v_shapes = self.model.get_cache_kvs_shape(
self.model_config, self.config.batch_size, self.config.total_max_length
)
self._buffer_maker(self.config, self.tokenizer)
if not hasattr(self, "_buffer_attrs"):
self._buffer_attrs = set(self.__dict__.keys()) - pre_attrs
def _clear_caches(self):
# del or offload
for attr in self._buffer_attrs:
delattr(self, attr)
def disable(self, model, onload_model=True):
# clear caches
self._clear_caches()
# clear params
for _, param in self.model.state_dict().items():
param._clear_data()
# param._clear()
if onload_model:
model.to(paddle.device.get_device())
self.is_available = False
def enable(self, model, offload_model=True):
if self.is_available:
return
# set params
self.set_state_dict(model, offload_model)
self.is_available = True
@paddle.no_grad()
def set_state_dict(self, model, offload_model=True):
offload_place = paddle.CUDAPinnedPlace()
state_dict = {}
for k, v in model.state_dict().items():
state_dict[k] = v
if getattr(self, "_weights_mapping", None) is None:
self._weights_mapping = self.model.get_weights_mapping()
# non_share_params = []
for k, v in self._weights_mapping.items():
param, (convert_fun, args) = k, v
args = [state_dict[name] for name in args]
value = convert_fun(*args)
if offload_model:
for arg in args:
# shared params no need to offload
if value is not arg:
cpu_arg = arg._copy_to(offload_place, blocking=False)
cpu_arg._share_buffer_to(arg)
if not isinstance(value, paddle.Tensor):
param.set_value(value)
# elif isinstance(value.place, paddle.CUDAPlace):
elif value.place.is_gpu_place():
# NOTE: _share_buffer_to seems do not work
# value._share_buffer_to(param)
# value._share_underline_tensor_to(param)
param.get_tensor()._share_data_with(value.get_tensor())
else:
param.copy_(value, True)
paddle.device.cuda.synchronize()
def _preprocess(self, source):
# make cache when infer happens to get actual shape to save memory
self._create_caches()
with guard_set_args(self.config, {"src_length": getattr(self, "input_length", self.config.src_length)}):
inputs = self._inputs_processer(source)
# We want to use a defined input_length to create cache and input_ids.
# However predictor could not use a specified length to pad currently.
# Thus we use this way to let get the actual input length.
self.infer_input_length = inputs["input_ids"].shape[-1]
return inputs
@paddle.no_grad()
def _infer(self, inputs):
for key in inputs.keys():
if paddle.is_tensor(inputs[key]):
continue
if isinstance(inputs[key], list):
if paddle.is_tensor(inputs[key]):
continue
inputs[key] = [paddle.to_tensor(item) for item in inputs[key]]
else:
inputs[key] = paddle.to_tensor(inputs[key])
inputs["cache_kvs"] = self.cache_kvs
return self.model.generate(**inputs)
def _postprocess(self, predictions):
return predictions
@paddle.no_grad()
def predict(self, input_texts: str | list[str]):
tokenized_source = self._preprocess(input_texts)
predictions = self._infer(tokenized_source)
decoded_predictions = self._postprocess(predictions)
return decoded_predictions
policy_predictor: Predictor = None
@contextmanager
def infer_guard(trainer, offload_model=True):
# trainer might use an extra model instead of trainer.model for eval
eval_model = getattr(trainer, "_inner_eval_model", None)
model = trainer.model if eval_model is None else eval_model
# PipelineParallel does not support inference speedup
if not getattr(trainer, "use_fusemt", False) or isinstance(
model, (dist.fleet.meta_parallel.PipelineLayer, dist.fleet.model.PipelineParallel)
):
yield
return
try:
try_import("paddlenlp_ops")
except:
logger.warning("paddlenlp_ops does not exist, please install paddlenlp_ops for generation speedup.")
yield
return
global policy_predictor
if policy_predictor is None:
policy_predictor = Predictor.create_predictor(trainer)
if not policy_predictor.is_available:
policy_predictor.enable(model, offload_model=offload_model)
# TODO(guosheng): patch for dist.all_recude to use tp group, fix it later
ori_all_reduce = dist.all_reduce
ori_broadcast = dist.broadcast
hcg = dist.fleet.get_hybrid_communicate_group()
dist.all_reduce = lambda x: ori_all_reduce(x, group=hcg.get_model_parallel_group())
dist.broadcast = lambda x, rank: ori_broadcast(
x, src=hcg.get_model_parallel_group_src_rank(), group=hcg.get_model_parallel_group()
)
yield
dist.all_reduce = ori_all_reduce
dist.broadcast = ori_broadcast
policy_predictor.disable(model, onload_model=offload_model)
class InferEvalModel:
"""For faster generation, not support PipelineParallel yet."""
def __init__(self, trainer: Trainer):
# trainer might use an extra model instead of trainer.model for eval
eval_model = getattr(trainer, "_inner_eval_model", None)
self.model: PretrainedModel = trainer.model if eval_model is None else eval_model
self.tokenizer: PretrainedTokenizer = trainer.tokenizer
self.trainer = trainer
def enable(self):
trainer = self.trainer
if trainer.model is not self.model:
trainer.export_evaluate_model(
trainer.model,
self.model,
with_offload="train_model" in trainer.args.offload_level,
)
else:
reload_tensor_to_gpu(self.model.state_dict())
def disable(self):
trainer = self.trainer
if trainer.model is not self.model:
cleanup_tensor_space(self.model.state_dict())
else:
offload_tensor_to_cpu(self.model.state_dict())
def __getattr__(self, name):
try:
return super().__getattr__(name)
except AttributeError:
return getattr(self.model, name)
def eval(self):
self.model.eval()
def train(self):
self.model.train()
def __call__(self, *args, **kwargs):
# assert model is on GPU
assert policy_predictor is None or not policy_predictor.is_available
return self.model(*args, **kwargs)
def generate(self, *args, **kwargs):
if policy_predictor is None or not policy_predictor.is_available:
return self.model.generate(*args, **kwargs)
arg_dict = inspect.signature(self.model.generate).bind(*args, **kwargs).arguments
input_ids = arg_dict["input_ids"]
generation_config = arg_dict["generation_config"]
# convert text and tokenize again to convert left padding to right padding
# remove this if inputs is right padding
# TODO(guosheng): allow to use right padding to infer directly
prompts = self.tokenizer.batch_decode(input_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False)
# decoded prompts has been applied with chat_template
# NOTE(guosheng): Whether to add special token should be checked, None
# chat_template would not add special token in predictor, since it assumes
# chat_template includes special tokens. While Beaver dataset tokenization
# does not use chat_template, it uses hard coded template which excludes
# special tokens.
with guard_set_args(
policy_predictor.tokenizer,
{
# predictor use right padding for infer model by default
# "padding_side": "right",
# "chat_template": None
},
):
# NOTE: right padding in predictor according to prompt might have a
# different length with input_ids, espically when input_ids has more
# paddings than the necessary. Thus pass input_length to predictor to:
# 1. use a consistent length to replace input_ids back to output to
# keep the same padding format. however predictor could not use a
# specified length to pad currently
# 2. allow to use a dynamic length for memory efficiency (by a smaller
# cache)
policy_predictor.input_length = input_ids.shape[-1]
outputs = policy_predictor.predict(prompts)
if generation_config.trunc_input:
outputs = (outputs[0][:, policy_predictor.infer_input_length :],)
return outputs
if policy_predictor.input_length != policy_predictor.infer_input_length:
outputs = (paddle.concat([input_ids, outputs[0][:, policy_predictor.infer_input_length :]], axis=-1),)
return outputs
outputs = (outputs[0],)
if self.tokenizer.padding_side == "left":
# convert back to left padding inputs
outputs[0][:, : input_ids.shape[-1]] = input_ids
return outputs

19
slm/examples/RLHF/models/__init__.py
View File

@@ -1,19 +0,0 @@
# Copyright (c) 2023 PaddlePaddle Authors. All Rights Reserved.
# Copyright 2023 PKU-Alignment Team. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Utility functions for score models."""
from .score_model_utils import AutoModelForScore, ScoreModelOutput
__all__ = ["AutoModelForScore", "ScoreModelOutput"]

185
slm/examples/RLHF/models/infer_model_utils.py
View File

@@ -1,185 +0,0 @@
# Copyright (c) 2023 PaddlePaddle Authors. All Rights Reserved.
# Copyright 2023 PKU-Alignment Team. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Utilities for inference model."""
import numpy as np
import paddle
def patch_paddlenlp_ops(eos_token_id, pad_token_id):
import paddlenlp_ops
paddlenlp_ops.save_with_output = lambda *args, **kwargs: None
# TODO(guosheng): update the custom op code directly.
ori_set_ends = paddlenlp_ops.set_stop_value_multi_ends
def _set_ends(topk_ids, stop_flags, end_ids, mode):
# infer model uses eos_token_id to pad and discriminate ending,
# patch to use pad_token_id to pad to unify with non-infer model.
topk_ids_out, stop_flags_out = ori_set_ends(topk_ids, stop_flags, end_ids, mode)
if pad_token_id != eos_token_id:
topk_ids_out = paddle.where(stop_flags, pad_token_id, topk_ids_out)
return topk_ids_out, stop_flags_out
paddlenlp_ops.set_stop_value_multi_ends = _set_ends
def patch_infer_generate(eos_token_id, pad_token_id):
"""patches for inference model to make FuseMT adapt"""
# patch paddlenlp_ops, maybe update the custom op code directly later
# NOTE: should patch paddlenlp_ops before infer model import
patch_paddlenlp_ops(eos_token_id, pad_token_id)
# patch get_weights_mapping for InferenceModel
patch_infer_model()
# patch GenerationInferenceModel.sample
from paddlenlp.experimental.transformers.generation_utils import (
GenerationInferenceModel,
)
ori_update_model_kwargs = GenerationInferenceModel.update_model_kwargs_for_generation
def _update_model_kwargs(self, *args, **kwargs):
# update_model_kwargs_for_generation only returns , hack to.
model_kwargs = ori_update_model_kwargs(self, *args, **kwargs)
next_tokens = model_kwargs["next_tokens"]
all_input_ids = paddle.concat([model_kwargs["all_input_ids"], next_tokens], axis=1)
model_kwargs["next_tokens"] = all_input_ids
model_kwargs["all_input_ids"] = None
return model_kwargs
GenerationInferenceModel.update_model_kwargs_for_generation = _update_model_kwargs
_model_weights_mapping_dict = {}
def register_model(model_cls_name):
def mark_cls_name(func):
# Do not register here although we can, otherwise infer model would import
# before paddlenlp_ops.
_model_weights_mapping_dict[model_cls_name] = func
return func
return mark_cls_name
def patch_infer_model():
import paddlenlp.experimental.transformers as infer_transformers
for model_cls_name, get_weights_mapping in _model_weights_mapping_dict.items():
model_cls = getattr(infer_transformers, model_cls_name)
model_cls.get_weights_mapping = get_weights_mapping
@register_model("LlamaForCausalLMInferenceModel")
def get_weights_mapping(self):
"""model to infer model"""
head_size = self.config.hidden_size // self.config.num_attention_heads
def _concat_qkv(q, k, v):
if isinstance(q, paddle.Tensor):
concated_qkv_weight = paddle.concat([q, k, v], axis=-1).T.reshape(
[
3 * (self.config.num_attention_heads // self.config.tensor_parallel_degree) * (head_size),
self.config.hidden_size,
]
)
else:
concated_qkv_weight = (
np.concatenate(
[q, k, v],
axis=-1,
)
.transpose(1, 0)
.reshape(
3 * (self.config.num_attention_heads // self.config.tensor_parallel_degree) * (head_size),
self.config.hidden_size,
)
)
return concated_qkv_weight
def _concat_ffn1(w1, w2):
if isinstance(w1, paddle.Tensor):
concated_ffn1_weight = paddle.concat([w1, w2], axis=-1)
else:
concated_ffn1_weight = np.concatenate([w1, w2], axis=-1)
return concated_ffn1_weight
identity = lambda x: x
weight_mapping = {}
weight_mapping[self.lm_head.weight] = [
identity,
[
"lm_head.weight",
],
]
weight_mapping[self.llama.embed_tokens.weight] = [
identity,
[
"llama.embed_tokens.weight",
],
]
weight_mapping[self.llama.norm.weight] = [
identity,
[
"llama.norm.weight",
],
]
for idx in range(self.config.num_hidden_layers):
weight_mapping[self.llama.transformer_block.qkv_weights[idx]] = [
_concat_qkv,
[
f"llama.layers.{idx}.self_attn.q_proj.weight",
f"llama.layers.{idx}.self_attn.k_proj.weight",
f"llama.layers.{idx}.self_attn.v_proj.weight",
],
]
weight_mapping[self.llama.transformer_block.ffn1_weights[idx]] = [
_concat_ffn1,
[
f"llama.layers.{idx}.mlp.gate_proj.weight",
f"llama.layers.{idx}.mlp.up_proj.weight",
],
]
weight_mapping[self.llama.transformer_block.linear_weights[idx]] = [
identity,
[
f"llama.layers.{idx}.self_attn.o_proj.weight",
],
]
weight_mapping[self.llama.transformer_block.ffn2_weights[idx]] = [
identity,
[
f"llama.layers.{idx}.mlp.down_proj.weight",
],
]
weight_mapping[self.llama.transformer_block.ln_scales[idx]] = [
identity,
[
f"llama.layers.{idx}.input_layernorm.weight",
],
]
weight_mapping[self.llama.transformer_block.ffn_ln_scales[idx]] = [
identity,
[
f"llama.layers.{idx}.post_attention_layernorm.weight",
],
]
return weight_mapping

261
slm/examples/RLHF/models/model_pp.py
View File

@@ -1,261 +0,0 @@
# Copyright (c) 2023 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import paddle
import paddle.nn as nn
from paddle.distributed.fleet.meta_parallel import LayerDesc
from paddlenlp.transformers import LlamaForCausalLM, LlamaForCausalLMPipe
from paddlenlp.transformers.llama.modeling import LlamaDecoderLayer
from paddlenlp.transformers.llama.modeling_pp import (
LlamaRMSNormPipe,
parse_args,
return_args,
)
from .pp_model_utils import fwd_args_to_dict, get_expected_keys, pad_batches_inputs
from .ppo_model_utils import (
RLHFPPOMixedLoss,
RLHFValueLoss,
create_loss,
make_position_ids,
)
from .score_model_utils import ScoreModelMixin
# patches for base pipe model
# non-pipe model class, can be used to parse and convert forward args
# mainly used for generation with PipelienParallel model
LlamaForCausalLMPipe._non_pipe_model_class = LlamaForCausalLM
LlamaForCausalLMPipe._non_pipe_decoder_layer_class = LlamaDecoderLayer
class LlamaPolicyPipe(LlamaForCausalLMPipe):
# TODO(guosheng): maybe make a Mixin is better
@fwd_args_to_dict
def _prepare_pipeline_inputs_func(self, inputs):
# first_stage_keys = ["input_ids", "attention_mask"]
first_stage_keys = ["input_ids", "attention_mask", "position_ids"]
# last_stage_keys = [
# "labels", "input_ids", "log_probs", "advantages", "sequence_mask"
# ]
# TODO(guosheng): make input keys same with model arg names, maybe we
# can use inspect and set as global var which can then be used here and
# in PPOTrainer.
last_stage_keys = ["labels", "input_ids", "old_log_probs", "reward_advantages", "sequence_mask"]
if type(inputs) is dict:
# for left padding, position_ids is nececessary
if "position_ids" not in inputs:
inputs["position_ids"] = make_position_ids(inputs["attention_mask"])
# ppo-loss and ptx-loss need different labels, and data iter provides
# corrensponding data, thus add the not provided fields here.
# policy trian and infer has different inputs, infer uses position_ids.
# for key in last_stage_keys:
for key in first_stage_keys + last_stage_keys:
if key not in inputs:
inputs[key] = None
return [
get_expected_keys(inputs, first_stage_keys),
get_expected_keys(inputs, last_stage_keys),
]
for data in inputs:
# for key in last_stage_keys:
for key in first_stage_keys + last_stage_keys:
if key not in data:
if key == "position_ids":
data[key] = make_position_ids(data["attention_mask"])
continue
data[key] = None
# keys = list(inputs[0].keys())
inputs_batch = {key: [data.get(key) for data in inputs] for key in first_stage_keys + last_stage_keys}
# NOTE(guosheng): PipelineParallel requires send/recv tensors among
# micro-batches/accu-steps have the same shape. Thus pad here, maybe
# should make data collator do padding and pad optionally here, since
# padding strategy may not be clear here.
# 1. For input_ids/attention_mask/labels (prompt+target) padding:
# Some data fields, such as input_ids/attention_mask/labels, should
# have same shape after padding, and each of them cannot pad only
# according to its own max length which might be different since the
# filed value is None for different batches/tasks.
src_tgt_keys = ["input_ids", "attention_mask", "labels", "position_ids"]
max_len = max([x.shape[-1] for x in inputs_batch["input_ids"]])
pad_len = [max_len - x.shape[-1] for x in inputs_batch["input_ids"]]
for key in src_tgt_keys:
# Do not pad position_ids with 0 since 0s in position_ids has special
# usage in reward model. We use 1 to pad.
padding_value = self._ignore_index if key == "labels" else 1 if key == "position_ids" else 0
inputs_batch[key] = pad_batches_inputs(inputs_batch[key], padding_value, pad_len=pad_len)
# 2. For old_log_probs/reward_advantages/sequence_mask (target) padding:
# hard to pad acorss batches, think in some cases one batch might have the
# longest prompt+target length but the shortest target lengh, which might
# cause mismatch between inputs with prompt+target length and labels with
# target length. NOTE: however trick can be used here, label fields with
# target length such as old_log_probs/reward_advantages/sequence_mask do
# not need to join comm and thus there is no need to keep same shape among
# batches of accumulation steps, they just need to pad as prompt+target
# fields such as input_ids.
tgt_keys = ["old_log_probs", "reward_advantages", "sequence_mask"]
for key in tgt_keys:
padding_value = 0
inputs_batch[key] = pad_batches_inputs(inputs_batch[key], padding_value, pad_len=pad_len)
# for key, value in inputs_batch.items():
# padding_value = self._ignore_index if key == "labels" else 0
# max_len = max_len if key in [
# "input_ids", "attention_mask", "labels"
# ] else None
# inputs_batch[key] = pad_batches_inputs(value, padding_value, max_len)
return [
get_expected_keys(inputs_batch, first_stage_keys),
get_expected_keys(inputs_batch, last_stage_keys),
]
def __init__(self, config, **kwargs):
# NOTE: make _sequential_layers/_single_to_pp_mapping/_pp_to_single_mapping
# instance attrs instead of class attrs to support more than one pipeline
# models. Maybe make all sequential_layers add once.
self._sequential_layers = []
self._single_to_pp_mapping = None
self._pp_to_single_mapping = None
# To be consistent with score model init and allow hyper-param be passed
# using __init__/from_pretrained
self._init_kwargs = kwargs
super().__init__(config)
self._ignore_index = self._loss_fn.sft_criterion.ignore_index
def get_loss_fn(self, config):
return create_loss(RLHFPPOMixedLoss, config, self._init_kwargs)
@property
def head_out_meta(self):
"""mainly for eval/generation with PipelineParallel"""
# None means to use actual data info
return paddle.static.InputSpec(shape=[None, None, self.config.vocab_size], dtype=None)
class _LlamaRMSNormPipe(LlamaRMSNormPipe):
"""
We need position_ids for reward model, so wrap LlamaRMSNormPipe to pass position_ids
"""
def __init__(self, config):
super().__init__(config)
def forward(self, args):
hidden_states, attention_mask, position_ids, alibi = parse_args(args)
return return_args(self.norm(hidden_states), attention_mask, position_ids)
# LayerDesc of PipelineParallel requires head to be a nn.Layer
class ValueHead(nn.Layer, ScoreModelMixin):
def __init__(self, config, **kwargs):
super().__init__()
self.config = config
self.init_score_head(config, hidden_size=config.hidden_size, **kwargs)
def forward(self, args):
# attention_mask passed from pre-stage is shaped (bs, 1, seq_len, seq_len)
hidden_state, attention_mask, position_ids, alibi = parse_args(args)
outputs = self.get_score(
hidden_state, attention_mask=attention_mask, position_ids=position_ids, return_dict=True
)
return outputs
class LlamaValuePipe(LlamaForCausalLMPipe):
# TODO(guosheng): maybe make a Mixin is better
@fwd_args_to_dict
def _prepare_pipeline_inputs_func(self, inputs):
# ValueHead/get_score needs original attention_mask or position_ids,
# while attention_mask passed from pre-stage is not the original, thus
# hack for position_ids here.
# Maybe add position_ids into inputs later and use position_ids instead
# of attention_mask to get score not only for pipeline parallel.
first_stage_keys = ["input_ids", "attention_mask", "position_ids"]
# TODO(guosheng): make input keys same with model arg names, maybe we
# can use inspect and set as global var which can then be used here and
# in PPOTrainer.
last_stage_keys = ["old_reward_values", "reward_returns", "sequence_mask"]
if type(inputs) is dict:
if "position_ids" not in inputs:
inputs["position_ids"] = make_position_ids(inputs["attention_mask"])
return [
get_expected_keys(inputs, first_stage_keys),
get_expected_keys(inputs, last_stage_keys),
]
for data in inputs:
if "position_ids" not in data:
data["position_ids"] = make_position_ids(data["attention_mask"])
# keys = list(inputs[0].keys())
inputs_batch = {key: [data.get(key) for data in inputs] for key in first_stage_keys + last_stage_keys}
# 1. For input_ids/attention_mask (prompt+target) padding:
# src_tgt_keys = ["input_ids", "attention_mask"]
src_tgt_keys = ["input_ids", "attention_mask", "position_ids"]
max_len = max([x.shape[-1] for x in inputs_batch["input_ids"]])
pad_len = [max_len - x.shape[-1] for x in inputs_batch["input_ids"]]
for key in src_tgt_keys:
# Do not pad position_ids with 0 since 0s in position_ids has special
# usage in reward model. We use 1 to pad.
padding_value = self._ignore_index if key == "labels" else 1 if key == "position_ids" else 0
inputs_batch[key] = pad_batches_inputs(inputs_batch[key], padding_value, pad_len=pad_len)
# 2. For old_reward_values/reward_returns/sequence_mask (target) padding:
tgt_keys = ["old_reward_values", "reward_returns", "sequence_mask"]
for key in tgt_keys:
padding_value = 0
inputs_batch[key] = pad_batches_inputs(inputs_batch[key], padding_value, pad_len=pad_len)
# for key, value in inputs_batch.items():
# inputs_batch[key] = pad_batches_inputs(value, padding_value=0)
# if "position_ids" not in inputs[0]:
# inputs_batch["position_ids"] = [
# make_position_ids(attention_mask) for attention_mask in inputs_batch["attention_mask"]
# ]
return [
get_expected_keys(inputs_batch, first_stage_keys),
get_expected_keys(inputs_batch, last_stage_keys),
]
def __init__(self, config, **kwargs):
# NOTE: make _sequential_layers/_single_to_pp_mapping/_pp_to_single_mapping
# instance attrs instead of class attrs to support more than one pipeline
# models. Maybe make all sequential_layers add once.
self._sequential_layers = []
self._single_to_pp_mapping = None
self._pp_to_single_mapping = None
# To be consistent with score model init and allow hyper-param be passed
# using __init__/from_pretrained
self._init_kwargs = kwargs
super().__init__(config)
def add_head(self, config):
init_kwargs = self._init_kwargs
# hack to replace original RMSNormPipe to support ValueHead inputs
norm_prefix = self._sequential_layers.pop(-1)["name_prefix"]
self.add_sequential_layer(LayerDesc(_LlamaRMSNormPipe, config=config), norm_prefix)
self.add_sequential_layer(LayerDesc(ValueHead, config, **init_kwargs), "")
def get_loss_fn(self, config):
return create_loss(RLHFValueLoss, config, self._init_kwargs)
@property
def head_out_meta(self):
# None means to use actual data info
return (
paddle.static.InputSpec(shape=[None, None, 1], dtype=None),
paddle.static.InputSpec(shape=[None, 1], dtype=None),
)

92
slm/examples/RLHF/models/pp_model_utils.py
View File

@@ -1,92 +0,0 @@
# Copyright (c) 2023 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import importlib
import inspect
import paddle
def fwd_step_patch(func, output, self, *args, **kwargs):
# training patch
if self.training and self.is_pipeline_last_stage():
if getattr(self, "_step_losses", None):
self._step_losses.append(output.detach())
else:
self._step_losses = [output.detach()]
def make_wrapper(func, pre_patch=None, post_patch=None):
def wrapper(*args, **kwargs):
if pre_patch is not None:
pre_patch(func, None, *args, **kwargs)
output = func(*args, **kwargs)
if post_patch is not None:
post_patch(func, output, *args, **kwargs)
return output
return wrapper
funcs = [(paddle.distributed.fleet.model.PipelineParallel._forward_step, fwd_step_patch)]
for func in funcs:
fun, patch = func
module = importlib.import_module(fun.__module__)
cls_name = fun.__qualname__[: -len(fun.__name__) - 1]
wrap_fun = make_wrapper(fun, post_patch=patch)
cls_obj = getattr(module, cls_name)
setattr(cls_obj, fun.__name__, wrap_fun)
@paddle.no_grad()
def pad_batches_inputs(inputs, padding_value=0, max_len=None, pad_len=None):
"""Pad length for tensors shaped [bs, seq_len] to [bs, max(seq_lens)]"""
if pad_len is not None:
pad_len = [pad_len] * len(inputs) if isinstance(pad_len, int) else pad_len
elif max_len is None:
# max_len = max([x.shape[-1] for x in inputs if x is not None])
max_len = max([x.shape[-1] if isinstance(x, paddle.Tensor) else 0 for x in inputs])
pad_len = [max_len - x.shape[-1] if isinstance(x, paddle.Tensor) else 0 for x in inputs]
for i in range(len(inputs)):
x = inputs[i]
# if x is None or x.shape[-1] == max_len:
if not isinstance(x, paddle.Tensor) or x.shape[-1] == max_len:
continue
inputs[i] = paddle.concat([x, paddle.full([x.shape[0], pad_len[i]], padding_value, dtype=x.dtype)], -1)
return inputs
def get_expected_keys(inputs, keys):
ret = tuple([inputs.get(k, None) for k in keys if k in inputs])
if len(ret) == 1:
ret = ret[0]
return ret
def fwd_args_to_dict(fun):
def _impl(self, *args, **kwargs):
try:
return fun(self, *args, **kwargs)
except TypeError:
# otherwise, inputs is any valid format of non_pipe_model forward args,
# convert to dict, to support more args format in prediction_pipeline_step
# assume no arg is inspect.Parameter.VAR_KEYWORD
arg_dict = (
inspect.signature(self._non_pipe_model_class.forward).bind(*((self,) + args), **kwargs).arguments
)
arg_dict.pop("self")
return fun(self, arg_dict)
return _impl

117
slm/examples/RLHF/models/ppo_model.py
View File

@@ -1,117 +0,0 @@
# Copyright (c) 2023 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from paddlenlp.transformers import LlamaForCausalLM, PretrainedConfig
from .ppo_model_utils import PolicyOutput, RLHFPPOMixedLoss, RLHFValueLoss, ValueOutput
from .score_model import LlamaModelForScore
# TODO(guosheng): create Mixin and make model classes using metaclass.
class LlamaPolicyModel(LlamaForCausalLM):
def __init__(self, config: PretrainedConfig, **kwargs):
super().__init__(config)
self.loss_fn = RLHFPPOMixedLoss(config, **kwargs)
def forward(
self,
input_ids=None,
position_ids=None,
attention_mask=None,
inputs_embeds=None,
labels=None,
use_cache=False,
past_key_values=None,
log_probs=None,
advantages=None,
sequence_mask=None,
output_attentions=None,
output_hidden_states=None,
return_dict=None,
):
outputs = super().forward(
input_ids=input_ids,
position_ids=position_ids,
attention_mask=attention_mask,
inputs_embeds=inputs_embeds,
labels=None,
use_cache=use_cache,
past_key_values=past_key_values,
output_attentions=output_attentions,
output_hidden_states=output_hidden_states,
return_dict=return_dict,
)
logits = outputs[0]
loss = None
if labels is not None or advantages is not None:
loss = self.loss_fn(logits, (labels, input_ids, log_probs, advantages, sequence_mask))
if not return_dict:
return (loss,) + outputs if loss is not None else outputs
return PolicyOutput(
loss=loss,
logits=logits,
past_key_values=outputs.past_key_values,
hidden_states=outputs.hidden_states,
attentions=outputs.attentions,
)
class LlamaValueModel(LlamaModelForScore):
def __init__(self, config, **kwargs):
super().__init__(config, **kwargs)
self.loss_fn = RLHFValueLoss(config, **kwargs)
def forward(
self,
input_ids=None,
position_ids=None,
attention_mask=None,
inputs_embeds=None,
use_cache=False,
past_key_values=None,
old_values=None,
returns=None,
sequence_mask=None,
output_attentions=None,
output_hidden_states=None,
return_dict=None,
):
outputs = super().forward(
input_ids=input_ids,
position_ids=position_ids,
attention_mask=attention_mask,
inputs_embeds=inputs_embeds,
use_cache=use_cache,
past_key_values=past_key_values,
output_attentions=output_attentions,
output_hidden_states=output_hidden_states,
return_dict=return_dict,
)
reward_values, rewards = outputs
loss = None
if returns is not None:
loss = self.loss_fn(reward_values, old_values, returns, sequence_mask)
if not return_dict:
return (loss,) + outputs if loss is not None else outputs
return ValueOutput(
loss=loss,
value=reward_values,
reward=rewards,
past_key_values=outputs.past_key_values,
hidden_states=outputs.hidden_states,
attentions=outputs.attentions,
)

274
slm/examples/RLHF/models/ppo_model_utils.py
View File

@@ -1,274 +0,0 @@
# Copyright (c) 2023 PaddlePaddle Authors. All Rights Reserved.
# Copyright 2023 PKU-Alignment Team. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Utilities for score models."""
from __future__ import annotations
import inspect
from dataclasses import dataclass
from typing import Optional, Tuple
import paddle
import paddle.nn as nn
import paddle.nn.functional as F
# use LlamaPretrainingCriterion as common PretrainingCriterion
from paddlenlp.transformers import LlamaPretrainingCriterion as PretrainingCriterion
from paddlenlp.transformers.model_outputs import ModelOutput
@dataclass
class PolicyOutput(ModelOutput):
loss: Optional[paddle.Tensor] = None
logits: paddle.Tensor = None
# logits_entropy: Optional[paddle.Tensor] = None
past_key_values: Optional[Tuple[Tuple[paddle.Tensor]]] = None
hidden_states: Optional[Tuple[paddle.Tensor]] = None
attentions: Optional[Tuple[paddle.Tensor]] = None
cross_attentions: Optional[Tuple[paddle.Tensor]] = None
@dataclass
class ValueOutput(ModelOutput):
loss: Optional[paddle.Tensor] = None
value: paddle.Tensor = None
reward: paddle.Tensor = None
past_key_values: Optional[Tuple[Tuple[paddle.Tensor]]] = None
hidden_states: Optional[Tuple[paddle.Tensor]] = None
attentions: Optional[Tuple[paddle.Tensor]] = None
cross_attentions: Optional[Tuple[paddle.Tensor]] = None
def merge_fwd_labels(loss_cls):
"""
PipelineParallel and trainer.criterion both use labels as tuple, thus wrap.
"""
ori_fwd = loss_cls.forward
def loss_fwd(self, predict, labels):
return ori_fwd(self, predict, *labels)
fwd_params = inspect.signature(ori_fwd).parameters
# forward(self, predict, label1, label2, ...)
loss_cls.label_names = list(fwd_params.keys())[2:]
loss_cls.label_default_values = {}
for label_name in loss_cls.label_names:
if fwd_params[label_name].default is not inspect.Parameter.empty:
loss_cls.label_default_values[label_name] = fwd_params[label_name].default
loss_cls.forward = loss_fwd
return loss_cls
def create_loss(loss_cls, config, extra_args, merge_labels=None):
"""
loss_cls(paddle.nn.Layer): loss class
config(PratrainedConfig): model config, to be consistent with loss defined
in transformers
extra_args(dict): create loss with more args not in config
merge_labels: use a wrapped loss_cls whose label args are merged into one arg,
this is useful to PipelineParallel and trainer.criterion since they only
support loss format corresponding to this format.
"""
# TODO(guosheng): merge_labels if loss_cls not
ori_fwd = loss_cls.forward
if merge_labels:
fwd_params = inspect.signature(ori_fwd).parameters
if len(fwd_params.keys()) > 3: # merge_fwd_labels has not done
loss_cls = merge_fwd_labels(loss_cls)
# forward(self, predict, label1, label2, ...)
loss_arg_names = list(inspect.signature(loss_cls.__init__).parameters.keys())[2:]
if isinstance(extra_args, dict):
loss_kwargs = dict([(name, extra_args[name]) for name in loss_arg_names if name in extra_args])
else:
# create from TrainingArguments
loss_kwargs = dict([(name, getattr(extra_args, name)) for name in loss_arg_names if hasattr(extra_args, name)])
loss = loss_cls(config, **loss_kwargs)
return loss
@paddle.no_grad()
def make_position_ids(attention_mask, source=None):
if len(attention_mask.shape) == 4: # causal mask
position_ids_p1 = attention_mask.cast(paddle.int64).sum(-1)
position_ids = position_ids_p1 - 1
position_ids = paddle.where(position_ids == -1, position_ids_p1, position_ids)
return position_ids[:, 0, :]
assert len(attention_mask.shape) == 2 # padding mask
attention_mask_bool = attention_mask
attention_mask = attention_mask.cast(paddle.int64)
position_ids = attention_mask.cumsum(-1) - 1
# Make padding positions in source be 0, since reward model use position_ids
# plus with padding size (number of 0s) in source to calculate end offsets.
# It does not matter when source is left padding and target is right padding
# which is the output of non-FuseMT generation, while when using FuseMT whose
# output is right padding source and right padding target, we have to set
# padding positions in source be 0 to make compatible.
if source is not None:
src_len = position_ids[:, source.shape[-1] - 1].unsqueeze(-1)
position_ids = paddle.where(
paddle.logical_and(paddle.logical_not(attention_mask_bool), position_ids <= src_len),
attention_mask,
position_ids,
)
return position_ids
position_ids = paddle.where(position_ids == -1, attention_mask, position_ids)
return position_ids
@paddle.no_grad()
def make_attention_mask(input_ids, pad_id, unk_id=None, past_key_values_length=0, causal_mask=True):
attention_mask = input_ids != pad_id
if unk_id is not None and pad_id != unk_id:
attention_mask = paddle.logical_and(attention_mask, input_ids != unk_id)
if not causal_mask:
return attention_mask
batch_size, target_length = input_ids.shape # target_length: seq_len
mask = paddle.tril(paddle.ones((target_length, target_length), dtype="bool"))
if past_key_values_length > 0:
# [tgt_len, tgt_len + past_len]
mask = paddle.concat([paddle.ones([target_length, past_key_values_length], dtype="bool"), mask], axis=-1)
# [bs, 1, tgt_len, tgt_len + past_len]
causal_mask = mask[None, None, :, :].expand([batch_size, 1, target_length, target_length + past_key_values_length])
attention_mask = attention_mask[:, None, None, :]
expanded_attn_mask = attention_mask & causal_mask
return expanded_attn_mask
def gather_log_probabilities(logits: paddle.Tensor, labels: paddle.Tensor) -> paddle.Tensor:
"""Gather log probabilities of the given labels from the logits."""
log_probs = F.log_softmax(logits, axis=-1)
log_probs_labels = paddle.take_along_axis(log_probs, axis=-1, indices=labels.unsqueeze(axis=-1))
return log_probs_labels.squeeze(axis=-1)
class RLHFPPOLoss(nn.Layer):
def __init__(self, config, clip_range_ratio=0.2):
super().__init__()
self.clip_range_ratio = clip_range_ratio
self.config = config
def actor_loss_fn(
self, log_probs: paddle.Tensor, old_log_probs: paddle.Tensor, advantages: paddle.Tensor, mask: paddle.Tensor
) -> paddle.Tensor:
# policy gradient loss
ratio = paddle.exp(log_probs - old_log_probs)
pg_loss1 = -advantages * ratio
pg_loss2 = -advantages * paddle.clip(
ratio,
1.0 - self.clip_range_ratio,
1.0 + self.clip_range_ratio,
)
return paddle.sum(paddle.maximum(pg_loss1, pg_loss2) * mask) / mask.sum()
def forward(self, logits, input_ids, old_log_probs, reward_advantages, sequence_mask):
log_probs = gather_log_probabilities(logits[:, :-1], input_ids[:, 1:])
if log_probs.shape[1] == old_log_probs.shape[1]:
# labels (old_log_probs, reward_advantages, sequence_mask) has
# src+tgt-1 length, valid length is determined by sequence_mask
pass
elif log_probs.shape[1] < old_log_probs.shape[1]:
# labels (old_log_probs, reward_advantages, sequence_mask) has
# src+tgt length and the last one is a padding to be consistent
# with input_ids
assert log_probs.shape[1] == old_log_probs.shape[1] - 1
log_probs = paddle.concat([log_probs, paddle.zeros([log_probs.shape[0], 1], dtype=log_probs.dtype)], -1)
else:
# labels (old_log_probs, reward_advantages, sequence_mask) has tgt length
log_probs = log_probs[:, -old_log_probs.shape[1] :]
actor_loss = self.actor_loss_fn(
log_probs,
old_log_probs,
reward_advantages,
sequence_mask,
)
return actor_loss
@merge_fwd_labels
class RLHFPPOMixedLoss(nn.Layer):
"""provide two losses, one for PPO loss, the other for SFT loss."""
def __init__(self, config, ptx_coeff=16, clip_range_ratio=0.2):
super(RLHFPPOMixedLoss, self).__init__()
self.ptx_coeff = ptx_coeff
self.ppo_criterion = RLHFPPOLoss(config, clip_range_ratio)
self.sft_criterion = PretrainingCriterion(config)
def forward(self, logits, labels, input_ids, old_log_probs, reward_advantages, sequence_mask):
logits = logits if isinstance(logits, paddle.Tensor) else logits[0]
loss = None
# sft, pt loss
if labels is not None:
loss = self.ptx_coeff * self.sft_criterion(logits, labels)
# ppo loss
if reward_advantages is not None:
loss = self.ppo_criterion(logits, input_ids, old_log_probs, reward_advantages, sequence_mask)
return loss
@merge_fwd_labels
class RLHFValueLoss(nn.Layer):
def __init__(self, config, clip_range_value=5.0):
super().__init__()
self.clip_range_value = clip_range_value
self.config = config
def critic_loss_fn(
self,
values: paddle.Tensor,
old_values: paddle.Tensor,
returns: paddle.Tensor,
mask: paddle.Tensor,
) -> paddle.Tensor:
"""Compute critic loss."""
# TODO(guosheng): use paddle.clip when its min/max can support more than
# 0D Tensor
values_clipped = paddle.minimum(
paddle.maximum(values, old_values - self.clip_range_value), old_values + self.clip_range_value
)
vf_loss1 = paddle.square(values - returns)
vf_loss2 = paddle.square(values_clipped - returns)
return 0.5 * paddle.sum(paddle.maximum(vf_loss1, vf_loss2) * mask) / mask.sum()
def forward(self, reward_values, old_reward_values, reward_returns, sequence_mask):
reward_values = reward_values if isinstance(reward_values, paddle.Tensor) else reward_values[0]
reward_values = reward_values.squeeze(axis=-1)[:, :-1]
if reward_values.shape[1] == old_reward_values.shape[1]:
# labels (old_reward_values, reward_returns, sequence_mask) has
# src+tgt-1 length, valid length is determined by sequence_mask
pass
elif reward_values.shape[1] < old_reward_values.shape[1]:
# labels (old_reward_values, reward_returns, sequence_mask) has
# src+tgt length and the last one is a padding to be consistent
# with input_ids
assert reward_values.shape[1] == old_reward_values.shape[1] - 1
reward_values = paddle.concat(
[reward_values, paddle.zeros([reward_values.shape[0], 1], dtype=reward_values.dtype)], -1
)
else:
# labels (old_reward_values, reward_returns, sequence_mask) has
# tgt length
reward_values = reward_values[:, -old_reward_values.shape[1] :]
reward_critic_loss = self.critic_loss_fn(
reward_values,
old_reward_values,
reward_returns,
sequence_mask,
)
return reward_critic_loss

140
slm/examples/RLHF/models/score_model.py
View File

@@ -1,140 +0,0 @@
# Copyright (c) 2023 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from __future__ import annotations
from typing import Any
import paddle
from paddle import nn
import paddlenlp
from paddlenlp.transformers import (
LlamaConfig,
LlamaModel,
LlamaPretrainedModel,
PretrainedConfig,
PretrainedModel,
)
from paddlenlp.transformers.conversion_utils import (
StateDictNameMapping,
init_name_mappings,
)
from .score_model_utils import ScoreModelMixin, ScoreModelOutput
class LlamaModelForScore(ScoreModelMixin, LlamaPretrainedModel):
_keys_to_ignore_on_load_missing = ["lm_head.weight"]
def __init__(self, config: PretrainedConfig, **kwargs: Any) -> None:
super().__init__(config)
self.llama = LlamaModel(config)
# config.architectures = [self.__class__.__name__]
self.init_score_head(config, hidden_size=config.hidden_size, **kwargs)
def get_input_embeddings(self) -> nn.Embedding:
return self.llama.embed_tokens
def set_input_embeddings(self, value: nn.Embedding) -> None:
self.llama.embed_tokens = value
def get_decoder(self) -> PretrainedModel:
return self.llama
def set_decoder(self, decoder: PretrainedModel) -> None:
self.llama = decoder
def forward( # pylint: disable=too-many-arguments
self,
input_ids: paddle.Tensor,
attention_mask: paddle.Tensor,
position_ids: paddle.Tensor | None = None,
past_key_values: list[paddle.Tensor] | None = None,
inputs_embeds: paddle.Tensor | None = None,
use_cache: bool | None = None,
output_attentions: bool | None = None,
output_hidden_states: bool | None = None,
return_dict: bool | None = None,
) -> tuple[paddle.Tensor, paddle.Tensor] | ScoreModelOutput:
assert attention_mask is not None
output_attentions = output_attentions if output_attentions is not None else self.config.output_attentions
output_hidden_states = (
output_hidden_states if output_hidden_states is not None else self.config.output_hidden_states
)
return_dict = return_dict if return_dict is not None else self.config.use_return_dict
outputs = self.llama(
input_ids=input_ids,
attention_mask=attention_mask,
position_ids=position_ids,
past_key_values=past_key_values,
inputs_embeds=inputs_embeds,
use_cache=use_cache,
output_attentions=output_attentions,
output_hidden_states=output_hidden_states,
return_dict=return_dict,
)
hidden_states = outputs[0] # size = (B, L, E)
return self.get_score(
hidden_states,
attention_mask=attention_mask,
position_ids=position_ids,
return_dict=return_dict,
)
@classmethod
def _get_name_mappings(cls, config: LlamaConfig) -> list[StateDictNameMapping]:
mappings: list[StateDictNameMapping] = []
model_mappings = [
["embed_tokens.weight"],
["norm.weight"],
]
for layer_index in range(config.num_hidden_layers):
layer_mappings = [
[f"layers.{layer_index}.self_attn.q_proj.weight", None, "transpose"],
[f"layers.{layer_index}.self_attn.k_proj.weight", None, "transpose"],
[f"layers.{layer_index}.self_attn.v_proj.weight", None, "transpose"],
[f"layers.{layer_index}.self_attn.o_proj.weight", None, "transpose"],
[f"layers.{layer_index}.self_attn.rotary_emb.inv_freq"],
[f"layers.{layer_index}.mlp.gate_proj.weight", None, "transpose"],
[f"layers.{layer_index}.mlp.down_proj.weight", None, "transpose"],
[f"layers.{layer_index}.mlp.up_proj.weight", None, "transpose"],
[f"layers.{layer_index}.input_layernorm.weight"],
[f"layers.{layer_index}.post_attention_layernorm.weight"],
]
model_mappings.extend(layer_mappings)
init_name_mappings(mappings=model_mappings)
# base-model prefix "LlamaModel"
if "LlamaModel" not in config.architectures:
for mapping in model_mappings:
mapping[0] = "model." + mapping[0]
mapping[1] = "llama." + mapping[1]
model_mappings.append(["lm_head.weight", "lm_head.weight", "transpose"])
model_mappings.extend(
[
["score_head.weight", "score_head.weight", "transpose"],
["normalizer.var", "normalizer.var"],
["normalizer.mean", "normalizer.mean"],
["normalizer.count", "normalizer.count"],
]
)
mappings = [StateDictNameMapping(*mapping, index=index) for index, mapping in enumerate(model_mappings)]
return mappings
paddlenlp.transformers.LlamaModelForScore = LlamaModelForScore

417
slm/examples/RLHF/models/score_model_utils.py
View File

@@ -1,417 +0,0 @@
# Copyright (c) 2023 PaddlePaddle Authors. All Rights Reserved.
# Copyright 2023 PKU-Alignment Team. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Utilities for score models."""
from __future__ import annotations
import importlib
import io
import json
from abc import abstractmethod
from collections import OrderedDict
from dataclasses import dataclass
from typing import Any, Literal
import paddle
import paddle.distributed as dist
import paddle.nn as nn
from paddlenlp.transformers import PretrainedConfig
from paddlenlp.transformers.auto.modeling import (
MAPPING_NAMES,
_BaseAutoModelClass,
get_init_configurations,
get_name_mapping,
is_standard_config,
)
from paddlenlp.transformers.model_outputs import ModelOutput
class AutoModelForScore(_BaseAutoModelClass):
CONFIGURATION_MODEL_MAPPING = get_init_configurations()
_pretrained_model_dict = CONFIGURATION_MODEL_MAPPING
# "ForScore" might be more consistent with other tasks, while this suffix is
# consistent with Beaver now.
_task_suffix = "ModelForScore"
_name_mapping: OrderedDict[str, Any] = get_name_mapping(_task_suffix)
_score_module_name: str = "models.score_model"
@classmethod
def _get_model_class_from_config(cls, pretrained_model_name_or_path, config_file_path, config=None):
if config is None:
with io.open(config_file_path, encoding="utf-8") as f:
config = json.load(f)
# Get class name corresponds to this configuration
if is_standard_config(config):
architectures = config["architectures"]
init_class = architectures.pop() if len(architectures) > 0 else None
else:
init_class = config.pop("init_class", None)
# Get class name corresponds to this task, since we might init Score
# model with CausalLM model.
init_class = init_class[:-5] if init_class is not None and init_class.endswith("Model") else init_class
model_name = None
if init_class:
for model_flag, name in MAPPING_NAMES.items():
if model_flag in init_class:
model_name = model_flag + "Model"
break
if model_name is None:
raise AttributeError(
f"Unable to parse 'architectures' or 'init_class' from {config_file_path}. Also unable to infer model class from 'pretrained_model_name_or_path'"
)
init_class = cls._name_mapping[model_name + "_Import_Class"]
# module_name = cls._name_mapping[init_class]
module_name = cls._score_module_name
import_class = importlib.import_module(module_name)
model_class = getattr(import_class, init_class)
return model_class
@classmethod
def from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs):
return cls._from_pretrained(pretrained_model_name_or_path, *model_args, **kwargs)
@dataclass
class ScoreModelOutput(ModelOutput):
"""
Output of the score model.
Args:
scores (:obj:`torch.FloatTensor` of shape :obj:`(batch_size, score_dim, sequence_length)`):
Prediction scores of the score model.
end_scores (:obj:`torch.FloatTensor` of shape :obj:`(batch_size, score_dim)`):
Prediction scores of the end of the sequence.
"""
scores: paddle.Tensor | None = None # size = (B, L, D)
end_scores: paddle.Tensor | None = None # size = (B, D)
class ScoreModelMixin:
"""Base class for score models."""
score_head: nn.Linear
normalizer: Normalizer
do_normalize: bool = False
normalize_function: NormalizeFunction = "affine"
_initialized: bool = False
def init_score_head(self, config: PretrainedConfig, hidden_size: int, **kwargs: Any) -> None:
"""Initialize the score head."""
if self._initialized:
return
config.score_dim = kwargs.pop("score_dim", getattr(config, "score_dim", 1))
config.bias = kwargs.pop("bias", getattr(config, "bias", False))
config.score_type = kwargs.pop("score_type", getattr(config, "score_type", "reward"))
if config.score_type == "reward":
self.normalize_function = "affine"
elif config.score_type == "cost":
self.normalize_function = "scale"
elif config.score_type == "critic":
self.normalize_function = "identity"
else:
raise ValueError(
f"Invalid score type: {config.score_type}. Expected one of 'reward', 'cost', or 'critic'.",
)
config.do_normalize = kwargs.pop(
"do_normalize",
getattr(config, "do_normalize", False),
)
self.do_normalize = config.do_normalize
config.normalizer_type = kwargs.pop(
"normalizer_type",
getattr(config, "normalizer_type", None),
)
if config.normalizer_type not in {"RunningMeanStd", "ExponentialMovingAverage", None}:
raise ValueError(
f"Invalid norm type: {config.normalizer_type}."
"Expected one of 'RunningMeadStd', 'ExponentialMovingAverage', or None.",
)
if config.normalizer_type == "ExponentialMovingAverage":
config.momentum = kwargs.pop("momentum", getattr(config, "momentum", None))
momentum = getattr(config, "momentum", None)
self.score_head = nn.Linear(hidden_size, config.score_dim, bias_attr=config.bias)
self.normalizer = Normalizer.instantiate(
normalizer_type=config.normalizer_type,
normalize_function=self.normalize_function,
shape=(config.score_dim,),
momentum=momentum,
)
mean = getattr(config, "mean", None)
var = getattr(config, "var", None)
self.normalizer.set_mean_var(mean, var)
self._initialized = True
def get_score(
self,
hidden_state: paddle.Tensor, # size = (B, L, E)
attention_mask: paddle.Tensor | None = None, # size = (B, L)
position_ids: paddle.Tensor | None = None, # size = (B, L)
return_dict: bool | None = None,
) -> ScoreModelOutput:
"""Forward pass of the score model."""
scores = self.score_head(hidden_state) # size = (B, L, D)
if position_ids is not None:
first_pos = paddle.arange(hidden_state.shape[0]).unsqueeze(-1)
# Take left padding into account, which has 0s in left and max_len
# in right.
left_pad_mask = position_ids == 0
# position_ids = paddle.where(
# left_pad_mask, position_ids, position_ids + left_pad_mask.sum(-1, keepdim=True) - 1
# )
# the above limits right padding must not be 0s, the following suits
# to both left and right padding with 0s
left_pad_num = (
paddle.where(left_pad_mask, position_ids.shape[-1] + 100, position_ids).argmin(axis=-1, keepdim=True)
- 1
)
position_ids = left_pad_num + position_ids
second_pos = paddle.max(position_ids, axis=-1, keepdim=True)
end_pos = paddle.stack([first_pos, second_pos], axis=-1).squeeze(1)
end_score = scores.gather_nd(end_pos)
else:
# attention_mask passed from pipeline pre-stage is shaped (bs, 1, seq_len, seq_len)
assert attention_mask is not None and len(attention_mask.shape) == 2
end_score = []
end_pos = []
for i in range(hidden_state.shape[0]):
end_index = attention_mask[i].nonzero()[-1].item()
end_pos.append((i, end_index))
end_score.append(scores[i, end_index]) # size = (D,)
end_score = paddle.stack(end_score, axis=0) # size = (B, D)
if self.training and self.do_normalize:
if dist.is_initialized():
gathered_end_score_list = []
try:
# gather among data parallel group
hcg = dist.fleet.get_hybrid_communicate_group()
group = hcg.get_sharding_parallel_group()
dist.all_gather(gathered_end_score_list, end_score, group)
except:
dist.all_gather(gathered_end_score_list, end_score)
gathered_end_score = paddle.concat(gathered_end_score_list, axis=0)
self.normalizer.update(gathered_end_score)
else:
self.normalizer.update(end_score)
self.config.mean = self.normalizer.mean.tolist()
self.config.var = self.normalizer.var.tolist()
if self.do_normalize:
scores = self.normalizer.normalize(scores)
end_score = self.normalizer.normalize(end_score)
if not return_dict:
return scores, end_score
return ScoreModelOutput(
scores=scores, # size = (B, L, D)
end_scores=end_score, # size = (B, D)
)
def set_normalize(self, mode: bool = True) -> None:
if self.do_normalize == mode:
return
self.do_normalize = self.config.do_normalize = mode
NormalizeFunction = Literal["affine", "scale", "translate", "identity"]
NormalizerType = Literal["RunningMeanStd", "ExponentialMovingAverage"]
class Normalizer(nn.Layer):
"""Normalize input to have zero mean and unit variance."""
mean: paddle.Tensor
var: paddle.Tensor
count: paddle.Tensor
normalize_function: NormalizeFunction
def __init__(
self,
normalize_function: NormalizeFunction,
shape: tuple[int, ...],
device: str | None = None,
) -> None:
"""Initialize."""
super().__init__()
if normalize_function not in {"affine", "scale", "translate", "identity"}:
raise ValueError(
f"Invalid normalization function type: {normalize_function}. ",
'Expected one of "affine", "scale", "translate", "identity".',
)
self.normalize_function = normalize_function
self.register_buffer("mean", paddle.zeros(shape, dtype=paddle.get_default_dtype())) # align torch zeros/ones
self.register_buffer("var", paddle.ones(shape, dtype=paddle.get_default_dtype())) # align torch zeros/ones
self.register_buffer("count", paddle.zeros(1, dtype=paddle.int64))
@abstractmethod
def update(self, data: paddle.Tensor) -> None:
"""Update mean and variance."""
raise NotImplementedError
@property
def std(self) -> paddle.Tensor:
"""Return standard deviation."""
return self.var.sqrt()
def set_mean_var(
self,
mean: paddle.Tensor | list[float] | tuple[float, ...] | None,
var: paddle.Tensor | list[float] | tuple[float, ...] | None,
) -> None:
"""Set mean and variance."""
mean = paddle.to_tensor(mean, dtype=self.mean.dtype, place=self.mean.place) if mean is not None else self.mean
var = paddle.to_tensor(var, dtype=self.var.dtype, place=self.var.place) if var is not None else self.var
assert mean.shape == self.mean.shape
assert var.shape == self.var.shape
self.mean = mean
self.var = var
def forward(
self,
data: paddle.Tensor,
epsilon=1e-8,
) -> paddle.Tensor:
"""Update and normalize input."""
if self.training:
self.update(data)
return self.normalize(data, epsilon=epsilon)
def normalize(
self,
data: paddle.Tensor,
epsilon=1e-8,
) -> paddle.Tensor:
"""Normalize input."""
if self.normalize_function == "affine":
return (data - self.mean.detach()) / (self.std.detach() + epsilon)
if self.normalize_function == "scale":
return data / (self.std.detach() + epsilon)
if self.normalize_function == "translate":
return data - self.mean.detach()
if self.normalize_function == "identity":
return data
raise ValueError(
f"Invalid normalization function type: {self.normalize_function}. ",
'Expected one of "affine", "scale", "translate", "identity".',
)
@classmethod
def instantiate(
cls,
normalizer_type: NormalizerType | None,
normalize_function: NormalizeFunction,
shape: tuple[int, ...],
device: str | None = None,
**kwargs: Any,
) -> Normalizer:
"""Get a normalizer."""
if normalizer_type == "RunningMeanStd":
return RunningMeanStd(
normalize_function,
shape=shape,
device=device,
)
if normalizer_type == "ExponentialMovingAverage":
return ExponentialMovingAverage(
normalize_function,
shape=shape,
device=device,
**kwargs,
)
if normalizer_type is None:
return IdentityNormalizer(
normalize_function,
shape=shape,
device=device,
)
raise ValueError(
f"Invalid normalization function type: {normalizer_type}. "
'Expected one of "RunningMeanStd", "ExponentialMovingAverage".',
)
class RunningMeanStd(Normalizer):
"""Running mean and standard deviation."""
def update(self, data: paddle.Tensor) -> None:
"""Update mean and variance."""
batch_mean = data.mean(0)
batch_var = data.var(0)
batch_count = data.shape[0]
delta = batch_mean - self.mean
total_count = self.count + batch_count
new_mean = self.mean + delta * batch_count / total_count
m_a = self.var * self.count
m_b = batch_var * batch_count
m2 = ( # pylint: disable=invalid-name
m_a + m_b + paddle.square(delta) * (self.count * batch_count / total_count)
)
new_var = m2 / total_count
self.mean = new_mean
self.var = new_var
self.count = total_count
class ExponentialMovingAverage(Normalizer):
"""Exponential moving average."""
def __init__(
self,
normalize_function: NormalizeFunction,
shape: tuple[int, ...],
device: str | None = None,
momentum: float = 0.9,
) -> None:
super().__init__(normalize_function, shape=shape, device=device)
self.momentum = momentum
def update(self, data: paddle.Tensor) -> None:
"""Update mean and variance."""
batch_mean = data.mean(0)
batch_var = data.var(0)
batch_count = data.shape[0]
self.mean = self.momentum * self.mean + (1.0 - self.momentum) * batch_mean
self.var = self.momentum * self.var + (1.0 - self.momentum) * batch_var
self.count += batch_count # pylint: disable=no-member
class IdentityNormalizer(Normalizer):
"""Identity normalizer."""
def update(self, data: paddle.Tensor) -> None:
"""Update mean and variance."""
self.count += data.shape[0] # pylint: disable=no-member

57
slm/examples/RLHF/ppo_config.json
View File

@@ -1,57 +0,0 @@
{
"train_datasets": "PKU-SafeRLHF/train",
"eval_datasets": "PKU-SafeRLHF/test",
"ptx_datasets": "alpaca",
"actor_model_name_or_path": "PKU-Alignment/alpaca-7b-reproduced",
"reward_model_name_or_path": "PKU-Alignment/beaver-7b-v1.0-reward",
"output_dir": "/root/paddlejob/workspace/guosheng/ckpts/ppo-reshard-sd38",
"max_length": 512,
"top_p": 0.8,
"temperature": 1.0,
"num_return_sequences":1,
"repetition_penalty": 1.0,
"num_train_epochs": 1,
"update_iters": 1,
"per_device_prompt_batch_size": 16,
"per_device_train_batch_size": 16,
"gradient_accumulation_steps": 1,
"learning_rate": 1e-5,
"weight_decay": 0.01,
"lr_scheduler_type": "cosine",
"warmup_ratio": 0.03,
"recompute": true,
"critic_learning_rate": 5e-6,
"critic_weight_decay": 0.0,
"critic_lr_scheduler_type": "constant",
"critic_warmup_ratio": 0.03,
"critic_recompute": true,
"normalize_reward": false,
"kl_coeff": 0.02,
"clip_range_ratio": 0.2,
"clip_range_score": 50.0,
"clip_range_value": 5.0,
"ptx_coeff": 16.0,
"per_device_eval_batch_size": 16,
"logging_steps": 1,
"evaluation_strategy": "steps",
"eval_steps": 100,
"save_strategy": "epoch",
"save_steps": 100000,
"bf16": true,
"fp16_opt_level": "O2",
"do_train": true,
"do_eval": true,
"disable_tqdm": true,
"save_total_limit": 1,
"sharding_parallel_degree": 4,
"sharding": "stage1",
"tensor_parallel_degree": 2,
"pipeline_parallel_degree": 1,
"pipeline_parallel_config": "disable_p2p_cache_shape",
"max_grad_norm": 1.0,
"adam_beta1": 0.9,
"adam_beta2": 0.95,
"dataloader_drop_last": false,
"eval_mode": "",
"offload_level": "freeze_model"
}

511
slm/examples/RLHF/ppo_main.py
View File

@@ -1,511 +0,0 @@
# Copyright (c) 2023 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import copy
import os
import sys
import types
from dataclasses import dataclass, field
from functools import partial
from typing import Any, Dict, Tuple
import paddle
from data import PromptOnlyDataset, SupervisedDataset, parse_dataset
from models import AutoModelForScore
from models.score_model import LlamaModelForScore # noqa
from ppo_trainer import PPOTrainer, cleanup_tensor_space, offload_tensor_to_cpu
from paddlenlp.trainer import PdArgumentParser, TrainingArguments, get_last_checkpoint
from paddlenlp.transformers import (
AutoConfig,
AutoModelForCausalLM,
AutoTokenizer,
LlamaTokenizer,
)
from paddlenlp.utils.log import logger
@dataclass
class TrainingArguments(TrainingArguments):
kl_coeff: float = field(
default=0.02,
metadata={"help": "The coefficient for the KL divergence between the reference and actor policy."},
)
clip_range_ratio: float = field(
default=0.2,
metadata={
"help": "The clipping range for ratio between the old and new policy. This is the epsilon parameter in the PPO algorithm."
},
)
clip_range_score: float = field(
default=50.0,
metadata={
"help": "The clipping range for the output of the score model. The reward is clipped into [-clip_range_score, clip_range_score]."
},
)
clip_range_value: float = field(
default=5.0,
metadata={
"help": "The clipping range for the value function. The value is clipped into [value_estimate - clip_range_value, value_estimate + clip_range_value] during training."
},
)
ptx_coeff: float = field(
default=0.0,
metadata={"help": "The coefficient for the ptx loss."},
)
update_iters: int = field(
default=1,
metadata={"help": "The number of repeated updates on a generated batch."},
)
critic_learning_rate: float = field(
default=None,
metadata={"help": "Initial learning rate (after the potential warmup period) for the critic model training."},
)
critic_weight_decay: float = field(
default=None,
metadata={"help": "Weight decay to for the critic model training."},
)
critic_lr_scheduler_type: str = field(
default=None,
metadata={"help": "The scheduler type for critic model."},
)
critic_warmup_ratio: float = field(
default=None,
metadata={"help": "Ratio of warm steps over total training steps for the critic lr scheduler."},
)
critic_recompute: bool = field(
default=None,
metadata={"help": "Enable gradient checkpointing for critic model."},
)
normalize_reward: bool = field(
default=None,
metadata={"help": "Whether to normalize the reward during RL training."},
)
temperature: float = field(
default=1.0,
metadata={"help": "The value used to module the next token probabilities."},
)
# top_k: int = field(
# default=1,
# metadata={"help": "top_k"},
# )
top_p: float = field(
default=0.8,
metadata={
"help": "If set to float < 1, only the smallest set of most probable tokens with probabilities that add up to`top_p` or higher are kept for generation."
},
)
num_return_sequences: int = field(
default=1,
metadata={"help": "The number of independently computed returned sequences for each element in the batch."},
)
repetition_penalty: float = field(
default=1.0,
metadata={"help": "The parameter for repetition penalty. 1.0 means no penalty."},
)
per_device_prompt_batch_size: int = field(
default=16,
metadata={"help": "Batch size (per device) for the training dataloader."},
)
eval_mode: str = field(
default=None,
metadata={
"help": "eval mode for actor model and reward_critic_model, optional for: None, single, tensor_parallel."
},
)
offload_level: str = field(
default="",
metadata={"help": "Offload model, optional for: eval, reward, optimizer, train_model"},
)
use_fusemt: bool = field(
default=True,
metadata={"help": "use inference model to speedup in rollout generation"},
)
# save_generation_output: bool = field(
# default=False,
# metadata={"help": "Whether to save generated text to file when eval"},
# )
@dataclass
class ModelArgument:
actor_model_name_or_path: str = field(
default=None, metadata={"help": "Built-in pretrained model name or the path to local model."}
)
reward_model_name_or_path: str = field(
default=None, metadata={"help": "Built-in pretrained model name or the path to local model."}
)
reward_critic_model_name_or_path: str = field(
default=None, metadata={"help": "Built-in pretrained model name or the path to local model."}
)
use_flash_attention: bool = field(default=False, metadata={"help": "Whether to use flash attention"})
# # LoRA related parameters
# lora: bool = field(default=False, metadata={"help": "Whether to use LoRA technique"})
# lora_path: str = field(default=None, metadata={"help": "Initialize lora state dict."})
# lora_rank: int = field(default=8, metadata={"help": "Lora attention dimension"})
# # prefix tuning related parameters
# prefix_tuning: bool = field(default=False, metadata={"help": "Whether to use Prefix technique"})
# num_prefix_tokens: int = field(default=128, metadata={"help": "Number of prefix tokens"})
@dataclass
class DataArgument:
train_datasets: str = field(default=None, metadata={"help": "Dataset name(s) registered in the raw dataset."})
eval_datasets: str = field(default=None, metadata={"help": "Dataset name(s) registered in the raw dataset."})
eval_split_ratio: float = field(default=None, metadata={"help": "Ratio of eval data to train data"})
ptx_datasets: str = field(default=None, metadata={"help": "Dataset name(s) registered in the raw dataset."})
max_length: int = field(
default=2048,
metadata={
"help": "The maximum length that model input tokens can have. When intokens is set to True, it's also the maximum length for InTokens data stream"
},
)
@property
def parsed_train_datasets(self) -> Tuple[str, Dict[str, Any]]:
"""Parse dataset path and its proportion and optionally additional arguments from `train_datasets`."""
return [parse_dataset(string) for string in self.train_datasets.split(",")]
@property
def parsed_eval_datasets(self) -> Tuple[str, Dict[str, Any]]:
"""Parse dataset path and its proportion and optionally additional arguments from `eval_datasets`."""
if self.eval_datasets is None:
return None
return [parse_dataset(string) for string in self.eval_datasets.split(",")]
@property
def parsed_ptx_datasets(self) -> Tuple[str, Dict[str, Any]]:
"""Parse dataset path and its proportion and optionally additional arguments from `ptx_datasets`."""
if self.ptx_datasets is None:
return None
return [parse_dataset(string) for string in self.ptx_datasets.split(",")]
def main():
# Arguments
parser = PdArgumentParser((ModelArgument, DataArgument, TrainingArguments))
if len(sys.argv) == 2 and sys.argv[1].endswith(".json"):
model_args, data_args, training_args = parser.parse_json_file(json_file=os.path.abspath(sys.argv[1]))
else:
model_args, data_args, training_args = parser.parse_args_into_dataclasses()
training_args.print_config(model_args, "Model")
training_args.print_config(data_args, "Data")
if training_args.eval_mode is not None and len(training_args.eval_mode) == 0:
training_args.eval_mode = None
if training_args.eval_mode is None and training_args.offload_level is not None:
training_args.offload_level = training_args.offload_level.replace("eval", "")
# Setup GPU & distributed training
paddle.set_device(training_args.device)
logger.warning(
f"Process rank: {training_args.local_rank}, device: {training_args.device}, world_size: {training_args.world_size}, "
+ f"distributed training: {bool(training_args.local_rank != -1)}, 16-bits training: {training_args.fp16 or training_args.bf16}"
)
# Detecting last checkpoint.
last_checkpoint = None
if os.path.isdir(training_args.output_dir) and training_args.do_train and not training_args.overwrite_output_dir:
last_checkpoint = get_last_checkpoint(training_args.output_dir)
if last_checkpoint is None and len(os.listdir(training_args.output_dir)) > 1:
raise ValueError(
f"Output directory ({training_args.output_dir}) already exists and is not empty. "
"Use --overwrite_output_dir to overcome."
)
if last_checkpoint is not None and training_args.resume_from_checkpoint is None:
logger.info(
f"Checkpoint detected, resuming training at {last_checkpoint}. To avoid this behavior, change "
"the `--output_dir` or add `--overwrite_output_dir` to train from scratch."
)
# Load model
if training_args.fp16_opt_level == "O2":
if training_args.fp16:
dtype = "float16"
elif training_args.bf16:
dtype = "bfloat16"
else:
raise ValueError("Please specific dtype: --fp16 or --bf16")
else:
dtype = "float32"
training_args.max_length = data_args.max_length
model_class_lm, model_class_score = AutoModelForCausalLM, AutoModelForScore
if training_args.pipeline_parallel_degree > 1:
from models.model_pp import LlamaPolicyPipe, LlamaValuePipe
model_class_lm = LlamaPolicyPipe
model_class_score = LlamaValuePipe
extra_args = {
"ptx_coeff": training_args.ptx_coeff,
"clip_range_ratio": training_args.clip_range_ratio,
}
else:
# non-pipe modelForCausalLM does not accept extra_args and use other ways
# (StepTrainer.create_criterion) to set hyper-parameters
extra_args = {}
# actor model
model_config = AutoConfig.from_pretrained(
model_args.actor_model_name_or_path,
tensor_parallel_output=False,
tensor_parallel_degree=training_args.tensor_parallel_degree,
tensor_parallel_rank=training_args.tensor_parallel_rank,
dtype=dtype,
)
if hasattr(model_config, "use_flash_attention"):
model_config.use_flash_attention = model_args.use_flash_attention
# model_config.num_hidden_layers = 2
actor_model = model_class_lm.from_pretrained(
model_args.actor_model_name_or_path,
config=model_config,
**extra_args,
# ptx_coeff=training_args.ptx_coeff,
# clip_range_ratio=training_args.clip_range_ratio,
)
if training_args.eval_mode is not None:
config = copy.deepcopy(actor_model.config)
if training_args.eval_mode == "single":
config.tensor_parallel_degree = -1
config.tensor_parallel_rank = 0
actor_eval_model = AutoModelForCausalLM.from_config(config)
# TODO(guosheng): AutoModel (in `_get_model_class_from_config`) pop out
# architecture which is necessary for infer predictor currently
config.architectures = actor_model.config.architectures
# actor_eval_model = AutoModelForCausalLM.from_pretrained(model_args.actor_model_name_or_path, config=config)
else:
actor_eval_model = None
# todo reference model
if training_args.eval_mode is not None:
config = copy.deepcopy(model_config)
if training_args.eval_mode == "single":
config.tensor_parallel_degree = -1
config.tensor_parallel_rank = 0
actor_reference_model = AutoModelForCausalLM.from_pretrained(
model_args.actor_model_name_or_path,
config=config,
)
else:
actor_reference_model = model_class_lm.from_pretrained(
model_args.actor_model_name_or_path,
config=model_config,
)
actor_tokenizer = AutoTokenizer.from_pretrained(
model_args.actor_model_name_or_path, model_max_length=data_args.max_length, padding_side="left"
)
# reward model
model_config = AutoConfig.from_pretrained(
model_args.reward_model_name_or_path,
tensor_parallel_output=False,
tensor_parallel_degree=training_args.tensor_parallel_degree,
tensor_parallel_rank=training_args.tensor_parallel_rank,
dtype=dtype,
)
if hasattr(model_config, "use_flash_attention"):
model_config.use_flash_attention = model_args.use_flash_attention
# model_config.num_hidden_layers = 2
# todo
if training_args.eval_mode is not None:
config = copy.deepcopy(model_config)
if training_args.eval_mode == "single":
config.tensor_parallel_degree = -1
config.tensor_parallel_rank = 0
reward_model = AutoModelForScore.from_pretrained(
model_args.reward_model_name_or_path,
config=config,
score_type="reward",
do_normalize=training_args.normalize_reward,
)
else:
reward_model = model_class_score.from_pretrained(
model_args.reward_model_name_or_path,
config=model_config,
score_type="reward",
do_normalize=training_args.normalize_reward,
)
reward_tokenizer = AutoTokenizer.from_pretrained(
model_args.reward_model_name_or_path, model_max_length=data_args.max_length, padding_side="right"
)
# critic model
if model_args.reward_critic_model_name_or_path is None:
model_args.reward_critic_model_name_or_path = model_args.reward_model_name_or_path
reward_critic_model = model_class_score.from_pretrained(
model_args.reward_critic_model_name_or_path,
config=model_config,
score_type="critic",
do_normalize=False,
clip_range_value=training_args.clip_range_value,
)
reward_critic_tokenizer = AutoTokenizer.from_pretrained(
model_args.reward_critic_model_name_or_path, model_max_length=data_args.max_length, padding_side="left"
)
if training_args.eval_mode is not None:
config = copy.deepcopy(reward_critic_model.config)
if training_args.eval_mode == "single":
config.tensor_parallel_degree = -1
config.tensor_parallel_rank = 0
reward_critic_eval_model = AutoModelForScore.from_config(config)
# reward_critic_eval_model = AutoModelForScore.from_pretrained(
# model_args.reward_critic_model_name_or_path,config=model_config
# )
else:
reward_critic_eval_model = None
# # actor model
# model_config = AutoConfig.from_pretrained(
# model_args.actor_model_name_or_path,
# tensor_parallel_output=False,
# tensor_parallel_degree=training_args.tensor_parallel_degree,
# tensor_parallel_rank=training_args.tensor_parallel_rank,
# dtype=dtype,
# )
# model_config.num_hidden_layers = 2
# if hasattr(model_config, "use_flash_attention"):
# model_config.use_flash_attention = model_args.use_flash_attention
# actor_model = AutoModelForCausalLM.from_pretrained(
# model_args.actor_model_name_or_path,
# config=model_config,
# )
#
# if training_args.eval_mode is not None:
# config = copy.deepcopy(actor_model.config)
# if training_args.eval_mode == "single":
# config.tensor_parallel_degree = -1
# config.tensor_parallel_rank = 0
# actor_eval_model = AutoModelForCausalLM.from_config(config)
# else:
# actor_eval_model = None
#
# # reference model
# actor_reference_model = AutoModelForCausalLM.from_pretrained(
# model_args.actor_model_name_or_path,
# config=model_config,
# )
# actor_tokenizer = AutoTokenizer.from_pretrained(
# model_args.actor_model_name_or_path, model_max_length=data_args.max_length, padding_side="left"
# )
#
# # reward model
# model_config = AutoConfig.from_pretrained(
# model_args.reward_model_name_or_path,
# tensor_parallel_output=False,
# tensor_parallel_degree=training_args.tensor_parallel_degree,
# tensor_parallel_rank=training_args.tensor_parallel_rank,
# dtype=dtype,
# )
# model_config.num_hidden_layers = 2
# if hasattr(model_config, "use_flash_attention"):
# model_config.use_flash_attention = model_args.use_flash_attention
# reward_model = AutoModelForScore.from_pretrained(
# model_args.reward_model_name_or_path,
# config=model_config,
# score_type="reward",
# do_normalize=training_args.normalize_reward,
# )
# reward_tokenizer = AutoTokenizer.from_pretrained(
# model_args.reward_model_name_or_path, model_max_length=data_args.max_length, padding_side="right"
# )
#
# # critic model
# if model_args.reward_critic_model_name_or_path is None:
# model_args.reward_critic_model_name_or_path = model_args.reward_model_name_or_path
# reward_critic_model = AutoModelForScore.from_pretrained(
# model_args.reward_critic_model_name_or_path, config=model_config, score_type="critic", do_normalize=False
# )
# reward_critic_tokenizer = AutoTokenizer.from_pretrained(
# model_args.reward_critic_model_name_or_path, model_max_length=data_args.max_length, padding_side="left"
# )
#
# if training_args.eval_mode is not None:
# config = copy.deepcopy(reward_critic_model.config)
# if training_args.eval_mode == "single":
# config.tensor_parallel_degree = -1
# config.tensor_parallel_rank = 0
# reward_critic_eval_model = AutoModelForScore.from_config(config)
# else:
# reward_critic_eval_model = None
for tokenizer in [actor_tokenizer, reward_tokenizer, reward_critic_tokenizer]:
if isinstance(tokenizer, LlamaTokenizer) and tokenizer.pad_token_id is None:
tokenizer.pad_token_id = tokenizer.eos_token_id
train_ds = PromptOnlyDataset(data_args.parsed_train_datasets, tokenizer=actor_tokenizer)
if data_args.eval_datasets is None and data_args.eval_split_ratio:
train_ds, dev_ds = train_ds.split_train_test(split_ratio=data_args.eval_split_ratio)
elif data_args.eval_datasets is not None:
dev_ds = PromptOnlyDataset(data_args.parsed_eval_datasets, tokenizer=actor_tokenizer)
else:
dev_ds = None
ptx_ds = (
SupervisedDataset(data_args.parsed_ptx_datasets, tokenizer=actor_tokenizer)
if data_args.ptx_datasets is not None
else None
)
if ptx_ds is not None:
# PretrainingCriterion requires shifted inputs and labels
ptx_ds.get_collator = types.MethodType(partial(ptx_ds.get_collator.__func__, shift=True), ptx_ds)
# offload
# cleanup actor_eval_model, reward_critic_eval_model
# offload actor_reference_model reward_model
if training_args.offload_level is not None:
if "eval" in training_args.offload_level:
cleanup_tensor_space(actor_eval_model.state_dict())
cleanup_tensor_space(reward_critic_eval_model.state_dict())
if "reward" in training_args.offload_level:
# if pp mode, should lazy offload
offload_tensor_to_cpu(actor_reference_model.state_dict())
offload_tensor_to_cpu(reward_model.state_dict())
trainer = PPOTrainer(
# (policy_model, reference_model, reward_model, value_model)
# policy_model, sft_model, reward_model, value_model
# (policy_model, reference_model, reward_model, value_model,
# (policy_model, reference_model, reward_model, value_model, policy_eval_model, value_eval_model
# (actor_model, actor_reference_model, reward_model, reward_critic_model, actor_eval_model, reward_critic_eval_model
model=(
actor_model,
actor_reference_model,
reward_model,
reward_critic_model,
actor_eval_model,
reward_critic_eval_model,
),
args=training_args,
train_dataset=train_ds,
eval_dataset=dev_ds,
ptx_dataset=ptx_ds,
tokenizer=(actor_tokenizer, actor_tokenizer, reward_tokenizer, reward_critic_tokenizer),
data_collator=train_ds.get_collator(),
)
checkpoint = None
if training_args.resume_from_checkpoint is not None:
checkpoint = training_args.resume_from_checkpoint
elif last_checkpoint is not None:
checkpoint = last_checkpoint
trainer.train(resume_from_checkpoint=checkpoint)
if __name__ == "__main__":
main()

1803
slm/examples/RLHF/ppo_trainer.py
View File

File diff suppressed because it is too large Load Diff

40
slm/examples/RLHF/reward_config.json
View File

@@ -1,40 +0,0 @@
{
"model_name_or_path": "PKU-Alignment/alpaca-7b-reproduced",
"train_datasets": "PKU-SafeRLHF-30K/train",
"eval_datasets": "PKU-SafeRLHF-30K/test",
"output_dir": "/root/paddlejob/workspace/guosheng/checkpoints/llama_sft_ckpts-test",
"per_device_train_batch_size": 16,
"gradient_accumulation_steps": 1,
"per_device_eval_batch_size": 16,
"eval_accumulation_steps": 1,
"num_train_epochs": 2,
"logging_steps": 1,
"evaluation_strategy": "epoch",
"eval_steps": 1000,
"save_strategy": "epoch",
"save_steps": 1000,
"max_length": 512,
"bf16": true,
"fp16_opt_level": "O2",
"do_train": true,
"do_eval": true,
"disable_tqdm": true,
"load_best_model_at_end": false,
"metric_for_best_model": "accuracy",
"recompute": true,
"save_total_limit": 1,
"sharding_parallel_degree": 4,
"sharding": "stage2",
"weight_decay": 0.1,
"max_grad_norm": 0.0,
"adam_beta1": 0.9,
"adam_beta2": 0.95,
"lr_scheduler_type": "cosine",
"learning_rate": 2e-5,
"warmup_ratio": 0.03,
"normalize_score_during_training": false,
"normalizer_type": "ExponentialMovingAverage",
"normalizer_momentum": 0.9,
"loss_type": "sequence-wise",
"regularization": 0.001
}

202
slm/examples/RLHF/reward_main.py
View File

@@ -1,202 +0,0 @@
# Copyright (c) 2023 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import os
import sys
from dataclasses import dataclass, field
from typing import Any, Dict, Tuple
import paddle
try:
from typing import Literal
except ImportError:
from typing_extensions import Literal
from data import PreferenceDataset, parse_dataset
from models import AutoModelForScore
from reward_trainer import RewardTrainer
from paddlenlp.trainer import PdArgumentParser, TrainingArguments, get_last_checkpoint
from paddlenlp.transformers import AutoConfig, AutoTokenizer, LlamaTokenizer
from paddlenlp.utils.log import logger
@dataclass
class TrainingArguments(TrainingArguments):
loss_type: Literal["token-wise", "sequence-wise"] = field(
default="sequence-wise",
metadata={
"help": "Calculate ranking loss with all token-wise reward outputs in the sequence or the "
"sequence-wise reward output only (the reward of the last token in each sequence)."
},
)
# regularization
regularization: float = field(
default=0.0,
metadata={"help": "The regularization strength for the L2 regularization for score outputs."},
)
@dataclass
class ModelArgument:
model_name_or_path: str = field(
default=None, metadata={"help": "Built-in pretrained model name or the path to local model."}
)
normalize_score_during_training: bool = field(
default=False, metadata={"help": "Whether to normalize score during training."}
)
normalizer_type: Literal["RunningMeanStd", "ExponentialMovingAverage"] = field(
default=None, metadata={"help": "The type of the reward normalizer."}
)
normalizer_momentum: float = field(
default=None,
metadata={
"help": "The momentum use in ExponentialMovingAverage, EMA_{t+1} = momentum * x + (1 - momentum) * EMA_t."
},
)
use_flash_attention: bool = field(default=False, metadata={"help": "Whether to use flash attention"})
@property
def extra_model_kwargs(self):
"""Extra keyword arguments for initializing the model."""
return {
"score_type": "reward",
"do_normalize": self.normalize_score_during_training,
"normalizer_type": self.normalizer_type,
"momentum": self.normalizer_momentum,
}
@dataclass
class DataArgument:
dataset_name_or_path: str = field(default=None, metadata={"help": "Name or path for dataset"})
task_name: str = field(default=None, metadata={"help": "Additional name to select a more specific task."})
train_datasets: str = field(default=None, metadata={"help": "Dataset name(s) registered in the raw dataset."})
eval_datasets: str = field(default=None, metadata={"help": "Dataset name(s) registered in the raw dataset."})
max_length: int = field(
default=2048,
metadata={"help": "The maximum length that model input tokens can ."},
)
eval_with_do_generation: bool = field(default=False, metadata={"help": "Whether to do generation for evaluation"})
save_generation_output: bool = field(
default=False,
metadata={"help": "Whether to save generated text to file when eval_with_do_generation set to True."},
)
lazy: bool = field(
default=False,
metadata={
"help": "Weather to return `MapDataset` or an `IterDataset`.True for `IterDataset`. False for `MapDataset`."
},
)
@property
def parsed_train_datasets(self) -> Tuple[str, Dict[str, Any]]:
"""Parse dataset path and its proportion and optionally additional arguments from `train_datasets`."""
return [parse_dataset(string) for string in self.train_datasets.split(",")]
@property
def parsed_eval_datasets(self) -> Tuple[str, Dict[str, Any]]:
"""Parse dataset path and its proportion and optionally additional arguments from `eval_datasets`."""
return [parse_dataset(string) for string in self.eval_datasets.split(",")]
def main():
# Arguments
parser = PdArgumentParser((ModelArgument, DataArgument, TrainingArguments))
if len(sys.argv) == 2 and sys.argv[1].endswith(".json"):
model_args, data_args, training_args = parser.parse_json_file(json_file=os.path.abspath(sys.argv[1]))
else:
model_args, data_args, training_args = parser.parse_args_into_dataclasses()
training_args.print_config(model_args, "Model")
training_args.print_config(data_args, "Data")
# Setup GPU & distributed training
paddle.set_device(training_args.device)
logger.warning(
f"Process rank: {training_args.local_rank}, device: {training_args.device}, world_size: {training_args.world_size}, "
+ f"distributed training: {bool(training_args.local_rank != -1)}, 16-bits training: {training_args.fp16 or training_args.bf16}"
)
# Detecting last checkpoint.
last_checkpoint = None
if os.path.isdir(training_args.output_dir) and training_args.do_train and not training_args.overwrite_output_dir:
last_checkpoint = get_last_checkpoint(training_args.output_dir)
if last_checkpoint is None and len(os.listdir(training_args.output_dir)) > 1:
raise ValueError(
f"Output directory ({training_args.output_dir}) already exists and is not empty. "
"Use --overwrite_output_dir to overcome."
)
if last_checkpoint is not None and training_args.resume_from_checkpoint is None:
logger.info(
f"Checkpoint detected, resuming training at {last_checkpoint}. To avoid this behavior, change "
"the `--output_dir` or add `--overwrite_output_dir` to train from scratch."
)
# Load model
if training_args.fp16_opt_level == "O2":
if training_args.fp16:
dtype = "float16"
elif training_args.bf16:
dtype = "bfloat16"
else:
raise ValueError("Please specific dtype: --fp16 or --bf16")
else:
dtype = "float32"
if training_args.pipeline_parallel_degree > 1:
raise ValueError("Not support pipeline parallel mode.")
else:
model_config = AutoConfig.from_pretrained(
model_args.model_name_or_path,
tensor_parallel_output=False,
tensor_parallel_degree=training_args.tensor_parallel_degree,
tensor_parallel_rank=training_args.tensor_parallel_rank,
dtype=dtype,
)
if hasattr(model_config, "use_flash_attention"):
model_config.use_flash_attention = model_args.use_flash_attention
model = AutoModelForScore.from_pretrained(
model_args.model_name_or_path, config=model_config, **model_args.extra_model_kwargs
)
tokenizer = AutoTokenizer.from_pretrained(
model_args.model_name_or_path, model_max_length=data_args.max_length, padding_side="right"
)
if isinstance(tokenizer, LlamaTokenizer) and tokenizer.pad_token_id is None:
# tokenizer.pad_token_id = tokenizer.eos_token_id
# to be consistent with PKU-Alignment/alpaca-7b-reproduced
tokenizer.pad_token_id = tokenizer.eos_token_id
train_ds = PreferenceDataset(data_args.parsed_train_datasets, tokenizer=tokenizer)
dev_ds = PreferenceDataset(data_args.parsed_eval_datasets, tokenizer=tokenizer)
trainer = RewardTrainer(
model=model,
args=training_args,
train_dataset=train_ds,
eval_dataset=dev_ds,
tokenizer=tokenizer,
data_collator=train_ds.get_collator(),
)
checkpoint = None
if training_args.resume_from_checkpoint is not None:
checkpoint = training_args.resume_from_checkpoint
elif last_checkpoint is not None:
checkpoint = last_checkpoint
trainer.train(resume_from_checkpoint=checkpoint)
if __name__ == "__main__":
main()

219
slm/examples/RLHF/reward_trainer.py
View File

@@ -1,219 +0,0 @@
# Copyright (c) 2023 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from typing import Any, Callable, Dict, List, Optional, Tuple, Union
import numpy as np
import paddle
import paddle.nn as nn
import paddle.nn.functional as F
from models import ScoreModelOutput
from paddle.io import Dataset
import paddlenlp.trainer.trainer as trainer
from paddlenlp.data import DataCollator
from paddlenlp.trainer import (
EvalPrediction,
Trainer,
TrainerCallback,
TrainingArguments,
)
from paddlenlp.trainer.utils import nested_detach
from paddlenlp.transformers import PretrainedModel, PretrainedTokenizer
_tr_acc = None
speed_metrics = trainer.speed_metrics
def patch_speed_metrics(split, start_time, num_samples=None, num_steps=None, seq_length=None):
# split: interval, train, eval, test
result = speed_metrics(split, start_time, num_samples, num_steps, seq_length)
if split not in ["train", "interval"]:
return result
# accuracy
global _tr_acc
tr_acc, total_acc_scalar, nested_gather = _tr_acc
tr_acc_scalar = nested_gather(tr_acc).mean().item()
total_acc_scalar += tr_acc_scalar
tr_acc.subtract_(tr_acc)
_tr_acc[1] = total_acc_scalar
result["accuracy"] = round(tr_acc_scalar / num_steps, 8)
if split == "train":
result["train_accuracy"] = round(total_acc_scalar / num_steps, 8)
return result
trainer.speed_metrics = patch_speed_metrics
def compute_accuracy(eval_pred) -> Dict[str, float]:
higher_end_rewards, lower_end_rewards = eval_pred
accuracy = (higher_end_rewards > lower_end_rewards).astype("float32").mean().item()
rewards = np.concatenate([higher_end_rewards, lower_end_rewards], axis=0)
reward_mean = rewards.mean().item()
reward_std = rewards.std().item()
return {
"accuracy": accuracy,
"rewards_mean": reward_mean,
"rewards_std": reward_std,
}
class RewardTrainer(Trainer):
def __init__(
self,
model: Union[PretrainedModel, nn.Layer] = None,
criterion: nn.Layer = None,
args: TrainingArguments = None,
data_collator: Optional[DataCollator] = None,
train_dataset: Optional[Dataset] = None,
eval_dataset: Union[Dataset, Dict[str, Dataset]] = None,
tokenizer: Optional[PretrainedTokenizer] = None,
compute_metrics: Optional[Callable[[EvalPrediction], Dict]] = None,
callbacks: Optional[List[TrainerCallback]] = None,
optimizers: Tuple[paddle.optimizer.Optimizer, paddle.optimizer.lr.LRScheduler] = (None, None),
preprocess_logits_for_metrics: Callable[[paddle.Tensor, paddle.Tensor], paddle.Tensor] = None,
):
if compute_metrics is None:
compute_metrics = compute_accuracy
super().__init__(
model,
criterion,
args,
data_collator,
train_dataset,
eval_dataset,
tokenizer,
compute_metrics,
callbacks,
optimizers,
preprocess_logits_for_metrics,
)
def compute_loss(self, model, inputs, return_outputs=False):
"""
How the loss is computed by Trainer. By default, all models return the loss in the first element.
Subclass and override for custom behavior.
"""
better_input_ids = inputs["better_input_ids"]
worse_input_ids = inputs["worse_input_ids"]
better_attention_mask = inputs["better_attention_mask"]
worse_attention_mask = inputs["worse_attention_mask"]
assert better_input_ids.shape[0] == worse_input_ids.shape[0], "batch size mismatch!"
batch_size = better_input_ids.shape[0]
output: ScoreModelOutput = model(
paddle.concat([better_input_ids, worse_input_ids], axis=0),
attention_mask=paddle.concat([better_attention_mask, worse_attention_mask], axis=0),
)
if isinstance(output, dict):
scores = output.scores # size = (2 * B, L, 1)
end_scores = output.end_scores # size = (2 * B, 1)
else:
scores, end_scores = output
# size = (B, L)
higher_rewards, lower_rewards = scores.squeeze(axis=-1).chunk(chunks=2, axis=0)
# size = (B,)
higher_end_rewards, lower_end_rewards = end_scores.squeeze(axis=-1).chunk(chunks=2, axis=0)
if self.args.loss_type == "token-wise":
losses = []
for i in range(batch_size):
assert not paddle.all(
paddle.equal(better_input_ids[i], worse_input_ids[i]),
).item(), "The better and worse answers are the same!"
higher_end_index = better_attention_mask[i].nonzero()[-1]
lower_end_index = worse_attention_mask[i].nonzero()[-1]
end_index = max(higher_end_index, lower_end_index)
diverge_index = (better_input_ids[i] != worse_input_ids[i]).nonzero()[0]
assert 0 <= diverge_index <= end_index, "diverge index is out of range!"
# size = (L,)
higher_truncated_rewards = higher_rewards[i, diverge_index : end_index + 1]
lower_truncated_rewards = lower_rewards[i, diverge_index : end_index + 1]
losses.append(
-F.log_sigmoid(higher_truncated_rewards - lower_truncated_rewards).mean(),
)
if self.args.regularization > 0.0:
losses[-1] = losses[-1] + self.args.regularization * (
paddle.square(lower_truncated_rewards).mean() + paddle.square(higher_truncated_rewards).mean()
)
loss = paddle.stack(losses).mean() # size = ()
elif self.args.loss_type == "sequence-wise":
loss = -F.log_sigmoid(higher_end_rewards - lower_end_rewards).mean()
if self.args.regularization > 0.0:
loss = loss + self.args.regularization * (
paddle.square(lower_end_rewards).mean() + paddle.square(higher_end_rewards).mean()
)
else:
raise ValueError(f"Unknown loss type: {self.args.loss_type}")
accuracy = (higher_end_rewards > lower_end_rewards).cast("float32").mean() # size = ()
# TODO(guosheng): use a formal way to replace this hack for accuracy track
# in training
global _tr_acc
if _tr_acc is None:
_tr_acc = [paddle.to_tensor(0.0), 0.0, self._nested_gather]
_tr_acc[0] = _tr_acc[0] + accuracy.detach()
if return_outputs:
return loss, {
"higher_end_rewards": higher_end_rewards,
"lower_end_rewards": lower_end_rewards,
"accuracy": accuracy,
}
return loss
def prediction_step(
self,
model: nn.Layer,
inputs: Dict[str, Union[paddle.Tensor, Any]],
prediction_loss_only: bool,
ignore_keys: Optional[List[str]] = None,
):
if self.args.pipeline_parallel_degree > 1:
# hack for pipeline mode
inputs = self._prepare_inputs(inputs)
return self.prediction_pipeline_step(model, inputs, prediction_loss_only, ignore_keys)
else:
inputs = self._prepare_inputs(inputs)
better_input_ids = inputs["better_input_ids"]
worse_input_ids = inputs["worse_input_ids"]
better_attention_mask = inputs["better_attention_mask"]
worse_attention_mask = inputs["worse_attention_mask"]
with paddle.no_grad():
with self.autocast_smart_context_manager():
higher_rewards = self.model(better_input_ids, better_attention_mask)
lower_rewards = self.model(worse_input_ids, worse_attention_mask)
if isinstance(higher_rewards, dict):
higher_end_rewards = higher_rewards.end_scores.squeeze(axis=-1)
lower_end_rewards = lower_rewards.end_scores.squeeze(axis=-1)
else:
higher_end_rewards = higher_rewards[-1].squeeze(axis=-1)
lower_end_rewards = lower_rewards[-1].squeeze(axis=-1)
higher_end_rewards = nested_detach(higher_end_rewards)
lower_end_rewards = nested_detach(lower_end_rewards)
return None, higher_end_rewards.cast("float32"), lower_end_rewards.cast("float32")

15
slm/examples/RLHF/run.sh
View File

@@ -1,15 +0,0 @@
# Copyright (c) 2024 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
PYTHONPATH=../../ GLOG_minloglevel=2 python3.10 -m paddle.distributed.launch --gpus "0,1,2,3,4,5,6,7" ppo_main.py ppo_config.json

125
slm/examples/RLHF/tests/run_model.py
View File

@@ -1,125 +0,0 @@
# Copyright (c) 2024 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import copy
from dataclasses import dataclass, field
import numpy
import paddle
from paddle.distributed import fleet
from ppo_trainer import Trainer, data_group_merge, data_group_split, group_rank_guard
from paddlenlp.trainer import PdArgumentParser, TrainingArguments
from paddlenlp.transformers import (
AutoConfig,
AutoModelForCausalLM,
AutoModelForCausalLMPipe,
)
@dataclass
class ModelArgument:
model_name_or_path: str = field(
default=None, metadata={"help": "Built-in pretrained model name or the path to local model."}
)
test_mode: str = field(default="export", metadata={"help": "export data_split or rank_guard."})
def test_group_rank_guard(group):
@group_rank_guard(group=group, rank=0)
def func():
tensor = paddle.randn([4, 64])
return tensor
t = func()
ret = []
paddle.distributed.stream.all_gather(ret, t, group=group)
for x in ret:
assert x._md5sum() == t._md5sum(), f"{x} {t}"
def main():
# Arguments
parser = PdArgumentParser((ModelArgument, TrainingArguments))
model_args, training_args = parser.parse_args_into_dataclasses()
hcg = fleet.get_hybrid_communicate_group()
pp_group = hcg.get_pipe_parallel_group()
tp_group = hcg.get_model_parallel_group()
if model_args.test_mode == "rank_guard":
test_group_rank_guard(tp_group)
return 0
model_config = AutoConfig.from_pretrained(
model_args.model_name_or_path,
tensor_parallel_output=False,
tensor_parallel_degree=training_args.tensor_parallel_degree,
tensor_parallel_rank=training_args.tensor_parallel_rank,
dtype="float32",
)
model_class = AutoModelForCausalLM
if training_args.pipeline_parallel_degree > 1:
model_class = AutoModelForCausalLMPipe
actor_model = model_class.from_pretrained(
model_args.model_name_or_path,
config=model_config,
)
if True: # test export_evaluate_model
# 随机初始化
config = copy.deepcopy(model_config)
if training_args.pipeline_parallel_degree <= 1:
config.tensor_parallel_degree = -1
config.tensor_parallel_rank = 0
actor_eval_model = AutoModelForCausalLM.from_config(config)
# ground truth模型
actor_gt_model = AutoModelForCausalLM.from_pretrained(model_args.model_name_or_path, config=config)
trainer = Trainer(
model=actor_model,
args=training_args,
)
trainer.export_evaluate_model(actor_model, actor_eval_model)
gp_state = actor_gt_model.state_dict()
export_state = actor_eval_model.state_dict()
for k, v in gp_state.items():
assert (
v._md5sum() == export_state[k]._md5sum()
), f"{k} groud_truth: {v.shape}, export: {export_state[k].shape}"
split_group = tp_group
if training_args.pipeline_parallel_degree > 1:
split_group = pp_group
input_ids = paddle.randint(low=1, high=50, shape=[8, 64])
paddle.distributed.broadcast(input_ids, src=0)
split_input_ids = data_group_split(input_ids, group=split_group)
ret = actor_eval_model(input_ids=split_input_ids, return_dict=True)
eval_loggits = data_group_merge(ret.logits, group=split_group)
gt_ret = actor_gt_model(input_ids=input_ids, return_dict=True)
gt_loggits = gt_ret.logits
numpy.testing.assert_almost_equal(eval_loggits.numpy(), gt_loggits.numpy(), decimal=5)
if __name__ == "__main__":
main()

68
slm/examples/RLHF/tests/test_export.py
View File

@@ -1,68 +0,0 @@
# Copyright (c) 2024 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import os
from tests.parallel_launch import TestMultipleGpus
environment_variables = {
"NCCL_ALGO": "Tree",
"NVIDIA_TF32_OVERRIDE": "0",
"NCCL_IB_TIMEOUT": "22",
"NCCL_DEBUG": "INFO",
"FLAGS_embedding_deterministic": "1",
"FLAGS_cudnn_deterministic": "1",
"Flags_mp_aysnc_allreduce": "1",
"Flags_skip_mp_c_identity": "1",
"FLAGS_shard_norm_align_dp": "0",
"FLAGS_shard_use_reduce": "1",
"test_ci_no_save_model": "1",
}
class TestExportEvalModel(TestMultipleGpus):
def setUp(self):
os.environ.update(environment_variables)
super().setUp()
def test_pptp_to_tp(self):
config = {
"output_dir": "./tmp",
"model_name_or_path": "__internal_testing__/tiny-random-llama",
"tensor_parallel_degree": 2,
"pipeline_parallel_degree": 2,
}
scripts = "tests/run_model.py"
self.run_4gpu(scripts, **config)
def test_tp_to_single(self):
config = {
"output_dir": "./tmp",
"model_name_or_path": "__internal_testing__/tiny-random-llama",
"tensor_parallel_degree": 2,
"pipeline_parallel_degree": 1,
}
scripts = "tests/run_model.py"
self.run_2gpu(scripts, **config)
def test_group_rank_guard(self):
config = {
"output_dir": "./tmp",
"model_name_or_path": "__internal_testing__/tiny-random-llama",
"tensor_parallel_degree": 2,
"pipeline_parallel_degree": 1,
"test_mode": "rank_guard",
}
scripts = "tests/run_model.py"
self.run_2gpu(scripts, **config)

662
slm/examples/RLHF/trainer_utils.py
View File

@@ -1,662 +0,0 @@
# Copyright (c) 2023 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from __future__ import annotations
import inspect
import os
import time
from contextlib import contextmanager
from typing import Dict
import numpy as np
import paddle
import tqdm
from paddle.distributed import fleet
from paddle.io import DataLoader
from paddlenlp.generation.utils import GenerationMixin
from paddlenlp.trainer.trainer import (
TRAINER_STATE_NAME,
HybridParallelOptimizer,
NlpDistributedBatchSampler,
ShardingOption,
Trainer,
TrainerCallback,
TrainerControl,
TrainerState,
TrainingArguments,
_obtain_optimizer_parameters_list,
distributed_file,
distributed_isfile,
fused_allreduce_gradients,
logger,
reshard_util,
split_inputs_sequence_dim,
)
from paddlenlp.transformers import BatchEncoding, PretrainedModel, PretrainedTokenizer
from paddlenlp.transformers.configuration_utils import PretrainedConfig
from paddlenlp.transformers.model_outputs import ModelOutput
from paddlenlp.transformers.tokenizer_utils_base import (
PaddingStrategy,
TruncationStrategy,
)
# ########## patches for Trianer ##########
def init_train_model_opt(
self: Trainer, max_steps: int, resume_from_checkpoint: bool = False, clear_master_weight: bool = False
) -> PretrainedModel:
# Copy of model/optimizer init and resuming related code in `Trainer.train`.
# NOTE: this `_load_from_checkpoint` is indeed to load model states in the
# following elif-else branches, though they are apart away in `Trainer.train`.
if not self.args.should_load_sharding_stage1_model:
self._load_from_checkpoint(resume_from_checkpoint)
# delay_optimizer_creation = (
# self.sharding is not None
# and ShardingOption.SHARD_OP in self.args.sharding
# )
delay_optimizer_creation = False
if not delay_optimizer_creation:
self.create_optimizer_and_scheduler(num_training_steps=max_steps)
if self.args.should_load_sharding_stage1_model:
model = self._wrap_model_and_load_sharded_checkpoint(resume_from_checkpoint)
elif self.args.should_save_sharding_stage1_model:
# In the non-sharded mode, should invoke _load_from_checkpoint before _wrap_model.
# In this mode, the rank0 load all params and the _wrap_model implicitly broadcast params from rank0 to the other ranks.
model = self._wrap_model(self.model_wrapped)
if self.sharding_io is not None:
assert delay_optimizer_creation is False, "delay_optimizer_creation should be False"
# the self.optimizer should be wrapped and it is done in _wrap_model
self.sharding_io.set_optimizer(self.optimizer)
# for the rest of this function `model` is the outside model, whether it was wrapped or not
if model is not self.model:
self.model_wrapped = model
if delay_optimizer_creation:
self.create_optimizer_and_scheduler(num_training_steps=max_steps)
self._load_optimizer_and_scheduler(resume_from_checkpoint)
else:
model = self._wrap_model(self.model_wrapped)
# for the rest of this function `model` is the outside model, whether it was wrapped or not
if model is not self.model:
self.model_wrapped = model
if delay_optimizer_creation:
self.create_optimizer_and_scheduler(num_training_steps=max_steps)
self._load_optimizer_and_scheduler(resume_from_checkpoint)
if ShardingOption.FULL_SHARD in self.args.sharding and clear_master_weight:
# for inference model to use Trainer sharding stage3, clear master_weight
# which is created in GroupShardedStage3.__init__
self.optimizer._master_weights = None
if self.args.device == "npu" and self.args.flatten_param_grads:
from .plugins.npu_plugin import npu_accelerate_plugin
npu_accelerate_plugin(self.optimizer)
return model
def init_train_state(
self: Trainer,
resume_from_checkpoint: bool,
train_dataloader: DataLoader,
max_steps: int,
num_train_epochs: int,
num_update_steps_per_epoch: int,
):
args = self.args
self.state = TrainerState()
self.state.epoch = 0
epochs_trained = 0
steps_trained_in_current_epoch = 0
steps_trained_progress_bar = None
# Check if continuing training from a checkpoint
if resume_from_checkpoint is not None and distributed_isfile(
os.path.join(resume_from_checkpoint, TRAINER_STATE_NAME)
):
self.state = TrainerState.load_from_json(
distributed_file(os.path.join(resume_from_checkpoint, TRAINER_STATE_NAME))
)
epochs_trained = self.state.global_step // num_update_steps_per_epoch
if not args.ignore_data_skip:
steps_trained_in_current_epoch = self.state.global_step % (num_update_steps_per_epoch)
steps_trained_in_current_epoch *= args.gradient_accumulation_steps
else:
steps_trained_in_current_epoch = 0
logger.info(" Continuing training from checkpoint, will skip to saved global_step")
logger.info(f" Continuing training from epoch {epochs_trained}")
logger.info(f" Continuing training from global step {self.state.global_step}")
if not args.ignore_data_skip:
logger.info(
f" Will skip the first {epochs_trained} epochs then the first {steps_trained_in_current_epoch} "
"batches in the first epoch. If this takes a lot of time, you can add the `--ignore_data_skip` "
"flag to your launch command, but you will resume the training on data already seen by your model."
)
if self.is_local_process_zero() and not args.disable_tqdm:
steps_trained_progress_bar = tqdm(total=steps_trained_in_current_epoch)
steps_trained_progress_bar.set_description("Skipping the first batches")
if not args.ignore_data_skip:
if isinstance(train_dataloader, paddle.io.DataLoader) and isinstance(
train_dataloader.batch_sampler, NlpDistributedBatchSampler
):
consumed_samples = (
self.state.global_step
* args.train_batch_size
* args.gradient_accumulation_steps
* args.dataset_world_size
)
train_dataloader.batch_sampler.set_epoch(consumed_samples=consumed_samples)
logger.info(f"Set DistributedBatchSampler consumed_samples to {consumed_samples}")
self.state.max_steps = int(max_steps)
self.state.num_train_epochs = num_train_epochs
self.state.is_local_process_zero = self.is_local_process_zero()
self.state.is_world_process_zero = self.is_world_process_zero()
return epochs_trained, steps_trained_in_current_epoch, steps_trained_progress_bar
def init_train_log(
self: Trainer,
num_examples: int,
num_train_epochs: int,
total_train_batch_size: int,
max_steps: int,
num_train_samples: int,
model: PretrainedModel,
):
args = self.args
logger.info("***** Running training *****")
logger.info(f" Num examples = {num_examples:,}")
logger.info(f" Num Epochs = {num_train_epochs}")
logger.info(f" Instantaneous batch size per device = {args.per_device_train_batch_size}")
logger.info(f" Total train batch size (w. parallel, distributed & accumulation) = {total_train_batch_size}")
logger.info(f" Gradient Accumulation steps = {args.gradient_accumulation_steps}")
logger.info(f" Total optimization steps = {max_steps:,}")
logger.info(f" Total num train samples = {num_train_samples:,}")
# per_device_trainable_numel = sum(p.numel().item() for p in model.parameters() if not p.stop_gradient)
# TODO: Temporary fix since Tensor.numel() not supported in distributed mode
per_device_trainable_numel = sum(np.prod(p.shape) for p in model.parameters() if not p.stop_gradient)
logger.info(f" Number of trainable parameters = {per_device_trainable_numel:,} (per device)")
if self.args.use_hybrid_parallel:
# todo fix for pipeline_parallel_degree
parts_num = max(self.args.tensor_parallel_degree, 1) * max(self.args.pipeline_parallel_degree, 1)
if parts_num > 1:
all_reduce_dtype = "int64"
if paddle.get_device().split(":")[0] in ["npu", "xpu"]:
# TODO(duanyanhui): fix when NPU all_reduce supports int64
all_reduce_dtype = "float32"
trainable_numel_tensor = paddle.to_tensor(per_device_trainable_numel, dtype=all_reduce_dtype)
paddle.distributed.all_reduce(trainable_numel_tensor)
trainable_numel = int(trainable_numel_tensor.item()) // self.args.dataset_world_size
# the numel is roughly, because the tensor parallel still hold own bias or layer_norm weight without splited
# so, the trainable numel is a little bigger than real.
logger.info(f" Number of trainable parameters = {trainable_numel:,} (all devices, roughly)")
def full_training_step(self: Trainer, inputs: Dict[str, paddle.Tensor], **kwargs):
"""
Just a copy of single training step complete code in Trainer.train while loop
which including forward+backward+step, while wraps the inputs and outputs to
make the complicated copied code no need to change. Maybe a better way is to
add fine-grained methods including these steps to Trainer which is similar to
DeepSpeed engine.
"""
# TODO(guosheng): step, steps_trained_in_current_epoch and steps_trained_progress_bar
# should use reference since they would be overwrite.
# for state update
epoch = kwargs.get("epoch", 0)
step = kwargs.get("step", 0)
steps_in_epoch = kwargs.get("steps_in_epoch", 0)
step_control = kwargs.get("step_control", 0)
# for step and progress update when resuming data
train_dataloader = kwargs.get("train_dataloader", None)
resume_from_checkpoint = kwargs.get("resume_from_checkpoint", None)
steps_trained_in_current_epoch = kwargs.get("steps_trained_in_current_epoch", 0)
steps_trained_progress_bar = kwargs.get("steps_trained_progress_bar", None)
# for eval output ignore to gather
ignore_keys_for_eval = kwargs.get("ignore_keys_for_eval", None)
tr_loss = kwargs.get("tr_loss", 0.0)
model = kwargs.get("model", self.model_wrapped)
# needed in _maybe_log_save_evaluate
self._globalstep_last_logged = getattr(self, "_globalstep_last_logged", 0)
self._globalstep_last_start_time = getattr(self, "_globalstep_last_start_time", time.time())
args = self.args
if self.args.use_hybrid_parallel and self.args.sep_parallel_degree > 1:
inputs = split_inputs_sequence_dim(inputs)
# self.timers and self.timers("read-data").stop()
os.environ["TRAINER_GLOBAL_STEP"] = str(self.state.global_step)
self.callback_handler.on_load_data_end(args, self.state, self.control, inputs=inputs)
# Skip past any already trained steps if resuming training
# for paddlenlp.utils.batch_sampler.DistributedBatchSampler
# We use consumed_samples to reset the status
if isinstance(train_dataloader, paddle.io.DataLoader) and isinstance(
train_dataloader.batch_sampler, NlpDistributedBatchSampler
):
if step == 0:
if steps_trained_progress_bar is not None:
steps_trained_progress_bar.update(steps_trained_in_current_epoch)
steps_trained_progress_bar.close()
steps_trained_progress_bar = None
self._load_rng_state(resume_from_checkpoint)
step += steps_trained_in_current_epoch
elif steps_trained_in_current_epoch > 0:
steps_trained_in_current_epoch -= 1
if steps_trained_progress_bar is not None:
steps_trained_progress_bar.update(1)
if steps_trained_in_current_epoch == 0:
self._load_rng_state(resume_from_checkpoint)
# continue
final_local_vars = locals()
for k in kwargs.keys():
if k in final_local_vars:
kwargs[k] = final_local_vars[k]
return kwargs
elif steps_trained_progress_bar is not None:
steps_trained_progress_bar.close()
steps_trained_progress_bar = None
if step_control % args.gradient_accumulation_steps == 0:
self.control = self.callback_handler.on_step_begin(args, self.state, self.control)
self.timers and self.timers("forward-backward").start()
dp_enabled = self.args.data_parallel_degree > 1 if self.args.use_hybrid_parallel else args.local_rank != -1
forbidden_no_sync = False
# stage2 and stage3 should not no_sync, because the is no DDP wrapper and no_sync API
# hybrid_parallel (tp or pp or sharding stage 1) should not no_sync
if self.args.use_hybrid_parallel:
forbidden_no_sync = True
availiable_no_sync = dp_enabled and not forbidden_no_sync
is_no_sync = (
((step_control + 1) % args.gradient_accumulation_steps != 0)
and availiable_no_sync
and args._no_sync_in_gradient_accumulation
) or (args.recompute and availiable_no_sync)
# sharding
# stage1. the same as ddp
# stage2. manualy collect gradient on dp group
dp_master_grad = self.args.world_size > 1 and self.args.amp_master_grad and not self.args.use_hybrid_parallel
if dp_master_grad:
is_no_sync = True
if is_no_sync:
# Avoid unnecessary DDP synchronization since there will be no backward pass on this example.
with model.no_sync():
tr_loss_step = self.training_step(model, inputs)
else:
tr_loss_step = self.training_step(model, inputs)
tr_loss += tr_loss_step
if (step_control + 1) % args.gradient_accumulation_steps == 0 or (
# last step in epoch but step is always smaller than gradient_accumulation_steps
steps_in_epoch <= args.gradient_accumulation_steps
and (step + 1) == steps_in_epoch
):
if self.args.pipeline_parallel_degree <= 1 and self._enable_delay_scale_loss():
tr_loss /= self.args.gradient_accumulation_steps
self.timers and self.timers("forward-backward").stop()
# Maunally collect gradients
# Case 1: Use recompute and dp
# Case 2: Hack dp with master_grad
# Case 3: Pipeline or sharding overlap
# local_rank != -1 don't means dp in networks.
self.timers and self.timers("all-reduce").start()
# Case 1: Use recompute and dp / sharding stage1,
# manualy collect gradient for dp.
if args.recompute and availiable_no_sync:
fused_allreduce_gradients(list(model.parameters()), None)
# Case 2: hack dp with master_grad
if dp_master_grad and not (args.recompute and availiable_no_sync):
fused_allreduce_gradients(list(model.parameters()), None)
# Pipeline parallel mode, handle gradient reduce here to overlap
enable_dp_comm_overlap = False
enable_release_grads = False
if args.pipeline_parallel_degree > 1:
enable_dp_comm_overlap = "enable_dp_comm_overlap" in args.pipeline_parallel_config
enable_release_grads = "enable_release_grads" in args.pipeline_parallel_config
# Case 3: Pipeline parallel mode, overlap with dp
if isinstance(self.optimizer, HybridParallelOptimizer) and not self.do_grad_scaling:
parameters_list = _obtain_optimizer_parameters_list(self.optimizer._inner_opt)
if not enable_dp_comm_overlap:
if self.optimizer._sharding_enable:
assert reshard_util.is_sharding_opt(self.optimizer)
self.optimizer._inner_opt.reduce_gradients(list(parameters_list), self.optimizer._hcg)
if self.optimizer._dp_enable or getattr(self.optimizer, "_sep_enable", False):
fused_allreduce_gradients(list(parameters_list), self.optimizer._hcg)
self.timers and self.timers("all-reduce").stop()
self.timers and self.timers("optimizer-step").start()
if self.args.gradient_accumulation_steps > 1 and self._enable_delay_scale_loss():
for p in model._layers.parameters():
with paddle.no_grad():
if hasattr(p, "main_grad") and p.main_grad is not None:
assert p.grad is None
p.main_grad.scale_(1.0 / self.args.gradient_accumulation_steps)
elif p.grad is not None:
p.grad.scale_(1.0 / self.args.gradient_accumulation_steps)
# Optimizer step
self.callback_handler.on_optimizer_begin(
args, self.state, self.control, scaler=self.scaler if self.do_grad_scaling else None
)
optimizer_was_run = True
if self.do_grad_scaling:
scale_before = paddle.assign(self.scaler._scale)
self.scaler.step(self.optimizer)
self.scaler.update()
scale_after = self.scaler._scale
# Compatible with paddlepaddle 2.6.0 using typo word.
if hasattr(self.scaler, "_cache_founf_inf"):
optimizer_was_run = not self.scaler._cache_founf_inf
else:
optimizer_was_run = not self.scaler._cache_found_inf
if not optimizer_was_run:
scale_before_value = scale_before.cpu().numpy()
scale_after_value = scale_after.cpu().numpy()
logger.warning(
f"optimizer not run, scale_before: {scale_before_value[0]}, scale_after: {scale_after_value[0]}"
)
elif isinstance(self.optimizer, HybridParallelOptimizer):
self.optimizer._step(parameters_list)
else:
self.optimizer.step()
self.timers and self.timers("optimizer-step").stop()
if optimizer_was_run:
self.lr_scheduler.step()
if enable_release_grads and args.pipeline_parallel_degree > 1:
self.optimizer.clear_grad(set_to_zero=False)
for _, buffers in model._chunk_2_comm_buffers.items():
for buffer in buffers:
buffer._clear_grad_storage()
else:
self.optimizer.clear_grad(set_to_zero=False)
self.callback_handler.on_optimizer_end(
args, self.state, self.control, scaler=self.scaler if self.do_grad_scaling else None
)
self.state.global_step += 1
self.state.epoch = epoch + (step + 1) / steps_in_epoch
self.control = self.callback_handler.on_step_end(args, self.state, self.control)
self._maybe_log_save_evaluate(tr_loss, model, epoch, ignore_keys_for_eval, inputs=inputs)
self._print_timer()
step_control = 0
else:
self.control = self.callback_handler.on_substep_end(args, self.state, self.control)
step_control += 1
if self.control.should_epoch_stop or self.control.should_training_stop:
# break
final_local_vars = locals()
for k in kwargs.keys():
if k in final_local_vars:
kwargs[k] = final_local_vars[k]
return kwargs
# self.timers and self.timers("read-data").start()
final_local_vars = locals()
for k in kwargs.keys():
if k in final_local_vars:
kwargs[k] = final_local_vars[k]
return kwargs
Trainer.init_train_model_opt = init_train_model_opt
Trainer.init_train_log = init_train_log
Trainer.init_train_state = init_train_state
Trainer.full_training_step = full_training_step
# ########## patches for Trianer ##########
class MuteDefaultFlowCallback(TrainerCallback):
"""
Add this callback can cencel logging/evaluation/saving by DefaultFlowCallback.
Use this when having multi trainer.
"""
def on_step_end(self, args: TrainingArguments, state: TrainerState, control: TrainerControl, **kwargs):
control.should_save = False
control.should_evaluate = False
control.should_log = False
return control
@contextmanager
def guard_set_args(args, arg_name_values):
for k, v in arg_name_values.items():
old_value = getattr(args, k, None)
setattr(args, k, v)
arg_name_values[k] = old_value
yield
for k, v in arg_name_values.items():
old_value = getattr(args, k)
setattr(args, k, v)
arg_name_values[k] = old_value
class PipeEvalModel(GenerationMixin):
"""
Wrapper for PipelineParallel to do evaluate and generate. Currently only
support .
"""
def __init__(self, trainer: Trainer):
eval_model = getattr(trainer, "_inner_eval_model", None)
self.model: fleet.model.PipelineParallel = trainer.model_wrapped if eval_model is None else eval_model
self.config: PretrainedConfig = trainer.model.config
self._is_gen = False
self.update_model_kwargs_for_generation = (
self.model._layers._non_pipe_model_class.update_model_kwargs_for_generation
)
@property
def pp_group(self):
return self.model.pp_group
def eval(self):
self.model.eval()
def train(self):
self.model.train()
def __getattr__(self, name):
try:
return super().__getattr__(name)
except AttributeError:
return getattr(self.model, name)
def _broadcast_outputs(self, outputs):
# outputs is PipelineParallel.eval_batch which is a list of batches.
out = []
outputs = (outputs,) if isinstance(outputs, paddle.Tensor) else outputs
for tensors in outputs:
if not self.model.is_pipeline_last_stage():
tensor = tensors if isinstance(tensors, paddle.Tensor) else tensors[0]
head_out_meta = (
(self.model._layers.head_out_meta,)
if isinstance(self.model._layers.head_out_meta, paddle.static.InputSpec)
else self.model._layers.head_out_meta
)
tensors = tuple(
paddle.empty(
shape=[
tensor.shape[i] if (meta.shape[i] is None or meta.shape[i] < 0) else meta.shape[i]
for i in range(len(meta.shape))
],
dtype=tensor.dtype if meta.dtype is None else meta.dtype,
)
for meta in head_out_meta
)
else:
# Currently use tuple instead of ModelOutput and require the
# caller use the return result as tuple.
tensors = (
(tensors,)
if isinstance(tensors, paddle.Tensor)
else tensors.to_tuple()
if isinstance(tensors, ModelOutput)
else tensors
)
# use map_structure seems hung
for tensor in tensors:
paddle.distributed.broadcast(tensor, src=self.model.pp_group.ranks[-1], group=self.model.pp_group)
out.append(tensors[0] if len(tensors) == 1 else tensors)
return out[0] if len(out) == 1 else out
def __call__(self, *args, **kwargs):
model = self.model
assert self.model.training is False
if self._is_gen:
# inputs by `prepare_inputs_for_generation` is a dict with following keys:
# "input_ids", "position_ids", "past_key_values", "use_cache", "attention_mask"
# NOTE: 1. cache/past_key_values should be passed across decoding steps
# by using as model attr rather than input args to reduce comm overhead.
# Also, pipe model defined for training not support this cache input.
# 2. ignore use_cache since _check_data_vaild requires tensor if not None.
# 3. attention_mask can reuse _prepare_decoder_attention_mask in LlamaEmbeddingPipe.
# 4. position_ids pass through _prepare_pipeline_inputs_func and PipeLayer.
inputs, labels = model._prepare_pipeline_inputs_func(*args, **kwargs)
# currently, set accumulate_steps to 1 to avoid multi-batch eval/gen
with guard_set_args(model, {"_compute_loss": False, "accumulate_steps": 1}):
outputs = model.eval_batch([inputs, labels], compute_loss=False)
# TODO(guosheng): Broadcasted logits are used to get next_scores, remove
# it to reduce comm overhead. Also note that we still need broadcast
# next_tokens though logits are broadcasted since pp ranks' seeds differs.
# Currently, just slice the last token to reduce comm overhead.
outputs = [
micro_batch_output[:, -1, :].unsqueeze(1)
if isinstance(micro_batch_output, paddle.Tensor)
else micro_batch_output[0][:, -1, :].unsqueeze(1)
for micro_batch_output in outputs
]
outputs = self._broadcast_outputs(outputs)
else:
# use _prepare_pipeline_inputs_func to convert pipeline inputs
inputs, labels = model._prepare_pipeline_inputs_func(*args, **kwargs)
# NOTE(guosheng): bug seems exist. pp.eval_batch(compute_loss=False)
# will set pp._compute_loss to False and would not set it back. Thus
# hack here to set it back.
with guard_set_args(model, {"_compute_loss": False, "accumulate_steps": 1}):
outputs = model.eval_batch([inputs, labels], compute_loss=False)
outputs = self._broadcast_outputs(outputs)
return outputs
def generate(self, *args, **kwargs):
self._is_gen = True
# patch DecoderLayerPipe to use cache, DecoderLayerPipe is subclass of
# DecoderLayer, and would call super().forward
ori_decoder_layer_forward = self.model._layers._non_pipe_decoder_layer_class.forward
def decoder_layer_forward(layer_self, *args, **kwargs):
kwargs.update({"use_cache": True, "past_key_value": getattr(layer_self, "_cache", None)})
outputs = ori_decoder_layer_forward(layer_self, *args, **kwargs)
output = outputs[0]
layer_self._cache = outputs[1]
self._has_cache = True
return output
with guard_set_args(self.model._layers._non_pipe_decoder_layer_class, {"forward": decoder_layer_forward}):
outputs = super().generate(*args, **kwargs)
self._is_gen = False
# clear cache of decoder layers, sublayers is incursive thus suitable
# to both 1F1B and interleave
for layer in self.model._layers.sublayers():
if isinstance(layer, self.model._layers._non_pipe_decoder_layer_class):
layer._cache = None
self._has_cache = False
return outputs
def prepare_inputs_for_generation(self, *args, **kwargs):
arg_bind = inspect.signature(self.model._layers._non_pipe_model_class.prepare_inputs_for_generation).bind(
*((self,) + args), **kwargs
)
arg_bind.apply_defaults()
arg_dict = arg_bind.arguments
last_arg_name, last_arg_value = arg_dict.popitem()
if arg_bind.signature.parameters[last_arg_name].kind == inspect.Parameter.VAR_KEYWORD:
arg_dict.update(last_arg_value)
else:
arg_dict[last_arg_name] = last_arg_value
arg_dict.pop("self")
past_key_values = arg_dict.get("past_key_values", None)
# prepare_inputs_for_generation use past_key_values to discrimate prefill
# or decode and slice inputs accordingly.
if getattr(self, "_has_cache", False):
arg_dict.update({"past_key_values": True})
model_inputs = self.model._layers._non_pipe_model_class.prepare_inputs_for_generation(self, **arg_dict)
model_inputs.update({"past_key_values": past_key_values})
return model_inputs
def is_same_tokenizer(
tokenizer: PretrainedTokenizer,
other_tokenizer: PretrainedTokenizer,
) -> bool:
"""Check if two tokenizers are the same."""
return tokenizer is other_tokenizer or (
tokenizer.__class__ == other_tokenizer.__class__ and tokenizer.get_vocab() == other_tokenizer.get_vocab()
)
def batch_retokenize(
input_ids: paddle.Tensor,
src_tokenizer: PretrainedTokenizer,
dest_tokenizer: PretrainedTokenizer,
*,
padding: bool | str | PaddingStrategy = PaddingStrategy.LONGEST,
truncation: bool | str | TruncationStrategy = TruncationStrategy.DO_NOT_TRUNCATE,
skip_special_tokens: bool = True,
) -> BatchEncoding:
"""Re-tokenize a batch of input ids from one tokenizer to another."""
output = dest_tokenizer(
[
text + dest_tokenizer.eos_token
for text in src_tokenizer.batch_decode(
input_ids,
skip_special_tokens=skip_special_tokens,
)
],
padding=padding,
truncation=truncation,
return_tensors="pd",
)
return output

145
slm/examples/dialogue/README_DGU.md
View File

@@ -1,145 +0,0 @@
# 对话通用理解模型 (DGU, Dialogue General Understanding)
> **注意** 部分内容在 PaddleNLP 3.0以后不再进行维护，更多历史内容请参考[PaddleNLP 2.8](https://github.com/PaddlePaddle/PaddleNLP/tree/release/2.8/examples/dialogue)。
## 模型简介
对话系统 (Dialogue System) 常常需要根据应用场景的变化去解决多种多样的任务。任务的多样性（意图识别、槽填充、行为识别、状态追踪等等），以及领域训练数据的稀少，给 Dialogue System 的研究和应用带来了巨大的困难和挑战，要使得 Dialogue System 得到更好的发展，需要开发一个通用的对话理解模型。为此，我们给出了基于 BERT 的对话通用理解模型 (DGU: Dialogue General Understanding)，通过实验表明，使用 base-model (BERT)并结合常见的学习范式，就可以在几乎全部对话理解任务上取得比肩甚至超越各个领域业内最好的模型的效果，展现了学习一个通用对话理解模型的巨大潜力。
DGU 模型内共包含6个任务，全部基于公开数据集在 Paddle2.0上完成训练及评估，详细说明如下：
```
udc: 使用UDC (Ubuntu Corpus V1) 数据集完成对话匹配 (Dialogue Response Selection) 任务;
dstc2: 使用DSTC2 (Dialog State Tracking Challenge 2) 数据集完成对话状态追踪 (Dialogue State Tracking) 任务;
atis_slot: 使用ATIS (Airline Travel Information System) 数据集完成对话槽填充 (Dialogue Slot Filling) 任务；
atis_intent: 使用ATIS (Airline Travel Information System) 数据集完成对话意图识别 (Dialogue Intent Detection) 任务；
mrda: 使用MRDAC (Meeting Recorder Dialogue Act Corpus) 数据集完成对话行为识别 (Dialogue Act Detection) 任务；
swda: 使用SwDAC (Switchboard Dialogue Act Corpus) 数据集完成对话行为识别 (Dialogue Act Detection) 任务;
```
## 模型效果
DGU 模型中的6个任务，分别采用不同的评估指标在 test 集上进行评估，结果如下：
<table>
<tr><th style="text-align:center">任务</th><th style="text-align:center">评估指标</th><th style="text-align:center">DGU</th></tr>
<tr align="center"><td rowspan="3" style="vertical-align:middle;">udc</td><td>R1@10</td><td>81.04%</td></tr>
<tr align="center"><td>R2@10</td><td>89.85%</td></tr>
<tr align="center"><td>R5@10</td><td>97.59%</td></tr>
<tr align="center"><td>dstc2</td><td>Joint_Acc</td><td>90.43%</td></tr>
<tr align="center"><td>atis_slot</td><td>F1_Micro</td><td>97.98%</td></tr>
<tr align="center"><td>atis_intent</td><td>Acc</td><td>97.42%</td></tr>
<tr align="center"><td>mrda</td><td>Acc</td><td>90.94%</td></tr>
<tr align="center"><td>swda</td><td>Acc</td><td>80.61%</td></tr>
</table>
**NOTE:** 以上结果均是采用默认配置在 GPU 单卡上训练和评估得到的，用户如需复现效果，可采用默认配置在单卡上进行训练评估。
## 快速开始
### 数据准备
下载数据集压缩包并解压后，DGU_datasets 目录下共存在6个目录，分别对应每个任务的训练集 train.txt、评估集 dev.txt 和测试集 test.txt。
```shell
wget https://bj.bcebos.com/paddlenlp/datasets/DGU_datasets.tar.gz
tar -zxf DGU_datasets.tar.gz
```
DGU_datasets 目录结构：
```text
DGU_datasets/
├── atis_intent
│   ├── dev.txt
│   ├── map_tag_intent_id.txt
│   ├── test.txt
│   └── train.txt
├── udc
│   ├── dev.txt
│   ├── dev.txt-small
│   ├── test.txt
│   └── train.txt
├── atis_slot
│   ├── dev.txt
│   ├── map_tag_slot_id.txt
│   ├── test.txt
│   └── train.txt
├── dstc2
│   ├── dev.txt
│   ├── map_tag_id.txt
│   ├── test.txt
│   └── train.txt
├── mrda
│   ├── dev.txt
│   ├── map_tag_id.txt
│   ├── test.txt
│   └── train.txt
└── swda
├── dev.txt
├── map_tag_id.txt
├── test.txt
└── train.txt
```
数据的每一行由多列组成，都以"\t"作为分割符，详细数据格式说明如下：
```
udc：由label、多轮对话conv和回应response组成
格式：label \t conv1 \t conv2 \t conv3 \t ... \t response
dstc2：由多轮对话id、当前轮QA对(使用\1拼接)和对话状态序列state_list(state_list中每个state由空格分割)组成
格式：conversation_id \t question \1 answer \t state1 state2 state3 ...
atis_slot：由对话内容conversation_content和标签序列label_list (label_list中每个label由空格分割) 组成, 其中标签序列和对话内容中word为一一对应关系
格式：conversation_content \t label1 label2 label3 ...
atis_intent：由标签label和对话内容conversation_content组成
格式： label \t conversation_content
mrda：由多轮对话id、标签label、发言人caller、对话内容conversation_content组成
格式：conversation_id \t label \t caller \t conversation_content
swda：由多轮对话id、标签label、发言人caller、对话内容conversation_content组成
格式：conversation_id \t label \t caller \t conversation_content
```
**NOTE:** 上述数据集来自于 [Paddle1.8静态图版本](https://github.com/PaddlePaddle/models/tree/release/1.8/PaddleNLP/dialogue_system/dialogue_general_understanding)，是由相应的开源数据集经过数据格式转换而得来的，本项目中暂未包含数据格式转换脚本，细节请参考 [Paddle1.8静态图版本](https://github.com/PaddlePaddle/models/tree/release/1.8/PaddleNLP/dialogue_system/dialogue_general_understanding)。
### 模型训练
运行如下命令即可在训练集 (train.tsv) 上进行模型训练，并在开发集 (dev.tsv) 验证，训练结束后会在测试集 (test.txt) 上进行模型评估
```shell
# GPU启动，gpus指定训练所用的GPU卡号，可以是单卡，也可以多卡。默认会进行训练、验证和评估
unset CUDA_VISIBLE_DEVICES
python -m paddle.distributed.launch --gpus "0" --log_dir ./log main.py --task_name=udc --data_dir=./DGU_datasets/udc --output_dir=./checkpoints/udc --device=gpu
# 若只需进行评估，do_train设为False，并且必须指定init_from_ckpt
# python -m paddle.distributed.launch --gpus "0" --log_dir ./log main.py --task_name=udc --data_dir=./DGU_datasets/udc --do_train=False --init_from_ckpt=./checkpoints/udc/best --device=gpu
```
以上参数表示：
* `task_name`：任务名称，可以为 udc、dstc2、atis_slot、atis_intent、mrda 或 swda。
* `data_dir`：训练数据路径。
* `output_dir`：训练保存模型的文件路径。
* `do_train：是否进行训练，默认为`True`。
* `init_from_ckpt`：恢复模型参数的路径。
* `device`：表示训练使用的设备。
其他可选参数和参数的默认值请参考`args.py`。
程序运行时将会自动进行训练，验证和评估。同时训练过程中会自动保存模型在指定的`output_dir`中。
如：
```text
checkpoints/
├── 1000.pdopt
├── 1000.pdparams
├── 2000.pdopt
├── 2000.pdparams
├── ...
├── best.pdopt
└── best.pdparams
```
**NOTE:** 如需恢复模型训练，则 init_from_ckpt 只需指定到文件名即可，不需要添加文件尾缀。如`--init_from_ckpt=checkpoints/1000`即可，程序会自动加载模型参数`checkpoints/1000.pdparams`，也会自动加载优化器状态`checkpoints/1000.pdopt`。

148
slm/examples/dialogue/README_LIC2021.md
View File

@@ -1,148 +0,0 @@
# LIC 2021对话比赛 baseline
> **注意** 部分内容在 PaddleNLP 3.0以后不再进行维护，更多历史内容请参考[PaddleNLP 2.8](https://github.com/PaddlePaddle/PaddleNLP/tree/release/2.8/examples/dialogue)。
## 模型简介
近年来，人机对话系统受到了学术界和产业界的广泛关注并取得了不错的发展。开放域对话系统旨在建立一个开放域的多轮对话系统，使得机器可以流畅自然地与人进行语言交互，既可以进行日常问候类的闲聊，又可以完成特定功能，以使得开放域对话系统具有实际应用价值，例如进行对话式推荐，或围绕一个主题进行深入的知识对话等。具体的说，开放域对话可以继续拆分为支持不同功能的对话形式，例如对话式推荐，知识对话技术等，如何解决并有效融合以上多个技能面临诸多挑战。
LIC 2021对话比赛收集了一系列公开的开放域对话数据并提供了统一的评测方式，旨在为研究人员和开发者提供学术和技术交流的平台，进一步提升开放域对话的研究水平，推动自然语言理解和人工智能领域技术的应用和发展。
为了方便参赛者快速了解 LIC 2021对话比赛的流程，并快速地参与到比赛中，本项目基于 UnifiedTransformer 模型提供了一个基础 baseline，利用小规模样例数据在预训练模型上完成了微调及预测。参赛者可以针对赛题进行其他改进，例如修改数据预处理方法、修改网络结构、修改训练方式、修改预测的解码方式或对结果的后处理策略等方式提升模型效果。
UnifiedTransformer 模型的细节可以[参阅论文](https://arxiv.org/abs/2006.16779)。
## 快速开始
### 环境依赖
- sentencepiece
安装方式：`pip install sentencepiece`
### 数据准备
由于样例数据涉及 LIC 2021对话比赛，暂不开放。
关于数据集及数据集的预处理过程，详见[2021语言与智能技术竞赛：多技能对话](https://aistudio.baidu.com/aistudio/competition/detail/67)及官方提供的基线系统 Baselines。
模型的输入由3部分组成：词向量 token_ids，句向量 token_type_ids 和位置向量 position_ids。本项目的数据集是样例文本经过数据预处理脚本得到的 id 化的数据集。数据的每一行由3列组成，以";"作为分割符，格式：token_ids;token_type_ids;position_ids。具体细节请参考`data.py`。
### 模型训练
运行如下命令即可在样例训练集上进行 finetune，并在样例验证集上进行验证
```shell
# GPU启动，参数`--gpus`指定训练所用的GPU卡号，可以是单卡，也可以多卡
unset CUDA_VISIBLE_DEVICES
python -m paddle.distributed.launch --gpus "0" --log_dir ./log finetune.py \
--model_name_or_path=unified_transformer-12L-cn \
--train_data_path=./datasets/train.txt \
--valid_data_path=./datasets/valid.txt \
--save_dir=./checkpoints \
--logging_steps=500 \
--save_steps=8000 \
--seed=2021 \
--epochs=10 \
--batch_size=8192 \
--lr=1e-5 \
--weight_decay=0.01 \
--warmup_steps=4000 \
--max_grad_norm=0.1 \
--sort_pool_size=65536 \
--device=gpu
```
其中参数释义如下：
- `gpus` 指示了训练所用的 GPU 卡号。
- `model_name_or_path` 指示了 finetune 使用的具体预训练模型，可以是 PaddleNLP 提供的预训练模型，或者是本地的预训练模型。如果使用本地的预训练模型，可以配置本地模型的目录地址，例如: ./checkpoints/model_xx/，目录中需包含 paddle 预训练模型 model_state.pdparams。如果使用 PaddleNLP 提供的预训练模型，可以选择下面其中之一。
| PaddleNLP 提供的预训练模型 |
|---------------------------------|
| unified_transformer-12L-cn |
| unified_transformer-12L-cn-luge |
- `train_data_path` 表示训练集文件路径。
- `valid_data_path` 表示验证集文件路径。
- `save_dir` 表示模型的保存路径。
- `logging_steps` 表示日志打印间隔。
- `save_steps` 表示模型保存及评估间隔。
- `seed` 表示随机数生成器的种子。
- `epochs` 表示训练轮数。
- `batch_size` 表示每次迭代**每张卡**上的样本数目。
- `lr` 表示基础学习率大小，将于 learning rate scheduler 产生的值相乘作为当前学习率。
- `weight_decay` 表示 AdamW 优化器中使用的 weight_decay 的系数。
- `warmup_steps` 表示学习率逐渐升高到基础学习率（即上面配置的 lr）所需要的迭代数，最早的使用可以参考[这篇论文](https://arxiv.org/pdf/1706.02677.pdf)。
- `max_grad_norm` 表示梯度裁剪允许的最大梯度值。
- `sort_pool_size` 表示在构建 batch 数据时，用来排序的 pool size。
- `device` 表示训练使用的设备。
参数详情和参数的默认值请参考`args.py`。
程序运行时将会自动进行训练和验证，训练过程中会自动保存模型在指定的`save_dir`中。
如：
```text
./checkpoints/
├── model_8000
│ ├── model_config.json
│ ├── model_state.pdparams
│ ├── spm.model
│ ├── tokenizer_config.json
│ └── vocab.txt
└── ...
```
**NOTE:** 如需恢复模型训练，`model_name_or_path`配置本地模型的目录地址即可。
### 模型预测
运行如下命令即可在样例测试集上进行测试
```shell
export CUDA_VISIBLE_DEVICES=0
# GPU启动，预测仅支持单卡
python infer.py \
--model_name_or_path=./checkpoints/model_80000 \
--test_data_path=./datasets/test.txt \
--output_path=./predict.txt \
--logging_steps=500 \
--seed=2021 \
--batch_size=4 \
--min_dec_len=1 \
--max_dec_len=64 \
--num_samples=20 \
--decode_strategy=sampling \
--top_k=5 \
--device=gpu
```
其中参数释义如下：
- `model_name_or_path` 指示了 finetune 使用的具体预训练模型，可以是 PaddleNLP 提供的预训练模型，或者是本地的预训练模型。如果使用本地的预训练模型，可以配置本地模型的目录地址，例如: ./checkpoints/model_xx/，目录中需包含 paddle 预训练模型 model_state.pdparams。如果使用 PaddleNLP 提供的预训练模型，可以选择下面其中之一。
| PaddleNLP 提供的预训练模型 |
|---------------------------------|
| unified_transformer-12L-cn |
| unified_transformer-12L-cn-luge |
- `test_data_path` 表示预测集文件路径。
- `output_path` 表示预测结果的保存路径。
- `logging_steps` 表示日志打印间隔。
- `seed` 表示随机数生成器的种子。
- `batch_size` 表示每次迭代**每张卡**上的样本数目。
- `min_dec_len` 表示预测生成的句子的最小长度。
- `max_dec_len` 表示预测生成的句子的最大长度。
- `num_samples` 表示每条样本生成的句子的数量。对于每条样本，模型会生成`num_samples`个句子，根据每个句子的概率得分进行排序，得分最高的句子作为最终的生成结果。
- `decode_strategy` 表示预测解码时采取的策略，可选"sampling"、"greedy_search"和"beam_search"之一。
- `top_k` 表示采用"sampling"解码策略时，token 的概率按从大到小排序，生成的 token 只从前`top_k`个中进行采样。
- `device` 表示训练使用的设备。
参数详情和参数的默认值请参考`args.py`。
程序运行结束后会将预测结果保存在`output_path`中。将预测结果准备成比赛官网要求的格式，提交评估即可得评估结果。
采用不同的模型在样例测试集上有如下结果：
| model_name_or_path | F1 | BLEU1 / BLEU2 | DISTINCT1 / DISTINCT2 |
| :-----------------------------: | :---: | :-----------: | :-------------------: |
| unified_transformer-12L-cn | 10.62 | 0.070 / 0.022 | 0.065 / 0.304 |
| unified_transformer-12L-cn-luge | 33.11 | 0.245 / 0.157 | 0.074 / 0.238 |
| ./checkpoints/model_80000 | 32.38 | 0.239 / 0.150 | 0.070 / 0.219 |

69
slm/examples/dialogue/README_PLATO-2.md
View File

@@ -1,69 +0,0 @@
# PLATO-2
> **注意** 部分内容在 PaddleNLP 3.0以后不再进行维护，更多历史内容请参考[PaddleNLP 2.8](https://github.com/PaddlePaddle/PaddleNLP/tree/release/2.8/examples/dialogue)。
## 模型简介
构建高质量的开放领域（Open-Domain）的对话机器人，使得它能用自然语言与人自由地交流，这一直是自然语言处理领域终极目标之一。
为了能够简易地构建一个高质量的开放域聊天机器人，本项目在 Paddle2.0上实现了 PLATO-2的预测模型，并基于终端实现了简单的人机交互。用户可以通过下载预训练模型快速构建一个开放域聊天机器人。
PLATO-2的网络结构见下图：
<p align="center"><img src="./imgs/network.png" hspace="10"/></p>
PLATO-2的训练过程及其他细节详见 [Knover](https://github.com/PaddlePaddle/Knover)
## 快速开始
### 环境依赖
- python 3.7+
- sentencepiece
- termcolor
安装方式：`pip install sentencepiece termcolor`
### 数据准备
您可以从以下位置下载预训练模型文件：
* PLATO-2, 24-layers, 16-heads, 1024-hidden, EN: [预训练模型](https://bj.bcebos.com/paddlenlp/models/transformers/plato2/24L.pdparams)
* PLATO-2, 32-layers, 32-heads, 2048-hidden, EN: [预训练模型](https://bj.bcebos.com/paddlenlp/models/transformers/plato2/32L.pdparams)
以24层预训练模型为例：
```shell
wget https://bj.bcebos.com/paddlenlp/models/transformers/plato2/24L.pdparams
```
**NOTE:** PLATO-2网络参数量较大，24层网络至少需要显存16G，32层网络至少需要显存22G，用户可选择合适的网络层数及预训练模型。
sentencepiece 分词预训练模型和词表文件下载：
```shell
wget https://bj.bcebos.com/paddlenlp/models/transformers/plato2/data.tar.gz
tar -zxf data.tar.gz
```
### 人机交互
运行如下命令即可开始与聊天机器人用英语进行简单的对话
```shell
export CUDA_VISIBLE_DEVICES=0
python interaction.py --vocab_path ./data/vocab.txt --spm_model_file ./data/spm.model --num_layers 24 --init_from_ckpt ./24L.pdparams
```
以上参数表示：
* vocab_path：词表文件路径。
* spm_model_file：sentencepiece 分词预训练模型路径。
* num_layers：PLATO-2组网层数。
* init_from_ckpt：PLATO-2预训练模型路径。
32层 PLATO-2网络交互示例：
<p><img src="./imgs/case.jpg" hspace="10"/></p>
**NOTE:** 输入"[EXIT]"退出交互程序，输入"[NEXT]"开启下一轮新的对话。

150
slm/examples/dialogue/README_PLATO-XL.md
View File

@@ -1,150 +0,0 @@
# PLATO-XL: Exploring the Large-scale Pre-training of Dialogue Generation
> **注意** 部分内容在 PaddleNLP 3.0以后不再进行维护，更多历史内容请参考[PaddleNLP 2.8](https://github.com/PaddlePaddle/PaddleNLP/tree/release/2.8/examples/dialogue)。
## 模型简介
构建高质量的开放领域（Open-Domain）的对话机器人，使得它能用自然语言与人自由地交流，这一直是自然语言处理领域终极目标之一。
PLATO-XL 是业界首个开源的百亿超大规模开放域对话预训练模型，其使用了参数高效(encoder-decoder 共享参数)的 UnifiedTransformer（prefix LM）模型架构，将模型参数量提升到了11B 量级，经过了十亿级样本对话数据的预训练，并引入 role embedding 区分多方对话中的对话角色提升预训练效果，最终模型闲聊测试效果超过了众多代表性的对话模型。可以直接使用 PLATO-XL 构建高质量的开放领域对话机器人。
PaddleNLP 内置了 PLATO-XL 英文预训练模型以供使用。由于 PLATO-XL 模型规模较大，这使得其在预测时生成对话回复的时间较长，并且 11B 的参数量也可能超出部分型号 GPU 显存容量，这是大模型推理与落地存在的普遍和关键问题。PaddleNLP FastGeneration 为 PLATO-XL 提供了 GPU 上的高性能生成加速能力，并且支持模型并行（张量并行）推理允许通过多张小显存容量的 GPU 使用百亿大模型，相比单卡代码中也只增加了`enable_ft_para()`一行，此外模型并行能进一步提升预测速度。
本项目提供了 PLATO-XL 英文模型使用 PaddleNLP FastGeneration 进行高性能预测的使用示例。PLATO-XL 的训练及更多内容请参考 [PaddlePaddle/Knover](https://github.com/PaddlePaddle/Knover/tree/develop/projects/PLATO-XL)。
## 开始运行
### 单卡高性能推理
`infer.py` 是 PLATO-XL 高性能预测使用示例脚本，可以使用如下命令运行：
```shell
python infer.py --topk 4 --max_out_len 64 --use_faster --use_fp16
```
该脚本各个参数含义如下：
- `topk` 用于 Top-K 采样策略，采样时将只从概率最高 K 个 token 中采样，默认为1，即 greedy search。
- `topp` 用于 Top-P 采样策略，采样时将只从概率最高且累加概率不超过该值的 token 中采样，默认为1.0。
- `max_out_len` 指定生成的最大长度，默认为64。
- `min_out_len` 指定生成的最小长度，默认为1。
- `temperature` 用于调整预测概率分布，默认为1.0，即保持模型原有的预测概率。
- `use_faster` 使用 FastGeneration
- `use_fp16` 使用 FP16，只在使用 FastGeneration 时生效
脚本中使用了一条如下的多轮对话的样本数据， 由`List[str]`表示，其中每个`str`表示一句话，将根据历史对话内容生成回复。
```python
history = [
"hi , Mary ! What do you usually like to do in your spare time ?",
"well , I spend a lot of time watching movies .",
"what a confidence ! I always watch a lot of movies , too ."
"oh really , Frank ? What kind of movies do you like ?"
]
```
**注意** 由于 PLATO-XL 模型较大，单卡预测至少需要22G 显存（使用 FP16时），且模型下载需要一定时间（FP32的权重文件约41G）。
### 多卡并行推理
多卡并行推理当前依赖 MPI（[MPICH](https://www.mpich.org)、[OpenMPI](https://www.open-mpi.org)均可）和[NCCL](https://developer.nvidia.com/nccl)，如需使用还请先安装依赖。安装完成后仍然使用 `infer.py` 来进行预测，相比单卡时不同的只是通过 mpi 来启动运行，如下：
```shell
mpirun -n 4 python infer.py --topk 4 --max_out_len 64 --use_faster --use_fp16
```
其中`-n 4`指明使用的进程和 GPU 数，在`n`设置为1时仍将进行单卡推理。由于多卡并行推理使用和单卡使用不同的依赖库，第一次运行时将重新进行 JIT 编译。
### 性能测试
`infer.py` 中同时提供了性能测试的支持，在上面预测命令的基础上加上 `--profile` 即可，如下：
```shell
mpirun -n 4 python infer.py --batch_size 8 --min_out_len 20 --max_out_len 20 --topk 1 --use_faster --use_fp16 --profile
```
此外还指定了`batch_size`和`min_out_len`来得到特定输入输出大小下的性能，性能测试将给出循环运行多次的平均时延。以下为单卡高性能推理和4卡张量并行推理性能数据（V100，CUDA 10.2，输入长度60、输出长度20），可以看出4卡并行速度为单卡的2倍左右。
<table>
<caption>PLATO-XL 高性能推理速度&nbsp;&nbsp;(in ms/batch)</caption>
<tr style="text-align:center;">
<td align=center>batch size</td>
<td align=center>K</td>
<td align=center>FastGeneration</br>1卡</br>FP16</td>
<td align=center>FastGeneration</br>4卡</br>FP16</td>
<td align=center>多卡并行</br>SpeedUp</td>
</tr>
<tr style="text-align:center;">
<td align=center>1</td>
<td align=center>1</td>
<td align=center>706.937</td>
<td align=center>348.653</td>
<td align=center>2.027</td>
</tr>
<tr style="text-align:center;">
<td align=center>1</td>
<td align=center>10</td>
<td align=center>707.514</td>
<td align=center>348.699</td>
<td align=center>2.029</td>
</tr>
<tr style="text-align:center;">
<td align=center>4</td>
<td align=center>1</td>
<td align=center>768.597</td>
<td align=center>384.730</td>
<td align=center>1.997</td>
</tr>
<tr style="text-align:center;">
<td align=center>4</td>
<td align=center>10</td>
<td align=center>770.008</td>
<td align=center>385.244</td>
<td align=center>1.998</td>
</tr>
<tr style="text-align:center;">
<td align=center>8</td>
<td align=center>1</td>
<td align=center>862.017</td>
<td align=center>418.313</td>
<td align=center>2.060</td>
</tr>
<tr style="text-align:center;">
<td align=center>8</td>
<td align=center>10</td>
<td align=center>866.490</td>
<td align=center>418.965</td>
<td align=center>2.068</td>
</tr>
<tr style="text-align:center;">
<td align=center>16</td>
<td align=center>1</td>
<td align=center>1016.362</td>
<td align=center>486.974</td>
<td align=center>2.087</td>
</tr>
<tr style="text-align:center;">
<td align=center>16</td>
<td align=center>10</td>
<td align=center>1060.472</td>
<td align=center>488.156</td>
<td align=center>2.172</td>
</tr>
<tr style="text-align:center;">
<td align=center>32</td>
<td align=center>1</td>
<td align=center>1325.700</td>
<td align=center>606.770</td>
<td align=center>2.184</td>
</tr>
<tr style="text-align:center;">
<td align=center>32</td>
<td align=center>10</td>
<td align=center>1326.222</td>
<td align=center>608.479</td>
<td align=center>2.179</td>
</tr>
</table>
## Reference
1. Bao S, He H, Wang F, et al. PLATO-XL: Exploring the Large-scale Pre-training of Dialogue Generation[J]. arXiv preprint arXiv:2109.09519, 2021.

219
slm/examples/dialogue/README_UnifiedTransformer.md
View File

@@ -1,219 +0,0 @@
# UnifiedTransformer
> **注意** 部分内容在 PaddleNLP 3.0以后不再进行维护，更多历史内容请参考[PaddleNLP 2.8](https://github.com/PaddlePaddle/PaddleNLP/tree/release/2.8/examples/dialogue)。
## 模型简介
近年来，人机对话系统受到了学术界和产业界的广泛关注并取得了不错的发展。开放域对话系统旨在建立一个开放域的多轮对话系统，使得机器可以流畅自然地与人进行语言交互，既可以进行日常问候类的闲聊，又可以完成特定功能，以使得开放域对话系统具有实际应用价值。具体的说，开放域对话可以继续拆分为支持不同功能的对话形式，例如对话式推荐，知识对话技术等，如何解决并有效融合以上多个技能面临诸多挑战。
[UnifiedTransformer](https://arxiv.org/abs/2006.16779)以[Transformer](https://arxiv.org/abs/1706.03762) 编码器为网络基本组件，采用灵活的注意力机制，十分适合对话生成任务。
本项目是 UnifiedTransformer 在 Paddle 2.0上的开源实现，介绍了如何使用 UnifiedTransformer 在 DuConv 任务型对话数据集上进行微调，并给出了一个搭建简单中文聊天机器人的例子。
## 快速开始
### 环境依赖
- sentencepiece
- termcolor
安装方式：`pip install sentencepiece termcolor`
### 代码结构说明
以下是本项目主要代码结构及说明：
```text
.
├── finetune.py # 模型finetune主程序入口
├── infer.py # 模型预测主程序入口
├── utils.py # 定义参数及一些工具函数
└── README.md # 文档说明
```
### 数据准备
**DuConv**是百度发布的基于知识图谱的主动聊天任务数据集，让机器根据构建的知识图谱进行主动聊天，使机器具备模拟人类用语言进行信息传递的能力。数据集的创新性是：强调了 bot 的主动性，并且在闲聊对话中引入了明确的对话目标，即将对话引导到特定实体上。数据中的知识信息来源于电影和娱乐人物领域有聊天价值的知识信息，如票房、导演、评价等，以三元组 SPO 的形式组织，对话目标中的话题为电影或娱乐人物实体。数据集中共有3万 session，约12万轮对话，划分为训练集、开发集、测试集1和测试集2，其中测试集1中包含对话的 response，而测试集2中只有对话历史。
为了方便用户快速测试，PaddleNLP Dataset API 内置了 DuConv 数据集，一键即可完成数据集加载，示例代码如下：
```python
from paddlenlp.datasets import load_dataset
train_ds, dev_ds, test1_ds, test2_ds = load_dataset('duconv', splits=('train', 'dev', 'test_1', 'test_2'))
```
### 预训练模型
以下是 PaddleNLP 支持的对话类预训练模型：
|模型名称| 模型参数 | 模型特点 |
|:-----:|:------:|:-------:|
|unified_transformer-12L-cn| 12-layers, 12-heads, 768-hidden| 在千万级别的中文会话数据上进行预训练。|
|unified_transformer-12L-cn-luge| 12-layers, 12-heads, 768-hidden|由 unified_transformer-12L-cn 预训练模型在千言对话数据集上进行微调。并且模型输入中加入了标识不同对话技能的 special token，使得模型能同时支持闲聊对话、推荐对话和知识对话。|
|plato-mini| 6-layers, 12-heads, 768-hidden|在十亿级别的中文对话数据上进行预训练。参数量更小，但效果更好。只支持闲聊型对话。|
### 模型训练
运行如下命令即可在训练集上进行 finetune，并在验证集上进行验证
```shell
# GPU启动，参数`--gpus`指定训练所用的GPU卡号，可以是单卡，也可以多卡
# 例如使用1号和2号卡，则：`--gpu '1,2'`
unset CUDA_VISIBLE_DEVICES
python -m paddle.distributed.launch --gpus '0' --log_dir ./log finetune.py \
--model_name_or_path=unified_transformer-12L-cn-luge \
--save_dir=./checkpoints \
--logging_steps=100 \
--save_steps=1000 \
--seed=2021 \
--epochs=3 \
--batch_size=16 \
--lr=5e-5 \
--weight_decay=0.01 \
--warmup_steps=2500 \
--max_grad_norm=0.1 \
--max_seq_len=512 \
--max_response_len=128 \
--max_knowledge_len=256 \
--device=gpu
```
其中参数释义如下：
- `gpus` 指示了训练所用的 GPU
- `log_dir` 指示了日志保存目录
- `model_name_or_path` 指示了 finetune 使用的预训练模型，可以是 PaddleNLP 提供的预训练模型，或者是本地的模型。如果使用本地的模型，则配置为本地模型的目录地址，例如: ./checkpoints/model_xx/，目录中需包含 paddle 模型参数 model_state.pdparams。如果使用 PaddleNLP 提供的预训练模型，可以选择下面其中之一。
| PaddleNLP 提供的预训练模型 |
|---------------------------------|
| unified_transformer-12L-cn |
| unified_transformer-12L-cn-luge |
- `save_dir` 表示模型的保存路径。
- `logging_steps` 表示日志打印间隔。
- `save_steps` 表示模型保存及评估间隔。
- `seed` 表示随机数生成器的种子。
- `epochs` 表示训练轮数。
- `batch_size` 表示每次迭代**每张卡**上的样本数目。
- `lr` 表示基础学习率大小，将于 learning rate scheduler 产生的值相乘作为当前学习率。
- `weight_decay` 表示 AdamW 优化器中使用的 weight_decay 的系数。
- `warmup_steps` 表示学习率逐渐升高到基础学习率（即上面配置的 lr）所需要的迭代数，最早的使用可以参考[这篇论文](https://arxiv.org/pdf/1706.02677.pdf)。
- `max_grad_norm` 表示梯度裁剪允许的最大梯度值。
- `max_seq_len` 表示输入序列的最大长度。
- `max_response_len` 表示输入 response 的最大长度。
- `max_knowledge_len` 表示输入 knowledge 序列的最大长度。
- `device` 表示使用的设备。
程序运行时将会自动进行训练和验证，训练过程中会自动保存模型在指定的`save_dir`中，其中 loss 最小的模型会被保存在`save_dir/model_best`中。如：
```text
./checkpoints/
├── model_1000
│ ├── model_config.json
│ ├── model_state.pdparams
│ ├── spm.model
│ ├── tokenizer_config.json
│ └── vocab.txt
└── ...
```
**NOTE:** 如需恢复模型训练，只需指定`model_name_or_path`为本地微调模型的路径即可。
### 模型预测
运行如下命令即可在测试集上进行测试。
```shell
# GPU启动，预测仅支持单卡
export CUDA_VISIBLE_DEVICES=0
python infer.py \
--model_name_or_path=./checkpoints/model_best \
--output_path=./predict.txt \
--logging_steps=10 \
--seed=2021 \
--max_seq_len=512 \
--max_knowledge_len=256 \
--batch_size=4 \
--min_dec_len=1 \
--max_dec_len=64 \
--num_return_sequences=20 \
--decode_strategy=sampling \
--top_k=5 \
--device=gpu
```
其中参数释义如下：
- `model_name_or_path` 指示了预测使用的模型，可以是 PaddleNLP 提供的预训练模型，或者是本地的模型。如果使用本地的模型，则配置为本地模型的目录地址，例如: ./checkpoints/model_xx/，目录中需包含 paddle 模型参数 model_state.pdparams。如果使用 PaddleNLP 提供的预训练模型，可以选择下面其中之一。
| PaddleNLP 提供的预训练模型 |
|---------------------------------|
| unified_transformer-12L-cn |
| unified_transformer-12L-cn-luge |
- `output_path` 表示预测结果的保存路径。
- `logging_steps` 表示日志打印间隔。
- `seed` 表示随机数生成器的种子。
- `max_seq_len` 表示输入序列的最大长度。
- `max_knowledge_len` 表示输入 knowledge 序列的最大长度。
- `batch_size` 表示每次迭代**每张卡**上的样本数目。
- `min_dec_len` 表示预测生成的句子的最小长度。
- `max_dec_len` 表示预测生成的句子的最大长度。
- `num_return_sequences` 表示每条样本生成的句子的数量。对于每条样本，模型会生成`num_return_sequences`个句子，根据每个句子的概率得分进行排序，得分最高的句子作为最终的生成结果。
- `decode_strategy` 表示预测解码时采取的策略，可选"sampling"、"greedy_search"和"beam_search"之一。
- `top_k` 表示采用"sampling"解码策略时，token 的概率按从大到小排序，生成的 token 只从前`top_k`个中进行采样。
- `device` 表示使用的设备。
同时，我们提供了基于 FastGeneration 的高性能预测的选项，可以选择性开启是否需要采用高性能预测，PaddleNLP 提供了 JIT 的方式，可以自动完成对所需的自定义 op 的动态库编译：
- `faster` 表示是否开启高性能预测。设置 `--faster` 即表示开启。
- `use_fp16_decoding` 表示在开启高性能预测的时候，是否使用 fp16 来完成预测过程。设置 `--use_fp16_decoding` 即表示使用 fp16 进行预测，否则使用 fp32。
程序运行结束后会将预测生成的 response 保存在`output_path`中。同时终端中会输出评估结果。
采用预训练模型及微调模型在测试集上有如下结果：
| model_name_or_path | BLEU1 / BLEU2 | DISTINCT1 / DISTINCT2 |
| :-----------------------------: | :-------------: | :-------------------: |
| unified_transformer-12L-cn-luge | 0.2606 / 0.1576 | 0.1168 / 0.2977 |
| ./checkpoints/model_best | 0.2808 / 0.1744 | 0.1124 / 0.2899 |
**NOTE:** `./checkpoints/model_best`是按本项目中的超参在单卡上 finetune 得到的结果。
### 人机交互
运行如下命令即可开始与聊天机器人用中文进行简单的对话。
```shell
# GPU启动，仅支持单卡
export CUDA_VISIBLE_DEVICES=0
python interaction.py \
--model_name_or_path=plato-mini \
--min_dec_len=0 \
--max_dec_len=64 \
--num_return_sequences=20 \
--decode_strategy=sampling \
--top_k=5 \
--device=gpu
```
其中参数释义如下：
- `model_name_or_path` 指示了预测使用的模型，可以是 PaddleNLP 提供的预训练模型，或者是本地的模型。如果使用本地的模型，则配置为本地模型的目录地址，例如: ./checkpoints/model_xx/，目录中需包含 paddle 模型参数 model_state.pdparams。如果使用 PaddleNLP 提供的预训练模型，可以选择下面其中之一。
| PaddleNLP 提供的预训练模型 |
|---------------------------------|
| unified_transformer-12L-cn |
| unified_transformer-12L-cn-luge |
| plato-mini |
- `min_dec_len` 表示预测生成的句子的最小长度。
- `max_dec_len` 表示预测生成的句子的最大长度。
- `num_return_sequences` 表示每条样本生成的句子的数量。对于每条样本，模型会生成`num_return_sequences`个句子，根据每个句子的概率得分进行排序，得分最高的句子作为最终的生成结果。
- `decode_strategy` 表示预测解码时采取的策略，可选"sampling"、"greedy_search"和"beam_search"之一。
- `top_k` 表示采用"sampling"解码策略时，token 的概率按从大到小排序，生成的 token 只从前`top_k`个中进行采样。
- `device` 表示使用的设备。
**NOTE:** 输入"[EXIT]"退出交互程序，输入"[NEXT]"开启下一轮新的对话。需要注意使用退格会导致错误。
## Reference
- [UnifiedTransformer](https://arxiv.org/abs/2006.16779)
- [Knover/luge-dialogue](https://github.com/PaddlePaddle/Knover/tree/luge-dialogue/luge-dialogue)
- [DuConv](https://www.aclweb.org/anthology/P19-1369/)

143
slm/examples/dialogue/dgu/README.md
View File

@@ -1,143 +0,0 @@
# 对话通用理解模型 (DGU, Dialogue General Understanding)
## 模型简介
对话系统 (Dialogue System) 常常需要根据应用场景的变化去解决多种多样的任务。任务的多样性（意图识别、槽填充、行为识别、状态追踪等等），以及领域训练数据的稀少，给 Dialogue System 的研究和应用带来了巨大的困难和挑战，要使得 Dialogue System 得到更好的发展，需要开发一个通用的对话理解模型。为此，我们给出了基于 BERT 的对话通用理解模型 (DGU: Dialogue General Understanding)，通过实验表明，使用 base-model (BERT)并结合常见的学习范式，就可以在几乎全部对话理解任务上取得比肩甚至超越各个领域业内最好的模型的效果，展现了学习一个通用对话理解模型的巨大潜力。
DGU 模型内共包含6个任务，全部基于公开数据集在 Paddle2.0上完成训练及评估，详细说明如下：
```
udc: 使用UDC (Ubuntu Corpus V1) 数据集完成对话匹配 (Dialogue Response Selection) 任务;
dstc2: 使用DSTC2 (Dialog State Tracking Challenge 2) 数据集完成对话状态追踪 (Dialogue State Tracking) 任务;
atis_slot: 使用ATIS (Airline Travel Information System) 数据集完成对话槽填充 (Dialogue Slot Filling) 任务；
atis_intent: 使用ATIS (Airline Travel Information System) 数据集完成对话意图识别 (Dialogue Intent Detection) 任务；
mrda: 使用MRDAC (Meeting Recorder Dialogue Act Corpus) 数据集完成对话行为识别 (Dialogue Act Detection) 任务；
swda: 使用SwDAC (Switchboard Dialogue Act Corpus) 数据集完成对话行为识别 (Dialogue Act Detection) 任务;
```
## 模型效果
DGU 模型中的6个任务，分别采用不同的评估指标在 test 集上进行评估，结果如下：
<table>
<tr><th style="text-align:center">任务</th><th style="text-align:center">评估指标</th><th style="text-align:center">DGU</th></tr>
<tr align="center"><td rowspan="3" style="vertical-align:middle;">udc</td><td>R1@10</td><td>81.04%</td></tr>
<tr align="center"><td>R2@10</td><td>89.85%</td></tr>
<tr align="center"><td>R5@10</td><td>97.59%</td></tr>
<tr align="center"><td>dstc2</td><td>Joint_Acc</td><td>90.43%</td></tr>
<tr align="center"><td>atis_slot</td><td>F1_Micro</td><td>97.98%</td></tr>
<tr align="center"><td>atis_intent</td><td>Acc</td><td>97.42%</td></tr>
<tr align="center"><td>mrda</td><td>Acc</td><td>90.94%</td></tr>
<tr align="center"><td>swda</td><td>Acc</td><td>80.61%</td></tr>
</table>
**NOTE:** 以上结果均是采用默认配置在 GPU 单卡上训练和评估得到的，用户如需复现效果，可采用默认配置在单卡上进行训练评估。
## 快速开始
### 数据准备
下载数据集压缩包并解压后，DGU_datasets 目录下共存在6个目录，分别对应每个任务的训练集 train.txt、评估集 dev.txt 和测试集 test.txt。
```shell
wget https://bj.bcebos.com/paddlenlp/datasets/DGU_datasets.tar.gz
tar -zxf DGU_datasets.tar.gz
```
DGU_datasets 目录结构：
```text
DGU_datasets/
├── atis_intent
│   ├── dev.txt
│   ├── map_tag_intent_id.txt
│   ├── test.txt
│   └── train.txt
├── udc
│   ├── dev.txt
│   ├── dev.txt-small
│   ├── test.txt
│   └── train.txt
├── atis_slot
│   ├── dev.txt
│   ├── map_tag_slot_id.txt
│   ├── test.txt
│   └── train.txt
├── dstc2
│   ├── dev.txt
│   ├── map_tag_id.txt
│   ├── test.txt
│   └── train.txt
├── mrda
│   ├── dev.txt
│   ├── map_tag_id.txt
│   ├── test.txt
│   └── train.txt
└── swda
├── dev.txt
├── map_tag_id.txt
├── test.txt
└── train.txt
```
数据的每一行由多列组成，都以"\t"作为分割符，详细数据格式说明如下：
```
udc：由label、多轮对话conv和回应response组成
格式：label \t conv1 \t conv2 \t conv3 \t ... \t response
dstc2：由多轮对话id、当前轮QA对(使用\1拼接)和对话状态序列state_list(state_list中每个state由空格分割)组成
格式：conversation_id \t question \1 answer \t state1 state2 state3 ...
atis_slot：由对话内容conversation_content和标签序列label_list (label_list中每个label由空格分割) 组成, 其中标签序列和对话内容中word为一一对应关系
格式：conversation_content \t label1 label2 label3 ...
atis_intent：由标签label和对话内容conversation_content组成
格式： label \t conversation_content
mrda：由多轮对话id、标签label、发言人caller、对话内容conversation_content组成
格式：conversation_id \t label \t caller \t conversation_content
swda：由多轮对话id、标签label、发言人caller、对话内容conversation_content组成
格式：conversation_id \t label \t caller \t conversation_content
```
**NOTE:** 上述数据集来自于 [Paddle1.8静态图版本](https://github.com/PaddlePaddle/models/tree/release/1.8/PaddleNLP/dialogue_system/dialogue_general_understanding)，是由相应的开源数据集经过数据格式转换而得来的，本项目中暂未包含数据格式转换脚本，细节请参考 [Paddle1.8静态图版本](https://github.com/PaddlePaddle/models/tree/release/1.8/PaddleNLP/dialogue_system/dialogue_general_understanding)。
### 模型训练
运行如下命令即可在训练集 (train.tsv) 上进行模型训练，并在开发集 (dev.tsv) 验证，训练结束后会在测试集 (test.txt) 上进行模型评估
```shell
# GPU启动，gpus指定训练所用的GPU卡号，可以是单卡，也可以多卡。默认会进行训练、验证和评估
unset CUDA_VISIBLE_DEVICES
python -m paddle.distributed.launch --gpus "0" --log_dir ./log main.py --task_name=udc --data_dir=./DGU_datasets/udc --output_dir=./checkpoints/udc --device=gpu
# 若只需进行评估，do_train设为False，并且必须指定init_from_ckpt
# python -m paddle.distributed.launch --gpus "0" --log_dir ./log main.py --task_name=udc --data_dir=./DGU_datasets/udc --do_train=False --init_from_ckpt=./checkpoints/udc/best --device=gpu
```
以上参数表示：
* `task_name`：任务名称，可以为 udc、dstc2、atis_slot、atis_intent、mrda 或 swda。
* `data_dir`：训练数据路径。
* `output_dir`：训练保存模型的文件路径。
* `do_train：是否进行训练，默认为`True`。
* `init_from_ckpt`：恢复模型参数的路径。
* `device`：表示训练使用的设备。
其他可选参数和参数的默认值请参考`args.py`。
程序运行时将会自动进行训练，验证和评估。同时训练过程中会自动保存模型在指定的`output_dir`中。
如：
```text
checkpoints/
├── 1000.pdopt
├── 1000.pdparams
├── 2000.pdopt
├── 2000.pdparams
├── ...
├── best.pdopt
└── best.pdparams
```
**NOTE:** 如需恢复模型训练，则 init_from_ckpt 只需指定到文件名即可，不需要添加文件尾缀。如`--init_from_ckpt=checkpoints/1000`即可，程序会自动加载模型参数`checkpoints/1000.pdparams`，也会自动加载优化器状态`checkpoints/1000.pdopt`。

118
slm/examples/dialogue/dgu/args.py
View File

@@ -1,118 +0,0 @@
# Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import argparse
# yapf: disable
def parse_args():
parser = argparse.ArgumentParser(__doc__)
parser.add_argument("--task_name", default=None, type=str, required=True, help="The name of the task to train.")
parser.add_argument("--model_name_or_path", default='bert-base-uncased', type=str, help="Path to pre-trained bert model or shortcut name.")
parser.add_argument("--output_dir", default=None, type=str, help="The output directory where the checkpoints will be saved.")
parser.add_argument("--data_dir", default=None, type=str, help="The directory where the dataset will be load.")
parser.add_argument("--init_from_ckpt", default=None, type=str, help="The path of checkpoint to be loaded.")
parser.add_argument("--max_seq_len", default=None, type=int, help="The maximum total input sequence length after tokenization for trainng. Sequences longer than this will be truncated, sequences shorter will be padded.")
parser.add_argument("--test_max_seq_len", default=None, type=int, help="The maximum total input sequence length after tokenization for testing. Sequences longer than this will be truncated, sequences shorter will be padded.")
parser.add_argument("--batch_size", default=None, type=int, help="Batch size per GPU/CPU for training.")
parser.add_argument("--test_batch_size", default=None, type=int, help="Batch size per GPU/CPU for testing.")
parser.add_argument("--learning_rate", default=None, type=float, help="The initial learning rate for Adam.")
parser.add_argument("--weight_decay", default=0.01, type=float, help="Weight decay if we apply some.")
parser.add_argument("--epochs", default=None, type=int, help="Total number of training epochs to perform.")
parser.add_argument("--logging_steps", default=None, type=int, help="Log every X updates steps.")
parser.add_argument("--save_steps", default=None, type=int, help="Save checkpoint every X updates steps.")
parser.add_argument("--seed", default=42, type=int, help="Random seed for initialization.")
parser.add_argument("--warmup_proportion", default=0.1, type=float, help="The proportion of warmup.")
parser.add_argument("--max_grad_norm", default=1.0, type=float, help="The max value of grad norm.")
parser.add_argument("--do_train", default=True, type=eval, help="Whether training.")
parser.add_argument("--do_eval", default=True, type=eval, help="Whether evaluation.")
parser.add_argument("--do_test", default=True, type=eval, help="Whether testing.")
parser.add_argument("--device", type=str, default="gpu", help="Device for selecting for the training.")
args = parser.parse_args()
return args
# yapf: enable
def set_default_args(args):
args.task_name = args.task_name.lower()
if args.task_name == "udc":
if not args.save_steps:
args.save_steps = 1000
if not args.logging_steps:
args.logging_steps = 100
if not args.epochs:
args.epochs = 2
if not args.max_seq_len:
args.max_seq_len = 210
if not args.test_batch_size:
args.test_batch_size = 100
elif args.task_name == "dstc2":
if not args.save_steps:
args.save_steps = 400
if not args.logging_steps:
args.logging_steps = 20
if not args.epochs:
args.epochs = 40
if not args.learning_rate:
args.learning_rate = 5e-5
if not args.max_seq_len:
args.max_seq_len = 256
if not args.test_max_seq_len:
args.test_max_seq_len = 512
elif args.task_name == "atis_slot":
if not args.save_steps:
args.save_steps = 100
if not args.logging_steps:
args.logging_steps = 10
if not args.epochs:
args.epochs = 50
elif args.task_name == "atis_intent":
if not args.save_steps:
args.save_steps = 100
if not args.logging_steps:
args.logging_steps = 10
if not args.epochs:
args.epochs = 20
elif args.task_name == "mrda":
if not args.save_steps:
args.save_steps = 500
if not args.logging_steps:
args.logging_steps = 200
if not args.epochs:
args.epochs = 7
elif args.task_name == "swda":
if not args.save_steps:
args.save_steps = 500
if not args.logging_steps:
args.logging_steps = 200
if not args.epochs:
args.epochs = 3
else:
raise ValueError("Not support task: %s." % args.task_name)
if not args.data_dir:
args.data_dir = "./DGU_datasets/" + args.task_name
if not args.output_dir:
args.output_dir = "./checkpoints/" + args.task_name
if not args.learning_rate:
args.learning_rate = 2e-5
if not args.batch_size:
args.batch_size = 32
if not args.test_batch_size:
args.test_batch_size = args.batch_size
if not args.max_seq_len:
args.max_seq_len = 128
if not args.test_max_seq_len:
args.test_max_seq_len = args.max_seq_len

509
slm/examples/dialogue/dgu/data.py
View File

@@ -1,509 +0,0 @@
# Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import os
from typing import List
import numpy as np
from paddle.io import Dataset
# The input data bigin with '[CLS]', using '[SEP]' split conversation content(
# Previous part, current part, following part, etc.). If there are multiple
# conversation in split part, using 'INNER_SEP' to further split.
INNER_SEP = "[unused0]"
def get_label_map(label_list):
"""Create label maps"""
label_map = {}
for (i, l) in enumerate(label_list):
label_map[l] = i
return label_map
class UDCv1(Dataset):
"""
The UDCv1 dataset is using in task Dialogue Response Selection.
The source dataset is UDCv1(Ubuntu Dialogue Corpus v1.0). See detail at
http://dataset.cs.mcgill.ca/ubuntu-corpus-1.0/
"""
MAX_LEN_OF_RESPONSE = 60
LABEL_MAP = get_label_map(["0", "1"])
def __init__(self, data_dir, mode="train"):
super(UDCv1, self).__init__()
self._data_dir = data_dir
self._mode = mode
self.read_data()
def read_data(self):
if self._mode == "train":
data_path = os.path.join(self._data_dir, "train.txt")
elif self._mode == "dev":
data_path = os.path.join(self._data_dir, "dev.txt-small")
elif self._mode == "test":
data_path = os.path.join(self._data_dir, "test.txt")
self.data = []
with open(data_path, "r", encoding="utf8") as fin:
for line in fin:
if not line:
continue
arr = line.rstrip("\n").split("\t")
if len(arr) < 3:
print("Data format error: %s" % "\t".join(arr))
print("Data row contains at least three parts: label\tconversation1\t.....\tresponse.")
continue
label = arr[0]
text_a = arr[1:-1]
text_b = arr[-1]
self.data.append([label, text_a, text_b])
@classmethod
def get_label(cls, label):
return cls.LABEL_MAP[label]
@classmethod
def num_classes(cls):
return len(cls.LABEL_MAP)
@classmethod
def convert_example(cls, example, tokenizer, max_seq_length=512):
"""Convert a glue example into necessary features."""
def _truncate_and_concat(text_a: List[str], text_b: str, tokenizer, max_seq_length):
tokens_b = tokenizer.tokenize(text_b)
tokens_b = tokens_b[: min(cls.MAX_LEN_OF_RESPONSE, len(tokens_b))]
tokens_a = []
for text in text_a:
tokens_a.extend(tokenizer.tokenize(text))
tokens_a.append(INNER_SEP)
tokens_a = tokens_a[:-1]
if len(tokens_a) > max_seq_length - len(tokens_b) - 3:
tokens_a = tokens_a[len(tokens_a) - max_seq_length + len(tokens_b) + 3 :]
tokens, segment_ids = [], []
tokens.extend([tokenizer.cls_token] + tokens_a + [tokenizer.sep_token])
segment_ids.extend([0] * len(tokens))
tokens.extend(tokens_b + [tokenizer.sep_token])
segment_ids.extend([1] * (len(tokens_b) + 1))
input_ids = tokenizer.convert_tokens_to_ids(tokens)
return input_ids, segment_ids
label, text_a, text_b = example
label = np.array([cls.get_label(label)], dtype="int64")
input_ids, segment_ids = _truncate_and_concat(text_a, text_b, tokenizer, max_seq_length)
return input_ids, segment_ids, label
def __getitem__(self, index):
return self.data[index]
def __len__(self):
return len(self.data)
class DSTC2(Dataset):
"""
The dataset DSTC2 is using in task Dialogue State Tracking.
The source dataset is DSTC2(Dialog State Tracking Challenges 2). See detail at
https://github.com/matthen/dstc
"""
LABEL_MAP = get_label_map([str(i) for i in range(217)])
def __init__(self, data_dir, mode="train"):
super(DSTC2, self).__init__()
self._data_dir = data_dir
self._mode = mode
self.read_data()
def read_data(self):
def _concat_dialogues(examples):
"""concat multi turns dialogues"""
new_examples = []
max_turns = 20
for i in range(len(examples)):
multi_turns = examples[max(i - max_turns, 0) : i + 1]
new_qa = "\1".join([example[0] for example in multi_turns])
new_examples.append((new_qa.split("\1"), examples[i][1]))
return new_examples
if self._mode == "train":
data_path = os.path.join(self._data_dir, "train.txt")
elif self._mode == "dev":
data_path = os.path.join(self._data_dir, "dev.txt")
elif self._mode == "test":
data_path = os.path.join(self._data_dir, "test.txt")
self.data = []
with open(data_path, "r", encoding="utf8") as fin:
pre_idx = -1
examples = []
for line in fin:
if not line:
continue
arr = line.rstrip("\n").split("\t")
if len(arr) != 3:
print("Data format error: %s" % "\t".join(arr))
print("Data row should contains three parts: id\tquestion\1answer\tlabel1 label2 ...")
continue
idx = arr[0]
qa = arr[1]
label_list = arr[2].split()
if idx != pre_idx:
if idx != 0:
examples = _concat_dialogues(examples)
self.data.extend(examples)
examples = []
pre_idx = idx
examples.append((qa, label_list))
if examples:
examples = _concat_dialogues(examples)
self.data.extend(examples)
@classmethod
def get_label(cls, label):
return cls.LABEL_MAP[label]
@classmethod
def num_classes(cls):
return len(cls.LABEL_MAP)
@classmethod
def convert_example(cls, example, tokenizer, max_seq_length=512):
"""Convert a glue example into necessary features."""
def _truncate_and_concat(texts: List[str], tokenizer, max_seq_length):
tokens = []
for text in texts:
tokens.extend(tokenizer.tokenize(text))
tokens.append(INNER_SEP)
tokens = tokens[:-1]
if len(tokens) > max_seq_length - 2:
tokens = tokens[len(tokens) - max_seq_length + 2 :]
tokens = [tokenizer.cls_token] + tokens + [tokenizer.sep_token]
segment_ids = [0] * len(tokens)
input_ids = tokenizer.convert_tokens_to_ids(tokens)
return input_ids, segment_ids
texts, labels = example
input_ids, segment_ids = _truncate_and_concat(texts, tokenizer, max_seq_length)
labels = [cls.get_label(l) for l in labels]
label = np.zeros(cls.num_classes(), dtype="int64")
for l in labels:
label[l] = 1
return input_ids, segment_ids, label
def __getitem__(self, index):
return self.data[index]
def __len__(self):
return len(self.data)
class ATIS_DSF(Dataset):
"""
The dataset ATIS_DSF is using in task Dialogue Slot Filling.
The source dataset is ATIS(Airline Travel Information System). See detail at
https://www.kaggle.com/siddhadev/ms-cntk-atis
"""
LABEL_MAP = get_label_map([str(i) for i in range(130)])
def __init__(self, data_dir, mode="train"):
super(ATIS_DSF, self).__init__()
self._data_dir = data_dir
self._mode = mode
self.read_data()
def read_data(self):
if self._mode == "train":
data_path = os.path.join(self._data_dir, "train.txt")
elif self._mode == "dev":
data_path = os.path.join(self._data_dir, "dev.txt")
elif self._mode == "test":
data_path = os.path.join(self._data_dir, "test.txt")
self.data = []
with open(data_path, "r", encoding="utf8") as fin:
for line in fin:
if not line:
continue
arr = line.rstrip("\n").split("\t")
if len(arr) != 2:
print("Data format error: %s" % "\t".join(arr))
print("Data row should contains two parts: conversation_content\tlabel1 label2 label3.")
continue
text = arr[0]
label_list = arr[1].split()
self.data.append([text, label_list])
@classmethod
def get_label(cls, label):
return cls.LABEL_MAP[label]
@classmethod
def num_classes(cls):
return len(cls.LABEL_MAP)
@classmethod
def convert_example(cls, example, tokenizer, max_seq_length=512):
"""Convert a glue example into necessary features."""
text, labels = example
tokens, label_list = [], []
words = text.split()
assert len(words) == len(labels)
for word, label in zip(words, labels):
piece_words = tokenizer.tokenize(word)
tokens.extend(piece_words)
label = cls.get_label(label)
label_list.extend([label] * len(piece_words))
if len(tokens) > max_seq_length - 2:
tokens = tokens[len(tokens) - max_seq_length + 2 :]
label_list = label_list[len(tokens) - max_seq_length + 2 :]
tokens = [tokenizer.cls_token] + tokens + [tokenizer.sep_token]
label_list = [0] + label_list + [0]
segment_ids = [0] * len(tokens)
input_ids = tokenizer.convert_tokens_to_ids(tokens)
label = np.array(label_list, dtype="int64")
return input_ids, segment_ids, label
def __getitem__(self, index):
return self.data[index]
def __len__(self):
return len(self.data)
class ATIS_DID(Dataset):
"""
The dataset ATIS_ID is using in task Dialogue Intent Detection.
The source dataset is ATIS(Airline Travel Information System). See detail at
https://www.kaggle.com/siddhadev/ms-cntk-atis
"""
LABEL_MAP = get_label_map([str(i) for i in range(26)])
def __init__(self, data_dir, mode="train"):
super(ATIS_DID, self).__init__()
self._data_dir = data_dir
self._mode = mode
self.read_data()
def read_data(self):
if self._mode == "train":
data_path = os.path.join(self._data_dir, "train.txt")
elif self._mode == "dev":
data_path = os.path.join(self._data_dir, "dev.txt")
elif self._mode == "test":
data_path = os.path.join(self._data_dir, "test.txt")
self.data = []
with open(data_path, "r", encoding="utf8") as fin:
for line in fin:
if not line:
continue
arr = line.rstrip("\n").split("\t")
if len(arr) != 2:
print("Data format error: %s" % "\t".join(arr))
print("Data row should contains two parts: label\tconversation_content.")
continue
label = arr[0]
text = arr[1]
self.data.append([label, text])
@classmethod
def get_label(cls, label):
return cls.LABEL_MAP[label]
@classmethod
def num_classes(cls):
return len(cls.LABEL_MAP)
@classmethod
def convert_example(cls, example, tokenizer, max_seq_length=512):
"""Convert a glue example into necessary features."""
label, text = example
tokens = tokenizer.tokenize(text)
if len(tokens) > max_seq_length - 2:
tokens = tokens[len(tokens) - max_seq_length + 2 :]
tokens = [tokenizer.cls_token] + tokens + [tokenizer.sep_token]
segment_ids = [0] * len(tokens)
input_ids = tokenizer.convert_tokens_to_ids(tokens)
label = np.array([cls.get_label(label)], dtype="int64")
return input_ids, segment_ids, label
def __getitem__(self, index):
return self.data[index]
def __len__(self):
return len(self.data)
def read_da_data(data_dir, mode):
def _concat_dialogues(examples):
"""concat multi turns dialogues"""
new_examples = []
for i in range(len(examples)):
label, caller, text = examples[i]
cur_txt = "%s : %s" % (caller, text)
pre_txt = ["%s : %s" % (item[1], item[2]) for item in examples[max(0, i - 5) : i]]
suf_txt = ["%s : %s" % (item[1], item[2]) for item in examples[i + 1 : min(len(examples), i + 3)]]
sample = [label, pre_txt, cur_txt, suf_txt]
new_examples.append(sample)
return new_examples
if mode == "train":
data_path = os.path.join(data_dir, "train.txt")
elif mode == "dev":
data_path = os.path.join(data_dir, "dev.txt")
elif mode == "test":
data_path = os.path.join(data_dir, "test.txt")
data = []
with open(data_path, "r", encoding="utf8") as fin:
pre_idx = -1
examples = []
for line in fin:
if not line:
continue
arr = line.rstrip("\n").split("\t")
if len(arr) != 4:
print("Data format error: %s" % "\t".join(arr))
print("Data row should contains four parts: id\tlabel\tcaller\tconversation_content.")
continue
idx, label, caller, text = arr
if idx != pre_idx:
if idx != 0:
examples = _concat_dialogues(examples)
data.extend(examples)
examples = []
pre_idx = idx
examples.append((label, caller, text))
if examples:
examples = _concat_dialogues(examples)
data.extend(examples)
return data
def truncate_and_concat(
pre_txt: List[str], cur_txt: str, suf_txt: List[str], tokenizer, max_seq_length, max_len_of_cur_text
):
cur_tokens = tokenizer.tokenize(cur_txt)
cur_tokens = cur_tokens[: min(max_len_of_cur_text, len(cur_tokens))]
pre_tokens = []
for text in pre_txt:
pre_tokens.extend(tokenizer.tokenize(text))
pre_tokens.append(INNER_SEP)
pre_tokens = pre_tokens[:-1]
suf_tokens = []
for text in suf_txt:
suf_tokens.extend(tokenizer.tokenize(text))
suf_tokens.append(INNER_SEP)
suf_tokens = suf_tokens[:-1]
if len(cur_tokens) + len(pre_tokens) + len(suf_tokens) > max_seq_length - 4:
left_num = max_seq_length - 4 - len(cur_tokens)
if len(pre_tokens) > len(suf_tokens):
suf_num = int(left_num / 2)
suf_tokens = suf_tokens[:suf_num]
pre_num = left_num - len(suf_tokens)
pre_tokens = pre_tokens[max(0, len(pre_tokens) - pre_num) :]
else:
pre_num = int(left_num / 2)
pre_tokens = pre_tokens[max(0, len(pre_tokens) - pre_num) :]
suf_num = left_num - len(pre_tokens)
suf_tokens = suf_tokens[:suf_num]
tokens, segment_ids = [], []
tokens.extend([tokenizer.cls_token] + pre_tokens + [tokenizer.sep_token])
segment_ids.extend([0] * len(tokens))
tokens.extend(cur_tokens + [tokenizer.sep_token])
segment_ids.extend([1] * (len(cur_tokens) + 1))
if suf_tokens:
tokens.extend(suf_tokens + [tokenizer.sep_token])
segment_ids.extend([0] * (len(suf_tokens) + 1))
input_ids = tokenizer.convert_tokens_to_ids(tokens)
return input_ids, segment_ids
class MRDA(Dataset):
"""
The dataset MRDA is using in task Dialogue Act.
The source dataset is MRDA(Meeting Recorder Dialogue Act). See detail at
https://www.aclweb.org/anthology/W04-2319.pdf
"""
MAX_LEN_OF_CUR_TEXT = 50
LABEL_MAP = get_label_map([str(i) for i in range(5)])
def __init__(self, data_dir, mode="train"):
super(MRDA, self).__init__()
self.data = read_da_data(data_dir, mode)
@classmethod
def get_label(cls, label):
return cls.LABEL_MAP[label]
@classmethod
def num_classes(cls):
return len(cls.LABEL_MAP)
@classmethod
def convert_example(cls, example, tokenizer, max_seq_length=512):
"""Convert a glue example into necessary features."""
label, pre_txt, cur_txt, suf_txt = example
label = np.array([cls.get_label(label)], dtype="int64")
input_ids, segment_ids = truncate_and_concat(
pre_txt, cur_txt, suf_txt, tokenizer, max_seq_length, cls.MAX_LEN_OF_CUR_TEXT
)
return input_ids, segment_ids, label
def __getitem__(self, index):
return self.data[index]
def __len__(self):
return len(self.data)
class SwDA(Dataset):
"""
The dataset SwDA is using in task Dialogue Act.
The source dataset is SwDA(Switchboard Dialog Act). See detail at
http://compprag.christopherpotts.net/swda.html
"""
MAX_LEN_OF_CUR_TEXT = 50
LABEL_MAP = get_label_map([str(i) for i in range(42)])
def __init__(self, data_dir, mode="train"):
super(SwDA, self).__init__()
self.data = read_da_data(data_dir, mode)
@classmethod
def get_label(cls, label):
return cls.LABEL_MAP[label]
@classmethod
def num_classes(cls):
return len(cls.LABEL_MAP)
@classmethod
def convert_example(cls, example, tokenizer, max_seq_length=512):
"""Convert a glue example into necessary features."""
label, pre_txt, cur_txt, suf_txt = example
label = np.array([cls.get_label(label)], dtype="int64")
input_ids, segment_ids = truncate_and_concat(
pre_txt, cur_txt, suf_txt, tokenizer, max_seq_length, cls.MAX_LEN_OF_CUR_TEXT
)
return input_ids, segment_ids, label
def __getitem__(self, index):
return self.data[index]
def __len__(self):
return len(self.data)

293
slm/examples/dialogue/dgu/main.py
View File

@@ -1,293 +0,0 @@
# Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import os
import random
import time
from functools import partial
import data
import metric
import numpy as np
import paddle
import paddle.distributed as dist
import paddle.nn as nn
import paddle.nn.functional as F
from args import parse_args, set_default_args
from paddle.io import BatchSampler, DataLoader, DistributedBatchSampler
from paddle.metric import Accuracy
from paddle.optimizer import AdamW
from paddlenlp.data import Pad, Stack, Tuple
from paddlenlp.datasets import MapDataset
from paddlenlp.transformers import (
BertForSequenceClassification,
BertForTokenClassification,
BertTokenizer,
LinearDecayWithWarmup,
)
TASK_CLASSES = {
"udc": (data.UDCv1, metric.RecallAtK),
"dstc2": (data.DSTC2, metric.JointAccuracy),
"atis_slot": (data.ATIS_DSF, metric.F1Score),
"atis_intent": (data.ATIS_DID, Accuracy),
"mrda": (data.MRDA, Accuracy),
"swda": (data.SwDA, Accuracy),
}
def set_seed(seed):
random.seed(seed)
np.random.seed(seed)
paddle.seed(seed)
def load_ckpt(args, model, optimizer=None):
if args.init_from_ckpt:
params_state_dict = paddle.load(args.init_from_ckpt + ".pdparams")
model.set_state_dict(params_state_dict)
if optimizer:
opt_state_dict = paddle.load(args.init_from_ckpt + ".pdopt")
optimizer.set_state_dict(opt_state_dict)
print("Loaded checkpoint from %s" % args.init_from_ckpt)
def save_ckpt(model, optimizer, output_dir, name):
params_path = os.path.join(output_dir, "{}.pdparams".format(name))
opt_path = os.path.join(output_dir, "{}.pdopt".format(name))
paddle.save(model.state_dict(), params_path)
paddle.save(optimizer.state_dict(), opt_path)
class DGULossFunction(nn.Layer):
def __init__(self, task_name):
super(DGULossFunction, self).__init__()
self.task_name = task_name
self.loss_fn = self.get_loss_fn()
def get_loss_fn(self):
if self.task_name in ["udc", "atis_slot", "atis_intent", "mrda", "swda"]:
return F.cross_entropy
elif self.task_name == "dstc2":
return nn.BCEWithLogitsLoss(reduction="sum")
def forward(self, logits, labels):
if self.task_name in ["udc", "atis_intent", "mrda", "swda"]:
loss = self.loss_fn(logits, labels)
elif self.task_name == "dstc2":
loss = self.loss_fn(logits, paddle.cast(labels, dtype=logits.dtype))
elif self.task_name == "atis_slot":
labels = paddle.unsqueeze(labels, axis=-1)
loss = self.loss_fn(logits, labels)
return loss
def print_logs(args, step, logits, labels, loss, total_time, metric):
if args.task_name in ["udc", "atis_intent", "mrda", "swda"]:
if args.task_name == "udc":
metric = Accuracy()
metric.reset()
correct = metric.compute(logits, labels)
metric.update(correct)
acc = metric.accumulate()
print("step %d - loss: %.4f - acc: %.4f - %.3fs/step" % (step, loss, acc, total_time / args.logging_steps))
elif args.task_name == "dstc2":
metric.reset()
metric.update(logits, labels)
joint_acc = metric.accumulate()
print(
"step %d - loss: %.4f - joint_acc: %.4f - %.3fs/step"
% (step, loss, joint_acc, total_time / args.logging_steps)
)
elif args.task_name == "atis_slot":
metric.reset()
metric.update(logits, labels)
f1_micro = metric.accumulate()
print(
"step %d - loss: %.4f - f1_micro: %.4f - %.3fs/step"
% (step, loss, f1_micro, total_time / args.logging_steps)
)
def train(args, model, train_data_loader, dev_data_loader, metric, n_procs, rank):
num_examples = len(train_data_loader) * args.batch_size * n_procs
max_train_steps = args.epochs * len(train_data_loader)
print("\nNum train examples: %d" % num_examples)
print("Max train steps: %d" % max_train_steps)
print("Warmup proportion: %.2f" % args.warmup_proportion)
lr_scheduler = LinearDecayWithWarmup(args.learning_rate, max_train_steps, args.warmup_proportion)
# Generate parameter names needed to perform weight decay.
# All bias and LayerNorm parameters are excluded.
decay_params = [p.name for n, p in model.named_parameters() if not any(nd in n for nd in ["bias", "norm"])]
optimizer = AdamW(
learning_rate=lr_scheduler,
parameters=model.parameters(),
weight_decay=args.weight_decay,
apply_decay_param_fun=lambda x: x in decay_params,
grad_clip=nn.ClipGradByGlobalNorm(args.max_grad_norm),
)
loss_fn = DGULossFunction(args.task_name)
load_ckpt(args, model, optimizer)
step = 0
best_metric = 0.0
total_time = 0.0
for epoch in range(args.epochs):
print("\nEpoch %d/%d" % (epoch + 1, args.epochs))
batch_start_time = time.time()
for batch in train_data_loader:
step += 1
input_ids, segment_ids, labels = batch
logits = model(input_ids, segment_ids)
loss = loss_fn(logits, labels)
loss.backward()
optimizer.step()
lr_scheduler.step()
optimizer.clear_grad()
total_time += time.time() - batch_start_time
if step % args.logging_steps == 0:
print_logs(args, step, logits, labels, loss, total_time, metric)
total_time = 0.0
if step % args.save_steps == 0 or step == max_train_steps:
if rank == 0:
save_ckpt(model, optimizer, args.output_dir, step)
if args.do_eval:
print("\nEval begin...")
metric_out = evaluation(args, model, dev_data_loader, metric)
if rank == 0 and metric_out > best_metric:
best_metric = metric_out
save_ckpt(model, optimizer, args.output_dir, "best")
print("Best model, step: %d\n" % step)
batch_start_time = time.time()
@paddle.no_grad()
def evaluation(args, model, data_loader, metric):
model.eval()
metric.reset()
for batch in data_loader:
input_ids, segment_ids, labels = batch
logits = model(input_ids, segment_ids)
if args.task_name in ["atis_intent", "mrda", "swda"]:
correct = metric.compute(logits, labels)
metric.update(correct)
else:
metric.update(logits, labels)
model.train()
metric_out = metric.accumulate()
print("Total samples: %d" % (len(data_loader) * args.test_batch_size))
if args.task_name == "udc":
print("R1@10: %.4f - R2@10: %.4f - R5@10: %.4f\n" % (metric_out[0], metric_out[1], metric_out[2]))
return metric_out[0]
elif args.task_name == "dstc2":
print("Joint_acc: %.4f\n" % metric_out)
return metric_out
elif args.task_name == "atis_slot":
print("F1_micro: %.4f\n" % metric_out)
return metric_out
elif args.task_name in ["atis_intent", "mrda", "swda"]:
print("Acc: %.4f\n" % metric_out)
return metric_out
def create_data_loader(args, dataset_class, trans_func, batchify_fn, mode):
dataset = dataset_class(args.data_dir, mode)
dataset = MapDataset(dataset).map(trans_func, lazy=True)
if mode == "train":
batch_sampler = DistributedBatchSampler(dataset, batch_size=args.batch_size, shuffle=True)
else:
batch_sampler = BatchSampler(dataset, batch_size=args.test_batch_size, shuffle=False)
data_loader = DataLoader(dataset, batch_sampler=batch_sampler, collate_fn=batchify_fn, return_list=True)
return data_loader
def main(args):
paddle.set_device(args.device)
world_size = dist.get_world_size()
rank = dist.get_rank()
if world_size > 1 and args.do_train:
dist.init_parallel_env()
set_seed(args.seed)
dataset_class, metric_class = TASK_CLASSES[args.task_name]
tokenizer = BertTokenizer.from_pretrained(args.model_name_or_path)
trans_func = partial(dataset_class.convert_example, tokenizer=tokenizer, max_seq_length=args.max_seq_len)
test_trans_func = partial(dataset_class.convert_example, tokenizer=tokenizer, max_seq_length=args.test_max_seq_len)
metric = metric_class()
if args.task_name in ("udc", "dstc2", "atis_intent", "mrda", "swda"):
batchify_fn = lambda samples, fn=Tuple(
Pad(axis=0, pad_val=tokenizer.pad_token_id), # input
Pad(axis=0, pad_val=tokenizer.pad_token_type_id), # segment
Stack(dtype="int64"), # label
): fn(samples)
model = BertForSequenceClassification.from_pretrained(
args.model_name_or_path, num_classes=dataset_class.num_classes()
)
elif args.task_name == "atis_slot":
batchify_fn = lambda samples, fn=Tuple(
Pad(axis=0, pad_val=tokenizer.pad_token_id), # input
Pad(axis=0, pad_val=tokenizer.pad_token_type_id), # segment
Pad(axis=0, pad_val=0, dtype="int64"), # label
): fn(samples)
model = BertForTokenClassification.from_pretrained(
args.model_name_or_path, num_classes=dataset_class.num_classes(), dropout=0.0
)
if world_size > 1 and args.do_train:
model = paddle.DataParallel(model)
if args.do_train:
train_data_loader = create_data_loader(args, dataset_class, trans_func, batchify_fn, "train")
if args.do_eval:
dev_data_loader = create_data_loader(args, dataset_class, test_trans_func, batchify_fn, "dev")
else:
dev_data_loader = None
train(args, model, train_data_loader, dev_data_loader, metric, world_size, rank)
if args.do_test:
if rank == 0:
test_data_loader = create_data_loader(args, dataset_class, test_trans_func, batchify_fn, "test")
if args.do_train:
# If do_eval=True, use best model to evaluate the test data.
# Otherwise, use final model to evaluate the test data.
if args.do_eval:
args.init_from_ckpt = os.path.join(args.output_dir, "best")
load_ckpt(args, model)
else:
if not args.init_from_ckpt:
raise ValueError('"init_from_ckpt" should be set.')
load_ckpt(args, model)
print("\nTest begin...")
evaluation(args, model, test_data_loader, metric)
def print_args(args):
print("----------- Configuration Arguments -----------")
for arg, value in sorted(vars(args).items()):
print("%s: %s" % (arg, value))
print("------------------------------------------------")
if __name__ == "__main__":
args = parse_args()
set_default_args(args)
print_args(args)
main(args)

244
slm/examples/dialogue/dgu/metric.py
View File

@@ -1,244 +0,0 @@
# Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import numpy as np
import paddle
import paddle.nn as nn
from paddle.metric import Metric
class RecallAtK(Metric):
"""
Recall@K is the fraction of relevant results among the retrieved Top K
results, using to evaluate the performance of Dialogue Response Selection.
Noted that this class manages the Recall@K score only for binary
classification task.
"""
def __init__(self, name="Recall@K", *args, **kwargs):
super(RecallAtK, self).__init__(*args, **kwargs)
self._name = name
self.softmax = nn.Softmax()
self.reset()
def reset(self):
"""
Resets all of the metric state.
"""
self.num_sampls = 0
self.p_at_1_in_10 = 0.0
self.p_at_2_in_10 = 0.0
self.p_at_5_in_10 = 0.0
def get_p_at_n_in_m(self, data, n, m, idx):
"""
calculate precision in recall n
"""
pos_score = data[idx][0]
curr = data[idx : idx + m]
curr = sorted(curr, key=lambda x: x[0], reverse=True)
if curr[n - 1][0] <= pos_score:
return 1
return 0
def update(self, logits, labels):
"""
Update the states based on the current mini-batch prediction results.
Args:
logits (Tensor): The predicted value is a Tensor with
shape [batch_size, 2] and type float32 or float64.
labels (Tensor): The ground truth value is a 2D Tensor,
its shape is [batch_size, 1] and type is int64.
"""
probs = self.softmax(logits)
probs = probs.numpy()
labels = labels.numpy()
assert probs.shape[0] == labels.shape[0]
data = []
for prob, label in zip(probs, labels):
data.append((prob[1], label))
assert len(data) % 10 == 0
length = int(len(data) / 10)
self.num_sampls += length
for i in range(length):
idx = i * 10
assert data[idx][1] == 1
self.p_at_1_in_10 += self.get_p_at_n_in_m(data, 1, 10, idx)
self.p_at_2_in_10 += self.get_p_at_n_in_m(data, 2, 10, idx)
self.p_at_5_in_10 += self.get_p_at_n_in_m(data, 5, 10, idx)
def accumulate(self):
"""
Calculate the final Recall@K.
Returns:
A list with scaler float: results of the calculated R1@K, R2@K, R5@K.
"""
metrics_out = [
self.p_at_1_in_10 / self.num_sampls,
self.p_at_2_in_10 / self.num_sampls,
self.p_at_5_in_10 / self.num_sampls,
]
return metrics_out
def name(self):
"""
Returns metric name
"""
return self._name
class JointAccuracy(Metric):
"""
The joint accuracy rate is used to evaluate the performance of multi-turn
Dialogue State Tracking. For each turn, if and only if all state in
state_list are correctly predicted, the dialog state prediction is
considered correct. And the joint accuracy rate is equal to 1, otherwise
it is equal to 0.
"""
def __init__(self, name="JointAccuracy", *args, **kwargs):
super(JointAccuracy, self).__init__(*args, **kwargs)
self._name = name
self.sigmoid = nn.Sigmoid()
self.reset()
def reset(self):
"""
Resets all of the metric state.
"""
self.num_samples = 0
self.correct_joint = 0.0
def update(self, logits, labels):
"""
Update the states based on the current mini-batch prediction results.
Args:
logits (Tensor): The predicted value is a Tensor with
shape [batch_size, num_classes] and type float32 or float64.
labels (Tensor): The ground truth value is a 2D Tensor,
its shape is [batch_size, num_classes] and type is int64.
"""
probs = self.sigmoid(logits)
probs = probs.numpy()
labels = labels.numpy()
assert probs.shape[0] == labels.shape[0]
assert probs.shape[1] == labels.shape[1]
for i in range(probs.shape[0]):
pred, refer = [], []
for j in range(probs.shape[1]):
if probs[i][j] >= 0.5:
pred.append(j)
if labels[i][j] == 1:
refer.append(j)
if not pred:
pred = [np.argmax(probs[i])]
if pred == refer:
self.correct_joint += 1
self.num_samples += probs.shape[0]
def accumulate(self):
"""
Calculate the final JointAccuracy.
Returns:
A scaler float: results of the calculated JointAccuracy.
"""
joint_acc = self.correct_joint / self.num_samples
return joint_acc
def name(self):
"""
Returns metric name
"""
return self._name
class F1Score(Metric):
"""
F1-score is the harmonic mean of precision and recall. Micro-averaging is
to create a global confusion matrix for all examples, and then calculate
the F1-score. This class is using to evaluate the performance of Dialogue
Slot Filling.
"""
def __init__(self, name="F1Score", *args, **kwargs):
super(F1Score, self).__init__(*args, **kwargs)
self._name = name
self.reset()
def reset(self):
"""
Resets all of the metric state.
"""
self.tp = {}
self.fn = {}
self.fp = {}
def update(self, logits, labels):
"""
Update the states based on the current mini-batch prediction results.
Args:
logits (Tensor): The predicted value is a Tensor with
shape [batch_size, seq_len, num_classes] and type float32 or
float64.
labels (Tensor): The ground truth value is a 2D Tensor,
its shape is [batch_size, seq_len] and type is int64.
"""
probs = paddle.argmax(logits, axis=-1)
probs = probs.numpy()
labels = labels.numpy()
assert probs.shape[0] == labels.shape[0]
assert probs.shape[1] == labels.shape[1]
for i in range(probs.shape[0]):
start, end = 1, probs.shape[1]
while end > start:
if labels[i][end - 1] != 0:
break
end -= 1
prob, label = probs[i][start:end], labels[i][start:end]
for y_pred, y in zip(prob, label):
if y_pred == y:
self.tp[y] = self.tp.get(y, 0) + 1
else:
self.fp[y_pred] = self.fp.get(y_pred, 0) + 1
self.fn[y] = self.fn.get(y, 0) + 1
def accumulate(self):
"""
Calculate the final micro F1 score.
Returns:
A scaler float: results of the calculated micro F1 score.
"""
tp_total = sum(self.tp.values())
fn_total = sum(self.fn.values())
fp_total = sum(self.fp.values())
p_total = float(tp_total) / (tp_total + fp_total)
r_total = float(tp_total) / (tp_total + fn_total)
if p_total + r_total == 0:
return 0
f1_micro = 2 * p_total * r_total / (p_total + r_total)
return f1_micro
def name(self):
"""
Returns metric name
"""
return self._name

146
slm/examples/dialogue/lic2021_baseline/README.md
View File

@@ -1,146 +0,0 @@
# LIC 2021对话比赛 baseline
## 模型简介
近年来，人机对话系统受到了学术界和产业界的广泛关注并取得了不错的发展。开放域对话系统旨在建立一个开放域的多轮对话系统，使得机器可以流畅自然地与人进行语言交互，既可以进行日常问候类的闲聊，又可以完成特定功能，以使得开放域对话系统具有实际应用价值，例如进行对话式推荐，或围绕一个主题进行深入的知识对话等。具体的说，开放域对话可以继续拆分为支持不同功能的对话形式，例如对话式推荐，知识对话技术等，如何解决并有效融合以上多个技能面临诸多挑战。
LIC 2021对话比赛收集了一系列公开的开放域对话数据并提供了统一的评测方式，旨在为研究人员和开发者提供学术和技术交流的平台，进一步提升开放域对话的研究水平，推动自然语言理解和人工智能领域技术的应用和发展。
为了方便参赛者快速了解 LIC 2021对话比赛的流程，并快速地参与到比赛中，本项目基于 UnifiedTransformer 模型提供了一个基础 baseline，利用小规模样例数据在预训练模型上完成了微调及预测。参赛者可以针对赛题进行其他改进，例如修改数据预处理方法、修改网络结构、修改训练方式、修改预测的解码方式或对结果的后处理策略等方式提升模型效果。
UnifiedTransformer 模型的细节可以[参阅论文](https://arxiv.org/abs/2006.16779)。
## 快速开始
### 环境依赖
- sentencepiece
安装方式：`pip install sentencepiece`
### 数据准备
由于样例数据涉及 LIC 2021对话比赛，暂不开放。
关于数据集及数据集的预处理过程，详见[2021语言与智能技术竞赛：多技能对话](https://aistudio.baidu.com/aistudio/competition/detail/67)及官方提供的基线系统 Baselines。
模型的输入由3部分组成：词向量 token_ids，句向量 token_type_ids 和位置向量 position_ids。本项目的数据集是样例文本经过数据预处理脚本得到的 id 化的数据集。数据的每一行由3列组成，以";"作为分割符，格式：token_ids;token_type_ids;position_ids。具体细节请参考`data.py`。
### 模型训练
运行如下命令即可在样例训练集上进行 finetune，并在样例验证集上进行验证
```shell
# GPU启动，参数`--gpus`指定训练所用的GPU卡号，可以是单卡，也可以多卡
unset CUDA_VISIBLE_DEVICES
python -m paddle.distributed.launch --gpus "0" --log_dir ./log finetune.py \
--model_name_or_path=unified_transformer-12L-cn \
--train_data_path=./datasets/train.txt \
--valid_data_path=./datasets/valid.txt \
--save_dir=./checkpoints \
--logging_steps=500 \
--save_steps=8000 \
--seed=2021 \
--epochs=10 \
--batch_size=8192 \
--lr=1e-5 \
--weight_decay=0.01 \
--warmup_steps=4000 \
--max_grad_norm=0.1 \
--sort_pool_size=65536 \
--device=gpu
```
其中参数释义如下：
- `gpus` 指示了训练所用的 GPU 卡号。
- `model_name_or_path` 指示了 finetune 使用的具体预训练模型，可以是 PaddleNLP 提供的预训练模型，或者是本地的预训练模型。如果使用本地的预训练模型，可以配置本地模型的目录地址，例如: ./checkpoints/model_xx/，目录中需包含 paddle 预训练模型 model_state.pdparams。如果使用 PaddleNLP 提供的预训练模型，可以选择下面其中之一。
| PaddleNLP 提供的预训练模型 |
|---------------------------------|
| unified_transformer-12L-cn |
| unified_transformer-12L-cn-luge |
- `train_data_path` 表示训练集文件路径。
- `valid_data_path` 表示验证集文件路径。
- `save_dir` 表示模型的保存路径。
- `logging_steps` 表示日志打印间隔。
- `save_steps` 表示模型保存及评估间隔。
- `seed` 表示随机数生成器的种子。
- `epochs` 表示训练轮数。
- `batch_size` 表示每次迭代**每张卡**上的样本数目。
- `lr` 表示基础学习率大小，将于 learning rate scheduler 产生的值相乘作为当前学习率。
- `weight_decay` 表示 AdamW 优化器中使用的 weight_decay 的系数。
- `warmup_steps` 表示学习率逐渐升高到基础学习率（即上面配置的 lr）所需要的迭代数，最早的使用可以参考[这篇论文](https://arxiv.org/pdf/1706.02677.pdf)。
- `max_grad_norm` 表示梯度裁剪允许的最大梯度值。
- `sort_pool_size` 表示在构建 batch 数据时，用来排序的 pool size。
- `device` 表示训练使用的设备。
参数详情和参数的默认值请参考`args.py`。
程序运行时将会自动进行训练和验证，训练过程中会自动保存模型在指定的`save_dir`中。
如：
```text
./checkpoints/
├── model_8000
│ ├── model_config.json
│ ├── model_state.pdparams
│ ├── spm.model
│ ├── tokenizer_config.json
│ └── vocab.txt
└── ...
```
**NOTE:** 如需恢复模型训练，`model_name_or_path`配置本地模型的目录地址即可。
### 模型预测
运行如下命令即可在样例测试集上进行测试
```shell
export CUDA_VISIBLE_DEVICES=0
# GPU启动，预测仅支持单卡
python infer.py \
--model_name_or_path=./checkpoints/model_80000 \
--test_data_path=./datasets/test.txt \
--output_path=./predict.txt \
--logging_steps=500 \
--seed=2021 \
--batch_size=4 \
--min_dec_len=1 \
--max_dec_len=64 \
--num_samples=20 \
--decode_strategy=sampling \
--top_k=5 \
--device=gpu
```
其中参数释义如下：
- `model_name_or_path` 指示了 finetune 使用的具体预训练模型，可以是 PaddleNLP 提供的预训练模型，或者是本地的预训练模型。如果使用本地的预训练模型，可以配置本地模型的目录地址，例如: ./checkpoints/model_xx/，目录中需包含 paddle 预训练模型 model_state.pdparams。如果使用 PaddleNLP 提供的预训练模型，可以选择下面其中之一。
| PaddleNLP 提供的预训练模型 |
|---------------------------------|
| unified_transformer-12L-cn |
| unified_transformer-12L-cn-luge |
- `test_data_path` 表示预测集文件路径。
- `output_path` 表示预测结果的保存路径。
- `logging_steps` 表示日志打印间隔。
- `seed` 表示随机数生成器的种子。
- `batch_size` 表示每次迭代**每张卡**上的样本数目。
- `min_dec_len` 表示预测生成的句子的最小长度。
- `max_dec_len` 表示预测生成的句子的最大长度。
- `num_samples` 表示每条样本生成的句子的数量。对于每条样本，模型会生成`num_samples`个句子，根据每个句子的概率得分进行排序，得分最高的句子作为最终的生成结果。
- `decode_strategy` 表示预测解码时采取的策略，可选"sampling"、"greedy_search"和"beam_search"之一。
- `top_k` 表示采用"sampling"解码策略时，token 的概率按从大到小排序，生成的 token 只从前`top_k`个中进行采样。
- `device` 表示训练使用的设备。
参数详情和参数的默认值请参考`args.py`。
程序运行结束后会将预测结果保存在`output_path`中。将预测结果准备成比赛官网要求的格式，提交评估即可得评估结果。
采用不同的模型在样例测试集上有如下结果：
| model_name_or_path | F1 | BLEU1 / BLEU2 | DISTINCT1 / DISTINCT2 |
| :-----------------------------: | :---: | :-----------: | :-------------------: |
| unified_transformer-12L-cn | 10.62 | 0.070 / 0.022 | 0.065 / 0.304 |
| unified_transformer-12L-cn-luge | 33.11 | 0.245 / 0.157 | 0.074 / 0.238 |
| ./checkpoints/model_80000 | 32.38 | 0.239 / 0.150 | 0.070 / 0.219 |

58
slm/examples/dialogue/lic2021_baseline/args.py
View File

@@ -1,58 +0,0 @@
# Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import argparse
# yapf: disable
def parse_args():
parser = argparse.ArgumentParser(__doc__)
parser.add_argument('--model_name_or_path', type=str, default='unified_transformer-12L-cn', help='The path or shortcut name of the pre-trained model.')
parser.add_argument('--save_dir', type=str, default='./checkpoints', help='The directory where the checkpoints will be saved.')
parser.add_argument('--output_path', type=str, default='./predict.txt', help='The file path where the infer result will be saved.')
parser.add_argument('--train_data_path', type=str, default='./datasets/train.txt', help='Specify the path to load train data.')
parser.add_argument('--valid_data_path', type=str, default='./datasets/valid.txt', help='Specify the path to load valid data.')
parser.add_argument('--test_data_path', type=str, default='./datasets/test.txt', help='Specify the path to load test data.')
parser.add_argument('--logging_steps', type=int, default=500, help='Log every X updates steps.')
parser.add_argument('--save_steps', type=int, default=8000, help='Save checkpoint every X updates steps.')
parser.add_argument('--seed', type=int, default=2021, help='Random seed for initialization.')
parser.add_argument('--batch_size', type=int, default=8192, required=True, help='Batch size per GPU/CPU for training.')
parser.add_argument('--lr', type=float, default=1e-5, help='The initial learning rate.')
parser.add_argument('--weight_decay', type=float, default=0.01, help='The weight decay for optimizer.')
parser.add_argument('--epochs', type=int, default=10, help='Total number of training epochs to perform.')
parser.add_argument('--warmup_steps', type=int, default=4000, help='The number of warmup steps.')
parser.add_argument('--max_grad_norm', type=float, default=0.1, help='The max value of grad norm.')
parser.add_argument('--sort_pool_size', type=int, default=65536, help='The pool size for sort in build batch data.')
parser.add_argument('--min_dec_len', type=int, default=1, help='The minimum sequence length of generation.')
parser.add_argument('--max_dec_len', type=int, default=64, help='The maximum sequence length of generation.')
parser.add_argument('--num_samples', type=int, default=1, help='The decode numbers in generation.')
parser.add_argument('--decode_strategy', type=str, default='sampling', help='The decode strategy in generation.')
parser.add_argument('--top_k', type=int, default=0, help='The number of highest probability vocabulary tokens to keep for top-k sampling.')
parser.add_argument('--temperature', type=float, default=1.0, help='The value used to module the next token probabilities.')
parser.add_argument('--top_p', type=float, default=1.0, help='The cumulative probability for top-p sampling.')
parser.add_argument('--num_beams', type=int, default=0, help='The number of beams for beam search.')
parser.add_argument('--length_penalty', type=float, default=1.0, help='The exponential penalty to the sequence length for beam search.')
parser.add_argument('--early_stopping', type=eval, default=False, help='Whether to stop the beam search when at least `num_beams` sentences are finished per batch or not.')
parser.add_argument('--device', type=str, default='gpu', help='Device for selecting for the training.')
args = parser.parse_args()
return args
# yapf: enable
def print_args(args):
print("----------- Configuration Arguments -----------")
for arg, value in sorted(vars(args).items()):
print("%s: %s" % (arg, value))
print("------------------------------------------------")

258
slm/examples/dialogue/lic2021_baseline/data.py
View File

@@ -1,258 +0,0 @@
# Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from glob import glob
import numpy as np
import paddle.distributed as dist
from paddle.io import IterableDataset
from paddlenlp.transformers.tokenizer_utils import convert_to_unicode
class DialogueDataset(IterableDataset):
def __init__(
self,
filepattern,
batch_size,
pad_token_id,
bos_token_id,
sort_pool_size=2**16,
seed=1,
n_procs=None,
rank=None,
mode="test",
):
super(DialogueDataset, self).__init__()
self.file_list = glob(filepattern)
self.sort_pool_size = 0 if mode == "test" else sort_pool_size
self.n_procs = n_procs if n_procs else dist.get_world_size()
self.rank = rank if rank else dist.get_rank()
self.batch_size = batch_size * self.n_procs
self.shuffle = True if mode == "train" else False
self.mode = mode
self.pad_id = pad_token_id
self.bos_id = bos_token_id
self.global_rng = np.random.RandomState(seed)
assert len(self.file_list) > 0, "There is no files in %s." % filepattern
def load_file(self, file_path):
with open(file_path, "r", encoding="utf-8") as fin:
for i, line in enumerate(fin):
cols = convert_to_unicode(line).strip().split(";")
cols = list(map(lambda x: list(map(int, x.split(" "))), cols))
if len(cols) > 3:
cols = cols[:3]
token_ids, type_ids, pos_ids = cols
if self.mode == "test":
tgt_start_idx = len(cols[0])
else:
tgt_start_idx = token_ids.index(self.bos_id, 1)
sample = [token_ids, type_ids, pos_ids, tgt_start_idx]
yield sample
def get_sorted_batch(self, pool):
"""Generate sorted batches from pool."""
pool = sorted(pool, key=lambda sample: len(sample[0]))
batches = []
batch, max_len = [], 0
for sample in pool:
max_len = max(max_len, len(sample[0]))
if self.mode == "test":
to_append = len(batch) < self.batch_size
else:
to_append = (len(batch) + 1) * max_len <= self.batch_size
if to_append:
batch.append(sample)
else:
batches.append(batch)
batch, max_len = [sample], len(sample[0])
if len(batch) > 0:
batches.append(batch)
if self.shuffle:
self.global_rng.shuffle(batches)
for batch in batches:
yield batch
@property
def get_batch(self):
all_files = list(self.file_list)
if self.shuffle:
self.global_rng.shuffle(all_files)
if self.sort_pool_size > 0:
pool = []
for file_path in all_files:
for sample in self.load_file(file_path):
pool.append(sample)
if len(pool) == self.sort_pool_size:
for batch in self.get_sorted_batch(pool):
yield batch
pool = []
if len(pool) > 0:
for batch in self.get_sorted_batch(pool):
yield batch
else:
batch, max_len = [], 0
for file_path in all_files:
for sample in self.load_file(file_path):
max_len = max(max_len, len(sample[0]))
if self.mode == "test":
to_append = len(batch) < self.batch_size
else:
to_append = (len(batch) + 1) * max_len <= self.batch_size
if to_append:
batch.append(sample)
else:
yield batch
batch, max_len = [sample], len(sample[0])
if len(batch) > 0:
yield batch
def pad_batch_data(self, batch):
"""Pad the instances to the max sequence length in batch."""
max_len = max(map(len, batch))
batch_data = np.array([list(data) + [self.pad_id] * (max_len - len(data)) for data in batch], dtype="int64")
return batch_data
def gen_tgt_label_and_pos(self, batch_token_ids, batch_tgt_start_idx):
max_len = max(map(len, batch_token_ids))
tgt_label = []
tgt_pos = []
for sent_index, sent in enumerate(batch_token_ids):
sent_b_index = batch_tgt_start_idx[sent_index]
tgt_label.extend(sent[sent_b_index + 1 :])
tgt_pos.extend([sent_index * max_len + i for i in range(sent_b_index, len(sent) - 1)])
tgt_label = np.array(tgt_label).astype("int64")
tgt_pos = np.array(tgt_pos).astype("int64")
return tgt_label, tgt_pos
def gen_self_attn_mask(self, batch_token_ids, batch_tgt_start_idx):
max_len = max(map(len, batch_token_ids))
input_mask_data = np.zeros((len(batch_token_ids), max_len, max_len))
for index, mask_data in enumerate(input_mask_data):
start = batch_tgt_start_idx[index]
end = len(batch_token_ids[index])
mask_data[:end, :start] = 1.0
# Generate the lower triangular matrix using the slice of matrix
b = np.tril(np.ones([end - start, end - start]), 0)
mask_data[start:end, start:end] = b
return input_mask_data.astype("float32")
def __iter__(self):
for batch_data in self.get_batch:
# sample [token_ids, type_ids, pos_ids, tgt_start_idx]
# raw_batch [sample0, sample1, ...]
if self.n_procs > 1:
batch_data = batch_data[self.rank :: self.n_procs]
batch_data = zip(*batch_data)
token_ids, type_ids, pos_ids, tgt_start_idx = batch_data
pad_token_ids = self.pad_batch_data(token_ids)
pad_type_ids = self.pad_batch_data(type_ids)
pad_pos_ids = self.pad_batch_data(pos_ids)
generation_mask = self.gen_self_attn_mask(token_ids, tgt_start_idx)
if self.mode == "test":
# [batch_size, 1]
tgt_ids = np.array([[self.bos_id]] * len(token_ids), dtype="int64")
tgt_type = np.ones((len(token_ids), 1), dtype="int64")
tgt_pos = np.array(tgt_start_idx, dtype="int64").reshape(-1, 1)
tgt_generation_mask = generation_mask[:, 0:1, :].astype("float32")
pad_token_ids = np.concatenate((pad_token_ids, tgt_ids), axis=1)
pad_type_ids = np.concatenate((pad_type_ids, tgt_type), axis=1)
pad_pos_ids = np.concatenate((pad_pos_ids, tgt_pos), axis=1)
generation_mask = np.concatenate((generation_mask, tgt_generation_mask), axis=1)
append_mask = np.zeros((generation_mask.shape[0], generation_mask.shape[1], 1), dtype="float32")
append_mask[:, -1, :] = 1.0
generation_mask = np.concatenate((generation_mask, append_mask), axis=2)
generation_mask = (generation_mask - 1.0) * 1e9
generation_mask = np.expand_dims(generation_mask, axis=1)
yield (pad_token_ids, pad_type_ids, pad_pos_ids, generation_mask)
else:
tgt_label, tgt_pos = self.gen_tgt_label_and_pos(token_ids, tgt_start_idx)
generation_mask = (generation_mask - 1.0) * 1e9
generation_mask = np.expand_dims(generation_mask, axis=1)
yield (pad_token_ids, pad_type_ids, pad_pos_ids, generation_mask, tgt_label, tgt_pos)
def post_process_response(token_ids, tokenizer):
"""
Post-process the decoded sequence. Truncate from the first
<eos> and remove the <bos> and <eos> tokens currently.
"""
eos_pos = len(token_ids)
for i, tok_id in enumerate(token_ids):
if tok_id == tokenizer.sep_token_id:
eos_pos = i
break
token_ids = token_ids[:eos_pos]
tokens = tokenizer.convert_ids_to_tokens(token_ids)
response = tokenizer.merge_subword(tokens)
return token_ids, response
def get_in_turn_repetition(pred, is_cn=False):
"""Get in-turn repetition."""
if len(pred) == 0:
return 1.0
if isinstance(pred[0], str):
pred = [tok.lower() for tok in pred]
if is_cn:
pred = "".join(pred)
tri_grams = set()
for i in range(len(pred) - 2):
tri_gram = tuple(pred[i : i + 3])
if tri_gram in tri_grams:
return True
tri_grams.add(tri_gram)
return False
def select_response(ids, scores, tokenizer, max_dec_len=None, num_samples=1):
ids = ids.numpy().tolist()
scores = scores.numpy()
if len(ids) != len(scores) or (len(ids) % num_samples) != 0:
raise ValueError("the length of `ids` is {}, but the `num_samples` is {}".format(len(ids), num_samples))
group = []
tmp = []
for pred, score in zip(ids, scores):
pred_token_ids, pred_tokens = post_process_response(pred, tokenizer)
num_token = len(pred_token_ids)
response = " ".join(pred_tokens)
in_turn_repetition = get_in_turn_repetition(pred_tokens, True) or get_in_turn_repetition(pred_token_ids)
# not ending
if max_dec_len is not None and num_token >= max_dec_len:
score -= 1e3
elif in_turn_repetition:
score -= 1e3
tmp.append([response, score])
if len(tmp) == num_samples:
group.append(tmp)
tmp = []
results = []
for preds in group:
preds = sorted(preds, key=lambda x: -x[1])
results.append(preds[0][0])
return results

149
slm/examples/dialogue/lic2021_baseline/finetune.py
View File

@@ -1,149 +0,0 @@
# Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import math
import os
import time
import paddle
import paddle.distributed as dist
import paddle.nn as nn
import paddle.nn.functional as F
from args import parse_args, print_args
from data import DialogueDataset
from paddle.io import DataLoader
from paddle.optimizer import AdamW
from paddle.optimizer.lr import NoamDecay
from paddlenlp.transformers import (
UnifiedTransformerLMHeadModel,
UnifiedTransformerTokenizer,
)
def save_ckpt(model, tokenizer, save_dir, name):
output_dir = os.path.join(save_dir, "model_{}".format(name))
if not os.path.exists(output_dir):
os.makedirs(output_dir)
# Need better way to get inner model of DataParallel
model_to_save = model._layers if isinstance(model, paddle.DataParallel) else model
model_to_save.save_pretrained(output_dir)
tokenizer.save_pretrained(output_dir)
def main(args):
paddle.set_device(args.device)
paddle.seed(args.seed)
world_size = dist.get_world_size()
if world_size > 1:
dist.init_parallel_env()
model = UnifiedTransformerLMHeadModel.from_pretrained(args.model_name_or_path)
tokenizer = UnifiedTransformerTokenizer.from_pretrained(args.model_name_or_path)
if world_size > 1:
model = paddle.DataParallel(model)
train_dataset = DialogueDataset(
args.train_data_path,
args.batch_size,
tokenizer.pad_token_id,
tokenizer.cls_token_id,
args.sort_pool_size,
args.seed,
mode="train",
)
train_dataloader = DataLoader(train_dataset, return_list=True, batch_size=None)
valid_dataset = DialogueDataset(
args.valid_data_path,
args.batch_size,
tokenizer.pad_token_id,
tokenizer.cls_token_id,
args.sort_pool_size,
mode="valid",
)
valid_dataloader = DataLoader(valid_dataset, return_list=True, batch_size=None)
lr_scheduler = NoamDecay(1 / (args.warmup_steps * (args.lr**2)), args.warmup_steps)
# Generate parameter names needed to perform weight decay.
# All bias and LayerNorm parameters are excluded.
decay_params = [p.name for n, p in model.named_parameters() if not any(nd in n for nd in ["bias", "norm"])]
optimizer = AdamW(
learning_rate=lr_scheduler,
parameters=model.parameters(),
weight_decay=args.weight_decay,
apply_decay_param_fun=lambda x: x in decay_params,
grad_clip=nn.ClipGradByGlobalNorm(args.max_grad_norm),
)
step = 0
total_time = 0.0
for epoch in range(args.epochs):
print("\nEpoch %d/%d" % (epoch + 1, args.epochs))
batch_start_time = time.time()
for inputs in train_dataloader:
step += 1
token_ids, type_ids, pos_ids, generation_mask, tgt_label, tgt_pos = inputs
logits = model(token_ids, type_ids, pos_ids, generation_mask, tgt_pos)
loss = F.cross_entropy(logits, tgt_label)
loss.backward()
optimizer.step()
lr_scheduler.step()
optimizer.clear_grad()
total_time += time.time() - batch_start_time
if step % args.logging_steps == 0:
ppl = paddle.exp(loss)
print(
"step %d - loss: %.4f - ppl: %.4f - lr: %.7f - %.3fs/step"
% (step, loss, ppl, optimizer.get_lr(), total_time / args.logging_steps)
)
total_time = 0.0
if step % args.save_steps == 0:
evaluation(model, valid_dataloader)
if dist.get_rank() == 0:
save_ckpt(model, tokenizer, args.save_dir, step)
batch_start_time = time.time()
@paddle.no_grad()
def evaluation(model, data_loader):
print("\nEval begin...")
model.eval()
total_tokens = 0
total_loss = 0.0
start_time = time.time()
step = 0
for inputs in data_loader:
step += 1
token_ids, type_ids, pos_ids, generation_mask, tgt_label, tgt_pos = inputs
logits = model(token_ids, type_ids, pos_ids, generation_mask, tgt_pos)
loss = F.cross_entropy(logits, tgt_label, reduction="sum")
total_loss += float(loss.numpy())
total_tokens += tgt_label.shape[0]
avg_loss = total_loss / total_tokens
ppl = math.exp(avg_loss)
avg_speed = (time.time() - start_time) / step
print("loss: %.4f - ppl: %.4f - %.3fs/step\n" % (avg_loss, ppl, avg_speed))
model.train()
if __name__ == "__main__":
args = parse_args()
print_args(args)
main(args)

88
slm/examples/dialogue/lic2021_baseline/infer.py
View File

@@ -1,88 +0,0 @@
# Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import time
import paddle
from args import parse_args, print_args
from data import DialogueDataset, select_response
from paddle.io import DataLoader
from paddlenlp.transformers import (
UnifiedTransformerLMHeadModel,
UnifiedTransformerTokenizer,
)
def main(args):
paddle.set_device(args.device)
paddle.seed(args.seed)
model = UnifiedTransformerLMHeadModel.from_pretrained(args.model_name_or_path)
tokenizer = UnifiedTransformerTokenizer.from_pretrained(args.model_name_or_path)
test_dataset = DialogueDataset(
args.test_data_path, args.batch_size, tokenizer.pad_token_id, tokenizer.cls_token_id, mode="test"
)
test_dataloader = DataLoader(test_dataset, return_list=True, batch_size=None)
infer(model, test_dataloader, tokenizer)
@paddle.no_grad()
def infer(model, data_loader, tokenizer):
print("\nInfer begin...")
model.eval()
total_time = 0.0
start_time = time.time()
responses = []
for step, inputs in enumerate(data_loader, 1):
token_ids, type_ids, pos_ids, generation_mask = inputs
ids, scores = model.generate(
input_ids=token_ids,
token_type_ids=type_ids,
position_ids=pos_ids,
attention_mask=generation_mask,
max_length=args.max_dec_len,
min_length=args.min_dec_len,
decode_strategy=args.decode_strategy,
temperature=args.temperature,
top_k=args.top_k,
top_p=args.top_p,
num_beams=args.num_beams,
length_penalty=args.length_penalty,
early_stopping=args.early_stopping,
num_return_sequences=args.num_samples,
use_fast=False,
)
total_time += time.time() - start_time
if step % args.logging_steps == 0:
print("step %d - %.3fs/step" % (step, total_time / args.logging_steps))
total_time = 0.0
results = select_response(ids, scores, tokenizer, args.max_dec_len, args.num_samples)
responses.extend(results)
start_time = time.time()
with open(args.output_path, "w", encoding="utf-8") as fout:
for response in responses:
fout.write(response + "\n")
print("\nSave inference result into: %s" % args.output_path)
if __name__ == "__main__":
args = parse_args()
print_args(args)
main(args)

67
slm/examples/dialogue/plato-2/README.md
View File

@@ -1,67 +0,0 @@
# PLATO-2
## 模型简介
构建高质量的开放领域（Open-Domain）的对话机器人，使得它能用自然语言与人自由地交流，这一直是自然语言处理领域终极目标之一。
为了能够简易地构建一个高质量的开放域聊天机器人，本项目在 Paddle2.0上实现了 PLATO-2的预测模型，并基于终端实现了简单的人机交互。用户可以通过下载预训练模型快速构建一个开放域聊天机器人。
PLATO-2的网络结构见下图：
<p align="center"><img src="./imgs/network.png" hspace="10"/></p>
PLATO-2的训练过程及其他细节详见 [Knover](https://github.com/PaddlePaddle/Knover)
## 快速开始
### 环境依赖
- python 3.7+
- sentencepiece
- termcolor
安装方式：`pip install sentencepiece termcolor`
### 数据准备
您可以从以下位置下载预训练模型文件：
* PLATO-2, 24-layers, 16-heads, 1024-hidden, EN: [预训练模型](https://bj.bcebos.com/paddlenlp/models/transformers/plato2/24L.pdparams)
* PLATO-2, 32-layers, 32-heads, 2048-hidden, EN: [预训练模型](https://bj.bcebos.com/paddlenlp/models/transformers/plato2/32L.pdparams)
以24层预训练模型为例：
```shell
wget https://bj.bcebos.com/paddlenlp/models/transformers/plato2/24L.pdparams
```
**NOTE:** PLATO-2网络参数量较大，24层网络至少需要显存16G，32层网络至少需要显存22G，用户可选择合适的网络层数及预训练模型。
sentencepiece 分词预训练模型和词表文件下载：
```shell
wget https://bj.bcebos.com/paddlenlp/models/transformers/plato2/data.tar.gz
tar -zxf data.tar.gz
```
### 人机交互
运行如下命令即可开始与聊天机器人用英语进行简单的对话
```shell
export CUDA_VISIBLE_DEVICES=0
python interaction.py --vocab_path ./data/vocab.txt --spm_model_file ./data/spm.model --num_layers 24 --init_from_ckpt ./24L.pdparams
```
以上参数表示：
* vocab_path：词表文件路径。
* spm_model_file：sentencepiece 分词预训练模型路径。
* num_layers：PLATO-2组网层数。
* init_from_ckpt：PLATO-2预训练模型路径。
32层 PLATO-2网络交互示例：
<p><img src="./imgs/case.jpg" hspace="10"/></p>
**NOTE:** 输入"[EXIT]"退出交互程序，输入"[NEXT]"开启下一轮新的对话。

BIN
slm/examples/dialogue/plato-2/imgs/case.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 99 KiB

BIN
slm/examples/dialogue/plato-2/imgs/network.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 110 KiB

104
slm/examples/dialogue/plato-2/interaction.py
View File

@@ -1,104 +0,0 @@
# Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import argparse
from collections import namedtuple
import paddle
from model import Plato2InferModel
from readers.nsp_reader import NSPReader
from readers.plato_reader import PlatoReader
from termcolor import colored, cprint
from utils import gen_inputs
from utils.args import parse_args
from paddlenlp.trainer.argparser import strtobool
def setup_args():
"""Setup arguments."""
parser = argparse.ArgumentParser()
group = parser.add_argument_group("Model")
group.add_argument("--init_from_ckpt", type=str, default="")
group.add_argument("--vocab_size", type=int, default=8001)
group.add_argument("--latent_type_size", type=int, default=20)
group.add_argument("--num_layers", type=int, default=24)
group = parser.add_argument_group("Task")
group.add_argument("--is_cn", type=strtobool, default=False)
args, _ = parser.parse_known_args()
NSPReader.add_cmdline_args(parser)
args = parse_args(parser)
args.batch_size *= args.latent_type_size
# print(json.dumps(args, indent=2))
return args
def load_params(model, init_from_ckpt):
state_dict = paddle.load(init_from_ckpt)
model.set_state_dict(state_dict)
def interact(args):
"""Inference main function."""
plato_reader = PlatoReader(args)
nsp_reader = NSPReader(args)
if args.num_layers == 24:
n_head = 16
hidden_size = 1024
elif args.num_layers == 32:
n_head = 32
hidden_size = 2048
else:
raise ValueError(
"The pre-trained model only support 24 or 32 layers, " "but received num_layers=%d." % args.num_layers
)
model = Plato2InferModel(nsp_reader, args.num_layers, n_head, hidden_size)
load_params(model, args.init_from_ckpt)
model.eval()
Example = namedtuple("Example", ["src", "data_id"])
context = []
start_info = "Enter [EXIT] to quit the interaction, [NEXT] to start a new conversation."
cprint(start_info, "yellow", attrs=["bold"])
while True:
user_utt = input(colored("[Human]: ", "red", attrs=["bold"])).strip()
if user_utt == "[EXIT]":
break
elif user_utt == "[NEXT]":
context = []
cprint(start_info, "yellow", attrs=["bold"])
else:
context.append(user_utt)
example = Example(src=" [SEP] ".join(context), data_id=0)
record = plato_reader._convert_example_to_record(example, is_infer=True)
data = plato_reader._pad_batch_records([record], is_infer=True)
inputs = gen_inputs(data, args.latent_type_size)
inputs["tgt_ids"] = inputs["tgt_ids"].astype("int64")
pred = model(inputs)[0]
bot_response = pred["response"]
print(colored("[Bot]:", "blue", attrs=["bold"]), colored(bot_response, attrs=["bold"]))
context.append(bot_response)
return
if __name__ == "__main__":
args = setup_args()
interact(args)

458
slm/examples/dialogue/plato-2/model.py
View File

@@ -1,458 +0,0 @@
# Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from collections import namedtuple
import paddle
import paddle.nn as nn
import paddle.nn.functional as F
def post_process_context(token_ids, reader, merge=True):
"""Post-process the context sequence."""
context = []
utt = []
for tok_id in token_ids[1:]:
if tok_id == reader.eos_id:
utt = reader.tokenizer.convert_ids_to_tokens(utt)
if merge:
utt = reader.tokenizer.merge_subword(utt)
context.append(utt)
utt = []
else:
utt.append(tok_id)
return context
def post_process_response(token_ids, reader, merge=True):
"""
Post-process the decoded sequence. Truncate from the first
<eos> and remove the <bos> and <eos> tokens currently.
"""
eos_pos = len(token_ids)
for i, tok_id in enumerate(token_ids):
if tok_id == reader.eos_id:
eos_pos = i
break
token_ids = token_ids[1:eos_pos]
response = reader.tokenizer.convert_ids_to_tokens(token_ids)
if merge:
response = reader.tokenizer.merge_subword(response)
return token_ids, response
def get_cross_turn_repetition(context, pred_tokens, eos_idx, is_cn=False):
"""Get cross-turn repetition."""
if len(pred_tokens) == 0:
return 1.0
if is_cn:
context = ["".join(utt) for utt in context]
pred_tokens = "".join(pred_tokens)
pred_tri_grams = set()
for i in range(len(pred_tokens) - 2):
tri_gram = tuple(pred_tokens[i : i + 3])
pred_tri_grams.add(tri_gram)
for utt in context:
for i in range(len(utt) - 2):
tri_gram = tuple(utt[i : i + 3])
if tri_gram in pred_tri_grams:
return 1.0
return 0.0
def get_in_turn_repetition(pred, is_cn=False):
"""Get in-turn repetition."""
if len(pred) == 0:
return 1.0
if isinstance(pred[0], str):
pred = [tok.lower() for tok in pred]
if is_cn:
pred = "".join(pred)
tri_grams = set()
for i in range(len(pred) - 2):
tri_gram = tuple(pred[i : i + 3])
if tri_gram in tri_grams:
return 1.0
tri_grams.add(tri_gram)
return 0.0
class Plato2EncoderLayer(nn.Layer):
def __init__(self, n_head, hidden_size, attn_dropout, act_dropout):
super(Plato2EncoderLayer, self).__init__()
self.self_attn = nn.MultiHeadAttention(hidden_size, n_head, attn_dropout)
self.pre_norm_layer = nn.LayerNorm(hidden_size)
self.post_norm_layer = nn.LayerNorm(hidden_size)
self.fc1 = nn.Linear(hidden_size, hidden_size * 4)
self.fc2 = nn.Linear(hidden_size * 4, hidden_size)
self.dropout_layer = nn.Dropout(act_dropout)
self.gelu_layer = nn.GELU()
def forward(self, x, attn_mask, cache):
query = self.pre_norm_layer(x)
attn_output, new_cache = self.self_attn(query, None, None, attn_mask, cache)
attn_output = self.dropout_layer(attn_output)
attn_output = attn_output + x
ffd_input = self.post_norm_layer(attn_output)
ffd_output = self.fc1(ffd_input)
ffd_output = self.gelu_layer(ffd_output)
ffd_output = self.dropout_layer(ffd_output)
ffd_output = self.fc2(ffd_output)
ffd_output = self.dropout_layer(ffd_output)
out = ffd_output + attn_output
return out, new_cache
def gen_cache(self, key):
return self.self_attn.gen_cache(key)
class Plato2Encoder(nn.Layer):
def __init__(
self, vocab_size, type_size, max_position_seq_len, num_layers, n_head, hidden_size, attn_dropout, act_dropout
):
super(Plato2Encoder, self).__init__()
self.n_head = n_head
self.word_embedding_layer = nn.Embedding(vocab_size, hidden_size)
self.sent_embedding_layer = nn.Embedding(type_size, hidden_size)
self.pos_embedding_layer = nn.Embedding(max_position_seq_len, hidden_size)
self.encoder_layers = []
for i in range(num_layers):
encoder_layer = Plato2EncoderLayer(n_head, hidden_size, attn_dropout, act_dropout)
self.encoder_layers.append(encoder_layer)
self.add_sublayer("layers." + str(i), encoder_layer)
self.post_encoder_layer_norm = nn.LayerNorm(hidden_size)
self.dropout_layer = nn.Dropout(act_dropout)
def forward(self, caches, token_ids, type_ids, pos_ids, generation_mask, aux_emb=None):
out, self_attn_mask = self.gen_input(token_ids, type_ids, pos_ids, generation_mask, aux_emb)
new_caches = []
for i, encoder_layer in enumerate(self.encoder_layers):
out, new_cache = encoder_layer(out, self_attn_mask, caches[i])
new_caches.append(new_cache)
enc_output = self.post_encoder_layer_norm(out)
return enc_output, new_caches
def gen_input(self, token_ids, type_ids, pos_ids, input_mask, aux_emb=None):
token_emb_out = self.word_embedding_layer(token_ids)
type_emb_out = self.sent_embedding_layer(type_ids)
pos_emb_out = self.pos_embedding_layer(pos_ids)
emb_out = token_emb_out + type_emb_out + pos_emb_out
# auxiliary memory embeddings
if aux_emb is not None:
emb_out = paddle.concat([aux_emb, emb_out], axis=1)
emb_out = self.dropout_layer(emb_out)
# generate n-head self-attention mask
self_attn_mask = input_mask
self_attn_mask = paddle.scale(x=self_attn_mask, scale=1e4, bias=-1.0, bias_after_scale=False)
n_head_self_attn_mask = paddle.stack(x=[self_attn_mask] * self.n_head, axis=1)
n_head_self_attn_mask.stop_gradient = True
return emb_out, n_head_self_attn_mask
def gen_caches(self, key):
caches = [encoder_layer.gen_cache(key) for encoder_layer in self.encoder_layers]
return caches
class NSP(nn.Layer):
def __init__(
self, vocab_size, type_size, max_position_seq_len, num_layers, n_head, hidden_size, attn_dropout, act_dropout
):
super(NSP, self).__init__()
self.n_head = n_head
self.hidden_size = hidden_size
self.word_embedding_layer = nn.Embedding(vocab_size, hidden_size)
self.sent_embedding_layer = nn.Embedding(type_size, hidden_size)
self.pos_embedding_layer = nn.Embedding(max_position_seq_len, hidden_size)
encoder_layer = nn.TransformerEncoderLayer(
hidden_size, n_head, hidden_size * 4, act_dropout, "gelu", attn_dropout, act_dropout, "True"
)
encoder_norm = nn.LayerNorm(hidden_size)
self.encoder = nn.TransformerEncoder(encoder_layer, num_layers, encoder_norm)
self.fc1 = nn.Linear(hidden_size, hidden_size)
self.fc2 = nn.Linear(hidden_size, 2)
self.dropout_layer = nn.Dropout(act_dropout)
self.tanh_layer = nn.Tanh()
self.softmax = nn.Softmax()
def forward(self, inputs):
token_ids = inputs["token_ids"]
type_ids = inputs["type_ids"]
pos_ids = inputs["pos_ids"]
attention_mask = inputs["attention_mask"]
label_pos = inputs["label_pos"]
out, self_attn_mask = self.gen_input(token_ids, type_ids, pos_ids, attention_mask)
# [-1, seq_len, hidden_size]
enc_out = self.encoder(out, self_attn_mask)
enc_out = paddle.reshape(enc_out, [-1, self.hidden_size])
label_pos = paddle.cast(label_pos, "int64")
out = paddle.gather(enc_out, label_pos)
pooled_out = self.fc1(out)
pooled_out = self.tanh_layer(pooled_out)
# [-1, 2]
logits = self.fc2(pooled_out)
probs = self.softmax(logits)
return probs
def gen_input(self, token_ids, type_ids, pos_ids, input_mask, aux_emb=None):
token_emb_out = self.word_embedding_layer(token_ids)
type_emb_out = self.sent_embedding_layer(type_ids)
pos_emb_out = self.pos_embedding_layer(pos_ids)
emb_out = token_emb_out + type_emb_out + pos_emb_out
# auxiliary memory embeddings
if aux_emb is not None:
emb_out = paddle.concat([aux_emb, emb_out], axis=1)
emb_out = self.dropout_layer(emb_out)
# generate n-head self-attention mask
self_attn_mask = input_mask
self_attn_mask = paddle.scale(x=self_attn_mask, scale=1e4, bias=-1.0, bias_after_scale=False)
n_head_self_attn_mask = paddle.stack(x=[self_attn_mask] * self.n_head, axis=1)
n_head_self_attn_mask.stop_gradient = True
return emb_out, n_head_self_attn_mask
class Plato2InferModel(nn.Layer):
def __init__(
self,
nsp_reader,
num_layers,
n_head,
hidden_size,
vocab_size=8001,
type_size=2,
latent_type_size=20,
max_position_seq_len=256,
act_dropout=0.1,
attn_dropout=0.1,
max_dec_len=64,
min_dec_len=1,
topk=10,
):
super(Plato2InferModel, self).__init__()
self.nsp_reader = nsp_reader
self.num_layers = num_layers
self.latent_type_size = latent_type_size
self.max_dec_len = max_dec_len
self.min_dec_len = min_dec_len
self.topk = topk
self.unk_id = 0
self.bos_id = 1
self.eos_id = 2
self.mask_id = 8000
self.after_eos = paddle.ones([vocab_size]) * -1e9
self.after_eos[self.eos_id] = 0
self.is_cn = False
self.batch_size = 1
self.latent_weight = paddle.create_parameter([hidden_size, latent_type_size], "float32")
self.plato2_encoder = Plato2Encoder(
vocab_size, type_size, max_position_seq_len, num_layers, n_head, hidden_size, attn_dropout, act_dropout
)
self.logits_fc_layer = nn.Linear(hidden_size, hidden_size)
self.logits_layer_norm = nn.LayerNorm(hidden_size)
self.logits_bias = paddle.create_parameter([vocab_size], "float32", is_bias=True)
self.nsp_predictor = NSP(
vocab_size, type_size, max_position_seq_len, num_layers, n_head, hidden_size, attn_dropout, act_dropout
)
self.gelu_layer = nn.GELU()
self.softmax = nn.Softmax()
@paddle.no_grad()
def forward(self, inputs):
token_ids = inputs["token_ids"]
type_ids = inputs["type_ids"]
pos_ids = inputs["pos_ids"]
generation_mask = inputs["generation_mask"]
latent_id = inputs["latent_id"]
data_id = inputs["data_id"]
# [-1, 1, latent_type_size]
latent_id = F.one_hot(latent_id, self.latent_type_size)
# [-1, 1, hidden_size]
latent_emb = paddle.matmul(latent_id, self.latent_weight, transpose_y=True)
caches = self.plato2_encoder.gen_caches(token_ids)
# [-1, seq_len + 1, hidden_size]
enc_out, new_caches = self.plato2_encoder(caches, token_ids, type_ids, pos_ids, generation_mask, latent_emb)
pred_ids = self.decode(inputs, new_caches)
nsp_inputs = self.gen_nsp_input(token_ids, pred_ids)
# [-1, 2]
probs = self.nsp_predictor(nsp_inputs)
return self.get_results(data_id, token_ids, pred_ids, probs)
def decode(self, inputs, caches):
tgt_ids = inputs["tgt_ids"]
tgt_pos = inputs["tgt_pos"]
tgt_generation_mask = inputs["tgt_generation_mask"]
predictions = tgt_ids
# TODO
step = 0
while step < self.max_dec_len:
# [-1, 1]
append_mask = paddle.cast(tgt_ids != self.eos_id, dtype=tgt_generation_mask.dtype)
tgt_generation_mask = paddle.concat([tgt_generation_mask, paddle.unsqueeze(append_mask, 1)], axis=-1)
tgt_sent = paddle.ones([tgt_generation_mask.shape[0], 1], dtype=tgt_ids.dtype)
# [-1, 1, hidden_size]
out, caches = self.plato2_encoder(caches, tgt_ids, tgt_sent, tgt_pos, tgt_generation_mask)
out = paddle.squeeze(out, axis=1)
# [-1, hidden_size]
trans = self.logits_fc_layer(out)
trans = self.gelu_layer(trans)
trans = self.logits_layer_norm(trans)
# [-1, vocab_size]
logits = (
paddle.matmul(trans, self.plato2_encoder.word_embedding_layer.weight, transpose_y=True)
+ self.logits_bias
)
logits[:, self.unk_id] = -1e9
logits[:, self.bos_id] = -1e9
logits[:, self.mask_id] = -1e9
if step < self.min_dec_len:
logits[:, self.eos_id] = -1e9
logits = logits * append_mask + (1 - append_mask) * self.after_eos
probs = self.softmax(logits)
# [-1, topk]
topk_probs, _ = paddle.topk(probs, k=self.topk)
mask = paddle.cast(probs >= topk_probs[:, -1:], "float32")
sums = paddle.sum(topk_probs, axis=-1, keepdim=True)
new_probs = probs * mask / sums
# [-1, 1]
sampling_ids = paddle.multinomial(new_probs)
step = step + 1
tgt_ids = sampling_ids
tgt_pos = tgt_pos + 1
predictions = paddle.concat([predictions, tgt_ids], axis=1)
return predictions
def gen_nsp_input(self, token_ids, pred_ids):
token_ids = token_ids.numpy()
pred_ids = pred_ids.numpy()
def __reader__():
headers = ["src", "tgt", "data_id"]
Example = namedtuple("Example", headers)
for i, (raw, pred) in enumerate(zip(token_ids, pred_ids)):
context = post_process_context(raw, self.nsp_reader, merge=False)
_, response = post_process_response(pred, self.nsp_reader, merge=False)
context_tokenized_input = " [SEP] ".join(" ".join(utt) for utt in context)
response_tokenized_input = " ".join(response)
example = Example(src=context_tokenized_input, tgt=response_tokenized_input, data_id=i)
data = self.nsp_reader._convert_example_to_record(example, is_infer=True)
yield data
return
generator = self.nsp_reader.data_generator(
reader=__reader__,
is_infer=True,
phase="test",
)
inputs = next(generator())
# print('\nnsp_inputs:')
for key in inputs:
inputs[key] = paddle.to_tensor(inputs[key])
if key in ["token_ids", "type_ids", "pos_ids"]:
inputs[key] = paddle.squeeze(inputs[key], axis=-1)
# print(key, inputs[key].shape)
# print(inputs[key])
return inputs
def get_results(self, data_id, token_ids, pred_ids, probs):
data_id = data_id.numpy()
token_ids = token_ids.numpy()
pred_ids = pred_ids.numpy()
probs = probs.numpy()
infos = []
for raw, pred, prob in zip(token_ids, pred_ids, probs):
tokens = post_process_context(raw, self.nsp_reader)
pred_token_ids, pred_tokens = post_process_response(pred, self.nsp_reader)
info = {}
info["response"] = " ".join(pred_tokens)
cross_turn_repetition = get_cross_turn_repetition(tokens, pred_tokens, self.nsp_reader.eos_id, self.is_cn)
in_turn_repetition = max(
get_in_turn_repetition(pred_tokens, self.is_cn), get_in_turn_repetition(pred_token_ids)
)
info["score"] = float(prob[1])
if len(pred_token_ids) >= self.max_dec_len:
info["score"] -= 1e3
elif cross_turn_repetition > 0:
info["score"] -= 1e3
elif in_turn_repetition > 0:
info["score"] -= 1e3
infos.append(info)
results = []
pre_idx = 0
sample = []
for idx, info in zip(data_id, infos):
if idx != pre_idx:
sample = sorted(sample, key=lambda info: -info["score"])
result = sample[0]
result["data_id"] = pre_idx
results.apeend(result)
sample = []
pre_idx = idx
sample.append(info)
if sample:
sample = sorted(sample, key=lambda info: -info["score"])
result = sample[0]
result["data_id"] = pre_idx
results.append(result)
return results

436
slm/examples/dialogue/plato-2/readers/dialog_reader.py
View File

@@ -1,436 +0,0 @@
# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Dialogue Reader."""
import csv
import gzip
from collections import namedtuple
from contextlib import contextmanager
import numpy as np
import utils.tokenization as tokenization
from utils import pad_batch_data
from utils.masking import mask
from paddlenlp.trainer.argparser import strtobool
class DialogReader(object):
"""The implement of DialogReader."""
@classmethod
def add_cmdline_args(cls, parser):
"""Add cmdline argurments."""
group = parser.add_argument_group("Reader")
group.add_argument("--max_src_len", type=int, default=128)
group.add_argument("--max_tgt_len", type=int, default=128)
group.add_argument("--truncate_first_turn", type=strtobool, default=False)
group.add_argument("--file_format", type=str, default="file", choices=["file", "filelist"])
group.add_argument("--data_format", type=str, default="raw", choices=["raw", "tokenized", "numerical"])
group.add_argument("--in_tokens", type=strtobool, default=False)
group.add_argument("--batch_size", type=int, default=16)
group.add_argument("--continuous_position", type=strtobool, default=True)
group.add_argument("--random_seed", type=int, default=11)
group.add_argument("--sort_pool_size", type=int, default=2**16)
group = parser.add_argument_group("Tokenizer")
group.add_argument("--tokenizer", type=str, default="SentencePieceTokenizer")
args, _ = parser.parse_known_args()
tokenizer_cls = getattr(tokenization, args.tokenizer)
tokenizer_cls.add_cmdline_args(parser)
return group
def __init__(self, args):
tokenizer_cls = getattr(tokenization, args.tokenizer)
self.tokenizer = tokenizer_cls(args)
self.vocab = self.tokenizer.vocab
self.pad_id = args.pad_id = self.vocab["[PAD]"]
self.bos_id = args.bos_id = self.vocab["[CLS]"]
self.eos_id = args.eos_id = self.vocab["[SEP]"]
self.unk_id = args.unk_id = self.vocab["[UNK]"]
self.mask_id = args.mask_id = self.vocab["[MASK]"]
self.vocab_size = args.get("vocab_size", 0)
self.max_src_len = args.max_src_len
self.max_tgt_len = args.max_tgt_len
self.truncate_first_turn = args.truncate_first_turn
self.file_format = args.file_format
self.data_format = args.data_format
self.in_tokens = args.in_tokens
self.batch_size = args.batch_size
self.continuous_position = args.continuous_position
self.sort_pool_size = args.sort_pool_size
# random_seed must be set for data slicing when using multi-gpu
self.global_rng = np.random.RandomState(args.random_seed)
# training progress
self.current_example = 0
self.current_epoch = 0
self.num_examples = 0
# model related
self.fields = ["token_ids", "type_ids", "pos_ids"]
self.num_numerical_fields = len(self.fields)
self.fields += ["tgt_start_idx", "data_id"]
self.sort_key = lambda record: [len(record.token_ids)]
self.Record = namedtuple("Record", self.fields, defaults=(None,) * len(self.fields))
self.features = {}
return
def get_train_progress(self):
"""Gets progress for training phase."""
return self.current_epoch, self.current_file_index, self.total_file
def _convert_example_to_record(self, example, is_infer):
# process src
src_token_ids = []
src_pos_ids = []
# tokenize src
s_token_ids_list = []
for s in example.src.split("[SEP]"):
s = tokenization.convert_to_unicode(s).strip()
if self.data_format == "tokenized":
s_tokens = s.split(" ")
else:
s_tokens = self.tokenizer.tokenize(s)
s_token_ids = self.tokenizer.convert_tokens_to_ids(s_tokens) + [self.eos_id]
s_token_ids_list.append(s_token_ids)
# trim src
idx = len(s_token_ids_list) - 1
total_token_num = 1
while idx >= 0:
total_token_num += len(s_token_ids_list[idx])
if total_token_num > self.max_src_len:
if self.truncate_first_turn and idx == 0:
truncated_ids = s_token_ids_list[idx][: self.max_src_len - total_token_num]
if len(truncated_ids) > 1:
s_token_ids_list[idx] = truncated_ids[:-1] + [self.eos_id]
idx -= 1
break
idx -= 1
for i, s_token_ids in enumerate(s_token_ids_list[idx + 1 :], idx + 1):
src_token_ids += s_token_ids
src_pos_ids += list(range(1, len(s_token_ids) + 1))
src_token_ids = [self.bos_id] + src_token_ids
src_type_ids = [0] * len(src_token_ids)
src_pos_ids = [0] + src_pos_ids
assert (
len(src_token_ids) == len(src_type_ids) == len(src_pos_ids)
), "not len(src_token_ids) == len(src_type_ids) == len(src_pos_ids)"
token_ids = src_token_ids
type_ids = src_type_ids
pos_ids = src_pos_ids
tgt_start_idx = len(token_ids)
if not is_infer:
# process tgt
# tokenize tgt
tgt = tokenization.convert_to_unicode(example.tgt).strip()
if self.data_format == "tokenized":
tgt_tokens = tgt.split(" ")
else:
tgt_tokens = self.tokenizer.tokenize(tgt)
tgt_token_ids = self.tokenizer.convert_tokens_to_ids(tgt_tokens)
tgt_token_ids.append(self.eos_id)
# trim tgt
if len(tgt_token_ids) > self.max_tgt_len - 1:
tgt_token_ids = tgt_token_ids[: self.max_tgt_len - 1]
tgt_token_ids = [self.bos_id] + tgt_token_ids
tgt_type_ids = [1] * len(tgt_token_ids)
tgt_pos_ids = list(range(1, len(tgt_token_ids) + 1))
assert (
len(tgt_token_ids) == len(tgt_type_ids) == len(tgt_pos_ids)
), "not len(tgt_token_ids) == len(tgt_type_ids) == len(tgt_pos_ids)"
token_ids += tgt_token_ids
type_ids += tgt_type_ids
pos_ids += tgt_pos_ids
assert len(token_ids) == len(type_ids) == len(pos_ids), "not len(token_ids) == len(type_ids) == len(pos_ids)"
if self.continuous_position:
src_pos_ids = list(range(len(src_token_ids)))
if not is_infer:
tgt_pos_ids = list(range(len(tgt_token_ids)))
pos_ids = list(range(len(token_ids)))
field_values = {"token_ids": src_token_ids, "type_ids": src_type_ids, "pos_ids": src_pos_ids}
field_values["tgt_start_idx"] = tgt_start_idx
field_values["data_id"] = example.data_id
record = self.Record(**field_values)
return record
def _read_tsv(self, fp, phase, is_infer, delimiter="\t", quotechar=None):
"""Reads a tab separated value file."""
csv.field_size_limit(2**20)
reader = csv.reader(fp, delimiter=delimiter, quotechar=quotechar)
headers = next(reader)
headers.append("data_id")
Example = namedtuple("Example", headers)
for i, line in enumerate(reader):
example = Example(*line, data_id=i)
if is_infer or phase.endswith("test"):
self.features[phase][i] = example
record = self._convert_example_to_record(example, is_infer)
yield record
def _read_numerical_file(self, fp, delimiter=";"):
for i, line in enumerate(fp):
cols = tokenization.convert_to_unicode(line).strip().split(delimiter)
cols = list(map(lambda x: list(map(int, x.split(" "))), cols))
if len(cols) > self.num_numerical_fields:
cols = cols[: self.num_numerical_fields]
tgt_start_idx = cols[0].index(self.bos_id, 1)
record = self.Record(*cols, tgt_start_idx=tgt_start_idx, data_id=i)
yield record
def _read_file(self, input_file, phase, is_infer):
def __wrapper__():
with open_file(input_file) as fp:
if self.data_format == "numerical":
records = self._read_numerical_file(fp)
else:
records = self._read_tsv(fp, phase, is_infer)
for record in records:
yield record
return __wrapper__
def _read_files(self, filelist, phase, is_infer, shuffle_files):
input_files = open(filelist).readlines()
def __wrapper__():
if shuffle_files:
self.global_rng.shuffle(input_files)
if phase == "train":
self.total_file = len(input_files)
for file_index, input_file in enumerate(input_files, 1):
if phase == "train":
self.current_file_index = file_index
self.current_file = input_file
file_reader = self._read_file(input_file.strip(), phase, is_infer)
for record in file_reader():
yield record
return __wrapper__
def _batch_reader(self, reader, phase=None, is_infer=False, sort_pool_size=2**16):
"""Construct a batch reader."""
def update_max_lens(max_lens, record):
"""Update max_lens."""
if max_lens is None:
return self.sort_key(record)
else:
return [max(max_len, l) for max_len, l in zip(max_lens, self.sort_key(record))]
def get_batch(reader):
"""Generate batches from reader."""
batch, max_lens = [], None
for record in reader():
if record is None:
yield batch
batch, max_lens = [], None
continue
self.current_example += 1
max_lens = update_max_lens(max_lens, record)
if self.in_tokens:
to_append = (len(batch) + 1) * sum(max_lens) <= self.batch_size
else:
to_append = len(batch) < self.batch_size
if to_append:
batch.append(record)
else:
yield batch
batch, max_lens = [record], self.sort_key(record)
if len(batch) > 0:
yield batch
def get_sorted_batch(pool):
"""Generate sorted batches from pool."""
pool = sorted(pool, key=self.sort_key)
batches = []
batch, max_lens = [], None
for record in pool:
self.current_example += 1
max_lens = update_max_lens(max_lens, record)
if self.in_tokens:
to_append = (len(batch) + 1) * sum(max_lens) <= self.batch_size
else:
to_append = len(batch) < self.batch_size
if to_append:
batch.append(record)
else:
batches.append(batch)
batch, max_lens = [record], self.sort_key(record)
if len(batch) > 0:
batches.append(batch)
self.global_rng.shuffle(batches)
for batch in batches:
yield batch
def __wrapper__():
if sort_pool_size > 0:
pool = []
for record in reader():
pool.append(record)
if len(pool) == sort_pool_size:
for batch in get_sorted_batch(pool):
yield batch
pool = []
if len(pool) > 0:
for batch in get_sorted_batch(pool):
yield batch
else:
for batch in get_batch(reader):
yield batch
return __wrapper__
def _distributed_batch_reader(self, batch_reader, num_part, part_id, is_test=False):
def __wrapper__():
batches = []
for batch in batch_reader():
batches.append(batch)
if len(batches) == num_part:
yield batches[part_id]
batches = []
if is_test and 0 <= part_id < len(batches):
yield batches[part_id]
return
return __wrapper__
def data_generator(
self, input_file=None, reader=None, num_epochs=1, num_part=1, part_id=0, phase=None, is_infer=False
):
"""Data generator."""
def __wrapper__():
if is_infer or phase.endswith("test"):
self.features[phase] = {}
nonlocal reader
if reader is None:
if self.file_format == "filelist":
reader = self._read_files(input_file, phase, is_infer, not phase.endswith("test"))
else:
if phase == "train":
self.total_file = 1
self.current_file_index = 1
self.current_file = input_file
reader = self._read_file(input_file, phase, is_infer)
batch_reader = self._batch_reader(
reader, phase, is_infer, sort_pool_size=self.sort_pool_size if not is_infer else 0
)
if phase == "train":
batch_reader = self._distributed_batch_reader(batch_reader, num_part, part_id)
elif phase.startswith("distributed"):
batch_reader = self._distributed_batch_reader(batch_reader, num_part, part_id, is_test=True)
for epoch_index in range(num_epochs):
if phase == "train":
self.current_example = 0
self.current_epoch = epoch_index + 1
for batch in batch_reader():
yield self._pad_batch_records(batch, is_infer)
return __wrapper__
def _gen_self_attn_mask(self, batch_token_ids, batch_tgt_start_idx=None, is_unidirectional=True, shift_len=0):
max_len = max(map(len, batch_token_ids))
input_mask_data = np.zeros((len(batch_token_ids), max_len + shift_len, max_len + shift_len))
if is_unidirectional:
for index, mask_data in enumerate(input_mask_data):
start = 0 if batch_tgt_start_idx is None else batch_tgt_start_idx[index]
end = len(batch_token_ids[index])
mask_data[: end + shift_len, : start + shift_len] = 1.0
# Generate the lower triangular matrix using the slice of matrix
b = np.tril(np.ones([end - start, end - start]), 0)
mask_data[start + shift_len : end + shift_len, start + shift_len : end + shift_len] = b
else:
for index, token_ids in enumerate(batch_token_ids):
input_mask_data[index, : len(token_ids) + shift_len, : len(token_ids) + shift_len] = 1.0
return input_mask_data.astype("float32")
def _pad_batch_records(self, batch_records, is_infer):
"""
Padding batch records and construct model's inputs.
"""
batch = {}
batch_token_ids = [record.token_ids for record in batch_records]
batch_type_ids = [record.type_ids for record in batch_records]
batch_pos_ids = [record.pos_ids for record in batch_records]
batch["token_ids"] = pad_batch_data(batch_token_ids, pad_id=self.pad_id)
batch["type_ids"] = pad_batch_data(batch_type_ids, pad_id=self.pad_id)
batch["pos_ids"] = pad_batch_data(batch_pos_ids, pad_id=self.pad_id)
batch_tgt_start_idx = [record.tgt_start_idx for record in batch_records]
batch["generation_mask"] = self._gen_self_attn_mask(batch_token_ids, batch_tgt_start_idx=batch_tgt_start_idx)
if is_infer:
tgt_ids = np.array([[[self.bos_id]]] * len(batch_token_ids), dtype="int64")
if self.continuous_position:
tgt_pos = np.array(batch_tgt_start_idx, dtype="int64")
else:
tgt_pos = np.zeros_like(batch_tgt_start_idx, dtype="int64")
tgt_pos = tgt_pos.reshape(-1, 1, 1)
batch["init_score"] = np.zeros_like(tgt_ids, dtype="float32").reshape(-1, 1).tolist()
batch["tgt_ids"] = tgt_ids.tolist()
batch["tgt_pos"] = tgt_pos.tolist()
batch["tgt_generation_mask"] = batch["generation_mask"][:, 0:1, :].astype("float32")
else:
batch["tgt_label"], batch["tgt_pos"] = mask(
batch_tokens=batch_token_ids,
vocab_size=self.vocab_size,
sent_b_starts=batch_tgt_start_idx,
is_unidirectional=True,
)
batch_data_id = [record.data_id for record in batch_records]
batch["data_id"] = np.array(batch_data_id).astype("int64").reshape([-1, 1])
return batch
@contextmanager
def open_file(filename):
"""Open file."""
if filename.endswith(".gz"):
fp = gzip.open(filename, "rt")
else:
fp = open(filename)
yield fp
fp.close()

152
slm/examples/dialogue/plato-2/readers/nsp_reader.py
View File

@@ -1,152 +0,0 @@
# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""NSP Reader."""
from collections import namedtuple
import numpy as np
from readers.dialog_reader import DialogReader
from utils import pad_batch_data
from utils.masking import mask
from paddlenlp.trainer.argparser import strtobool
class NSPReader(DialogReader):
"""NSP Reader."""
@classmethod
def add_cmdline_args(cls, parser):
"""Add cmdline argurments."""
group = DialogReader.add_cmdline_args(parser)
group.add_argument(
"--attention_style", type=str, default="bidirectional", choices=["bidirectional", "unidirectional"]
)
group.add_argument("--mix_negative_sample", type=strtobool, default=False)
return group
def __init__(self, args):
super(NSPReader, self).__init__(args)
self.fields.append("label")
self.Record = namedtuple("Record", self.fields, defaults=(None,) * len(self.fields))
self.attention_style = args.attention_style
self.mix_negative_sample = args.mix_negative_sample
return
def _convert_example_to_record(self, example, is_infer):
record = super(NSPReader, self)._convert_example_to_record(example, False)
if "label" in example._fields:
record = record._replace(label=int(example.label))
return record
def _mix_negative_sample(self, reader, neg_pool_size=2**16):
def gen_from_pool(pool):
num_samples = len(pool)
if num_samples == 1:
# only one sample: it is impossible to generate negative sample
yield pool[0]._replace(label=1)
return
self.global_rng.shuffle(pool)
for i in range(num_samples):
pool[i] = pool[i]._replace(label=1)
j = (i + 1) % num_samples
idx_i = pool[i].tgt_start_idx
idx_j = pool[j].tgt_start_idx
field_values = {}
field_values["token_ids"] = pool[i].token_ids[:idx_i] + pool[j].token_ids[idx_j:]
field_values["type_ids"] = pool[i].type_ids[:idx_i] + pool[j].type_ids[idx_j:]
field_values["pos_ids"] = list(range(len(field_values["token_ids"])))
neg_record = self.Record(**field_values, tgt_start_idx=idx_i, data_id=-1, label=0)
pool.append(neg_record)
assert len(neg_record.token_ids) <= self.max_seq_len
self.global_rng.shuffle(pool)
for record in pool:
yield record
def __wrapper__():
pool = []
for record in reader():
pool.append(record)
if len(pool) == neg_pool_size:
for record in gen_from_pool(pool):
yield record
pool = []
if len(pool) > 0:
for record in gen_from_pool(pool):
yield record
return __wrapper__
def _batch_reader(self, reader, phase=None, is_infer=False, sort_pool_size=2**16):
if self.mix_negative_sample:
reader = self._mix_negative_sample(reader)
return super(NSPReader, self)._batch_reader(
reader, phase=phase, is_infer=is_infer, sort_pool_size=sort_pool_size
)
def _pad_batch_records(self, batch_records, is_infer):
"""
Padding batch records and construct model's inputs.
"""
batch = {}
batch_token_ids = [record.token_ids for record in batch_records]
batch_type_ids = [record.type_ids for record in batch_records]
batch_pos_ids = [record.pos_ids for record in batch_records]
batch_tgt_start_idx = [record.tgt_start_idx for record in batch_records]
batch_label = [record.label for record in batch_records]
if self.attention_style == "unidirectional":
batch["token_ids"] = pad_batch_data(batch_token_ids, pad_id=self.pad_id)
batch["type_ids"] = pad_batch_data(batch_type_ids, pad_id=self.pad_id)
batch["pos_ids"] = pad_batch_data(batch_pos_ids, pad_id=self.pad_id)
tgt_label, tgt_pos, label_pos = mask(
batch_tokens=batch_token_ids,
vocab_size=self.vocab_size,
bos_id=self.bos_id,
sent_b_starts=batch_tgt_start_idx,
labels=batch_label,
is_unidirectional=True,
)
attention_mask = self._gen_self_attn_mask(batch_token_ids, batch_tgt_start_idx)
else:
batch_mask_token_ids, tgt_label, tgt_pos, label_pos = mask(
batch_tokens=batch_token_ids,
vocab_size=self.vocab_size,
bos_id=self.bos_id,
eos_id=self.eos_id,
mask_id=self.mask_id,
sent_b_starts=batch_tgt_start_idx,
labels=batch_label,
is_unidirectional=False,
)
if not is_infer:
batch_token_ids = batch_mask_token_ids
batch["token_ids"] = pad_batch_data(batch_token_ids, pad_id=self.pad_id)
batch["type_ids"] = pad_batch_data(batch_type_ids, pad_id=self.pad_id)
batch["pos_ids"] = pad_batch_data(batch_pos_ids, pad_id=self.pad_id)
attention_mask = self._gen_self_attn_mask(batch_token_ids, is_unidirectional=False)
batch["attention_mask"] = attention_mask
batch["label_pos"] = label_pos
if not is_infer:
batch_label = np.array(batch_label).astype("int64").reshape([-1, 1])
batch["label"] = batch_label
batch["tgt_label"] = tgt_label
batch["tgt_pos"] = tgt_pos
batch_data_id = [record.data_id for record in batch_records]
batch["data_id"] = np.array(batch_data_id).astype("int64").reshape([-1, 1])
return batch

84
slm/examples/dialogue/plato-2/readers/plato_reader.py
View File

@@ -1,84 +0,0 @@
# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Plato Reader."""
import numpy as np
from readers.dialog_reader import DialogReader
from utils import pad_batch_data
from utils.masking import mask
class PlatoReader(DialogReader):
"""The implement of PlatoReader"""
def __init__(self, args):
super(PlatoReader, self).__init__(args)
self.latent_type_size = args.latent_type_size
self.use_bow = args.use_bow
def _pad_batch_records(self, batch_records, is_infer):
"""
Padding batch records and construct model's inputs.
"""
batch = {}
batch_token_ids = [record.token_ids for record in batch_records]
batch_type_ids = [record.type_ids for record in batch_records]
batch_pos_ids = [record.pos_ids for record in batch_records]
batch_tgt_start_idx = [record.tgt_start_idx for record in batch_records]
batch_size = len(batch_token_ids)
# padding
batch["token_ids"] = pad_batch_data(batch_token_ids, pad_id=self.pad_id)
batch["type_ids"] = pad_batch_data(batch_type_ids, pad_id=self.pad_id)
batch["pos_ids"] = pad_batch_data(batch_pos_ids, pad_id=self.pad_id)
batch["generation_mask"] = self._gen_self_attn_mask(
batch_token_ids, batch_tgt_start_idx=batch_tgt_start_idx, is_unidirectional=True, shift_len=1
)
if not is_infer:
batch["recognition_mask"] = self._gen_self_attn_mask(batch_token_ids, is_unidirectional=False, shift_len=1)
if is_infer:
tgt_ids = np.array([[[self.bos_id]]] * batch_size, dtype="int64")
if self.continuous_position:
tgt_pos = np.array(batch_tgt_start_idx, dtype="int64")
else:
tgt_pos = np.zeros_like(batch_tgt_start_idx, dtype="int64")
tgt_pos = tgt_pos.reshape(-1, 1, 1)
batch["init_score"] = np.zeros_like(tgt_ids, dtype="float32").reshape(-1, 1).tolist()
batch["tgt_ids"] = tgt_ids.tolist()
batch["tgt_pos"] = tgt_pos.tolist()
batch["parent_idx"] = np.array(range(batch_size), dtype="int32")
batch["tgt_generation_mask"] = batch["generation_mask"][:, 0:1, :].astype("float32")
else:
mask_return_list = mask(
batch_tokens=batch_token_ids,
vocab_size=self.vocab_size,
sent_b_starts=batch_tgt_start_idx,
is_unidirectional=True,
use_latent=True,
use_bow=self.use_bow,
)
batch["tgt_label"] = mask_return_list[0]
batch["tgt_pos"] = mask_return_list[1]
if self.use_bow:
batch["bow_label"] = mask_return_list[2]
batch["bow_pos"] = mask_return_list[3]
batch_data_id = [record.data_id for record in batch_records]
batch["data_id"] = np.array(batch_data_id).astype("int64").reshape([-1, 1])
return batch

51
slm/examples/dialogue/plato-2/utils/__init__.py
View File

@@ -1,51 +0,0 @@
# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Utils."""
from itertools import chain
import numpy as np
import paddle
def repeat_array(array, times):
"""Repeate numpy array."""
if isinstance(array, list):
return list(chain(*([array] * times)))
else:
return np.concatenate([array] * times, axis=0)
def gen_inputs(inputs, latent_type_size):
batch_size = len(inputs["data_id"])
inputs = {name: repeat_array(array, latent_type_size) for name, array in inputs.items()}
# Add latent_id
inputs["latent_id"] = np.array(
[i for i in range(latent_type_size) for _ in range(batch_size)], dtype="int64"
).reshape([-1, 1])
# print('\nplato_inputs:')
for key in inputs:
inputs[key] = paddle.to_tensor(inputs[key])
if key in ["token_ids", "type_ids", "pos_ids", "tgt_ids", "tgt_pos", "data_id"]:
inputs[key] = paddle.squeeze(inputs[key], axis=-1)
# print(key, inputs[key].shape, inputs[key].dtype)
return inputs
def pad_batch_data(insts, pad_id=0):
"""Pad the instances to the max sequence length in batch."""
max_len = max(map(len, insts))
inst_data = np.array([list(inst) + [pad_id] * (max_len - len(inst)) for inst in insts])
return inst_data.astype("int64").reshape([-1, max_len, 1])

88
slm/examples/dialogue/plato-2/utils/args.py
View File

@@ -1,88 +0,0 @@
# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Parse argument."""
import argparse
import json
class Args(dict):
"""Arguments class
Store arguments in training / infer / ... scripts.
"""
def __getattr__(self, name):
if name in self.keys():
return self[name]
for v in self.values():
if isinstance(v, Args):
if name in v:
return v[name]
return None
def get(self, key, default_value=None):
"""Get the value of corresponding key."""
if key in self.keys():
return self[key]
for v in self.values():
if isinstance(v, Args):
if key in v:
return v[key]
return default_value
def __setattr__(self, name, value):
self[name] = value
def save(self, filename):
with open(filename, "w") as fp:
json.dump(self, fp, ensure_ascii=False, indent=4, sort_keys=False)
def load(self, filename, group_name=None):
if group_name is not None:
if group_name not in self:
self[group_name] = Args()
self[group_name].load(filename)
return
with open(filename, "r") as fp:
params_dict = json.load(fp)
for k, v in params_dict.items():
if isinstance(v, dict):
self[k].update(Args(v))
else:
self[k] = v
def parse_args(parser: argparse.ArgumentParser, allow_unknown=False) -> Args:
"""Parse hyper-parameters from cmdline."""
if allow_unknown:
parsed, _ = parser.parse_known_args()
else:
parsed = parser.parse_args()
args = Args()
optional_args = parser._action_groups[1]
for action in optional_args._group_actions[1:]:
arg_name = action.dest
args[arg_name] = getattr(parsed, arg_name)
for group in parser._action_groups[2:]:
group_args = Args()
for action in group._group_actions:
arg_name = action.dest
group_args[arg_name] = getattr(parsed, arg_name)
if len(group_args) > 0:
if group.title in args:
args[group.title].update(group_args)
else:
args[group.title] = group_args
return args

119
slm/examples/dialogue/plato-2/utils/masking.py
View File

@@ -1,119 +0,0 @@
# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Reader utils."""
import numpy as np
def mask(
batch_tokens,
vocab_size,
bos_id=1,
eos_id=2,
mask_id=3,
sent_b_starts=None,
labels=None,
is_unidirectional=False,
use_latent=False,
use_bow=False,
):
"""
Add mask for batch_tokens, return out, mask_label, mask_pos;
Note: mask_pos responding the batch_tokens after padded;
"""
batch_tokens = np.copy(batch_tokens)
max_len = max(map(len, batch_tokens))
mask_label = []
mask_pos = []
if labels is not None:
label_pos = []
if is_unidirectional:
# unidirectional language model
if use_latent:
max_len += 1
shift_len = 1
else:
shift_len = 0
for sent_index, sent in enumerate(batch_tokens):
sent_b_index = sent_b_starts[sent_index] if sent_b_starts is not None else 0
if labels is not None:
label_pos.append(sent_index * max_len + len(sent) - 1 + shift_len)
mask_label.extend(sent[sent_b_index + 1 :])
mask_pos.extend([sent_index * max_len + i + shift_len for i in range(sent_b_index, len(sent) - 1)])
mask_label = np.array(mask_label).astype("int64").reshape([-1, 1])
mask_pos = np.array(mask_pos).astype("int64").reshape([-1, 1])
return_list = [mask_label, mask_pos]
# latent related (bow label and pos)
if use_latent and use_bow:
bow_label = []
bow_pos = []
for sent_index, sent in enumerate(batch_tokens):
sent_b_index = sent_b_starts[sent_index] if sent_b_starts is not None else 0
def __filter__(tok_id):
# TODO: exclude [EOS] from bow loss
return True
bow_pos.extend([sent_index for i in range(sent_b_index + 1, len(sent)) if __filter__(sent[i])])
bow_label.extend([sent[i] for i in range(sent_b_index + 1, len(sent)) if __filter__(sent[i])])
bow_label = np.array(bow_label).astype("int64").reshape([-1, 1])
bow_pos = np.array(bow_pos).astype("int64").reshape([-1, 1])
return_list += [bow_label, bow_pos]
else:
# bidirectional mask language model
total_token_num = sum(map(len, batch_tokens))
prob_mask = np.random.rand(total_token_num)
# TODO: fix replace_ids, include [UNK]
replace_ids = np.random.randint(3, high=vocab_size, size=total_token_num)
prob_index = 0
for sent_index, sent in enumerate(batch_tokens):
# add pair label position
if labels is not None:
label_pos.append(sent_index * max_len)
# add mask label and position
for token_index, token in enumerate(sent):
if token == eos_id or token == bos_id:
continue
prob = prob_mask[prob_index + token_index]
if prob > 0.15:
continue
elif 0.03 < prob <= 0.15:
# mask
mask_label.append(sent[token_index])
sent[token_index] = mask_id
mask_pos.append(sent_index * max_len + token_index)
elif 0.015 < prob <= 0.03:
# random replace
mask_label.append(sent[token_index])
sent[token_index] = replace_ids[prob_index + token_index]
mask_pos.append(sent_index * max_len + token_index)
else:
# keep the original token
mask_label.append(sent[token_index])
mask_pos.append(sent_index * max_len + token_index)
prob_index += len(sent)
mask_label = np.array(mask_label).astype("int64").reshape([-1, 1])
mask_pos = np.array(mask_pos).astype("int64").reshape([-1, 1])
return_list = [batch_tokens, mask_label, mask_pos]
if labels is not None:
label_pos = np.array(label_pos).astype("int64").reshape([-1, 1])
assert len(labels) == len(label_pos)
return_list.append(label_pos)
return return_list

189
slm/examples/dialogue/plato-2/utils/tokenization.py
View File

@@ -1,189 +0,0 @@
# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Tokenization classes."""
import collections
import unicodedata
import sentencepiece as spm
from utils.args import str2bool
def clean_text(text):
"""Performs invalid character removal and whitespace cleanup on text."""
text = text.replace("“", '"').replace("”", '"').replace("‘", "'").replace("’", "'").replace("—", "-")
output = []
for char in text:
if _is_control(char):
continue
if _is_whitespace(char):
output.append(" ")
else:
output.append(char)
return "".join(output)
def preprocess_text(inputs, remove_space=True, lower=False):
"""preprocess data by removing extra space and normalize data."""
outputs = inputs
if remove_space:
outputs = " ".join(inputs.strip().split())
outputs = unicodedata.normalize("NFKD", outputs)
outputs = "".join([c for c in outputs if not unicodedata.combining(c)])
if lower:
outputs = outputs.lower()
return outputs
def encode_pieces(spm_model, text, return_unicode=True, sample=False):
"""turn sentences into word pieces."""
# liujiaxiang: add for ernie-albert, mainly consider for “/”/‘/’/— causing too many unk
text = clean_text(text)
if not sample:
pieces = spm_model.EncodeAsPieces(text)
else:
pieces = spm_model.SampleEncodeAsPieces(text, 64, 0.1)
return pieces
def encode_ids(spm_model, text, sample=False):
"""turn sentences into word pieces."""
pieces = encode_pieces(spm_model, text, return_unicode=False, sample=sample)
ids = [spm_model.PieceToId(piece) for piece in pieces]
return ids
def convert_to_unicode(text):
"""Converts `text` to Unicode (if it's not already), assuming utf-8 input."""
if isinstance(text, str):
return text
elif isinstance(text, bytes):
return text.decode("utf-8", "ignore")
else:
raise ValueError("Unsupported string type: %s" % (type(text)))
def load_vocab(vocab_file):
"""Loads a vocabulary file into a dictionary."""
vocab = collections.OrderedDict()
fin = open(vocab_file, "r", encoding="UTF-8")
for num, line in enumerate(fin):
items = convert_to_unicode(line.rstrip()).split("\t")
if len(items) > 2:
break
token = items[0]
index = items[1] if len(items) == 2 else num
token = token.strip()
vocab[token] = int(index)
return vocab
def convert_by_vocab(vocab, items):
"""Converts a sequence of [tokens|ids] using the vocab."""
output = []
for item in items:
output.append(vocab[item])
return output
class SentencePieceTokenizer(object):
"""Runs end-to-end tokenziation."""
@classmethod
def add_cmdline_args(cls, parser):
"""Add cmdline argurments."""
group = parser.add_argument_group("Tokenizer")
group.add_argument("--vocab_path", type=str, required=True)
group.add_argument("--do_lower_case", type=str2bool, default=False)
group.add_argument("--spm_model_file", type=str, required=True)
return group
def __init__(self, args):
self.spm_model = spm.SentencePieceProcessor()
self.spm_model.Load(args.spm_model_file)
self.vocab = load_vocab(args.vocab_path)
self.do_lower_case = args.do_lower_case
self.inv_vocab = {v: k for k, v in self.vocab.items()}
def tokenize(self, text):
"""Tokenizes a piece of text."""
text = preprocess_text(text, lower=self.do_lower_case)
return encode_pieces(self.spm_model, text, return_unicode=True)
def convert_tokens_to_ids(self, tokens):
"""Convert tokens to ids."""
ret = []
unk_id = self.vocab["<unk>"]
for token in tokens:
if token in self.vocab:
ret.append(self.vocab[token])
else:
ret.append(unk_id)
return ret
def convert_ids_to_tokens(self, ids):
"""Convert ids to tokens."""
return convert_by_vocab(self.inv_vocab, ids)
def merge_subword(self, tokens):
"""Merge subword."""
ret = []
for token in tokens:
if token.startswith("▁"):
ret.append(token[1:])
else:
if len(ret):
ret[-1] += token
else:
ret.append(token)
ret = [token for token in ret if token]
return ret
def convert_ids_to_str(self, ids):
"""Convert ids to string."""
tokens = self.convert_ids_to_tokens(ids)
tokens = self.merge_subword(tokens)
res = " ".join(tokens).replace("<s>", "")
res = res.replace("</s>", "\n").replace("\n ", "\n").strip()
return res
def _is_whitespace(char):
"""Checks whether `chars` is a whitespace character."""
# \t, \n, and \r are technically contorl characters but we treat them
# as whitespace since they are generally considered as such.
if char == " " or char == "\t" or char == "\n" or char == "\r":
return True
cat = unicodedata.category(char)
if cat == "Zs":
return True
return False
def _is_control(char):
"""Checks whether `chars` is a control character."""
# These are technically control characters but we count them as whitespace
# characters.
if char == "\t" or char == "\n" or char == "\r":
return False
cat = unicodedata.category(char)
if cat.startswith("C"):
return True
return False

1
slm/examples/dialogue/plato-xl
View File

@@ -1 +0,0 @@
../../model_zoo/plato-xl

217
slm/examples/dialogue/unified_transformer/README.md
View File

@@ -1,217 +0,0 @@
# UnifiedTransformer
## 模型简介
近年来，人机对话系统受到了学术界和产业界的广泛关注并取得了不错的发展。开放域对话系统旨在建立一个开放域的多轮对话系统，使得机器可以流畅自然地与人进行语言交互，既可以进行日常问候类的闲聊，又可以完成特定功能，以使得开放域对话系统具有实际应用价值。具体的说，开放域对话可以继续拆分为支持不同功能的对话形式，例如对话式推荐，知识对话技术等，如何解决并有效融合以上多个技能面临诸多挑战。
[UnifiedTransformer](https://arxiv.org/abs/2006.16779)以[Transformer](https://arxiv.org/abs/1706.03762) 编码器为网络基本组件，采用灵活的注意力机制，十分适合对话生成任务。
本项目是 UnifiedTransformer 在 Paddle 2.0上的开源实现，介绍了如何使用 UnifiedTransformer 在 DuConv 任务型对话数据集上进行微调，并给出了一个搭建简单中文聊天机器人的例子。
## 快速开始
### 环境依赖
- sentencepiece
- termcolor
安装方式：`pip install sentencepiece termcolor`
### 代码结构说明
以下是本项目主要代码结构及说明：
```text
.
├── finetune.py # 模型finetune主程序入口
├── infer.py # 模型预测主程序入口
├── utils.py # 定义参数及一些工具函数
└── README.md # 文档说明
```
### 数据准备
**DuConv**是百度发布的基于知识图谱的主动聊天任务数据集，让机器根据构建的知识图谱进行主动聊天，使机器具备模拟人类用语言进行信息传递的能力。数据集的创新性是：强调了 bot 的主动性，并且在闲聊对话中引入了明确的对话目标，即将对话引导到特定实体上。数据中的知识信息来源于电影和娱乐人物领域有聊天价值的知识信息，如票房、导演、评价等，以三元组 SPO 的形式组织，对话目标中的话题为电影或娱乐人物实体。数据集中共有3万 session，约12万轮对话，划分为训练集、开发集、测试集1和测试集2，其中测试集1中包含对话的 response，而测试集2中只有对话历史。
为了方便用户快速测试，PaddleNLP Dataset API 内置了 DuConv 数据集，一键即可完成数据集加载，示例代码如下：
```python
from paddlenlp.datasets import load_dataset
train_ds, dev_ds, test1_ds, test2_ds = load_dataset('duconv', splits=('train', 'dev', 'test_1', 'test_2'))
```
### 预训练模型
以下是 PaddleNLP 支持的对话类预训练模型：
|模型名称| 模型参数 | 模型特点 |
|:-----:|:------:|:-------:|
|unified_transformer-12L-cn| 12-layers, 12-heads, 768-hidden| 在千万级别的中文会话数据上进行预训练。|
|unified_transformer-12L-cn-luge| 12-layers, 12-heads, 768-hidden|由 unified_transformer-12L-cn 预训练模型在千言对话数据集上进行微调。并且模型输入中加入了标识不同对话技能的 special token，使得模型能同时支持闲聊对话、推荐对话和知识对话。|
|plato-mini| 6-layers, 12-heads, 768-hidden|在十亿级别的中文对话数据上进行预训练。参数量更小，但效果更好。只支持闲聊型对话。|
### 模型训练
运行如下命令即可在训练集上进行 finetune，并在验证集上进行验证
```shell
# GPU启动，参数`--gpus`指定训练所用的GPU卡号，可以是单卡，也可以多卡
# 例如使用1号和2号卡，则：`--gpu '1,2'`
unset CUDA_VISIBLE_DEVICES
python -m paddle.distributed.launch --gpus '0' --log_dir ./log finetune.py \
--model_name_or_path=unified_transformer-12L-cn-luge \
--save_dir=./checkpoints \
--logging_steps=100 \
--save_steps=1000 \
--seed=2021 \
--epochs=3 \
--batch_size=16 \
--lr=5e-5 \
--weight_decay=0.01 \
--warmup_steps=2500 \
--max_grad_norm=0.1 \
--max_seq_len=512 \
--max_response_len=128 \
--max_knowledge_len=256 \
--device=gpu
```
其中参数释义如下：
- `gpus` 指示了训练所用的 GPU
- `log_dir` 指示了日志保存目录
- `model_name_or_path` 指示了 finetune 使用的预训练模型，可以是 PaddleNLP 提供的预训练模型，或者是本地的模型。如果使用本地的模型，则配置为本地模型的目录地址，例如: ./checkpoints/model_xx/，目录中需包含 paddle 模型参数 model_state.pdparams。如果使用 PaddleNLP 提供的预训练模型，可以选择下面其中之一。
| PaddleNLP 提供的预训练模型 |
|---------------------------------|
| unified_transformer-12L-cn |
| unified_transformer-12L-cn-luge |
- `save_dir` 表示模型的保存路径。
- `logging_steps` 表示日志打印间隔。
- `save_steps` 表示模型保存及评估间隔。
- `seed` 表示随机数生成器的种子。
- `epochs` 表示训练轮数。
- `batch_size` 表示每次迭代**每张卡**上的样本数目。
- `lr` 表示基础学习率大小，将于 learning rate scheduler 产生的值相乘作为当前学习率。
- `weight_decay` 表示 AdamW 优化器中使用的 weight_decay 的系数。
- `warmup_steps` 表示学习率逐渐升高到基础学习率（即上面配置的 lr）所需要的迭代数，最早的使用可以参考[这篇论文](https://arxiv.org/pdf/1706.02677.pdf)。
- `max_grad_norm` 表示梯度裁剪允许的最大梯度值。
- `max_seq_len` 表示输入序列的最大长度。
- `max_response_len` 表示输入 response 的最大长度。
- `max_knowledge_len` 表示输入 knowledge 序列的最大长度。
- `device` 表示使用的设备。
程序运行时将会自动进行训练和验证，训练过程中会自动保存模型在指定的`save_dir`中，其中 loss 最小的模型会被保存在`save_dir/model_best`中。如：
```text
./checkpoints/
├── model_1000
│ ├── model_config.json
│ ├── model_state.pdparams
│ ├── spm.model
│ ├── tokenizer_config.json
│ └── vocab.txt
└── ...
```
**NOTE:** 如需恢复模型训练，只需指定`model_name_or_path`为本地微调模型的路径即可。
### 模型预测
运行如下命令即可在测试集上进行测试。
```shell
# GPU启动，预测仅支持单卡
export CUDA_VISIBLE_DEVICES=0
python infer.py \
--model_name_or_path=./checkpoints/model_best \
--output_path=./predict.txt \
--logging_steps=10 \
--seed=2021 \
--max_seq_len=512 \
--max_knowledge_len=256 \
--batch_size=4 \
--min_dec_len=1 \
--max_dec_len=64 \
--num_return_sequences=20 \
--decode_strategy=sampling \
--top_k=5 \
--device=gpu
```
其中参数释义如下：
- `model_name_or_path` 指示了预测使用的模型，可以是 PaddleNLP 提供的预训练模型，或者是本地的模型。如果使用本地的模型，则配置为本地模型的目录地址，例如: ./checkpoints/model_xx/，目录中需包含 paddle 模型参数 model_state.pdparams。如果使用 PaddleNLP 提供的预训练模型，可以选择下面其中之一。
| PaddleNLP 提供的预训练模型 |
|---------------------------------|
| unified_transformer-12L-cn |
| unified_transformer-12L-cn-luge |
- `output_path` 表示预测结果的保存路径。
- `logging_steps` 表示日志打印间隔。
- `seed` 表示随机数生成器的种子。
- `max_seq_len` 表示输入序列的最大长度。
- `max_knowledge_len` 表示输入 knowledge 序列的最大长度。
- `batch_size` 表示每次迭代**每张卡**上的样本数目。
- `min_dec_len` 表示预测生成的句子的最小长度。
- `max_dec_len` 表示预测生成的句子的最大长度。
- `num_return_sequences` 表示每条样本生成的句子的数量。对于每条样本，模型会生成`num_return_sequences`个句子，根据每个句子的概率得分进行排序，得分最高的句子作为最终的生成结果。
- `decode_strategy` 表示预测解码时采取的策略，可选"sampling"、"greedy_search"和"beam_search"之一。
- `top_k` 表示采用"sampling"解码策略时，token 的概率按从大到小排序，生成的 token 只从前`top_k`个中进行采样。
- `device` 表示使用的设备。
同时，我们提供了基于 FastGeneration 的高性能预测的选项，可以选择性开启是否需要采用高性能预测，PaddleNLP 提供了 JIT 的方式，可以自动完成对所需的自定义 op 的动态库编译：
- `faster` 表示是否开启高性能预测。设置 `--faster` 即表示开启。
- `use_fp16_decoding` 表示在开启高性能预测的时候，是否使用 fp16 来完成预测过程。设置 `--use_fp16_decoding` 即表示使用 fp16 进行预测，否则使用 fp32。
程序运行结束后会将预测生成的 response 保存在`output_path`中。同时终端中会输出评估结果。
采用预训练模型及微调模型在测试集上有如下结果：
| model_name_or_path | BLEU1 / BLEU2 | DISTINCT1 / DISTINCT2 |
| :-----------------------------: | :-------------: | :-------------------: |
| unified_transformer-12L-cn-luge | 0.2606 / 0.1576 | 0.1168 / 0.2977 |
| ./checkpoints/model_best | 0.2808 / 0.1744 | 0.1124 / 0.2899 |
**NOTE:** `./checkpoints/model_best`是按本项目中的超参在单卡上 finetune 得到的结果。
### 人机交互
运行如下命令即可开始与聊天机器人用中文进行简单的对话。
```shell
# GPU启动，仅支持单卡
export CUDA_VISIBLE_DEVICES=0
python interaction.py \
--model_name_or_path=plato-mini \
--min_dec_len=0 \
--max_dec_len=64 \
--num_return_sequences=20 \
--decode_strategy=sampling \
--top_k=5 \
--device=gpu
```
其中参数释义如下：
- `model_name_or_path` 指示了预测使用的模型，可以是 PaddleNLP 提供的预训练模型，或者是本地的模型。如果使用本地的模型，则配置为本地模型的目录地址，例如: ./checkpoints/model_xx/，目录中需包含 paddle 模型参数 model_state.pdparams。如果使用 PaddleNLP 提供的预训练模型，可以选择下面其中之一。
| PaddleNLP 提供的预训练模型 |
|---------------------------------|
| unified_transformer-12L-cn |
| unified_transformer-12L-cn-luge |
| plato-mini |
- `min_dec_len` 表示预测生成的句子的最小长度。
- `max_dec_len` 表示预测生成的句子的最大长度。
- `num_return_sequences` 表示每条样本生成的句子的数量。对于每条样本，模型会生成`num_return_sequences`个句子，根据每个句子的概率得分进行排序，得分最高的句子作为最终的生成结果。
- `decode_strategy` 表示预测解码时采取的策略，可选"sampling"、"greedy_search"和"beam_search"之一。
- `top_k` 表示采用"sampling"解码策略时，token 的概率按从大到小排序，生成的 token 只从前`top_k`个中进行采样。
- `device` 表示使用的设备。
**NOTE:** 输入"[EXIT]"退出交互程序，输入"[NEXT]"开启下一轮新的对话。需要注意使用退格会导致错误。
## Reference
- [UnifiedTransformer](https://arxiv.org/abs/2006.16779)
- [Knover/luge-dialogue](https://github.com/PaddlePaddle/Knover/tree/luge-dialogue/luge-dialogue)
- [DuConv](https://www.aclweb.org/anthology/P19-1369/)

165
slm/examples/dialogue/unified_transformer/finetune.py
View File

@@ -1,165 +0,0 @@
# Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import argparse
import math
import os
import time
import paddle
import paddle.distributed as dist
import paddle.nn as nn
import paddle.nn.functional as F
from datasets import load_dataset
from paddle.optimizer import AdamW
from paddle.optimizer.lr import NoamDecay
from utils import create_data_loader, print_args, set_seed
from paddlenlp.transformers import (
UnifiedTransformerLMHeadModel,
UnifiedTransformerTokenizer,
)
# yapf: disable
def parse_args():
parser = argparse.ArgumentParser(__doc__)
parser.add_argument('--model_name_or_path', type=str, default='unified_transformer-12L-cn-luge', help='The path or shortcut name of the pre-trained model.')
parser.add_argument('--save_dir', type=str, default='./checkpoints', help='The directory where the checkpoints will be saved.')
parser.add_argument('--logging_steps', type=int, default=100, help='Log every X updates steps.')
parser.add_argument('--save_steps', type=int, default=1000, help='Save checkpoint every X updates steps.')
parser.add_argument('--seed', type=int, default=2021, help='Random seed for initialization.')
parser.add_argument('--batch_size', type=int, default=16, help='Batch size per GPU/CPU for training.')
parser.add_argument('--lr', type=float, default=5e-5, help='The initial learning rate.')
parser.add_argument('--weight_decay', type=float, default=0.01, help='The weight decay for optimizer.')
parser.add_argument('--epochs', type=int, default=3, help='Total number of training epochs to perform.')
parser.add_argument('--warmup_steps', type=int, default=2500, help='The number of warmup steps.')
parser.add_argument('--max_grad_norm', type=float, default=0.1, help='The max value of grad norm.')
parser.add_argument('--max_seq_len', type=int, default=512, help='The maximum sequence length of training.')
parser.add_argument('--max_response_len', type=int, default=128, help='The maximum response sequence length of training.')
parser.add_argument('--max_knowledge_len', type=int, default=256, help='The maximum knowledge sequence length of training.')
parser.add_argument('--device', type=str, default='gpu', help='The device to select for training the model.')
args = parser.parse_args()
return args
# yapf: enable
def save_ckpt(model, tokenizer, save_dir, name):
output_dir = os.path.join(save_dir, "model_{}".format(name))
if not os.path.exists(output_dir):
os.makedirs(output_dir)
# Need better way to get inner model of DataParallel
model_to_save = model._layers if isinstance(model, paddle.DataParallel) else model
model_to_save.save_pretrained(output_dir)
tokenizer.save_pretrained(output_dir)
def train(args):
paddle.set_device(args.device)
world_size = dist.get_world_size()
if world_size > 1:
dist.init_parallel_env()
set_seed(args.seed)
model = UnifiedTransformerLMHeadModel.from_pretrained(args.model_name_or_path)
tokenizer = UnifiedTransformerTokenizer.from_pretrained(args.model_name_or_path)
if world_size > 1:
model = paddle.DataParallel(model)
train_ds, dev_ds = load_dataset("duconv", split=("train", "dev"))
train_ds, train_data_loader = create_data_loader(train_ds, tokenizer, args, "train")
dev_ds, dev_data_loader = create_data_loader(dev_ds, tokenizer, args, "dev")
lr_scheduler = NoamDecay(1 / (args.warmup_steps * (args.lr**2)), args.warmup_steps)
# Generate parameter names needed to perform weight decay.
# All bias and LayerNorm parameters are excluded.
decay_params = [p.name for n, p in model.named_parameters() if not any(nd in n for nd in ["bias", "norm"])]
optimizer = AdamW(
learning_rate=lr_scheduler,
parameters=model.parameters(),
weight_decay=args.weight_decay,
apply_decay_param_fun=lambda x: x in decay_params,
grad_clip=nn.ClipGradByGlobalNorm(args.max_grad_norm),
)
step = 0
total_time = 0.0
best_ppl = 1e9
for epoch in range(args.epochs):
print("\nEpoch %d/%d" % (epoch + 1, args.epochs))
batch_start_time = time.time()
for inputs in train_data_loader:
step += 1
labels = inputs[-1]
logits = model(*inputs[:-1])
loss = F.cross_entropy(logits, labels)
loss.backward()
optimizer.step()
lr_scheduler.step()
optimizer.clear_grad()
total_time += time.time() - batch_start_time
if step % args.logging_steps == 0:
ppl = paddle.exp(loss)
print(
"step %d - loss: %.4f - ppl: %.4f - lr: %.7f - %.3fs/step"
% (step, loss, ppl, optimizer.get_lr(), total_time / args.logging_steps)
)
total_time = 0.0
if step % args.save_steps == 0:
ppl = evaluation(model, dev_data_loader)
if dist.get_rank() == 0:
save_ckpt(model, tokenizer, args.save_dir, step)
if ppl < best_ppl:
best_ppl = ppl
save_ckpt(model, tokenizer, args.save_dir, "best")
print("Saved step {} as best model.\n".format(step))
batch_start_time = time.time()
print("\nTraining completed.")
@paddle.no_grad()
def evaluation(model, data_loader):
print("\nEval begin...")
model.eval()
total_tokens = 0
total_loss = 0.0
start_time = time.time()
step = 0
for inputs in data_loader:
step += 1
labels = inputs[-1]
logits = model(*inputs[:-1])
loss = F.cross_entropy(logits, labels, reduction="sum")
total_loss += loss.item()
total_tokens += labels.shape[0]
avg_loss = total_loss / total_tokens
ppl = math.exp(avg_loss)
avg_speed = (time.time() - start_time) / step
print("loss: %.4f - ppl: %.4f - %.3fs/step" % (avg_loss, ppl, avg_speed))
model.train()
return ppl
if __name__ == "__main__":
args = parse_args()
print_args(args)
train(args)

146
slm/examples/dialogue/unified_transformer/infer.py
View File

@@ -1,146 +0,0 @@
# Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import argparse
import time
import paddle
from datasets import load_dataset
from utils import create_data_loader, print_args, select_response, set_seed
from paddlenlp.metrics import BLEU, Distinct
from paddlenlp.transformers import (
UnifiedTransformerLMHeadModel,
UnifiedTransformerTokenizer,
)
# yapf: disable
def parse_args():
parser = argparse.ArgumentParser(__doc__)
parser.add_argument('--model_name_or_path', type=str, default='unified_transformer-12L-cn-luge', help='The path or shortcut name of the pre-trained model.')
parser.add_argument('--output_path', type=str, default='./predict.txt', help='The file path where the infer result will be saved.')
parser.add_argument('--logging_steps', type=int, default=100, help='Log every X updates steps.')
parser.add_argument('--seed', type=int, default=2021, help='Random seed for initialization.')
parser.add_argument('--batch_size', type=int, default=16, help='Batch size per GPU/CPU for training.')
parser.add_argument('--max_seq_len', type=int, default=512, help='The maximum sequence length of training.')
parser.add_argument('--max_response_len', type=int, default=128, help='The maximum response sequence length of training.')
parser.add_argument('--max_knowledge_len', type=int, default=256, help='The maximum knowledge sequence length of training.')
parser.add_argument('--min_dec_len', type=int, default=1, help='The minimum sequence length of generation.')
parser.add_argument('--max_dec_len', type=int, default=64, help='The maximum sequence length of generation.')
parser.add_argument('--num_return_sequences', type=int, default=20, help='The numbers of returned sequences for one input in generation.')
parser.add_argument('--decode_strategy', type=str, default='sampling', help='The decode strategy in generation.')
parser.add_argument('--top_k', type=int, default=0, help='The number of highest probability vocabulary tokens to keep for top-k sampling.')
parser.add_argument('--temperature', type=float, default=1.0, help='The value used to module the next token probabilities.')
parser.add_argument('--top_p', type=float, default=1.0, help='The cumulative probability for top-p sampling.')
parser.add_argument('--num_beams', type=int, default=0, help='The number of beams for beam search.')
parser.add_argument('--length_penalty', type=float, default=1.0, help='The exponential penalty to the sequence length for beam search.')
parser.add_argument('--early_stopping', type=eval, default=False, help='Whether to stop the beam search when at least `num_beams` sentences are finished per batch or not.')
parser.add_argument('--device', type=str, default='gpu', help='The device to select for training the model.')
parser.add_argument('--faster', action='store_true', help='Whether to process inference using faster transformer. ')
parser.add_argument('--use_fp16_decoding', action='store_true', help='Whether to use fp16 when using faster transformer. Only works when using faster transformer. ')
args = parser.parse_args()
return args
# yapf: enable
def calc_bleu_and_distinct(preds, targets):
assert len(preds) == len(targets), (
"The length of pred_responses should be equal to the length of "
"target_responses. But received {} and {}.".format(len(preds), len(targets))
)
bleu1 = BLEU(n_size=1)
bleu2 = BLEU(n_size=2)
distinct1 = Distinct(n_size=1)
distinct2 = Distinct(n_size=2)
for pred, target in zip(preds, targets):
pred_tokens = pred.split()
target_token = target.split()
bleu1.add_inst(pred_tokens, [target_token])
bleu2.add_inst(pred_tokens, [target_token])
distinct1.add_inst(pred_tokens)
distinct2.add_inst(pred_tokens)
print("\n" + "*" * 15)
print("The auto evaluation result is:")
print("BLEU-1:", bleu1.score())
print("BLEU-2:", bleu2.score())
print("DISTINCT-1:", distinct1.score())
print("DISTINCT-2:", distinct2.score())
@paddle.no_grad()
def infer(args):
paddle.set_device(args.device)
set_seed(args.seed)
model = UnifiedTransformerLMHeadModel.from_pretrained(args.model_name_or_path)
tokenizer = UnifiedTransformerTokenizer.from_pretrained(args.model_name_or_path)
test_ds = load_dataset("duconv", split="test_1")
test_ds, test_data_loader = create_data_loader(test_ds, tokenizer, args, "test")
model.eval()
total_time = 0.0
start_time = time.time()
pred_responses = []
for step, inputs in enumerate(test_data_loader, 1):
input_ids, token_type_ids, position_ids, attention_mask, seq_len = inputs
output = model.generate(
input_ids=input_ids,
token_type_ids=token_type_ids,
position_ids=position_ids,
attention_mask=attention_mask,
seq_len=seq_len,
max_length=args.max_dec_len,
min_length=args.min_dec_len,
decode_strategy=args.decode_strategy,
temperature=args.temperature,
top_k=args.top_k,
top_p=args.top_p,
num_beams=args.num_beams,
length_penalty=args.length_penalty,
early_stopping=args.early_stopping,
num_return_sequences=args.num_return_sequences,
use_fp16_decoding=args.use_fp16_decoding,
use_fast=args.faster,
)
total_time += time.time() - start_time
if step % args.logging_steps == 0:
print("step %d - %.3fs/step" % (step, total_time / args.logging_steps))
total_time = 0.0
ids, scores = output
results = select_response(ids, scores, tokenizer, args.max_dec_len, args.num_return_sequences)
pred_responses.extend(results)
start_time = time.time()
with open(args.output_path, "w", encoding="utf-8") as fout:
for response in pred_responses:
fout.write(response + "\n")
print("\nSave inference result into: %s" % args.output_path)
target_responses = [example["response"] for example in test_ds]
calc_bleu_and_distinct(pred_responses, target_responses)
if __name__ == "__main__":
args = parse_args()
print_args(args)
infer(args)

107
slm/examples/dialogue/unified_transformer/interaction.py
View File

@@ -1,107 +0,0 @@
# Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import argparse
import paddle
from termcolor import colored, cprint
from utils import print_args, select_response, set_seed
from paddlenlp.transformers import (
UnifiedTransformerLMHeadModel,
UnifiedTransformerTokenizer,
)
# yapf: disable
def parse_args():
parser = argparse.ArgumentParser(__doc__)
parser.add_argument('--model_name_or_path', type=str, default='plato-mini', help='The path or shortcut name of the pre-trained model.')
parser.add_argument('--seed', type=int, default=None, help='Random seed for initialization.')
parser.add_argument('--min_dec_len', type=int, default=1, help='The minimum sequence length of generation.')
parser.add_argument('--max_dec_len', type=int, default=64, help='The maximum sequence length of generation.')
parser.add_argument('--num_return_sequences', type=int, default=20, help='The numbers of returned sequences for one input in generation.')
parser.add_argument('--decode_strategy', type=str, default='sampling', help='The decode strategy in generation.')
parser.add_argument('--top_k', type=int, default=5, help='The number of highest probability vocabulary tokens to keep for top-k sampling.')
parser.add_argument('--temperature', type=float, default=1.0, help='The value used to module the next token probabilities.')
parser.add_argument('--top_p', type=float, default=1.0, help='The cumulative probability for top-p sampling.')
parser.add_argument('--num_beams', type=int, default=0, help='The number of beams for beam search.')
parser.add_argument('--length_penalty', type=float, default=1.0, help='The exponential penalty to the sequence length for beam search.')
parser.add_argument('--early_stopping', type=eval, default=False, help='Whether to stop the beam search when at least `num_beams` sentences are finished per batch or not.')
parser.add_argument('--device', type=str, default='gpu', help='The device to select for training the model.')
args = parser.parse_args()
return args
# yapf: enable
def interaction(args, model, tokenizer):
history = []
start_info = "Enter [EXIT] to quit the interaction, [NEXT] to start a new conversation."
cprint(start_info, "yellow", attrs=["bold"])
while True:
user_utt = input(colored("[Human]: ", "red", attrs=["bold"])).strip()
if user_utt == "[EXIT]":
break
elif user_utt == "[NEXT]":
history = []
cprint(start_info, "yellow", attrs=["bold"])
else:
history.append(user_utt)
inputs = tokenizer.dialogue_encode(
history, add_start_token_as_response=True, return_tensors=True, is_split_into_words=False
)
inputs["input_ids"] = inputs["input_ids"].astype("int64")
ids, scores = model.generate(
input_ids=inputs["input_ids"],
token_type_ids=inputs["token_type_ids"],
position_ids=inputs["position_ids"],
attention_mask=inputs["attention_mask"],
max_length=args.max_dec_len,
min_length=args.min_dec_len,
decode_strategy=args.decode_strategy,
temperature=args.temperature,
top_k=args.top_k,
top_p=args.top_p,
num_beams=args.num_beams,
length_penalty=args.length_penalty,
early_stopping=args.early_stopping,
num_return_sequences=args.num_return_sequences,
use_fast=True,
)
bot_response = select_response(
ids, scores, tokenizer, args.max_dec_len, args.num_return_sequences, keep_space=False
)[0]
print(colored("[Bot]:", "blue", attrs=["bold"]), colored(bot_response, attrs=["bold"]))
history.append(bot_response)
return
def main(args):
paddle.set_device(args.device)
if args.seed is not None:
set_seed(args.seed)
# Initialize the model and tokenizer
model = UnifiedTransformerLMHeadModel.from_pretrained(args.model_name_or_path)
tokenizer = UnifiedTransformerTokenizer.from_pretrained(args.model_name_or_path)
model.eval()
interaction(args, model, tokenizer)
if __name__ == "__main__":
args = parse_args()
print_args(args)
main(args)

265
slm/examples/dialogue/unified_transformer/utils.py
View File

@@ -1,265 +0,0 @@
# Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import random
from functools import partial
import numpy as np
import paddle
import paddle.distributed as dist
from paddle.io import BatchSampler, DataLoader, DistributedBatchSampler
from paddlenlp.data import Pad
def print_args(args):
print("----------- Configuration Arguments -----------")
for arg, value in sorted(vars(args).items()):
print("%s: %s" % (arg, value))
print("------------------------------------------------")
def set_seed(seed):
# Use the same data seed(for data shuffle) for all procs to guarantee data
# consistency after sharding.
random.seed(seed)
np.random.seed(seed)
# Maybe different op seeds(for dropout) for different procs is better.
paddle.seed(seed + dist.get_rank())
def preprocess_examples(examples, mode="train"):
"""
For training set and dev set, treat each utterance of the first speaker as
the response, and concatenate the goal, knowledge and the dialog’s previous
utterances as the history. In this way, multiple history-response pairs
are constructed.
"""
if mode == "test":
return examples
new_examples = {}
goal = []
knowledge = []
history = []
response = []
conv = examples["conversation"]
for index, conversation in enumerate(conv):
for i in range(0, len(conversation), 2):
goal.append(examples["goal"][index])
knowledge.append(examples["knowledge"][index])
history.append(conversation[:i])
response.append(conversation[i])
new_examples["goal"] = goal
new_examples["knowledge"] = knowledge
new_examples["history"] = history
new_examples["response"] = response
return new_examples
def convert_example(example, tokenizer, max_seq_len=512, max_response_len=128, max_knowledge_len=256, mode="train"):
"""Convert all examples into necessary features."""
goal = example["goal"]
knowledge = example["knowledge"]
goal_knowledge = " ".join([" ".join(lst) for lst in goal + knowledge])
if mode != "test":
tokenized_example = tokenizer.dialogue_encode(
example["history"],
response=example["response"],
knowledge=goal_knowledge,
task_type="knowledge",
max_seq_len=max_seq_len,
max_response_len=max_response_len,
max_knowledge_len=max_knowledge_len,
return_length=True,
)
response_start = tokenized_example["input_ids"].index(tokenizer.cls_token_id, 1)
response_end = tokenized_example["seq_len"]
# Use to gather the logits corresponding to the labels during training
tokenized_example["masked_positions"] = list(range(response_start, response_end - 1))
tokenized_example["labels"] = tokenized_example["input_ids"][response_start + 1 : response_end]
return tokenized_example
else:
tokenized_example = tokenizer.dialogue_encode(
example["history"],
knowledge=goal_knowledge,
task_type="knowledge",
max_seq_len=max_seq_len,
max_knowledge_len=max_knowledge_len,
add_start_token_as_response=True,
return_length=True,
)
if "response" in example:
tokenized_example["response"] = example["response"]
return tokenized_example
def batchify_fn(batch_examples, pad_val, mode):
def pad_mask(batch_attention_mask):
batch_size = len(batch_attention_mask)
max_len = max(map(len, batch_attention_mask))
attention_mask = np.ones((batch_size, max_len, max_len), dtype="float32") * -1e9
for i, mask_data in enumerate(attention_mask):
seq_len = len(batch_attention_mask[i])
mask_data[-seq_len:, -seq_len:] = np.array(batch_attention_mask[i], dtype="float32")
# In order to ensure the correct broadcasting mechanism, expand one
# dimension to the second dimension (n_head of Transformer).
attention_mask = np.expand_dims(attention_mask, axis=1)
return attention_mask
pad_func = Pad(pad_val=pad_val, pad_right=False, dtype="int64")
input_ids = pad_func([example["input_ids"] for example in batch_examples])
token_type_ids = pad_func([example["token_type_ids"] for example in batch_examples])
position_ids = pad_func([example["position_ids"] for example in batch_examples])
attention_mask = pad_mask([example["attention_mask"] for example in batch_examples])
if mode != "test":
max_len = max([example["seq_len"] for example in batch_examples])
masked_positions = np.concatenate(
[
np.array(example["masked_positions"]) + (max_len - example["seq_len"]) + i * max_len
for i, example in enumerate(batch_examples)
]
)
labels = np.concatenate([np.array(example["labels"], dtype="int64") for example in batch_examples])
return input_ids, token_type_ids, position_ids, attention_mask, masked_positions, labels
else:
seq_len = np.asarray([example["seq_len"] for example in batch_examples]).astype("int32")
return input_ids, token_type_ids, position_ids, attention_mask, seq_len
def create_data_loader(dataset, tokenizer, args, mode):
trans_func1 = partial(preprocess_examples, mode=mode)
trans_func2 = partial(
convert_example,
tokenizer=tokenizer,
max_seq_len=args.max_seq_len,
max_response_len=args.max_response_len,
max_knowledge_len=args.max_knowledge_len,
mode=mode,
)
remove_columns = None
if mode in ["train", "dev"]:
remove_columns = ["id", "conversation"]
dataset = dataset.map(trans_func1, batched=True, batch_size=None, remove_columns=remove_columns).map(trans_func2)
if mode == "train":
batch_sampler = DistributedBatchSampler(dataset, batch_size=args.batch_size, shuffle=True)
else:
batch_sampler = BatchSampler(dataset, batch_size=args.batch_size, shuffle=False)
collate_fn = partial(batchify_fn, pad_val=tokenizer.pad_token_id, mode=mode)
data_loader = DataLoader(dataset, batch_sampler=batch_sampler, collate_fn=collate_fn, return_list=True)
return dataset, data_loader
def post_process_response(token_ids, tokenizer):
"""Post-process the decoded sequence. Truncate from the first <eos>."""
eos_pos = len(token_ids)
for i, tok_id in enumerate(token_ids):
if tok_id == tokenizer.sep_token_id:
eos_pos = i
break
token_ids = token_ids[:eos_pos]
tokens = tokenizer.convert_ids_to_tokens(token_ids)
tokens = tokenizer.merge_subword(tokens)
return token_ids, tokens
def get_in_turn_repetition(pred, is_cn=False):
"""Get in-turn repetition."""
if len(pred) == 0:
return 1.0
if isinstance(pred[0], str):
pred = [tok.lower() for tok in pred]
if is_cn:
pred = "".join(pred)
tri_grams = set()
for i in range(len(pred) - 2):
tri_gram = tuple(pred[i : i + 3])
if tri_gram in tri_grams:
return True
tri_grams.add(tri_gram)
return False
def select_response(ids, scores, tokenizer, max_dec_len=None, num_return_sequences=1, keep_space=True):
results = []
group = []
tmp = []
if scores is not None:
ids = ids.numpy()
scores = scores.numpy()
if len(ids) != len(scores) or (len(ids) % num_return_sequences) != 0:
raise ValueError(
"the length of `ids` is {}, but the `num_return_sequences` is {}".format(
len(ids), num_return_sequences
)
)
for pred, score in zip(ids, scores):
pred_token_ids, pred_tokens = post_process_response(pred, tokenizer)
num_token = len(pred_token_ids)
if keep_space:
response = " ".join(pred_tokens)
else:
response = "".join(pred_tokens)
in_turn_repetition = get_in_turn_repetition(pred_tokens, True) or get_in_turn_repetition(pred_token_ids)
# not ending
if max_dec_len is not None and num_token >= max_dec_len:
score -= 1e3
elif in_turn_repetition:
score -= 1e3
tmp.append([response, score])
if len(tmp) == num_return_sequences:
group.append(tmp)
tmp = []
for preds in group:
preds = sorted(preds, key=lambda x: -x[1])
results.append(preds[0][0])
else:
ids = ids.numpy()
for pred in ids:
pred_token_ids, pred_tokens = post_process_response(pred, tokenizer)
num_token = len(pred_token_ids)
if keep_space:
response = " ".join(pred_tokens)
else:
response = "".join(pred_tokens)
in_turn_repetition = get_in_turn_repetition(pred_tokens, True) or get_in_turn_repetition(pred_token_ids)
last_pos = 0
if (max_dec_len is not None and num_token >= max_dec_len) or in_turn_repetition:
tmp.append([response])
else:
tmp.insert(last_pos, [response])
last_pos += 1
if len(tmp) == num_return_sequences:
group.append(tmp)
tmp = []
for preds in group:
results.append(preds[0][0])
return results

3
slm/examples/information_extraction/waybill_ie/model.py
View File

@@ -15,7 +15,6 @@
import paddle
import paddle.nn as nn
from paddlenlp.embeddings import TokenEmbedding
from paddlenlp.layers.crf import LinearChainCrf, LinearChainCrfLoss
from paddlenlp.utils.tools import compare_version
@@ -30,7 +29,7 @@ class BiGRUWithCRF(nn.Layer):
def __init__(self, emb_size, hidden_size, word_num, label_num, use_w2v_emb=False):
super(BiGRUWithCRF, self).__init__()
if use_w2v_emb:
self.word_emb = TokenEmbedding(extended_vocab_path="./data/word.dic", unknown_token="OOV")
raise ValueError("Please set use_w2v_emb=False")
else:
self.word_emb = nn.Embedding(word_num, emb_size)
self.gru = nn.GRU(emb_size, hidden_size, num_layers=2, direction="bidirect")

2
slm/examples/model_compression/distill_lstm/README.md
View File

@@ -69,7 +69,7 @@ python -u ./run_glue.py \
```
如果需要训练基于 ChnSentiCorp 数据集的 BERT finetuning 模型，可以进入[文本分类目录](../../text_classification/pretrained_models)下，将预训练模型改成 BERT，并基于 bert-base-chinese 和 bert-wwm-ext-chinese 模型进行 fine-tuning 训练。
如果需要训练基于 ChnSentiCorp 数据集的 BERT finetuning 模型，可以进入[文本分类目录](https://github.com/PaddlePaddle/PaddleNLP/tree/develop/slm/applications/text_classification/multi_class)下，将预训练模型改成 BERT，并基于 bert-base-chinese 和 bert-wwm-ext-chinese 模型进行 fine-tuning 训练。
训练完成之后，可将训练效果最好的模型保存在本项目下的`pretrained_models/$TASK_NAME/`下。模型目录下有`model_config.json`, `model_state.pdparams`, `tokenizer_config.json`及`vocab.txt`这几个文件。

6
slm/examples/model_compression/distill_lstm/small.py
View File

@@ -22,7 +22,6 @@ from args import parse_args
from data import create_data_loader_for_small_model, create_pair_loader_for_small_model
from paddle.metric import Accuracy
from paddlenlp.embeddings import TokenEmbedding
from paddlenlp.metrics import AccuracyAndF1
METRIC_CLASSES = {"sst-2": Accuracy, "qqp": AccuracyAndF1, "chnsenticorp": Accuracy}
@@ -44,10 +43,7 @@ class BiLSTM(nn.Layer):
):
super(BiLSTM, self).__init__()
if embedding_name is not None:
self.embedder = TokenEmbedding(
embedding_name, extended_vocab_path=vocab_path, keep_extended_vocab_only=True
)
embed_dim = self.embedder.embedding_dim
raise ValueError("TokenEmbedding is depercated in PaddleNLP since 3.0, please set embedding_name to None ")
else:
self.embedder = nn.Embedding(vocab_size, embed_dim, padding_idx)

20
slm/examples/question_generation/README.md
View File

@@ -1,20 +0,0 @@
# 问题生成
Question Generation（QG），即问题生成，指的是给定一段上下文和答案，自动生成一个流畅且符合上下文主题的问句。问题生成技术在教育、咨询、搜索、问答等多个领域均有着巨大的应用价值。
PaddleNLP 提供英文和中文问题生成任务示例，分别基于英文预训练语言模型[t5](./t5)和中文预训练语言模型[unimo-text](./unimo-text)。
## 英文
[t5](./t5) 展示了如何使用英文预训练模型 T5完成问题生成任务，支持模型微调预测评估，并提供相关预训练模型。
## 中文
[unimo-text](./unimo-text) 展示了如何使用中文预训练模型 UNIMO-Text 完成问题生成任务，提供数据准备、训练、预测、推理部署全流程定制化训练，并提供相关预训练模型。
# 参考文献
1. Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W. and Liu, P.J., 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. J. Mach. Learn. Res., 21(140), pp.1-67.
2. Li, Wei, et al. "Unimo: Towards unified-modal understanding and generation via cross-modal contrastive learning." arXiv preprint arXiv:2012.15409 (2020).

208
slm/examples/question_generation/t5/README.md
View File

@@ -1,208 +0,0 @@
# 问题生成(Question Generation)
## 简介
Question Generation（QG），即问题生成，指的是给定一段上下文（passage 或 sentence），自动生成一个流畅且符合上下文主题的问句。问题生成通常可以分为两个分支，即无答案问题生成（answer-agnostic question generation）和有答案问题生成（answer-aware question generation）。
本项目是 T5在 PaddlePaddle 上开源实现的有答案问题生成的例子，包含了在 SQuAD 数据集上微调和生成的代码。
## 快速开始
### 环境依赖
- nltk
- evaluate
安装方式：`pip install -r requirements.txt`
### 代码结构说明
以下是本项目主要代码结构及说明：
```text
.
├── finetune.py # 模型微调主程序入口
├── generate.py # 模型生成主程序入口
├── utils.py # 定义参数及一些工具函数
├── requirements.txt # 环境依赖文件
└── README.md # 文档说明
```
### 数据准备
#### 数据加载
**SQuAD**（Stanford Question Answering Dataset）数据集是一个英文问答数据集，现有的问题生成研究主要在该数据集上进行评价。**SQuAD**中的数据由段落、问题、答案3个主要部分组成，其中段落从维基百科中获取，问题和答案通过众包的方式由人工标注。
为了方便用户快速测试，PaddleNLP Dataset API 内置了 Squad 数据集，一键即可完成数据集加载，示例代码如下：
```python
from paddlenlp.datasets import load_dataset
train_set, dev_set, test_set = load_dataset("squad", splits=["train_v1", "dev_v1"])
```
#### 数据处理
针对**SQuAD**数据集，我们需要将 QA 任务格式的数据进行转换从而得到 text2text 形式的数据，默认构造方式如下，其他形式输入数据用户可以在 convert_example 函数中自行定义
```text
answer: {answer_text} context: {context_text}
question: {question_text}
```
具体案例如下，
```text
answer: the Miller–Rabin primality test context: The property of being prime (or not) is called primality. A simple but slow method of verifying the primality of a given number n is known as trial division. It consists of testing whether n is a multiple of any integer between 2 and . Algorithms much more efficient than trial division have been devised to test the primality of large numbers. These include the Miller–Rabin primality test, which is fast but has a small probability of error, and the AKS primality test, which always produces the correct answer in polynomial time but is too slow to be practical. Particularly fast methods are available for numbers of special forms, such as Mersenne numbers. As of January 2016[update], the largest known prime number has 22,338,618 decimal digits.
question: What is the name of the process which confirms the primality of a number n?
```
### 模型训练
运行如下命令即可在训练集上进行 finetune，并在验证集上进行验证
```shell
# GPU启动，参数`--gpus`指定训练所用的GPU卡号，可以是单卡，也可以多卡
# 例如使用1号和2号卡，则：`--gpu 1,2`
unset CUDA_VISIBLE_DEVICES
python -m paddle.distributed.launch --gpus 1,2 train.py \
--model_name_or_path=t5-base \
--dataset_name=squad \
--output_dir=output \
--max_source_length=1024 \
--max_target_length=142 \
--learning_rate=1e-4 \
--num_train_epochs=6 \
--logging_steps=100 \
--save_steps=1000 \
--seed=42 \
--train_batch_size=4 \
--eval_batch_size=64 \
--warmup_proportion=0.1 \
--ignore_pad_token_for_loss \
--device=gpu
```
其中参数释义如下：
- `gpus` 指示了训练所用的 GPU
- `model_name_or_path` 指示了 finetune 使用的预训练模型，可以是 PaddleNLP 提供的预训练模型，或者是本地的模型。如果使用本地的模型，则配置为本地模型的目录地址，例如: ./checkpoints/model_xx/，目录中需包含 paddle 模型参数 model_state.pdparams。如果使用 PaddleNLP 提供的预训练模型，可以选择下面其中之一。
| PaddleNLP 提供的预训练模型 |
|---------------------------------|
| t5-base |
| t5-large |
- `dataset_name` 表示训练的数据集。
- `output_dir` 表示模型的保存路径。
- `max_source_length` 表示输入序列的长度，超过该长度将被截断。
- `max_target_length` 表示输出的最大长度。
- `learning_rate` 表示基础学习率大小，将与 learning rate scheduler 产生的值相乘作为当前学习率。
- `num_train_epochs` 表示训练轮数。
- `epochs` 表示训练轮数。
- `logging_steps` 表示日志打印间隔。
- `save_steps` 表示模型保存及评估间隔。
- `seed` 表示随机数生成器的种子。
- `train_batch_size` 表示训练每张卡上的样本数目。
- `eval_batch_size` 表示预测单卡上的样本数目。
- `warmup_proportion` 表示 warmup_steps 所占总步数的比例。学习率逐渐升高到基础学习率（即上面配置的 learning_rate）所需要的迭代数。
- `device` 表示使用的设备。
程序运行时将会自动进行训练和验证，训练过程中会自动保存模型在指定的`output_dir`中。如：
```text
./output/
├── t5_model_1000
│ ├── model_config.json
│ ├── model_state.pdparams
│ ├── special_tokens_map.json
│ ├── spiece.model
│ └── tokenizer_config.json
└── ...
```
**NOTE:** 如需恢复模型训练，只需指定`model_name_or_path`为本地微调模型的路径即可。
### 模型预测
运行如下命令即可在验证集上进行测试
```shell
# GPU启动，预测仅支持单卡
export CUDA_VISIBLE_DEVICES=0
python predict.py \
--model_name_or_path=./checkpoints/model_xx/ \
--dataset_name=squad \
--output_path=generate.txt \
--max_source_length=1024 \
--max_target_length=142 \
--decode_strategy=greedy_search \
--top_k=2 \
--top_p=1.0 \
--num_beams=1 \
--length_penalty=0.0 \
--batch_size=64 \
--seed=42 \
--ignore_pad_token_for_loss \
--logging_steps=20 \
--device=gpu
```
其中参数释义如下：
- `model_name_or_path` 指示了预测使用的模型，可以是 PaddleNLP 提供的预训练模型，或者是本地的模型。如果使用本地的模型，则配置为本地模型的目录地址，例如: ./checkpoints/model_xx/，目录中需包含 paddle 模型参数 model_state.pdparams。如果使用 PaddleNLP 提供的预训练模型，可以选择下面其中之一。
| PaddleNLP 提供的预训练模型 |
|---------------------------------|
| t5-base |
| t5-large |
| mrm8488/t5-base-finetuned-question-generation-ap |
- `dataset_name` 表示预测的数据集。
- `output_path` 表示预测结果的保存路径。
- `max_source_length` 表示输入序列的长度，超过该长度将被截断。
- `max_target_length` 表示输出的最大长度。
- `decode_strategy` 表示预测解码时采取的策略，可选"sampling"、"greedy_search"和"beam_search"之一。
- `top_k` 表示采用"sampling"解码策略时，token 的概率按从大到小排序，生成的 token 只从前`top_k`个中进行采样。
- `top_p` 表示采用"sampling"解码策略时，从词表中采样并选择概率之和大于给定阈值`top_p`的 token。
- `num_beams` 表示 besm search 的 beam size。
- `length_penalty` 表示 besm search 生成长度的指数惩罚。
- `batch_size` 表示每次迭代**单卡**上的样本数目。
- `seed` 表示随机数生成器的种子。
- `logging_steps` 表示日志打印间隔。
- `device` 表示使用的设备。
程序运行结束后会将预测生成的问题保存在`output_path`中。同时终端中会输出评估结果。
采用社区微调模型 mrm8488/t5-base-finetuned-question-generation-ap 在验证集上有如下结果：
| model_name_or_path | BLEU-1 | BLEU-2 | BLEU-3 | BLEU-4 |
| :----------------------: | :-------------: | :-------------: |:-------------: |:-------------: |
| [mrm8488/t5-base-finetuned-question-generation-ap](https://huggingface.co/mrm8488/t5-base-finetuned-question-generation-ap ) | 50.11 | 35.83 | 27.68 | 22.03 |
## 参考文献
1. Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W. and Liu, P.J., 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. J. Mach. Learn. Res., 21(140), pp.1-67.

160
slm/examples/question_generation/t5/predict.py
View File

@@ -1,160 +0,0 @@
# Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import argparse
import random
import time
from functools import partial
from pprint import pprint
import numpy as np
import paddle
from paddle.io import BatchSampler, DataLoader
from utils import compute_metrics, convert_example
from paddlenlp.data import Pad, Tuple
from paddlenlp.datasets import load_dataset
from paddlenlp.transformers import T5ForConditionalGeneration, T5Tokenizer
# yapf: disable
def parse_args():
parser = argparse.ArgumentParser()
# Required parameters
parser.add_argument("--model_name_or_path", default="t5-base", type=str, required=True, help="Path to pre-trained model. ")
parser.add_argument("--dataset_name", default="squad", type=str, required=True, help="The name of the dataset to use. Selected in the list: " + "squad")
parser.add_argument('--output_path', type=str, default='generate.txt', help='The file path where the infer result will be saved.')
parser.add_argument("--max_source_length", default=1024, type=int, help="The maximum total input sequence length after tokenization.Sequences longer than this will be truncated, sequences shorter will be padded.",)
parser.add_argument("--min_target_length", default=0, type=int, help="The minimum total sequence length for target text when generating. ")
parser.add_argument("--max_target_length", default=142, type=int, help="The maximum total sequence length for target text after tokenization. Sequences longer than this will be truncated, sequences shorter will be padded during ``evaluate`` and ``predict``.",)
parser.add_argument('--decode_strategy', default='greedy_search', type=str, help='The decode strategy in generation.')
parser.add_argument('--top_k', default=2, type=int, help='The number of highest probability vocabulary tokens to keep for top-k sampling.')
parser.add_argument('--top_p', default=1.0, type=float, help='The cumulative probability for top-p sampling.')
parser.add_argument('--num_beams', default=1, type=int, help='The number of beams for beam search.')
parser.add_argument('--length_penalty', default=0.6, type=float, help='The exponential penalty to the sequence length for beam search.')
parser.add_argument('--early_stopping', default=False, type=eval, help='Whether to stop the beam search when at least `num_beams` sentences are finished per batch or not.')
parser.add_argument("--diversity_rate", default=0.0, type=float, help="The diversity of beam search. ")
parser.add_argument('--faster', action='store_true', help='Whether to process inference using FastGeneration. ')
parser.add_argument('--use_fp16_decoding', action='store_true', help='Whether to use fp16 when using FastGeneration. Only works when using FastGeneration. ')
parser.add_argument("--batch_size", default=64, type=int, help="Batch size per GPU/CPU for testing or evaluation.")
parser.add_argument("--seed", default=42, type=int, help="random seed for initialization")
parser.add_argument("--device", default="gpu", type=str, choices=["cpu", "gpu", "xpu"], help="The device to select to train the model, is must be cpu/gpu/xpu.")
parser.add_argument("--logging_steps", type=int, default=100, help="Log every X updates steps.")
parser.add_argument("--is_debug", action='store_true', help="Whether to debug.")
parser.add_argument("--ignore_pad_token_for_loss", action='store_true', help="Whether to ignore the tokens corresponding to padded labels in the loss computation or not.")
args = parser.parse_args()
return args
def set_seed(args):
# Use the same data seed(for data shuffle) for all procs to guarantee data
# consistency after sharding.
random.seed(args.seed)
np.random.seed(args.seed)
# Maybe different op seeds(for dropout) for different procs is better. By:
# `paddle.seed(args.seed + paddle.distributed.get_rank())`
paddle.seed(args.seed)
@paddle.no_grad()
def generate(args):
paddle.set_device(args.device)
set_seed(args)
tokenizer = T5Tokenizer.from_pretrained(args.model_name_or_path)
model = T5ForConditionalGeneration.from_pretrained(args.model_name_or_path)
dataset = load_dataset(args.dataset_name, splits=["dev_v1"])
# dataset = load_dataset(args.dataset_name, splits=["dev_v2"])
trans_func = partial(
convert_example,
tokenizer=tokenizer,
decoder_start_token_id=model.t5.bos_token_id,
max_source_length=args.max_source_length,
max_target_length=args.max_target_length,
ignore_pad_token_for_loss=args.ignore_pad_token_for_loss,
is_train=False)
def batchify_fn(samples, tokenizer):
fn = Tuple(
Pad(axis=0, pad_val=tokenizer.pad_token_id, dtype="int64"), # input_ids
Pad(axis=0, pad_val=tokenizer.pad_token_id, dtype="int64"
), # attention_mask
Pad(axis=0, pad_val=-100, dtype="int64"), # mem_seq_lens
Pad(axis=0, pad_val=tokenizer.pad_token_id, dtype="int64"
), # decoder_input_ids
Pad(axis=0, pad_val=tokenizer.pad_token_id, dtype="int64"), # labels
)
return fn(samples)
dataset = dataset.map(trans_func, lazy=True)
# debug
if args.is_debug:
dataset.data = dataset.data[:20]
dataset.new_data = dataset.new_data[:20]
batch_sampler = BatchSampler(dataset,
batch_size=args.batch_size,
shuffle=False)
data_loader = DataLoader(dataset=dataset,
batch_sampler=batch_sampler,
num_workers=0,
collate_fn=batchify_fn,
return_list=True)
data_loader.pin_memory = False
model.eval()
total_time = 0.0
start_time = time.time()
all_preds = []
all_labels = []
for step, batch in enumerate(data_loader):
input_ids, _, mem_seq_lens, _, labels = batch
preds, _ = model.generate(input_ids=input_ids,
max_length=args.max_target_length,
min_length=args.min_target_length,
decode_strategy=args.decode_strategy,
top_k=args.top_k,
top_p=args.top_p,
num_beams=args.num_beams,
length_penalty=args.length_penalty,
early_stopping=args.early_stopping,
diversity_rate=args.diversity_rate,
use_fast=args.faster)
total_time += (time.time() - start_time)
if step % args.logging_steps == 0:
print('step %d - %.3fs/step' %
(step, total_time / args.logging_steps))
total_time = 0.0
all_preds.extend(preds.numpy())
all_labels.extend(labels.numpy())
start_time = time.time()
bleu_result, decoded_preds, decoded_labels = compute_metrics(
all_preds, all_labels, tokenizer, args.ignore_pad_token_for_loss)
print("BLEU result: ", bleu_result)
with open(args.output_path, 'w', encoding='utf-8') as fout:
for decoded_pred in decoded_preds:
fout.write(' '.join(decoded_pred) + '\n')
print('Save generated result into: %s' % args.output_path)
with open(args.output_path + '.reference.txt', 'w',
encoding='utf-8') as fout:
for decoded_label in decoded_labels:
fout.write(' '.join(decoded_label) + '\n')
print('Save referenced labels into: {}.reference.txt'.format(args.output_path))
if __name__ == '__main__':
args = parse_args()
pprint(args)
generate(args)

2
slm/examples/question_generation/t5/requirements.txt
View File

@@ -1,2 +0,0 @@
nltk==3.6.2
evaluate==0.2.2

236
slm/examples/question_generation/t5/train.py
View File

@@ -1,236 +0,0 @@
# Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import argparse
import os
import random
import time
from functools import partial
from pprint import pprint
import numpy as np
import paddle
from paddle.io import BatchSampler, DataLoader, DistributedBatchSampler
from tqdm import tqdm
from utils import compute_metrics, convert_example
from paddlenlp.data import Pad, Tuple
from paddlenlp.datasets import load_dataset
from paddlenlp.trainer.argparser import strtobool
from paddlenlp.transformers import (
LinearDecayWithWarmup,
T5ForConditionalGeneration,
T5Tokenizer,
)
from paddlenlp.utils.log import logger
# yapf: disable
def parse_args():
parser = argparse.ArgumentParser()
# Required parameters
parser.add_argument("--model_name_or_path", default="t5-base", type=str, required=True, help="Path to pre-trained model. ")
parser.add_argument("--dataset_name", default="squad", type=str, required=True, help="The name of the dataset to use. Selected in the list: " + "squad")
parser.add_argument("--output_dir", default="output", type=str, required=True, help="The output directory where the model predictions and checkpoints will be written.",)
parser.add_argument("--max_source_length", default=1024, type=int, help="The maximum total input sequence length after tokenization.Sequences longer than this will be truncated, sequences shorter will be padded.",)
parser.add_argument("--min_target_length", default=0, type=int, help="The minimum total sequence length for target text when generating. ")
parser.add_argument("--max_target_length", default=142, type=int, help="The maximum total sequence length for target text after tokenization. Sequences longer than this will be truncated, sequences shorter will be padded. during ``evaluate`` and ``predict``.",)
parser.add_argument("--learning_rate", default=1e-4, type=float, help="The initial learning rate for Adam.")
parser.add_argument("--num_train_epochs", default=3, type=int, help="Total number of training epochs to perform.")
parser.add_argument("--logging_steps", type=int, default=100, help="Log every X updates steps.")
parser.add_argument("--save_steps", type=int, default=100, help="Save checkpoint every X updates steps.")
parser.add_argument("--train_batch_size", default=20, type=int, help="Batch size per GPU/CPU for training.")
parser.add_argument("--eval_batch_size", default=12, type=int, help="Batch size per GPU/CPU for evaluation.")
parser.add_argument("--weight_decay", default=0.0, type=float, help="Weight decay if we apply some.")
parser.add_argument("--warmup_steps", default=0, type=int, help="Linear warmup over warmup_steps. If > 0: Override warmup_proportion")
parser.add_argument("--warmup_proportion", default=0.1, type=float, help="Linear warmup proportion over total steps.")
parser.add_argument("--adam_epsilon", default=1e-6, type=float, help="Epsilon for Adam optimizer.")
parser.add_argument("--max_steps", default=-1, type=int, help="If > 0: set total number of training steps to perform. Override num_train_epochs.")
parser.add_argument("--seed", default=42, type=int, help="random seed for initialization")
parser.add_argument("--device", default="gpu", type=str, choices=["cpu", "gpu", "xpu"], help="The device to select to train the model, is must be cpu/gpu/xpu.")
parser.add_argument("--use_amp", default=False, type=strtobool, help="Enable mixed precision training.")
parser.add_argument("--scale_loss", default=2**15, type=float, help="The value of scale_loss for fp16.")
parser.add_argument("--ignore_pad_token_for_loss", action='store_true', help="Whether to ignore the tokens corresponding to padded labels in the loss computation or not.")
args = parser.parse_args()
return args
def set_seed(args):
# Use the same data seed(for data shuffle) for all procs to guarantee data
# consistency after sharding.
random.seed(args.seed)
np.random.seed(args.seed)
# Maybe different op seeds(for dropout) for different procs is better. By:
# `paddle.seed(args.seed + paddle.distributed.get_rank())`
paddle.seed(args.seed)
@paddle.no_grad()
def evaluate(model, data_loader, tokenizer, ignore_pad_token_for_loss,
min_target_length, max_target_length):
model.eval()
all_preds = []
all_labels = []
model = model._layers if isinstance(model, paddle.DataParallel) else model
for batch in tqdm(data_loader, total=len(data_loader), desc="Eval step"):
input_ids, _, _, labels = batch
preds = model.generate(input_ids=input_ids,
min_length=min_target_length,
max_length=max_target_length,
use_cache=True)[0]
all_preds.extend(preds.numpy())
all_labels.extend(labels.numpy())
bleu_result, decoded_preds, decoded_labels = compute_metrics(
all_preds, all_labels, tokenizer, ignore_pad_token_for_loss)
logger.info(bleu_result)
model.train()
def do_train(args):
paddle.set_device(args.device)
if paddle.distributed.get_world_size() > 1:
paddle.distributed.init_parallel_env()
set_seed(args)
tokenizer = T5Tokenizer.from_pretrained(args.model_name_or_path)
model = T5ForConditionalGeneration.from_pretrained(args.model_name_or_path)
trans_func = partial(
convert_example,
tokenizer=tokenizer,
decoder_start_token_id=model.t5.bos_token_id,
max_source_length=args.max_source_length,
max_target_length=args.max_target_length,
ignore_pad_token_for_loss=args.ignore_pad_token_for_loss)
logger.info("Loading train and dev dataset: %s" % args.dataset_name)
train_set, dev_set = load_dataset(args.dataset_name,
splits=["train_v1", "dev_v1"])
logger.info("Loaded train and dev dataset: %s" % args.dataset_name)
train_set = train_set.map(trans_func, lazy=True)
train_batch_sampler = DistributedBatchSampler(
train_set, batch_size=args.train_batch_size, shuffle=True)
batchify_fn = lambda samples, fn=Tuple(
Pad(axis=0, pad_val=tokenizer.pad_token_id, dtype="int64"), # input_ids
Pad(axis=0, pad_val=tokenizer.pad_token_id, dtype="int64"
), # attention_mask
Pad(axis=0, pad_val=tokenizer.pad_token_id, dtype="int64"
), # decoder_input_ids
Pad(axis=0, pad_val=tokenizer.pad_token_id, dtype="int64"), # labels
): fn(samples)
train_data_loader = DataLoader(dataset=train_set,
batch_sampler=train_batch_sampler,
num_workers=0,
collate_fn=batchify_fn,
return_list=True)
dev_set = dev_set.map(trans_func, lazy=True)
dev_batch_sampler = BatchSampler(dev_set,
batch_size=args.eval_batch_size,
shuffle=False)
dev_data_loader = DataLoader(dataset=dev_set,
batch_sampler=dev_batch_sampler,
num_workers=0,
collate_fn=batchify_fn,
return_list=True)
if paddle.distributed.get_world_size() > 1:
model = paddle.DataParallel(model)
num_training_steps = args.max_steps if args.max_steps > 0 else (
len(train_data_loader) * args.num_train_epochs)
warmup = args.warmup_steps if args.warmup_steps > 0 else args.warmup_proportion
lr_scheduler = LinearDecayWithWarmup(args.learning_rate, num_training_steps,
warmup)
# Generate parameter names needed to perform weight decay.
# All bias and LayerNorm parameters are excluded.
decay_params = [
p.name for n, p in model.named_parameters()
if not any(nd in n for nd in ["bias", "norm"])
]
optimizer = paddle.optimizer.AdamW(
learning_rate=lr_scheduler,
epsilon=args.adam_epsilon,
parameters=model.parameters(),
weight_decay=args.weight_decay,
apply_decay_param_fun=lambda x: x in decay_params)
if args.use_amp:
scaler = paddle.amp.GradScaler(init_loss_scaling=args.scale_loss)
global_step = 0
tic_train = time.time()
for epoch in tqdm(range(args.num_train_epochs), desc="Epoch"):
for step, batch in tqdm(enumerate(train_data_loader),
desc="Train step",
total=len(train_data_loader)):
global_step += 1
input_ids, attention_mask, decoder_input_ids, labels = batch
with paddle.amp.auto_cast(
args.use_amp,
custom_white_list=["layer_norm", "softmax", "gelu"]):
output = model(input_ids,
attention_mask,
decoder_input_ids,
labels=labels)
loss = output[0]
if args.use_amp:
scaled_loss = scaler.scale(loss)
scaled_loss.backward()
scaler.minimize(optimizer, scaled_loss)
else:
loss.backward()
optimizer.step()
lr_scheduler.step()
optimizer.clear_grad()
if global_step % args.logging_steps == 0:
logger.info(
"global step %d/%d, epoch: %d, batch: %d, rank_id: %s, loss: %f, lr: %.10f, speed: %.4f step/s"
% (global_step, num_training_steps, epoch, step,
paddle.distributed.get_rank(), loss, optimizer.get_lr(),
args.logging_steps / (time.time() - tic_train)))
tic_train = time.time()
if global_step % args.save_steps == 0 or global_step == num_training_steps:
tic_eval = time.time()
evaluate(model, dev_data_loader, tokenizer,
args.ignore_pad_token_for_loss, args.min_target_length,
args.max_target_length)
logger.info("eval done total : %s s" % (time.time() - tic_eval))
if paddle.distributed.get_rank() == 0:
output_dir = os.path.join(
args.output_dir, "t5_model_%d.pdparams" % global_step)
if not os.path.exists(output_dir):
os.makedirs(output_dir)
# Need better way to get inner model of DataParallel
model_to_save = model._layers if isinstance(
model, paddle.DataParallel) else model
model_to_save.save_pretrained(output_dir)
tokenizer.save_pretrained(output_dir)
if global_step >= num_training_steps:
return
if paddle.distributed.get_rank() == 0:
output_dir = os.path.join(args.output_dir,
"t5_model_final_%d.pdparams" % global_step)
if not os.path.exists(output_dir):
os.makedirs(output_dir)
# Need better way to get inner model of DataParallel
model_to_save = model._layers if isinstance(
model, paddle.DataParallel) else model
model_to_save.save_pretrained(output_dir)
tokenizer.save_pretrained(output_dir)
if __name__ == "__main__":
args = parse_args()
pprint(args)
do_train(args)

166
slm/examples/question_generation/t5/utils.py
View File

@@ -1,166 +0,0 @@
# Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import evaluate
import nltk
import numpy as np
from paddlenlp.metrics import BLEU
def convert_example(
example,
tokenizer,
decoder_start_token_id,
max_source_length,
max_target_length,
ignore_pad_token_for_loss=True,
is_train=True,
):
"""
Convert an example into necessary features.
"""
# Tokenize our examples with truncation and maybe padding, but keep the overflows using a stride. This results
# in one example possible giving several features when a context is long, each of those features having a
# context that overlaps a bit the context of the previous feature.
# NOTE: Almost the same functionality as HuggingFace's prepare_train_features function. The main difference is
# that HugggingFace uses ArrowTable as basic data structure, while we use list of dictionary instead.
context = example["context"]
question = example["question"]
try:
answer = example["answers"][0]
except:
print(example["context"])
print(example["question"])
print(example["answers"])
print(example["answer_starts"])
print(example["is_impossible"])
input_seq = f"answer: {answer} context: {context} </s>"
output_seq = f"question: {question} </s>"
outputs = tokenizer(
output_seq,
max_seq_len=max_target_length,
pad_to_max_seq_len=True,
truncation_strategy="longest_first",
)
output_ids = [decoder_start_token_id] + outputs["input_ids"][:-1]
if ignore_pad_token_for_loss:
# Replace all tokenizer.pad_token_id in the outputs by -100 when we want to ignore padding in the loss.
outputs["input_ids"] = [(l if l != tokenizer.pad_token_id else -100) for l in outputs["input_ids"]]
if is_train:
inputs = tokenizer(
input_seq,
max_seq_len=max_source_length,
pad_to_max_seq_len=True,
truncation_strategy="longest_first",
return_attention_mask=True,
return_length=False,
)
return inputs["input_ids"], inputs["attention_mask"], output_ids, outputs["input_ids"]
else:
inputs = tokenizer(
input_seq,
max_seq_len=max_source_length,
pad_to_max_seq_len=True,
truncation_strategy="longest_first",
return_attention_mask=True,
return_length=True,
)
return inputs["input_ids"], inputs["attention_mask"], inputs["length"], output_ids, outputs["input_ids"]
def compute_metrics(preds, labels, tokenizer, ignore_pad_token_for_loss=True):
def compute_bleu(predictions, references, rouge_types=None, use_stemmer=True):
bleu1 = BLEU(n_size=1)
bleu2 = BLEU(n_size=2)
bleu3 = BLEU(n_size=3)
bleu4 = BLEU(n_size=4)
assert len(predictions) == len(references)
for i in range(len(predictions)):
bleu1.add_inst(predictions[i], [references[i]])
bleu2.add_inst(predictions[i], [references[i]])
bleu3.add_inst(predictions[i], [references[i]])
bleu4.add_inst(predictions[i], [references[i]])
result = {
"BLEU-1": bleu1.score() * 100,
"BLEU-2": bleu2.score() * 100,
"BLEU-3": bleu3.score() * 100,
"BLEU-4": bleu4.score() * 100,
}
return result
def compute_bleu_hf(predictions, references, rouge_types=None, use_stemmer=True):
predictions = [" ".join(prediction) for prediction in predictions]
references = [[" ".join(reference)] for reference in references]
bleu = evaluate.load("bleu")
assert len(predictions) == len(references)
bleu1_results = bleu.compute(predictions=predictions, references=references, max_order=1)
bleu2_results = bleu.compute(predictions=predictions, references=references, max_order=2)
bleu3_results = bleu.compute(predictions=predictions, references=references, max_order=3)
bleu4_results = bleu.compute(predictions=predictions, references=references, max_order=4)
result = {
"BLEU-1": bleu1_results["bleu"] * 100,
"BLEU-2": bleu2_results["bleu"] * 100,
"BLEU-3": bleu3_results["bleu"] * 100,
"BLEU-4": bleu4_results["bleu"] * 100,
}
return result
def post_process_text(preds, labels):
preds = [pred.strip() for pred in preds]
labels = [label.strip() for label in labels]
preds = [pred.strip("question:") for pred in preds]
labels = [label.strip("question:") for label in labels]
labels = [label.strip() for label in labels]
# expects newline after each sentence
preds = ["\n".join(nltk.sent_tokenize(pred)) for pred in preds]
labels = ["\n".join(nltk.sent_tokenize(label)) for label in labels]
preds = [pred.split() for pred in preds]
labels = [label.split() for label in labels]
return preds, labels
def post_process_seq(seq, bos_idx, eos_idx, output_bos=False, output_eos=False):
"""
Post-process the decoded sequence.
"""
eos_pos = len(seq) - 1
for i, idx in enumerate(seq):
if idx == eos_idx:
eos_pos = i
break
seq = [idx for idx in seq[: eos_pos + 1] if (output_bos or idx != bos_idx) and (output_eos or idx != eos_idx)]
return seq
if ignore_pad_token_for_loss:
labels = np.asarray(labels)
labels = np.where(labels != -100, labels, tokenizer.pad_token_id)
decoded_preds, decoded_labels = [], []
for pred, label in zip(preds, labels):
pred_id = post_process_seq(pred, tokenizer.bos_token_id, tokenizer.eos_token_id)
label_id = post_process_seq(label, tokenizer.bos_token_id, tokenizer.eos_token_id)
decoded_preds.append(tokenizer.decode(pred_id))
decoded_labels.append(tokenizer.decode(label_id))
decoded_preds, decoded_labels = post_process_text(decoded_preds, decoded_labels)
# bleu_result = compute_bleu(decoded_preds, decoded_labels)
bleu_result = compute_bleu_hf(decoded_preds, decoded_labels)
return bleu_result, decoded_preds, decoded_labels

343
slm/examples/question_generation/unimo-text/README.md
View File

@@ -1,343 +0,0 @@
# 问题生成
**目录**
- [问题生成](#问题生成)
- [简介](#简介)
<!-- - [基于预训练语言模型的问题生成](#基于预训练语言模型的问题生成) -->
<!-- - [效果展示](#效果展示) -->
- [开箱即用](#开箱即用)
- [训练定制](#训练定制)
- [环境依赖](#环境依赖)
- [代码结构说明](#代码结构说明)
- [问题生成应用定制训练全流程介绍](#问题生成定制训练全流程介绍)
- [数据准备](#数据准备)
- [数据加载](#数据加载)
- [数据处理](#数据处理)
- [从本地文件创建数据集-可选](#从本地文件创建数据集-可选)
- [模型训练](#模型训练)
- [模型预测](#模型预测)
- [模型转换部署](#模型转换部署)
- [FasterTransformer 加速及模型静态图导出](#fastertransformer 加速及模型静态图导出)
- [模型部署](#模型部署)
- [References](#references)
## 简介
Question Generation（QG），即问题生成，指的是给定一段上下文，自动生成一个流畅且符合上下文主题的问句。问题生成通常可以分为，无答案问题生成和有答案问题生成，这里只关注应用更广的有答案问题生成。
问题生成技术在教育、咨询、搜索、推荐等多个领域均有着巨大的应用价值。具体来说，问题生成可广泛应用于问答系统语料库构建，事实性问题生成，教育行业题库生成，对话提问，聊天机器人意图理解，对话式搜索意图提问，闲聊机器人主动提问等等场景。
本项目是基于预训练语言模型 UNIMO-Text 的问题生成，具有以下优势：
- 效果领先。基于百度自研中文预训练语言模型 UNIMO-Text，并提供基于模版策略和大规模多领域问题生成数据集训练的通用问题生成预训练模型`unimo-text-1.0-question-generation`。
- 开箱即用。本项目提供 TaskFlow 接口，无需训练，仅需几行代码便可预测。
- 高性能推理。本项目基于 FasterTransformer 进行推理加速，能够提供更高性能的推理体验，优化后的推理模型在 dureader_qg 开发集的推理耗时缩短为优化前的1/5。
- 训练推理部署全流程打通。本项目提供了全面的定制训练流程，从数据准备、模型训练预测，到模型推理部署，一应俱全。
<!-- ### 基于预训练语言模型的问题生成
基于预训练语言模型（Pretrained Language Models, PLMs）范式的问题生成是目前最常用、效果最好(SOTA)的方式。
预训练模型是在超大规模的语料采用无监督或者弱监督的方式进行预训练，能够学习如何准确地理解自然语言并以自然语言的形式流畅表达，这两项都是完成文本生成任务的重要能力。
PaddleNLP 提供了方便易用的接口，可指定模型名或模型参数文件路径通过 from_pretrained()方法加载不同网络结构的预训练模型，且相应预训练模型权重下载速度快速、稳定。
Transformer 预训练模型汇总包含了如 ERNIE、BERT、T5、UNIMO 等主流预训练模型。下面以中文 unimo-text-1.0模型为例，演示如何加载预训练模型和分词器：
```
from paddlenlp.transformers import ErnieForGeneration, ErnieTokenizer
model_name = "ernie-1.0"
model = UNIMOLMHeadModel.from_pretrained(model_name)
tokenizer = UNIMOTokenizer.from_pretrained(model_name)
``` -->
## 开箱即用
PaddleNLP 提供开箱即用的产业级 NLP 预置任务能力，无需训练，一键预测。
#### 支持单条、批量预测
```python
>>> from paddlenlp import Taskflow
# 默认模型为 unimo-text-1.0-dureader_qg
>>> question_generator = Taskflow("question_generation")
# 单条输入
>>> question_generator([
{"context": "奇峰黄山千米以上的山峰有77座，整座黄山就是一座花岗岩的峰林，自古有36大峰，36小峰，最高峰莲花峰、最险峰天都峰和观日出的最佳点光明顶构成黄山的三大主峰。", "answer": "莲花峰"}
])
'''
['黄山最高峰是什么']
'''
# 多条输入
>>> question_generator([
{"context": "奇峰黄山千米以上的山峰有77座，整座黄山就是一座花岗岩的峰林，自古有36大峰，36小峰，最高峰莲花峰、最险峰天都峰和观日出的最佳点光明顶构成黄山的三大主峰。", "answer": "莲花峰"},
{"context": "弗朗索瓦·韦达外文名：franciscusvieta国籍：法国出生地：普瓦图出生日期：1540年逝世日期：1603年12月13日职业：数学家主要成就：为近代数学的发展奠定了基础。", "answer": "法国"}
])
'''
['黄山最高峰是什么', '弗朗索瓦是哪里人']
'''
```
关键配置参数说明：
* `model`：可选模型，默认为 unimo-text-1.0-dureader_qg，支持的模型有["unimo-text-1.0", "unimo-text-1.0-dureader_qg", "unimo-text-1.0-question-generation", "unimo-text-1.0-question-generation-dureader_qg"]。
具体参数配置可参考[Taskflow 文档](../../../../docs/zh/model_zoo/taskflow.md)。
## 训练定制
### 环境依赖
- nltk
- evaluate
- tqdm
安装方式：`pip install -r requirements.txt`
### 代码结构说明
以下是本项目主要代码结构及说明：
```text
├── deploy # 部署
│ ├── paddle_inference # PaddleInference高性能推理部署
│ │ ├── inference_unimo_text.py # 推理部署脚本
│ │ └── README.md # 说明文档
│ └── paddle_serving
│ ├── config.yml # 配置文件
│ ├── pipeline_client.py # 客户端程序
│ ├── pipeline_service.py # 服务器程序
│ └── README.md # 说明文档
├── export_model.py # 动态图参数导出静态图参数脚本
├── train.py # 训练脚本
├── predict.py # 预测评估脚本
├── utils.py # 工具函数脚本
└── README.md # 说明文档
```
### 问题生成定制训练全流程介绍
接下来，我们将按数据准备、训练、预测、推理部署等四个阶段对问题生成应用的全流程进行介绍。
1. **数据准备**
- 默认使用中文问题生成数据集 DuReader_QG 进行实验，该数据集已集成到 PaddleNLP。
- 如果已有标注好的本地数据集，我们需要根据将数据集整理为文档要求的格式，请参考[从本地文件创建数据集（可选）](#从本地文件创建数据集（可选）)。
2. **模型训练**
- 数据准备完成后，可以开始使用我们的数据集对预训练模型进行微调训练。我们可以根据任务需求，调整可配置参数，选择使用 GPU 或 CPU 进行模型训练，脚本默认保存在开发集最佳表现模型。中文任务默认使用`unimo-text-1.0`模型，unimo-text-1.0还支持 large 模型。此外本项目还提供基于大规模多领域问题生成数据集训练的通用问题生成预训练模型`unimo-text-1.0-question-generation`，详见[UNIMO 模型汇总](https://paddlenlp.readthedocs.io/zh/latest/model_zoo/transformers/UNIMO/contents.html)，用户可以根据任务和设备需求进行选择。
3. **模型预测**
- 训练结束后，我们可以加载保存的最佳模型进行模型测试，打印模型预测结果。
4. **模型转换部署**
- 在现实部署场景中，我们通常不仅对模型的精度表现有要求，也需要考虑模型性能上的表现。我们可以使用模型裁剪进一步压缩模型体积，问题生成应用已提供裁剪 API 对上一步微调后的模型进行裁剪，模型裁剪之后会默认导出静态图模型。
- 模型部署需要将保存的最佳模型参数（动态图）导出成静态图参数，用于后续的推理部署。
- 问题生成应用提供了基于 Paddle Serving 的本地部署 predictor，并且支持在 GPU 设备使用 Faster Generation 进行加速。
- 问题生成应用提供了基于 Paddle Serving 的服务端部署方案。
### 数据准备
#### 数据加载
[**DuReader_QG**数据集](https://www.luge.ai/#/luge/dataDetail?id=8)是一个中文问题生成数据集，我们使用该数据集作为应用案例进行实验。**DuReader_QG**中的数据主要由由上下文、问题、答案3个主要部分组成，其任务描述为给定上下文 p 和答案 a，生成自然语言表述的问题 q，且该问题符合段落和上下文的限制。
为了方便用户快速测试，PaddleNLP Dataset API 内置了 DuReader_QG 数据集，一键即可完成数据集加载，示例代码如下：
```python
from paddlenlp.datasets import load_dataset
train_ds, dev_ds = load_dataset('dureader_qg', splits=('train', 'dev'))
```
#### 数据处理
针对**DuReader_QG**数据集，我们需要将 QA 任务格式的数据进行转换从而得到 text2text 形式的数据，我们默认使用模版的方式构造输入数据，默认模版如下，其他形式输入数据用户可以在 convert_example 函数中自行定义。
```text
答案: <answer_text> 上下文: <context_text>
问题: <question_text>
```
#### 从本地文件创建数据集-可选
在许多情况下，我们需要使用本地数据集来训练我们的问题生成模型，本项目支持使用固定格式本地数据集文件进行训练。
使用本地文件，只需要在模型训练时指定`train_file` 为本地训练数据地址，`predict_file` 为本地测试数据地址即可。
本地数据集目录结构如下：
```text
data/
├── train.json # 训练数据集文件
├── dev.json # 开发数据集文件
└── test.json # 可选，待预测数据文件
```
本地数据集文件格式如下：
- train.json/dev.json/test.json 文件格式：
```text
{
"context": <context_text>,
"answer": <answer_text>,
"question": <question_text>,
}
...
```
- train.json/dev.json/test.json 文件样例：
```text
{
"context": "欠条是永久有效的,未约定还款期限的借款合同纠纷,诉讼时效自债权人主张债权之日起计算,时效为2年。 根据《中华人民共和国民法通则》第一百三十五条:向人民法院请求保护民事权利的诉讼时效期间为二年,法律另有规定的除外。 第一百三十七条:诉讼时效期间从知道或者应当知道权利被侵害时起计算。但是,从权利被侵害之日起超过二十年的,人民法院不予保护。有特殊情况的,人民法院可以延长诉讼时效期间。 第六十二条第(四)项:履行期限不明确的,债务人可以随时履行,债权人也可以随时要求履行,但应当给对方必要的准备时间。",
"answer": "永久有效",
"question": "欠条的有效期是多久"
}
...
```
更多数据集读取格式详见[数据集加载](https://paddlenlp.readthedocs.io/zh/latest/data_prepare/dataset_load.html#)和[自定义数据集](https://paddlenlp.readthedocs.io/zh/latest/data_prepare/dataset_self_defined.html)。
### 模型训练
运行如下命令即可在样例训练集上进行 finetune，并在样例验证集上进行验证。
```shell
# GPU启动，参数`--gpus`指定训练所用的GPU卡号，可以是单卡，也可以多卡
# 例如使用1号和2号卡，则：`--gpu 1,2`
unset CUDA_VISIBLE_DEVICES
python -m paddle.distributed.launch --gpus "1,2" --log_dir ./unimo/finetune/log train.py \
--dataset_name=dureader_qg \
--model_name_or_path="unimo-text-1.0" \
--save_dir=./unimo/finetune/checkpoints \
--output_path ./unimo/finetune/predict.txt \
--logging_steps=100 \
--save_steps=500 \
--epochs=20 \
--batch_size=16 \
--learning_rate=1e-5 \
--warmup_proportion=0.02 \
--weight_decay=0.01 \
--max_seq_len=512 \
--max_target_len=30 \
--do_train \
--do_predict \
--max_dec_len=20 \
--min_dec_len=3 \
--num_return_sequences=1 \
--template=1 \
--device=gpu
```
关键参数释义如下：
- `gpus` 指示了训练所用的 GPU，使用多卡训练可以指定多个 GPU 卡号，例如 --gpus "0,1"。
- `dataset_name` 数据集名称，当`train_file`和`predict_file`为 None 时将加载`dataset_name`的训练集和开发集，默认为`dureader_qg`。
- `train_file` 本地训练数据地址，数据格式必须与`dataset_name`所指数据集格式相同，默认为 None。
- `predict_file` 本地测试数据地址，数据格式必须与`dataset_name`所指数据集格式相同，默认为 None。
- `model_name_or_path` 指示了 finetune 使用的具体预训练模型，可以是 PaddleNLP 提供的预训练模型，或者是本地的预训练模型。如果使用本地的预训练模型，可以配置本地模型的目录地址，例如: ./checkpoints/model_xx/，目录中需包含 paddle 预训练模型 model_state.pdparams。如果使用 PaddleNLP 提供的预训练模型，可以选择下面其中之一。
| 可选预训练模型 |
|---------------------------------|
| unimo-text-1.0 |
| unimo-text-1.0-large |
| unimo-text-1.0-question-generation |
<!-- | T5-PEGASUS |
| ernie-1.0 |
| ernie-gen-base-en |
| ernie-gen-large-en |
| ernie-gen-large-en-430g | -->
- `save_dir` 表示模型的保存路径。
- `output_path` 表示预测结果的保存路径。
- `logging_steps` 表示日志打印间隔。
- `save_steps` 表示模型保存及评估间隔。
- `seed` 表示随机数生成器的种子。
- `epochs` 表示训练轮数。
- `batch_size` 表示每次迭代**每张卡**上的样本数目。
- `learning_rate` 表示基础学习率大小，将于 learning rate scheduler 产生的值相乘作为当前学习率。
- `weight_decay` 表示 AdamW 优化器中使用的 weight_decay 的系数。
- `warmup_proportion` 表示学习率逐渐升高到基础学习率（即上面配置的 learning_rate）所需要的迭代数占总步数的比例。
- `max_seq_len` 模型输入序列的最大长度。
- `max_target_len` 模型训练时标签的最大长度。
- `min_dec_len` 模型生成序列的最小长度。
- `max_dec_len` 模型生成序列的最大长度。
- `do_train` 是否进行训练。
- `do_predict` 是否进行预测，在验证集上会自动评估。
- `device` 表示使用的设备，从 gpu 和 cpu 中选择。
- `template` 表示使用的模版，从[0, 1, 2, 3, 4]中选择，0表示不选择模版，1表示使用默认模版。
程序运行时将会自动进行训练和验证，训练过程中会自动保存模型在指定的`save_dir`中。如：
```text
./unimo/finetune/checkpoints
├── model_1000
│ ├── model_config.json
│ ├── model_state.pdparams
│ ├── special_tokens_map.json
│ ├── tokenizer_config.json
│ └── vocab.txt
└── ...
```
**NOTE:** 如需恢复模型训练，`model_name_or_path`配置本地模型的目录地址即可。
微调的模型在 dureader_qg 验证集上有如下结果(指标为 BLEU-4)，其中`unimo-text-1.0-dureader_qg-w/o-template`表示不使用模版策略微调的结果，`unimo-text-1.0-large-dureader_qg`表示使用 large 模型微调的结果，`unimo-text-1.0-question-generation-dureader_qg`表示在通用问题生成预训练模型`unimo-text-1.0-question-generation`上微调的结果：
| model_name | DuReaderQG |
| :-----------------------------: | :-----------: |
| unimo-text-1.0-dureader_qg-w/o-template | 39.61 |
| unimo-text-1.0-dureader_qg | 41.08 |
| unimo-text-1.0-large-dureader_qg | 41.51 |
| unimo-text-1.0-question-generation-dureader_qg | 44.02 |
### 模型预测
运行下方脚本可以使用训练好的模型进行预测。
```shell
export CUDA_VISIBLE_DEVICES=0
python -u predict.py \
--dataset_name=dureader_qg \
--model_name_or_path=your_model_path \
--output_path=./predict.txt \
--logging_steps=100 \
--batch_size=16 \
--max_seq_len=512 \
--max_target_len=30 \
--do_predict \
--max_dec_len=20 \
--min_dec_len=3 \
--template=1 \
--device=gpu
```
关键参数释义如下：
- `output_path` 表示预测输出结果保存的文件路径，默认为./predict.txt。
- `model_name_or_path` 指示了 finetune 使用的具体预训练模型，可以是 PaddleNLP 提供的预训练模型，或者是本地的微调好的预训练模型。如果使用本地的预训练模型，可以配置本地模型的目录地址，例如: ./checkpoints/model_xx/，目录中需包含 paddle 预训练模型 model_state.pdparams。
### 模型转换部署
#### FasterTransformer 加速及模型静态图导出
使用动态图训练结束之后，可以通过[静态图导出脚本](export_model.py)实现基于 FasterTransformer 的高性能预测加速，并将动态图参数导出成静态图参数，静态图参数保存在`output_path`指定路径中。运行方式：
```shell
python export_model.py \
--model_name_or_path ./checkpoint \
--inference_model_dir ./export_checkpoint \
--max_dec_len 50 \
--use_fp16_decoding
```
关键参数释义如下：
* `model_name_or_path`：动态图训练保存的参数路径；默认为"./checkpoint"。
* `inference_model_dir`：静态图图保存的参数路径；默认为"./export_checkpoint"。
* `max_dec_len`：最大输出长度。
* `use_fp16_decoding`:是否使用 fp16解码进行预测。
执行命令后将会自动导出模型到指定的 `inference_model_dir` 中，保存模型文件结构如下所示：
```text
├── unimo_text.pdiparams
├── unimo_text.pdiparams.info
└── unimo_text.pdmodel
```
#### 模型部署
本项目提供多种不同场景的部署方案，请根据实际情况进行选择：
|部署方案|特色|场景|硬件|
|-|-|-|-|
|Paddle Inference<br>服务端／云端|通用性|模型算法复杂<br>硬件高性能|X86 CPU<br>NVIDIA 全系列 GPU<br>龙芯／飞腾等国产 CPU<br>昆仑／昇腾／海光 DCU 等 AI 加速芯片
|Paddle Serving<br>服务化|高并发|大流量、高并发、低延时、高吞吐<br>资源弹性调控应对服务流量变化<br>支持模型组合、加密、热更新等|X86/Arm CPU<br>NVIDIA GPU<br>昆仑／昇腾等
问题生成应用已打通多种场景部署方案，点击链接获取具体的使用教程。
- [Paddle Inference 推理 (Python)](./deploy/paddle_inference/README.md)
- [Paddle Serving 服务化部署（Python）](./deploy/paddle_serving/README.md)
## References
Zheng, Chujie, and Minlie Huang. "Exploring prompt-based few-shot learning for grounded dialog generation." arXiv preprint arXiv:2109.06513 (2021).
Li, Wei, et al. "Unimo: Towards unified-modal understanding and generation via cross-modal contrastive learning." arXiv preprint arXiv:2012.15409 (2020).

54
slm/examples/question_generation/unimo-text/deploy/paddle_inference/README.md
View File

@@ -1,54 +0,0 @@
# Paddle Inference 部署
本文档将介绍如何使用[Paddle Inference](https://paddle-inference.readthedocs.io/en/latest/guides/introduction/index_intro.html#paddle-inference)工具进行问题生成应用高性能推理推理部署。
**目录**
* [背景介绍](#背景介绍)
* [导出预测部署模型](#导出预测部署模型)
* [基于 Python 预测](#基于 Python 预测)
## 背景介绍
Paddle inference 和主框架的 Model.predict 均可实现推理预测，Paddle Inference 是飞桨的原生推理库， 作用于服务器端和云端，提供高性能的推理能力，主框架的 Model 对象是一个具备训练、测试、推理的神经网络。相比于 Model.predict，inference 可使用 MKLDNN、CUDNN、TensorRT 进行预测加速。Model.predict 适用于训练好的模型直接进行预测，paddle inference 适用于对推理性能、通用性有要求的用户，针对不同平台不同的应用场景进行了深度的适配优化，保证模型在服务器端即训即用，快速部署。由于 Paddle Inference 能力直接基于飞桨的训练算子，因此它支持飞桨训练出的所有模型的推理。
Paddle Inference Python 端预测部署主要包含两个步骤：
- 导出预测部署模型
- 基于 Python 预测
## 导出预测部署模型
部署时需要使用预测格式的模型（即动态图转静态图操作）。预测格式模型相对训练格式模型而言，在拓扑上裁剪掉了预测不需要的算子，并且会做特定部署优化。具体操作详见[FasterTransformer 加速及模型静态图导出](../../README.md)。
## 基于 Python 预测
<!-- 同上，高性能预测的默认输入和输出形式也为文件，可分别通过 test_path 和 save_path 进行指定，通过如下命令便可以基于 Paddle Inference 进行高性能预测： -->
在终端输入以下命令可在 GPU 上进行预测：
```shell
python deploy/paddle_inference/inference.py \
--inference_model_dir ./export_checkpoint \
--model_name_or_path "unimo-text-1.0" \
--predict_file predict_file_name \
--output_path output_path_name \
--device gpu \
```
<!-- 在终端输入以下命令可在 CPU 上进行预测：
```shell
python deploy/paddle_inference/inference_unimo_text.py --inference_model_dir ./export_checkpoint --device cpu
``` -->
经静态图转换，FastTransformer 性能优化，Paddle Inference 加速后的部署模型在 dureader_qg devset 的预测时间为27.74秒，相较于未优化前169.24秒，耗时缩减为原来的16.39%。
关键参数释义如下：
* `inference_model_dir`：用于高性能推理的静态图模型参数路径，默认为"./export_checkpoint"。
* `model_name_or_path`：tokenizer 对应模型或路径，默认为"unimo-text-1.0"。
* `dataset_name`：数据集名称，默认为`dureader_qg`。
* `predict_file`：本地预测数据地址，数据格式必须与`dataset_name`所指数据集格式相同，默认为 None，当为 None 时默认加载`dataset_name`的 dev 集。
* `output_path`：表示预测结果的保存路径。
* `device`：推理时使用的设备，可选项["gpu"]，默认为"gpu"。
* `batch_size`：进行推理时的批大小，默认为16。
* `precision`：当使用 TensorRT 进行加速推理时，所使用的 TensorRT 精度，可选项["fp32", "fp16"]，默认为"fp32"。
<!-- * `precision`：当使用 TensorRT 进行加速推理时，所使用的 TensorRT 精度，可选项["fp32", "fp16", "int8"]，默认为"fp32"。 -->
<!-- * `device`：推理时使用的设备，可选项["gpu", "cpu", "xpu"]，默认为"gpu"。 -->
<!-- * `enable_mkldnn`：当使用 cpu 时，选择是否使用 MKL-DNN(oneDNN)进行加速推理，默认为 False。 -->
<!-- * `cpu_threads`：当使用 cpu 时，推理所用的进程数，默认为10。 -->
<!-- * `use_tensorrt`：当使用 gpu 时，选择是否使用 TensorRT 进行加速推理，默认为 False。 -->

260
slm/examples/question_generation/unimo-text/deploy/paddle_inference/infer_utils.py
View File

@@ -1,260 +0,0 @@
# Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import random
from functools import partial
import numpy as np
import paddle
import paddle.distributed as dist
from paddle.io import BatchSampler, DataLoader, DistributedBatchSampler
from paddlenlp.data import Pad
def postprocess_response(token_ids, tokenizer):
"""Post-process the decoded sequence. Truncate from the first <eos>."""
eos_pos = len(token_ids)
for i, tok_id in enumerate(token_ids):
if tok_id == tokenizer.mask_token_id:
eos_pos = i
break
token_ids = token_ids[:eos_pos]
tokens = tokenizer.convert_ids_to_tokens(token_ids)
tokens = tokenizer.merge_subword(tokens)
return tokens
def print_args(args):
print("----------- Configuration Arguments -----------")
for arg, value in sorted(vars(args).items()):
print("%s: %s" % (arg, value))
print("------------------------------------------------")
def set_seed(seed):
# Use the same data seed(for data shuffle) for all procs to guarantee data
# consistency after sharding.
random.seed(seed)
np.random.seed(seed)
# Maybe different op seeds(for dropout) for different procs is better.
paddle.seed(seed + dist.get_rank())
def convert_example(
example, tokenizer, max_seq_len=512, max_target_len=128, max_title_len=256, mode="test", template=0
):
"""Convert all examples into necessary features."""
if mode == "pretrain" or mode == "pretrain_test":
context = example["context"]
answer = example["answer"]
target = example["target"]
source = "答案：" + answer + tokenizer.sep_token + "上下文：" + context
title = None
elif mode == "train" or mode == "test":
target = None
if "source" in example and "title" in example:
source = example["source"]
title = None
if "title" in example.keys():
title = example["title"]
elif "context" in example and "answer" in example:
source = example["context"]
title = None
if "answer" in example.keys():
title = example["answer"]
else:
assert False, "Source and title are not in the input dictionary, nor are context and answer."
if "target" in example.keys():
target = example["target"]
if template == 1:
source = "答案：" + title + tokenizer.sep_token + "上下文：" + source
title = None
if target:
target = "问题：" + target
elif template == 2:
source = "答案：" + title + tokenizer.sep_token + "上下文：" + source
title = None
if target:
target = "在已知答案的前提下，问题：" + target
elif template == 3:
source = "这是一个问题生成任务，根据提供的答案和上下文，来生成问题。" + title + tokenizer.sep_token + "上下文：" + source
title = None
if target:
target = "问题：" + target
if mode == "train" or mode == "pretrain":
tokenized_example = tokenizer.gen_encode(
source,
title=title,
target=target,
max_seq_len=max_seq_len,
max_target_len=max_target_len,
max_title_len=max_title_len,
return_position_ids=True,
return_length=True,
)
target_start = tokenized_example["input_ids"].index(tokenizer.cls_token_id, 1)
target_end = tokenized_example["seq_len"]
# Use to gather the logits corresponding to the labels during training
tokenized_example["masked_positions"] = list(range(target_start, target_end - 1))
tokenized_example["labels"] = tokenized_example["input_ids"][target_start + 1 : target_end]
return tokenized_example
elif mode == "test" or mode == "pretrain_test":
tokenized_example = tokenizer.gen_encode(
source,
title=title,
max_seq_len=max_seq_len,
max_title_len=max_title_len,
add_start_token_for_decoding=True,
return_position_ids=True,
return_length=True,
)
if "target" in example and example["target"]:
tokenized_example["target"] = example["target"]
return tokenized_example
def batchify_fn(batch_examples, pad_val, mode="test"):
def pad_mask(batch_attention_mask):
batch_size = len(batch_attention_mask)
max_len = max(map(len, batch_attention_mask))
attention_mask = np.ones((batch_size, max_len, max_len), dtype="float32") * -1e9
for i, mask_data in enumerate(attention_mask):
seq_len = len(batch_attention_mask[i])
mask_data[-seq_len:, -seq_len:] = np.array(batch_attention_mask[i], dtype="float32")
# In order to ensure the correct broadcasting mechanism, expand one
# dimension to the second dimension (n_head of Transformer).
attention_mask = np.expand_dims(attention_mask, axis=1)
return attention_mask
pad_func = Pad(pad_val=pad_val, pad_right=False, dtype="int64")
input_ids = pad_func([example["input_ids"] for example in batch_examples])
token_type_ids = pad_func([example["token_type_ids"] for example in batch_examples])
position_ids = pad_func([example["position_ids"] for example in batch_examples])
attention_mask = pad_mask([example["attention_mask"] for example in batch_examples])
seq_len = np.asarray([example["seq_len"] for example in batch_examples], dtype="int32")
if mode == "train" or mode == "pretrain":
max_len = max([example["seq_len"] for example in batch_examples])
masked_positions = np.concatenate(
[
np.array(example["masked_positions"]) + (max_len - example["seq_len"]) + i * max_len
for i, example in enumerate(batch_examples)
]
)
labels = np.concatenate([np.array(example["labels"], dtype="int64") for example in batch_examples])
return input_ids, token_type_ids, position_ids, attention_mask, masked_positions, labels
elif mode == "test" or mode == "pretrain_test":
return input_ids, token_type_ids, position_ids, attention_mask, seq_len
def create_data_loader(dataset, tokenizer, args, mode="test"):
trans_func = partial(convert_example, tokenizer=tokenizer, mode="test", template=1)
dataset = dataset.map(trans_func, lazy=True)
if mode == "pretrain":
batch_sampler = DistributedBatchSampler(dataset, batch_size=args.batch_size, shuffle=True)
elif mode == "train":
batch_sampler = DistributedBatchSampler(dataset, batch_size=args.batch_size, shuffle=True)
elif mode == "test" or mode == "pretrain_test":
batch_sampler = BatchSampler(dataset, batch_size=args.batch_size // 2, shuffle=False)
collate_fn = partial(batchify_fn, pad_val=tokenizer.pad_token_id, mode=mode)
data_loader = DataLoader(dataset, batch_sampler=batch_sampler, collate_fn=collate_fn, return_list=True)
return dataset, data_loader
def post_process_sum(token_ids, tokenizer):
"""Post-process the decoded sequence. Truncate from the first <eos>."""
eos_pos = len(token_ids)
for i, tok_id in enumerate(token_ids):
if tok_id == tokenizer.mask_token_id:
eos_pos = i
break
token_ids = token_ids[:eos_pos]
tokens = tokenizer.convert_ids_to_tokens(token_ids)
tokens = tokenizer.merge_subword(tokens)
special_tokens = ["[UNK]"]
tokens = [token for token in tokens if token not in special_tokens]
return token_ids, tokens
def remove_template(instr):
"""Remove template prefix of decoded sequence."""
outstr = instr.strip("问题：")
outstr = instr.strip("在已知答案的前提下，问题：")
return outstr
def select_sum(ids, scores, tokenizer, max_dec_len=None, num_return_sequences=1):
results = []
group = []
tmp = []
if scores is not None:
ids = ids.numpy()
scores = scores.numpy()
if len(ids) != len(scores) or (len(ids) % num_return_sequences) != 0:
raise ValueError(
"the length of `ids` is {}, but the `num_return_sequences` is {}".format(
len(ids), num_return_sequences
)
)
for pred, score in zip(ids, scores):
pred_token_ids, pred_tokens = post_process_sum(pred, tokenizer)
num_token = len(pred_token_ids)
target = "".join(pred_tokens)
target = remove_template(target)
# not ending
if max_dec_len is not None and num_token >= max_dec_len:
score -= 1e3
tmp.append([target, score])
if len(tmp) == num_return_sequences:
group.append(tmp)
tmp = []
for preds in group:
preds = sorted(preds, key=lambda x: -x[1])
results.append(preds[0][0])
else:
ids = ids.numpy()
for pred in ids:
pred_token_ids, pred_tokens = post_process_sum(pred, tokenizer)
num_token = len(pred_token_ids)
response = "".join(pred_tokens)
response = remove_template(response)
# TODO: Support return scores in FT.
tmp.append([response])
if len(tmp) == num_return_sequences:
group.append(tmp)
tmp = []
for preds in group:
results.append(preds[0][0])
return results

223
slm/examples/question_generation/unimo-text/deploy/paddle_inference/inference.py
View File

@@ -1,223 +0,0 @@
# Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import argparse
import os
import time
from pprint import pprint
import numpy as np
import paddle
from infer_utils import create_data_loader, postprocess_response, select_sum
from paddle import inference
from paddlenlp.datasets import load_dataset
from paddlenlp.ops.ext_utils import load
from paddlenlp.transformers import UNIMOTokenizer
def setup_args():
"""Setup arguments."""
parser = argparse.ArgumentParser()
parser.add_argument(
"--inference_model_dir", default="./infer_model", type=str, help="Path to save inference model of UNIMOText. "
)
parser.add_argument(
"--model_name_or_path", type=str, default="unimo-text-1.0", help="The path or shortcut name of the tokenizer."
)
parser.add_argument(
"--device", default="gpu", choices=["gpu", "cpu", "xpu"], help="Device selected for inference."
)
parser.add_argument(
"--use_tensorrt",
default=False,
type=eval,
choices=[True, False],
help="Whether to use inference engin TensorRT when using gpu.",
)
parser.add_argument(
"--enable_mkldnn",
default=False,
type=eval,
choices=[True, False],
help="Enable to use mkldnn to speed up when using cpu.",
)
parser.add_argument("--cpu_threads", default=10, type=int, help="Number of threads to predict when using cpu.")
parser.add_argument(
"--precision", default="fp32", type=str, choices=["fp32", "fp16", "int8"], help="The tensorrt precision."
)
parser.add_argument("--batch_size", type=int, default=16, help="Batch size per GPU/CPU for training.")
parser.add_argument(
"--output_path", type=str, default="./predict.txt", help="The file path where the infer result will be saved."
)
parser.add_argument("--logging_steps", type=int, default=100, help="Log every X updates steps.")
parser.add_argument("--dataset_name", type=str, default="dureader_qg", help="The name of the dataset to load.")
parser.add_argument("--predict_file", type=str, required=False, default=None, help="Predict data path.")
parser.add_argument("--max_dec_len", type=int, default=20, help="The maximum sequence length of decoding.")
parser.add_argument(
"--num_return_sequences",
type=int,
default=1,
help="The numbers of returned sequences for one input in generation.",
)
args = parser.parse_args()
return args
def setup_predictor(args):
"""Setup inference predictor."""
# Load FastGeneration lib.
load("FastGeneration", verbose=True)
model_file = os.path.join(args.inference_model_dir, "unimo_text.pdmodel")
params_file = os.path.join(args.inference_model_dir, "unimo_text.pdiparams")
if not os.path.exists(model_file):
raise ValueError("not find model file path {}".format(model_file))
if not os.path.exists(params_file):
raise ValueError("not find params file path {}".format(params_file))
config = inference.Config(model_file, params_file)
if args.device == "gpu":
config.enable_use_gpu(100, 0)
config.switch_ir_optim()
config.enable_memory_optim()
config.disable_glog_info()
precision_map = {
"fp16": inference.PrecisionType.Half,
"fp32": inference.PrecisionType.Float32,
"int8": inference.PrecisionType.Int8,
}
precision_mode = precision_map[args.precision]
if args.use_tensorrt:
config.enable_tensorrt_engine(
max_batch_size=args.batch_size, min_subgraph_size=30, precision_mode=precision_mode
)
elif args.device == "cpu":
config.disable_gpu()
if args.enable_mkldnn:
config.enable_mkldnn()
config.set_mkldnn_cache_capacity(10)
config.set_cpu_math_library_num_threads(args.cpu_threads)
elif args.device == "xpu":
config.enable_xpu(100)
predictor = inference.create_predictor(config)
return predictor
@paddle.no_grad()
def infer_one(args, predictor, inputs=None):
"""Use predictor to inference."""
tokenizer = UNIMOTokenizer.from_pretrained("unimo-text-1.0")
if not inputs:
inputs = {
"context": "奇峰黄山千米以上的山峰有77座，整座黄山就是一座花岗岩的峰林，自古有36大峰，36小峰，最高峰莲花峰、最险峰天都峰和观日出的最佳点光明顶构成黄山的三大主峰。",
"answer": "莲花峰",
}
inputs = "答案：" + inputs["answer"] + tokenizer.sep_token + "上下文：" + inputs["context"]
data = tokenizer.gen_encode(
inputs, add_start_token_for_decoding=True, return_length=True, is_split_into_words=False
)
input_handles = {}
for name in predictor.get_input_names():
input_handles[name] = predictor.get_input_handle(name)
if name == "attention_mask":
input_handles[name].copy_from_cpu(np.expand_dims(np.asarray(data[name], dtype="float32"), axis=(0, 1)))
else:
input_handles[name].copy_from_cpu(np.asarray(data[name], dtype="int32").reshape([1, -1]))
output_handles = [predictor.get_output_handle(name) for name in predictor.get_output_names()]
predictor.run()
output = [output_handle.copy_to_cpu() for output_handle in output_handles]
for sample in output[0][:, :, 0].tolist():
print("".join(postprocess_response(sample, tokenizer)))
@paddle.no_grad()
def infer(args, predictor, data_loader, tokenizer):
print("Infer begin...")
pred_ref = []
total_time = 0.0
start_time = time.time()
for step, inputs in enumerate(data_loader, 1):
input_ids, token_type_ids, position_ids, attention_mask, seq_len = inputs
data = {
"input_ids": input_ids,
"token_type_ids": token_type_ids,
"position_ids": position_ids,
"attention_mask": attention_mask,
"seq_len": seq_len,
}
input_handles = {}
for name in predictor.get_input_names():
input_handles[name] = predictor.get_input_handle(name)
if name == "attention_mask":
input_handles[name].copy_from_cpu(np.asarray(data[name], dtype="float32"))
else:
input_handles[name].copy_from_cpu(np.asarray(data[name], dtype="int32"))
output_handles = [predictor.get_output_handle(name) for name in predictor.get_output_names()]
predictor.run()
output = [output_handle.copy_to_cpu() for output_handle in output_handles]
ids = output[0]
scores = output[1]
ids = paddle.to_tensor(ids, dtype="int32")[:, 0, :]
scores = paddle.to_tensor(scores, dtype="float32")
total_time += time.time() - start_time
if step % args.logging_steps == 0:
print("step %d - %.3fs/step" % (step, total_time / args.logging_steps))
total_time = 0.0
results = select_sum(ids, scores, tokenizer, args.max_dec_len, args.num_return_sequences)
pred_ref.extend(results)
start_time = time.time()
with open(args.output_path, "w", encoding="utf-8") as fout:
for ref in pred_ref:
fout.write(ref + "\n")
print("\nSave inference result into: %s" % args.output_path)
if "target" in data_loader.dataset[0].keys():
with open(args.output_path + ".reference.txt", "w", encoding="utf-8") as fout:
targets = [example["target"] for example in data_loader.dataset]
for target in targets:
fout.write(target + "\n")
if __name__ == "__main__":
args = setup_args()
pprint(args)
predictor = setup_predictor(args)
tokenizer = UNIMOTokenizer.from_pretrained(args.model_name_or_path)
ds = load_dataset(args.dataset_name, splits="dev", data_files=args.predict_file)
ds, data_loader = create_data_loader(ds, tokenizer, args, "test")
time_begin = time.time()
infer(args, predictor, data_loader, tokenizer)
print("inference cost time:", time.time() - time_begin)

142
slm/examples/question_generation/unimo-text/deploy/paddle_serving/README.md
View File

@@ -1,142 +0,0 @@
# Paddle Serving 服务化部署
本文档将介绍如何使用[Paddle Serving](https://github.com/PaddlePaddle/Serving/blob/develop/README_CN.md)工具部署问题生成在线服务。
## 目录
- [Paddle Serving 服务化部署](#paddle-serving 服务化部署)
- [目录](#目录)
- [背景介绍](#背景介绍)
- [环境准备](#环境准备)
- [安装 Paddle Serving](#安装 paddle-serving)
- [模型转换](#模型转换)
- [pipeline 部署](#pipeline 部署)
- [修改配置文件](#修改配置文件)
- [server 启动服务](#server 启动服务)
- [client 发送服务请求](#client 发送服务请求)
## 背景介绍
Paddle Serving 依托深度学习框架 PaddlePaddle 旨在帮助深度学习开发者和企业提供高性能、灵活易用的工业级在线推理服务。Paddle Serving 支持 RESTful、gRPC、bRPC 等多种协议，提供多种异构硬件和多种操作系统环境下推理解决方案，和多种经典预训练模型示例。集成高性能服务端推理引擎 Paddle Inference 和端侧引擎 Paddle Lite。设计并实现基于有向无环图(DAG) 的异步流水线高性能推理框架，具有多模型组合、异步调度、并发推理、动态批量、多卡多流推理、请求缓存等特性。
Paddle Serving Python 端预测部署主要包含以下步骤：
- 环境准备
- 模型转换
- 部署模型
## 环境准备
### 安装 Paddle Serving
安装 client 和 serving app，用于向服务发送请求:
```shell
pip install paddle_serving_app paddle_serving_client
```
安装 server，用于启动服务，根据服务器设备选择安装 CPU server 或 GPU server：
- 安装 CPU server
```shell
pip install paddle_serving_server
```
- 安装 GPU server, 注意选择跟本地环境一致的命令
```shell
# CUDA10.2 + Cudnn7 + TensorRT6
pip install paddle-serving-server-gpu==0.8.3.post102 # -i https://pypi.tuna.tsinghua.edu.cn/simple
# CUDA10.1 + TensorRT6
pip install paddle-serving-server-gpu==0.8.3.post101 # -i https://pypi.tuna.tsinghua.edu.cn/simple
# CUDA11.2 + TensorRT8
pip install paddle-serving-server-gpu==0.8.3.post112 # -i https://pypi.tuna.tsinghua.edu.cn/simple
```
**NOTE:**
- 可以开启国内清华镜像源来加速下载
- 如果要安装最新版本的 PaddleServing 参考[链接](https://github.com/PaddlePaddle/Serving/blob/develop/doc/Latest_Packages_CN.md)。
## 模型转换
使用 Paddle Serving 做服务化部署时，需要将保存的 inference 模型转换为 serving 易于部署的模型。
用已安装的 paddle_serving_client 将静态图参数模型转换成 serving 格式。关于如何使用将训练后的动态图模型转为静态图模型详见[FasterTransformer 加速及模型静态图导出](../../README.md)。
模型转换命令如下：
```shell
python -m paddle_serving_client.convert --dirname ./export_checkpoint \
--model_filename unimo_text.pdmodel \
--params_filename unimo_text.pdiparams \
--serving_server ./deploy/paddle_serving/export_checkpoint_server \
--serving_client ./deploy/paddle_serving/export_checkpoint_client
```
关键参数释义如下：
* `dirname`：静态图模型文件夹地址。
* `model_filename`：模型文件名。
* `params_filename`：模型参数名。
* `serving_server`：server 的模型文件和配置文件路径，默认"serving_server"。
* `serving_client`：client 的配置文件路径，默认"serving_client"。
更多参数可通过以下命令查询：
```shell
python -m paddle_serving_client.convert --help
```
模型转换完成后，会在./delopy/paddle_serving 文件夹多出 export_checkpoint_server 和 export_checkpoint_client 的文件夹，文件夹目录格式如下：
```
export_checkpoint_server/
├── unimo_text.pdiparams
├── unimo_text.pdmodel
├── serving_server_conf.prototxt
└── serving_server_conf.stream.prototxt
export_checkpoint_server/
├── serving_client_conf.prototxt
└── serving_client_conf.stream.prototxt
```
## pipeline 部署
paddle_serving 目录包含启动 pipeline 服务和发送预测请求的代码，包括：
```
paddle_serving/
├──config.yml # 启动服务端的配置文件
├──pipeline_client.py # 发送pipeline预测请求的脚本
└──pipeline_service.py # 启动pipeline服务端的脚本
```
### 修改配置文件
目录中的`config.yml`文件解释了每一个参数的含义，可以根据实际需要修改其中的配置。
### server 启动服务
修改好配置文件后，执行下面命令启动服务:
```shell
cd deploy/paddle_serving
# 启动服务，运行日志保存在log.txt
python pipeline_service.py &> log.txt &
```
成功启动服务后，log.txt 中会打印类似如下日志
```
--- Running analysis [ir_graph_to_program_pass]
I0901 12:09:27.248943 12190 analysis_predictor.cc:1035] ======= optimize end =======
I0901 12:09:27.249596 12190 naive_executor.cc:102] --- skip [feed], feed -> seq_len
I0901 12:09:27.249608 12190 naive_executor.cc:102] --- skip [feed], feed -> attention_mask
I0901 12:09:27.249614 12190 naive_executor.cc:102] --- skip [feed], feed -> token_type_ids
I0901 12:09:27.249617 12190 naive_executor.cc:102] --- skip [feed], feed -> input_ids
I0901 12:09:27.250080 12190 naive_executor.cc:102] --- skip [_generated_var_3], fetch -> fetch
I0901 12:09:27.250090 12190 naive_executor.cc:102] --- skip [transpose_0.tmp_0], fetch -> fetch
[2022-09-01 12:09:27,251] [ INFO] - Already cached /root/.paddlenlp/models/unimo-text-1.0/unimo-text-1.0-vocab.txt
[2022-09-01 12:09:27,269] [ INFO] - tokenizer config file saved in /root/.paddlenlp/models/unimo-text-1.0/tokenizer_config.json
[2022-09-01 12:09:27,269] [ INFO] - Special tokens file saved in /root/.paddlenlp/models/unimo-text-1.0/special_tokens_map.json
[PipelineServicer] succ init
[OP Object] init success
2022/09/01 12:09:27 start proxy service
```
### client 发送服务请求
执行以下命令发送文本摘要服务请求：
```shell
cd deploy/paddle_serving
python pipeline_client.py
```
注意执行客户端请求时关闭代理，并根据实际情况修改 server_url 地址(启动服务所在的机器)
成功运行后，输出打印如下:
```
time cost :0.03429532051086426 seconds
--------------------
input: {'context': '平安银行95511电话按9转报案人工服务。 1.寿险 :95511转1 2.信用卡 95511转2 3.平安银行 95511转3 4.一账通 95511转4转8 5.产险 95511转5 6.养老险团体险 95511转6 7.健康险 95511转7 8.证券 95511转8 9.车险报案95511转9 0.重听', 'answer': '95511'}
output: 问题：平安银行人工服务电话
--------------------
```

59
slm/examples/question_generation/unimo-text/deploy/paddle_serving/config.yml
View File

@@ -1,59 +0,0 @@
#rpc端口, rpc_port和http_port不允许同时为空。当rpc_port为空且http_port不为空时，会自动将rpc_port设置为http_port+1
rpc_port: 18011
#http端口, rpc_port和http_port不允许同时为空。当rpc_port可用且http_port为空时，不自动生成http_port
http_port: 9999
#worker_num, 最大并发数。
#当build_dag_each_worker=True时, 框架会创建worker_num个进程，每个进程内构建grpcSever和DAG
#当build_dag_each_worker=False时，框架会设置主线程grpc线程池的max_workers=worker_num
worker_num: 10
#build_dag_each_worker, False，框架在进程内创建一条DAG；True，框架会每个进程内创建多个独立的DAG
build_dag_each_worker: false
dag:
#op资源类型, True, 为线程模型；False，为进程模型
is_thread_op: True
#重试次数
retry: 1
#使用性能分析, True，生成Timeline性能数据，对性能有一定影响；False为不使用
use_profile: false
tracer:
interval_s: 10
op:
question_generation:
#并发数，is_thread_op=True时，为线程并发；否则为进程并发
concurrency: 11
#当op配置没有server_endpoints时，从local_service_conf读取本地服务配置
local_service_conf:
#client类型，包括brpc, grpc和local_predictor.local_predictor不启动Serving服务，进程内预测
client_type: local_predictor
#模型路径
model_config: ../../unimo/serving/export_checkpoint_server
#Fetch结果列表，以client_config中fetch_var的alias_name为准，不设置默认取全部输出变量
# fetch_list: ["_generated_var_3", "slice_0.tmp_0"]
# device_type, 0=cpu, 1=gpu, 2=tensorRT, 3=arm cpu, 4=kunlun xpu
device_type: 1
#计算硬件ID，当devices为""或不写时为CPU预测；当devices为"0", "0,1,2"时为GPU预测，表示使用的GPU卡
devices: "0"
#开启MKLDNN加速
use_mkldnn: False
#thread_num
thread_num: 12
#ir_optim
ir_optim: False
#开启tensorrt后，进行优化的子图包含的最少节点数
#min_subgraph_size: 10

Some files were not shown because too many files have changed in this diff Show More