fix bugs.

* rename files and add readme for llama auto_parallel

* rename files and add readme for llama auto_parallel

* fix ci

.flake8

@@ -0,0 +1,7 @@
+[flake8]
+ignore = E203, E402, E501, E731, E741, W503, W605, E722
+max-line-length = 119
+
+# E402: module level import not at top of file
+per-file-ignores =
+    __init__.py:F401,F403,E402

.github/CODE_OF_CONDUCT.md

@@ -1,3 +1,4 @@
+**简体中文**🀄 | [English🌎](./CODE_OF_CONDUCT_en.md)
 
 # 贡献者公约
 

.github/CODE_OF_CONDUCT_en.md

@@ -1,3 +1,4 @@
+[简体中文🀄](./CODE_OF_CONDUCT.md) |  **English**🌎
 
 # Contributor Covenant Code of Conduct
 

.github/CONTRIBUTING_en.md

@@ -0,0 +1,154 @@
+[简体中文🀄](../CONTRIBUTING.md) |  **English**🌎
+
+# Contributing to PaddleNLP
+
+We highly welcome and value your contributions to `PaddleNLP`. The first step to start your contribution is to sign the [PaddlePaddle Contributor License Agreement](https://cla-assistant.io/PaddlePaddle/PaddleNLP).
+
+This document explains our workflow and work style:
+
+## Finding out what to work on Workflow
+
+## Development Workflow
+
+PaddleNLP uses the [Git branching model](http://nvie.com/posts/a-successful-git-branching-model/).  The following steps guide usual contributions.
+
+#### 1. Fork
+
+   Our development community has been growing fastly; it doesn't make sense for everyone to write into the official repo.  So, please file Pull Requests from your fork.  To make a fork,  just head over to the GitHub page and click the ["Fork" button](https://help.github.com/articles/fork-a-repo/).
+
+#### 2. Clone
+
+   To make a copy of your fork to your local computers, please run
+
+   ```bash
+   git clone https://github.com/<your-github-account>/PaddleNLP
+   cd PaddleNLP
+   ```
+
+#### 3. Create the local feature branch
+
+   For daily works like adding a new feature or fixing a bug, please open your feature branch before coding:
+
+   ```bash
+   git checkout -b my-cool-feature
+   ```
+
+#### 4. Set up the development environment
+
+   Before you start coding, you need to setup the development environment. We highly recommend doing all your development in a virtual environment such as
+   [venv](https://docs.python.org/3/library/venv.html) or [conda](https://docs.conda.io/en/latest/). After you setup and activated your virtual environment,
+   run the following command:
+
+   ```bash
+   make install
+   ```
+
+   This will setup all the dependencies of `PaddleNLP` as well as the [`pre-commit`](http://pre-commit.com/) tool.
+
+   If you are working on the `examples` or `applications` module and require importing from `PaddleNLP`, make sure you install `PaddleNLP` in editable mode.
+   If `PaddleNLP` is already installed in the virtual environment, remove it with `pip uninstall paddlenlp` before reinstalling it in editable mode with
+   `pip install -e .`
+
+#### 5. Develop
+
+   As you develop your new exciting feature, keep in mind that it should be covered by unit tests. All of our unit tests can be found under the `tests` directory.
+   You can either modify existing unit test to cover the new feature, or create a new test from scratch.
+   As you finish up the your code, you should make sure the test suite passes. You can run the tests impacted by your changes like this:
+
+   ```bash
+   pytest tests/<test_to_run>.py
+   ```
+
+#### 6. Commit
+
+   We utilizes [`pre-commit`](http://pre-commit.com/) (with [black](https://black.readthedocs.io/en/stable/), [isort](https://pycqa.github.io/isort/) and
+   [flake8](https://flake8.pycqa.org/en/latest/) under the hood) to check the style of code and documentation in every commit. When you run run `git commit`, you will see
+   something like the following:
+
+   ```
+    ➜  (my-virtual-env) git commit -m "commiting my cool feature"
+    black....................................................................Passed
+    isort....................................................................Passed
+    flake8...................................................................Passed
+    check for merge conflicts................................................Passed
+    check for broken symlinks............................(no files to check)Skipped
+    detect private key.......................................................Passed
+    fix end of files.....................................(no files to check)Skipped
+    trim trailing whitespace.............................(no files to check)Skipped
+    CRLF end-lines checker...............................(no files to check)Skipped
+    CRLF end-lines remover...............................(no files to check)Skipped
+    No-tabs checker......................................(no files to check)Skipped
+    Tabs remover.........................................(no files to check)Skipped
+    copyright_checker........................................................Passed
+   ```
+
+   But most of the time things don't go so smoothly. When your code or documentation doesn't meet the standard, the `pre-commit` check will fail.
+   ```
+    ➜  (my-virtual-env) git commit -m "commiting my cool feature"
+    black....................................................................Passed
+    isort....................................................................Failed
+    - hook id: isort
+    - files were modified by this hook
+
+    Fixing examples/information_extraction/waybill_ie/run_ernie_crf.py
+
+    flake8...................................................................Passed
+    check for merge conflicts................................................Passed
+    check for broken symlinks............................(no files to check)Skipped
+    detect private key.......................................................Passed
+    fix end of files.....................................(no files to check)Skipped
+    trim trailing whitespace.............................(no files to check)Skipped
+    CRLF end-lines checker...............................(no files to check)Skipped
+    CRLF end-lines remover...............................(no files to check)Skipped
+    No-tabs checker......................................(no files to check)Skipped
+    Tabs remover.........................................(no files to check)Skipped
+    copyright_checker........................................................Passed
+   ```
+
+   But **don't panic**!
+   Our tooling will fix most of the style errors automatically. Some errors will need to be addressed manually. Fortunately, the error messages are straight forward and
+   the errors are usually simple to fix. After addressing the errors, you can run `git add <files>` and `git commit` again, which will trigger `pre-commit` again.
+   Once the `pre-commit` checks pass, you are ready to push the code.
+
+   [Google][http://google.com/] or [StackOverflow](https://stackoverflow.com/) are great tools to help you understand the code style errors.
+   Don't worry if you still can't figure it out. You can commit with `git commit -m "style error" --no-verify` and we are happy to help you once you create a Pull Request.
+
+#### 7. Keep pulling
+
+   An experienced Git user pulls from the official repo often -- daily or even hourly, so they notice conflicts with others work early, and it's easier to resolve smaller conflicts.
+
+   ```bash
+   git remote add upstream https://github.com/PaddlePaddle/PaddleNLP
+   git pull upstream develop
+   ```
+
+#### 8. Push and file a pull request
+
+   You can "push" your local work into your forked repo:
+
+   ```bash
+   git push origin my-cool-stuff
+   ```
+
+   The push allows you to create a pull request, requesting owners of this [official repo](https://github.com/PaddlePaddle/PaddleNLP) to pull your change into the official one.
+
+   To create a pull request, please follow [these steps](https://help.github.com/articles/creating-a-pull-request/).
+
+#### 9. Delete local and remote branches
+
+   To keep your local workspace and your fork clean, you might want to remove merged branches:
+
+   ```bash
+   git push origin my-cool-stuff
+   git checkout develop
+   git pull upstream develop
+   git branch -d my-cool-stuff
+   ```
+
+## Code Review
+
+-  Please feel free to ping your reviewers by @-mentioning the in the Pull Request.  Please do this after your pull request passes the CI.
+
+- Please answer reviewers' every comment.  If you are to follow the comment, please write "Done"; Otherwise, please start a discussion under the comment.
+
+- If you don't want your reviewers to get overwhelmed by email notifications, you might reply their comments by [in a batch](https://help.github.com/articles/reviewing-proposed-changes-in-a-pull-request/).

.github/codecov.yml

@@ -0,0 +1,13 @@
+codecov:
+  notify:
+    wait_for_ci: false
+
+coverage:
+  status:
+    project:
+      default: 
+        target: 58% # overall project Coverage 
+        threshold: 1% # Allow the coverage to drop by 1%, and posting a success status.
+    patch:
+      default: 
+        target: 80% # lines adjusted  Coverage < 80%  CI will fail

.github/workflows/fast_tokenizer.yml

@@ -0,0 +1,48 @@
+name: FastTokenizer
+
+on:
+  push:
+    paths:
+      - 'fast_tokenizer/*'
+  pull_request:
+    paths:
+      - 'fast_tokenizer/*'
+
+jobs:
+  fast_tokenizer_cpp:
+    name: fast_tokenizer_cpp
+    runs-on: ubuntu-22.04
+    permissions:
+      pull-requests: write
+      contents: read
+      id-token: write
+    steps:
+      - uses: actions/checkout@v3
+      - name: compile
+        working-directory: ./fast_tokenizer
+        run: make fast_tokenizer_cpp_compile
+      - name: test
+        working-directory: ./fast_tokenizer
+        run: make fast_tokenizer_cpp_test
+  fast_tokenizer_python38:
+      name: fast_tokenizer_python38
+      runs-on: ubuntu-22.04
+      permissions:
+        pull-requests: write
+        contents: read
+        id-token: write
+      steps:
+        - uses: actions/checkout@v3
+        - uses: actions/setup-python@v1
+          with:
+            python-version: 3.8
+        - name: install
+          working-directory: ./fast_tokenizer
+          run: make fast_tokenizer_python_install
+        - name: compile
+          working-directory: ./fast_tokenizer
+          run: make fast_tokenizer_python_compile
+        - name: test
+          working-directory: ./fast_tokenizer
+          run: make fast_tokenizer_python_test
+  

.github/workflows/lint.yml

@@ -0,0 +1,32 @@
+name: Lint
+
+on: [push, pull_request]
+
+jobs:
+  Lint:
+    name: Lint
+    runs-on: ubuntu-20.04
+    steps:
+      - uses: actions/checkout@v3
+        with:
+          fetch-depth: 0
+      - name: checkout develop
+        run: |
+          if ! git show-ref --quiet refs/heads/develop; then \
+              echo "local develop branch is missing, creating local develop branch that tracks remote develop branch"
+              git fetch origin develop
+              git branch develop --track origin/develop
+          else
+            echo "local develop branch exist, skipping"
+          fi
+      - uses: actions/setup-python@v4
+        with:
+          python-version: 3.8
+          cache: 'pip' # caching pip dependencies
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip
+          make install
+      - name: run the command
+        run: make lint
+    

.github/workflows/pipelines.yml

@@ -0,0 +1,30 @@
+name: Pipelines-Test
+
+on:
+  push:
+    paths:
+      - 'pipelines/*'
+  pull_request:
+    paths:
+      - 'pipelines/*'
+
+
+jobs:
+  Pipelines-Test:
+    name: Pipelines-Test
+    runs-on: ubuntu-20.04
+    steps:
+      - uses: actions/checkout@v3
+      - uses: actions/setup-python@v4
+        with:
+          python-version: 3.8
+          cache: 'pip' # caching pip dependencies
+      - name: Install dependencies
+        working-directory: ./pipelines
+        run: |
+          python -m pip install --upgrade pip
+          make install
+          pip install -r tests/requirements.txt
+      - name: run the command
+        working-directory: ./pipelines
+        run: make test

.github/workflows/pypi.yml

@@ -1,60 +0,0 @@
-name: PyPI
-
-on: [push, pull_request]
-
-jobs:
-  build:
-    name: Build
-    runs-on: ubuntu-latest
-    steps:
-      - uses: actions/checkout@v2
-      - uses: actions/setup-python@v1
-        with:
-          python-version: 3.8
-      - name: Install dependencies
-        run: |
-          python -m pip install --upgrade pip
-          pip install -r requirements.txt
-          pip install -r docs/requirements.txt
-          pip install -r ppdiffusers/requirements.txt
-
-  ppdiffusers:
-    name: Pack
-    needs: build
-    runs-on: ubuntu-latest
-    steps:
-      - uses: actions/checkout@v2
-      - uses: actions/setup-python@v1
-        with:
-          python-version: 3.8
-      - name: Install dependencies
-        run: |
-          python -m pip install --upgrade pip
-          pip install setuptools wheel twine
-          make install-ppdiffusers
-
-      - name: Should Deploy to Pypi Server
-        id: should-deploy
-        run: python scripts/should_deploy.py  --name ppdiffusers >> $GITHUB_OUTPUT
-      - name: Check Branch
-        id: check-branch
-        run: |
-          if [[ ${{ github.ref }} =~ ^refs/heads/(develop)$ ]]; then
-              echo "match=true" >> $GITHUB_OUTPUT
-          fi  # See: https://stackoverflow.com/a/58869470/1123955
-
-      - name: Is A Publish Branch
-        if: steps.check-branch.outputs.match == 'true' && steps.should-deploy.outputs.should_deploy == 'true'
-        env:
-          TWINE_USERNAME: paddle-dev
-          TWINE_PASSWORD: ${{ secrets.paddlenlp }}
-        run: |
-          make deploy-ppdiffusers
-
-      - name: Should Not Deploy To Pypi Server
-        if: steps.should-deploy.outputs.should_deploy != 'true'
-        run: echo 'should not deploy pypi server'
-
-      - name: Is Not A Publish Branch
-        if: steps.check-branch.outputs.match != 'true'
-        run: echo 'Not A Publish Branch'

.github/workflows/release.yml

@@ -0,0 +1,58 @@
+name: Release
+
+on: 
+  workflow_dispatch:
+    inputs:
+      package:
+        description: 'Package to be deployed'
+        required: true
+        default: 'paddlenlp'
+        type: choice
+        options:
+        - paddlenlp
+        - paddle-pipelines
+        - ppdiffusers
+
+jobs:
+  Release:
+    name: Release
+    runs-on: ubuntu-20.04
+    steps:
+      - uses: actions/checkout@v2
+      - uses: actions/setup-python@v1
+        with:
+          python-version: 3.8
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip
+          pip install setuptools wheel twine
+      - name: Should Deploy
+        id: should-deploy
+        env: 
+          PACKAGE: ${{ inputs.package }}
+        run: python scripts/should_deploy.py  --name $PACKAGE >> $GITHUB_OUTPUT
+
+      - name: Check Branch
+        id: check-branch
+        run: |
+          if [[ ${{ github.ref }} =~ ^refs/tags/v[0-9]+\.[0-9]+\.[0-9]+$ ]]; then
+              echo "match=true" >> $GITHUB_OUTPUT
+          fi  # See: https://stackoverflow.com/a/58869470/1123955
+
+      - name: Is A Publish Branch
+        if: steps.check-branch.outputs.match == 'true' && steps.should-deploy.outputs.should_deploy == 'true'
+        env:
+          TWINE_USERNAME: __token__
+          TWINE_PASSWORD: ${{ secrets.paddlenlp }}
+          PACKAGE: ${{ inputs.package }}
+          PADDLENLP_STABLE_VERSION: "true"
+        run: |
+          make deploy-$PACKAGE
+
+      - name: Should Not Deploy To Pypi Server
+        if: steps.should-deploy.outputs.should_deploy != 'true'
+        run: echo 'should not deploy pypi server'
+
+      - name: Is Not A Publish Branch
+        if: steps.check-branch.outputs.match != 'true'
+        run: echo 'Not A Publish Branch'

.github/workflows/stale.yml

@@ -0,0 +1,35 @@
+name: Stale
+
+on:
+  # Allow manual run via GitHub web or CLI
+  workflow_dispatch:
+  schedule:
+    # Run daily at midnight UTC
+    - cron: 0 0 * * *
+
+permissions:
+  issues: write
+  pull-requests: write
+
+jobs:
+  stale:
+    runs-on: ubuntu-20.04
+
+    permissions:
+      issues: write
+      pull-requests: write
+
+    steps:
+      - uses: actions/stale@v6.0.1
+        with:
+          days-before-issue-stale: 60
+          days-before-issue-close: 14
+          stale-issue-label: "stale"
+          stale-issue-message: "This issue is stale because it has been open for 60 days with no activity. 当前issue 60天内无活动，被标记为stale。"
+          close-issue-message: "This issue was closed because it has been inactive for 14 days since being marked as stale. 当前issue 被标记为stale已有14天，即将关闭。"
+          exempt-issue-labels: 'triage,keep'
+          days-before-pr-stale: 60
+          days-before-pr-close: -1
+          stale-pr-label: "stale"
+          stale-pr-message: "This Pull Request is stale because it has been open for 60 days with no activity. 当前Pull Request 60天内无活动，被标记为stale。"
+          operations-per-run: 400

.github/workflows/tests.yml

@@ -0,0 +1,27 @@
+name: Test
+
+on: [push, pull_request]
+
+jobs:
+  Test:
+    name: Test
+    runs-on: ubuntu-20.04
+    permissions:
+      pull-requests: write
+      contents: read
+      id-token: write
+    steps:
+      - uses: actions/checkout@v3
+      - uses: actions/setup-python@v4
+        with:
+          python-version: 3.8
+          cache: 'pip' # caching pip dependencies
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip
+          pip install -r tests/requirements.txt
+          make install
+      - name: run the command
+        run: make test
+      - name: Upload coverage to Codecov
+        uses: codecov/codecov-action@v3

.gitignore

@@ -123,4 +123,4 @@ FETCH_HEAD
 
 # vscode
 .vscode
-./ppdiffusers/ppdiffusers/version.py
+./ppdiffusers/ppdiffusers/version.py

.pre-commit-config.yaml

@@ -1,9 +1,20 @@
+exclude: 'model_zoo/gpt-3'
 repos:
--   repo: https://github.com/google/yapf
-    rev: v0.32.0
+# For Python files
+-   repo: https://github.com/psf/black.git
+    rev: 22.8.0
     hooks:
-    -   id: yapf
-        files: \.py$
+    -   id: black
+        files: \.(py|pyi)$
+        additional_dependencies: [toml]
+-   repo: https://github.com/PyCQA/isort
+    rev: 5.11.5
+    hooks:
+    -   id: isort
+-   repo: https://github.com/PyCQA/flake8
+    rev: 4.0.1
+    hooks:
+    -   id: flake8
 -   repo: https://github.com/pre-commit/pre-commit-hooks
     rev: v4.1.0
     hooks:

.readthedocs.yaml

@@ -4,6 +4,10 @@
 
 # Required
 version: 2
+build:
+  os: "ubuntu-20.04"
+  tools:
+    python: "3.8"
 
 submodules:
   include: all
@@ -19,10 +23,5 @@ sphinx:
 
 # Optionally set the version of Python and requirements required to build your docs
 python:
-  version: 3.8
   install:
     - requirements: docs/requirements.txt
-    - requirements: requirements.txt
-    - method: setuptools
-      path: .
-  system_packages: true

.style.yapf

@@ -1,3 +0,0 @@
-[style]
-based_on_style = pep8
-column_limit = 80

CONTRIBUTING.md

@@ -0,0 +1,157 @@
+**简体中文**🀄 | [English🌎](.github/CONTRIBUTING_en.md)
+
+# Contributing to PaddleNLP
+
+我们非常欢迎并希望您对`PaddleNLP`做出开源贡献。在您开始提交您的贡献之前，请先行签署[PaddlePaddle 贡献者许可协议](https://cla-assistant.io/PaddlePaddle/PaddleNLP)。
+本文接下来将介绍我们的开发与贡献流程：
+
+## 贡献方式
+
+我们欢迎不同的向`PaddleNLP`做出贡献的方式，例如：
+
+- 修复已知的Issue
+- 提交新的Issue，例如提出功能需求或者bug报告
+- 实现新的模型结构
+
+如果您不知道从哪里开始，请查看Issues板块中的`Good First Issue`标签。它为您提供一个对初学者友好的已知Issue列表，可以降低贡献的门槛，帮助您开始为开源做出贡献。您只需在您想处理的Issue中告知我们您想负责此Issue即可。
+
+## 开发流程
+
+PaddleNLP 使用 [Git 分支模型](http://nvie.com/posts/a-successful-git-branching-model/)。对于常见的开源贡献，我们有以下的贡献流程：
+
+#### 1. Fork
+
+   因为PaddleNLP的开发社区一直在发展，如果每位贡献者都直接向官方Repo提交commit将会难以管理。因此，请从您的分支中提交 Pull Requests。建议您通过GitHub的[“Fork”按钮](https://help.github.com/articles/fork-a-repo/)来创建您的Fork分支。
+
+#### 2. Clone
+
+   请运行一下命令将您的分支clone到本地
+
+   ```bash
+   git clone https://github.com/<your-github-account>/PaddleNLP
+   cd PaddleNLP
+   ```
+
+#### 3. 创建本地开发分支
+
+   对于添加新功能或修复错误等日常工作，请在开发前创建您的本地开发分支：
+
+   ```bash
+   git checkout -b my-cool-feature
+   ```
+
+#### 4. 配置开发环境
+   在开始编码之前，您需要设置开发环境。我们强烈建议您在虚拟环境中进行所有开发，例如[venv](https://docs.python.org/3/library/venv.html)或[conda](https://docs.conda.io/en/latest/)。
+   请您设置并激活虚拟环境后，运行以下命令：
+
+   ```bash
+   make install
+   ```
+
+   这将设置 `PaddleNLP` 的所有依赖以及 [`pre-commit`](http://pre-commit.com/) 工具。
+
+   如果您需要开发 `examples` 或 `applications` 模块并加载 `PaddleNLP`，请确保以可编辑模式（`-e`）安装 `PaddleNLP`。
+   如果在虚拟环境中已经安装 `PaddleNLP` ，请使用 `pip uninstall paddlenlp` 将其删除，然后以可编辑模式重新安装它
+   `pip install -e .`
+
+
+#### 5. 开发
+
+   当您开发时，请确保您新增的代码会被单元测试所覆盖。我们所有的单元测试都可以在 `tests` 目录下找到。
+   您可以修改现有单元测试以覆盖新功能，也可以从头开始创建新测试。
+   当您完成代码时，您应该确保相关的单元测试可以通过。您可以像这样运行受更改影响的测试：
+
+   ```bash
+   pytest tests/<test_to_run>.py
+   ```
+
+#### 6. Commit
+
+   我们使用 [`pre-commit`](http://pre-commit.com/)工具（包括[black](https://black.readthedocs.io/en/stable/)、[isort](https:/ /pycqa.github.io/isort/) 和
+   [flake8](https://flake8.pycqa.org/en/latest/)）来检查每次提交中的代码和文档的风格。当你运行 `git commit` 时，你会看到
+   类似于以下内容：
+
+   ```
+    ➜  (my-virtual-env) git commit -m "commiting my cool feature"
+    black....................................................................Passed
+    isort....................................................................Passed
+    flake8...................................................................Passed
+    check for merge conflicts................................................Passed
+    check for broken symlinks............................(no files to check)Skipped
+    detect private key.......................................................Passed
+    fix end of files.....................................(no files to check)Skipped
+    trim trailing whitespace.............................(no files to check)Skipped
+    CRLF end-lines checker...............................(no files to check)Skipped
+    CRLF end-lines remover...............................(no files to check)Skipped
+    No-tabs checker......................................(no files to check)Skipped
+    Tabs remover.........................................(no files to check)Skipped
+    copyright_checker........................................................Passed
+   ```
+
+   但大多数时候事情并没有那么顺利。当您的代码或文档不符合标准时，`pre-commit` 检查将失败。
+   ```
+    ➜  (my-virtual-env) git commit -m "commiting my cool feature"
+    black....................................................................Passed
+    isort....................................................................Failed
+    - hook id: isort
+    - files were modified by this hook
+
+    Fixing examples/information_extraction/waybill_ie/run_ernie_crf.py
+
+    flake8...................................................................Passed
+    check for merge conflicts................................................Passed
+    check for broken symlinks............................(no files to check)Skipped
+    detect private key.......................................................Passed
+    fix end of files.....................................(no files to check)Skipped
+    trim trailing whitespace.............................(no files to check)Skipped
+    CRLF end-lines checker...............................(no files to check)Skipped
+    CRLF end-lines remover...............................(no files to check)Skipped
+    No-tabs checker......................................(no files to check)Skipped
+    Tabs remover.........................................(no files to check)Skipped
+    copyright_checker........................................................Passed
+   ```
+
+   我们的工具将自动修复大部分样式错误，但是有些错误需要手动解决。幸运的是，错误信息一般通俗易懂，很容易修复。
+   解决错误后，您可以再次运行 git add <files> 和 git commit ，这将再次触发 pre-commit 。
+   一旦 pre-commit 检查通过，您就可以推送代码了。
+
+   [Google][http://google.com/] 或 [StackOverflow](https://stackoverflow.com/) 是帮助您了解代码风格错误的好工具。
+   如果您仍然无法弄清楚，请不要担心。您可以使用 `git commit -m "style error" --no-verify` 提交，我们很乐意在您创建 Pull Request 后帮助您。
+
+#### 7. git pull与代码冲突
+
+   有经验的 Git 用户经常从官方Repo中git pull。因为这样子他们会及早注意到与其他人的代码冲突，并且让代码冲突更容易解决
+
+   ```bash
+   git remote add upstream https://github.com/PaddlePaddle/PaddleNLP
+   git pull upstream develop
+   ```
+
+#### 8. git push与提交Pull Request
+
+   您可以将您的本地开发分支中的工作 push 到您的fork的分支中：
+
+   ```bash
+   git push origin my-cool-stuff
+   ```
+
+   git push之后，您可以提交Pull Request，请求[官方repo](https://github.com/PaddlePaddle/PaddleNLP) 采纳您的开发工作。请您依照[这些步骤](https://help.github.com/articles/creating-a-pull-request/)创建Pull Request。
+
+#### 9. 删除已经合入的本地和远程分支
+
+   为了保持您本地的工作区和fork分支的干净整洁，建议您在Pull Request合入之后删除本地的残余分支：
+
+   ```bash
+   git push origin my-cool-stuff
+   git checkout develop
+   git pull upstream develop
+   git branch -d my-cool-stuff
+   ```
+
+## 代码Review
+
+- 在您的Pull Request能够顺利通过本地测试以及CI的情况下，您可以在Pull Request中 @ 相关的Reviewer，提醒他们尽快对您的Pull Request进行Review。
+
+- 请处理Reviewer的每一条评论。如果您已按照评论修改，请回复“完成”；否则，可以在评论下展开讨论。
+
+- 如果您不希望您的Reviewer被电子邮件通知淹没，您可以[批量回复](https://help.github.com/articles/reviewing-proposed-changes-in-a-pull-request/)。

Makefile

@@ -1,10 +1,77 @@
+# Makefile for PaddleNLP
+#
+# 	GitHb: https://github.com/PaddlePaddle/PaddleNLP
+# 	Author: Paddle Team https://github.com/PaddlePaddle
+#
+
+.PHONY: all
+all : lint test
+check_dirs := applications examples model_zoo paddlenlp pipelines ppdiffusers scripts tests 
+# # # # # # # # # # # # # # # Format Block # # # # # # # # # # # # # # # 
+
+format:
+	pre-commit run isort
+	pre-commit run black
+
+# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
+
+# # # # # # # # # # # # # # # Lint Block # # # # # # # # # # # # # # # 
+
+.PHONY: lint
+lint:
+	$(eval modified_py_files := $(shell python scripts/get_modified_files.py $(check_dirs)))
+	@if test -n "$(modified_py_files)"; then \
+		echo ${modified_py_files}; \
+		pre-commit run --files ${modified_py_files}; \
+	else \
+		echo "No library .py files were modified"; \
+	fi	
+
+# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
+
+# # # # # # # # # # # # # # # Test Block # # # # # # # # # # # # # # # 
+
+.PHONY: test
+test: unit-test
+
+unit-test:
+	PYTHONPATH=$(shell pwd) pytest -v \
+		-n auto \
+		--durations 20 \
+		--cov paddlenlp \
+		--cov-report xml:coverage.xml
+
+# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
+
+.PHONY: install
+install:
+	pip install -r requirements-dev.txt
+	pip install -r requirements.txt
+	pip install -r paddlenlp/experimental/autonlp/requirements.txt
+	pre-commit install
 
 
 .PHONY: deploy-ppdiffusers
 deploy-ppdiffusers:
-	cd ppdiffusers && make
+	cd ppdiffusers && make install && make
 
-.PHONY: install-ppdiffusers
-install-ppdiffusers:
-	cd ppdiffusers && make install
+.PHONY: deploy-paddle-pipelines
+deploy-paddle-pipelines:
+	cd pipelines && make install && make
 
+.PHONY: deploy-paddlenlp
+deploy-paddlenlp:
+	# install related package
+	make install
+	# build
+	python3 setup.py sdist bdist_wheel
+	# upload
+	twine upload --skip-existing dist/*
+
+.PHONY: regression-all
+release: 
+	bash ./scripts/regression/run_release.sh 0 0,1 all
+
+.PHONY: regression-key
+key: 
+	bash ./scripts/regression/run_release.sh 0 0,1 p0

README.md

@@ -1 +0,0 @@
-README_cn.md

README.md

@@ -0,0 +1,347 @@
+**简体中文**🀄 | [English🌎](./README_en.md)
+
+<p align="center">
+  <img src="https://user-images.githubusercontent.com/1371212/175816733-8ec25eb0-9af3-4380-9218-27c154518258.png" align="middle"  width="500" />
+</p>
+
+------------------------------------------------------------------------------------------
+
+<p align="center">
+    <a href="./LICENSE"><img src="https://img.shields.io/badge/license-Apache%202-dfd.svg"></a>
+    <a href="https://github.com/PaddlePaddle/PaddleNLP/releases"><img src="https://img.shields.io/github/v/release/PaddlePaddle/PaddleNLP?color=ffa"></a>
+    <a href=""><img src="https://img.shields.io/badge/python-3.7+-aff.svg"></a>
+    <a href=""><img src="https://img.shields.io/badge/os-linux%2C%20win%2C%20mac-pink.svg"></a>
+    <a href="https://github.com/PaddlePaddle/PaddleNLP/graphs/contributors"><img src="https://img.shields.io/github/contributors/PaddlePaddle/PaddleNLP?color=9ea"></a>
+    <a href="https://github.com/PaddlePaddle/PaddleNLP/commits"><img src="https://img.shields.io/github/commit-activity/m/PaddlePaddle/PaddleNLP?color=3af"></a>
+    <a href="https://pypi.org/project/paddlenlp/"><img src="https://img.shields.io/pypi/dm/paddlenlp?color=9cf"></a>
+    <a href="https://github.com/PaddlePaddle/PaddleNLP/issues"><img src="https://img.shields.io/github/issues/PaddlePaddle/PaddleNLP?color=9cc"></a>
+    <a href="https://github.com/PaddlePaddle/PaddleNLP/stargazers"><img src="https://img.shields.io/github/stars/PaddlePaddle/PaddleNLP?color=ccf"></a>
+</p>
+
+
+<h4 align="center">
+  <a href=#安装> 安装 </a> |
+  <a href=#快速开始> 快速开始 </a> |
+  <a href=#特性> 特性 </a> |
+  <a href=#社区交流> 社区交流 </a>
+</h4>
+
+**PaddleNLP**是一款**简单易用**且**功能强大**的自然语言处理和大语言模型(LLM)开发库。聚合业界**优质预训练模型**并提供**开箱即用**的开发体验，覆盖NLP多场景的模型库搭配**产业实践范例**可满足开发者**灵活定制**的需求。
+
+## News 📢
+
+* **2024.01.04 [PaddleNLP v2.7](https://github.com/PaddlePaddle/PaddleNLP/releases/tag/v2.7.1)**： 大模型体验全面升级，统一工具链大模型入口。统一预训练、精调、压缩、推理以及部署等环节的实现代码，到 `PaddleNLP/llm`目录。全新大[模型工具链文档](https://paddlenlp.readthedocs.io/zh/latest/llm/finetune.html)，一站式指引用户从大模型入门到业务部署上线。全断点存储机制 Unified Checkpoint，大大提高大模型存储的通用性。高效微调升级，支持了高效微调+LoRA同时使用，支持了QLoRA等算法。
+
+* **2023.08.15 [PaddleNLP v2.6](https://github.com/PaddlePaddle/PaddleNLP/releases/tag/v2.6.0)**： 发布[全流程大模型工具链](./llm)，涵盖预训练，精调，压缩，推理以及部署等各个环节，为用户提供端到端的大模型方案和一站式的开发体验；内置[4D并行分布式Trainer](./docs/trainer.md)，[高效微调算法LoRA/Prefix Tuning](./llm#33-lora), [自研INT8/INT4量化算法](./llm#6-量化)等等；全面支持[LLaMA 1/2](./llm/llama), [BLOOM](.llm/bloom), [ChatGLM 1/2](./llm/chatglm), [GLM](./llm/glm), [OPT](./llm/opt)等主流大模型
+
+
+## 安装
+
+### 环境依赖
+
+- python >= 3.7
+- paddlepaddle >= 2.6.0
+- 如需大模型功能，请使用 paddlepaddle-gpu >= 2.6.0
+
+### pip安装
+
+```shell
+pip install --upgrade paddlenlp
+```
+
+或者可通过以下命令安装最新 develop 分支代码：
+
+```shell
+pip install --pre --upgrade paddlenlp -f https://www.paddlepaddle.org.cn/whl/paddlenlp.html
+```
+
+更多关于PaddlePaddle和PaddleNLP安装的详细教程请查看[Installation](./docs/get_started/installation.rst)。
+
+## 快速开始
+
+
+### 大模型文本生成
+
+PaddleNLP提供了方便易用的Auto API，能够快速的加载模型和Tokenizer。这里以使用 `linly-ai/chinese-llama-2-7b` 大模型做文本生成为例：
+
+```python
+>>> from paddlenlp.transformers import AutoTokenizer, AutoModelForCausalLM
+>>> tokenizer = AutoTokenizer.from_pretrained("linly-ai/chinese-llama-2-7b")
+>>> model = AutoModelForCausalLM.from_pretrained("linly-ai/chinese-llama-2-7b", dtype="float16")
+>>> input_features = tokenizer("你好！请自我介绍一下。", return_tensors="pd")
+>>> outputs = model.generate(**input_features, max_length=128)
+>>> tokenizer.batch_decode(outputs[0])
+['\n你好！我是一个AI语言模型，可以回答你的问题和提供帮助。']
+```
+
+### 一键UIE预测
+
+PaddleNLP提供[一键预测功能](./docs/model_zoo/taskflow.md)，无需训练，直接输入数据即可开放域抽取结果。这里以信息抽取-命名实体识别任务，UIE模型为例：
+
+```python
+>>> from pprint import pprint
+>>> from paddlenlp import Taskflow
+
+>>> schema = ['时间', '选手', '赛事名称'] # Define the schema for entity extraction
+>>> ie = Taskflow('information_extraction', schema=schema)
+>>> pprint(ie("2月8日上午北京冬奥会自由式滑雪女子大跳台决赛中中国选手谷爱凌以188.25分获得金牌！"))
+[{'时间': [{'end': 6,
+          'probability': 0.9857378532924486,
+          'start': 0,
+          'text': '2月8日上午'}],
+  '赛事名称': [{'end': 23,
+            'probability': 0.8503089953268272,
+            'start': 6,
+            'text': '北京冬奥会自由式滑雪女子大跳台决赛'}],
+  '选手': [{'end': 31,
+          'probability': 0.8981548639781138,
+          'start': 28,
+          'text': '谷爱凌'}]}]
+```
+
+更多PaddleNLP内容可参考：
+- [大模型全流程工具链](./llm)，包含主流中文大模型的全流程方案。
+- [精选模型库](./model_zoo)，包含优质预训练模型的端到端全流程使用。
+- [多场景示例](./examples)，了解如何使用PaddleNLP解决NLP多种技术问题，包含基础技术、系统应用与拓展应用。
+- [交互式教程](https://aistudio.baidu.com/aistudio/personalcenter/thirdview/574995)，在🆓免费算力平台AI Studio上快速学习PaddleNLP。
+
+
+## 特性
+
+#### <a href=#开箱即用的nlp工具集> 📦 开箱即用的NLP工具集 </a>
+
+#### <a href=#丰富完备的中文模型库> 🤗 丰富完备的中文模型库 </a>
+
+#### <a href=#产业级端到端系统范例> 🎛️ 产业级端到端系统范例 </a>
+
+#### <a href=#高性能分布式训练与推理> 🚀 高性能分布式训练与推理 </a>
+
+
+### 开箱即用的NLP工具集
+
+Taskflow提供丰富的**📦开箱即用**的产业级NLP预置模型，覆盖自然语言理解与生成两大场景，提供**💪产业级的效果**与**⚡️极致的推理性能**。
+
+![taskflow1](https://user-images.githubusercontent.com/11793384/159693816-fda35221-9751-43bb-b05c-7fc77571dd76.gif)
+
+更多使用方法可参考[Taskflow文档](./docs/model_zoo/taskflow.md)。
+### 丰富完备的中文模型库
+
+#### 🀄 业界最全的中文预训练模型
+
+精选 45+ 个网络结构和 500+ 个预训练模型参数，涵盖业界最全的中文预训练模型：既包括文心NLP大模型的ERNIE、PLATO等，也覆盖BERT、GPT、RoBERTa、T5等主流结构。通过`AutoModel` API一键⚡**高速下载**⚡。
+
+```python
+from paddlenlp.transformers import *
+
+ernie = AutoModel.from_pretrained('ernie-3.0-medium-zh')
+bert = AutoModel.from_pretrained('bert-wwm-chinese')
+albert = AutoModel.from_pretrained('albert-chinese-tiny')
+roberta = AutoModel.from_pretrained('roberta-wwm-ext')
+electra = AutoModel.from_pretrained('chinese-electra-small')
+gpt = AutoModelForPretraining.from_pretrained('gpt-cpm-large-cn')
+```
+
+针对预训练模型计算瓶颈，可以使用API一键使用文心ERNIE-Tiny全系列轻量化模型，降低预训练模型部署难度。
+
+```python
+# 6L768H
+ernie = AutoModel.from_pretrained('ernie-3.0-medium-zh')
+# 6L384H
+ernie = AutoModel.from_pretrained('ernie-3.0-mini-zh')
+# 4L384H
+ernie = AutoModel.from_pretrained('ernie-3.0-micro-zh')
+# 4L312H
+ernie = AutoModel.from_pretrained('ernie-3.0-nano-zh')
+```
+
+对预训练模型应用范式如语义表示、文本分类、句对匹配、序列标注、问答等，提供统一的API体验。
+
+```python
+import paddle
+from paddlenlp.transformers import *
+
+tokenizer = AutoTokenizer.from_pretrained('ernie-3.0-medium-zh')
+text = tokenizer('自然语言处理')
+
+# 语义表示
+model = AutoModel.from_pretrained('ernie-3.0-medium-zh')
+sequence_output, pooled_output = model(input_ids=paddle.to_tensor([text['input_ids']]))
+# 文本分类 & 句对匹配
+model = AutoModelForSequenceClassification.from_pretrained('ernie-3.0-medium-zh')
+# 序列标注
+model = AutoModelForTokenClassification.from_pretrained('ernie-3.0-medium-zh')
+# 问答
+model = AutoModelForQuestionAnswering.from_pretrained('ernie-3.0-medium-zh')
+```
+
+#### 💯 全场景覆盖的应用示例
+
+覆盖从学术到产业的NLP应用示例，涵盖NLP基础技术、NLP系统应用以及拓展应用。全面基于飞桨核心框架2.0全新API体系开发，为开发者提供飞桨文本领域的最佳实践。
+
+精选预训练模型示例可参考[Model Zoo](./model_zoo)，更多场景示例文档可参考[examples目录](./examples)。更有免费算力支持的[AI Studio](https://aistudio.baidu.com)平台的[Notbook交互式教程](https://aistudio.baidu.com/aistudio/personalcenter/thirdview/574995)提供实践。
+
+<details><summary> PaddleNLP预训练模型适用任务汇总（<b>点击展开详情</b>）</summary><div>
+
+| Model              | Sequence Classification | Token Classification | Question Answering | Text Generation | Multiple Choice |
+| :----------------- | ----------------------- | -------------------- | ------------------ | --------------- | --------------- |
+| ALBERT             | ✅                       | ✅                    | ✅                  | ❌               | ✅               |
+| BART               | ✅                       | ✅                    | ✅                  | ✅               | ❌               |
+| BERT               | ✅                       | ✅                    | ✅                  | ❌               | ✅               |
+| BigBird            | ✅                       | ✅                    | ✅                  | ❌               | ✅               |
+| BlenderBot         | ❌                       | ❌                    | ❌                  | ✅               | ❌               |
+| ChineseBERT        | ✅                       | ✅                    | ✅                  | ❌               | ❌               |
+| ConvBERT           | ✅                       | ✅                    | ✅                  | ❌               | ✅               |
+| CTRL               | ✅                       | ❌                    | ❌                  | ❌               | ❌               |
+| DistilBERT         | ✅                       | ✅                    | ✅                  | ❌               | ❌               |
+| ELECTRA            | ✅                       | ✅                    | ✅                  | ❌               | ✅               |
+| ERNIE              | ✅                       | ✅                    | ✅                  | ❌               | ✅               |
+| ERNIE-CTM          | ❌                       | ✅                    | ❌                  | ❌               | ❌               |
+| ERNIE-Doc          | ✅                       | ✅                    | ✅                  | ❌               | ❌               |
+| ERNIE-GEN          | ❌                       | ❌                    | ❌                  | ✅               | ❌               |
+| ERNIE-Gram         | ✅                       | ✅                    | ✅                  | ❌               | ❌               |
+| ERNIE-M            | ✅                       | ✅                    | ✅                  | ❌               | ❌               |
+| FNet               | ✅                       | ✅                    | ✅                  | ❌               | ✅               |
+| Funnel-Transformer | ✅                       | ✅                    | ✅                  | ❌               | ❌               |
+| GPT                | ✅                       | ✅                    | ❌                  | ✅               | ❌               |
+| LayoutLM           | ✅                       | ✅                    | ❌                  | ❌               | ❌               |
+| LayoutLMv2         | ❌                       | ✅                    | ❌                  | ❌               | ❌               |
+| LayoutXLM          | ❌                       | ✅                    | ❌                  | ❌               | ❌               |
+| LUKE               | ❌                       | ✅                    | ✅                  | ❌               | ❌               |
+| mBART              | ✅                       | ❌                    | ✅                  | ❌               | ✅               |
+| MegatronBERT       | ✅                       | ✅                    | ✅                  | ❌               | ✅               |
+| MobileBERT         | ✅                       | ❌                    | ✅                  | ❌               | ❌               |
+| MPNet              | ✅                       | ✅                    | ✅                  | ❌               | ✅               |
+| NEZHA              | ✅                       | ✅                    | ✅                  | ❌               | ✅               |
+| PP-MiniLM          | ✅                       | ❌                    | ❌                  | ❌               | ❌               |
+| ProphetNet         | ❌                       | ❌                    | ❌                  | ✅               | ❌               |
+| Reformer           | ✅                       | ❌                    | ✅                  | ❌               | ❌               |
+| RemBERT            | ✅                       | ✅                    | ✅                  | ❌               | ✅               |
+| RoBERTa            | ✅                       | ✅                    | ✅                  | ❌               | ✅               |
+| RoFormer           | ✅                       | ✅                    | ✅                  | ❌               | ❌               |
+| SKEP               | ✅                       | ✅                    | ❌                  | ❌               | ❌               |
+| SqueezeBERT        | ✅                       | ✅                    | ✅                  | ❌               | ❌               |
+| T5                 | ❌                       | ❌                    | ❌                  | ✅               | ❌               |
+| TinyBERT           | ✅                       | ❌                    | ❌                  | ❌               | ❌               |
+| UnifiedTransformer | ❌                       | ❌                    | ❌                  | ✅               | ❌               |
+| XLNet              | ✅                       | ✅                    | ✅                  | ❌               | ✅               |
+
+</div></details>
+
+可参考[Transformer 文档](/docs/model_zoo/index.rst) 查看目前支持的预训练模型结构、参数和详细用法。
+
+### 产业级端到端系统范例
+
+PaddleNLP针对信息抽取、语义检索、智能问答、情感分析等高频NLP场景，提供了端到端系统范例，打通*数据标注*-*模型训练*-*模型调优*-*预测部署*全流程，持续降低NLP技术产业落地门槛。更多详细的系统级产业范例使用说明请参考[Applications](./applications)。
+
+#### 🔍 语义检索系统
+
+针对无监督数据、有监督数据等多种数据情况，结合SimCSE、In-batch Negatives、ERNIE-Gram单塔模型等，推出前沿的语义检索方案，包含召回、排序环节，打通训练、调优、高效向量检索引擎建库和查询全流程。
+
+<div align="center">
+    <img src="https://user-images.githubusercontent.com/11793384/168514909-8817d79a-72c4-4be1-8080-93d1f682bb46.gif" width="400">
+</div>
+
+
+更多使用说明请参考[语义检索系统](./applications/neural_search)。
+
+#### ❓ 智能问答系统
+
+基于[🚀RocketQA](https://github.com/PaddlePaddle/RocketQA)技术的检索式问答系统，支持FAQ问答、说明书问答等多种业务场景。
+
+<div align="center">
+    <img src="https://user-images.githubusercontent.com/11793384/168514868-1babe981-c675-4f89-9168-dd0a3eede315.gif" width="400">
+</div>
+
+
+更多使用说明请参考[智能问答系统](./applications/question_answering)与[文档智能问答](./applications/document_intelligence/doc_vqa)
+
+#### 💌 评论观点抽取与情感分析
+
+基于情感知识增强预训练模型SKEP，针对产品评论进行评价维度和观点抽取，以及细粒度的情感分析。
+
+<div align="center">
+    <img src="https://user-images.githubusercontent.com/11793384/168407260-b7f92800-861c-4207-98f3-2291e0102bbe.png" width="400">
+</div>
+
+更多使用说明请参考[情感分析](./applications/sentiment_analysis)。
+
+#### 🎙️ 智能语音指令解析
+
+集成了[PaddleSpeech](https://github.com/PaddlePaddle/PaddleSpeech)和[百度开放平台](https://ai.baidu.com/)的语音识别和[UIE](./model_zoo/uie)通用信息抽取等技术，打造智能一体化的语音指令解析系统范例，该方案可应用于智能语音填单、智能语音交互、智能语音检索等场景，提高人机交互效率。
+
+<div align="center">
+    <img src="https://user-images.githubusercontent.com/16698950/168589100-a6c6f346-97bb-47b2-ac26-8d50e71fddc5.png" width="400">
+</div>
+
+更多使用说明请参考[智能语音指令解析](./applications/speech_cmd_analysis)。
+
+### 高性能分布式训练与推理
+
+#### ⚡ FastTokenizer：高性能文本处理库
+
+<div align="center">
+    <img src="https://user-images.githubusercontent.com/11793384/168407921-b4395b1d-44bd-41a0-8c58-923ba2b703ef.png" width="400">
+</div>
+
+```python
+AutoTokenizer.from_pretrained("ernie-3.0-medium-zh", use_fast=True)
+```
+
+为了实现更极致的模型部署性能，安装FastTokenizer后只需在`AutoTokenizer` API上打开 `use_fast=True`选项，即可调用C++实现的高性能分词算子，轻松获得超Python百余倍的文本处理加速，更多使用说明可参考[FastTokenizer文档](./fast_tokenizer)。
+
+#### ⚡️ FastGeneration：高性能生成加速库
+
+<div align="center">
+    <img src="https://user-images.githubusercontent.com/11793384/168407831-914dced0-3a5a-40b8-8a65-ec82bf13e53c.gif" width="400">
+</div>
+
+```python
+model = GPTLMHeadModel.from_pretrained('gpt-cpm-large-cn')
+...
+outputs, _ = model.generate(
+    input_ids=inputs_ids, max_length=10, decode_strategy='greedy_search',
+    use_fast=True)
+```
+
+简单地在`generate()`API上打开`use_fast=True`选项，轻松在Transformer、GPT、BART、PLATO、UniLM等生成式预训练模型上获得5倍以上GPU加速，更多使用说明可参考[FastGeneration文档](./fast_generation)。
+
+#### 🚀 Fleet：飞桨4D混合并行分布式训练技术
+
+<div align="center">
+    <img src="https://user-images.githubusercontent.com/11793384/168515134-513f13e0-9902-40ef-98fa-528271dcccda.png" width="300">
+</div>
+
+
+更多关于千亿级AI模型的分布式训练使用说明可参考[GPT-3](./examples/language_model/gpt-3)。
+
+## 社区交流
+
+- 微信扫描二维码并填写问卷，回复小助手关键词（NLP）之后，即可加入交流群领取福利
+
+  - 与众多社区开发者以及官方团队深度交流。
+  - 10G重磅NLP学习大礼包！
+
+  <div align="center">
+  <img src="https://user-images.githubusercontent.com/11987277/245085922-0aa68d24-00ff-442e-9c53-2f1e898151ce.png" width="150" height="150" />
+  </div>
+
+## Citation
+
+如果PaddleNLP对您的研究有帮助，欢迎引用
+
+```
+@misc{=paddlenlp,
+    title={PaddleNLP: An Easy-to-use and High Performance NLP Library},
+    author={PaddleNLP Contributors},
+    howpublished = {\url{https://github.com/PaddlePaddle/PaddleNLP}},
+    year={2021}
+}
+```
+
+## Acknowledge
+
+我们借鉴了Hugging Face的[Transformers](https://github.com/huggingface/transformers)🤗关于预训练模型使用的优秀设计，在此对Hugging Face作者及其开源社区表示感谢。
+
+## License
+
+PaddleNLP遵循[Apache-2.0开源协议](./LICENSE)。

README_cn.md

@@ -1,358 +0,0 @@
-**简体中文**🀄 | [English🌎](./README_en.md)
-
-<p align="center">
-  <img src="https://user-images.githubusercontent.com/1371212/175816733-8ec25eb0-9af3-4380-9218-27c154518258.png" align="middle"  width="500" />
-</p>
-
-------------------------------------------------------------------------------------------
-
-<p align="center">
-    <a href="./LICENSE"><img src="https://img.shields.io/badge/license-Apache%202-dfd.svg"></a>
-    <a href="https://github.com/PaddlePaddle/PaddleNLP/releases"><img src="https://img.shields.io/github/v/release/PaddlePaddle/PaddleNLP?color=ffa"></a>
-    <a href=""><img src="https://img.shields.io/badge/python-3.6.2+-aff.svg"></a>
-    <a href=""><img src="https://img.shields.io/badge/os-linux%2C%20win%2C%20mac-pink.svg"></a>
-    <a href="https://github.com/PaddlePaddle/PaddleNLP/graphs/contributors"><img src="https://img.shields.io/github/contributors/PaddlePaddle/PaddleNLP?color=9ea"></a>
-    <a href="https://github.com/PaddlePaddle/PaddleNLP/commits"><img src="https://img.shields.io/github/commit-activity/m/PaddlePaddle/PaddleNLP?color=3af"></a>
-    <a href="https://pypi.org/project/paddlenlp/"><img src="https://img.shields.io/pypi/dm/paddlenlp?color=9cf"></a>
-    <a href="https://github.com/PaddlePaddle/PaddleNLP/issues"><img src="https://img.shields.io/github/issues/PaddlePaddle/PaddleNLP?color=9cc"></a>
-    <a href="https://github.com/PaddlePaddle/PaddleNLP/stargazers"><img src="https://img.shields.io/github/stars/PaddlePaddle/PaddleNLP?color=ccf"></a>
-</p>
-
-
-<h4 align="center">
-  <a href=#特性> 特性 </a> |
-  <a href=#安装> 安装 </a> |
-  <a href=#快速开始> 快速开始 </a> |
-  <a href=#api文档> API文档 </a> |
-  <a href=#社区交流> 社区交流 </a>
-</h4>
-
-**PaddleNLP**是一款**简单易用**且**功能强大**的自然语言处理开发库。聚合业界**优质预训练模型**并提供**开箱即用**的开发体验，覆盖NLP多场景的模型库搭配**产业实践范例**可满足开发者**灵活定制**的需求。
-
-## News 📢
-* 🔥 **2022.10.27 发布 [PaddleNLP v2.4.2](https://github.com/PaddlePaddle/PaddleNLP/releases/tag/v2.4.2)**
-  * NLG能力扩充：新增📄[**基于Pegasus的中文文本摘要方案**](https://github.com/PaddlePaddle/PaddleNLP/tree/develop/applications/text_summarization/pegasus)，效果领先；新增❓[**问题生成解决方案**](./examples/question_generation)，提供基于业界领先模型UNIMO-Text和大规模多领域问题生成数据集训练的通用问题生成预训练模型。均支持Taskflow一键调用，支持FasterGeneration高性能推理，训练推理部署全流程打通。
-  * 发布 🖼[**PPDiffusers**](https://github.com/PaddlePaddle/PaddleNLP/tree/develop/ppdiffusers)：支持跨模态（如图像与语音）训练和推理的扩散模型（Diffusion Model）工具箱，可快速体验、二次开发 **Stable Diffusion**，持续支持更多模型。
-
-* 🔥 **2022.10.14 发布 [PaddleNLP v2.4.1](https://github.com/PaddlePaddle/PaddleNLP/releases/tag/v2.4.1)**
-  * 🧾 发布多语言跨模态布局增强文档智能大模型 [**ERNIE-Layout**](./model_zoo/ernie-layout/)，刷新11项任务SOTA。同步发布基于ERNIE-Layout的**文档抽取问答模型DocPrompt** 🔖，精准理解文档图片布局与语义信息，轻松应对各类业务场景。
-
-* 🔥 **2022.9.6 发布 [PaddleNLP v2.4](https://github.com/PaddlePaddle/PaddleNLP/releases/tag/v2.4.0)**
-  * 💎 NLP工具：**[NLP 流水线系统 Pipelines](./pipelines)** 发布，支持快速搭建搜索引擎、问答系统，可扩展支持各类NLP系统，让解决 NLP 任务像搭积木一样便捷、灵活、高效！
-  * 💢 产业应用：新增 **[文本分类全流程应用方案](./applications/text_classification)** ，覆盖多分类、多标签、层次分类各类场景，支持 **小样本学习** 和 **TrustAI** 可信计算模型训练与调优；[**通用信息抽取 UIE 能力升级**](./model_zoo/uie)，发布 **UIE-M**，支持中英文混合抽取，新增**UIE 数据蒸馏**方案，打破 UIE 推理瓶颈，推理速度提升 100 倍以上；
-  * 🍭 AIGC 内容生成：新增代码生成 SOTA 模型[**CodeGen**](./examples/code_generation/codegen)，支持多种编程语言代码生成；集成[**文图生成潮流模型**](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/docs/model_zoo/taskflow.md#%E6%96%87%E5%9B%BE%E7%94%9F%E6%88%90) DALL·E Mini、Disco Diffusion、Stable Diffusion，更多趣玩模型等你来玩；新增[**中文文本摘要应用**](./applications/text_summarization)，基于大规模语料的中文摘要模型首次发布，可支持 Taskflow 一键调用和定制训练；
-  * 💪 框架升级：[**模型自动压缩 API**](./docs/compression.md) 发布，自动对模型进行裁减和量化，大幅降低模型压缩技术使用门槛；[**小样本 Prompt**](./applications/text_classification/multi_class/few-shot)能力发布，集成 PET、P-Tuning、RGL 等经典算法。
-
-
-## 社区交流
-
-- 微信扫描二维码并填写问卷，回复小助手关键词（NLP）之后，即可加入交流群领取福利
-
-  - 与众多社区开发者以及官方团队深度交流。
-  - 10G重磅NLP学习大礼包！
-
-  <div align="center">
-  <img src="https://user-images.githubusercontent.com/11793384/200770385-a2c02093-0e1d-4189-b324-8b0251cba316.jpg" width="150" height="150" />
-</div>
-## 特性
-
-#### <a href=#开箱即用的nlp工具集> 📦 开箱即用的NLP工具集 </a>
-
-#### <a href=#丰富完备的中文模型库> 🤗 丰富完备的中文模型库 </a>
-
-#### <a href=#产业级端到端系统范例> 🎛️ 产业级端到端系统范例 </a>
-
-#### <a href=#高性能分布式训练与推理> 🚀 高性能分布式训练与推理 </a>
-
-
-### 开箱即用的NLP工具集
-
-Taskflow提供丰富的**📦开箱即用**的产业级NLP预置模型，覆盖自然语言理解与生成两大场景，提供**💪产业级的效果**与**⚡️极致的推理性能**。
-
-![taskflow1](https://user-images.githubusercontent.com/11793384/159693816-fda35221-9751-43bb-b05c-7fc77571dd76.gif)
-
-Taskflow最新集成了文生图的趣玩应用，三行代码体验 **Stable Diffusion**
-```python
-from paddlenlp import Taskflow
-text_to_image = Taskflow("text_to_image", model="CompVis/stable-diffusion-v1-4")
-image_list = text_to_image('"In the morning light,Chinese ancient buildings in the mountains,Magnificent and fantastic John Howe landscape,lake,clouds,farm,Fairy tale,light effect,Dream,Greg Rutkowski,James Gurney,artstation"')
-```
-<img width="300" alt="image" src="https://user-images.githubusercontent.com/16698950/194882669-f7cc7c98-d63a-45f4-99c1-0514c6712368.png">
-
-更多使用方法可参考[Taskflow文档](./docs/model_zoo/taskflow.md)。
-
-### 丰富完备的中文模型库
-
-#### 🀄 业界最全的中文预训练模型
-
-精选 45+ 个网络结构和 500+ 个预训练模型参数，涵盖业界最全的中文预训练模型：既包括文心NLP大模型的ERNIE、PLATO等，也覆盖BERT、GPT、RoBERTa、T5等主流结构。通过`AutoModel` API一键⚡**高速下载**⚡。
-
-```python
-from paddlenlp.transformers import *
-
-ernie = AutoModel.from_pretrained('ernie-3.0-medium-zh')
-bert = AutoModel.from_pretrained('bert-wwm-chinese')
-albert = AutoModel.from_pretrained('albert-chinese-tiny')
-roberta = AutoModel.from_pretrained('roberta-wwm-ext')
-electra = AutoModel.from_pretrained('chinese-electra-small')
-gpt = AutoModelForPretraining.from_pretrained('gpt-cpm-large-cn')
-```
-
-针对预训练模型计算瓶颈，可以使用API一键使用文心ERNIE-Tiny全系列轻量化模型，降低预训练模型部署难度。
-
-```python
-# 6L768H
-ernie = AutoModel.from_pretrained('ernie-3.0-medium-zh')
-# 6L384H
-ernie = AutoModel.from_pretrained('ernie-3.0-mini-zh')
-# 4L384H
-ernie = AutoModel.from_pretrained('ernie-3.0-micro-zh')
-# 4L312H
-ernie = AutoModel.from_pretrained('ernie-3.0-nano-zh')
-```
-
-对预训练模型应用范式如语义表示、文本分类、句对匹配、序列标注、问答等，提供统一的API体验。
-
-```python
-import paddle
-from paddlenlp.transformers import *
-
-tokenizer = AutoTokenizer.from_pretrained('ernie-3.0-medium-zh')
-text = tokenizer('自然语言处理')
-
-# 语义表示
-model = AutoModel.from_pretrained('ernie-3.0-medium-zh')
-sequence_output, pooled_output = model(input_ids=paddle.to_tensor([text['input_ids']]))
-# 文本分类 & 句对匹配
-model = AutoModelForSequenceClassification.from_pretrained('ernie-3.0-medium-zh')
-# 序列标注
-model = AutoModelForTokenClassification.from_pretrained('ernie-3.0-medium-zh')
-# 问答
-model = AutoModelForQuestionAnswering.from_pretrained('ernie-3.0-medium-zh')
-```
-
-#### 💯 全场景覆盖的应用示例
-
-覆盖从学术到产业的NLP应用示例，涵盖NLP基础技术、NLP系统应用以及拓展应用。全面基于飞桨核心框架2.0全新API体系开发，为开发者提供飞桨文本领域的最佳实践。
-
-精选预训练模型示例可参考[Model Zoo](./model_zoo)，更多场景示例文档可参考[examples目录](./examples)。更有免费算力支持的[AI Studio](https://aistudio.baidu.com)平台的[Notbook交互式教程](https://aistudio.baidu.com/aistudio/personalcenter/thirdview/574995)提供实践。
-
-<details><summary> PaddleNLP预训练模型适用任务汇总（<b>点击展开详情</b>）</summary><div>
-
-| Model              | Sequence Classification | Token Classification | Question Answering | Text Generation | Multiple Choice |
-| :----------------- | ----------------------- | -------------------- | ------------------ | --------------- | --------------- |
-| ALBERT             | ✅                       | ✅                    | ✅                  | ❌               | ✅               |
-| BART               | ✅                       | ✅                    | ✅                  | ✅               | ❌               |
-| BERT               | ✅                       | ✅                    | ✅                  | ❌               | ✅               |
-| BigBird            | ✅                       | ✅                    | ✅                  | ❌               | ✅               |
-| BlenderBot         | ❌                       | ❌                    | ❌                  | ✅               | ❌               |
-| ChineseBERT        | ✅                       | ✅                    | ✅                  | ❌               | ❌               |
-| ConvBERT           | ✅                       | ✅                    | ✅                  | ❌               | ✅               |
-| CTRL               | ✅                       | ❌                    | ❌                  | ❌               | ❌               |
-| DistilBERT         | ✅                       | ✅                    | ✅                  | ❌               | ❌               |
-| ELECTRA            | ✅                       | ✅                    | ✅                  | ❌               | ✅               |
-| ERNIE              | ✅                       | ✅                    | ✅                  | ❌               | ✅               |
-| ERNIE-CTM          | ❌                       | ✅                    | ❌                  | ❌               | ❌               |
-| ERNIE-Doc          | ✅                       | ✅                    | ✅                  | ❌               | ❌               |
-| ERNIE-GEN          | ❌                       | ❌                    | ❌                  | ✅               | ❌               |
-| ERNIE-Gram         | ✅                       | ✅                    | ✅                  | ❌               | ❌               |
-| ERNIE-M            | ✅                       | ✅                    | ✅                  | ❌               | ❌               |
-| FNet               | ✅                       | ✅                    | ✅                  | ❌               | ✅               |
-| Funnel-Transformer | ✅                       | ✅                    | ✅                  | ❌               | ❌               |
-| GPT                | ✅                       | ✅                    | ❌                  | ✅               | ❌               |
-| LayoutLM           | ✅                       | ✅                    | ❌                  | ❌               | ❌               |
-| LayoutLMv2         | ❌                       | ✅                    | ❌                  | ❌               | ❌               |
-| LayoutXLM          | ❌                       | ✅                    | ❌                  | ❌               | ❌               |
-| LUKE               | ❌                       | ✅                    | ✅                  | ❌               | ❌               |
-| mBART              | ✅                       | ❌                    | ✅                  | ❌               | ✅               |
-| MegatronBERT       | ✅                       | ✅                    | ✅                  | ❌               | ✅               |
-| MobileBERT         | ✅                       | ❌                    | ✅                  | ❌               | ❌               |
-| MPNet              | ✅                       | ✅                    | ✅                  | ❌               | ✅               |
-| NEZHA              | ✅                       | ✅                    | ✅                  | ❌               | ✅               |
-| PP-MiniLM          | ✅                       | ❌                    | ❌                  | ❌               | ❌               |
-| ProphetNet         | ❌                       | ❌                    | ❌                  | ✅               | ❌               |
-| Reformer           | ✅                       | ❌                    | ✅                  | ❌               | ❌               |
-| RemBERT            | ✅                       | ✅                    | ✅                  | ❌               | ✅               |
-| RoBERTa            | ✅                       | ✅                    | ✅                  | ❌               | ✅               |
-| RoFormer           | ✅                       | ✅                    | ✅                  | ❌               | ❌               |
-| SKEP               | ✅                       | ✅                    | ❌                  | ❌               | ❌               |
-| SqueezeBERT        | ✅                       | ✅                    | ✅                  | ❌               | ❌               |
-| T5                 | ❌                       | ❌                    | ❌                  | ✅               | ❌               |
-| TinyBERT           | ✅                       | ❌                    | ❌                  | ❌               | ❌               |
-| UnifiedTransformer | ❌                       | ❌                    | ❌                  | ✅               | ❌               |
-| XLNet              | ✅                       | ✅                    | ✅                  | ❌               | ✅               |
-
-</div></details>
-
-可参考[Transformer 文档](/docs/model_zoo/index.rst) 查看目前支持的预训练模型结构、参数和详细用法。
-
-### 产业级端到端系统范例
-
-PaddleNLP针对信息抽取、语义检索、智能问答、情感分析等高频NLP场景，提供了端到端系统范例，打通*数据标注*-*模型训练*-*模型调优*-*预测部署*全流程，持续降低NLP技术产业落地门槛。更多详细的系统级产业范例使用说明请参考[Applications](./applications)。
-
-#### 🔍 语义检索系统
-
-针对无监督数据、有监督数据等多种数据情况，结合SimCSE、In-batch Negatives、ERNIE-Gram单塔模型等，推出前沿的语义检索方案，包含召回、排序环节，打通训练、调优、高效向量检索引擎建库和查询全流程。
-
-<div align="center">
-    <img src="https://user-images.githubusercontent.com/11793384/168514909-8817d79a-72c4-4be1-8080-93d1f682bb46.gif" width="400">
-</div>
-
-
-更多使用说明请参考[语义检索系统](./applications/neural_search)。
-
-#### ❓ 智能问答系统
-
-基于[🚀RocketQA](https://github.com/PaddlePaddle/RocketQA)技术的检索式问答系统，支持FAQ问答、说明书问答等多种业务场景。
-
-<div align="center">
-    <img src="https://user-images.githubusercontent.com/11793384/168514868-1babe981-c675-4f89-9168-dd0a3eede315.gif" width="400">
-</div>
-
-
-更多使用说明请参考[智能问答系统](./applications/question_answering)与[文档智能问答](./applications/document_intelligence/doc_vqa)
-
-#### 💌 评论观点抽取与情感分析
-
-基于情感知识增强预训练模型SKEP，针对产品评论进行评价维度和观点抽取，以及细粒度的情感分析。
-
-<div align="center">
-    <img src="https://user-images.githubusercontent.com/11793384/168407260-b7f92800-861c-4207-98f3-2291e0102bbe.png" width="400">
-</div>
-
-更多使用说明请参考[情感分析](./applications/sentiment_analysis)。
-
-#### 🎙️ 智能语音指令解析
-
-集成了[PaddleSpeech](https://github.com/PaddlePaddle/PaddleSpeech)和[百度开放平台](https://ai.baidu.com/)的的语音识别和[UIE](./model_zoo/uie)通用信息抽取等技术，打造智能一体化的语音指令解析系统范例，该方案可应用于智能语音填单、智能语音交互、智能语音检索等场景，提高人机交互效率。
-
-<div align="center">
-    <img src="https://user-images.githubusercontent.com/16698950/168589100-a6c6f346-97bb-47b2-ac26-8d50e71fddc5.png" width="400">
-</div>
-
-更多使用说明请参考[智能语音指令解析](./applications/speech_cmd_analysis)。
-
-### 高性能分布式训练与推理
-
-#### ⚡ FasterTokenizer：高性能文本处理库
-
-<div align="center">
-    <img src="https://user-images.githubusercontent.com/11793384/168407921-b4395b1d-44bd-41a0-8c58-923ba2b703ef.png" width="400">
-</div>
-
-```python
-AutoTokenizer.from_pretrained("ernie-3.0-medium-zh", use_faster=True)
-```
-
-为了实现更极致的模型部署性能，安装FastTokenizers后只需在`AutoTokenizer` API上打开 `use_faster=True`选项，即可调用C++实现的高性能分词算子，轻松获得超Python百余倍的文本处理加速，更多使用说明可参考[FasterTokenizer文档](./faster_tokenizer)。
-
-#### ⚡️ FasterGeneration：高性能生成加速库
-
-<div align="center">
-    <img src="https://user-images.githubusercontent.com/11793384/168407831-914dced0-3a5a-40b8-8a65-ec82bf13e53c.gif" width="400">
-</div>
-
-```python
-model = GPTLMHeadModel.from_pretrained('gpt-cpm-large-cn')
-...
-outputs, _ = model.generate(
-    input_ids=inputs_ids, max_length=10, decode_strategy='greedy_search',
-    use_faster=True)
-```
-
-简单地在`generate()`API上打开`use_faster=True`选项，轻松在Transformer、GPT、BART、PLATO、UniLM等生成式预训练模型上获得5倍以上GPU加速，更多使用说明可参考[FasterGeneration文档](./faster_generation)。
-
-#### 🚀 Fleet：飞桨4D混合并行分布式训练技术
-
-<div align="center">
-    <img src="https://user-images.githubusercontent.com/11793384/168515134-513f13e0-9902-40ef-98fa-528271dcccda.png" width="300">
-</div>
-
-
-更多关于千亿级AI模型的分布式训练使用说明可参考[GPT-3](./examples/language_model/gpt-3)。
-
-## 安装
-
-### 环境依赖
-
-- python >= 3.6
-- paddlepaddle >= 2.2
-
-### pip安装
-
-```shell
-pip install --upgrade paddlenlp
-```
-
-更多关于PaddlePaddle和PaddleNLP安装的详细教程请查看[Installation](./docs/get_started/installation.rst)。
-
-## 快速开始
-
-这里以信息抽取-命名实体识别任务，UIE模型为例，来说明如何快速使用PaddleNLP:
-
-### 一键预测
-
-PaddleNLP提供[一键预测功能](./docs/model_zoo/taskflow.md)，无需训练，直接输入数据即可开放域抽取结果：
-
-```python
->>> from pprint import pprint
->>> from paddlenlp import Taskflow
-
->>> schema = ['时间', '选手', '赛事名称'] # Define the schema for entity extraction
->>> ie = Taskflow('information_extraction', schema=schema)
->>> pprint(ie("2月8日上午北京冬奥会自由式滑雪女子大跳台决赛中中国选手谷爱凌以188.25分获得金牌！"))
-[{'时间': [{'end': 6,
-          'probability': 0.9857378532924486,
-          'start': 0,
-          'text': '2月8日上午'}],
-  '赛事名称': [{'end': 23,
-            'probability': 0.8503089953268272,
-            'start': 6,
-            'text': '北京冬奥会自由式滑雪女子大跳台决赛'}],
-  '选手': [{'end': 31,
-          'probability': 0.8981548639781138,
-          'start': 28,
-          'text': '谷爱凌'}]}]
-```
-
-### 小样本学习
-
-如果对一键预测效果不满意，也可以使用少量数据进行模型精调，进一步提升特定场景的效果，详见[UIE小样本定制训练](./model_zoo/uie/)。
-
-更多PaddleNLP内容可参考：
-- [精选模型库](./model_zoo)，包含优质预训练模型的端到端全流程使用。
-- [多场景示例](./examples)，了解如何使用PaddleNLP解决NLP多种技术问题，包含基础技术、系统应用与拓展应用。
-- [交互式教程](https://aistudio.baidu.com/aistudio/personalcenter/thirdview/574995)，在🆓免费算力平台AI Studio上快速学习PaddleNLP。
-
-
-## API文档
-
-PaddleNLP提供全流程的文本领域API，可大幅提升NLP任务建模的效率：
-
-- 支持[千言](https://www.luge.ai)等丰富中文数据集加载的[Dataset API](https://paddlenlp.readthedocs.io/zh/latest/data_prepare/dataset_list.html)。
-- 提供🤗Hugging Face Style的API，支持 **500+** 优质预训练模型加载的[Transformers API](https://paddlenlp.readthedocs.io/zh/latest/model_zoo/index.html)。
-- 提供30+多语言词向量的[Embedding API](https://paddlenlp.readthedocs.io/zh/latest/model_zoo/embeddings.html)
-
-更多使用方法请参考[API文档](https://paddlenlp.readthedocs.io/zh/latest/)。
-
-
-## Citation
-
-如果PaddleNLP对您的研究有帮助，欢迎引用
-
-```
-@misc{=paddlenlp,
-    title={PaddleNLP: An Easy-to-use and High Performance NLP Library},
-    author={PaddleNLP Contributors},
-    howpublished = {\url{https://github.com/PaddlePaddle/PaddleNLP}},
-    year={2021}
-}
-```
-
-## Acknowledge
-
-我们借鉴了Hugging Face的[Transformers](https://github.com/huggingface/transformers)🤗关于预训练模型使用的优秀设计，在此对Hugging Face作者及其开源社区表示感谢。
-
-## License
-
-PaddleNLP遵循[Apache-2.0开源协议](./LICENSE)。

README_en.md

@@ -1,15 +1,14 @@
-[简体中文🀄](./README_cn.md) |  **English**🌎
 
-<p align="center">
-  <img src="https://user-images.githubusercontent.com/1371212/175816733-8ec25eb0-9af3-4380-9218-27c154518258.png" align="middle"  width="500" />
-</p>
+[简体中文🀄](./README.md) | **English🌎**
+
+<p align="center"> <img src="https://user-images.githubusercontent.com/1371212/175816733-8ec25eb0-9af3-4380-9218-27c154518258.png" align="middle"  width="500" /> </p>
 
 ------------------------------------------------------------------------------------------
 
 <p align="center">
     <a href="./LICENSE"><img src="https://img.shields.io/badge/license-Apache%202-dfd.svg"></a>
     <a href="https://github.com/PaddlePaddle/PaddleNLP/releases"><img src="https://img.shields.io/github/v/release/PaddlePaddle/PaddleNLP?color=ffa"></a>
-    <a href=""><img src="https://img.shields.io/badge/python-3.6.2+-aff.svg"></a>
+    <a href=""><img src="https://img.shields.io/badge/python-3.7+-aff.svg"></a>
     <a href=""><img src="https://img.shields.io/badge/os-linux%2C%20win%2C%20mac-pink.svg"></a>
     <a href="https://github.com/PaddlePaddle/PaddleNLP/graphs/contributors"><img src="https://img.shields.io/github/contributors/PaddlePaddle/PaddleNLP?color=9ea"></a>
     <a href="https://github.com/PaddlePaddle/PaddleNLP/commits"><img src="https://img.shields.io/github/commit-activity/m/PaddlePaddle/PaddleNLP?color=3af"></a>
@@ -18,33 +17,37 @@
     <a href="https://github.com/PaddlePaddle/PaddleNLP/stargazers"><img src="https://img.shields.io/github/stars/PaddlePaddle/PaddleNLP?color=ccf"></a>
 </p>
 
-<h4 align="center">
-  <a href=#features> Features </a> |
-  <a href=#installation> Installation </a> |
-  <a href=#quick-start> Quick Start </a> |
-  <a href=#api-reference> API Reference </a> |
-  <a href=#community> Community </a>
-</h4>
+<h4 align="center"> <a href=#features> Features </a> | <a href=#installation> Installation </a> | <a href=#quick-start> Quick Start </a> | <a href=#api-reference> API Reference </a> | <a href=#community> Community </a>
 
-**PaddleNLP** is an *easy-to-use* and *powerful* NLP library with **Awesome** pre-trained model zoo, supporting wide-range of NLP tasks from research to industrial applications.
+**PaddleNLP** is a NLP library that is both **easy to use** and **powerful**. It aggregates high-quality pretrained models in the industry and provides a **plug-and-play** development experience, covering a model library for various NLP scenarios. With practical examples from industry practices, PaddleNLP can meet the needs of developers who require **flexible customization**.
 
 ## News 📢
 
-* 🔥 **2022.10.27 [PaddleNLP v2.4.2](https://github.com/PaddlePaddle/PaddleNLP/releases/tag/v2.4.2) Released!**
-  * NLG Upgrade: 📄 Release [**Solution of Text Summarization**](https://github.com/PaddlePaddle/PaddleNLP/tree/develop/applications/text_summarization/pegasus) based on Pegasus;❓ Release [**Solution of Problem Generation**](./examples/question_generation), providing **general problem generation pre-trained model** based on Baidu's UNIMO Text and large-scale multi domain problem generation dataset。Supporting high-performance inference ability based on FasterGeneration , and covering the whole process of training , inference and deployment.
-* 🔥 **2022.10.14 [PaddleNLP v2.4.1](https://github.com/PaddlePaddle/PaddleNLP/releases/tag/v2.4.1) Released!**
-  * 🧾 Release multilingual/cross-lingual pre-trained models [**ERNIE-Layout**](./model_zoo/ernie-layout/) which achieves new SOTA results in 11 downstream tasks. **DocPrompt** 🔖 based on ERNIE-Layout is also released which has the ability for multilingual document information extraction and question ansering.
-* 🔥 **2022.9.6 [PaddleNLPv2.4](https://github.com/PaddlePaddle/PaddleNLP/releases/tag/v2.4.0) Released!**
+* **2024.01.04 [PaddleNLP v2.7](https://github.com/PaddlePaddle/PaddleNLP/releases/tag/v2.7.0)**: The LLM experience is fully upgraded, and the tool chain LLM entrance is unified. Unify the implementation code of pre-training, fine-tuning, compression, inference and deployment to the `PaddleNLP/llm` directory. The new [LLM Toolchain Documentation](https://paddlenlp.readthedocs.io/zh/latest/llm/finetune.html) provides one-stop guidance for users from getting started with LLM to business deployment and launch. The full breakpoint storage mechanism Unified Checkpoint greatly improves the versatility of LLM storage. Efficient fine-tuning upgrade supports the simultaneous use of efficient fine-tuning + LoRA, and supports QLoRA and other algorithms.
 
-  * 💎 NLP Tool：**[Pipelines](./pipelines)** released. Supports for fast construction of search engine and question answering systems, and it is expandable to all kinds of NLP systems. Building end-to-end pipelines for NLP tasks like playing Lego!
-
-  * 💢 Industrial application：Release **[Complete Solution of Text Classification](./applications/text_classification)** covering various scenarios of text classification: multi-class, multi-label and hierarchical, it also supports for **few-shot learning** and the training and optimization of **TrustAI**. Upgrade for [**Universal Information Extraction**](./model_zoo/uie) and release **UIE-M**, support both Chinese and English information extraction in a single model; release the data distillation solution for UIE to break the bottleneck of time-consuming of inference.
-
-  * 🍭 AIGC: Release code generation SOTA model [**CodeGen**](./examples/code_generation/codegen), supports for multiple programming languages code generation. Integrate [**Text to Image Model**](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/docs/model_zoo/taskflow.md#%E6%96%87%E5%9B%BE%E7%94%9F%E6%88%90) DALL·E Mini, Disco Diffusion, Stable Diffusion, let's play and have some fun! Release [**Chinese Text Summarization Application**](./applications/text_summarization), first release of chinese text summarization model pretrained on a large scale of corpus, it can be use via Taskflow API and support for finetuning on your own data.
-
-  * 💪 Framework upgrade: Release [**Auto Model Compression API**](./docs/compression.md), supports for pruning and quantization automatically, lower the barriers of model compression; Release [**Few-shot Prompt**](./applications/text_classification/multi_class/few-shot), includes the algorithms such as PET, P-Tuning and RGL.
+* **2023.08.15 [PaddleNLP v2.6](https://github.com/PaddlePaddle/PaddleNLP/releases/tag/v2.6.0)**: Release [Full-process LLM toolchain](./llm) , covering all aspects of pre-training, fine-tuning, compression, inference and deployment, providing users with end-to-end LLM solutions and one-stop development experience; built-in [4D parallel distributed Trainer](./docs/trainer.md ), [Efficient fine-tuning algorithm LoRA/Prefix Tuning](./llm#33-lora), [Self-developed INT8/INT4 quantization algorithm](./llm#6-quantization), etc.; fully supports [LLaMA 1/2](./llm/llama), [BLOOM](.llm/bloom), [ChatGLM 1/2](./llm/chatglm), [GLM](./llm/glm), [OPT](./llm/opt) and other mainstream LLMs.
 
 
+## Installation
+
+### Prerequisites
+
+* python >= 3.7
+* paddlepaddle >= 2.6.0
+
+More information about PaddlePaddle installation please refer to [PaddlePaddle's Website](https://www.paddlepaddle.org.cn/install/quick?docurl=/documentation/docs/zh/install/conda/linux-conda.html).
+
+### Python pip Installation
+
+```
+pip install --upgrade paddlenlp
+```
+
+or you can install the latest develop branch code with the following command:
+
+```shell
+pip install --pre --upgrade paddlenlp -f https://www.paddlepaddle.org.cn/whl/paddlenlp.html
+```
 
 
 ## Features
@@ -60,7 +63,7 @@
 
 ### Out-of-Box NLP Toolset
 
-Taskflow aims to provide off-the-shelf NLP pre-built task covering NLU and NLG technique, in the meanwhile with extreamly fast infernece satisfying industrial scenario.
+Taskflow aims to provide off-the-shelf NLP pre-built task covering NLU and NLG technique, in the meanwhile with extremely fast inference satisfying industrial scenario.
 
 ![taskflow1](https://user-images.githubusercontent.com/11793384/159693816-fda35221-9751-43bb-b05c-7fc77571dd76.gif)
 
@@ -116,7 +119,7 @@ model = AutoModelForQuestionAnswering.from_pretrained('ernie-3.0-medium-zh')
 
 #### Wide-range NLP Task Support
 
-PaddleNLP provides rich examples covering mainstream NLP task to help developers accelerate problem solving. You can find our powerful transformer [Model Zoo](./model_zoo), and wide-range NLP application [exmaples](./examples) with detailed instructions.
+PaddleNLP provides rich examples covering mainstream NLP task to help developers accelerate problem solving. You can find our powerful transformer [Model Zoo](./model_zoo), and wide-range NLP application [examples](./examples) with detailed instructions.
 
 Also you can run our interactive [Notebook tutorial](https://aistudio.baidu.com/aistudio/personalcenter/thirdview/574995) on AI Studio, a powerful platform with **FREE** computing resource.
 
@@ -171,9 +174,9 @@ For more pretrained model usage, please refer to [Transformer API Docs](./docs/m
 
 ### Industrial End-to-end System
 
-We provide high value scenarios including information extraction, semantic retrieval, questionn answering high-value.
+We provide high value scenarios including information extraction, semantic retrieval, question answering high-value.
 
-For more details industial cases please refer to [Applications](./applications).
+For more details industrial cases please refer to [Applications](./applications).
 
 
 #### 🔍 Neural Search System
@@ -221,19 +224,19 @@ For more details please refer to [Speech Command Analysis](./applications/speech
 
 ### High Performance Distributed Training and Inference
 
-#### ⚡ FasterTokenizer: High Performance Text Preprocessing Library
+#### ⚡ FastTokenizer: High Performance Text Preprocessing Library
 
 <div align="center">
     <img src="https://user-images.githubusercontent.com/11793384/168407921-b4395b1d-44bd-41a0-8c58-923ba2b703ef.png" width="400">
 </div>
 
 ```python
-AutoTokenizer.from_pretrained("ernie-3.0-medium-zh", use_faster=True)
+AutoTokenizer.from_pretrained("ernie-3.0-medium-zh", use_fast=True)
 ```
 
-Set `use_faster=True` to use C++ Tokenizer kernel to achieve 100x faster on text pre-processing. For more usage please refer to [FasterTokenizer](./faster_tokenizer).
+Set `use_fast=True` to use C++ Tokenizer kernel to achieve 100x faster on text pre-processing. For more usage please refer to [FastTokenizer](./fast_tokenizer).
 
-#### ⚡ FasterGeneration: High Perforance Generation Library
+#### ⚡ FastGeneration: High Performance Generation Library
 
 <div align="center">
     <img src="https://user-images.githubusercontent.com/11793384/168407831-914dced0-3a5a-40b8-8a65-ec82bf13e53c.gif" width="400">
@@ -244,10 +247,10 @@ model = GPTLMHeadModel.from_pretrained('gpt-cpm-large-cn')
 ...
 outputs, _ = model.generate(
     input_ids=inputs_ids, max_length=10, decode_strategy='greedy_search',
-    use_faster=True)
+    use_fast=True)
 ```
 
-Set `use_faster=True` to achieve 5x speedup for Transformer, GPT, BART, PLATO, UniLM text generation. For more usage please refer to [FasterGeneration](./faster_generation).
+Set `use_fast=True` to achieve 5x speedup for Transformer, GPT, BART, PLATO, UniLM text generation. For more usage please refer to [FastGeneration](./fast_generation).
 
 #### 🚀 Fleet: 4D Hybrid Distributed Training
 
@@ -259,24 +262,9 @@ Set `use_faster=True` to achieve 5x speedup for Transformer, GPT, BART, PLATO, U
 For more super large-scale model pre-training details please refer to [GPT-3](./examples/language_model/gpt-3).
 
 
-## Installation
-
-### Prerequisites
-
-* python >= 3.6
-* paddlepaddle >= 2.2
-
-More information about PaddlePaddle installation please refer to [PaddlePaddle's Website](https://www.paddlepaddle.org.cn/install/quick?docurl=/documentation/docs/zh/install/conda/linux-conda.html).
-
-### Python pip Installation
-
-```
-pip install --upgrade paddlenlp
-```
-
 ## Quick Start
 
-**Taskflow** aims to provide off-the-shelf NLP pre-built task covering NLU and NLG scenario, in the meanwhile with extreamly fast infernece satisfying industrial applications.
+**Taskflow** aims to provide off-the-shelf NLP pre-built task covering NLU and NLG scenario, in the meanwhile with extremely fast inference satisfying industrial applications.
 
 ```python
 from paddlenlp import Taskflow
@@ -326,7 +314,7 @@ To connect with other users and contributors, welcome to join our [Slack channel
 Scan the QR code below with your Wechat⬇️. You can access to official technical exchange group. Look forward to your participation.
 
 <div align="center">
-<img src="https://user-images.githubusercontent.com/11793384/200770385-a2c02093-0e1d-4189-b324-8b0251cba316.jpg" width="150" height="150" />
+<img src="https://user-images.githubusercontent.com/11987277/245085922-0aa68d24-00ff-442e-9c53-2f1e898151ce.png" width="150" height="150" />
 </div>
 
 

VERSION

@@ -1 +0,0 @@
-4.2.3

applications/README.md

@@ -0,0 +1,130 @@
+# 产业级端到端系统范例
+
+## 1、简介
+
+PaddleNLP 从预训练模型库出发，提供了经典预训练模型在主流 NLP 任务上丰富的[应用示例](../examples)，满足了大量开发者的学习科研与基础应用需求。
+
+针对更广泛的产业落地需求、更复杂的 NLP 场景任务，PaddleNLP 推出**产业级端到端系统范例库**（下文简称产业范例），提供单个模型之上的产业解决方案。
+
+- 最强模型与实践———产业范例针对具体业务场景，提供最佳模型（组合），兼顾模型精度与性能，降低开发者模型选型成本；
+- 全流程———打通数据标注-模型训练-模型调优-模型压缩—预测部署全流程，帮助开发者更低成本得完成产业落地。
+
+## 2、基于 Pipelines 构建产业范例，加速落地
+
+在面向不同场景任务建设一系列产业方案的过程中，不难发现，从技术基础设施角度看：
+
+（1）NLP系统都可以抽象为由多个基础组件串接而成的流水线系统；
+（2）多个NLP流水线系统可共享使用相同的基础组件。
+
+因此，PaddleNLP 逐渐孵化出了一套 NLP 流水线系统 [Pipelines](../pipelines)，将各个 NLP 复杂系统的通用模块抽象封装为标准组件，支持开发者通过配置文件对标准组件进行组合，仅需几分钟即可定制化构建智能系统，让解决NLP任务像搭积木一样便捷、灵活、高效。同时，Pipelines 中预置了前沿的预训练模型和算法，在研发效率、模型效果和性能方面提供多重保障。因此，Pipelines 能够大幅加快开发者使用飞桨落地的效率。
+
+
+<div>
+    <img src="https://user-images.githubusercontent.com/11793384/212836991-d9132e46-b5bf-4389-80e1-4f9dee32f1fe.png" width="90%" length="90%">
+</div>
+
+<br>
+
+**PaddleNLP 提供了多个版本的产业范例:**
+
+- 如果你希望快速体验、直接应用、从零搭建一套完整系统，推荐使用 **Pipelines 版本**。这里集成了训练好的模型，无需关心模型训练细节；提供 Docker 环境，可快速一键部署端到端系统；打通前端 Demo 界面，便于直观展示、分析、调试效果。
+- 如果你希望使用自己的业务数据进行二次开发，推荐使用`./applications`目录下的**可定制版本**，训练好的模型可以直接集成进 Pipelines 中进行使用。
+- 也可以使用 [AI Studio](https://aistudio.baidu.com/aistudio/index) 在线 Jupyter Notebook 快速体验，有 GPU 算力哦。
+
+| 场景任务   | Pipelines版本地址 | 可定制版本地址 | Notebook |
+| :--------------- | ------- | ------- | ------- |
+| **检索**| [字面+语义检索](../pipelines/examples/semantic-search) | [语义检索](./neural_search) | [基于Pipelines搭建检索系统](https://aistudio.baidu.com/aistudio/projectdetail/4442670)<br>[二次开发语义检索](https://aistudio.baidu.com/aistudio/projectdetail/3351784) |
+| **问答** | [FAQ问答](../pipelines/examples/FAQ/)<br>[无监督检索式问答](../pipelines/examples/unsupervised-question-answering)<br>[有监督检索式问答](../pipelines/examples/question-answering) | [FAQ问答](./question_answering/supervised_qa)<br>[无监督检索式问答](./question_answering/unsupervised_qa) | [基于Pipelines搭建FAQ问答系统](https://aistudio.baidu.com/aistudio/projectdetail/4465498)<br>[基于Pipelines搭建抽取式问答系统](https://aistudio.baidu.com/aistudio/projectdetail/4442857)<br>[FAQ政务问答](https://aistudio.baidu.com/aistudio/projectdetail/3678873)<br>[FAQ保险问答](https://aistudio.baidu.com/aistudio/projectdetail/3882519) |
+| **文本分类**| 暂无 | [文本分类](./text_classification)  | [对话意图识别](https://aistudio.baidu.com/aistudio/projectdetail/2017202)<br>[法律文本多标签分类](https://aistudio.baidu.com/aistudio/projectdetail/3996601)<br>[层次分类](https://aistudio.baidu.com/aistudio/projectdetail/4568985) |
+| **通用文本分类** | 暂无 | [通用文本分类](./zero_shot_text_classification) |  |
+| **通用信息抽取** | 暂无 | [通用信息抽取](./information_extraction) | [UIE快速体验](https://aistudio.baidu.com/aistudio/projectdetail/3914778)<br>[UIE微调实体抽取](https://aistudio.baidu.com/aistudio/projectdetail/4038499)<br>[UIE微调关系抽取](https://aistudio.baidu.com/aistudio/projectdetail/4371345)<br>[UIE-X快速体验](https://aistudio.baidu.com/aistudio/projectdetail/5017442)<br>[UIE-X微调](https://aistudio.baidu.com/aistudio/projectdetail/5261592) |
+| **情感分析**  | [情感分析](../pipelines/examples/sentiment_analysis)  | [情感分析](./sentiment_analysis) |  [情感分析](https://aistudio.baidu.com/aistudio/projectdetail/5318177)|
+| **文档智能**  | [文档抽取问答](../pipelines/examples/document-intelligence) |  [跨模态文档问答](./document_intelligence/doc_vqa)| [文档抽取问答](https://aistudio.baidu.com/aistudio/projectdetail/4881278)<br>[汽车说明书问答](https://aistudio.baidu.com/aistudio/projectdetail/4049663)  |
+| **文生图**  | [文生图系统](../pipelines/examples/text_to_image)  | 可参考[PPDiffusers](../ppdiffusers) |   |
+| **语音指令解析**  | 暂无 | [语音指令解析](./speech_cmd_analysis) | [语音指令解析](https://aistudio.baidu.com/aistudio/projectdetail/4399703) |
+| **文本摘要**  | 暂无 | [文本摘要](./text_summarization) | [文本摘要](https://aistudio.baidu.com/aistudio/projectdetail/4903667) |
+
+## 3、典型范例介绍
+
+#### 📄 通用信息抽取系统
+
+- 首个产业级通用信息抽取方案 UIE，面向纯文本，实现多任务统一建模，提供强大的零样本抽取和少样本快速迁移能力；
+- 首个兼具文本及文档抽取能力、多语言、开放域的信息抽取方案 UIE-X，基于 [ERNIE-Layout](../model_zoo/ernie-layout) 跨模态布局增强预训练模型，集成 [PaddleOCR](https://github.com/PaddlePaddle/PaddleOCR) 的 PP-OCR、PP-Structure 版面分析能力，小样本文档信息抽取效果领先。
+
+<div align="center">
+    <img src="https://user-images.githubusercontent.com/11793384/213365046-69967745-b4a8-4435-98fb-c34f68cd22e9.png" width="60%" length="60%">
+</div>
+
+
+详细使用说明请参考[通用信息抽取系统](./information_extraction)，更多：[UIE 解读](https://mp.weixin.qq.com/s/-hHz8knHIKKqKCBTke7i5A)、[UIE-X 解读](https://zhuanlan.zhihu.com/p/592422623)。
+
+#### 🔍 语义检索系统
+
+- 前沿算法———基于 SimCSE、In-batch Negatives、ERNIE Pairwise、RocketQA Pointwise 等提供针对无监督、有监督等多种数据情况的多样化方案；
+- 全流程———覆盖召回、排序环节，集成主流 ANN 引擎，同时兼容 ElasticSearch 字面检索模式，提供多路召回方案。打通训练、调优、高效向量检索引擎建库和查询全流程。
+
+<div align="center">
+    <img src="https://user-images.githubusercontent.com/11793384/213134465-30cae5fd-4cd1-4e5b-a1cb-fa55c72980a7.gif" width="60%" length="60%">
+</div>
+
+详细使用说明请参考[语义检索系统](./neural_search)。
+
+#### ❓ 智能问答系统
+
+- 端到端问答技术 [🚀RocketQA](https://github.com/PaddlePaddle/RocketQA)，首个中文端到端问答模型，基于知识增强的预训练模型ERNIE和百万量级的人工标注数据集DuReader训练得到，效果优异；
+- 覆盖有监督（如 FAQ 问答）、无监督（自动生成 QA 对，生成的问答对语料可以通过无监督的方式构建检索式问答系统）等多种情况，适用各类业务场景。
+
+<div align="center">
+    <img src="https://user-images.githubusercontent.com/11793384/168514868-1babe981-c675-4f89-9168-dd0a3eede315.gif" width="60%" length="60%">
+</div>
+
+
+详细使用说明请参考[智能问答系统](./question_answering)与[文档智能问答](./document_intelligence/doc_vqa)。
+
+#### 📚 通用文本分类
+
+- 基于“任务架构统一、通用能力共享”的通用文本分类技术 UTC，实了良好的零/少样本迁移能力，实现大一统诸多任务的开放域分类，可支持情感分析、意图识别、语义匹配、蕴含推理等各种可转换为分类问题的 NLU 任务。
+
+<div align="center">
+    <img src="https://user-images.githubusercontent.com/11793384/213347595-e9c08bd1-3d32-4519-9a52-31fb69b841e8.png" width="60%" length="60%">
+</div>
+
+<br>
+
+详细使用说明请参考[通用文本分类](./zero_shot_text_classification)，更多：[文章解读](https://mp.weixin.qq.com/s/VV-nYv4y1r7oipJnURRL5w)。
+
+
+#### 🗂 文本分类
+
+- 场景方案全覆盖––––开源预训练模型-微调、提示学习、基于语义索引等多种分类技术方案，满足不同场景需求，涵盖多分类（multi-class）、多标签（multi-label）、层次分类（hierarchical）三类任务；
+- 模型高效调优––––强强结合数据增强能力与可信增强技术，解决脏数据、标注数据欠缺、数据不平衡等问题，大幅提升模型效果。
+
+<div align="center">
+    <img src="https://user-images.githubusercontent.com/63761690/186378697-630d3590-4e67-49a0-8d5f-7cabd9daa894.png" width="60%" length="60%">
+</div>
+
+<br>
+
+详细使用说明请参考[文本分类](./text_classification)，更多：[文章解读](https://mp.weixin.qq.com/s/tas7yM8vapxwtlJt-MRZdg)。
+
+#### 💌 评论观点抽取与情感分析
+
+- 经典方案：基于情感知识增强预训练模型SKEP，两阶段式抽取和分类，首先通过序列标注的方式定位属性词和观点词，然后进行属性集情感分类；
+- 前沿方案：基于UIE的情感分析方案采用 Prompt Learning 的方式进行情感信息抽取，精度更高。支持语句级和属性级情感分析，解决同义属性聚合、隐性观点抽取难点，并提供可视化分析能力。
+
+<div align="center">
+    <img src="https://user-images.githubusercontent.com/35913314/200259473-434888f7-c0ac-4253-ab23-ede1628e6ba2.png" width="60%" length="60%">
+</div>
+<br>
+
+详细使用说明请参考[情感分析](./sentiment_analysis)，更多：[文章解读](https://mp.weixin.qq.com/s/QAHjIRG9zxpYfM6YPRQ-9w)。
+
+#### 🎙️ 智能语音指令解析
+
+- 集成了[PaddleSpeech](https://github.com/PaddlePaddle/PaddleSpeech)和[百度开放平台](https://ai.baidu.com/)的语音识别和[UIE](./model_zoo/uie)通用信息抽取等技术，打造智能一体化的语音指令解析系统范例，该方案可应用于智能语音填单、智能语音交互、智能语音检索等场景，提高人机交互效率。
+
+<div align="center">
+    <img src="https://user-images.githubusercontent.com/16698950/168589100-a6c6f346-97bb-47b2-ac26-8d50e71fddc5.png" width="400">
+</div>
+
+详细使用说明请参考[智能语音指令解析](./applications/speech_cmd_analysis)。

applications/document_intelligence/doc_vqa/Extraction/docvqa.py

@@ -12,30 +12,20 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-import os
-import cv2
-import sys
-import json
-import copy
 import collections
-import numpy as np
-from tqdm import tqdm
+import json
+import sys
 
+import numpy as np
 import paddle
 from paddle.io import Dataset
+from tqdm import tqdm
 
 sys.path.insert(0, "../")
 
 
 class DocVQAExample(object):
-
-    def __init__(self,
-                 question,
-                 doc_tokens,
-                 doc_boxes=[],
-                 answer=None,
-                 labels=None,
-                 image=None):
+    def __init__(self, question, doc_tokens, doc_boxes=[], answer=None, labels=None, image=None):
         self.question = question
         self.doc_tokens = doc_tokens
         self.doc_boxes = doc_boxes
@@ -47,13 +37,7 @@ class DocVQAExample(object):
 class DocVQAFeatures(object):
     """A single set of features of data."""
 
-    def __init__(self,
-                 example_index,
-                 input_ids,
-                 input_mask,
-                 segment_ids,
-                 boxes=None,
-                 label=None):
+    def __init__(self, example_index, input_ids, input_mask, segment_ids, boxes=None, label=None):
         self.example_index = example_index
         self.input_ids = input_ids
         self.input_mask = input_mask
@@ -63,15 +47,9 @@ class DocVQAFeatures(object):
 
 
 class DocVQA(Dataset):
-
-    def __init__(self,
-                 args,
-                 tokenizer,
-                 label2id_map,
-                 max_seq_len=512,
-                 max_query_length=20,
-                 max_doc_length=512,
-                 max_span_num=1):
+    def __init__(
+        self, args, tokenizer, label2id_map, max_seq_len=512, max_query_length=20, max_doc_length=512, max_span_num=1
+    ):
         super(DocVQA, self).__init__()
         self.tokenizer = tokenizer
         self.label2id_map = label2id_map
@@ -113,17 +91,16 @@ class DocVQA(Dataset):
                 continue
             num_left_context = position - doc_span.start
             num_right_context = end - position
-            score = min(num_left_context,
-                        num_right_context) + 0.01 * doc_span.length
+            score = min(num_left_context, num_right_context) + 0.01 * doc_span.length
             if best_score is None or score > best_score:
                 best_score = score
             best_span_index = span_index
 
         return cur_span_index == best_span_index
 
-    def convert_examples_to_features(self, examples, tokenizer, label_map,
-                                     max_seq_length, max_span_num,
-                                     max_doc_length, max_query_length):
+    def convert_examples_to_features(
+        self, examples, tokenizer, label_map, max_seq_length, max_span_num, max_doc_length, max_query_length
+    ):
 
         if "[CLS]" in self.tokenizer.get_vocab():
             start_token = "[CLS]"
@@ -133,7 +110,6 @@ class DocVQA(Dataset):
             end_token = "</s>"
 
         features = []
-        total = len(examples)
         for (example_index, example) in enumerate(examples):
             query_tokens = tokenizer.tokenize(example.question)
             if len(query_tokens) > max_query_length:
@@ -188,8 +164,7 @@ class DocVQA(Dataset):
                 segment_ids.append(0)
                 for i in range(doc_span.length):
                     split_token_index = doc_span.start + i
-                    is_max_context = self.check_is_max_context(
-                        doc_spans, doc_span_index, split_token_index)
+                    is_max_context = self.check_is_max_context(doc_spans, doc_span_index, split_token_index)
                     token_is_max_context[len(tokens)] = is_max_context
                     tokens.append(all_doc_tokens[split_token_index])
                     boxes_tokens.append(all_doc_boxes_tokens[split_token_index])
@@ -292,12 +267,10 @@ class DocVQA(Dataset):
             question = sample["question"]
             doc_tokens = sample["document"]
             doc_boxes = sample["document_bbox"]
-            labels = sample['labels'] if not is_test else []
+            labels = sample["labels"] if not is_test else []
 
-            x_min, y_min = min(doc_boxes, key=lambda x: x[0])[0], min(
-                doc_boxes, key=lambda x: x[2])[2]
-            x_max, y_max = max(doc_boxes, key=lambda x: x[1])[1], max(
-                doc_boxes, key=lambda x: x[3])[3]
+            x_min, y_min = min(doc_boxes, key=lambda x: x[0])[0], min(doc_boxes, key=lambda x: x[2])[2]
+            x_max, y_max = max(doc_boxes, key=lambda x: x[1])[1], max(doc_boxes, key=lambda x: x[3])[3]
             width = x_max - x_min
             height = y_max - y_min
 
@@ -308,12 +281,15 @@ class DocVQA(Dataset):
                 scale_x = 1000 / max(width, height)
                 scale_y = 1000 / max(width, height)
 
-            scaled_doc_boxes = [[
-                round((b[0] - x_min) * scale_x),
-                round((b[2] - y_min) * scale_y),
-                round((b[1] - x_min) * scale_x),
-                round((b[3] - y_min) * scale_y)
-            ] for b in doc_boxes]
+            scaled_doc_boxes = [
+                [
+                    round((b[0] - x_min) * scale_x),
+                    round((b[2] - y_min) * scale_y),
+                    round((b[1] - x_min) * scale_x),
+                    round((b[3] - y_min) * scale_y),
+                ]
+                for b in doc_boxes
+            ]
 
             for box, oribox in zip(scaled_doc_boxes, doc_boxes):
                 if box[0] < 0:
@@ -326,10 +302,9 @@ class DocVQA(Dataset):
                     if pos > 1000:
                         print(width, height, box, oribox)
 
-            example = DocVQAExample(question=question,
-                                    doc_tokens=doc_tokens,
-                                    doc_boxes=scaled_doc_boxes,
-                                    labels=labels)
+            example = DocVQAExample(
+                question=question, doc_tokens=doc_tokens, doc_boxes=scaled_doc_boxes, labels=labels
+            )
             examples.append(example)
         return examples
 
@@ -339,7 +314,7 @@ class DocVQA(Dataset):
             dataset = self.args.train_file
         elif self.args.do_test:
             dataset = self.args.test_file
-        with open(dataset, 'r', encoding='utf8') as f:
+        with open(dataset, "r", encoding="utf8") as f:
             for index, line in enumerate(f):
                 data.append(json.loads(line.strip()))
 
@@ -353,30 +328,32 @@ class DocVQA(Dataset):
             max_seq_length=self.max_seq_len,
             max_doc_length=self.max_doc_length,
             max_span_num=self.max_span_num,
-            max_query_length=self.max_query_length)
+            max_query_length=self.max_query_length,
+        )
 
-        all_input_ids = paddle.to_tensor([f.input_ids for f in features],
-                                         dtype="int64")
-        all_input_mask = paddle.to_tensor([f.input_mask for f in features],
-                                          dtype="int64")
-        all_segment_ids = paddle.to_tensor([f.segment_ids for f in features],
-                                           dtype="int64")
-        all_bboxes = paddle.to_tensor([f.boxes for f in features],
-                                      dtype="int64")
-        all_labels = paddle.to_tensor([f.label for f in features],
-                                      dtype="int64")
+        all_input_ids = paddle.to_tensor([f.input_ids for f in features], dtype="int64")
+        all_input_mask = paddle.to_tensor([f.input_mask for f in features], dtype="int64")
+        all_segment_ids = paddle.to_tensor([f.segment_ids for f in features], dtype="int64")
+        all_bboxes = paddle.to_tensor([f.boxes for f in features], dtype="int64")
+        all_labels = paddle.to_tensor([f.label for f in features], dtype="int64")
         self.sample_list = [
             np.array(all_input_ids),
             np.array(all_input_mask),
             np.array(all_segment_ids),
             np.array(all_bboxes),
-            np.array(all_labels)
+            np.array(all_labels),
         ]
 
     def __getitem__(self, idx):
-        return self.sample_list[0][idx], self.sample_list[1][
-            idx], self.sample_list[2][idx], self.sample_list[3][
-                idx], self.sample_list[4][idx]
+        return (
+            self.sample_list[0][idx],
+            self.sample_list[1][idx],
+            self.sample_list[2][idx],
+            self.sample_list[3][idx],
+            self.sample_list[4][idx],
+        )
 
-    def __len__(self, ):
+    def __len__(
+        self,
+    ):
         return self.sample_list[0].shape[0]

applications/document_intelligence/doc_vqa/Extraction/model.py

@@ -12,17 +12,15 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-import numpy as np
-
 import paddle
-import paddle.nn as nn
 import paddle.fluid as fluid
+import paddle.nn as nn
+
 from paddlenlp.transformers import LayoutXLMPretrainedModel
 
 
 class Crf_decoding(paddle.fluid.dygraph.Layer):
-
-    def __init__(self, param_attr, size=None, is_test=True, dtype='float32'):
+    def __init__(self, param_attr, size=None, is_test=True, dtype="float32"):
         super(Crf_decoding, self).__init__()
 
         self._dtype = dtype
@@ -30,9 +28,8 @@ class Crf_decoding(paddle.fluid.dygraph.Layer):
         self._is_test = is_test
         self._param_attr = param_attr
         self._transition = self.create_parameter(
-            attr=self._param_attr,
-            shape=[self._size + 2, self._size],
-            dtype=self._dtype)
+            attr=self._param_attr, shape=[self._size + 2, self._size], dtype=self._dtype
+        )
 
     @property
     def weight(self):
@@ -44,30 +41,23 @@ class Crf_decoding(paddle.fluid.dygraph.Layer):
 
     def forward(self, input, label=None, length=None):
 
-        viterbi_path = self._helper.create_variable_for_type_inference(
-            dtype=self._dtype)
-        this_inputs = {
-            "Emission": [input],
-            "Transition": self._transition,
-            "Label": label
-        }
+        viterbi_path = self._helper.create_variable_for_type_inference(dtype=self._dtype)
+        this_inputs = {"Emission": [input], "Transition": self._transition, "Label": label}
         if length is not None:
-            this_inputs['Length'] = [length]
-        self._helper.append_op(type='crf_decoding',
-                               inputs=this_inputs,
-                               outputs={"ViterbiPath": [viterbi_path]},
-                               attrs={
-                                   "is_test": self._is_test,
-                               })
+            this_inputs["Length"] = [length]
+        self._helper.append_op(
+            type="crf_decoding",
+            inputs=this_inputs,
+            outputs={"ViterbiPath": [viterbi_path]},
+            attrs={
+                "is_test": self._is_test,
+            },
+        )
         return viterbi_path
 
 
 class Chunk_eval(paddle.fluid.dygraph.Layer):
-
-    def __init__(self,
-                 num_chunk_types,
-                 chunk_scheme,
-                 excluded_chunk_types=None):
+    def __init__(self, num_chunk_types, chunk_scheme, excluded_chunk_types=None):
         super(Chunk_eval, self).__init__()
         self.num_chunk_types = num_chunk_types
         self.chunk_scheme = chunk_scheme
@@ -75,48 +65,39 @@ class Chunk_eval(paddle.fluid.dygraph.Layer):
 
     def forward(self, input, label, seq_length=None):
 
-        precision = self._helper.create_variable_for_type_inference(
-            dtype="float32")
-        recall = self._helper.create_variable_for_type_inference(
-            dtype="float32")
-        f1_score = self._helper.create_variable_for_type_inference(
-            dtype="float32")
-        num_infer_chunks = self._helper.create_variable_for_type_inference(
-            dtype="int64")
-        num_label_chunks = self._helper.create_variable_for_type_inference(
-            dtype="int64")
-        num_correct_chunks = self._helper.create_variable_for_type_inference(
-            dtype="int64")
+        precision = self._helper.create_variable_for_type_inference(dtype="float32")
+        recall = self._helper.create_variable_for_type_inference(dtype="float32")
+        f1_score = self._helper.create_variable_for_type_inference(dtype="float32")
+        num_infer_chunks = self._helper.create_variable_for_type_inference(dtype="int64")
+        num_label_chunks = self._helper.create_variable_for_type_inference(dtype="int64")
+        num_correct_chunks = self._helper.create_variable_for_type_inference(dtype="int64")
 
         this_input = {"Inference": [input], "Label": [label]}
         if seq_length is not None:
             this_input["SeqLength"] = [seq_length]
 
-        self._helper.append_op(type='chunk_eval',
-                               inputs=this_input,
-                               outputs={
-                                   "Precision": [precision],
-                                   "Recall": [recall],
-                                   "F1-Score": [f1_score],
-                                   "NumInferChunks": [num_infer_chunks],
-                                   "NumLabelChunks": [num_label_chunks],
-                                   "NumCorrectChunks": [num_correct_chunks]
-                               },
-                               attrs={
-                                   "num_chunk_types":
-                                   self.num_chunk_types,
-                                   "chunk_scheme":
-                                   self.chunk_scheme,
-                                   "excluded_chunk_types":
-                                   self.excluded_chunk_types or []
-                               })
-        return (precision, recall, f1_score, num_infer_chunks, num_label_chunks,
-                num_correct_chunks)
+        self._helper.append_op(
+            type="chunk_eval",
+            inputs=this_input,
+            outputs={
+                "Precision": [precision],
+                "Recall": [recall],
+                "F1-Score": [f1_score],
+                "NumInferChunks": [num_infer_chunks],
+                "NumLabelChunks": [num_label_chunks],
+                "NumCorrectChunks": [num_correct_chunks],
+            },
+            attrs={
+                "num_chunk_types": self.num_chunk_types,
+                "chunk_scheme": self.chunk_scheme,
+                "excluded_chunk_types": self.excluded_chunk_types or [],
+            },
+        )
+        return (precision, recall, f1_score, num_infer_chunks, num_label_chunks, num_correct_chunks)
 
 
 class Linear_chain_crf(paddle.fluid.dygraph.Layer):
-
-    def __init__(self, param_attr, size=None, is_test=False, dtype='float32'):
+    def __init__(self, param_attr, size=None, is_test=False, dtype="float32"):
         super(Linear_chain_crf, self).__init__()
 
         self._param_attr = param_attr
@@ -124,9 +105,8 @@ class Linear_chain_crf(paddle.fluid.dygraph.Layer):
         self._size = size
         self._is_test = is_test
         self._transition = self.create_parameter(
-            attr=self._param_attr,
-            shape=[self._size + 2, self._size],
-            dtype=self._dtype)
+            attr=self._param_attr, shape=[self._size + 2, self._size], dtype=self._dtype
+        )
 
     @property
     def weight(self):
@@ -138,78 +118,64 @@ class Linear_chain_crf(paddle.fluid.dygraph.Layer):
 
     def forward(self, input, label, length=None):
 
-        alpha = self._helper.create_variable_for_type_inference(
-            dtype=self._dtype)
-        emission_exps = self._helper.create_variable_for_type_inference(
-            dtype=self._dtype)
-        transition_exps = self._helper.create_variable_for_type_inference(
-            dtype=self._dtype)
-        log_likelihood = self._helper.create_variable_for_type_inference(
-            dtype=self._dtype)
-        this_inputs = {
-            "Emission": [input],
-            "Transition": self._transition,
-            "Label": [label]
-        }
+        alpha = self._helper.create_variable_for_type_inference(dtype=self._dtype)
+        emission_exps = self._helper.create_variable_for_type_inference(dtype=self._dtype)
+        transition_exps = self._helper.create_variable_for_type_inference(dtype=self._dtype)
+        log_likelihood = self._helper.create_variable_for_type_inference(dtype=self._dtype)
+        this_inputs = {"Emission": [input], "Transition": self._transition, "Label": [label]}
         if length is not None:
-            this_inputs['Length'] = [length]
-        self._helper.append_op(type='linear_chain_crf',
-                               inputs=this_inputs,
-                               outputs={
-                                   "Alpha": [alpha],
-                                   "EmissionExps": [emission_exps],
-                                   "TransitionExps": transition_exps,
-                                   "LogLikelihood": log_likelihood
-                               },
-                               attrs={
-                                   "is_test": self._is_test,
-                               })
+            this_inputs["Length"] = [length]
+        self._helper.append_op(
+            type="linear_chain_crf",
+            inputs=this_inputs,
+            outputs={
+                "Alpha": [alpha],
+                "EmissionExps": [emission_exps],
+                "TransitionExps": transition_exps,
+                "LogLikelihood": log_likelihood,
+            },
+            attrs={
+                "is_test": self._is_test,
+            },
+        )
         return log_likelihood
 
 
 class LayoutXLMForTokenClassification_with_CRF(LayoutXLMPretrainedModel):
-
     def __init__(self, layoutxlm, num_classes, dropout=None):
         super(LayoutXLMForTokenClassification_with_CRF, self).__init__()
         self.num_classes = num_classes
-        if isinstance(layoutxlm, dict):
-            self.layoutxlm = LayoutXLMModel(**layoutxlm)
-        else:
-            self.layoutxlm = layoutxlm
-        self.dropout = nn.Dropout(dropout if dropout is not None else self.
-                                  layoutxlm.config["hidden_dropout_prob"])
-        self.emission_classifier = nn.Linear(
-            self.layoutxlm.config["hidden_size"], self.num_classes)
+        self.layoutxlm = layoutxlm
+        self.dropout = nn.Dropout(dropout if dropout is not None else self.layoutxlm.config["hidden_dropout_prob"])
+        self.emission_classifier = nn.Linear(self.layoutxlm.config["hidden_size"], self.num_classes)
         self.emission_classifier.apply(self.init_weights)
         self.linear_chain_crf = Linear_chain_crf(
-            size=self.num_classes,
-            param_attr=paddle.fluid.ParamAttr(name='liner_chain_crfw'))
-        self.crf_decoding = Crf_decoding(
-            param_attr=paddle.fluid.ParamAttr(name='crfw_decode'),
-            size=self.num_classes)
+            size=self.num_classes, param_attr=paddle.fluid.ParamAttr(name="liner_chain_crfw")
+        )
+        self.crf_decoding = Crf_decoding(param_attr=paddle.fluid.ParamAttr(name="crfw_decode"), size=self.num_classes)
         self.crf_decoding.weight = self.linear_chain_crf.weight
         self.crfw = fluid.layers.create_parameter(
-            shape=[self.num_classes + 2, self.num_classes],
-            dtype='float32',
-            name='crfw')
+            shape=[self.num_classes + 2, self.num_classes], dtype="float32", name="crfw"
+        )
         self.mask_crfw = fluid.layers.create_parameter(
-            shape=[self.num_classes + 2, self.num_classes],
-            dtype='float32',
-            name='mask_matrix')
+            shape=[self.num_classes + 2, self.num_classes], dtype="float32", name="mask_matrix"
+        )
 
     def get_input_embeddings(self):
         return self.layoutxlm.embeddings.word_embeddings
 
-    def forward(self,
-                input_ids=None,
-                bbox=None,
-                attention_mask=None,
-                token_type_ids=None,
-                labels=None,
-                image=None,
-                position_ids=None,
-                head_mask=None,
-                is_train=False):
+    def forward(
+        self,
+        input_ids=None,
+        bbox=None,
+        attention_mask=None,
+        token_type_ids=None,
+        labels=None,
+        image=None,
+        position_ids=None,
+        head_mask=None,
+        is_train=False,
+    ):
 
         input_ids = input_ids.squeeze(axis=1)
         bbox = bbox.squeeze(axis=1)
@@ -226,16 +192,13 @@ class LayoutXLMForTokenClassification_with_CRF(LayoutXLMPretrainedModel):
         )
         seq_length = input_ids.shape[1]
         # sequence out and image out
-        sequence_logits, image_output = outputs[0][:, :seq_length], outputs[
-            0][:, seq_length:]
+        sequence_logits, _ = outputs[0][:, :seq_length], outputs[0][:, seq_length:]
         emission = self.emission_classifier(sequence_logits)
         length = paddle.sum(attention_mask, axis=1)
         labels = labels.reshape([-1, seq_length, 1])
 
         # standard crf loss
-        crf_cost = self.linear_chain_crf(input=emission,
-                                         label=labels,
-                                         length=length)
+        crf_cost = self.linear_chain_crf(input=emission, label=labels, length=length)
         crf_decode = self.crf_decoding(input=emission, length=length)
         if is_train:
             return [crf_cost]

applications/document_intelligence/doc_vqa/Extraction/run_docvqa.py

@@ -12,25 +12,21 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-import os
-import sys
-import copy
-import json
-import random
-import logging
-import warnings
 import argparse
+import json
+import logging
+import os
+import random
+import warnings
+from collections import Counter
+
 import numpy as np
-from collections import OrderedDict, Counter
-
 import paddle
-from paddle.static import InputSpec
-from paddle.jit import to_static
-from paddlenlp.transformers import LayoutXLMModel, LayoutXLMTokenizer
-
 from docvqa import DocVQA
 from model import LayoutXLMForTokenClassification_with_CRF
 
+from paddlenlp.transformers import LayoutXLMModel, LayoutXLMTokenizer
+
 warnings.filterwarnings("ignore")
 logger = logging.getLogger(__name__)
 
@@ -81,12 +77,10 @@ def get_label_maps():
 def main(args):
     os.makedirs(args.output_dir, exist_ok=True)
     logging.basicConfig(
-        filename=os.path.join(args.output_dir, "train.log")
-        if paddle.distributed.get_rank() == 0 else None,
+        filename=os.path.join(args.output_dir, "train.log") if paddle.distributed.get_rank() == 0 else None,
         format="%(asctime)s - %(levelname)s - %(name)s -   %(message)s",
         datefmt="%m/%d/%Y %H:%M:%S",
-        level=logging.INFO
-        if paddle.distributed.get_rank() == 0 else logging.WARN,
+        level=logging.INFO if paddle.distributed.get_rank() == 0 else logging.WARN,
     )
 
     ch = logging.StreamHandler()
@@ -103,74 +97,62 @@ def main(args):
     tokenizer = LayoutXLMTokenizer.from_pretrained(args.model_name_or_path)
 
     if args.do_test:
-        model = LayoutXLMForTokenClassification_with_CRF.from_pretrained(
-            args.init_checkpoint)
-        evaluate(args,
-                 model,
-                 tokenizer,
-                 label2id_map,
-                 id2label_map,
-                 pad_token_label_id,
-                 global_step=0)
+        model = LayoutXLMForTokenClassification_with_CRF.from_pretrained(args.init_checkpoint)
+        evaluate(args, model, tokenizer, label2id_map, id2label_map, pad_token_label_id, global_step=0)
         exit(0)
 
     if args.init_checkpoint:
-        logger.info('Init checkpoint from {}'.format(args.init_checkpoint))
-        model = LayoutXLMForTokenClassification_with_CRF.from_pretrained(
-            args.init_checkpoint)
+        logger.info("Init checkpoint from {}".format(args.init_checkpoint))
+        model = LayoutXLMForTokenClassification_with_CRF.from_pretrained(args.init_checkpoint)
     else:
         base_model = LayoutXLMModel.from_pretrained(args.model_name_or_path)
-        model = LayoutXLMForTokenClassification_with_CRF(
-            base_model, num_classes=len(label2id_map), dropout=None)
+        model = LayoutXLMForTokenClassification_with_CRF(base_model, num_classes=len(label2id_map), dropout=None)
 
     # dist mode
     if paddle.distributed.get_world_size() > 1:
         model = paddle.DataParallel(model)
 
-    train_dataset = DocVQA(args,
-                           tokenizer,
-                           label2id_map,
-                           max_seq_len=args.max_seq_len,
-                           max_query_length=args.max_query_length,
-                           max_doc_length=args.max_doc_length,
-                           max_span_num=args.max_span_num)
+    train_dataset = DocVQA(
+        args,
+        tokenizer,
+        label2id_map,
+        max_seq_len=args.max_seq_len,
+        max_query_length=args.max_query_length,
+        max_doc_length=args.max_doc_length,
+        max_span_num=args.max_span_num,
+    )
 
     train_sampler = paddle.io.DistributedBatchSampler(
-        train_dataset, batch_size=args.per_gpu_train_batch_size, shuffle=False)
+        train_dataset, batch_size=args.per_gpu_train_batch_size, shuffle=False
+    )
 
-    args.train_batch_size = args.per_gpu_train_batch_size * max(
-        1, paddle.distributed.get_world_size())
+    args.train_batch_size = args.per_gpu_train_batch_size * max(1, paddle.distributed.get_world_size())
 
-    train_dataloader = paddle.io.DataLoader(train_dataset,
-                                            batch_sampler=train_sampler,
-                                            num_workers=0,
-                                            use_shared_memory=True,
-                                            collate_fn=None)
+    train_dataloader = paddle.io.DataLoader(
+        train_dataset, batch_sampler=train_sampler, num_workers=0, use_shared_memory=True, collate_fn=None
+    )
 
     t_total = len(train_dataloader) * args.num_train_epochs
     # build linear decay with warmup lr sch
     lr_scheduler = paddle.optimizer.lr.PolynomialDecay(
-        learning_rate=args.learning_rate,
-        decay_steps=t_total,
-        end_lr=0.0,
-        power=1.0)
+        learning_rate=args.learning_rate, decay_steps=t_total, end_lr=0.0, power=1.0
+    )
     if args.warmup_steps > 0:
         lr_scheduler = paddle.optimizer.lr.LinearWarmup(
-            lr_scheduler,
-            args.warmup_steps,
-            start_lr=0,
-            end_lr=args.learning_rate)
+            lr_scheduler, args.warmup_steps, start_lr=0, end_lr=args.learning_rate
+        )
 
-    optimizer = paddle.optimizer.AdamW(learning_rate=lr_scheduler,
-                                       parameters=model.parameters(),
-                                       epsilon=args.adam_epsilon,
-                                       weight_decay=args.weight_decay)
+    optimizer = paddle.optimizer.AdamW(
+        learning_rate=lr_scheduler,
+        parameters=model.parameters(),
+        epsilon=args.adam_epsilon,
+        weight_decay=args.weight_decay,
+    )
 
     logger.info("***** Running training *****")
     logger.info("  Num examples = %d", len(train_dataset))
     logger.info("  Num Epochs = %d", args.num_train_epochs)
-    logger.info("  Instantaneous batch size per GPU = %d",
-                args.per_gpu_train_batch_size)
+    logger.info("  Instantaneous batch size per GPU = %d", args.per_gpu_train_batch_size)
     logger.info(
         "  Total train batch size (w. parallel, distributed) = %d",
         args.train_batch_size * paddle.distributed.get_world_size(),
@@ -180,29 +162,35 @@ def main(args):
     global_step = 0
     tr_loss = 0.0
     set_seed(args)
-    best_metrics = None
     for epoch_id in range(args.num_train_epochs):
-        print('epoch id:{}'.format(epoch_id))
+        print("epoch id:{}".format(epoch_id))
         for step, batch in enumerate(train_dataloader):
             model.train()
             input_ids, input_mask, segment_ids, bboxes, labels = batch
             if input_ids.shape[0] != args.per_gpu_train_batch_size:
                 continue
-            outputs = model(input_ids=input_ids,
-                            bbox=bboxes,
-                            attention_mask=input_mask,
-                            token_type_ids=segment_ids,
-                            labels=labels,
-                            is_train=True)
+            outputs = model(
+                input_ids=input_ids,
+                bbox=bboxes,
+                attention_mask=input_mask,
+                token_type_ids=segment_ids,
+                labels=labels,
+                is_train=True,
+            )
             # model outputs are always tuple in paddlenlp (see doc)
             loss = outputs[0]
             loss = loss.mean()
             if global_step % 50 == 0:
                 logger.info(
-                    "[epoch {}/{}][iter: {}/{}] lr: {:.5f}, train loss: {:.5f}, "
-                    .format(epoch_id, args.num_train_epochs, step,
-                            len(train_dataloader), lr_scheduler.get_lr(),
-                            loss.numpy()[0]))
+                    "[epoch {}/{}][iter: {}/{}] lr: {:.5f}, train loss: {:.5f}, ".format(
+                        epoch_id,
+                        args.num_train_epochs,
+                        step,
+                        len(train_dataloader),
+                        lr_scheduler.get_lr(),
+                        float(loss),
+                    )
+                )
 
             loss.backward()
             tr_loss += loss.item()
@@ -211,17 +199,14 @@ def main(args):
             optimizer.clear_grad()
             global_step += 1
 
-            if paddle.distributed.get_rank(
-            ) == 0 and args.save_steps > 0 and global_step % args.save_steps == 0:
+            if paddle.distributed.get_rank() == 0 and args.save_steps > 0 and global_step % args.save_steps == 0:
                 # Save model checkpoint
-                output_dir = os.path.join(args.output_dir,
-                                          "checkpoint-{}".format(global_step))
+                output_dir = os.path.join(args.output_dir, "checkpoint-{}".format(global_step))
                 os.makedirs(output_dir, exist_ok=True)
                 if paddle.distributed.get_rank() == 0:
                     model.save_pretrained(output_dir)
                     tokenizer.save_pretrained(output_dir)
-                    paddle.save(args,
-                                os.path.join(output_dir, "training_args.bin"))
+                    paddle.save(args, os.path.join(output_dir, "training_args.bin"))
                     logger.info("Saving model checkpoint to %s", output_dir)
 
 
@@ -242,14 +227,16 @@ def _tokenize_chinese_chars(text):
         # as is Japanese Hiragana and Katakana. Those alphabets are used to write
         # space-separated words, so they are not treated specially and handled
         # like the all of the other languages.
-        if ((cp >= 0x4E00 and cp <= 0x9FFF) or  #
-            (cp >= 0x3400 and cp <= 0x4DBF) or  #
-            (cp >= 0x20000 and cp <= 0x2A6DF) or  #
-            (cp >= 0x2A700 and cp <= 0x2B73F) or  #
-            (cp >= 0x2B740 and cp <= 0x2B81F) or  #
-            (cp >= 0x2B820 and cp <= 0x2CEAF) or
-            (cp >= 0xF900 and cp <= 0xFAFF) or  #
-            (cp >= 0x2F800 and cp <= 0x2FA1F)):  #
+        if (
+            (cp >= 0x4E00 and cp <= 0x9FFF)
+            or (cp >= 0x3400 and cp <= 0x4DBF)  #
+            or (cp >= 0x20000 and cp <= 0x2A6DF)  #
+            or (cp >= 0x2A700 and cp <= 0x2B73F)  #
+            or (cp >= 0x2B740 and cp <= 0x2B81F)  #
+            or (cp >= 0x2B820 and cp <= 0x2CEAF)  #
+            or (cp >= 0xF900 and cp <= 0xFAFF)
+            or (cp >= 0x2F800 and cp <= 0x2FA1F)  #
+        ):  #
             return True
 
         return False
@@ -290,9 +277,35 @@ def _normalize(in_str):
     """
     in_str = in_str.lower()
     sp_char = [
-        u':', u'_', u'`', u'，', u'。', u'：', u'？', u'！', u'(', u')', u'“', u'”',
-        u'；', u'’', u'《', u'》', u'……', u'·', u'、', u',', u'「', u'」', u'（', u'）',
-        u'－', u'～', u'『', u'』', '|'
+        ":",
+        "_",
+        "`",
+        "，",
+        "。",
+        "：",
+        "？",
+        "！",
+        "(",
+        ")",
+        "“",
+        "”",
+        "；",
+        "’",
+        "《",
+        "》",
+        "……",
+        "·",
+        "、",
+        ",",
+        "「",
+        "」",
+        "（",
+        "）",
+        "－",
+        "～",
+        "『",
+        "』",
+        "|",
     ]
     out_segs = []
     for char in in_str:
@@ -300,11 +313,10 @@ def _normalize(in_str):
             continue
         else:
             out_segs.append(char)
-    return ''.join(out_segs)
+    return "".join(out_segs)
 
 
 def calc_f1_score(answer, prediction):
-    f1_scores = []
     ans_segs = _tokenize_chinese_chars(_normalize(answer))
     prediction_segs = _tokenize_chinese_chars(_normalize(prediction))
     f1 = fast_f1(prediction_segs, ans_segs)
@@ -318,38 +330,35 @@ def decode(tokenizer, res):
     save_f1 = []
     for i in range(len(res)):
         input_ids, label_ids, predict_ids, bbox = res[i]
-        remove_pos = len(' '.join(
-            [str(x) for x in input_ids]).split('2 6 ')[0].strip(' ').split(
-                ' ')) + 2  # remove the question bbox and sep bbox
+        remove_pos = (
+            len(" ".join([str(x) for x in input_ids]).split("2 6 ")[0].strip(" ").split(" ")) + 2
+        )  # remove the question bbox and sep bbox
         start_pos = input_ids.index(sep_id)
         query_text = []
         for idx in range(1, start_pos):
             input_id = input_ids[idx]
             query_text.append(tokenizer._convert_id_to_token(int(input_id)))
 
-        #label texts and predict texts
+        # label texts and predict texts
         text_label, text_predict = [], []
         label_bbox_index, predict_bbox_index = [], []
         for idx in range(start_pos + 1, len(input_ids)):
-            input_id, label_id, predict_id = input_ids[idx], label_ids[
-                idx], predict_ids[idx]
+            input_id, label_id, predict_id = input_ids[idx], label_ids[idx], predict_ids[idx]
 
             if label_id in [1, 2, 3]:
                 text_label.append(tokenizer._convert_id_to_token(int(input_id)))
                 label_bbox_index.append(idx - remove_pos + 1)
             if predict_id in [1, 2, 3]:
-                text_predict.append(
-                    tokenizer._convert_id_to_token(int(input_id)))
+                text_predict.append(tokenizer._convert_id_to_token(int(input_id)))
                 predict_bbox_index.append(idx - remove_pos + 1)
-        text_res.append([
-            ''.join(query_text), ''.join(text_label), ''.join(text_predict),
-            label_bbox_index, predict_bbox_index
-        ])
+        text_res.append(
+            ["".join(query_text), "".join(text_label), "".join(text_predict), label_bbox_index, predict_bbox_index]
+        )
 
-        f1 = calc_f1_score(''.join(text_label), ''.join(text_predict))
+        f1 = calc_f1_score("".join(text_label), "".join(text_predict))
         save_f1.append(f1)
 
-        if len(''.join(text_label)) > 10:
+        if len("".join(text_label)) > 10:
             all_f1.append(f1)
     if len(all_f1) > 0:
         print("F1: ", sum(all_f1) / len(all_f1))
@@ -358,31 +367,17 @@ def decode(tokenizer, res):
     return text_res
 
 
-def evaluate(args,
-             model,
-             tokenizer,
-             label2id_map,
-             id2label_map,
-             pad_token_label_id,
-             prefix="",
-             global_step=0):
+def evaluate(args, model, tokenizer, label2id_map, id2label_map, pad_token_label_id, prefix="", global_step=0):
 
-    eval_dataset = DocVQA(args,
-                          tokenizer,
-                          label2id_map,
-                          max_seq_len=512,
-                          max_query_length=20,
-                          max_doc_length=512,
-                          max_span_num=1)
+    eval_dataset = DocVQA(
+        args, tokenizer, label2id_map, max_seq_len=512, max_query_length=20, max_doc_length=512, max_span_num=1
+    )
 
-    args.eval_batch_size = args.per_gpu_eval_batch_size * max(
-        1, paddle.distributed.get_world_size())
+    args.eval_batch_size = args.per_gpu_eval_batch_size * max(1, paddle.distributed.get_world_size())
 
-    eval_dataloader = paddle.io.DataLoader(eval_dataset,
-                                           batch_size=args.eval_batch_size,
-                                           num_workers=0,
-                                           use_shared_memory=True,
-                                           collate_fn=None)
+    eval_dataloader = paddle.io.DataLoader(
+        eval_dataset, batch_size=args.eval_batch_size, num_workers=0, use_shared_memory=True, collate_fn=None
+    )
 
     # Eval!
     logger.info("***** Running evaluation %s *****", prefix)
@@ -396,66 +391,64 @@ def evaluate(args,
 
             if input_ids.shape[0] != args.eval_batch_size:
                 continue
-            outputs = model(input_ids=input_ids,
-                            bbox=bboxes,
-                            attention_mask=input_mask,
-                            token_type_ids=segment_ids,
-                            labels=labels,
-                            is_train=False)
+            outputs = model(
+                input_ids=input_ids,
+                bbox=bboxes,
+                attention_mask=input_mask,
+                token_type_ids=segment_ids,
+                labels=labels,
+                is_train=False,
+            )
             labels = labels.numpy()
             crf_decode = outputs[1].numpy()
             bboxes = bboxes.squeeze().numpy()
             input_ids = input_ids.squeeze(axis=1).numpy()
 
             for index in range(input_ids.shape[0]):
-                res.append([
-                    list(input_ids[index]),
-                    list(labels[index]),
-                    list(crf_decode[index]), bboxes[index]
-                ])
+                res.append([list(input_ids[index]), list(labels[index]), list(crf_decode[index]), bboxes[index]])
 
     origin_inputs = []
-    with open(args.test_file, 'r', encoding='utf8') as f:
+    with open(args.test_file, "r", encoding="utf8") as f:
         for line in f:
             line = json.loads(line.strip())
-            origin_inputs.append({
-                'img_name': line['img_name'],
-                'question': line['question'],
-                'bboxes': line['document_bbox'],
-                'img_id': line['img_id']
-            })
+            origin_inputs.append(
+                {
+                    "img_name": line["img_name"],
+                    "question": line["question"],
+                    "bboxes": line["document_bbox"],
+                    "img_id": line["img_id"],
+                }
+            )
 
     text_res = decode(tokenizer, res)
 
-    with open(args.save_path, 'w', encoding='utf8') as f:
-        for line_res, line_text, line_label in zip(res, text_res,
-                                                   origin_inputs):
+    with open(args.save_path, "w", encoding="utf8") as f:
+        for line_res, line_text, line_label in zip(res, text_res, origin_inputs):
             line_json = {}
-            line_json['img_name'] = line_label['img_name']
-            line_json['img_id'] = line_label['img_id']
-            line_json['question'] = line_label['question']
-            line_json['label_answer'] = line_text[1]
-            line_json['predict_answer'] = line_text[2]
-            all_boxes = line_res[3]
+            line_json["img_name"] = line_label["img_name"]
+            line_json["img_id"] = line_label["img_id"]
+            line_json["question"] = line_label["question"]
+            line_json["label_answer"] = line_text[1]
+            line_json["predict_answer"] = line_text[2]
             label_bbox_index, predict_bbox_index = line_text[3], line_text[4]
             label_bboxes, predict_bboxes = [], []
-            for i in range(len(line_label['bboxes'])):
+            for i in range(len(line_label["bboxes"])):
                 if i in label_bbox_index:
-                    label_bboxes.append(line_label['bboxes'][i])
+                    label_bboxes.append(line_label["bboxes"][i])
                 if i in predict_bbox_index:
-                    predict_bboxes.append(line_label['bboxes'][i])
-            line_json['label_bboxes'] = label_bboxes
-            line_json['predict_bboxes'] = predict_bboxes
+                    predict_bboxes.append(line_label["bboxes"][i])
+            line_json["label_bboxes"] = label_bboxes
+            line_json["predict_bboxes"] = predict_bboxes
             json.dump(line_json, f, ensure_ascii=False)
-            f.write('\n')
+            f.write("\n")
 
 
 def print_arguments(args):
     """print arguments"""
-    print('-----------  Configuration Arguments -----------')
+    print("-----------  Configuration Arguments -----------")
     for arg, value in sorted(vars(args).items()):
-        print('%s: %s' % (arg, value))
-    print('------------------------------------------------')
+        print("%s: %s" % (arg, value))
+    print("------------------------------------------------")
 
 
 if __name__ == "__main__":

applications/document_intelligence/doc_vqa/Extraction/view.py

@@ -30,12 +30,12 @@ def _highlight_bbox(img, bbox):
     w = bbox[1] - x
     y = bbox[2]
     h = bbox[3] - y
-    sub_img = img[y:y + h, x:x + w]
+    sub_img = img[y : y + h, x : x + w]
     colored_rect = np.zeros(sub_img.shape, dtype=np.uint8)
     colored_rect[:, :, 2] = 255
     colored_rect[:, :, 1] = 255
     res = cv2.addWeighted(sub_img, 0.5, colored_rect, 0.5, 1.0)
-    img[y:y + h, x:x + w] = res
+    img[y : y + h, x : x + w] = res
 
 
 def highlight_ans(source_img_path, output_img_path, ans_bbox):
@@ -54,14 +54,14 @@ def highlight_img(source_img_path, output_img_path):
     cv2.imwrite(output_img_path, image)
 
 
-if __name__ == '__main__':
+if __name__ == "__main__":
     res_path = "./data/decode_res.json"
     result = {}
     with open(res_path, "r", encoding="utf-8") as f:
         line = f.readline()
         result = json.loads(line.strip())
 
-    img_path = '../OCR_process/demo_pics/demo_{}.png'.format(result["img_id"])
+    img_path = "../OCR_process/demo_pics/demo_{}.png".format(result["img_id"])
     img_save_path = "../answer.png"
-    highlight_ans(img_path, img_save_path, result['predict_bboxes'])
+    highlight_ans(img_path, img_save_path, result["predict_bboxes"])
     print("extraction result has been saved to answer.png")

applications/document_intelligence/doc_vqa/OCR_process/ocr_process.py

@@ -12,14 +12,11 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-import os
 import json
-from paddleocr import PaddleOCR
-import numpy as np
+import os
 import re
-import time
-import sys
-from multiprocessing import Process, Queue
+
+from paddleocr import PaddleOCR
 
 from paddlenlp.transformers import LayoutXLMTokenizer
 
@@ -33,7 +30,7 @@ def get_all_chars(tokenizer):
         tok_chr = [tc.replace("▁", "") for tc in tok_chr]
         while "" in tok_chr:
             tok_chr.remove("")
-        tok_chr = ''.join(tok_chr)
+        tok_chr = "".join(tok_chr)
         if len(tok_chr) != 1:
             all_chr.append(i)
     return all_chr
@@ -53,16 +50,14 @@ def merge_bbox(tok_bboxes):
         height_m += y_max - y_min
         width_m += x_max - x_min
     height_m = height_m / len(tok_bboxes)
-    if (height_g -
-            height_m) < 0.5 * height_m and width_g - width_m < 0.1 * width_m:
+    if (height_g - height_m) < 0.5 * height_m and width_g - width_m < 0.1 * width_m:
         return False, [min_gx, max_gx, min_gy, max_gy]
     else:
         return True, tok_bboxes[0]
 
 
 def xlm_parse(ocr_res, tokenizer):
-
-    doc_tokens, doc_bboxes = [], []
+    doc_bboxes = []
     all_chr = get_all_chars(tokenizer)
 
     try:
@@ -72,7 +67,7 @@ def xlm_parse(ocr_res, tokenizer):
             new_token_boxes.extend(item["token_box"])
 
         # get layoutxlm tokenizer results and get the final results
-        temp_span_text = ''.join(new_tokens)
+        temp_span_text = "".join(new_tokens)
         temp_span_bbox = new_token_boxes
         span_text = ""
         span_bbox = []
@@ -138,20 +133,15 @@ def xlm_parse(ocr_res, tokenizer):
                     for j in range(i, len(span_text)):
                         if span_text[j].lower() == span_tokens[tid + 1][0]:
                             break
-                        if span_text[j].lower() == "，" and span_tokens[
-                                tid + 1][0] == ",":
+                        if span_text[j].lower() == "，" and span_tokens[tid + 1][0] == ",":
                             break
-                        if span_text[j].lower() == "；" and span_tokens[
-                                tid + 1][0] == ";":
+                        if span_text[j].lower() == "；" and span_tokens[tid + 1][0] == ";":
                             break
-                        if span_text[j].lower() == "）" and span_tokens[
-                                tid + 1][0] == ")":
+                        if span_text[j].lower() == "）" and span_tokens[tid + 1][0] == ")":
                             break
-                        if span_text[j].lower() == "（" and span_tokens[
-                                tid + 1][0] == "(":
+                        if span_text[j].lower() == "（" and span_tokens[tid + 1][0] == "(":
                             break
-                        if span_text[j].lower() == "￥" and span_tokens[
-                                tid + 1][0] == "¥":
+                        if span_text[j].lower() == "￥" and span_tokens[tid + 1][0] == "¥":
                             break
 
                     tok_len = j - i
@@ -159,7 +149,7 @@ def xlm_parse(ocr_res, tokenizer):
             else:
                 if "<EFBFBD>" == span_text[i]:
                     tok_len = len(tok) + 1
-                elif tok == "......" and "…" in span_text[i:i + 6]:
+                elif tok == "......" and "…" in span_text[i : i + 6]:
                     tok_len = len(tok) - 2
                 elif "ⅱ" in span_text[i + len(tok) - 1]:
                     if tok == "i":
@@ -172,21 +162,21 @@ def xlm_parse(ocr_res, tokenizer):
                     tok_len = len(tok)
 
             assert i + tok_len <= len(span_bbox)
-            tok_bboxes = span_bbox[i:i + tok_len]
+            tok_bboxes = span_bbox[i : i + tok_len]
             _, merged_bbox = merge_bbox(tok_bboxes)
 
             doc_bboxes.append(merged_bbox)
             i = i + tok_len
-    except:
-        print('Error')
-        span_tokens = ['▁'] * 512
+    except Exception:
+        print("Error")
+        span_tokens = ["▁"] * 512
         doc_bboxes = [[0, 0, 0, 0]] * 512
 
     return span_tokens, doc_bboxes
 
 
 def tokenize_ocr_res(ocr_reses):
-    '''
+    """
     input:
         ocr_res: the ocr result of the image
     return:
@@ -198,7 +188,7 @@ def tokenize_ocr_res(ocr_reses):
                 "token_box: bounding box of each chars in ocr_res
                 }
         }
-    '''
+    """
     new_reses = []
     for img_name, ocr_res in ocr_reses:
         new_res = []
@@ -227,16 +217,16 @@ def tokenize_ocr_res(ocr_reses):
                 tok_y_max = y_max
 
                 shift = tok_x_max
-                new_token_boxes.append(
-                    [round(tok_x_min),
-                     round(tok_x_max), tok_y_min, tok_y_max])
+                new_token_boxes.append([round(tok_x_min), round(tok_x_max), tok_y_min, tok_y_max])
                 new_tokens.append(char)
-            new_res.append({
-                "text": para["text"],
-                "bounding_box": para["bbox"],
-                "tokens": new_tokens,
-                "token_box": new_token_boxes
-            })
+            new_res.append(
+                {
+                    "text": para["text"],
+                    "bounding_box": para["bbox"],
+                    "tokens": new_tokens,
+                    "token_box": new_token_boxes,
+                }
+            )
         new_reses.append((img_name, new_res))
     return new_reses
 
@@ -247,19 +237,15 @@ def process_input(ocr_reses, tokenizer, save_ocr_path):
     examples = []
     for img_name, ocr_res in ocr_reses:
         doc_tokens, doc_bboxes = xlm_parse(ocr_res, tokenizer)
-        doc_tokens.insert(0, '▁')
+        doc_tokens.insert(0, "▁")
         doc_bboxes.insert(0, doc_bboxes[0])
-        example = {
-            "img_name": img_name,
-            "document": doc_tokens,
-            "document_bbox": doc_bboxes
-        }
+        example = {"img_name": img_name, "document": doc_tokens, "document_bbox": doc_bboxes}
         examples.append(example)
 
-    with open(save_ocr_path, 'w', encoding='utf8') as f:
+    with open(save_ocr_path, "w", encoding="utf8") as f:
         for example in examples:
             json.dump(example, f, ensure_ascii=False)
-            f.write('\n')
+            f.write("\n")
 
     print(f"ocr parsing results has been save to: {save_ocr_path}")
 
@@ -267,11 +253,10 @@ def process_input(ocr_reses, tokenizer, save_ocr_path):
 def ocr_preprocess(img_dir):
     ocr = PaddleOCR(use_angle_cls=True, lang="ch", use_gpu=True)
     ocr_reses = []
-    img_names = sorted(os.listdir(img_dir),
-                       key=lambda x: int(x.split("_")[1].split(".")[0]))
+    img_names = sorted(os.listdir(img_dir), key=lambda x: int(x.split("_")[1].split(".")[0]))
     for img_name in img_names:
         img_path = os.path.join(img_dir, img_name)
-        parsing_res = ocr.ocr(img_path, cls=True)
+        parsing_res = ocr.ocr(img_path, cls=True)[0]
         ocr_res = []
         for para in parsing_res:
             ocr_res.append({"text": para[1][0], "bbox": para[0]})
@@ -280,7 +265,7 @@ def ocr_preprocess(img_dir):
     return ocr_reses
 
 
-if __name__ == '__main__':
+if __name__ == "__main__":
     img_dir = "./demo_pics"
     save_path = "./demo_ocr_res.json"
     ocr_results = ocr_preprocess(img_dir)

applications/document_intelligence/doc_vqa/README.md

@@ -17,7 +17,7 @@
 - 售后客服：需要配置大量客服人员，且客服专业知识培训周期长
 - 构建问题库：需要投入大量人力整理常见问题库，并且固定的问题库难以覆盖灵活多变的提问
 
-对于用户来说，汽车说明书问答系统能够支持通过车机助手/APP/小程序为用户提供即问即答的的功能。对于常见问题，用户不再需要查阅说明书，也无需打客服电话，从而缓解了人工客服的压力。
+对于用户来说，汽车说明书问答系统能够支持通过车机助手/APP/小程序为用户提供即问即答的功能。对于常见问题，用户不再需要查阅说明书，也无需打客服电话，从而缓解了人工客服的压力。
 
 对于客服来讲，汽车说明书问答系统帮助客服人员快速定位答案，高效查阅文档，提高客服的专业水平，同时也能够缩短客服的培训周期。
 
@@ -25,9 +25,9 @@
 
 #### 环境要求
 
-- paddlepaddle >= 2.2.0
-- paddlenlp >= 2.3.1
-- paddleocr >= 2.5
+- paddlepaddle == 2.3.2
+- paddlenlp == 2.5.2
+- paddleocr == 2.6.1.3
 
 安装相关问题可参考[PaddlePaddle](https://www.paddlepaddle.org.cn/install/quick?docurl=/documentation/docs/zh/install/pip/linux-pip.html)和[PaddleNLP](https://paddlenlp.readthedocs.io/zh/latest/get_started/installation.html)文档。
 

applications/document_intelligence/doc_vqa/Rerank/change_to_rerank.py

@@ -17,17 +17,17 @@ import json
 
 question = sys.argv[1]
 
-with open('../OCR_process/demo_ocr_res.json', 'r', encoding='utf8') as f:
+with open("../OCR_process/demo_ocr_res.json", "r", encoding="utf8") as f:
     paras = []
     for line in f:
         line = json.loads(line.strip())
-        document = line['document']
+        document = line["document"]
         para = []
         for token in document:
-            token = token.replace('▁', '')
+            token = token.replace("▁", "")
             para.append(token)
-        paras.append(''.join(para))
+        paras.append("".join(para))
 
-with open('./data/demo.tsv', 'w', encoding='utf8') as f:
+with open("./data/demo.tsv", "w", encoding="utf8") as f:
     for para in paras:
-        f.write('{}\t\t{}\t0\n'.format(question, para))
+        f.write("{}\t\t{}\t0\n".format(question, para))

applications/document_intelligence/doc_vqa/Rerank/src/batching.py

@@ -13,21 +13,18 @@
 # limitations under the License.
 """Mask, padding and batching."""
 
-from __future__ import absolute_import
-from __future__ import division
-from __future__ import print_function
-
 import numpy as np
-from six.moves import xrange
 
 
-def pad_batch_data(insts,
-                   pad_idx=0,
-                   return_pos=False,
-                   return_input_mask=False,
-                   return_max_len=False,
-                   return_num_token=False,
-                   return_seq_lens=False):
+def pad_batch_data(
+    insts,
+    pad_idx=0,
+    return_pos=False,
+    return_input_mask=False,
+    return_max_len=False,
+    return_num_token=False,
+    return_seq_lens=False,
+):
     """
     Pad the instances to the max sequence length in batch, and generate the
     corresponding position data and attention bias.
@@ -37,23 +34,18 @@ def pad_batch_data(insts,
     # Any token included in dict can be used to pad, since the paddings' loss
     # will be masked out by weights and make no effect on parameter gradients.
 
-    inst_data = np.array(
-        [inst + list([pad_idx] * (max_len - len(inst))) for inst in insts])
+    inst_data = np.array([inst + list([pad_idx] * (max_len - len(inst))) for inst in insts])
     return_list += [inst_data.astype("int64").reshape([-1, max_len, 1])]
 
     # position data
     if return_pos:
-        inst_pos = np.array([
-            list(range(0, len(inst))) + [pad_idx] * (max_len - len(inst))
-            for inst in insts
-        ])
+        inst_pos = np.array([list(range(0, len(inst))) + [pad_idx] * (max_len - len(inst)) for inst in insts])
 
         return_list += [inst_pos.astype("int64").reshape([-1, max_len, 1])]
 
     if return_input_mask:
         # This is used to avoid attention on paddings.
-        input_mask_data = np.array(
-            [[1] * len(inst) + [0] * (max_len - len(inst)) for inst in insts])
+        input_mask_data = np.array([[1] * len(inst) + [0] * (max_len - len(inst)) for inst in insts])
         input_mask_data = np.expand_dims(input_mask_data, axis=-1)
         return_list += [input_mask_data.astype("float32")]
 

applications/document_intelligence/doc_vqa/Rerank/src/cross_encoder.py

@@ -13,72 +13,59 @@
 # limitations under the License.
 """Model for classifier."""
 
-from __future__ import absolute_import
-from __future__ import division
-from __future__ import print_function
-from __future__ import unicode_literals
-from __future__ import absolute_import
-
-import time
 import logging
+import time
+
 import numpy as np
-from six.moves import xrange
-from scipy.stats import pearsonr, spearmanr
-
 import paddle.fluid as fluid
-
 from model.ernie import ErnieModel
+from scipy.stats import pearsonr, spearmanr
 
 log = logging.getLogger(__name__)
 
 
-def create_model(args,
-                 pyreader_name,
-                 ernie_config,
-                 is_prediction=False,
-                 task_name=""):
-    pyreader = fluid.layers.py_reader(capacity=50,
-                                      shapes=[[-1, args.max_seq_len, 1],
-                                              [-1, args.max_seq_len, 1],
-                                              [-1, args.max_seq_len, 1],
-                                              [-1, args.max_seq_len, 1],
-                                              [-1, args.max_seq_len, 1],
-                                              [-1, 1], [-1, 1]],
-                                      dtypes=[
-                                          'int64', 'int64', 'int64', 'int64',
-                                          'float32', 'int64', 'int64'
-                                      ],
-                                      lod_levels=[0, 0, 0, 0, 0, 0, 0],
-                                      name=task_name + "_" + pyreader_name,
-                                      use_double_buffer=True)
+def create_model(args, pyreader_name, ernie_config, is_prediction=False, task_name=""):
+    pyreader = fluid.layers.py_reader(
+        capacity=50,
+        shapes=[
+            [-1, args.max_seq_len, 1],
+            [-1, args.max_seq_len, 1],
+            [-1, args.max_seq_len, 1],
+            [-1, args.max_seq_len, 1],
+            [-1, args.max_seq_len, 1],
+            [-1, 1],
+            [-1, 1],
+        ],
+        dtypes=["int64", "int64", "int64", "int64", "float32", "int64", "int64"],
+        lod_levels=[0, 0, 0, 0, 0, 0, 0],
+        name=task_name + "_" + pyreader_name,
+        use_double_buffer=True,
+    )
 
-    (src_ids, sent_ids, pos_ids, task_ids, input_mask, labels,
-     qids) = fluid.layers.read_file(pyreader)
+    (src_ids, sent_ids, pos_ids, task_ids, input_mask, labels, qids) = fluid.layers.read_file(pyreader)
 
     def _model(is_noise=False):
-        ernie = ErnieModel(src_ids=src_ids,
-                           position_ids=pos_ids,
-                           sentence_ids=sent_ids,
-                           task_ids=task_ids,
-                           input_mask=input_mask,
-                           config=ernie_config,
-                           is_noise=is_noise)
+        ernie = ErnieModel(
+            src_ids=src_ids,
+            position_ids=pos_ids,
+            sentence_ids=sent_ids,
+            task_ids=task_ids,
+            input_mask=input_mask,
+            config=ernie_config,
+            is_noise=is_noise,
+        )
 
         cls_feats = ernie.get_pooled_output()
         if not is_noise:
-            cls_feats = fluid.layers.dropout(
-                x=cls_feats,
-                dropout_prob=0.1,
-                dropout_implementation="upscale_in_train")
+            cls_feats = fluid.layers.dropout(x=cls_feats, dropout_prob=0.1, dropout_implementation="upscale_in_train")
         logits = fluid.layers.fc(
             input=cls_feats,
             size=args.num_labels,
             param_attr=fluid.ParamAttr(
-                name=task_name + "_cls_out_w",
-                initializer=fluid.initializer.TruncatedNormal(scale=0.02)),
-            bias_attr=fluid.ParamAttr(
-                name=task_name + "_cls_out_b",
-                initializer=fluid.initializer.Constant(0.)))
+                name=task_name + "_cls_out_w", initializer=fluid.initializer.TruncatedNormal(scale=0.02)
+            ),
+            bias_attr=fluid.ParamAttr(name=task_name + "_cls_out_b", initializer=fluid.initializer.Constant(0.0)),
+        )
         """
         if is_prediction:
             probs = fluid.layers.softmax(logits)
@@ -90,37 +77,30 @@ def create_model(args,
             return pyreader, probs, feed_targets_name
         """
 
-        num_seqs = fluid.layers.create_tensor(dtype='int64')
-        ## add focal loss
-        ce_loss, probs = fluid.layers.softmax_with_cross_entropy(
-            logits=logits, label=labels, return_softmax=True)
+        num_seqs = fluid.layers.create_tensor(dtype="int64")
+        # add focal loss
+        ce_loss, probs = fluid.layers.softmax_with_cross_entropy(logits=logits, label=labels, return_softmax=True)
         loss = fluid.layers.mean(x=ce_loss)
-        accuracy = fluid.layers.accuracy(input=probs,
-                                         label=labels,
-                                         total=num_seqs)
+        accuracy = fluid.layers.accuracy(input=probs, label=labels, total=num_seqs)
         graph_vars = {
             "loss": loss,
             "probs": probs,
             "accuracy": accuracy,
             "labels": labels,
             "num_seqs": num_seqs,
-            "qids": qids
+            "qids": qids,
         }
         return graph_vars
 
     if not is_prediction:
         graph_vars = _model(is_noise=True)
         old_loss = graph_vars["loss"]
-        token_emb = fluid.default_main_program().global_block().var(
-            "word_embedding")
-        # print(token_emb)
+        token_emb = fluid.default_main_program().global_block().var("word_embedding")
         token_emb.stop_gradient = False
         token_gradient = fluid.gradients(old_loss, token_emb)[0]
         token_gradient.stop_gradient = False
         epsilon = 1e-8
-        norm = (fluid.layers.sqrt(
-            fluid.layers.reduce_sum(fluid.layers.square(token_gradient)) +
-            epsilon))
+        norm = fluid.layers.sqrt(fluid.layers.reduce_sum(fluid.layers.square(token_gradient)) + epsilon)
         gp = (0.01 * token_gradient) / norm
         gp.stop_gradient = True
         fluid.layers.assign(token_emb + gp, token_emb)
@@ -153,17 +133,10 @@ def evaluate_mrr(preds):
     return total_mrr / qnum
 
 
-def evaluate(exe,
-             test_program,
-             test_pyreader,
-             graph_vars,
-             eval_phase,
-             use_multi_gpu_test=False,
-             metric='simple_accuracy'):
-    train_fetch_list = [
-        graph_vars["loss"].name, graph_vars["accuracy"].name,
-        graph_vars["num_seqs"].name
-    ]
+def evaluate(
+    exe, test_program, test_pyreader, graph_vars, eval_phase, use_multi_gpu_test=False, metric="simple_accuracy"
+):
+    train_fetch_list = [graph_vars["loss"].name, graph_vars["accuracy"].name, graph_vars["num_seqs"].name]
 
     if eval_phase == "train":
         if "learning_rate" in graph_vars:
@@ -185,18 +158,21 @@ def evaluate(exe,
     time_begin = time.time()
 
     fetch_list = [
-        graph_vars["loss"].name, graph_vars["accuracy"].name,
-        graph_vars["probs"].name, graph_vars["labels"].name,
-        graph_vars["num_seqs"].name, graph_vars["qids"].name
+        graph_vars["loss"].name,
+        graph_vars["accuracy"].name,
+        graph_vars["probs"].name,
+        graph_vars["labels"].name,
+        graph_vars["num_seqs"].name,
+        graph_vars["qids"].name,
     ]
     while True:
         try:
             if use_multi_gpu_test:
-                np_loss, np_acc, np_probs, np_labels, np_num_seqs, np_qids = exe.run(
-                    fetch_list=fetch_list)
+                np_loss, np_acc, np_probs, np_labels, np_num_seqs, np_qids = exe.run(fetch_list=fetch_list)
             else:
                 np_loss, np_acc, np_probs, np_labels, np_num_seqs, np_qids = exe.run(
-                    program=test_program, fetch_list=fetch_list)
+                    program=test_program, fetch_list=fetch_list
+                )
             total_cost += np.sum(np_loss * np_num_seqs)
             total_acc += np.sum(np_acc * np_num_seqs)
             total_num_seqs += np.sum(np_num_seqs)
@@ -218,31 +194,55 @@ def evaluate(exe,
     elapsed_time = time_end - time_begin
 
     evaluate_info = ""
-    if metric == 'acc_and_f1':
+    if metric == "acc_and_f1":
         ret = acc_and_f1(preds, labels)
-        evaluate_info = "[%s evaluation] ave loss: %f, ave_acc: %f, f1: %f, data_num: %d, elapsed time: %f s" \
-            % (eval_phase, cost, ret['acc'], ret['f1'], total_num_seqs, elapsed_time)
-    elif metric == 'matthews_corrcoef':
+        evaluate_info = "[%s evaluation] ave loss: %f, ave_acc: %f, f1: %f, data_num: %d, elapsed time: %f s" % (
+            eval_phase,
+            cost,
+            ret["acc"],
+            ret["f1"],
+            total_num_seqs,
+            elapsed_time,
+        )
+    elif metric == "matthews_corrcoef":
         ret = matthews_corrcoef(preds, labels)
-        evaluate_info = "[%s evaluation] ave loss: %f, matthews_corrcoef: %f, data_num: %d, elapsed time: %f s" \
-            % (eval_phase, cost, ret, total_num_seqs, elapsed_time)
-    elif metric == 'pearson_and_spearman':
+        evaluate_info = "[%s evaluation] ave loss: %f, matthews_corrcoef: %f, data_num: %d, elapsed time: %f s" % (
+            eval_phase,
+            cost,
+            ret,
+            total_num_seqs,
+            elapsed_time,
+        )
+    elif metric == "pearson_and_spearman":
         ret = pearson_and_spearman(scores, labels)
-        evaluate_info = "[%s evaluation] ave loss: %f, pearson:%f, spearman:%f, corr:%f, data_num: %d, elapsed time: %f s" \
-            % (eval_phase, cost, ret['pearson'], ret['spearman'], ret['corr'], total_num_seqs, elapsed_time)
-    elif metric == 'simple_accuracy':
+        evaluate_info = (
+            "[%s evaluation] ave loss: %f, pearson:%f, spearman:%f, corr:%f, data_num: %d, elapsed time: %f s"
+            % (eval_phase, cost, ret["pearson"], ret["spearman"], ret["corr"], total_num_seqs, elapsed_time)
+        )
+    elif metric == "simple_accuracy":
         ret = simple_accuracy(preds, labels)
-        evaluate_info = "[%s evaluation] ave loss: %f, acc:%f, data_num: %d, elapsed time: %f s" \
-            % (eval_phase, cost, ret, total_num_seqs, elapsed_time)
+        evaluate_info = "[%s evaluation] ave loss: %f, acc:%f, data_num: %d, elapsed time: %f s" % (
+            eval_phase,
+            cost,
+            ret,
+            total_num_seqs,
+            elapsed_time,
+        )
     elif metric == "acc_and_f1_and_mrr":
         ret_a = acc_and_f1(preds, labels)
-        preds = sorted(zip(qids, scores, labels),
-                       key=lambda elem: (elem[0], -elem[1]))
+        preds = sorted(zip(qids, scores, labels), key=lambda elem: (elem[0], -elem[1]))
         ret_b = evaluate_mrr(preds)
-        evaluate_info = "[%s evaluation] ave loss: %f, acc: %f, f1: %f, mrr: %f, data_num: %d, elapsed time: %f s" \
-            % (eval_phase, cost, ret_a['acc'], ret_a['f1'], ret_b, total_num_seqs, elapsed_time)
+        evaluate_info = "[%s evaluation] ave loss: %f, acc: %f, f1: %f, mrr: %f, data_num: %d, elapsed time: %f s" % (
+            eval_phase,
+            cost,
+            ret_a["acc"],
+            ret_a["f1"],
+            ret_b,
+            total_num_seqs,
+            elapsed_time,
+        )
     else:
-        raise ValueError('unsupported metric {}'.format(metric))
+        raise ValueError("unsupported metric {}".format(metric))
     return evaluate_info
 
 
@@ -254,8 +254,7 @@ def matthews_corrcoef(preds, labels):
     fp = np.sum((labels == 0) & (preds == 1))
     fn = np.sum((labels == 1) & (preds == 0))
 
-    mcc = ((tp * tn) - (fp * fn)) / np.sqrt(
-        (tp + fp) * (tp + fn) * (tn + fp) * (tn + fn))
+    mcc = ((tp * tn) - (fp * fn)) / np.sqrt((tp + fp) * (tp + fn) * (tn + fp) * (tn + fn))
     return mcc
 
 
@@ -264,7 +263,6 @@ def f1_score(preds, labels):
     labels = np.array(labels)
 
     tp = np.sum((labels == 1) & (preds == 1))
-    tn = np.sum((labels == 0) & (preds == 0))
     fp = np.sum((labels == 0) & (preds == 1))
     fn = np.sum((labels == 1) & (preds == 0))
     p = tp / (tp + fp)
@@ -307,7 +305,7 @@ def simple_accuracy(preds, labels):
 
 def predict(exe, test_program, test_pyreader, graph_vars, dev_count=1):
     test_pyreader.start()
-    qids, scores, probs = [], [], []
+    qids, probs = [], []
     preds = []
 
     fetch_list = [graph_vars["probs"].name, graph_vars["qids"].name]
@@ -315,8 +313,7 @@ def predict(exe, test_program, test_pyreader, graph_vars, dev_count=1):
     while True:
         try:
             if dev_count == 1:
-                np_probs, np_qids = exe.run(program=test_program,
-                                            fetch_list=fetch_list)
+                np_probs, np_qids = exe.run(program=test_program, fetch_list=fetch_list)
             else:
                 np_probs, np_qids = exe.run(fetch_list=fetch_list)
 

applications/document_intelligence/doc_vqa/Rerank/src/finetune_args.py

@@ -1,4 +1,4 @@
-#   Copyright (c) 2019 PaddlePaddle Authors. All Rights Reserved.
+# Copyright (c) 2023 PaddlePaddle Authors. All Rights Reserved.
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
@@ -11,107 +11,85 @@
 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 # See the License for the specific language governing permissions and
 # limitations under the License.
-from __future__ import absolute_import
-from __future__ import division
-from __future__ import print_function
-from __future__ import unicode_literals
-from __future__ import absolute_import
 
-import os
-import time
 import argparse
 
-from utils.args import ArgumentGroup
+from src.utils.args import ArgumentGroup
 
 # yapf: disable
 parser = argparse.ArgumentParser(__doc__)
 model_g = ArgumentGroup(parser, "model", "model configuration and paths.")
-model_g.add_arg("ernie_config_path",         str,  None,           "Path to the json file for ernie model config.")
-model_g.add_arg("init_checkpoint",          str,  None,           "Init checkpoint to resume training from.")
-model_g.add_arg("init_pretraining_params",  str,  None,
-                "Init pre-training params which preforms fine-tuning from. If the "
-                 "arg 'init_checkpoint' has been set, this argument wouldn't be valid.")
-model_g.add_arg("checkpoints",              str,  "checkpoints",  "Path to save checkpoints.")
+model_g.add_arg("ernie_config_path", str, None, "Path to the json file for ernie model config.")
+model_g.add_arg("init_checkpoint", str, None, "Init checkpoint to resume training from.")
+model_g.add_arg("init_pretraining_params", str, None, "Init pre-training params which preforms fine-tuning from. If the arg 'init_checkpoint' has been set, this argument wouldn't be valid.")
+model_g.add_arg("checkpoints", str, "checkpoints", "Path to save checkpoints.")
 
-model_g.add_arg("is_classify",    bool, True,  "is_classify")
-model_g.add_arg("is_regression",  bool, False, "is_regression")
-model_g.add_arg("task_id",           int,    0,       "task id")
+model_g.add_arg("is_classify", bool, True, "is_classify")
+model_g.add_arg("is_regression", bool, False, "is_regression")
+model_g.add_arg("task_id", int, 0, "task id")
 
 train_g = ArgumentGroup(parser, "training", "training options.")
-train_g.add_arg("epoch",             int,    3,       "Number of epoches for fine-tuning.")
-train_g.add_arg("learning_rate",     float,  5e-5,    "Learning rate used to train with warmup.")
-train_g.add_arg("lr_scheduler",      str,    "linear_warmup_decay",
-                "scheduler of learning rate.", choices=['linear_warmup_decay', 'noam_decay'])
-train_g.add_arg("weight_decay",      float,  0.01,    "Weight decay rate for L2 regularizer.")
-train_g.add_arg("warmup_proportion", float,  0.1,
-                "Proportion of training steps to perform linear learning rate warmup for.")
-train_g.add_arg("save_steps",        int,    10000,   "The steps interval to save checkpoints.")
-train_g.add_arg("validation_steps",  int,    1000,    "The steps interval to evaluate model performance.")
-train_g.add_arg("use_recompute",          bool,   False,   "Whether to use recompute optimizer for training.")
-train_g.add_arg("use_mix_precision",          bool,   False,   "Whether to use mix-precision optimizer for training.")
-train_g.add_arg("use_cross_batch",          bool,   False,   "Whether to use cross-batch for training.")
-train_g.add_arg("use_lamb",          bool,   False,   "Whether to use LambOptimizer for training.")
-train_g.add_arg("use_dynamic_loss_scaling",    bool,   True,   "Whether to use dynamic loss scaling.")
+train_g.add_arg("epoch", int, 3, "Number of epoches for fine-tuning.")
+train_g.add_arg("learning_rate", float, 5e-5, "Learning rate used to train with warmup.")
+train_g.add_arg("lr_scheduler", str, "linear_warmup_decay", "scheduler of learning rate.", choices=['linear_warmup_decay', 'noam_decay'])
+train_g.add_arg("weight_decay", float, 0.01, "Weight decay rate for L2 regularizer.")
+train_g.add_arg("warmup_proportion", float, 0.1, "Proportion of training steps to perform linear learning rate warmup for.")
+train_g.add_arg("save_steps", int, 10000, "The steps interval to save checkpoints.")
+train_g.add_arg("validation_steps", int, 1000, "The steps interval to evaluate model performance.")
+train_g.add_arg("use_recompute", bool, False, "Whether to use recompute optimizer for training.")
+train_g.add_arg("use_mix_precision", bool, False, "Whether to use mix-precision optimizer for training.")
+train_g.add_arg("use_cross_batch", bool, False, "Whether to use cross-batch for training.")
+train_g.add_arg("use_lamb", bool, False, "Whether to use LambOptimizer for training.")
+train_g.add_arg("use_dynamic_loss_scaling", bool, True, "Whether to use dynamic loss scaling.")
 
-train_g.add_arg("test_save",            str,    "./checkpoints/test_result",       "test_save")
-train_g.add_arg("metric",               str,    "simple_accuracy",   "metric")
-train_g.add_arg("incr_every_n_steps",          int,    100,   "Increases loss scaling every n consecutive.")
-train_g.add_arg("decr_every_n_nan_or_inf",     int,    2,
-                "Decreases loss scaling every n accumulated steps with nan or inf gradients.")
-train_g.add_arg("incr_ratio",                  float,  2.0,
-                "The multiplier to use when increasing the loss scaling.")
-train_g.add_arg("decr_ratio",                  float,  0.8,
-                "The less-than-one-multiplier to use when decreasing.")
+train_g.add_arg("test_save", str, "./checkpoints/test_result", "test_save")
+train_g.add_arg("metric", str, "simple_accuracy", "metric")
+train_g.add_arg("incr_every_n_steps", int, 100, "Increases loss scaling every n consecutive.")
+train_g.add_arg("decr_every_n_nan_or_inf", int, 2, "Decreases loss scaling every n accumulated steps with nan or inf gradients.")
+train_g.add_arg("incr_ratio", float, 2.0, "The multiplier to use when increasing the loss scaling.")
+train_g.add_arg("decr_ratio", float, 0.8, "The less-than-one-multiplier to use when decreasing.")
 
-
-
-log_g = ArgumentGroup(parser,     "logging", "logging related.")
-log_g.add_arg("skip_steps",          int,    10,    "The steps interval to print loss.")
-log_g.add_arg("verbose",             bool,   False, "Whether to output verbose log.")
+log_g = ArgumentGroup(parser, "logging", "logging related.")
+log_g.add_arg("skip_steps", int, 10, "The steps interval to print loss.")
+log_g.add_arg("verbose", bool, False, "Whether to output verbose log.")
 
 data_g = ArgumentGroup(parser, "data", "Data paths, vocab paths and data processing options")
-data_g.add_arg("tokenizer",           str, "FullTokenizer",
-              "ATTENTION: the INPUT must be splited by Word with blank while using SentencepieceTokenizer or WordsegTokenizer")
-data_g.add_arg("train_set",           str,  None,  "Path to training data.")
-data_g.add_arg("test_set",            str,  None,  "Path to test data.")
-data_g.add_arg("dev_set",             str,  None,  "Path to validation data.")
-data_g.add_arg("vocab_path",          str,  None,  "Vocabulary path.")
-data_g.add_arg("max_seq_len",         int,  512,   "Number of words of the longest seqence.")
-data_g.add_arg("q_max_seq_len",       int,  32,   "Number of words of the longest seqence.")
-data_g.add_arg("p_max_seq_len",       int,  256,   "Number of words of the longest seqence.")
-data_g.add_arg("train_data_size",     int,  0,     "Number of training data's total examples. Set for distribute.")
-data_g.add_arg("batch_size",          int,  32,    "Total examples' number in batch for training. see also --in_tokens.")
-data_g.add_arg("predict_batch_size",  int,  None,    "Total examples' number in batch for predict. see also --in_tokens.")
-data_g.add_arg("in_tokens",           bool, False,
-              "If set, the batch size will be the maximum number of tokens in one batch. "
-              "Otherwise, it will be the maximum number of examples in one batch.")
-data_g.add_arg("do_lower_case",       bool, True,
-               "Whether to lower case the input text. Should be True for uncased models and False for cased models.")
-data_g.add_arg("random_seed",         int,  None,     "Random seed.")
-data_g.add_arg("label_map_config",    str,  None,  "label_map_path.")
-data_g.add_arg("num_labels",          int,  2,     "label number")
-data_g.add_arg("diagnostic",          str,  None,  "GLUE Diagnostic Dataset")
-data_g.add_arg("diagnostic_save",     str,  None,  "GLUE Diagnostic save f")
-data_g.add_arg("max_query_length",          int,   64,    "Max query length.")
-data_g.add_arg("max_answer_length",         int,   100,    "Max answer length.")
-data_g.add_arg("doc_stride",                int,   128,
-               "When splitting up a long document into chunks, how much stride to take between chunks.")
-data_g.add_arg("n_best_size",               int,   20,
-               "The total number of n-best predictions to generate in the nbest_predictions.json output file.")
-data_g.add_arg("chunk_scheme", type=str,  default="IOB", choices=["IO", "IOB", "IOE", "IOBES"], help="chunk scheme")
+data_g.add_arg("tokenizer", str, "FullTokenizer", "ATTENTION: the INPUT must be splited by Word with blank while using SentencepieceTokenizer or WordsegTokenizer")
+data_g.add_arg("train_set", str, None, "Path to training data.")
+data_g.add_arg("test_set", str, None, "Path to test data.")
+data_g.add_arg("dev_set", str, None, "Path to validation data.")
+data_g.add_arg("vocab_path", str, None, "Vocabulary path.")
+data_g.add_arg("max_seq_len", int, 512, "Number of words of the longest sequence.")
+data_g.add_arg("q_max_seq_len", int, 32, "Number of words of the longest sequence.")
+data_g.add_arg("p_max_seq_len", int, 256, "Number of words of the longest sequence.")
+data_g.add_arg("train_data_size", int, 0, "Number of training data's total examples. Set for distribute.")
+data_g.add_arg("batch_size", int, 32, "Total examples' number in batch for training. see also --in_tokens.")
+data_g.add_arg("predict_batch_size", int, None, "Total examples' number in batch for predict. see also --in_tokens.")
+data_g.add_arg("in_tokens", bool, False, "If set, the batch size will be the maximum number of tokens in one batch. Otherwise, it will be the maximum number of examples in one batch.")
+data_g.add_arg("do_lower_case", bool, True, "Whether to lower case the input text. Should be True for uncased models and False for cased models.")
+data_g.add_arg("random_seed", int, None, "Random seed.")
+data_g.add_arg("label_map_config", str, None, "label_map_path.")
+data_g.add_arg("num_labels", int, 2, "label number")
+data_g.add_arg("diagnostic", str, None, "GLUE Diagnostic Dataset")
+data_g.add_arg("diagnostic_save", str, None, "GLUE Diagnostic save f")
+data_g.add_arg("max_query_length", int, 64, "Max query length.")
+data_g.add_arg("max_answer_length", int, 100, "Max answer length.")
+data_g.add_arg("doc_stride", int, 128, "When splitting up a long document into chunks, how much stride to take between chunks.")
+data_g.add_arg("n_best_size", int, 20, "The total number of n-best predictions to generate in the nbest_predictions.json output file.")
+data_g.add_arg("chunk_scheme", type=str, default="IOB", choices=["IO", "IOB", "IOE", "IOBES"], help="chunk scheme")
 
 run_type_g = ArgumentGroup(parser, "run_type", "running type options.")
-run_type_g.add_arg("use_cuda",                     bool,   True,  "If set, use GPU for training.")
-run_type_g.add_arg("is_distributed",    bool,   False,  "If set, then start distributed training.")
-run_type_g.add_arg("use_fast_executor",            bool,   False, "If set, use fast parallel executor (in experiment).")
-run_type_g.add_arg("num_iteration_per_drop_scope", int,    10,    "Iteration intervals to drop scope.")
-run_type_g.add_arg("do_train",                     bool,   True,  "Whether to perform training.")
-run_type_g.add_arg("do_val",                       bool,   True,  "Whether to perform evaluation on dev data set.")
-run_type_g.add_arg("do_test",                      bool,   True,  "Whether to perform evaluation on test data set.")
-run_type_g.add_arg("output_item",                  int,   3,  "Test output format.")
-run_type_g.add_arg("output_file_name",             str,   None,  "Test output file name")
-run_type_g.add_arg("test_data_cnt",             int,  1110000 ,  "total cnt of testset")
-run_type_g.add_arg("use_multi_gpu_test",           bool,   False, "Whether to perform evaluation using multiple gpu cards")
-run_type_g.add_arg("metrics",                      bool,   True,  "Whether to perform evaluation on test data set.")
-run_type_g.add_arg("shuffle",                      bool,   True,  "")
-run_type_g.add_arg("for_cn",                       bool,   False,  "model train for cn or for other langs.")
+run_type_g.add_arg("use_cuda", bool, True, "If set, use GPU for training.")
+run_type_g.add_arg("is_distributed", bool, False, "If set, then start distributed training.")
+run_type_g.add_arg("use_fast_executor", bool, False, "If set, use fast parallel executor (in experiment).")
+run_type_g.add_arg("num_iteration_per_drop_scope", int, 10, "Iteration intervals to drop scope.")
+run_type_g.add_arg("do_train", bool, True, "Whether to perform training.")
+run_type_g.add_arg("do_val", bool, True, "Whether to perform evaluation on dev data set.")
+run_type_g.add_arg("do_test", bool, True, "Whether to perform evaluation on test data set.")
+run_type_g.add_arg("output_item", int, 3, "Test output format.")
+run_type_g.add_arg("output_file_name", str, None, "Test output file name")
+run_type_g.add_arg("test_data_cnt", int, 1110000 , "total cnt of testset")
+run_type_g.add_arg("use_multi_gpu_test", bool, False, "Whether to perform evaluation using multiple gpu cards")
+run_type_g.add_arg("metrics", bool, True, "Whether to perform evaluation on test data set.")
+run_type_g.add_arg("shuffle", bool, True, "")
+run_type_g.add_arg("for_cn", bool, False, "model train for cn or for other langs.")

applications/document_intelligence/doc_vqa/Rerank/src/index_search.py

@@ -13,18 +13,17 @@
 # limitations under the License.
 
 import sys
-import time
+
 import faiss
-import math
 import numpy as np
 
 
 def read_embed(file_name, dim=768, bs=3000):
-    if file_name.endswith('npy'):
+    if file_name.endswith("npy"):
         i = 0
         emb_np = np.load(file_name)
-        while (i < len(emb_np)):
-            vec_list = emb_np[i:i + bs]
+        while i < len(emb_np):
+            vec_list = emb_np[i : i + bs]
             i += bs
             yield vec_list
     else:
@@ -32,7 +31,7 @@ def read_embed(file_name, dim=768, bs=3000):
         with open(file_name) as inp:
             for line in inp:
                 data = line.strip()
-                vector = [float(item) for item in data.split(' ')]
+                vector = [float(item) for item in data.split(" ")]
                 assert len(vector) == dim
                 vec_list.append(vector)
                 if len(vec_list) == bs:
@@ -47,24 +46,23 @@ def load_qid(file_name):
     with open(file_name) as inp:
         for line in inp:
             line = line.strip()
-            qid = line.split('\t')[0]
+            qid = line.split("\t")[0]
             qid_list.append(qid)
     return qid_list
 
 
 def search(index, emb_file, qid_list, outfile, top_k):
     q_idx = 0
-    with open(outfile, 'w') as out:
+    with open(outfile, "w") as out:
         for batch_vec in read_embed(emb_file):
             q_emb_matrix = np.array(batch_vec)
-            res_dist, res_p_id = index.search(q_emb_matrix.astype('float32'),
-                                              top_k)
+            res_dist, res_p_id = index.search(q_emb_matrix.astype("float32"), top_k)
             for i in range(len(q_emb_matrix)):
                 qid = qid_list[q_idx]
                 for j in range(top_k):
                     pid = res_p_id[i][j]
                     score = res_dist[i][j]
-                    out.write('%s\t%s\t%s\t%s\n' % (qid, pid, j + 1, score))
+                    out.write("%s\t%s\t%s\t%s\n" % (qid, pid, j + 1, score))
                 q_idx += 1
 
 
@@ -72,12 +70,12 @@ def main():
     part = sys.argv[1]
     topk = int(sys.argv[2])
     q_text_file = sys.argv[3]
-    outfile = 'output/res.top%s-part%s' % (topk, part)
+    outfile = "output/res.top%s-part%s" % (topk, part)
 
     qid_list = load_qid(q_text_file)
 
     engine = faiss.read_index("output/para.index.part%s" % part)
-    emb_file = 'output/query.emb.npy'
+    emb_file = "output/query.emb.npy"
     search(engine, emb_file, qid_list, outfile, topk)
 
 

applications/document_intelligence/doc_vqa/Rerank/src/merge.py

@@ -20,7 +20,7 @@ total_part = int(sys.argv[3])
 
 f_list = []
 for part in range(total_part):
-    f0 = open('output/res.top%s-part%s' % (top, part))
+    f0 = open("output/res.top%s-part%s" % (top, part))
     f_list.append(f0)
 
 line_list = []
@@ -28,22 +28,21 @@ for part in range(total_part):
     line = f_list[part].readline()
     line_list.append(line)
 
-out = open('output/dev.res.top%s' % top, 'w')
-last_q = ''
+out = open("output/dev.res.top%s" % top, "w")
+last_q = ""
 ans_list = {}
 while line_list[-1]:
     cur_list = []
     for line in line_list:
-        sub = line.strip().split('\t')
+        sub = line.strip().split("\t")
         cur_list.append(sub)
 
-    if last_q == '':
+    if last_q == "":
         last_q = cur_list[0][0]
     if cur_list[0][0] != last_q:
         rank = sorted(ans_list.items(), key=lambda a: a[1], reverse=True)
         for i in range(top):
-            out.write("%s\t%s\t%s\t%s\n" %
-                      (last_q, rank[i][0], i + 1, rank[i][1]))
+            out.write("%s\t%s\t%s\t%s\n" % (last_q, rank[i][0], i + 1, rank[i][1]))
         ans_list = {}
     for i, sub in enumerate(cur_list):
         ans_list[int(sub[1]) + shift * i] = float(sub[-1])
@@ -59,4 +58,4 @@ for i in range(top):
     out.write("%s\t%s\t%s\t%s\n" % (last_q, rank[i][0], i + 1, rank[i][1]))
 out.close()
 
-print('output/dev.res.top%s' % top)
+print("output/dev.res.top%s" % top)

applications/document_intelligence/doc_vqa/Rerank/src/model/ernie.py

@@ -13,36 +13,30 @@
 # limitations under the License.
 """Ernie model."""
 
-from __future__ import absolute_import
-from __future__ import division
-from __future__ import print_function
-from __future__ import unicode_literals
-from __future__ import absolute_import
+from __future__ import absolute_import, division, print_function, unicode_literals
 
 import json
-import six
 import logging
 from io import open
 
+import paddle
 import paddle.fluid as fluid
-
+import six
 from model.transformer_encoder import encoder, pre_process_layer
 
 log = logging.getLogger(__name__)
 
 
 class ErnieConfig(object):
-
     def __init__(self, config_path):
         self._config_dict = self._parse(config_path)
 
     def _parse(self, config_path):
         try:
-            with open(config_path, 'r', encoding='utf8') as json_file:
+            with open(config_path, "r", encoding="utf8") as json_file:
                 config_dict = json.load(json_file)
         except Exception:
-            raise IOError("Error in parsing Ernie model config file '%s'" %
-                          config_path)
+            raise IOError("Error in parsing Ernie model config file '%s'" % config_path)
         else:
             return config_dict
 
@@ -51,39 +45,40 @@ class ErnieConfig(object):
 
     def print_config(self):
         for arg, value in sorted(six.iteritems(self._config_dict)):
-            log.info('%s: %s' % (arg, value))
-        log.info('------------------------------------------------')
+            log.info("%s: %s" % (arg, value))
+        log.info("------------------------------------------------")
 
 
 class ErnieModel(object):
+    def __init__(
+        self,
+        src_ids,
+        position_ids,
+        sentence_ids,
+        task_ids,
+        input_mask,
+        config,
+        weight_sharing=True,
+        model_name="",
+        is_noise=False,
+    ):
 
-    def __init__(self,
-                 src_ids,
-                 position_ids,
-                 sentence_ids,
-                 task_ids,
-                 input_mask,
-                 config,
-                 weight_sharing=True,
-                 model_name='',
-                 is_noise=False):
-
-        self._emb_size = config['hidden_size']
-        self._n_layer = config['num_hidden_layers']
-        self._n_head = config['num_attention_heads']
-        self._voc_size = config['vocab_size']
-        self._max_position_seq_len = config['max_position_embeddings']
-        if config['sent_type_vocab_size']:
-            self._sent_types = config['sent_type_vocab_size']
+        self._emb_size = config["hidden_size"]
+        self._n_layer = config["num_hidden_layers"]
+        self._n_head = config["num_attention_heads"]
+        self._voc_size = config["vocab_size"]
+        self._max_position_seq_len = config["max_position_embeddings"]
+        if config["sent_type_vocab_size"]:
+            self._sent_types = config["sent_type_vocab_size"]
         else:
-            self._sent_types = config['type_vocab_size']
+            self._sent_types = config["type_vocab_size"]
 
-        self._use_task_id = config['use_task_id']
+        self._use_task_id = config["use_task_id"]
         if self._use_task_id:
-            self._task_types = config['task_type_vocab_size']
-        self._hidden_act = config['hidden_act']
-        self._prepostprocess_dropout = config['hidden_dropout_prob']
-        self._attention_dropout = config['attention_probs_dropout_prob']
+            self._task_types = config["task_type_vocab_size"]
+        self._hidden_act = config["hidden_act"]
+        self._prepostprocess_dropout = config["hidden_dropout_prob"]
+        self._attention_dropout = config["attention_probs_dropout_prob"]
         if is_noise:
             self._prepostprocess_dropout = 0
             self._attention_dropout = 0
@@ -96,38 +91,35 @@ class ErnieModel(object):
         self._task_emb_name = "task_embedding"
         self._emb_dtype = "float32"
 
-        # Initialize all weigths by truncated normal initializer, and all biases
+        # Initialize all weights by truncated normal initializer, and all biases
         # will be initialized by constant zero by default.
-        self._param_initializer = fluid.initializer.TruncatedNormal(
-            scale=config['initializer_range'])
+        self._param_initializer = fluid.initializer.TruncatedNormal(scale=config["initializer_range"])
 
-        self._build_model(model_name, src_ids, position_ids, sentence_ids,
-                          task_ids, input_mask)
+        self._build_model(model_name, src_ids, position_ids, sentence_ids, task_ids, input_mask)
 
-    def _build_model(self, model_name, src_ids, position_ids, sentence_ids,
-                     task_ids, input_mask):
+    def _build_model(self, model_name, src_ids, position_ids, sentence_ids, task_ids, input_mask):
         # padding id in vocabulary must be set to 0
         emb_out = fluid.layers.embedding(
             input=src_ids,
             size=[self._voc_size, self._emb_size],
             dtype=self._emb_dtype,
-            param_attr=fluid.ParamAttr(name=model_name + self._word_emb_name,
-                                       initializer=self._param_initializer),
-            is_sparse=False)
+            param_attr=fluid.ParamAttr(name=model_name + self._word_emb_name, initializer=self._param_initializer),
+            is_sparse=False,
+        )
 
         position_emb_out = fluid.layers.embedding(
             input=position_ids,
             size=[self._max_position_seq_len, self._emb_size],
             dtype=self._emb_dtype,
-            param_attr=fluid.ParamAttr(name=model_name + self._pos_emb_name,
-                                       initializer=self._param_initializer))
+            param_attr=fluid.ParamAttr(name=model_name + self._pos_emb_name, initializer=self._param_initializer),
+        )
 
         sent_emb_out = fluid.layers.embedding(
             sentence_ids,
             size=[self._sent_types, self._emb_size],
             dtype=self._emb_dtype,
-            param_attr=fluid.ParamAttr(name=model_name + self._sent_emb_name,
-                                       initializer=self._param_initializer))
+            param_attr=fluid.ParamAttr(name=model_name + self._sent_emb_name, initializer=self._param_initializer),
+        )
 
         emb_out = emb_out + position_emb_out
         emb_out = emb_out + sent_emb_out
@@ -137,28 +129,17 @@ class ErnieModel(object):
                 task_ids,
                 size=[self._task_types, self._emb_size],
                 dtype=self._emb_dtype,
-                param_attr=fluid.ParamAttr(name=model_name +
-                                           self._task_emb_name,
-                                           initializer=self._param_initializer))
+                param_attr=fluid.ParamAttr(name=model_name + self._task_emb_name, initializer=self._param_initializer),
+            )
 
             emb_out = emb_out + task_emb_out
 
-        emb_out = pre_process_layer(emb_out,
-                                    'nd',
-                                    self._prepostprocess_dropout,
-                                    name=model_name + 'pre_encoder')
+        emb_out = pre_process_layer(emb_out, "nd", self._prepostprocess_dropout, name=model_name + "pre_encoder")
 
-        self_attn_mask = fluid.layers.matmul(x=input_mask,
-                                             y=input_mask,
-                                             transpose_y=True)
+        self_attn_mask = paddle.matmul(x=input_mask, y=input_mask, transpose_y=True)
 
-        self_attn_mask = fluid.layers.scale(x=self_attn_mask,
-                                            scale=10000.0,
-                                            bias=-1.0,
-                                            bias_after_scale=False)
-        n_head_self_attn_mask = fluid.layers.stack(x=[self_attn_mask] *
-                                                   self._n_head,
-                                                   axis=1)
+        self_attn_mask = fluid.layers.scale(x=self_attn_mask, scale=10000.0, bias=-1.0, bias_after_scale=False)
+        n_head_self_attn_mask = fluid.layers.stack(x=[self_attn_mask] * self._n_head, axis=1)
         n_head_self_attn_mask.stop_gradient = True
 
         self._enc_out, self.checkpoints = encoder(
@@ -178,44 +159,38 @@ class ErnieModel(object):
             postprocess_cmd="dan",
             param_initializer=self._param_initializer,
             model_name=model_name,
-            name=model_name + 'encoder')
+            name=model_name + "encoder",
+        )
 
     def get_sequence_output(self):
         return self._enc_out
 
     def get_cls_output(self):
         """Get the first feature of each sequence for classification"""
-        cls_output = fluid.layers.slice(input=self._enc_out,
-                                        axes=[1],
-                                        starts=[0],
-                                        ends=[1])
+        cls_output = fluid.layers.slice(input=self._enc_out, axes=[1], starts=[0], ends=[1])
         cls_output = fluid.layers.squeeze(cls_output, axes=[1])
         return cls_output
 
     def get_pooled_output(self):
         """Get the first feature of each sequence for classification"""
-        next_sent_feat = fluid.layers.slice(input=self._enc_out,
-                                            axes=[1],
-                                            starts=[0],
-                                            ends=[1])
+        next_sent_feat = fluid.layers.slice(input=self._enc_out, axes=[1], starts=[0], ends=[1])
         next_sent_feat = fluid.layers.fc(
             input=next_sent_feat,
             size=self._emb_size,
             act="tanh",
-            param_attr=fluid.ParamAttr(name="pooled_fc.w_0",
-                                       initializer=self._param_initializer),
-            bias_attr="pooled_fc.b_0")
+            param_attr=fluid.ParamAttr(name="pooled_fc.w_0", initializer=self._param_initializer),
+            bias_attr="pooled_fc.b_0",
+        )
         return next_sent_feat
 
     def get_lm_output(self, mask_label, mask_pos):
         """Get the loss & accuracy for pretraining"""
 
-        mask_pos = fluid.layers.cast(x=mask_pos, dtype='int32')
+        mask_pos = fluid.layers.cast(x=mask_pos, dtype="int32")
 
         # extract the first token feature in each sentence
         self.next_sent_feat = self.get_pooled_output()
-        reshaped_emb_out = fluid.layers.reshape(x=self._enc_out,
-                                                shape=[-1, self._emb_size])
+        reshaped_emb_out = fluid.layers.reshape(x=self._enc_out, shape=[-1, self._emb_size])
         # extract masked tokens' feature
         mask_feat = fluid.layers.gather(input=reshaped_emb_out, index=mask_pos)
 
@@ -224,61 +199,61 @@ class ErnieModel(object):
             input=mask_feat,
             size=self._emb_size,
             act=self._hidden_act,
-            param_attr=fluid.ParamAttr(name='mask_lm_trans_fc.w_0',
-                                       initializer=self._param_initializer),
-            bias_attr=fluid.ParamAttr(name='mask_lm_trans_fc.b_0'))
+            param_attr=fluid.ParamAttr(name="mask_lm_trans_fc.w_0", initializer=self._param_initializer),
+            bias_attr=fluid.ParamAttr(name="mask_lm_trans_fc.b_0"),
+        )
 
         # transform: layer norm
         mask_trans_feat = fluid.layers.layer_norm(
             mask_trans_feat,
             begin_norm_axis=len(mask_trans_feat.shape) - 1,
             param_attr=fluid.ParamAttr(
-                name='mask_lm_trans_layer_norm_scale',
-                initializer=fluid.initializer.Constant(1.)),
+                name="mask_lm_trans_layer_norm_scale", initializer=fluid.initializer.Constant(1.0)
+            ),
             bias_attr=fluid.ParamAttr(
-                name='mask_lm_trans_layer_norm_bias',
-                initializer=fluid.initializer.Constant(1.)))
+                name="mask_lm_trans_layer_norm_bias", initializer=fluid.initializer.Constant(1.0)
+            ),
+        )
         # transform: layer norm
-        #mask_trans_feat = pre_process_layer(
+        # mask_trans_feat = pre_process_layer(
         #    mask_trans_feat, 'n', name='mask_lm_trans')
 
         mask_lm_out_bias_attr = fluid.ParamAttr(
-            name="mask_lm_out_fc.b_0",
-            initializer=fluid.initializer.Constant(value=0.0))
+            name="mask_lm_out_fc.b_0", initializer=fluid.initializer.Constant(value=0.0)
+        )
         if self._weight_sharing:
-            fc_out = fluid.layers.matmul(
+            fc_out = paddle.matmul(
                 x=mask_trans_feat,
-                y=fluid.default_main_program().global_block().var(
-                    self._word_emb_name),
-                transpose_y=True)
-            fc_out += fluid.layers.create_parameter(shape=[self._voc_size],
-                                                    dtype=self._emb_dtype,
-                                                    attr=mask_lm_out_bias_attr,
-                                                    is_bias=True)
+                y=fluid.default_main_program().global_block().var(self._word_emb_name),
+                transpose_y=True,
+            )
+            fc_out += fluid.layers.create_parameter(
+                shape=[self._voc_size], dtype=self._emb_dtype, attr=mask_lm_out_bias_attr, is_bias=True
+            )
 
         else:
-            fc_out = fluid.layers.fc(input=mask_trans_feat,
-                                     size=self._voc_size,
-                                     param_attr=fluid.ParamAttr(
-                                         name="mask_lm_out_fc.w_0",
-                                         initializer=self._param_initializer),
-                                     bias_attr=mask_lm_out_bias_attr)
+            fc_out = fluid.layers.fc(
+                input=mask_trans_feat,
+                size=self._voc_size,
+                param_attr=fluid.ParamAttr(name="mask_lm_out_fc.w_0", initializer=self._param_initializer),
+                bias_attr=mask_lm_out_bias_attr,
+            )
 
-        mask_lm_loss = fluid.layers.softmax_with_cross_entropy(logits=fc_out,
-                                                               label=mask_label)
+        mask_lm_loss = fluid.layers.softmax_with_cross_entropy(logits=fc_out, label=mask_label)
         mean_mask_lm_loss = fluid.layers.mean(mask_lm_loss)
 
         return mean_mask_lm_loss
 
     def get_task_output(self, task, task_labels):
-        task_fc_out = fluid.layers.fc(input=self.next_sent_feat,
-                                      size=task["num_labels"],
-                                      param_attr=fluid.ParamAttr(
-                                          name=task["task_name"] + "_fc.w_0",
-                                          initializer=self._param_initializer),
-                                      bias_attr=task["task_name"] + "_fc.b_0")
+        task_fc_out = fluid.layers.fc(
+            input=self.next_sent_feat,
+            size=task["num_labels"],
+            param_attr=fluid.ParamAttr(name=task["task_name"] + "_fc.w_0", initializer=self._param_initializer),
+            bias_attr=task["task_name"] + "_fc.b_0",
+        )
         task_loss, task_softmax = fluid.layers.softmax_with_cross_entropy(
-            logits=task_fc_out, label=task_labels, return_softmax=True)
+            logits=task_fc_out, label=task_labels, return_softmax=True
+        )
         task_acc = fluid.layers.accuracy(input=task_softmax, label=task_labels)
         mean_task_loss = fluid.layers.mean(task_loss)
         return mean_task_loss, task_acc

applications/document_intelligence/doc_vqa/Rerank/src/model/transformer_encoder.py

@@ -13,67 +13,70 @@
 # limitations under the License.
 """Transformer encoder."""
 
-from __future__ import absolute_import
-from __future__ import division
-from __future__ import print_function
+from __future__ import absolute_import, division, print_function
 
 from functools import partial
 
+import paddle
 import paddle.fluid as fluid
 import paddle.fluid.layers as layers
 
 
-def multi_head_attention(queries,
-                         keys,
-                         values,
-                         attn_bias,
-                         d_key,
-                         d_value,
-                         d_model,
-                         n_head=1,
-                         dropout_rate=0.,
-                         cache=None,
-                         param_initializer=None,
-                         name='multi_head_att'):
+def multi_head_attention(
+    queries,
+    keys,
+    values,
+    attn_bias,
+    d_key,
+    d_value,
+    d_model,
+    n_head=1,
+    dropout_rate=0.0,
+    cache=None,
+    param_initializer=None,
+    name="multi_head_att",
+):
     """
     Multi-Head Attention. Note that attn_bias is added to the logit before
-    computing softmax activiation to mask certain selected positions so that
+    computing softmax activation to mask certain selected positions so that
     they will not considered in attention weights.
     """
     keys = queries if keys is None else keys
     values = keys if values is None else values
 
     if not (len(queries.shape) == len(keys.shape) == len(values.shape) == 3):
-        raise ValueError(
-            "Inputs: quries, keys and values should all be 3-D tensors.")
+        raise ValueError("Inputs: queries, keys and values should all be 3-D tensors.")
 
     def __compute_qkv(queries, keys, values, n_head, d_key, d_value):
         """
         Add linear projection to queries, keys, and values.
         """
-        q = layers.fc(input=queries,
-                      size=d_key * n_head,
-                      num_flatten_dims=2,
-                      param_attr=fluid.ParamAttr(name=name + '_query_fc.w_0',
-                                                 initializer=param_initializer),
-                      bias_attr=name + '_query_fc.b_0')
-        k = layers.fc(input=keys,
-                      size=d_key * n_head,
-                      num_flatten_dims=2,
-                      param_attr=fluid.ParamAttr(name=name + '_key_fc.w_0',
-                                                 initializer=param_initializer),
-                      bias_attr=name + '_key_fc.b_0')
-        v = layers.fc(input=values,
-                      size=d_value * n_head,
-                      num_flatten_dims=2,
-                      param_attr=fluid.ParamAttr(name=name + '_value_fc.w_0',
-                                                 initializer=param_initializer),
-                      bias_attr=name + '_value_fc.b_0')
+        q = layers.fc(
+            input=queries,
+            size=d_key * n_head,
+            num_flatten_dims=2,
+            param_attr=fluid.ParamAttr(name=name + "_query_fc.w_0", initializer=param_initializer),
+            bias_attr=name + "_query_fc.b_0",
+        )
+        k = layers.fc(
+            input=keys,
+            size=d_key * n_head,
+            num_flatten_dims=2,
+            param_attr=fluid.ParamAttr(name=name + "_key_fc.w_0", initializer=param_initializer),
+            bias_attr=name + "_key_fc.b_0",
+        )
+        v = layers.fc(
+            input=values,
+            size=d_value * n_head,
+            num_flatten_dims=2,
+            param_attr=fluid.ParamAttr(name=name + "_value_fc.w_0", initializer=param_initializer),
+            bias_attr=name + "_value_fc.b_0",
+        )
         return q, k, v
 
     def __split_heads(x, n_head):
         """
-        Reshape the last dimension of inpunt tensor x so that it becomes two
+        Reshape the last dimension of input tensor x so that it becomes two
         dimensions and then transpose. Specifically, input a tensor with shape
         [bs, max_sequence_length, n_head * hidden_dim] then output a tensor
         with shape [bs, n_head, max_sequence_length, hidden_dim].
@@ -81,9 +84,7 @@ def multi_head_attention(queries,
         hidden_size = x.shape[-1]
         # The value 0 in shape attr means copying the corresponding dimension
         # size of the input as the output dimension size.
-        reshaped = layers.reshape(x=x,
-                                  shape=[0, 0, n_head, hidden_size // n_head],
-                                  inplace=True)
+        reshaped = layers.reshape(x=x, shape=[0, 0, n_head, hidden_size // n_head], inplace=True)
 
         # permuate the dimensions into:
         # [batch_size, n_head, max_sequence_len, hidden_size_per_head]
@@ -91,35 +92,33 @@ def multi_head_attention(queries,
 
     def __combine_heads(x):
         """
-        Transpose and then reshape the last two dimensions of inpunt tensor x
+        Transpose and then reshape the last two dimensions of input tensor x
         so that it becomes one dimension, which is reverse to __split_heads.
         """
-        if len(x.shape) == 3: return x
+        if len(x.shape) == 3:
+            return x
         if len(x.shape) != 4:
             raise ValueError("Input(x) should be a 4-D Tensor.")
 
         trans_x = layers.transpose(x, perm=[0, 2, 1, 3])
         # The value 0 in shape attr means copying the corresponding dimension
         # size of the input as the output dimension size.
-        return layers.reshape(x=trans_x,
-                              shape=[0, 0, trans_x.shape[2] * trans_x.shape[3]],
-                              inplace=True)
+        return layers.reshape(x=trans_x, shape=[0, 0, trans_x.shape[2] * trans_x.shape[3]], inplace=True)
 
     def scaled_dot_product_attention(q, k, v, attn_bias, d_key, dropout_rate):
         """
         Scaled Dot-Product Attention
         """
         scaled_q = layers.scale(x=q, scale=d_key**-0.5)
-        product = layers.matmul(x=scaled_q, y=k, transpose_y=True)
+        product = paddle.matmul(x=scaled_q, y=k, transpose_y=True)
         if attn_bias:
             product += attn_bias
         weights = layers.softmax(product)
         if dropout_rate:
-            weights = layers.dropout(weights,
-                                     dropout_prob=dropout_rate,
-                                     dropout_implementation="upscale_in_train",
-                                     is_test=False)
-        out = layers.matmul(weights, v)
+            weights = layers.dropout(
+                weights, dropout_prob=dropout_rate, dropout_implementation="upscale_in_train", is_test=False
+            )
+        out = paddle.matmul(weights, v)
         return out
 
     q, k, v = __compute_qkv(queries, keys, values, n_head, d_key, d_value)
@@ -128,72 +127,59 @@ def multi_head_attention(queries,
         # Since the inplace reshape in __split_heads changes the shape of k and
         # v, which is the cache input for next time step, reshape the cache
         # input from the previous time step first.
-        k = cache["k"] = layers.concat(
-            [layers.reshape(cache["k"], shape=[0, 0, d_model]), k], axis=1)
-        v = cache["v"] = layers.concat(
-            [layers.reshape(cache["v"], shape=[0, 0, d_model]), v], axis=1)
+        k = cache["k"] = layers.concat([layers.reshape(cache["k"], shape=[0, 0, d_model]), k], axis=1)
+        v = cache["v"] = layers.concat([layers.reshape(cache["v"], shape=[0, 0, d_model]), v], axis=1)
 
     q = __split_heads(q, n_head)
     k = __split_heads(k, n_head)
     v = __split_heads(v, n_head)
 
-    ctx_multiheads = scaled_dot_product_attention(q, k, v, attn_bias, d_key,
-                                                  dropout_rate)
+    ctx_multiheads = scaled_dot_product_attention(q, k, v, attn_bias, d_key, dropout_rate)
 
     out = __combine_heads(ctx_multiheads)
 
     # Project back to the model size.
-    proj_out = layers.fc(input=out,
-                         size=d_model,
-                         num_flatten_dims=2,
-                         param_attr=fluid.ParamAttr(
-                             name=name + '_output_fc.w_0',
-                             initializer=param_initializer),
-                         bias_attr=name + '_output_fc.b_0')
+    proj_out = layers.fc(
+        input=out,
+        size=d_model,
+        num_flatten_dims=2,
+        param_attr=fluid.ParamAttr(name=name + "_output_fc.w_0", initializer=param_initializer),
+        bias_attr=name + "_output_fc.b_0",
+    )
     return proj_out
 
 
-def positionwise_feed_forward(x,
-                              d_inner_hid,
-                              d_hid,
-                              dropout_rate,
-                              hidden_act,
-                              param_initializer=None,
-                              name='ffn'):
+def positionwise_feed_forward(x, d_inner_hid, d_hid, dropout_rate, hidden_act, param_initializer=None, name="ffn"):
     """
     Position-wise Feed-Forward Networks.
     This module consists of two linear transformations with a ReLU activation
     in between, which is applied to each position separately and identically.
     """
-    hidden = layers.fc(input=x,
-                       size=d_inner_hid,
-                       num_flatten_dims=2,
-                       act=hidden_act,
-                       param_attr=fluid.ParamAttr(
-                           name=name + '_fc_0.w_0',
-                           initializer=param_initializer),
-                       bias_attr=name + '_fc_0.b_0')
+    hidden = layers.fc(
+        input=x,
+        size=d_inner_hid,
+        num_flatten_dims=2,
+        act=hidden_act,
+        param_attr=fluid.ParamAttr(name=name + "_fc_0.w_0", initializer=param_initializer),
+        bias_attr=name + "_fc_0.b_0",
+    )
     if dropout_rate:
-        hidden = layers.dropout(hidden,
-                                dropout_prob=dropout_rate,
-                                dropout_implementation="upscale_in_train",
-                                is_test=False)
-    out = layers.fc(input=hidden,
-                    size=d_hid,
-                    num_flatten_dims=2,
-                    param_attr=fluid.ParamAttr(name=name + '_fc_1.w_0',
-                                               initializer=param_initializer),
-                    bias_attr=name + '_fc_1.b_0')
+        hidden = layers.dropout(
+            hidden, dropout_prob=dropout_rate, dropout_implementation="upscale_in_train", is_test=False
+        )
+    out = layers.fc(
+        input=hidden,
+        size=d_hid,
+        num_flatten_dims=2,
+        param_attr=fluid.ParamAttr(name=name + "_fc_1.w_0", initializer=param_initializer),
+        bias_attr=name + "_fc_1.b_0",
+    )
     return out
 
 
-def pre_post_process_layer(prev_out,
-                           out,
-                           process_cmd,
-                           dropout_rate=0.,
-                           name=''):
+def pre_post_process_layer(prev_out, out, process_cmd, dropout_rate=0.0, name=""):
     """
-    Add residual connection, layer normalization and droput to the out tensor
+    Add residual connection, layer normalization and dropout to the out tensor
     optionally according to the value of process_cmd.
     This will be used before or after multi-head attention and position-wise
     feed-forward networks.
@@ -209,19 +195,17 @@ def pre_post_process_layer(prev_out,
                 out,
                 begin_norm_axis=len(out.shape) - 1,
                 param_attr=fluid.ParamAttr(
-                    name=name + '_layer_norm_scale',
-                    initializer=fluid.initializer.Constant(1.)),
-                bias_attr=fluid.ParamAttr(
-                    name=name + '_layer_norm_bias',
-                    initializer=fluid.initializer.Constant(0.)))
+                    name=name + "_layer_norm_scale", initializer=fluid.initializer.Constant(1.0)
+                ),
+                bias_attr=fluid.ParamAttr(name=name + "_layer_norm_bias", initializer=fluid.initializer.Constant(0.0)),
+            )
             if out_dtype == fluid.core.VarDesc.VarType.FP16:
                 out = layers.cast(x=out, dtype="float16")
         elif cmd == "d":  # add dropout
             if dropout_rate:
-                out = layers.dropout(out,
-                                     dropout_prob=dropout_rate,
-                                     dropout_implementation="upscale_in_train",
-                                     is_test=False)
+                out = layers.dropout(
+                    out, dropout_prob=dropout_rate, dropout_implementation="upscale_in_train", is_test=False
+                )
     return out
 
 
@@ -229,68 +213,88 @@ pre_process_layer = partial(pre_post_process_layer, None)
 post_process_layer = pre_post_process_layer
 
 
-def encoder_layer(enc_input,
-                  attn_bias,
-                  n_head,
-                  d_key,
-                  d_value,
-                  d_model,
-                  d_inner_hid,
-                  prepostprocess_dropout,
-                  attention_dropout,
-                  relu_dropout,
-                  hidden_act,
-                  preprocess_cmd="n",
-                  postprocess_cmd="da",
-                  param_initializer=None,
-                  name=''):
+def encoder_layer(
+    enc_input,
+    attn_bias,
+    n_head,
+    d_key,
+    d_value,
+    d_model,
+    d_inner_hid,
+    prepostprocess_dropout,
+    attention_dropout,
+    relu_dropout,
+    hidden_act,
+    preprocess_cmd="n",
+    postprocess_cmd="da",
+    param_initializer=None,
+    name="",
+):
     """The encoder layers that can be stacked to form a deep encoder.
-    This module consits of a multi-head (self) attention followed by
+    This module consists of a multi-head (self) attention followed by
     position-wise feed-forward networks and both the two components companied
     with the post_process_layer to add residual connection, layer normalization
-    and droput.
+    and dropout.
     """
-    attn_output = multi_head_attention(pre_process_layer(enc_input,
-                                                         preprocess_cmd,
-                                                         prepostprocess_dropout,
-                                                         name=name +
-                                                         '_pre_att'),
-                                       None,
-                                       None,
-                                       attn_bias,
-                                       d_key,
-                                       d_value,
-                                       d_model,
-                                       n_head,
-                                       attention_dropout,
-                                       param_initializer=param_initializer,
-                                       name=name + '_multi_head_att')
-    attn_output = post_process_layer(enc_input,
-                                     attn_output,
-                                     postprocess_cmd,
-                                     prepostprocess_dropout,
-                                     name=name + '_post_att')
-    ffd_output = positionwise_feed_forward(pre_process_layer(
-        attn_output,
-        preprocess_cmd,
-        prepostprocess_dropout,
-        name=name + '_pre_ffn'),
-                                           d_inner_hid,
-                                           d_model,
-                                           relu_dropout,
-                                           hidden_act,
-                                           param_initializer=param_initializer,
-                                           name=name + '_ffn')
-    return post_process_layer(attn_output,
-                              ffd_output,
-                              postprocess_cmd,
-                              prepostprocess_dropout,
-                              name=name + '_post_ffn'), ffd_output
+    attn_output = multi_head_attention(
+        pre_process_layer(enc_input, preprocess_cmd, prepostprocess_dropout, name=name + "_pre_att"),
+        None,
+        None,
+        attn_bias,
+        d_key,
+        d_value,
+        d_model,
+        n_head,
+        attention_dropout,
+        param_initializer=param_initializer,
+        name=name + "_multi_head_att",
+    )
+    attn_output = post_process_layer(
+        enc_input, attn_output, postprocess_cmd, prepostprocess_dropout, name=name + "_post_att"
+    )
+    ffd_output = positionwise_feed_forward(
+        pre_process_layer(attn_output, preprocess_cmd, prepostprocess_dropout, name=name + "_pre_ffn"),
+        d_inner_hid,
+        d_model,
+        relu_dropout,
+        hidden_act,
+        param_initializer=param_initializer,
+        name=name + "_ffn",
+    )
+    return (
+        post_process_layer(attn_output, ffd_output, postprocess_cmd, prepostprocess_dropout, name=name + "_post_ffn"),
+        ffd_output,
+    )
 
 
-def encoder(enc_input,
+def encoder(
+    enc_input,
+    attn_bias,
+    n_layer,
+    n_head,
+    d_key,
+    d_value,
+    d_model,
+    d_inner_hid,
+    prepostprocess_dropout,
+    attention_dropout,
+    relu_dropout,
+    hidden_act,
+    preprocess_cmd="n",
+    postprocess_cmd="da",
+    param_initializer=None,
+    model_name="",
+    name="",
+):
+    """
+    The encoder is composed of a stack of identical layers returned by calling
+    encoder_layer.
+    """
+    checkpoints = []
+    for i in range(n_layer):
+        enc_output, cp = encoder_layer(
+            enc_input,
             attn_bias,
-            n_layer,
             n_head,
             d_key,
             d_value,
@@ -300,37 +304,15 @@ def encoder(enc_input,
             attention_dropout,
             relu_dropout,
             hidden_act,
-            preprocess_cmd="n",
-            postprocess_cmd="da",
-            param_initializer=None,
-            model_name='',
-            name=''):
-    """
-    The encoder is composed of a stack of identical layers returned by calling
-    encoder_layer.
-    """
-    checkpoints = []
-    for i in range(n_layer):
-        enc_output, cp = encoder_layer(enc_input,
-                                       attn_bias,
-                                       n_head,
-                                       d_key,
-                                       d_value,
-                                       d_model,
-                                       d_inner_hid,
-                                       prepostprocess_dropout,
-                                       attention_dropout,
-                                       relu_dropout,
-                                       hidden_act,
-                                       preprocess_cmd,
-                                       postprocess_cmd,
-                                       param_initializer=param_initializer,
-                                       name=name + '_layer_' + str(i))
+            preprocess_cmd,
+            postprocess_cmd,
+            param_initializer=param_initializer,
+            name=name + "_layer_" + str(i),
+        )
         checkpoints.append(cp)
         enc_input = enc_output
-    enc_output = pre_process_layer(enc_output,
-                                   preprocess_cmd,
-                                   prepostprocess_dropout,
-                                   name=model_name + "post_encoder")
+    enc_output = pre_process_layer(
+        enc_output, preprocess_cmd, prepostprocess_dropout, name=model_name + "post_encoder"
+    )
 
     return enc_output, checkpoints

applications/document_intelligence/doc_vqa/Rerank/src/optimization.py

@@ -13,27 +13,16 @@
 # limitations under the License.
 """Optimization and learning rate scheduling."""
 
-from __future__ import absolute_import
-from __future__ import division
-from __future__ import print_function
-from __future__ import unicode_literals
-from __future__ import absolute_import
-
-import numpy as np
-
 import paddle.fluid as fluid
-from paddle.fluid.incubate.fleet.collective import fleet, DistributedStrategy
+from paddle.fluid.incubate.fleet.collective import fleet
 
 
 def linear_warmup_decay(learning_rate, warmup_steps, num_train_steps):
-    """ Applies linear warmup of learning rate from 0 and decay to 0."""
+    """Applies linear warmup of learning rate from 0 and decay to 0."""
     with fluid.default_main_program()._lr_schedule_guard():
         lr = fluid.layers.tensor.create_global_var(
-            shape=[1],
-            value=0.0,
-            dtype='float32',
-            persistable=True,
-            name="scheduled_learning_rate")
+            shape=[1], value=0.0, dtype="float32", persistable=True, name="scheduled_learning_rate"
+        )
 
         global_step = fluid.layers.learning_rate_scheduler._decay_step_counter()
 
@@ -47,41 +36,41 @@ def linear_warmup_decay(learning_rate, warmup_steps, num_train_steps):
                     decay_steps=num_train_steps,
                     end_learning_rate=0.0,
                     power=1.0,
-                    cycle=False)
+                    cycle=False,
+                )
                 fluid.layers.tensor.assign(decayed_lr, lr)
 
         return lr
 
 
-def optimization(loss,
-                 warmup_steps,
-                 num_train_steps,
-                 learning_rate,
-                 train_program,
-                 startup_prog,
-                 weight_decay,
-                 scheduler='linear_warmup_decay',
-                 use_dynamic_loss_scaling=False,
-                 incr_every_n_steps=1000,
-                 decr_every_n_nan_or_inf=2,
-                 incr_ratio=2.0,
-                 decr_ratio=0.8,
-                 dist_strategy=None,
-                 use_lamb=False):
+def optimization(
+    loss,
+    warmup_steps,
+    num_train_steps,
+    learning_rate,
+    train_program,
+    startup_prog,
+    weight_decay,
+    scheduler="linear_warmup_decay",
+    use_dynamic_loss_scaling=False,
+    incr_every_n_steps=1000,
+    decr_every_n_nan_or_inf=2,
+    incr_ratio=2.0,
+    decr_ratio=0.8,
+    dist_strategy=None,
+    use_lamb=False,
+):
     if warmup_steps > 0:
-        if scheduler == 'noam_decay':
-            scheduled_lr = fluid.layers.learning_rate_scheduler\
-             .noam_decay(1/(warmup_steps *(learning_rate ** 2)),
-                         warmup_steps)
-        elif scheduler == 'linear_warmup_decay':
-            scheduled_lr = linear_warmup_decay(learning_rate, warmup_steps,
-                                               num_train_steps)
+        if scheduler == "noam_decay":
+            scheduled_lr = fluid.layers.learning_rate_scheduler.noam_decay(
+                1 / (warmup_steps * (learning_rate**2)), warmup_steps
+            )
+        elif scheduler == "linear_warmup_decay":
+            scheduled_lr = linear_warmup_decay(learning_rate, warmup_steps, num_train_steps)
         else:
-            raise ValueError("Unkown learning rate scheduler, should be "
-                             "'noam_decay' or 'linear_warmup_decay'")
+            raise ValueError("Unknown learning rate scheduler, should be " "'noam_decay' or 'linear_warmup_decay'")
         if use_lamb:
-            optimizer = fluid.optimizer.LambOptimizer(
-                learning_rate=scheduled_lr)
+            optimizer = fluid.optimizer.LambOptimizer(learning_rate=scheduled_lr)
         else:
             optimizer = fluid.optimizer.Adam(learning_rate=scheduled_lr)
     else:
@@ -89,18 +78,16 @@ def optimization(loss,
             name=fluid.unique_name.generate("learning_rate"),
             shape=[1],
             value=learning_rate,
-            dtype='float32',
-            persistable=True)
+            dtype="float32",
+            persistable=True,
+        )
         if use_lamb:
-            optimizer = fluid.optimizer.LambOptimizer(
-                learning_rate=scheduled_lr)
+            optimizer = fluid.optimizer.LambOptimizer(learning_rate=scheduled_lr)
         else:
             optimizer = fluid.optimizer.Adam(learning_rate=scheduled_lr)
-        optimizer._learning_rate_map[
-            fluid.default_main_program()] = scheduled_lr
+        optimizer._learning_rate_map[fluid.default_main_program()] = scheduled_lr
 
-    fluid.clip.set_gradient_clip(clip=fluid.clip.GradientClipByGlobalNorm(
-        clip_norm=1.0))
+    fluid.clip.set_gradient_clip(clip=fluid.clip.GradientClipByGlobalNorm(clip_norm=1.0))
 
     def exclude_from_weight_decay(name):
         if name.find("layer_norm") > -1:
@@ -119,8 +106,7 @@ def optimization(loss,
 
     if dist_strategy is not None:
         # use fleet api
-        optimizer = fleet.distributed_optimizer(optimizer,
-                                                strategy=dist_strategy)
+        optimizer = fleet.distributed_optimizer(optimizer, strategy=dist_strategy)
 
     _, param_grads = optimizer.minimize(loss)
 
@@ -128,10 +114,8 @@ def optimization(loss,
         for param, grad in param_grads:
             if exclude_from_weight_decay(param.name):
                 continue
-            with param.block.program._optimized_guard(
-                [param, grad]), fluid.framework.name_scope("weight_decay"):
-                updated_param = param - param_list[
-                    param.name] * weight_decay * scheduled_lr
+            with param.block.program._optimized_guard([param, grad]), fluid.framework.name_scope("weight_decay"):
+                updated_param = param - param_list[param.name] * weight_decay * scheduled_lr
                 fluid.layers.assign(output=param, input=updated_param)
 
     return scheduled_lr

applications/document_intelligence/doc_vqa/Rerank/src/reader_ce.py

@@ -11,22 +11,15 @@
 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 # See the License for the specific language governing permissions and
 # limitations under the License.
-from __future__ import absolute_import
-from __future__ import division
-from __future__ import print_function
-from __future__ import unicode_literals
-from __future__ import absolute_import
 
-import sys
-import os
 import json
-import random
 import logging
+import sys
+from collections import namedtuple
+from io import open
+
 import numpy as np
 import six
-from io import open
-from collections import namedtuple
-
 import tokenization
 from batching import pad_batch_data
 
@@ -34,16 +27,16 @@ log = logging.getLogger(__name__)
 
 if six.PY3:
     import io
-    sys.stdout = io.TextIOWrapper(sys.stdout.buffer, encoding='utf-8')
-    sys.stderr = io.TextIOWrapper(sys.stderr.buffer, encoding='utf-8')
+
+    sys.stdout = io.TextIOWrapper(sys.stdout.buffer, encoding="utf-8")
+    sys.stderr = io.TextIOWrapper(sys.stderr.buffer, encoding="utf-8")
 
 
-def csv_reader(fd, delimiter='\t', trainer_id=0, trainer_num=1):
-
+def csv_reader(fd, delimiter="\t", trainer_id=0, trainer_num=1):
     def gen():
         for i, line in enumerate(fd):
             if i % trainer_num == trainer_id:
-                slots = line.rstrip('\n').split(delimiter)
+                slots = line.rstrip("\n").split(delimiter)
                 if len(slots) == 1:
                     yield slots,
                 else:
@@ -53,22 +46,22 @@ def csv_reader(fd, delimiter='\t', trainer_id=0, trainer_num=1):
 
 
 class BaseReader(object):
-
-    def __init__(self,
-                 vocab_path,
-                 label_map_config=None,
-                 max_seq_len=512,
-                 total_num=0,
-                 do_lower_case=True,
-                 in_tokens=False,
-                 is_inference=False,
-                 random_seed=None,
-                 tokenizer="FullTokenizer",
-                 for_cn=True,
-                 task_id=0):
+    def __init__(
+        self,
+        vocab_path,
+        label_map_config=None,
+        max_seq_len=512,
+        total_num=0,
+        do_lower_case=True,
+        in_tokens=False,
+        is_inference=False,
+        random_seed=None,
+        tokenizer="FullTokenizer",
+        for_cn=True,
+        task_id=0,
+    ):
         self.max_seq_len = max_seq_len
-        self.tokenizer = tokenization.FullTokenizer(vocab_file=vocab_path,
-                                                    do_lower_case=do_lower_case)
+        self.tokenizer = tokenization.FullTokenizer(vocab_file=vocab_path, do_lower_case=do_lower_case)
         self.vocab = self.tokenizer.vocab
         self.pad_id = self.vocab["[PAD]"]
         self.cls_id = self.vocab["[CLS]"]
@@ -86,7 +79,7 @@ class BaseReader(object):
         self.total_num = total_num
 
         if label_map_config:
-            with open(label_map_config, encoding='utf8') as f:
+            with open(label_map_config, encoding="utf8") as f:
                 self.label_map = json.load(f)
         else:
             self.label_map = None
@@ -97,10 +90,10 @@ class BaseReader(object):
 
     def _read_tsv(self, input_file, quotechar=None):
         """Reads a tab separated value file."""
-        with open(input_file, 'r', encoding='utf8') as f:
+        with open(input_file, "r", encoding="utf8") as f:
             reader = csv_reader(f)
             headers = next(reader)
-            Example = namedtuple('Example', headers)
+            Example = namedtuple("Example", headers)
 
             examples = []
             for line in reader:
@@ -157,7 +150,7 @@ class BaseReader(object):
         # it easier for the model to learn the concept of sequences.
         #
         # For classification tasks, the first vector (corresponding to [CLS]) is
-        # used as as the "sentence vector". Note that this only makes sense because
+        # used as the "sentence vector". Note that this only makes sense because
         # the entire model is fine-tuned.
         tokens = []
         text_type_ids = []
@@ -180,30 +173,23 @@ class BaseReader(object):
         position_ids = list(range(len(token_ids)))
 
         if self.is_inference:
-            Record = namedtuple('Record',
-                                ['token_ids', 'text_type_ids', 'position_ids'])
-            record = Record(token_ids=token_ids,
-                            text_type_ids=text_type_ids,
-                            position_ids=position_ids)
+            Record = namedtuple("Record", ["token_ids", "text_type_ids", "position_ids"])
+            record = Record(token_ids=token_ids, text_type_ids=text_type_ids, position_ids=position_ids)
         else:
             if self.label_map:
                 label_id = self.label_map[example.label]
             else:
                 label_id = example.label
 
-            Record = namedtuple('Record', [
-                'token_ids', 'text_type_ids', 'position_ids', 'label_id', 'qid'
-            ])
+            Record = namedtuple("Record", ["token_ids", "text_type_ids", "position_ids", "label_id", "qid"])
 
             qid = None
             if "qid" in example._fields:
                 qid = example.qid
 
-            record = Record(token_ids=token_ids,
-                            text_type_ids=text_type_ids,
-                            position_ids=position_ids,
-                            label_id=label_id,
-                            qid=qid)
+            record = Record(
+                token_ids=token_ids, text_type_ids=text_type_ids, position_ids=position_ids, label_id=label_id, qid=qid
+            )
         return record
 
     def _prepare_batch_data(self, examples, batch_size, phase=None):
@@ -212,8 +198,7 @@ class BaseReader(object):
         for index, example in enumerate(examples):
             if phase == "train":
                 self.current_example = index
-            record = self._convert_example_to_record(example, self.max_seq_len,
-                                                     self.tokenizer)
+            record = self._convert_example_to_record(example, self.max_seq_len, self.tokenizer)
             max_len = max(max_len, len(record.token_ids))
             if self.in_tokens:
                 to_append = (len(batch_records) + 1) * max_len <= batch_size
@@ -233,25 +218,18 @@ class BaseReader(object):
         #        return len(examples)
         return self.num_examples
 
-    def data_generator(self,
-                       input_file,
-                       batch_size,
-                       epoch,
-                       dev_count=1,
-                       trainer_id=0,
-                       trainer_num=1,
-                       shuffle=True,
-                       phase=None):
+    def data_generator(
+        self, input_file, batch_size, epoch, dev_count=1, trainer_id=0, trainer_num=1, shuffle=True, phase=None
+    ):
 
-        if phase == 'train':
+        if phase == "train":
             #            examples = examples[trainer_id: (len(examples) //trainer_num) * trainer_num : trainer_num]
             self.num_examples_per_node = self.total_num // trainer_num
             self.num_examples = self.num_examples_per_node * trainer_num
-            examples = self._read_tsv(input_file,
-                                      trainer_id=trainer_id,
-                                      trainer_num=trainer_num,
-                                      num_examples=self.num_examples_per_node)
-            log.info('apply sharding %d/%d' % (trainer_id, trainer_num))
+            examples = self._read_tsv(
+                input_file, trainer_id=trainer_id, trainer_num=trainer_num, num_examples=self.num_examples_per_node
+            )
+            log.info("apply sharding %d/%d" % (trainer_id, trainer_num))
         else:
             examples = self._read_tsv(input_file)
 
@@ -264,9 +242,7 @@ class BaseReader(object):
                 if shuffle:
                     np.random.shuffle(examples)
 
-                for batch_data in self._prepare_batch_data(examples,
-                                                           batch_size,
-                                                           phase=phase):
+                for batch_data in self._prepare_batch_data(examples, batch_size, phase=phase):
                     if len(all_dev_batches) < dev_count:
                         all_dev_batches.append(batch_data)
                     if len(all_dev_batches) == dev_count:
@@ -278,32 +254,23 @@ class BaseReader(object):
             try:
                 for i in wrapper():
                     yield i
-            except Exception as e:
+            except Exception:
                 import traceback
+
                 traceback.print_exc()
 
         return f
 
 
 class ClassifyReader(BaseReader):
-
-    def _read_tsv(self,
-                  input_file,
-                  quotechar=None,
-                  trainer_id=0,
-                  trainer_num=1,
-                  num_examples=0):
+    def _read_tsv(self, input_file, quotechar=None, trainer_id=0, trainer_num=1, num_examples=0):
         """Reads a tab separated value file."""
-        with open(input_file, 'r', encoding='utf8') as f:
-            reader = csv_reader(f,
-                                trainer_id=trainer_id,
-                                trainer_num=trainer_num)
+        with open(input_file, "r", encoding="utf8") as f:
+            reader = csv_reader(f, trainer_id=trainer_id, trainer_num=trainer_num)
             #            headers = next(reader)
-            headers = 'query\ttitle\tpara\tlabel'.split('\t')
-            text_indices = [
-                index for index, h in enumerate(headers) if h != "label"
-            ]
-            Example = namedtuple('Example', headers)
+            headers = "query\ttitle\tpara\tlabel".split("\t")
+            text_indices = [index for index, h in enumerate(headers) if h != "label"]
+            Example = namedtuple("Example", headers)
 
             examples = []
             for cnt, line in enumerate(reader):
@@ -312,7 +279,7 @@ class ClassifyReader(BaseReader):
                 for index, text in enumerate(line):
                     if index in text_indices:
                         if self.for_cn:
-                            line[index] = text.replace(' ', '')
+                            line[index] = text.replace(" ", "")
                         else:
                             line[index] = text
                 example = Example(*line)
@@ -326,31 +293,21 @@ class ClassifyReader(BaseReader):
 
         if not self.is_inference:
             batch_labels = [record.label_id for record in batch_records]
-            batch_labels = np.array(batch_labels).astype("int64").reshape(
-                [-1, 1])
+            batch_labels = np.array(batch_labels).astype("int64").reshape([-1, 1])
 
             if batch_records[0].qid:
                 batch_qids = [record.qid for record in batch_records]
-                batch_qids = np.array(batch_qids).astype("int64").reshape(
-                    [-1, 1])
+                batch_qids = np.array(batch_qids).astype("int64").reshape([-1, 1])
             else:
                 batch_qids = np.array([]).astype("int64").reshape([-1, 1])
 
         # padding
-        padded_token_ids, input_mask = pad_batch_data(batch_token_ids,
-                                                      pad_idx=self.pad_id,
-                                                      return_input_mask=True)
-        padded_text_type_ids = pad_batch_data(batch_text_type_ids,
-                                              pad_idx=self.pad_id)
-        padded_position_ids = pad_batch_data(batch_position_ids,
-                                             pad_idx=self.pad_id)
-        padded_task_ids = np.ones_like(padded_token_ids,
-                                       dtype="int64") * self.task_id
+        padded_token_ids, input_mask = pad_batch_data(batch_token_ids, pad_idx=self.pad_id, return_input_mask=True)
+        padded_text_type_ids = pad_batch_data(batch_text_type_ids, pad_idx=self.pad_id)
+        padded_position_ids = pad_batch_data(batch_position_ids, pad_idx=self.pad_id)
+        padded_task_ids = np.ones_like(padded_token_ids, dtype="int64") * self.task_id
 
-        return_list = [
-            padded_token_ids, padded_text_type_ids, padded_position_ids,
-            padded_task_ids, input_mask
-        ]
+        return_list = [padded_token_ids, padded_text_type_ids, padded_position_ids, padded_task_ids, input_mask]
         if not self.is_inference:
             return_list += [batch_labels, batch_qids]
 

applications/document_intelligence/doc_vqa/Rerank/src/tokenization.py

@@ -20,7 +20,6 @@ from __future__ import print_function
 from __future__ import unicode_literals
 from __future__ import absolute_import
 
-import six
 import collections
 import unicodedata
 from io import open
@@ -28,51 +27,28 @@ from io import open
 
 def convert_to_unicode(text):
     """Converts `text` to Unicode (if it's not already), assuming utf-8 input."""
-    if six.PY3:
-        if isinstance(text, str):
-            return text
-        elif isinstance(text, bytes):
-            return text.decode("utf-8", "ignore")
-        else:
-            raise ValueError("Unsupported string type: %s" % (type(text)))
-    elif six.PY2:
-        if isinstance(text, str):
-            return text.decode("utf-8", "ignore")
-        elif isinstance(text, unicode):
-            return text
-        else:
-            raise ValueError("Unsupported string type: %s" % (type(text)))
+    if isinstance(text, str):
+        return text
+    elif isinstance(text, bytes):
+        return text.decode("utf-8", "ignore")
     else:
-        raise ValueError("Not running on Python2 or Python 3?")
+        raise ValueError("Unsupported string type: %s" % (type(text)))
 
 
 def printable_text(text):
     """Returns text encoded in a way suitable for print or `tf.logging`."""
-
-    # These functions want `str` for both Python2 and Python3, but in one case
-    # it's a Unicode string and in the other it's a byte string.
-    if six.PY3:
-        if isinstance(text, str):
-            return text
-        elif isinstance(text, bytes):
-            return text.decode("utf-8", "ignore")
-        else:
-            raise ValueError("Unsupported string type: %s" % (type(text)))
-    elif six.PY2:
-        if isinstance(text, str):
-            return text
-        elif isinstance(text, unicode):
-            return text.encode("utf-8")
-        else:
-            raise ValueError("Unsupported string type: %s" % (type(text)))
+    if isinstance(text, str):
+        return text
+    elif isinstance(text, bytes):
+        return text.decode("utf-8", "ignore")
     else:
-        raise ValueError("Not running on Python2 or Python 3?")
+        raise ValueError("Unsupported string type: %s" % (type(text)))
 
 
 def load_vocab(vocab_file):
     """Loads a vocabulary file into a dictionary."""
     vocab = collections.OrderedDict()
-    with open(vocab_file, encoding='utf8') as fin:
+    with open(vocab_file, encoding="utf8") as fin:
         for num, line in enumerate(fin):
             items = convert_to_unicode(line.strip()).split("\t")
             if len(items) > 2:
@@ -101,7 +77,7 @@ def convert_ids_to_tokens(inv_vocab, ids):
 
 
 def whitespace_tokenize(text):
-    """Runs basic whitespace cleaning and splitting on a peice of text."""
+    """Runs basic whitespace cleaning and splitting on a piece of text."""
     text = text.strip()
     if not text:
         return []
@@ -110,7 +86,7 @@ def whitespace_tokenize(text):
 
 
 class FullTokenizer(object):
-    """Runs end-to-end tokenziation."""
+    """Runs end-to-end tokenization."""
 
     def __init__(self, vocab_file, do_lower_case=True):
         self.vocab = load_vocab(vocab_file)
@@ -134,7 +110,7 @@ class FullTokenizer(object):
 
 
 class CharTokenizer(object):
-    """Runs end-to-end tokenziation."""
+    """Runs end-to-end tokenization."""
 
     def __init__(self, vocab_file, do_lower_case=True):
         self.vocab = load_vocab(vocab_file)
@@ -245,14 +221,16 @@ class BasicTokenizer(object):
         # as is Japanese Hiragana and Katakana. Those alphabets are used to write
         # space-separated words, so they are not treated specially and handled
         # like the all of the other languages.
-        if ((cp >= 0x4E00 and cp <= 0x9FFF) or  #
-            (cp >= 0x3400 and cp <= 0x4DBF) or  #
-            (cp >= 0x20000 and cp <= 0x2A6DF) or  #
-            (cp >= 0x2A700 and cp <= 0x2B73F) or  #
-            (cp >= 0x2B740 and cp <= 0x2B81F) or  #
-            (cp >= 0x2B820 and cp <= 0x2CEAF) or
-            (cp >= 0xF900 and cp <= 0xFAFF) or  #
-            (cp >= 0x2F800 and cp <= 0x2FA1F)):  #
+        if (
+            (cp >= 0x4E00 and cp <= 0x9FFF)
+            or (cp >= 0x3400 and cp <= 0x4DBF)  #
+            or (cp >= 0x20000 and cp <= 0x2A6DF)  #
+            or (cp >= 0x2A700 and cp <= 0x2B73F)  #
+            or (cp >= 0x2B740 and cp <= 0x2B81F)  #
+            or (cp >= 0x2B820 and cp <= 0x2CEAF)  #
+            or (cp >= 0xF900 and cp <= 0xFAFF)
+            or (cp >= 0x2F800 and cp <= 0x2FA1F)  #
+        ):  #
             return True
 
         return False
@@ -262,7 +240,7 @@ class BasicTokenizer(object):
         output = []
         for char in text:
             cp = ord(char)
-            if cp == 0 or cp == 0xfffd or _is_control(char):
+            if cp == 0 or cp == 0xFFFD or _is_control(char):
                 continue
             if _is_whitespace(char):
                 output.append(" ")
@@ -272,7 +250,7 @@ class BasicTokenizer(object):
 
 
 class WordpieceTokenizer(object):
-    """Runs WordPiece tokenziation."""
+    """Runs WordPiece tokenization."""
 
     def __init__(self, vocab, unk_token="[UNK]", max_input_chars_per_word=100):
         self.vocab = vocab
@@ -335,7 +313,7 @@ class WordpieceTokenizer(object):
 
 def _is_whitespace(char):
     """Checks whether `chars` is a whitespace character."""
-    # \t, \n, and \r are technically contorl characters but we treat them
+    # \t, \n, and \r are technically control characters but we treat them
     # as whitespace since they are generally considered as such.
     if char == " " or char == "\t" or char == "\n" or char == "\r":
         return True
@@ -364,8 +342,7 @@ def _is_punctuation(char):
     # Characters such as "^", "$", and "`" are not in the Unicode
     # Punctuation class but we treat them as punctuation anyways, for
     # consistency.
-    if ((cp >= 33 and cp <= 47) or (cp >= 58 and cp <= 64)
-            or (cp >= 91 and cp <= 96) or (cp >= 123 and cp <= 126)):
+    if (cp >= 33 and cp <= 47) or (cp >= 58 and cp <= 64) or (cp >= 91 and cp <= 96) or (cp >= 123 and cp <= 126):
         return True
     cat = unicodedata.category(char)
     if cat.startswith("P"):
@@ -386,14 +363,16 @@ def tokenize_chinese_chars(text):
         # as is Japanese Hiragana and Katakana. Those alphabets are used to write
         # space-separated words, so they are not treated specially and handled
         # like the all of the other languages.
-        if ((cp >= 0x4E00 and cp <= 0x9FFF) or  #
-            (cp >= 0x3400 and cp <= 0x4DBF) or  #
-            (cp >= 0x20000 and cp <= 0x2A6DF) or  #
-            (cp >= 0x2A700 and cp <= 0x2B73F) or  #
-            (cp >= 0x2B740 and cp <= 0x2B81F) or  #
-            (cp >= 0x2B820 and cp <= 0x2CEAF) or
-            (cp >= 0xF900 and cp <= 0xFAFF) or  #
-            (cp >= 0x2F800 and cp <= 0x2FA1F)):  #
+        if (
+            (cp >= 0x4E00 and cp <= 0x9FFF)
+            or (cp >= 0x3400 and cp <= 0x4DBF)  #
+            or (cp >= 0x20000 and cp <= 0x2A6DF)  #
+            or (cp >= 0x2A700 and cp <= 0x2B73F)  #
+            or (cp >= 0x2B740 and cp <= 0x2B81F)  #
+            or (cp >= 0x2B820 and cp <= 0x2CEAF)  #
+            or (cp >= 0xF900 and cp <= 0xFAFF)
+            or (cp >= 0x2F800 and cp <= 0x2FA1F)  #
+        ):  #
             return True
 
         return False

applications/document_intelligence/doc_vqa/Rerank/src/train_ce.py

@@ -13,37 +13,36 @@
 # limitations under the License.
 """Finetuning on classification tasks."""
 
-from __future__ import absolute_import
-from __future__ import division
-from __future__ import print_function
-from __future__ import unicode_literals
-from __future__ import absolute_import
+from __future__ import absolute_import, division, print_function, unicode_literals
 
+import logging
+import multiprocessing
 import os
 import time
-import logging
 import warnings
-import multiprocessing
 
 # NOTE(paddle-dev): All of these flags should be
 # set before `import paddle`. Otherwise, it would
 # not take any effect.
-os.environ['FLAGS_eager_delete_tensor_gb'] = '0'  # enable gc
+os.environ["FLAGS_eager_delete_tensor_gb"] = "0"  # enable gc
 
-import paddle
-import paddle.fluid as fluid
-if hasattr(paddle, 'enable_static'):
+import paddle  # noqa: E402
+import paddle.fluid as fluid  # noqa: E402
+
+if hasattr(paddle, "enable_static"):
     paddle.enable_static()
-from paddle.fluid.incubate.fleet.collective import fleet, DistributedStrategy
-import paddle.fluid.incubate.fleet.base.role_maker as role_maker
-
-import reader_ce as reader_ce
-from model.ernie import ErnieConfig
-from cross_encoder import create_model, evaluate, predict
-from optimization import optimization
-from utils.args import print_arguments, check_cuda, prepare_logger
-from utils.init import init_pretraining_params, init_checkpoint
-from finetune_args import parser
+import paddle.fluid.incubate.fleet.base.role_maker as role_maker  # noqa: E402
+import reader_ce as reader_ce  # noqa: E402
+from cross_encoder import create_model, evaluate, predict  # noqa: E402
+from finetune_args import parser  # noqa: E402
+from model.ernie import ErnieConfig  # noqa: E402
+from optimization import optimization  # noqa: E402
+from paddle.fluid.incubate.fleet.collective import (  # noqa: E402
+    DistributedStrategy,
+    fleet,
+)
+from src.utils.args import check_cuda, prepare_logger, print_arguments  # noqa: E402
+from src.utils.init import init_checkpoint, init_pretraining_params  # noqa: E402
 
 warnings.filterwarnings("ignore")
 args = parser.parse_args()
@@ -60,23 +59,24 @@ def main(args):
         dev_count = len(dev_list)
     else:
         place = fluid.CPUPlace()
-        dev_count = int(os.environ.get('CPU_NUM', multiprocessing.cpu_count()))
+        dev_count = int(os.environ.get("CPU_NUM", multiprocessing.cpu_count()))
     exe = fluid.Executor(place)
 
-    reader = reader_ce.ClassifyReader(vocab_path=args.vocab_path,
-                                      label_map_config=args.label_map_config,
-                                      max_seq_len=args.max_seq_len,
-                                      total_num=args.train_data_size,
-                                      do_lower_case=args.do_lower_case,
-                                      in_tokens=args.in_tokens,
-                                      random_seed=args.random_seed,
-                                      tokenizer=args.tokenizer,
-                                      for_cn=args.for_cn,
-                                      task_id=args.task_id)
+    reader = reader_ce.ClassifyReader(
+        vocab_path=args.vocab_path,
+        label_map_config=args.label_map_config,
+        max_seq_len=args.max_seq_len,
+        total_num=args.train_data_size,
+        do_lower_case=args.do_lower_case,
+        in_tokens=args.in_tokens,
+        random_seed=args.random_seed,
+        tokenizer=args.tokenizer,
+        for_cn=args.for_cn,
+        task_id=args.task_id,
+    )
 
     if not (args.do_train or args.do_val or args.do_test):
-        raise ValueError("For args `do_train`, `do_val` and `do_test`, at "
-                         "least one of them must be True.")
+        raise ValueError("For args `do_train`, `do_val` and `do_test`, at " "least one of them must be True.")
 
     if args.do_test:
         assert args.test_save is not None
@@ -84,7 +84,7 @@ def main(args):
     if args.random_seed is not None:
         startup_prog.random_seed = args.random_seed
 
-    if args.predict_batch_size == None:
+    if args.predict_batch_size is None:
         args.predict_batch_size = args.batch_size
 
     if args.do_train:
@@ -100,13 +100,13 @@ def main(args):
             trainer_id=fleet.worker_index(),
             trainer_num=fleet.worker_num(),
             shuffle=True,
-            phase="train")
+            phase="train",
+        )
 
         num_train_examples = reader.get_num_examples(args.train_set)
 
         if args.in_tokens:
-            max_train_steps = args.epoch * num_train_examples // (
-                args.batch_size // args.max_seq_len) // dev_count
+            max_train_steps = args.epoch * num_train_examples // (args.batch_size // args.max_seq_len) // dev_count
         else:
             max_train_steps = args.epoch * num_train_examples // args.batch_size // dev_count
 
@@ -141,9 +141,8 @@ def main(args):
         with fluid.program_guard(train_program, startup_prog):
             with fluid.unique_name.guard():
                 train_pyreader, graph_vars = create_model(
-                    args,
-                    pyreader_name='train_reader',
-                    ernie_config=ernie_config)
+                    args, pyreader_name="train_reader", ernie_config=ernie_config
+                )
                 scheduled_lr = optimization(
                     loss=graph_vars["loss"],
                     warmup_steps=warmup_steps,
@@ -158,28 +157,27 @@ def main(args):
                     decr_every_n_nan_or_inf=args.decr_every_n_nan_or_inf,
                     incr_ratio=args.incr_ratio,
                     decr_ratio=args.decr_ratio,
-                    dist_strategy=dist_strategy)
+                    dist_strategy=dist_strategy,
+                )
 
         if args.verbose:
             if args.in_tokens:
                 lower_mem, upper_mem, unit = fluid.contrib.memory_usage(
-                    program=train_program,
-                    batch_size=args.batch_size // args.max_seq_len)
+                    program=train_program, batch_size=args.batch_size // args.max_seq_len
+                )
             else:
                 lower_mem, upper_mem, unit = fluid.contrib.memory_usage(
-                    program=train_program, batch_size=args.batch_size)
-            log.info("Theoretical memory usage in training: %.3f - %.3f %s" %
-                     (lower_mem, upper_mem, unit))
+                    program=train_program, batch_size=args.batch_size
+                )
+            log.info("Theoretical memory usage in training: %.3f - %.3f %s" % (lower_mem, upper_mem, unit))
 
     if args.do_val or args.do_test:
         test_prog = fluid.Program()
         with fluid.program_guard(test_prog, startup_prog):
             with fluid.unique_name.guard():
                 test_pyreader, graph_vars = create_model(
-                    args,
-                    pyreader_name='test_reader',
-                    ernie_config=ernie_config,
-                    is_prediction=True)
+                    args, pyreader_name="test_reader", ernie_config=ernie_config, is_prediction=True
+                )
 
         test_prog = test_prog.clone(for_test=True)
 
@@ -192,19 +190,15 @@ def main(args):
         if args.init_checkpoint and args.init_pretraining_params:
             log.warning(
                 "WARNING: args 'init_checkpoint' and 'init_pretraining_params' "
-                "both are set! Only arg 'init_checkpoint' is made valid.")
+                "both are set! Only arg 'init_checkpoint' is made valid."
+            )
         if args.init_checkpoint:
-            init_checkpoint(exe,
-                            args.init_checkpoint,
-                            main_program=startup_prog)
+            init_checkpoint(exe, args.init_checkpoint, main_program=startup_prog)
         elif args.init_pretraining_params:
-            init_pretraining_params(exe,
-                                    args.init_pretraining_params,
-                                    main_program=startup_prog)
+            init_pretraining_params(exe, args.init_pretraining_params, main_program=startup_prog)
     elif args.do_val or args.do_test:
         if not args.init_checkpoint:
-            raise ValueError("args 'init_checkpoint' should be set if"
-                             "only doing validation or testing!")
+            raise ValueError("args 'init_checkpoint' should be set if" "only doing validation or testing!")
         init_checkpoint(exe, args.init_checkpoint, main_program=startup_prog)
 
     if args.do_train:
@@ -237,19 +231,15 @@ def main(args):
                     train_exe.run(fetch_list=[], program=train_program)
 
                 else:
-                    outputs = evaluate(train_exe,
-                                       train_program,
-                                       train_pyreader,
-                                       graph_vars,
-                                       "train",
-                                       metric=args.metric)
+                    outputs = evaluate(
+                        train_exe, train_program, train_pyreader, graph_vars, "train", metric=args.metric
+                    )
 
                     if args.verbose:
-                        verbose = "train pyreader queue size: %d, " % train_pyreader.queue.size(
-                        )
+                        verbose = "train pyreader queue size: %d, " % train_pyreader.queue.size()
                         verbose += "learning rate: %f" % (
-                            outputs["learning_rate"]
-                            if warmup_steps > 0 else args.learning_rate)
+                            outputs["learning_rate"] if warmup_steps > 0 else args.learning_rate
+                        )
                         log.info(verbose)
 
                     current_example, current_epoch = reader.get_train_progress()
@@ -258,117 +248,89 @@ def main(args):
 
                     log.info(
                         "epoch: %d, progress: %d/%d, step: %d, ave loss: %f, "
-                        "ave acc: %f, speed: %f steps/s" %
-                        (current_epoch, current_example * dev_count,
-                         num_train_examples, steps, outputs["loss"],
-                         outputs["accuracy"], args.skip_steps / used_time))
-                    ce_info.append(
-                        [outputs["loss"], outputs["accuracy"], used_time])
+                        "ave acc: %f, speed: %f steps/s"
+                        % (
+                            current_epoch,
+                            current_example * dev_count,
+                            num_train_examples,
+                            steps,
+                            outputs["loss"],
+                            outputs["accuracy"],
+                            args.skip_steps / used_time,
+                        )
+                    )
+                    ce_info.append([outputs["loss"], outputs["accuracy"], used_time])
 
                     time_begin = time.time()
 
                 if steps % args.save_steps == 0:
-                    save_path = os.path.join(args.checkpoints,
-                                             "step_" + str(steps))
-                    fluid.io.save_persistables(exe, save_path,
-                                               fleet._origin_program)
+                    save_path = os.path.join(args.checkpoints, "step_" + str(steps))
+                    fluid.io.save_persistables(exe, save_path, fleet._origin_program)
 
                 if steps % args.validation_steps == 0:
                     # evaluate dev set
                     if args.do_val:
-                        evaluate_wrapper(args, reader, exe, test_prog,
-                                         test_pyreader, graph_vars,
-                                         current_epoch, steps)
+                        evaluate_wrapper(args, reader, exe, test_prog, test_pyreader, graph_vars, current_epoch, steps)
 
                     if args.do_test:
-                        predict_wrapper(args, reader, exe, test_prog,
-                                        test_pyreader, graph_vars,
-                                        current_epoch, steps)
+                        predict_wrapper(args, reader, exe, test_prog, test_pyreader, graph_vars, current_epoch, steps)
 
                 if last_epoch != current_epoch:
                     last_epoch = current_epoch
 
             except fluid.core.EOFException:
                 save_path = os.path.join(args.checkpoints, "step_" + str(steps))
-                fluid.io.save_persistables(exe, save_path,
-                                           fleet._origin_program)
+                fluid.io.save_persistables(exe, save_path, fleet._origin_program)
                 train_pyreader.reset()
                 break
 
     # final eval on dev set
     if args.do_val:
-        evaluate_wrapper(args, reader, exe, test_prog, test_pyreader,
-                         graph_vars, current_epoch, steps)
+        evaluate_wrapper(args, reader, exe, test_prog, test_pyreader, graph_vars, current_epoch, steps)
 
     # final eval on test set
     if args.do_test:
         predict_wrapper(args, reader, exe, test_prog, test_pyreader, graph_vars)
 
-    # final eval on dianostic, hack for glue-ax
+    # final eval on diagnostic, hack for glue-ax
     if args.diagnostic:
         test_pyreader.decorate_tensor_provider(
-            reader.data_generator(args.diagnostic,
-                                  batch_size=args.batch_size,
-                                  epoch=1,
-                                  dev_count=1,
-                                  shuffle=False))
+            reader.data_generator(args.diagnostic, batch_size=args.batch_size, epoch=1, dev_count=1, shuffle=False)
+        )
 
         log.info("Final diagnostic")
-        qids, preds, probs = predict(test_exe, test_prog, test_pyreader,
-                                     graph_vars)
-        assert len(qids) == len(preds), '{} v.s. {}'.format(
-            len(qids), len(preds))
-        with open(args.diagnostic_save, 'w') as f:
+        qids, preds, probs = predict(test_exe, test_prog, test_pyreader, graph_vars)
+        assert len(qids) == len(preds), "{} v.s. {}".format(len(qids), len(preds))
+        with open(args.diagnostic_save, "w") as f:
             for id, s, p in zip(qids, preds, probs):
-                f.write('{}\t{}\t{}\n'.format(id, s, p))
+                f.write("{}\t{}\t{}\n".format(id, s, p))
 
-        log.info("Done final diagnostic, saving to {}".format(
-            args.diagnostic_save))
+        log.info("Done final diagnostic, saving to {}".format(args.diagnostic_save))
 
 
-def evaluate_wrapper(args, reader, exe, test_prog, test_pyreader, graph_vars,
-                     epoch, steps):
+def evaluate_wrapper(args, reader, exe, test_prog, test_pyreader, graph_vars, epoch, steps):
     # evaluate dev set
-    for ds in args.dev_set.split(','):
+    for ds in args.dev_set.split(","):
         test_pyreader.decorate_tensor_provider(
-            reader.data_generator(ds,
-                                  batch_size=args.predict_batch_size,
-                                  epoch=1,
-                                  dev_count=1,
-                                  shuffle=False))
+            reader.data_generator(ds, batch_size=args.predict_batch_size, epoch=1, dev_count=1, shuffle=False)
+        )
         log.info("validation result of dataset {}:".format(ds))
-        evaluate_info = evaluate(exe,
-                                 test_prog,
-                                 test_pyreader,
-                                 graph_vars,
-                                 "dev",
-                                 metric=args.metric)
-        log.info(evaluate_info +
-                 ', file: {}, epoch: {}, steps: {}'.format(ds, epoch, steps))
+        evaluate_info = evaluate(exe, test_prog, test_pyreader, graph_vars, "dev", metric=args.metric)
+        log.info(evaluate_info + ", file: {}, epoch: {}, steps: {}".format(ds, epoch, steps))
 
 
-def predict_wrapper(args,
-                    reader,
-                    exe,
-                    test_prog,
-                    test_pyreader,
-                    graph_vars,
-                    epoch=None,
-                    steps=None):
-    test_sets = args.test_set.split(',')
-    save_dirs = args.test_save.split(',')
+def predict_wrapper(args, reader, exe, test_prog, test_pyreader, graph_vars, epoch=None, steps=None):
+    test_sets = args.test_set.split(",")
+    save_dirs = args.test_save.split(",")
     assert len(test_sets) == len(save_dirs)
 
     for test_f, save_f in zip(test_sets, save_dirs):
         test_pyreader.decorate_tensor_provider(
-            reader.data_generator(test_f,
-                                  batch_size=args.predict_batch_size,
-                                  epoch=1,
-                                  dev_count=1,
-                                  shuffle=False))
+            reader.data_generator(test_f, batch_size=args.predict_batch_size, epoch=1, dev_count=1, shuffle=False)
+        )
 
         if epoch is not None or steps is not None:
-            save_path = save_f + '.' + str(epoch) + '.' + str(steps)
+            save_path = save_f + "." + str(epoch) + "." + str(steps)
         else:
             save_path = save_f
         log.info("testing {}, save to {}".format(test_f, save_path))
@@ -378,14 +340,14 @@ def predict_wrapper(args,
         if not os.path.exists(save_dir):
             os.makedirs(save_dir)
         else:
-            log.warning('save dir exsits: %s, will skip saving' % save_dir)
+            log.warning("save dir exists: %s, will skip saving" % save_dir)
 
-        with open(save_path, 'w') as f:
+        with open(save_path, "w") as f:
             for p in probs:
-                f.write('{}\n'.format(p[1]))
+                f.write("{}\n".format(p[1]))
 
 
-if __name__ == '__main__':
+if __name__ == "__main__":
     prepare_logger(log)
     print_arguments(args)
     check_cuda(args.use_cuda)

applications/document_intelligence/doc_vqa/Rerank/src/utils/args.py

@@ -12,28 +12,22 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 """Arguments for configuration."""
-from __future__ import absolute_import
-from __future__ import division
-from __future__ import print_function
-from __future__ import unicode_literals
-from __future__ import absolute_import
+from __future__ import absolute_import, division, print_function, unicode_literals
 
-import six
+import logging
 import os
 import sys
-import argparse
-import logging
 
 import paddle.fluid as fluid
+import six
+
+from paddlenlp.trainer.argparser import strtobool
 
 log = logging.getLogger(__name__)
 
 
 def prepare_logger(logger, debug=False, save_to_file=None):
-    formatter = logging.Formatter(
-        fmt=
-        '[%(levelname)s] %(asctime)s [%(filename)12s:%(lineno)5d]:\t%(message)s'
-    )
+    formatter = logging.Formatter(fmt="[%(levelname)s] %(asctime)s [%(filename)12s:%(lineno)5d]:\t%(message)s")
     console_hdl = logging.StreamHandler()
     console_hdl.setFormatter(formatter)
     logger.addHandler(console_hdl)
@@ -45,47 +39,33 @@ def prepare_logger(logger, debug=False, save_to_file=None):
     logger.propagate = False
 
 
-def str2bool(v):
-    # because argparse does not support to parse "true, False" as python
-    # boolean directly
-    return v.lower() in ("true", "t", "1")
-
-
 class ArgumentGroup(object):
-
     def __init__(self, parser, title, des):
         self._group = parser.add_argument_group(title=title, description=des)
 
-    def add_arg(self,
-                name,
-                type,
-                default,
-                help,
-                positional_arg=False,
-                **kwargs):
+    def add_arg(self, name, type, default, help, positional_arg=False, **kwargs):
         prefix = "" if positional_arg else "--"
-        type = str2bool if type == bool else type
-        self._group.add_argument(prefix + name,
-                                 default=default,
-                                 type=type,
-                                 help=help + ' Default: %(default)s.',
-                                 **kwargs)
+        type = strtobool if type == bool else type
+        self._group.add_argument(
+            prefix + name, default=default, type=type, help=help + " Default: %(default)s.", **kwargs
+        )
 
 
 def print_arguments(args):
-    log.info('-----------  Configuration Arguments -----------')
+    log.info("-----------  Configuration Arguments -----------")
     for arg, value in sorted(six.iteritems(vars(args))):
-        log.info('%s: %s' % (arg, value))
-    log.info('------------------------------------------------')
+        log.info("%s: %s" % (arg, value))
+    log.info("------------------------------------------------")
 
 
-def check_cuda(use_cuda, err = \
-    "\nYou can not set use_cuda = True in the model because you are using paddlepaddle-cpu.\n \
-    Please: 1. Install paddlepaddle-gpu to run your models on GPU or 2. Set use_cuda = False to run models on CPU.\n"
-                                                                                                                     ):
+def check_cuda(
+    use_cuda,
+    err="\nYou can not set use_cuda = True in the model because you are using paddlepaddle-cpu.\n \
+    Please: 1. Install paddlepaddle-gpu to run your models on GPU or 2. Set use_cuda = False to run models on CPU.\n",
+):
     try:
-        if use_cuda == True and fluid.is_compiled_with_cuda() == False:
+        if use_cuda is True and fluid.is_compiled_with_cuda() is False:
             log.error(err)
             sys.exit(1)
-    except Exception as e:
+    except Exception:
         pass

applications/document_intelligence/doc_vqa/Rerank/src/utils/init.py

@@ -12,18 +12,8 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-from __future__ import absolute_import
-from __future__ import division
-from __future__ import print_function
-from __future__ import unicode_literals
-from __future__ import absolute_import
-
-import os
-import six
-import ast
-import copy
 import logging
-import numpy as np
+import os
 
 import paddle.fluid as fluid
 
@@ -31,43 +21,32 @@ log = logging.getLogger(__name__)
 
 
 def init_checkpoint(exe, init_checkpoint_path, main_program):
-    assert os.path.exists(
-        init_checkpoint_path), "[%s] cann't be found." % init_checkpoint_path
+    assert os.path.exists(init_checkpoint_path), "[%s] cann't be found." % init_checkpoint_path
 
     def existed_persitables(var):
         if not fluid.io.is_persistable(var):
             return False
         if not os.path.exists(os.path.join(init_checkpoint_path, var.name)):
-            print("Var not exists: [%s]\t%s" %
-                  (var.name, os.path.join(init_checkpoint_path, var.name)))
-        #else:
+            print("Var not exists: [%s]\t%s" % (var.name, os.path.join(init_checkpoint_path, var.name)))
+        # else:
         #    print ("Var exists: [%s]" % (var.name))
         return os.path.exists(os.path.join(init_checkpoint_path, var.name))
 
-    fluid.io.load_vars(exe,
-                       init_checkpoint_path,
-                       main_program=main_program,
-                       predicate=existed_persitables)
+    fluid.io.load_vars(exe, init_checkpoint_path, main_program=main_program, predicate=existed_persitables)
     log.info("Load model from {}".format(init_checkpoint_path))
 
 
 def init_pretraining_params(exe, pretraining_params_path, main_program):
-    assert os.path.exists(pretraining_params_path
-                          ), "[%s] cann't be found." % pretraining_params_path
+    assert os.path.exists(pretraining_params_path), "[%s] cann't be found." % pretraining_params_path
 
     def existed_params(var):
         if not isinstance(var, fluid.framework.Parameter):
             return False
         if not os.path.exists(os.path.join(pretraining_params_path, var.name)):
-            print("Var not exists: [%s]\t%s" %
-                  (var.name, os.path.join(pretraining_params_path, var.name)))
-        #else:
+            print("Var not exists: [%s]\t%s" % (var.name, os.path.join(pretraining_params_path, var.name)))
+        # else:
         #    print ("Var exists: [%s]" % (var.name))
         return os.path.exists(os.path.join(pretraining_params_path, var.name))
 
-    fluid.io.load_vars(exe,
-                       pretraining_params_path,
-                       main_program=main_program,
-                       predicate=existed_params)
-    log.info(
-        "Load pretraining parameters from {}.".format(pretraining_params_path))
+    fluid.io.load_vars(exe, pretraining_params_path, main_program=main_program, predicate=existed_params)
+    log.info("Load pretraining parameters from {}.".format(pretraining_params_path))

applications/document_intelligence/docprompt

@@ -1 +0,0 @@
-../../model_zoo/ernie-layout

applications/document_intelligence/docprompt/README.md

@@ -0,0 +1 @@
+[ERNIE-Layout](../../../model_zoo/ernie-layout)

applications/information_extraction/README.md

@@ -0,0 +1,171 @@
+简体中文 | [English](README_en.md)
+
+# 信息抽取应用
+
+**目录**
+- [1. 信息抽取应用简介](#1)
+- [2. 技术特色](#2)
+  - [2.1 信息抽取方案全覆盖](#21)
+  - [2.2 强大的训练基座](#22)
+  - [2.3 产业级全流程方案](#23)
+  - [2.4 效果展示](#24)
+- [3. 快速开始](#快速开始)
+  - [3.1 Taskflow开箱即用](#31)
+  - [3.2 文本信息抽取](#32)
+  - [3.3 文档信息抽取](#33)
+
+<a name="1"></a>
+
+## 1. 信息抽取应用简介
+
+信息抽取应用针对信息抽取一系列高频场景开源了产业级解决方案，**具备多领域、多任务、跨模态的能力**，打通**数据标注-模型训练-模型调优-预测部署全流程**，可快速实现信息抽取产品落地。
+
+信息抽取通俗地说就是从给定的文本/图片等输入数据中抽取出结构化信息的过程。在信息抽取的落地过程中通常面临领域多变、任务多样、数据稀缺等许多挑战。针对信息抽取领域的难点和痛点，PaddleNLP信息抽取应用**基于UIE统一建模的思想**，提供了信息抽取产业级应用方案，**除支持纯文本场景实体、关系、事件、观点等不同任务抽取外，还支持文档/图片/表格的端到端信息抽取**。该应用**不限定行业领域和抽取目标**，可实现从产品原型研发、业务POC阶段到业务落地、迭代阶段的无缝衔接，助力开发者实现特定领域抽取场景的快速适配与落地。
+
+**信息抽取应用亮点：**
+
+- **覆盖场景全面🎓：** 覆盖信息抽取各类主流任务，面向纯文本和文档场景，支持多语言，满足开发者多样信息抽取落地需求。
+- **效果领先🏃：** 以在纯文本、多模态上均有突出效果的UIE系列模型作为训练基座，提供多种尺寸的预训练模型满足不同需求，具有广泛成熟的实践应用性。
+- **简单易用⚡：** 通过Taskflow实现三行代码可实现无标注数据的情况下进行快速调用，一行命令即可开启信息抽取训练，轻松完成部署上线，降低信息抽取技术落地门槛。
+- **高效调优✊：** 开发者无需机器学习背景知识，即可轻松上手数据标注及模型训练流程。
+
+<a name="2"></a>
+
+## 2. 技术特色
+
+<a name="21"></a>
+
+### 2.1 信息抽取方案全覆盖
+
+多模型选择，满足精度、速度，适配不同信息抽取使用场景。
+
+|                           模型名称                           | 使用场景                                                   | 支持任务                                             |
+| :----------------------------------------------------------: | :--------------------------------------------------------- | :--------------------------------------------------- |
+| `uie-base`<br />`uie-medium`<br />`uie-mini`<br />`uie-micro`<br />`uie-nano` | 面向**纯文本**场景的**抽取式**模型，支持**中文**         | 具备实体、关系、事件、评论观点等通用信息抽取能力 |
+|                       `uie-base-en`                          | 面向**纯文本**场景的**抽取式**模型，支持**英文**         | 具备实体、关系、事件、评论观点等通用信息抽取能力 |
+|                     `uie-m-base`<br />`uie-m-large`          | 面向**纯文本**场景的**抽取式**模型，支持**中英**         | 具备实体、关系、事件、评论观点等通用信息抽取能力 |
+|                      <b>`uie-x-base`</b>                     | 面向**纯文本**和**文档**场景的**抽取式**模型，支持**中英** | 支持纯文本场景的全部功能，还支持文档/图片/表格的端到端信息抽取 |
+
+<a name="22"></a>
+
+### 2.2 强大的训练基座
+
+信息抽取应用使用ERNIE 3.0轻量级模型作为预训练模型，同时在大量信息抽取数据上进行了二次预训练，从而让模型适配固定prompt。
+
+- 中文文本数据集实验效果
+
+我们在互联网、医疗、金融三大垂类文本自建测试集上进行了实验：
+
+<table>
+<tr><th row_span='2'><th colspan='2'>金融<th colspan='2'>医疗<th colspan='2'>互联网
+<tr><td><th>0-shot<th>5-shot<th>0-shot<th>5-shot<th>0-shot<th>5-shot
+<tr><td>uie-base (12L768H)<td>46.43<td>70.92<td><b>71.83</b><td>85.72<td>78.33<td>81.86
+<tr><td>uie-medium (6L768H)<td>41.11<td>64.53<td>65.40<td>75.72<td>78.32<td>79.68
+<tr><td>uie-mini (6L384H)<td>37.04<td>64.65<td>60.50<td>78.36<td>72.09<td>76.38
+<tr><td>uie-micro (4L384H)<td>37.53<td>62.11<td>57.04<td>75.92<td>66.00<td>70.22
+<tr><td>uie-nano (4L312H)<td>38.94<td>66.83<td>48.29<td>76.74<td>62.86<td>72.35
+<tr><td>uie-m-large (24L1024H)<td><b>49.35</b><td><b>74.55</b><td>70.50<td><b>92.66</b><td>78.49<td><b>83.02</b>
+<tr><td>uie-m-base (12L768H)<td>38.46<td>74.31<td>63.37<td>87.32<td>76.27<td>80.13
+<tr><td>🧾 🎓<b>uie-x-base (12L768H)</b><td>48.84<td>73.87<td>65.60<td>88.81<td><b>79.36</b><td>81.65
+</table>
+
+0-shot表示无训练数据直接通过```paddlenlp.Taskflow```进行预测，5-shot表示每个类别包含5条标注数据进行模型微调。**实验表明UIE在垂类场景可以通过少量数据（few-shot）进一步提升效果**。
+
+- 多模态数据集实验效果
+
+我们在通用、金融、医疗三大场景自建多模态测试集上对UIE-X的零样本效果进行了实验：
+
+<table>
+<tr><th ><th>通用<th>金融<th colspan='2'>医疗
+<tr><td>🧾 🎓<b>uie-x-base (12L768H)</b><td>65.03<td>73.51<td>84.24
+</table>
+
+通用测试集包含了不同领域的复杂样本，抽取难度最大。
+
+<a name="23"></a>
+
+### 2.3 产业级全流程方案
+
+**调研阶段**
+
+- 该阶段目标需求开放且缺少数据积累。我们提供Taskflow三行代码极简调用的方式，无需标注数据即可在业务场景上快速验证效果。
+  - [文本抽取 Taskflow使用指南](./taskflow_text.md)
+  - [文档抽取 Taskflow使用指南](./taskflow_doc.md)
+
+**数据准备阶段**
+
+- 我们推荐在实际的业务场景中定制自己的信息抽取模型。我们提供了不同抽取场景的Label Studio标注解决方案，可基于该方案实现从数据标注到训练数据构造的无缝衔接，大大降低了数据标注、模型定制的时间成本。
+  - [文本抽取标注指南](./label_studio_text.md)
+  - [文档抽取标注指南](./label_studio_doc.md)。
+
+**模型微调及封闭域蒸馏**
+
+- 基于UIE优秀的小样本微调能力，实现低成本模型定制适配。同时提供封闭域蒸馏的加速方案，解决抽取速度慢的问题。
+  - [文本信息抽取全流程示例](./text/README.md)
+  - [文档信息抽取全流程示例](./document/README.md)
+
+**模型部署**
+
+- 提供HTTP部署方案，快速实现定制模型的部署上线。
+  - [文本抽取HTTP部署指南](./text/deploy/simple_serving/README.md)
+  - [文档抽取HTTP部署指南](./document/deploy/simple_serving/README.md)
+
+<a name="24"></a>
+
+### 2.4 效果展示
+
+- 🧾 通过[Huggingface网页](https://huggingface.co/spaces/PaddlePaddle/UIE-X)体验UIE-X功能：
+
+<div align="center">
+    <img src=https://user-images.githubusercontent.com/40840292/207856955-a01cd5dd-fd5c-48ae-b8fd-c69512a88845.png height=500 width=900 hspace='10'/>
+</div>
+
+- UIE-X端到端文档抽取产业应用示例
+
+  - 报关单
+
+    <div align="center">
+        <img src=https://user-images.githubusercontent.com/40840292/205879840-239ada90-1692-40e4-a17f-c5e963fdd204.png height=800 width=500 />
+    </div>
+
+  - Delivery Note（需微调）
+
+    <div align="center">
+        <img src=https://user-images.githubusercontent.com/40840292/205922422-f2615050-83cb-4bf5-8887-461f5633e85c.png height=250 width=700 />
+    </div>
+
+  - 增值税发票（需微调）
+
+    <div align="center">
+        <img src=https://user-images.githubusercontent.com/40840292/206084942-44ba477c-9244-4ce2-bbb5-ba430c9b926e.png height=550 width=700 />
+    </div>
+
+  - 表单（需微调）
+
+    <div align="center">
+        <img src=https://user-images.githubusercontent.com/40840292/207856330-7aa0d158-47e0-477f-a88f-e23a040504a3.png height=400 width=700 />
+    </div>
+
+<a name="3"></a>
+
+## 3. 快速开始
+
+<a name="31"></a>
+
+### 3.1 Taskflow开箱即用
+
+- 通过Taskflow实现开箱即用
+  👉 [文本抽取 Taskflow使用指南](./taskflow_text.md)
+  👉 [文档抽取 Taskflow使用指南](./taskflow_doc.md)
+
+<a name="32"></a>
+
+### 3.2 文本信息抽取
+
+- 快速开启文本信息抽取 👉 [文本信息抽取指南](./text/README.md)
+
+<a name="33"></a>
+
+### 3.3 文档信息抽取
+
+- 快速开启文档信息抽取 👉 [文档信息抽取指南](./document/README.md)

applications/information_extraction/README_en.md

@@ -0,0 +1,170 @@
+# Information Extraction Application
+
+**Table of contents**
+- [1. Introduction](#1)
+- [2. Features](#2)
+   - [2.1 Available Models](#21)
+   - [2.2 Performance](#22)
+   - [2.3 Full Development Lifecycle](#23)
+   - [2.4 Demo](#24)
+- [3. Quick Start](#3)
+   - [3.1 Taskflow](#31)
+   - [3.2 Text Information Extraction](#32)
+   - [3.3 Document Information Extraction](#33)
+
+<a name="1"></a>
+
+## 1. Introduction
+
+This Information Extraction (IE) guide introduces our open-source industry-grade solution that covers the most widely-used application scenarios of Information Extraction. It features **multi-domain, multi-task, and cross-modal capabilities** and goes through the full lifecycle of **data labeling, model training and model deployment**. We hope this guide can help you apply Information Extraction techniques in your own products or models.
+
+Information Extraction (IE) is the process of extracting structured information from given input data such as text, pictures or scanned document. While IE brings immense value, applying IE techniques is never easy with challenges such as domain adaptation, heterogeneous structures, lack of labeled data, etc. This PaddleNLP Information Extraction Guide builds on the foundation of our work in [Universal Information Extraction] (https://arxiv.org/abs/2203.12277) and provides an industrial-level solution that not only supports **extracting entities, relations, events and opinions from plain text**, but also supports **cross-modal extraction out of documents, tables and pictures.** Our method features a flexible prompt, which allows you to specify extraction targets with simple natural language. We also provide a few different domain-adapated models specialized for different industry sectors.
+
+**Highlights:**
+
+- **Comprehensive Coverage🎓:** Covers various mainstream tasks of information extraction for plain text and document scenarios, supports multiple languages
+- **State-of-the-Art Performance🏃:** Strong performance from the UIE model series models in plain text and multimodal datasets. We also provide pretrained models of various sizes to meet different needs
+- **Easy to use⚡:** three lines of code to use our `Taskflow` for out-of-box Information Extraction capabilities. One line of command to model training and model deployment
+- **Efficient Tuning✊:** Developers can easily get started with the data labeling and model training process without a background in Machine Learning.
+
+<a name="2"></a>
+
+## 2. Features
+
+<a name="21"></a>
+
+### 2.1 Available Models
+
+Multiple model selection, satisfying accuracy and speed, and adapting to different information extraction scenarios.
+
+|                           Model Name                           | Usage Scenarios                                                 | Supporting Tasks                                            |
+| :----------------------------------------------------------: | :--------------------------------------------------------- | :--------------------------------------------------- |
+| `uie-base`<br />`uie-medium`<br />`uie-mini`<br />`uie-micro`<br />`uie-nano` | For **plain text** The **extractive** model of the scene supports **Chinese**         | Supports entity, relation, event, opinion extraction |
+|                       `uie-base-en`                          | An **extractive** model for **plain text** scenarios, supports **English**         | Supports entity, relation, event, opinion extraction |
+|                     `uie-m-base`<br />`uie-m-large`          | An **extractive** model for **plain text** scenarios, supporting **Chinese and English**     | Supports entity, relation, event, opinion extraction |
+|                      <b>`uie-x-base`</b>                     | An **extractive** model for **plain text** and **document** scenarios, supports **Chinese and English** | Supports entity, relation, event, opinion extraction on both plain text and documents/pictures/tables |
+
+
+<a name="22"></a>
+
+### 2.2 Performance
+
+The UIE model series uses the ERNIE 3.0 lightweight models as the pre-trained language models and was finetuned on a large amount of information extraction data so that the model can be adapted to a fixed prompt.
+
+- Experimental results on Chinese dataset
+
+We conducted experiments on the in-house test sets of the three different domains of Internet, medical care, and finance:
+
+<table>
+<tr><th row_span='2'><th colspan='2'>finance<th colspan='2'>healthcare<th colspan='2'>internet
+<tr><td><th>0-shot<th>5-shot<th>0-shot<th>5-shot<th>0-shot<th>5-shot
+<tr><td>uie-base (12L768H)<td>46.43<td>70.92<td><b>71.83</b><td>85.72<td>78.33<td>81.86
+<tr><td>uie-medium (6L768H)<td>41.11<td>64.53<td>65.40<td>75.72<td>78.32<td>79.68
+<tr><td>uie-mini (6L384H)<td>37.04<td>64.65<td>60.50<td>78.36<td>72.09<td>76.38
+<tr><td>uie-micro (4L384H)<td>37.53<td>62.11<td>57.04<td>75.92<td>66.00<td>70.22
+<tr><td>uie-nano (4L312H)<td>38.94<td>66.83<td>48.29<td>76.74<td>62.86<td>72.35
+<tr><td>uie-m-large (24L1024H)<td><b>49.35</b><td><b>74.55</b><td>70.50<td><b>92.66</b ><td>78.49<td><b>83.02</b>
+<tr><td>uie-m-base (12L768H)<td>38.46<td>74.31<td>63.37<td>87.32<td>76.27<td>80.13
+<tr><td>🧾🎓<b>uie-x-base (12L768H)</b><td>48.84<td>73.87<td>65.60<td>88.81<td><b>79.36</b> <td>81.65
+</table>
+
+0-shot means that no training data is directly used for prediction through ```paddlenlp.Taskflow```, and 5-shot means that each category contains 5 pieces of labeled data for model fine-tuning. **Experiments show that UIE can further improve the performance with a small amount of data (few-shot)**.
+
+- Experimental results on multimodal datasets
+
+We experimented on the zero-shot performance of UIE-X on the in-house multi-modal test sets in three different domains of general, financial, and medical:
+
+<table>
+<tr><th ><th>General <th>Financial<th colspan='2'>Medical
+<tr><td>🧾🎓<b>uie-x-base (12L768H)</b><td>65.03<td>73.51<td>84.24
+</table>
+
+The general test set contains complex samples from different fields and is the most difficult task.
+
+<a name="23"></a>
+
+### 2.3 Full Development Lifecycle
+
+**Research stage**
+
+- At this stage, the target requirements are open and there is no labeled data. We provide a simple way of using Taskflow out-of-the-box with three lines of code, which allows you to build POC without any labeled data.
+   - [Text Extraction Taskflow User Guide](./taskflow_text_en.md)
+   - [Document Extraction Taskflow User Guide](./taskflow_doc_en.md)
+
+**Data preparation stage**
+
+- We recommend finetuning your own information extraction model for your use case. We provide Label Studio labeling solutions for different extraction scenarios. Based on this solution, the seamless connection from data labeling to training data construction can be realized, which greatly reduces the time cost of data labeling and model customization.
+   - [Text Extraction Labeling Guide](./label_studio_text_en.md)
+   - [Document Extraction and Labeling Guide](./label_studio_doc_en.md).
+
+**Model fine-tuning and closed domain distillation**
+
+- Based on UIE's few-shot capabilities, it realizes low-cost model customization and adaptation. At the same time, it provides an acceleration solution for closed domain distillation to solve the problem of slow extraction speed.
+   - [Example of the whole process of text information extraction](./text/README_en.md)
+   - [Example of document information extraction process](./document/README_en.md)
+
+**Model Deployment**
+
+- Provide an HTTP deployment solution to quickly implement the deployment and launch of customized models.
+   - [Text Extract HTTP Deployment Guide](./text/deploy/simple_serving/README_en.md)
+   - [Document Extract HTTP Deployment Guide](./document/deploy/simple_serving/README_en.md)
+
+<a name="24"></a>
+
+### 2.4 Demo
+
+- 🧾Try our UIE-X demo on [🤗 HuggingFace Space](https://huggingface.co/spaces/PaddlePaddle/UIE-X):
+
+<div align="center">
+     <img src=https://user-images.githubusercontent.com/40840292/207856955-a01cd5dd-fd5c-48ae-b8fd-c69512a88845.png height=500 width=900 hspace='10'/>
+</div>
+
+- UIE-X end-to-end document extraction industry application example
+
+   - Customs declaration
+
+     <div align="center">
+         <img src=https://user-images.githubusercontent.com/40840292/205879840-239ada90-1692-40e4-a17f-c5e963fdd204.png height=800 width=500 />
+     </div>
+
+   - Delivery Note (Need fine-tuning)
+
+     <div align="center">
+         <img src=https://user-images.githubusercontent.com/40840292/205922422-f2615050-83cb-4bf5-8887-461f5633e85c.png height=250 width=700 />
+     </div>
+
+   - VAT invoice (need fine-tuning)
+
+     <div align="center">
+         <img src=https://user-images.githubusercontent.com/40840292/206084942-44ba477c-9244-4ce2-bbb5-ba430c9b926e.png height=550 width=700 />
+     </div>
+
+  - Form (need fine-tuning)
+
+    <div align="center">
+        <img src=https://user-images.githubusercontent.com/40840292/207856330-7aa0d158-47e0-477f-a88f-e23a040504a3.png height=400 width=700 />
+    </div>
+
+<a name="3"></a>
+
+## 3. Quick Start
+
+<a name="31"></a>
+
+### 3.1 Taskflow
+
+- Out of the box with Taskflow
+   👉 [Text Extraction Taskflow User Guide](./taskflow_text_en.md)
+   👉 [Document Extraction Taskflow User Guide](./taskflow_doc_en.md)
+
+<a name="32"></a>
+
+### 3.2 Text Information Extraction
+
+- Quickly start text information extraction 👉 [Text Information Extraction Guide](./text/README_en.md)
+
+<a name="33"></a>
+
+### 3.3 Document Information Extraction
+
+- Quickly open document information extraction 👉 [Document Information Extraction Guide](./document/README_en.md)

applications/information_extraction/document/README.md

@@ -0,0 +1,301 @@
+简体中文 | [English](README_en.md)
+
+# 文档信息抽取
+
+**目录**
+- [1. 文档信息抽取应用](#1)
+- [2. 快速开始](#2)
+  - [2.1 代码结构](#代码结构)
+  - [2.2 数据标注](#数据标注)
+  - [2.3 模型微调](#模型微调)
+  - [2.4 模型评估](#模型评估)
+  - [2.5 定制模型一键预测](#定制模型一键预测)
+  - [2.6 实验指标](#实验指标)
+
+<a name="1"></a>
+
+## 1. 文档信息抽取应用
+
+本项目提供基于UIE微调的文档抽取端到端应用方案，打通**数据标注-模型训练-模型调优-预测部署全流程**，可快速实现文档信息抽取产品落地。
+
+信息抽取通俗地说就是从给定的文本/图片等输入数据中抽取出结构化信息的过程。在信息抽取的落地过程中通常面临领域多变、任务多样、数据稀缺等许多挑战。针对信息抽取领域的难点和痛点，PaddleNLP信息抽取应用UIE统一建模的思想，提供了文档信息抽取产业级应用方案，支持**文档/图片/表格和纯文本场景下实体、关系、事件、观点等不同任务信息抽取**。该应用**不限定行业领域和抽取目标**，可实现从产品原型研发、业务POC阶段到业务落地、迭代阶段的无缝衔接，助力开发者实现特定领域抽取场景的快速适配与落地。
+
+**文档信息抽取应用亮点：**
+
+- **覆盖场景全面🎓：** 覆盖文档信息抽取各类主流任务，支持多语言，满足开发者多样信息抽取落地需求。
+- **效果领先🏃：** 以在多模态信息抽取上有突出效果的模型UIE-X作为训练基座，具有广泛成熟的实践应用性。
+- **简单易用⚡：** 通过Taskflow实现三行代码可实现无标注数据的情况下进行快速调用，一行命令即可开启信息抽取训练，轻松完成部署上线，降低信息抽取技术落地门槛。
+- **高效调优✊：** 开发者无需机器学习背景知识，即可轻松上手数据标注及模型训练流程。
+
+<a name="2"></a>
+
+## 2. 快速开始
+
+对于简单的抽取目标可以直接使用```paddlenlp.Taskflow```实现零样本（zero-shot）抽取，对于细分场景我们推荐使用定制功能（标注少量数据进行模型微调）以进一步提升效果。
+
+<a name="代码结构"></a>
+
+### 2.1 代码结构
+
+```shell
+.
+├── deploy            # 部署目录
+│ └── simple_serving  # 基于PaddleNLP SimpleServing 服务化部署
+├── utils.py          # 数据处理工具
+├── finetune.py       # 模型微调、压缩脚本
+├── evaluate.py       # 模型评估脚本
+└── README.md
+```
+
+<a name="数据标注"></a>
+
+### 2.2 数据标注
+我们推荐使用 [Label Studio](https://labelstud.io/) 进行文档信息抽取数据标注，本项目打通了从数据标注到训练的通道，也即Label Studio导出数据可以通过 [label_studio.py](../label_studio.py) 脚本轻松将数据转换为输入模型时需要的形式，实现无缝衔接。标注方法的详细介绍请参考 [Label Studio数据标注指南](../label_studio_doc.md)。
+
+这里我们提供预先标注好的`增值税发票数据集`的文件，可以运行下面的命令行下载数据集，我们将展示如何使用数据转化脚本生成训练/验证/测试集文件，并使用UIE-X模型进行微调。
+
+下载增值税发票数据集：
+```shell
+wget https://paddlenlp.bj.bcebos.com/datasets/tax.tar.gz
+tar -zxvf tax.tar.gz
+mv tax data
+rm tax.tar.gz
+```
+
+生成训练/验证集文件：
+```shell
+python ../label_studio.py \
+    --label_studio_file ./data/label_studio.json \
+    --save_dir ./data \
+    --splits 0.8 0.2 0 \
+    --task_type ext
+```
+
+生成训练/验证集文件，可以使用PP-Structure的布局分析优化OCR结果的排序：
+```shell
+python ../label_studio.py \
+    --label_studio_file ./data/label_studio.json \
+    --save_dir ./data \
+    --splits 0.8 0.2 0\
+    --task_type ext \
+    --layout_analysis True
+```
+
+更多不同类型任务（含实体抽取、关系抽取、文档分类等）的标注规则及参数说明，请参考[Label Studio数据标注指南](../label_studio_doc.md)。
+
+<a name="模型微调"></a>
+
+### 2.3 模型微调
+
+推荐使用 [Trainer API ](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/docs/trainer.md) 对模型进行微调。只需输入模型、数据集等就可以使用 Trainer API 高效快速地进行预训练、微调和模型压缩等任务，可以一键启动多卡训练、混合精度训练、梯度累积、断点重启、日志显示等功能，Trainer API 还针对训练过程的通用训练配置做了封装，比如：优化器、学习率调度等。
+
+使用下面的命令，使用 `uie-x-base` 作为预训练模型进行模型微调，将微调后的模型保存至`./checkpoint/model_best`：
+
+单卡启动：
+
+```shell
+python finetune.py  \
+    --device gpu \
+    --logging_steps 5 \
+    --save_steps 25 \
+    --eval_steps 25 \
+    --seed 42 \
+    --model_name_or_path uie-x-base \
+    --output_dir ./checkpoint/model_best \
+    --train_path data/train.txt \
+    --dev_path data/dev.txt  \
+    --max_seq_len 512  \
+    --per_device_train_batch_size  8 \
+    --per_device_eval_batch_size 8 \
+    --num_train_epochs 10 \
+    --learning_rate 1e-5 \
+    --do_train \
+    --do_eval \
+    --do_export \
+    --export_model_dir ./checkpoint/model_best \
+    --overwrite_output_dir \
+    --disable_tqdm True \
+    --metric_for_best_model eval_f1 \
+    --load_best_model_at_end  True \
+    --save_total_limit 1
+```
+
+如果在GPU环境中使用，可以指定gpus参数进行多卡训练：
+
+```shell
+python -u -m paddle.distributed.launch --gpus "0" finetune.py \
+    --device gpu \
+    --logging_steps 5 \
+    --save_steps 25 \
+    --eval_steps 25 \
+    --seed 42 \
+    --model_name_or_path uie-x-base \
+    --output_dir ./checkpoint/model_best \
+    --train_path data/train.txt \
+    --dev_path data/dev.txt  \
+    --max_seq_len 512  \
+    --per_device_train_batch_size  8 \
+    --per_device_eval_batch_size 8 \
+    --num_train_epochs 10 \
+    --learning_rate 1e-5 \
+    --do_train \
+    --do_eval \
+    --do_export \
+    --export_model_dir ./checkpoint/model_best \
+    --overwrite_output_dir \
+    --disable_tqdm True \
+    --metric_for_best_model eval_f1 \
+    --load_best_model_at_end  True \
+    --save_total_limit 1
+```
+
+该示例代码中由于设置了参数 `--do_eval`，因此在训练完会自动进行评估。
+
+可配置参数说明：
+* `device`: 训练设备，可选择 'cpu'、'gpu'、'npu' 其中的一种；默认为 GPU 训练。
+* `logging_steps`: 训练过程中日志打印的间隔 steps 数，默认10。
+* `save_steps`: 训练过程中保存模型 checkpoint 的间隔 steps 数，默认100。
+* `eval_steps`: 训练过程中保存模型 checkpoint 的间隔 steps 数，默认100。
+* `seed`：全局随机种子，默认为 42。
+* `model_name_or_path`：进行 few shot 训练使用的预训练模型。默认为 "uie-x-base"。
+* `output_dir`：必须，模型训练或压缩后保存的模型目录；默认为 `None` 。
+* `train_path`：训练集路径；默认为 `None` 。
+* `dev_path`：开发集路径；默认为 `None` 。
+* `max_seq_len`：文本最大切分长度，输入超过最大长度时会对输入文本进行自动切分，默认为512。
+* `per_device_train_batch_size`:用于训练的每个 GPU 核心/NPU 核心/CPU 的batch大小，默认为8。
+* `per_device_eval_batch_size`:用于评估的每个 GPU 核心/NPU 核心/CPU 的batch大小，默认为8。
+* `num_train_epochs`: 训练轮次，使用早停法时可以选择 100；默认为10。
+* `learning_rate`：训练最大学习率，UIE-X 推荐设置为 1e-5；默认值为3e-5。
+* `label_names`：训练数据标签label的名称，UIE-X 设置为'start_positions' 'end_positions'；默认值为None。
+* `do_train`:是否进行微调训练，设置该参数表示进行微调训练，默认不设置。
+* `do_eval`:是否进行评估，设置该参数表示进行评估，默认不设置。
+* `do_export`:是否进行导出，设置该参数表示进行静态图导出，默认不设置。
+* `export_model_dir`:静态图导出地址，默认为None。
+* `overwrite_output_dir`： 如果 `True`，覆盖输出目录的内容。如果 `output_dir` 指向检查点目录，则使用它继续训练。
+* `disable_tqdm`： 是否使用tqdm进度条。
+* `metric_for_best_model`：最优模型指标,UIE-X 推荐设置为 `eval_f1`，默认为None。
+* `load_best_model_at_end`：训练结束后是否加载最优模型，通常与`metric_for_best_model`配合使用，默认为False。
+* `save_total_limit`：如果设置次参数，将限制checkpoint的总数。删除旧的checkpoints `输出目录`，默认为None。
+
+<a name="模型评估"></a>
+
+### 2.4 模型评估
+
+```shell
+python evaluate.py \
+    --device "gpu" \
+    --model_path ./checkpoint/model_best \
+    --test_path ./data/dev.txt \
+    --output_dir ./checkpoint/model_best \
+    --label_names 'start_positions' 'end_positions'\
+    --max_seq_len 512 \
+    --per_device_eval_batch_size 16
+```
+评估方式说明：采用单阶段评价的方式，即关系抽取、事件抽取等需要分阶段预测的任务对每一阶段的预测结果进行分别评价。验证/测试集默认会利用同一层级的所有标签来构造出全部负例。
+
+可开启`debug`模式对每个正例类别分别进行评估，该模式仅用于模型调试：
+
+```shell
+python evaluate.py \
+    --device "gpu" \
+    --model_path ./checkpoint/model_best \
+    --test_path ./data/dev.txt \
+    --output_dir ./checkpoint/model_best \
+    --label_names 'start_positions' 'end_positions'\
+    --max_seq_len 512 \
+    --per_device_eval_batch_size 16 \
+    --debug True
+```
+
+输出结果：
+```text
+[2022-11-14 09:41:18,424] [    INFO] - ***** Running Evaluation *****
+[2022-11-14 09:41:18,424] [    INFO] -   Num examples = 160
+[2022-11-14 09:41:18,424] [    INFO] -   Pre device batch size = 4
+[2022-11-14 09:41:18,424] [    INFO] -   Total Batch size = 4
+[2022-11-14 09:41:18,424] [    INFO] -   Total prediction steps = 40
+[2022-11-14 09:41:26,451] [    INFO] - -----Evaluate model-------
+[2022-11-14 09:41:26,451] [    INFO] - Class Name: ALL CLASSES
+[2022-11-14 09:41:26,451] [    INFO] - Evaluation Precision: 0.94521 | Recall: 0.88462 | F1: 0.91391
+[2022-11-14 09:41:26,451] [    INFO] - -----------------------------
+[2022-11-14 09:41:26,452] [    INFO] - ***** Running Evaluation *****
+[2022-11-14 09:41:26,452] [    INFO] -   Num examples = 8
+[2022-11-14 09:41:26,452] [    INFO] -   Pre device batch size = 4
+[2022-11-14 09:41:26,452] [    INFO] -   Total Batch size = 4
+[2022-11-14 09:41:26,452] [    INFO] -   Total prediction steps = 2
+[2022-11-14 09:41:26,692] [    INFO] - Class Name: 开票日期
+[2022-11-14 09:41:26,692] [    INFO] - Evaluation Precision: 1.00000 | Recall: 1.00000 | F1: 1.00000
+[2022-11-14 09:41:26,692] [    INFO] - -----------------------------
+[2022-11-14 09:41:26,693] [    INFO] - ***** Running Evaluation *****
+[2022-11-14 09:41:26,693] [    INFO] -   Num examples = 8
+[2022-11-14 09:41:26,693] [    INFO] -   Pre device batch size = 4
+[2022-11-14 09:41:26,693] [    INFO] -   Total Batch size = 4
+[2022-11-14 09:41:26,693] [    INFO] -   Total prediction steps = 2
+[2022-11-14 09:41:26,952] [    INFO] - Class Name: 名称
+[2022-11-14 09:41:26,952] [    INFO] - Evaluation Precision: 0.87500 | Recall: 0.87500 | F1: 0.87500
+[2022-11-14 09:41:26,952] [    INFO] - -----------------------------
+...
+```
+
+可配置参数：
+* `device`: 评估设备，可选择 'cpu'、'gpu'、'npu' 其中的一种；默认为 GPU 评估。
+* `model_path`: 进行评估的模型文件夹路径，路径下需包含模型权重文件`model_state.pdparams`及配置文件`model_config.json`。
+* `test_path`: 进行评估的测试集文件。
+* `label_names`：训练数据标签label的名称，UIE-X 设置为'start_positions' 'end_positions'；默认值为None。
+* `batch_size`: 批处理大小，请结合机器情况进行调整，默认为16。
+* `max_seq_len`: 文本最大切分长度，输入超过最大长度时会对输入文本进行自动切分，默认为512。
+* `per_device_eval_batch_size`:用于评估的每个 GPU 核心/NPU 核心/CPU 的batch大小，默认为8。
+* `debug`: 是否开启debug模式对每个正例类别分别进行评估，该模式仅用于模型调试，默认关闭。
+* `schema_lang`: 选择schema的语言，可选有`ch`和`en`。默认为`ch`，英文数据集请选择`en`。
+
+<a name="定制模型一键预测"></a>
+
+### 2.5 定制模型一键预测
+
+`paddlenlp.Taskflow`装载定制模型，通过`task_path`指定模型权重文件的路径，路径下需要包含训练好的模型权重文件`model_state.pdparams`。
+
+```python
+from pprint import pprint
+from paddlenlp import Taskflow
+from paddlenlp.utils.doc_parser import DocParser
+
+schema = ['开票日期', '名称', '纳税人识别号', '开户行及账号', '金额', '价税合计', 'No', '税率', '地址、电话', '税额']
+my_ie = Taskflow("information_extraction", model="uie-x-base", schema=schema, task_path='./checkpoint/model_best', precision='fp16')
+```
+
+我们可以根据设置的`schema`，对指定的`doc_path`文档进行信息抽取并进行可视化：
+
+```python
+doc_path = "./data/images/b199.jpg"
+results = my_ie({"doc": doc_path})
+pprint(results)
+
+# 结果可视化
+DocParser.write_image_with_results(
+    doc_path,
+    result=results[0],
+    save_path="./image_show.png")
+```
+
+<div align="center">
+    <img src=https://user-images.githubusercontent.com/40840292/206084942-44ba477c-9244-4ce2-bbb5-ba430c9b926e.png height=550 width=700 />
+</div>
+
+<a name="实验指标"></a>
+
+### 2.6 实验指标
+
+我们在自标注的增值税数据集上进行实验：
+
+
+  |  |  Precision  | Recall | F1 Score |
+  | :---: | :--------: | :--------: | :--------: |
+  | 0-shot| 0.44898 | 0.56410  | 0.50000 |
+  | 5-shot| 0.9000 | 0.9231 | 0.9114 |
+  | 10-shot| 0.9125 | 0.93590 |  0.9241 |
+  | 20-shot| 0.9737 | 0.9487 | 0.9610 |
+  | 30-shot|  0.9744  | 0.9744  | 0.9744 |
+  | 30-shot+PP-Structure| 1.0  | 0.9625 |  0.9809 |
+
+
+n-shot表示训练集包含n张标注图片数据进行模型微调，实验表明UIE-X可以通过少量数据（few-shot）和PP-Structure的布局分析进一步提升结果。

applications/information_extraction/document/README_en.md

@@ -0,0 +1,297 @@
+# document information extraction
+
+**Table of contents**
+- [1. Introduction](#1)
+- [2. Quick Start](#2)
+   - [2.1 Code Structure](#21)
+   - [2.2 Data Annotation](#22)
+   - [2.3 Finetuning](#23)
+   - [2.4 Evaluation](#24)
+   - [2.5 Inference](#25)
+   - [2.6 Experiments](#26)
+
+<a name="1"></a>
+
+## 1. Introduction
+
+This Information Extraction (IE) guide introduces our open-source industry-grade solution that covers the most widely-used application scenarios of Information Extraction. It features **multi-domain, multi-task, and cross-modal capabilities** and goes through the full lifecycle of **data labeling, model training and model deployment**. We hope this guide can help you apply Information Extraction techniques in your own products or models.
+
+Information Extraction (IE) is the process of extracting structured information from given input data such as text, pictures or scanned document. While IE brings immense value, applying IE techniques is never easy with challenges such as domain adaptation, heterogeneous structures, lack of labeled data, etc. This PaddleNLP Information Extraction Guide builds on the foundation of our work in [Universal Information Extraction](https://arxiv.org/abs/2203.12277) and provides an industrial-level solution that not only supports **extracting entities, relations, events and opinions from plain text**, but also supports **cross-modal extraction out of documents, tables and pictures.** Our method features a flexible prompt, which allows you to specify extraction targets with simple natural language. We also provide a few different domain-adapted models specialized for different industry sectors.
+
+**Highlights:**
+
+- **Comprehensive Coverage🎓:** Covers various mainstream tasks of information extraction for plain text and document scenarios, supports multiple languages
+- **State-of-the-Art Performance🏃:** Strong performance from the UIE model series models in plain text and multimodal datasets. We also provide pretrained models of various sizes to meet different needs
+- **Easy to use⚡:** three lines of code to use our `Taskflow` for out-of-box Information Extraction capabilities. One line of command to model training and model deployment
+- **Efficient Tuning✊:** Developers can easily get started with the data labeling and model training process without a background in Machine Learning.
+
+<a name="2"></a>
+
+## 2. Quick Start
+
+For quick start, you can directly use ```paddlenlp.Taskflow``` out-of-the-box, leveraging the zero-shot capability. For production use cases, we recommend labeling a small amount of data for model fine-tuning to further improve the performance.
+
+<a name="21"></a>
+
+### 2.1 Code Structure
+
+```shell
+.
+├── utils.py # data processing tools
+├── finetune.py # model fine-tuning, compression script
+├── evaluate.py # model evaluation script
+└── README.md
+```
+
+<a name="22"></a>
+
+### 2.2 Data Annotation
+
+We recommend using [Label Studio](https://labelstud.io/) for data labeling. We provide an end-to-end pipeline for the labeling -> training process. You can export the labeled data in Label Studio through [label_studio.py](../label_studio.py) script to export and convert the data into the required input form for the model. For a detailed introduction to labeling methods, please refer to [Label Studio Data Labeling Guide](../label_studio_doc_en.md).
+
+Here we provide the pre-labeled example dataset `VAT invoice dataset`, which you can download by running the following command. We will demonstrate how to use the data conversion script to generate training/validation/test set files for finetuning.
+
+Download the VAT invoice dataset:
+```shell
+wget https://paddlenlp.bj.bcebos.com/datasets/tax.tar.gz
+tar -zxvf tax.tar.gz
+mv tax data
+rm tax.tar.gz
+```
+
+Generate training/validation data files:
+
+```shell
+python ../label_studio.py \
+     --label_studio_file ./data/label_studio.json \
+     --save_dir ./data \
+     --splits 0.8 0.2 0 \
+     --task_type ext
+```
+
+Generate training/validation set files, you can use PP-Structure's layout analysis to optimize the sorting of OCR results:
+
+```shell
+python ../label_studio.py \
+     --label_studio_file ./data/label_studio.json \
+     --save_dir ./data \
+     --splits 0.8 0.2 0\
+     --task_type ext\
+     --layout_analysis True
+```
+
+For more labeling rules and parameter descriptions for different types of tasks (including entity extraction, relationship extraction, document classification, etc.), please refer to [Label Studio Data Labeling Guide](../label_studio_doc_en.md).
+
+<a name="23"></a>
+
+### 2.3 Finetuning
+
+Use the following command to fine-tune the model using `uie-x-base` as the pre-trained model, and save the fine-tuned model to `./checkpoint/model_best`:
+
+Single GPU:
+
+```shell
+python finetune.py\
+     --device gpu \
+     --logging_steps 5 \
+     --save_steps 25 \
+     --eval_steps 25 \
+     --seed 42 \
+     --model_name_or_path uie-x-base \
+     --output_dir ./checkpoint/model_best\
+     --train_path data/train.txt \
+     --dev_path data/dev.txt \
+     --max_seq_len 512 \
+     --per_device_train_batch_size 8 \
+     --per_device_eval_batch_size 8 \
+     --num_train_epochs 10 \
+     --learning_rate 1e-5 \
+     --do_train \
+     --do_eval \
+     --do_export \
+     --export_model_dir ./checkpoint/model_best\
+     --overwrite_output_dir \
+     --disable_tqdm True \
+     --metric_for_best_model eval_f1 \
+     --load_best_model_at_end True \
+     --save_total_limit 1
+```
+
+Multiple GPUs:
+
+```shell
+python -u -m paddle.distributed.launch --gpus "0" finetune.py \
+     --device gpu \
+     --logging_steps 5 \
+     --save_steps 25 \
+     --eval_steps 25 \
+     --seed 42 \
+     --model_name_or_path uie-x-base \
+     --output_dir ./checkpoint/model_best\
+     --train_path data/train.txt \
+     --dev_path data/dev.txt \
+     --max_seq_len 512 \
+     --per_device_train_batch_size 8 \
+     --per_device_eval_batch_size 8 \
+     --num_train_epochs 10 \
+     --learning_rate 1e-5 \
+     --do_train \
+     --do_eval \
+     --do_export \
+     --export_model_dir ./checkpoint/model_best\
+     --overwrite_output_dir \
+     --disable_tqdm True \
+     --metric_for_best_model eval_f1 \
+     --load_best_model_at_end True \
+     --save_total_limit 1
+```
+
+Since the parameter `--do_eval` is set in the sample code, it will be automatically evaluated after training.
+
+Parameters:
+
+* `device`: Training device, one of 'cpu', 'gpu' and 'npu' can be selected; the default is GPU training.
+* `logging_steps`: The interval steps of log printing during training, the default is 10.
+* `save_steps`: The number of interval steps to save the model checkpoint during training, the default is 100.
+* `eval_steps`: The number of interval steps to save the model checkpoint during training, the default is 100.
+* `seed`: global random seed, default is 42.
+* `model_name_or_path`: The pre-trained model used for few shot training. Defaults to "uie-x-base".
+* `output_dir`: required, the model directory saved after model training or compression; the default is `None`.
+* `train_path`: training set path; defaults to `None`.
+* `dev_path`: Development set path; defaults to `None`.
+* `max_seq_len`: The maximum segmentation length of the text. When the input exceeds the maximum length, the input text will be automatically segmented. The default is 512.
+* `per_device_train_batch_size`: The batch size of each GPU core/NPU core/CPU used for training, the default is 8.
+* `per_device_eval_batch_size`: Batch size per GPU core/NPU core/CPU for evaluation, default is 8.
+* `num_train_epochs`: Training rounds, 100 can be selected when using early stopping method; the default is 10.
+* `learning_rate`: The maximum learning rate for training, UIE-X recommends setting it to 1e-5; the default value is 3e-5.
+* `label_names`: the name of the training data label label, UIE-X is set to 'start_positions' 'end_positions'; the default value is None.
+* `do_train`: Whether to perform fine-tuning training, setting this parameter means to perform fine-tuning training, and it is not set by default.
+* `do_eval`: Whether to evaluate, setting this parameter means to evaluate, the default is not set.
+* `do_export`: Whether to export, setting this parameter means to export static images, and it is not set by default.
+* `export_model_dir`: Static map export address, the default is None.
+* `overwrite_output_dir`: If `True`, overwrite the contents of the output directory. If `output_dir` points to a checkpoint directory, use it to continue training.
+* `disable_tqdm`: Whether to use tqdm progress bar.
+* `metric_for_best_model`: Optimal model metric, UIE-X recommends setting it to `eval_f1`, the default is None.
+* `load_best_model_at_end`: Whether to load the best model after training, usually used in conjunction with `metric_for_best_model`, the default is False.
+* `save_total_limit`: If this parameter is set, the total number of checkpoints will be limited. Remove old checkpoints `output directory`, defaults to None.
+
+<a name="24"></a>
+
+### 2.4 Evaluation
+
+```shell
+python evaluate.py \
+    --device "gpu" \
+    --model_path ./checkpoint/model_best \
+    --test_path ./data/dev.txt \
+    --output_dir ./checkpoint/model_best \
+    --label_names 'start_positions' 'end_positions'\
+    --max_seq_len 512 \
+    --per_device_eval_batch_size 16
+```
+We adopt the single-stage method for evaluation, which means tasks that require multiple stages (e.g. relation extraction, event extraction) are evaluated separately for each stage. By default, the validation/test set uses all labels at the same level to construct the negative examples.
+The `debug` mode can be turned on to evaluate each positive category separately. This mode is only used for model debugging:
+
+```shell
+python evaluate.py \
+    --device "gpu" \
+    --model_path ./checkpoint/model_best \
+    --test_path ./data/dev.txt \
+    --output_dir ./checkpoint/model_best \
+    --label_names 'start_positions' 'end_positions' \
+    --max_seq_len 512 \
+    --per_device_eval_batch_size 16 \
+    --debug True
+```
+
+Output result:
+
+```text
+[2022-11-14 09:41:18,424] [    INFO] - ***** Running Evaluation *****
+[2022-11-14 09:41:18,424] [    INFO] -   Num examples = 160
+[2022-11-14 09:41:18,424] [    INFO] -   Pre device batch size = 4
+[2022-11-14 09:41:18,424] [    INFO] -   Total Batch size = 4
+[2022-11-14 09:41:18,424] [    INFO] -   Total prediction steps = 40
+[2022-11-14 09:41:26,451] [    INFO] - -----Evaluate model-------
+[2022-11-14 09:41:26,451] [    INFO] - Class Name: ALL CLASSES
+[2022-11-14 09:41:26,451] [    INFO] - Evaluation Precision: 0.94521 | Recall: 0.88462 | F1: 0.91391
+[2022-11-14 09:41:26,451] [    INFO] - -----------------------------
+[2022-11-14 09:41:26,452] [    INFO] - ***** Running Evaluation *****
+[2022-11-14 09:41:26,452] [    INFO] -   Num examples = 8
+[2022-11-14 09:41:26,452] [    INFO] -   Pre device batch size = 4
+[2022-11-14 09:41:26,452] [    INFO] -   Total Batch size = 4
+[2022-11-14 09:41:26,452] [    INFO] -   Total prediction steps = 2
+[2022-11-14 09:41:26,692] [    INFO] - Class Name: 开票日期
+[2022-11-14 09:41:26,692] [    INFO] - Evaluation Precision: 1.00000 | Recall: 1.00000 | F1: 1.00000
+[2022-11-14 09:41:26,692] [    INFO] - -----------------------------
+[2022-11-14 09:41:26,693] [    INFO] - ***** Running Evaluation *****
+[2022-11-14 09:41:26,693] [    INFO] -   Num examples = 8
+[2022-11-14 09:41:26,693] [    INFO] -   Pre device batch size = 4
+[2022-11-14 09:41:26,693] [    INFO] -   Total Batch size = 4
+[2022-11-14 09:41:26,693] [    INFO] -   Total prediction steps = 2
+[2022-11-14 09:41:26,952] [    INFO] - Class Name: 名称
+[2022-11-14 09:41:26,952] [    INFO] - Evaluation Precision: 0.87500 | Recall: 0.87500 | F1: 0.87500
+[2022-11-14 09:41:26,952] [    INFO] - -----------------------------
+...
+```
+
+Parameters:
+
+* `device`: Evaluation device, one of 'cpu', 'gpu' and 'npu' can be selected; the default is GPU evaluation.
+* `model_path`: The path of the model folder for evaluation, which must contain the model weight file `model_state.pdparams` and the configuration file `model_config.json`.
+* `test_path`: The test set file for evaluation.
+* `label_names`: the name of the training data label, UIE-X is set to 'start_positions' 'end_positions'; the default value is None.
+* `batch_size`: batch size, please adjust according to the machine situation, the default is 16.
+* `max_seq_len`: The maximum segmentation length of the text. When the input exceeds the maximum length, the input text will be automatically segmented. The default is 512.
+* `per_device_eval_batch_size`: Batch size per GPU core/NPU core/CPU for evaluation, default is 8.
+* `debug`: Whether to enable the debug mode to evaluate each positive category separately. This mode is only used for model debugging and is disabled by default.
+* `schema_lang`: Select the language of the schema, optional `ch` and `en`. The default is `ch`, please select `en` for the English dataset.
+
+<a name="25"></a>
+
+### 2.5 Inference
+
+Same with the pretrained models, you can use `paddlenlp.Taskflow` to load your custom model by specifying the path of the model weight file through `task_path`
+
+```python
+from pprint import pprint
+from paddlenlp import Taskflow
+from paddlenlp.utils.doc_parser import DocParser
+
+schema = ['开票日期', '名称', '纳税人识别号', '开户行及账号', '金额', '价税合计', 'No', '税率', '地址、电话', '税额']
+my_ie = Taskflow("information_extraction", model="uie-x-base", schema=schema, task_path='./checkpoint/model_best', precision='fp16')
+```
+
+We specify the extraction targets by setting `schema` and visualize the information of the specified `doc_path` document:
+
+```python
+doc_path = "./data/images/b199.jpg"
+results = my_ie({"doc": doc_path})
+pprint(results)
+
+# Result visualization
+DocParser.write_image_with_results(
+    doc_path,
+    result=results[0],
+    save_path="./image_show.png")
+```
+
+<div align="center">
+    <img src=https://user-images.githubusercontent.com/40840292/206084942-44ba477c-9244-4ce2-bbb5-ba430c9b926e.png height=550 width=700 />
+</div>
+
+<a name="26"></a>
+
+### 2.6 Experiments
+
+  |  |  Precision  | Recall | F1 Score |
+  | :---: | :--------: | :--------: | :--------: |
+  | 0-shot| 0.44898 | 0.56410  | 0.50000 |
+  | 5-shot| 0.9000 | 0.9231 | 0.9114 |
+  | 10-shot| 0.9125 | 0.93590 |  0.9241 |
+  | 20-shot| 0.9737 | 0.9487 | 0.9610 |
+  | 30-shot|  0.9744  | 0.9744  | 0.9744 |
+  | 30-shot+PP-Structure| 1.0  | 0.9625 |  0.9809 |
+
+
+n-shot means that the training set contains n labeled image data for model fine-tuning. Experiments show that UIE-X can further improve the results through a small amount of data (few-shot) and PP-Structure layout analysis.

applications/information_extraction/document/deploy/simple_serving/README.md

@@ -0,0 +1,57 @@
+# 基于PaddleNLP SimpleServing 的服务化部署
+
+## 目录
+- [环境准备](#环境准备)
+- [Server服务启动](#Server服务启动)
+- [Client请求启动](#Client请求启动)
+- [服务化自定义参数](#服务化自定义参数)
+
+## 环境准备
+使用有SimpleServing功能的PaddleNLP版本(或者最新的develop版本)
+
+```shell
+pip install paddlenlp >= 2.4.4
+```
+
+
+## Server服务启动
+
+```bash
+paddlenlp server server:app --workers 1 --host 0.0.0.0 --port 8189
+```
+
+## Client请求启动
+
+```bash
+python client.py
+```
+
+## 服务化自定义参数
+
+### Server 自定义参数
+#### schema替换
+```python
+# Default schema
+schema = ['开票日期', '名称', '纳税人识别号', '开户行及账号', '金额', '价税合计', 'No', '税率', '地址、电话', '税额']
+```
+
+#### 设置模型路径
+```
+# Default task_path
+uie = Taskflow('information_extraction', task_path='../../checkpoint/model_best/', schema=schema)
+```
+
+#### 多卡服务化预测
+PaddleNLP SimpleServing 支持多卡负载均衡预测，主要在服务化注册的时候，注册两个Taskflow的task即可，下面是示例代码
+```
+uie1 = Taskflow('information_extraction', task_path='../../checkpoint/model_best/', schema=schema, device_id=0)
+uie2 = Taskflow('information_extraction', task_path='../../checkpoint/model_best/', schema=schema, device_id=1)
+service.register_taskflow('uie', [uie1, uie2])
+```
+
+### Client 自定义参数
+
+```python
+# Changed to image paths you wanted
+image_paths = ['../../data/images/b1.jpg']
+```

applications/information_extraction/document/deploy/simple_serving/README_en.md

@@ -0,0 +1,65 @@
+# Service deployment based on PaddleNLP SimpleServing
+
+## Table of contents
+- [Environment Preparation](#1)
+- [Server](#2)
+- [Client](#3)
+- [Service Custom Parameters](#4)
+
+<a name="1"></a>
+
+## Environment Preparation
+Use the PaddleNLP version with SimpleServing function (or the latest develop version)
+
+```shell
+pip install paddlenlp >= 2.4.4
+```
+
+<a name="2"></a>
+
+## Server
+
+```bash
+paddlenlp server server:app --workers 1 --host 0.0.0.0 --port 8189
+```
+
+<a name="3"></a>
+
+## Client
+
+```bash
+python client.py
+```
+
+<a name="4"></a>
+
+## Service custom parameters
+
+### Server Custom Parameters
+
+#### schema replacement
+```python
+# Default schema
+schema = ['Billing Date', 'Name', 'Taxpayer Identification Number', 'Account Bank and Account Number', 'Amount', 'Total Price and Tax', 'No', 'Tax Rate', 'Address, Phone', 'tax']
+```
+
+#### Set model path
+```
+# Default task_path
+uie = Taskflow('information_extraction', task_path='../../checkpoint/model_best/', schema=schema)
+```
+
+#### Doka Service Prediction
+PaddleNLP SimpleServing supports multi-card load balancing prediction, mainly during service registration, just register two Taskflow tasks, the following is the sample code
+```
+uie1 = Taskflow('information_extraction', task_path='../../checkpoint/model_best/', schema=schema, device_id=0)
+uie2 = Taskflow('information_extraction', task_path='../../checkpoint/model_best/', schema=schema, device_id=1)
+service. register_taskflow('uie', [uie1, uie2])
+```
+
+### Client Custom Parameters
+
+```python
+# Changed to image paths you wanted
+image_paths = ['../../data/images/b1.jpg']
+```

applications/information_extraction/document/deploy/simple_serving/client.py

@@ -0,0 +1,42 @@
+# Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import json
+
+import requests
+
+from paddlenlp.utils.doc_parser import DocParser
+
+# Define the document parser
+doc_parser = DocParser()
+
+image_paths = ["../../data/images/b1.jpg"]
+image_base64_docs = []
+
+# Get the image base64 to post
+for image_path in image_paths:
+    req_dict = {}
+    doc = doc_parser.parse({"doc": image_path}, do_ocr=False)
+    base64 = doc["image"]
+    req_dict["doc"] = base64
+    image_base64_docs.append(req_dict)
+
+url = "http://0.0.0.0:8189/taskflow/uie"
+headers = {"Content-Type": "application/json"}
+data = {"data": {"text": image_base64_docs}}
+
+# Post the requests
+r = requests.post(url=url, headers=headers, data=json.dumps(data))
+datas = json.loads(r.text)
+print(datas)

applications/information_extraction/document/deploy/simple_serving/server.py

@@ -0,0 +1,27 @@
+# Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from paddlenlp import SimpleServer, Taskflow
+
+# The schema changed to your defined schema
+schema = ["开票日期", "名称", "纳税人识别号", "开户行及账号", "金额", "价税合计", "No", "税率", "地址、电话", "税额"]
+# The task path changed to your best model path
+uie = Taskflow(
+    "information_extraction",
+    schema=schema,
+    task_path="../../checkpoint/model_best",
+)
+# If you want to define the finetuned uie service
+app = SimpleServer()
+app.register_taskflow("taskflow/uie", uie)

applications/information_extraction/document/evaluate.py

@@ -0,0 +1,149 @@
+# Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from dataclasses import dataclass, field
+from functools import partial
+from typing import Optional
+
+import paddle
+from utils import convert_example, reader
+
+from paddlenlp.datasets import MapDataset, load_dataset
+from paddlenlp.trainer import PdArgumentParser, Trainer, TrainingArguments
+from paddlenlp.transformers import UIEX, AutoTokenizer
+from paddlenlp.utils.ie_utils import (
+    compute_metrics,
+    get_relation_type_dict,
+    uie_loss_func,
+    unify_prompt_name,
+)
+from paddlenlp.utils.log import logger
+
+
+@dataclass
+class DataArguments:
+    """
+    Arguments pertaining to what data we are going to input our model for evaluation.
+    Using `PdArgumentParser` we can turn this class into argparse arguments to be able to
+    specify them on the command line.
+    """
+
+    test_path: str = field(default=None, metadata={"help": "The path of test set."})
+
+    schema_lang: str = field(
+        default="ch", metadata={"help": "Select the language type for schema, such as 'ch', 'en'"}
+    )
+
+    max_seq_len: Optional[int] = field(
+        default=512,
+        metadata={
+            "help": "The maximum total input sequence length after tokenization. Sequences longer "
+            "than this will be truncated, sequences shorter will be padded."
+        },
+    )
+
+    debug: bool = field(
+        default=False,
+        metadata={"help": "Whether choose debug mode."},
+    )
+
+
+@dataclass
+class ModelArguments:
+    """
+    Arguments pertaining to which model/config/tokenizer we are going to fine-tune from.
+    """
+
+    model_path: Optional[str] = field(
+        default=None, metadata={"help": "The path of saved model that you want to load."}
+    )
+
+
+def do_eval():
+    parser = PdArgumentParser((ModelArguments, DataArguments, TrainingArguments))
+    model_args, data_args, training_args = parser.parse_args_into_dataclasses()
+
+    # Log model and data config
+    training_args.print_config(model_args, "Model")
+    training_args.print_config(data_args, "Data")
+
+    paddle.set_device(training_args.device)
+
+    tokenizer = AutoTokenizer.from_pretrained(model_args.model_path)
+    model = UIEX.from_pretrained(model_args.model_path)
+
+    test_ds = load_dataset(reader, data_path=data_args.test_path, max_seq_len=data_args.max_seq_len, lazy=False)
+    trans_fn = partial(convert_example, tokenizer=tokenizer, max_seq_len=data_args.max_seq_len)
+    if data_args.debug:
+        class_dict = {}
+        relation_data = []
+
+        for data in test_ds:
+            class_name = unify_prompt_name(data["prompt"])
+            # Only positive examples are evaluated in debug mode
+            if len(data["result_list"]) != 0:
+                p = "的" if data_args.schema_lang == "ch" else " of "
+                if p not in data["prompt"]:
+                    class_dict.setdefault(class_name, []).append(data)
+                else:
+                    relation_data.append((data["prompt"], data))
+
+        relation_type_dict = get_relation_type_dict(relation_data, schema_lang=data_args.schema_lang)
+    test_ds = test_ds.map(trans_fn)
+
+    trainer = Trainer(
+        model=model,
+        criterion=uie_loss_func,
+        args=training_args,
+        eval_dataset=test_ds,
+        tokenizer=tokenizer,
+        compute_metrics=compute_metrics,
+    )
+    eval_metrics = trainer.evaluate()
+    logger.info("-----Evaluate model-------")
+    logger.info("Class Name: ALL CLASSES")
+    logger.info(
+        "Evaluation Precision: %.5f | Recall: %.5f | F1: %.5f"
+        % (eval_metrics["eval_precision"], eval_metrics["eval_recall"], eval_metrics["eval_f1"])
+    )
+    logger.info("-----------------------------")
+    if data_args.debug:
+        for key in class_dict.keys():
+            test_ds = MapDataset(class_dict[key])
+            test_ds = test_ds.map(trans_fn)
+            eval_metrics = trainer.evaluate(eval_dataset=test_ds)
+
+            logger.info("Class Name: %s" % key)
+            logger.info(
+                "Evaluation Precision: %.5f | Recall: %.5f | F1: %.5f"
+                % (eval_metrics["eval_precision"], eval_metrics["eval_recall"], eval_metrics["eval_f1"])
+            )
+            logger.info("-----------------------------")
+        for key in relation_type_dict.keys():
+            test_ds = MapDataset(relation_type_dict[key])
+            test_ds = test_ds.map(trans_fn)
+            eval_metrics = trainer.evaluate(eval_dataset=test_ds)
+            logger.info("-----------------------------")
+            if data_args.schema_lang == "ch":
+                logger.info("Class Name: X的%s" % key)
+            else:
+                logger.info("Class Name: %s of X" % key)
+            logger.info(
+                "Evaluation Precision: %.5f | Recall: %.5f | F1: %.5f"
+                % (eval_metrics["eval_precision"], eval_metrics["eval_recall"], eval_metrics["eval_f1"])
+            )
+
+
+if __name__ == "__main__":
+    do_eval()

applications/information_extraction/document/finetune.py

@@ -0,0 +1,177 @@
+# Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import os
+from dataclasses import dataclass, field
+from functools import partial
+from typing import List, Optional
+
+import paddle
+from utils import convert_example, reader
+
+from paddlenlp.datasets import load_dataset
+from paddlenlp.trainer import (
+    CompressionArguments,
+    PdArgumentParser,
+    Trainer,
+    get_last_checkpoint,
+)
+from paddlenlp.transformers import UIEX, AutoTokenizer, export_model
+from paddlenlp.utils.ie_utils import compute_metrics, uie_loss_func
+from paddlenlp.utils.log import logger
+
+
+@dataclass
+class DataArguments:
+    """
+    Arguments pertaining to what data we are going to input our model for training and eval.
+    Using `PdArgumentParser` we can turn this class into argparse arguments to be able to
+    specify them on the command line.
+    """
+
+    train_path: str = field(
+        default=None, metadata={"help": "The name of the dataset to use (via the datasets library)."}
+    )
+
+    dev_path: str = field(
+        default=None, metadata={"help": "The name of the dataset to use (via the datasets library)."}
+    )
+
+    max_seq_len: Optional[int] = field(
+        default=512,
+        metadata={
+            "help": "The maximum total input sequence length after tokenization. Sequences longer "
+            "than this will be truncated, sequences shorter will be padded."
+        },
+    )
+
+    dynamic_max_length: Optional[List[int]] = field(
+        default=None,
+        metadata={"help": "dynamic max length from batch, it can be array of length, eg: 16 32 64 128"},
+    )
+
+
+@dataclass
+class ModelArguments:
+    """
+    Arguments pertaining to which model/config/tokenizer we are going to fine-tune from.
+    """
+
+    model_name_or_path: Optional[str] = field(default="uie-x-base", metadata={"help": "Path to pretrained model"})
+    export_model_dir: Optional[str] = field(
+        default=None,
+        metadata={"help": "Path to directory to store the exported inference model."},
+    )
+
+
+def main():
+    parser = PdArgumentParser((ModelArguments, DataArguments, CompressionArguments))
+    model_args, data_args, training_args = parser.parse_args_into_dataclasses()
+    training_args.label_names = ["start_positions", "end_positions"]
+
+    # Log model and data config
+    training_args.print_config(model_args, "Model")
+    training_args.print_config(data_args, "Data")
+
+    paddle.set_device(training_args.device)
+
+    # Log on each process the small summary:
+    logger.warning(
+        f"Process rank: {training_args.local_rank}, device: {training_args.device}, world_size: {training_args.world_size}, "
+        + f"distributed training: {bool(training_args.local_rank != -1)}, 16-bits training: {training_args.fp16}"
+    )
+
+    # Detecting last checkpoint.
+    last_checkpoint = None
+    if os.path.isdir(training_args.output_dir) and training_args.do_train and not training_args.overwrite_output_dir:
+        last_checkpoint = get_last_checkpoint(training_args.output_dir)
+        if last_checkpoint is None and len(os.listdir(training_args.output_dir)) > 0:
+            raise ValueError(
+                f"Output directory ({training_args.output_dir}) already exists and is not empty. "
+                "Use --overwrite_output_dir to overcome."
+            )
+        elif last_checkpoint is not None and training_args.resume_from_checkpoint is None:
+            logger.info(
+                f"Checkpoint detected, resuming training at {last_checkpoint}. To avoid this behavior, change "
+                "the `--output_dir` or add `--overwrite_output_dir` to train from scratch."
+            )
+
+    # Define model and tokenizer
+    model = UIEX.from_pretrained(model_args.model_name_or_path)
+    tokenizer = AutoTokenizer.from_pretrained(model_args.model_name_or_path)
+
+    # Load and preprocess dataset
+    train_ds = load_dataset(reader, data_path=data_args.train_path, max_seq_len=data_args.max_seq_len, lazy=False)
+    dev_ds = load_dataset(reader, data_path=data_args.dev_path, max_seq_len=data_args.max_seq_len, lazy=False)
+    trans_fn = partial(
+        convert_example,
+        tokenizer=tokenizer,
+        max_seq_len=data_args.max_seq_len,
+        dynamic_max_length=data_args.dynamic_max_length,
+    )
+    train_ds = train_ds.map(trans_fn)
+    dev_ds = dev_ds.map(trans_fn)
+
+    trainer = Trainer(
+        model=model,
+        criterion=uie_loss_func,
+        args=training_args,
+        train_dataset=train_ds if training_args.do_train else None,
+        eval_dataset=dev_ds if training_args.do_eval else None,
+        tokenizer=tokenizer,
+        compute_metrics=compute_metrics,
+    )
+
+    trainer.optimizer = paddle.optimizer.AdamW(
+        learning_rate=training_args.learning_rate, parameters=model.parameters()
+    )
+    checkpoint = None
+    if training_args.resume_from_checkpoint is not None:
+        checkpoint = training_args.resume_from_checkpoint
+    elif last_checkpoint is not None:
+        checkpoint = last_checkpoint
+
+    # Training
+    if training_args.do_train:
+        train_result = trainer.train(resume_from_checkpoint=checkpoint)
+        metrics = train_result.metrics
+        trainer.save_model()
+        trainer.log_metrics("train", metrics)
+        trainer.save_metrics("train", metrics)
+        trainer.save_state()
+
+    # Evaluate and tests model
+    if training_args.do_eval:
+        eval_metrics = trainer.evaluate()
+        trainer.log_metrics("eval", eval_metrics)
+
+    # export inference model
+    if training_args.do_export:
+        # You can also load from certain checkpoint
+        # trainer.load_state_dict_from_checkpoint("/path/to/checkpoint/")
+        input_spec = [
+            paddle.static.InputSpec(shape=[None, None], dtype="int64", name="input_ids"),
+            paddle.static.InputSpec(shape=[None, None], dtype="int64", name="token_type_ids"),
+            paddle.static.InputSpec(shape=[None, None], dtype="int64", name="position_ids"),
+            paddle.static.InputSpec(shape=[None, None], dtype="int64", name="attention_mask"),
+            paddle.static.InputSpec(shape=[None, None, 4], dtype="int64", name="bbox"),
+            paddle.static.InputSpec(shape=[None, 3, 224, 224], dtype="int64", name="image"),
+        ]
+        if model_args.export_model_dir is None:
+            model_args.export_model_dir = os.path.join(training_args.output_dir, "export")
+        export_model(model=trainer.model, input_spec=input_spec, path=model_args.export_model_dir)
+
+
+if __name__ == "__main__":
+    main()

applications/information_extraction/document/utils.py

@@ -0,0 +1,374 @@
+# Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import base64
+import json
+from typing import List, Optional
+
+import numpy as np
+
+from paddlenlp.utils.ie_utils import map_offset, pad_image_data
+from paddlenlp.utils.log import logger
+
+
+def reader(data_path, max_seq_len=512):
+    """
+    read json
+    """
+    with open(data_path, "r", encoding="utf-8") as f:
+        for line in f:
+            json_line = json.loads(line)
+            content = json_line["content"].strip()
+            prompt = json_line["prompt"]
+            boxes = json_line.get("bbox", None)
+            image = json_line.get("image", None)
+            # Model Input is aslike: [CLS] prompt [SEP] [SEP] text [SEP] for UIE-X
+            if boxes is not None and image is not None:
+                summary_token_num = 4
+            else:
+                summary_token_num = 3
+            if max_seq_len <= len(prompt) + summary_token_num:
+                raise ValueError("The value of max_seq_len is too small, please set a larger value")
+            max_content_len = max_seq_len - len(prompt) - summary_token_num
+            if len(content) <= max_content_len:
+                yield json_line
+            else:
+                result_list = json_line["result_list"]
+                json_lines = []
+                accumulate = 0
+                while True:
+                    cur_result_list = []
+                    for result in result_list:
+                        if result["end"] - result["start"] > max_content_len:
+                            logger.warning(
+                                "result['end'] - result ['start'] exceeds max_content_len, which will result in no valid instance being returned"
+                            )
+                        if (
+                            result["start"] + 1 <= max_content_len < result["end"]
+                            and result["end"] - result["start"] <= max_content_len
+                        ):
+                            max_content_len = result["start"]
+                            break
+
+                    cur_content = content[:max_content_len]
+                    res_content = content[max_content_len:]
+                    if boxes is not None and image is not None:
+                        cur_boxes = boxes[:max_content_len]
+                        res_boxes = boxes[max_content_len:]
+
+                    while True:
+                        if len(result_list) == 0:
+                            break
+                        elif result_list[0]["end"] <= max_content_len:
+                            if result_list[0]["end"] > 0:
+                                cur_result = result_list.pop(0)
+                                cur_result_list.append(cur_result)
+                            else:
+                                cur_result_list = [result for result in result_list]
+                                break
+                        else:
+                            break
+
+                    if boxes is not None and image is not None:
+                        json_line = {
+                            "content": cur_content,
+                            "result_list": cur_result_list,
+                            "prompt": prompt,
+                            "bbox": cur_boxes,
+                            "image": image,
+                        }
+                    else:
+                        json_line = {
+                            "content": cur_content,
+                            "result_list": cur_result_list,
+                            "prompt": prompt,
+                        }
+                    json_lines.append(json_line)
+
+                    for result in result_list:
+                        if result["end"] <= 0:
+                            break
+                        result["start"] -= max_content_len
+                        result["end"] -= max_content_len
+                    accumulate += max_content_len
+                    max_content_len = max_seq_len - len(prompt) - summary_token_num
+                    if len(res_content) == 0:
+                        break
+                    elif len(res_content) < max_content_len:
+                        if boxes is not None and image is not None:
+                            json_line = {
+                                "content": res_content,
+                                "result_list": result_list,
+                                "prompt": prompt,
+                                "bbox": res_boxes,
+                                "image": image,
+                            }
+                        else:
+                            json_line = {"content": res_content, "result_list": result_list, "prompt": prompt}
+
+                        json_lines.append(json_line)
+                        break
+                    else:
+                        content = res_content
+                        boxes = res_boxes
+
+                for json_line in json_lines:
+                    yield json_line
+
+
+def get_dynamic_max_len(examples, default_max_len: int, dynamic_max_length: List[int]) -> int:
+    """get max_length by examples which you can change it by examples in batch"""
+    cur_length = len(examples[0]["input_ids"])
+    max_length = default_max_len
+    for max_length_option in sorted(dynamic_max_length):
+        if cur_length <= max_length_option:
+            max_length = max_length_option
+            break
+    return max_length
+
+
+def convert_example(
+    example,
+    tokenizer,
+    max_seq_len,
+    pad_id=1,
+    c_sep_id=2,
+    summary_token_num=4,
+    dynamic_max_length: Optional[List[int]] = None,
+):
+
+    content = example["content"]
+    prompt = example["prompt"]
+    bbox_lines = example.get("bbox", None)
+    image_buff_string = example.get("image", None)
+    # Text
+    if bbox_lines is None or image_buff_string is None:
+        if dynamic_max_length is not None:
+            temp_encoded_inputs = tokenizer(
+                text=[example["prompt"]],
+                text_pair=[example["content"]],
+                truncation=True,
+                max_seq_len=max_seq_len,
+                return_attention_mask=True,
+                return_position_ids=True,
+                return_dict=False,
+                return_offsets_mapping=True,
+            )
+            max_length = get_dynamic_max_len(
+                examples=temp_encoded_inputs, default_max_len=max_seq_len, dynamic_max_length=dynamic_max_length
+            )
+            # always pad to max_length
+            encoded_inputs = tokenizer(
+                text=[example["prompt"]],
+                text_pair=[example["content"]],
+                truncation=True,
+                max_seq_len=max_length,
+                pad_to_max_seq_len=True,
+                return_attention_mask=True,
+                return_position_ids=True,
+                return_dict=False,
+                return_offsets_mapping=True,
+            )
+            max_seq_len = max_length
+        else:
+            encoded_inputs = tokenizer(
+                text=[example["prompt"]],
+                text_pair=[example["content"]],
+                truncation=True,
+                max_seq_len=max_seq_len,
+                pad_to_max_seq_len=True,
+                return_attention_mask=True,
+                return_position_ids=True,
+                return_offsets_mapping=True,
+                return_dict=False,
+            )
+
+        encoded_inputs = encoded_inputs[0]
+
+        inputs_ids = encoded_inputs["input_ids"]
+        position_ids = encoded_inputs["position_ids"]
+        attention_mask = encoded_inputs["attention_mask"]
+
+        q_sep_index = inputs_ids.index(2, 1)
+        c_sep_index = attention_mask.index(0)
+
+        offset_mapping = [list(x) for x in encoded_inputs["offset_mapping"]]
+
+        bias = 0
+        for index in range(len(offset_mapping)):
+            if index == 0:
+                continue
+            mapping = offset_mapping[index]
+            if mapping[0] == 0 and mapping[1] == 0 and bias == 0:
+                # bias = index
+                bias = offset_mapping[index - 1][-1] + 1
+
+            if mapping[0] == 0 and mapping[1] == 0:
+                continue
+            offset_mapping[index][0] += bias
+            offset_mapping[index][1] += bias
+
+        offset_bias = bias
+
+        bbox_list = [[0, 0, 0, 0] for x in range(len(inputs_ids))]
+        token_type_ids = [
+            1 if token_index <= q_sep_index or token_index > c_sep_index else 0 for token_index in range(max_seq_len)
+        ]
+        padded_image = np.zeros([3, 224, 224])
+
+    # Doc
+    else:
+        inputs_ids = []
+        prev_bbox = [-1, -1, -1, -1]
+        this_text_line = ""
+        q_sep_index = -1
+        offset_mapping = []
+        last_offset = 0
+        for char_index, (char, bbox) in enumerate(zip(content, bbox_lines)):
+            if char_index == 0:
+                prev_bbox = bbox
+                this_text_line = char
+                continue
+
+            if all([bbox[x] == prev_bbox[x] for x in range(4)]):
+                this_text_line += char
+            else:
+                offset_mapping, last_offset, q_sep_index, inputs_ids = _encode_doc(
+                    tokenizer,
+                    offset_mapping,
+                    last_offset,
+                    prompt,
+                    this_text_line,
+                    inputs_ids,
+                    q_sep_index,
+                    max_seq_len,
+                )
+                this_text_line = char
+            prev_bbox = bbox
+
+        if len(this_text_line) > 0:
+            offset_mapping, last_offset, q_sep_index, inputs_ids = _encode_doc(
+                tokenizer, offset_mapping, last_offset, prompt, this_text_line, inputs_ids, q_sep_index, max_seq_len
+            )
+
+        if len(inputs_ids) > max_seq_len:
+            inputs_ids = inputs_ids[: (max_seq_len - 1)] + [c_sep_id]
+            offset_mapping = offset_mapping[: (max_seq_len - 1)] + [[0, 0]]
+        else:
+            inputs_ids += [c_sep_id]
+            offset_mapping += [[0, 0]]
+
+        offset_bias = offset_mapping[q_sep_index - 1][-1] + 1
+
+        seq_len = len(inputs_ids)
+        inputs_ids += [pad_id] * (max_seq_len - seq_len)
+        token_type_ids = [1] * (q_sep_index + 1) + [0] * (seq_len - q_sep_index - 1)
+        token_type_ids += [pad_id] * (max_seq_len - seq_len)
+
+        bbox_list = _process_bbox(inputs_ids, bbox_lines, offset_mapping, offset_bias)
+
+        offset_mapping += [[0, 0]] * (max_seq_len - seq_len)
+
+        position_ids = list(range(seq_len))
+
+        position_ids = position_ids + [0] * (max_seq_len - seq_len)
+        attention_mask = [1] * seq_len + [0] * (max_seq_len - seq_len)
+
+        image_data = base64.b64decode(image_buff_string.encode("utf8"))
+        padded_image = pad_image_data(image_data)
+
+    start_ids = np.array([0.0 for x in range(max_seq_len)], dtype="int64")
+    end_ids = np.array([0.0 for x in range(max_seq_len)], dtype="int64")
+
+    for item in example["result_list"]:
+        start = map_offset(item["start"] + offset_bias, offset_mapping)
+        end = map_offset(item["end"] - 1 + offset_bias, offset_mapping)
+        start_ids[start] = 1.0
+        end_ids[end] = 1.0
+
+    assert len(inputs_ids) == max_seq_len
+    assert len(token_type_ids) == max_seq_len
+    assert len(position_ids) == max_seq_len
+    assert len(attention_mask) == max_seq_len
+    assert len(bbox_list) == max_seq_len
+    tokenized_output = {
+        "input_ids": inputs_ids,
+        "token_type_ids": token_type_ids,
+        "position_ids": position_ids,
+        "attention_mask": attention_mask,
+        "bbox": bbox_list,
+        "image": padded_image,
+        "start_positions": start_ids,
+        "end_positions": end_ids,
+    }
+    return tokenized_output
+
+
+def _process_bbox(tokens, bbox_lines, offset_mapping, offset_bias):
+    bbox_list = [[0, 0, 0, 0] for x in range(len(tokens))]
+
+    for index, bbox in enumerate(bbox_lines):
+        index_token = map_offset(index + offset_bias, offset_mapping)
+        if 0 <= index_token < len(bbox_list):
+            bbox_list[index_token] = bbox
+    return bbox_list
+
+
+def _encode_doc(tokenizer, offset_mapping, last_offset, prompt, this_text_line, inputs_ids, q_sep_index, max_seq_len):
+    if len(offset_mapping) == 0:
+        content_encoded_inputs = tokenizer(
+            text=[prompt],
+            text_pair=[this_text_line],
+            max_seq_len=max_seq_len,
+            return_dict=False,
+            return_offsets_mapping=True,
+        )
+        content_encoded_inputs = content_encoded_inputs[0]
+        inputs_ids = content_encoded_inputs["input_ids"][:-1]
+        sub_offset_mapping = [list(x) for x in content_encoded_inputs["offset_mapping"]]
+        q_sep_index = content_encoded_inputs["input_ids"].index(2, 1)
+
+        bias = 0
+        for i in range(len(sub_offset_mapping)):
+            if i == 0:
+                continue
+            mapping = sub_offset_mapping[i]
+            if mapping[0] == 0 and mapping[1] == 0 and bias == 0:
+                bias = sub_offset_mapping[i - 1][-1] + 1
+            if mapping[0] == 0 and mapping[1] == 0:
+                continue
+            if mapping == sub_offset_mapping[i - 1]:
+                continue
+            sub_offset_mapping[i][0] += bias
+            sub_offset_mapping[i][1] += bias
+
+        offset_mapping = sub_offset_mapping[:-1]
+        last_offset = offset_mapping[-1][-1]
+    else:
+        content_encoded_inputs = tokenizer(
+            text=this_text_line, max_seq_len=max_seq_len, return_dict=False, return_offsets_mapping=True
+        )
+        inputs_ids += content_encoded_inputs["input_ids"][1:-1]
+        sub_offset_mapping = [list(x) for x in content_encoded_inputs["offset_mapping"]]
+
+        for i, sub_list in enumerate(sub_offset_mapping[1:-1]):
+            if i == 0:
+                org_offset = sub_list[1]
+            else:
+                if sub_list[0] != org_offset and sub_offset_mapping[1:-1][i - 1] != sub_list:
+                    last_offset += 1
+                org_offset = sub_list[1]
+            offset_mapping += [[last_offset, sub_list[1] - sub_list[0] + last_offset]]
+            last_offset = offset_mapping[-1][-1]
+    return offset_mapping, last_offset, q_sep_index, inputs_ids

applications/information_extraction/label_studio.py

@@ -0,0 +1,139 @@
+# coding=utf-8
+# Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import argparse
+import json
+import os
+import random
+import time
+from decimal import Decimal
+
+import numpy as np
+import paddle
+
+from paddlenlp.trainer.argparser import strtobool
+from paddlenlp.utils.log import logger
+from paddlenlp.utils.tools import DataConverter
+
+
+def set_seed(seed):
+    paddle.seed(seed)
+    random.seed(seed)
+    np.random.seed(seed)
+
+
+def do_convert():
+    set_seed(args.seed)
+
+    tic_time = time.time()
+    if not os.path.exists(args.label_studio_file):
+        raise ValueError("Please input the correct path of label studio file.")
+
+    if not os.path.exists(args.save_dir):
+        os.makedirs(args.save_dir)
+
+    if len(args.splits) != 0 and len(args.splits) != 3:
+        raise ValueError("Only []/ len(splits)==3 accepted for splits.")
+
+    def _check_sum(splits):
+        return Decimal(str(splits[0])) + Decimal(str(splits[1])) + Decimal(str(splits[2])) == Decimal("1")
+
+    if len(args.splits) == 3 and not _check_sum(args.splits):
+        raise ValueError("Please set correct splits, sum of elements in splits should be equal to 1.")
+
+    with open(args.label_studio_file, "r", encoding="utf-8") as f:
+        raw_examples = json.loads(f.read())
+
+    if args.is_shuffle:
+        indexes = np.random.permutation(len(raw_examples))
+        index_list = indexes.tolist()
+        raw_examples = [raw_examples[i] for i in indexes]
+
+    i1, i2, _ = args.splits
+    p1 = int(len(raw_examples) * i1)
+    p2 = int(len(raw_examples) * (i1 + i2))
+
+    train_ids = index_list[:p1]
+    dev_ids = index_list[p1:p2]
+    test_ids = index_list[p2:]
+
+    with open(os.path.join(args.save_dir, "sample_index.json"), "w") as fp:
+        maps = {"train_ids": train_ids, "dev_ids": dev_ids, "test_ids": test_ids}
+        fp.write(json.dumps(maps))
+
+    if raw_examples[0]["data"].get("image"):
+        anno_type = "image"
+    else:
+        anno_type = "text"
+
+    data_converter = DataConverter(
+        args.label_studio_file,
+        negative_ratio=args.negative_ratio,
+        prompt_prefix=args.prompt_prefix,
+        options=args.options,
+        separator=args.separator,
+        layout_analysis=args.layout_analysis,
+        schema_lang=args.schema_lang,
+        ocr_lang=args.ocr_lang,
+        anno_type=anno_type,
+    )
+
+    if args.task_type == "ext":
+        train_examples = data_converter.convert_ext_examples(raw_examples[:p1])
+        dev_examples = data_converter.convert_ext_examples(raw_examples[p1:p2], is_train=False)
+        test_examples = data_converter.convert_ext_examples(raw_examples[p2:], is_train=False)
+    else:
+        train_examples = data_converter.convert_cls_examples(raw_examples[:p1])
+        dev_examples = data_converter.convert_cls_examples(raw_examples[p1:p2])
+        test_examples = data_converter.convert_cls_examples(raw_examples[p2:])
+
+    def _save_examples(save_dir, file_name, examples):
+        count = 0
+        save_path = os.path.join(save_dir, file_name)
+        with open(save_path, "w", encoding="utf-8") as f:
+            for example in examples:
+                f.write(json.dumps(example, ensure_ascii=False) + "\n")
+                count += 1
+        logger.info("Save %d examples to %s." % (count, save_path))
+
+    _save_examples(args.save_dir, "train.txt", train_examples)
+    _save_examples(args.save_dir, "dev.txt", dev_examples)
+    _save_examples(args.save_dir, "test.txt", test_examples)
+
+    logger.info("Finished! It takes %.2f seconds" % (time.time() - tic_time))
+
+
+if __name__ == "__main__":
+    # yapf: disable
+    parser = argparse.ArgumentParser()
+
+    parser.add_argument("--label_studio_file", default="./data/label_studio.json", type=str, help="The annotation file exported from label studio platform.")
+    parser.add_argument("--save_dir", default="./data", type=str, help="The path of data that you wanna save.")
+    parser.add_argument("--negative_ratio", default=5, type=int, help="Used only for the extraction task, the ratio of positive and negative samples, number of negtive samples = negative_ratio * number of positive samples")
+    parser.add_argument("--splits", default=[0.8, 0.1, 0.1], type=float, nargs="*", help="The ratio of samples in datasets. [0.6, 0.2, 0.2] means 60% samples used for training, 20% for evaluation and 20% for test.")
+    parser.add_argument("--task_type", choices=['ext', 'cls'], default="ext", type=str, help="Select task type, ext for the extraction task and cls for the classification task, defaults to ext.")
+    parser.add_argument("--options", default=["正向", "负向"], type=str, nargs="+", help="Used only for the classification task, the options for classification")
+    parser.add_argument("--prompt_prefix", default="情感倾向", type=str, help="Used only for the classification task, the prompt prefix for classification")
+    parser.add_argument("--is_shuffle", default="True", type=strtobool, help="Whether to shuffle the labeled dataset, defaults to True.")
+    parser.add_argument("--layout_analysis", default=False, type=bool, help="Enable layout analysis to optimize the order of OCR result.")
+    parser.add_argument("--seed", type=int, default=1000, help="Random seed for initialization")
+    parser.add_argument("--separator", type=str, default='##', help="Used only for entity/aspect-level classification task, separator for entity label and classification label")
+    parser.add_argument("--schema_lang", choices=["ch", "en"], default="ch", help="Select the language type for schema.")
+    parser.add_argument("--ocr_lang", choices=["ch", "en"], default="ch", help="Select the language type for OCR.")
+
+    args = parser.parse_args()
+    # yapf: enable
+
+    do_convert()

applications/information_extraction/label_studio_doc.md

@@ -0,0 +1,272 @@
+# 文档抽取任务Label Studio使用指南
+
+ **目录**
+
+- [1. 安装](#1)
+- [2. 文档抽取任务标注](#2)
+    - [2.1 项目创建](#21)
+    - [2.2 数据上传](#22)
+    - [2.3 标签构建](#23)
+    - [2.4 任务标注](#24)
+    - [2.5 数据导出](#25)
+    - [2.6 数据转换](#26)
+    - [2.7 更多配置](#27)
+
+<a name="1"></a>
+
+## 1. 安装
+**以下标注示例用到的环境配置：**
+
+- Python 3.8+
+- label-studio == 1.6.0
+- paddleocr >= 2.6.0.1
+
+在终端(terminal)使用pip安装label-studio：
+
+```shell
+pip install label-studio==1.6.0
+```
+
+安装完成后，运行以下命令行：
+```shell
+label-studio start
+```
+
+在浏览器打开[http://localhost:8080/](http://127.0.0.1:8080/)，输入用户名和密码登录，开始使用label-studio进行标注。
+
+
+<a name="2"></a>
+
+## 2. 文档抽取任务标注
+
+<a name="21"></a>
+
+#### 2.1 项目创建
+
+点击创建（Create）开始创建一个新的项目，填写项目名称、描述，然后选择``Object Detection with Bounding Boxes``。
+
+- 填写项目名称、描述
+
+<div align="center">
+    <img src=https://user-images.githubusercontent.com/40840292/199445809-1206f887-2782-459e-9001-fbd790d59a5e.png height=300 width=1200 />
+</div>
+
+- **命名实体识别、关系抽取、事件抽取、实体/评价维度分类**任务选择``Object Detection with Bounding Boxes`
+
+<div align="center">
+    <img src=https://user-images.githubusercontent.com/40840292/199660090-d84901dd-001d-4620-bffa-0101a4ecd6e5.png height=400 width=1200 />
+</div>
+
+- **文档分类**任务选择``Image Classification`
+
+<div align="center">
+    <img src=https://user-images.githubusercontent.com/40840292/199729973-53a994d8-da71-4ab9-84f5-83297e19a7a1.png height=400 width=1200 />
+</div>
+
+- 添加标签(也可跳过后续在Setting/Labeling Interface中添加)
+
+<div align="center">
+    <img src=https://user-images.githubusercontent.com/40840292/199450930-4c0cd189-6085-465a-aca0-6ba6f52a0c0d.png height=600 width=1200 />
+</div>
+
+图中展示了Span实体类型标签的构建，其他类型标签的构建可参考[2.3标签构建](#23)
+
+<a name="22"></a>
+
+#### 2.2 数据上传
+
+先从本地或HTTP链接上传图片，然后选择导入本项目。
+
+<div align="center">
+    <img src=https://user-images.githubusercontent.com/40840292/199452007-2d45f7ba-c631-46b4-b21f-729a2ed652e9.png height=270 width=1200 />
+</div>
+
+<a name="23"></a>
+
+#### 2.3 标签构建
+
+- Span实体类型标签
+
+<div align="center">
+    <img src=https://user-images.githubusercontent.com/40840292/199456432-ce601ab0-7d6c-458f-ac46-8839dbc4d013.png height=500 width=1200 />
+</div>
+
+
+- Relation关系类型标签
+
+<div align="center">
+    <img src=https://user-images.githubusercontent.com/40840292/199877621-f60e00c7-81ae-42e1-b498-8ebc5b5bd0fd.png height=650 width=1200 />
+</div>
+
+Relation XML模板：
+
+```xml
+  <Relations>
+    <Relation value="单位"/>
+    <Relation value="数量"/>
+    <Relation value="金额"/>
+  </Relations>
+```
+
+
+- 分类类别标签
+
+<div align="center">
+    <img src=https://user-images.githubusercontent.com/40840292/199891626-cc995783-18d2-41dc-88de-260b979edc56.png height=500 width=1200 />
+</div>
+
+<a name="24"></a>
+
+#### 2.4 任务标注
+
+- 实体抽取
+
+    - 标注示例：
+
+        <div align="center">
+            <img src=https://user-images.githubusercontent.com/40840292/199879427-82806ffc-dc60-4ec7-bda5-e16419ee9d15.png height=650 width=800 />
+        </div>
+
+    - 该标注示例对应的schema为：
+
+        ```text
+        schema = ['开票日期', '名称', '纳税人识别号', '地址、电话', '开户行及账号', '金额', '税额', '价税合计', 'No', '税率']
+        ```
+
+- 关系抽取
+
+    - Step 1. 标注主体（Subject）及客体（Object）
+
+        <div align="center">
+            <img src=https://user-images.githubusercontent.com/40840292/218974459-4bf989fc-0e40-4dea-b309-346364cca1b5.png height=400 width=1000 />
+        </div>
+
+    - Step 2. 关系连线，箭头方向由主体（Subject）指向客体（Object）
+
+        <div align="center">
+            <img src=https://user-images.githubusercontent.com/40840292/218975474-0cf933bc-7c1e-4e7d-ada5-685ee5265f61.png height=450 width=1000 />
+        </div>
+
+        <div align="center">
+            <img src=https://user-images.githubusercontent.com/40840292/218975743-dc718068-6d58-4352-8eb2-8973549dd971.png height=400 width=1000 />
+        </div>
+
+    - Step 3. 添加对应关系类型标签
+
+        <div align="center">
+            <img src=https://user-images.githubusercontent.com/40840292/218976095-ff5a84e8-302c-4789-98df-139a8cef8d5a.png height=360 width=1000 />
+        </div>
+
+        <div align="center">
+            <img src=https://user-images.githubusercontent.com/40840292/218976368-a4556441-46ca-4372-b68b-e00b45f59260.png height=360 width=1000 />
+        </div>
+
+    - Step 4. 完成标注
+
+        <div align="center">
+            <img src=https://user-images.githubusercontent.com/40840292/218976853-4903f2ec-b669-4c63-8c21-5f7184fc03db.png height=450 width=1000 />
+        </div>
+
+
+    - 该标注示例对应的schema为：
+
+        ```text
+        schema = {
+            '名称及规格': [
+                '金额',
+                '单位',
+                '数量'
+            ]
+        }
+        ```
+
+- 文档分类
+
+    - 标注示例
+
+        <div align="center">
+            <img src=https://user-images.githubusercontent.com/40840292/199879238-b8b41d4a-7e77-47cd-8def-2fc8ba89442f.png height=650 width=800 />
+        </div>
+
+    - 该标注示例对应的schema为：
+
+        ```text
+        schema = '文档类别[发票，报关单]'
+        ```
+
+
+<a name="25"></a>
+
+#### 2.5 数据导出
+
+勾选已标注图片ID，选择导出的文件类型为``JSON``，导出数据：
+
+<div align="center">
+    <img src=https://user-images.githubusercontent.com/40840292/199890897-b33ede99-97d8-4d44-877a-2518a87f8b67.png height=200 width=1200 />
+</div>
+
+<a name="26"></a>
+
+#### 2.6 数据转换
+
+将导出的文件重命名为``label_studio.json``后，放入``./document/data``目录下，并将对应的标注图片放入``./document/data/images``目录下（图片的文件名需与上传到label studio时的命名一致）。通过[label_studio.py](./label_studio.py)脚本可转为UIE的数据格式。
+
+- 路径示例
+
+```shell
+./document/data/
+├── images # 图片目录
+│   ├── b0.jpg # 原始图片（文件名需与上传到label studio时的命名一致）
+│   └── b1.jpg
+└── label_studio.json # 从label studio导出的标注文件
+```
+
+- 抽取式任务
+
+```shell
+python label_studio.py \
+    --label_studio_file ./document/data/label_studio.json \
+    --save_dir ./document/data \
+    --splits 0.8 0.1 0.1\
+    --task_type ext
+```
+
+- 文档分类任务
+
+```shell
+python label_studio.py \
+    --label_studio_file ./document/data/label_studio.json \
+    --save_dir ./document/data \
+    --splits 0.8 0.1 0.1 \
+    --task_type cls \
+    --prompt_prefix "文档类别" \
+    --options "发票" "报关单"
+```
+
+<a name="27"></a>
+
+#### 2.7 更多配置
+
+- ``label_studio_file``: 从label studio导出的数据标注文件。
+- ``save_dir``: 训练数据的保存目录，默认存储在``data``目录下。
+- ``negative_ratio``: 最大负例比例，该参数只对抽取类型任务有效，适当构造负例可提升模型效果。负例数量和实际的标签数量有关，最大负例数量 = negative_ratio * 正例数量。该参数只对训练集有效，默认为5。为了保证评估指标的准确性，验证集和测试集默认构造全负例。
+- ``splits``: 划分数据集时训练集、验证集所占的比例。默认为[0.8, 0.1, 0.1]表示按照``8:1:1``的比例将数据划分为训练集、验证集和测试集。
+- ``task_type``: 选择任务类型，可选有抽取和分类两种类型的任务。
+- ``options``: 指定分类任务的类别标签，该参数只对分类类型任务有效。默认为["正向", "负向"]。
+- ``prompt_prefix``: 声明分类任务的prompt前缀信息，该参数只对分类类型任务有效。默认为"情感倾向"。
+- ``is_shuffle``: 是否对数据集进行随机打散，默认为True。
+- ``seed``: 随机种子，默认为1000.
+- ``separator``: 实体类别/评价维度与分类标签的分隔符，该参数只对实体/评价维度分类任务有效。默认为"##"。
+- ``schema_lang``：选择schema的语言，将会应该训练数据prompt的构造方式，可选有`ch`和`en`。默认为`ch`。
+- ``ocr_lang``：选择OCR的语言，可选有`ch`和`en`。默认为`ch`。
+- ``layout_analysis``：是否使用PPStructure对文档进行布局分析，该参数只对文档类型标注任务有效。默认为False。
+
+备注：
+- 默认情况下 [label_studio.py](./label_studio.py) 脚本会按照比例将数据划分为 train/dev/test 数据集
+- 每次执行 [label_studio.py](./label_studio.py) 脚本，将会覆盖已有的同名数据文件
+- 在模型训练阶段我们推荐构造一些负例以提升模型效果，在数据转换阶段我们内置了这一功能。可通过`negative_ratio`控制自动构造的负样本比例；负样本数量 = negative_ratio * 正样本数量。
+- 对于从label_studio导出的文件，默认文件中的每条数据都是经过人工正确标注的。
+
+
+## References
+- **[Label Studio](https://labelstud.io/)**

applications/information_extraction/label_studio_doc_en.md

@@ -0,0 +1,273 @@
+# Label Studio User Guide - Document Information Extraction
+
+  **Table of contents**
+
+- [1. Installation](#1)
+- [2. Document Extraction Task Annotation](#2)
+     - [2.1 Project Creation](#21)
+     - [2.2 Data Upload](#22)
+     - [2.3 Label Construction](#23)
+     - [2.4 Task Annotation](#24)
+     - [2.5 Data Export](#25)
+     - [2.6 Data Conversion](#26)
+     - [2.7 More Configuration](#27)
+
+<a name="1"></a>
+
+## 1. Installation
+
+**Environmental configuration used in the following annotation examples:**
+
+- Python 3.8+
+- label-studio == 1.6.0
+- paddleocr >= 2.6.0.1
+
+Use pip to install label-studio in the terminal:
+
+```shell
+pip install label-studio==1.6.0
+```
+
+Once the installation is complete, run the following command line:
+
+```shell
+label-studio start
+```
+
+Open [http://localhost:8080/](http://127.0.0.1:8080/) in the browser, enter the user name and password to log in, and start using label-studio for labeling.
+
+<a name="2"></a>
+
+## 2. Document Extraction Task Annotation
+
+<a name="21"></a>
+
+#### 2.1 Project Creation
+
+Click Create to start creating a new project, fill in the project name, description, and select ``Object Detection with Bounding Boxes``.
+
+- Fill in the project name, description
+
+<div align="center">
+     <img src=https://user-images.githubusercontent.com/40840292/199445809-1206f887-2782-459e-9001-fbd790d59a5e.png height=300 width=1200 />
+</div>
+
+- For **Named Entity Recognition, Relation Extraction** tasks please select ``Object Detection with Bounding Boxes`
+
+<div align="center">
+     <img src=https://user-images.githubusercontent.com/40840292/199660090-d84901dd-001d-4620-bffa-0101a4ecd6e5.png height=400 width=1200 />
+</div>
+
+- For **Document Classification** task please select ``Image Classification`
+
+<div align="center">
+     <img src=https://user-images.githubusercontent.com/40840292/199729973-53a994d8-da71-4ab9-84f5-83297e19a7a1.png height=400 width=1200 />
+</div>
+
+- Define labels
+
+<div align="center">
+     <img src=https://user-images.githubusercontent.com/40840292/199450930-4c0cd189-6085-465a-aca0-6ba6f52a0c0d.png height=600 width=1200 />
+</div>
+
+The figure shows the construction of Span entity type tags. For the construction of other types of tags, please refer to [2.3 Label Construction](#23)
+
+<a name="22"></a>
+
+#### 2.2 Data upload
+
+First upload the picture from a local or HTTP link, and then choose to import this project.
+
+<div align="center">
+     <img src=https://user-images.githubusercontent.com/40840292/199452007-2d45f7ba-c631-46b4-b21f-729a2ed652e9.png height=270 width=1200 />
+</div>
+
+<a name="23"></a>
+
+#### 2.3 Label Construction
+
+- Entity Label
+
+<div align="center">
+     <img src=https://user-images.githubusercontent.com/40840292/199456432-ce601ab0-7d6c-458f-ac46-8839dbc4d013.png height=500 width=1200 />
+</div>
+
+
+- Relation label
+
+<div align="center">
+     <img src=https://user-images.githubusercontent.com/40840292/199877621-f60e00c7-81ae-42e1-b498-8ebc5b5bd0fd.png height=650 width=1200 />
+</div>
+
+Relation XML template:
+
+```xml
+   <Relations>
+     <Relation value="unit"/>
+     <Relation value="Quantity"/>
+     <Relation value="amount"/>
+   </Relations>
+```
+
+- Classification label
+
+<div align="center">
+     <img src=https://user-images.githubusercontent.com/40840292/199891626-cc995783-18d2-41dc-88de-260b979edc56.png height=500 width=1200 />
+</div>
+
+<a name="24"></a>
+
+#### 2.4 Task Annotation
+
+- Entity extraction
+
+     - Callout example:
+
+         <div align="center">
+             <img src=https://user-images.githubusercontent.com/40840292/199879427-82806ffc-dc60-4ec7-bda5-e16419ee9d15.png height=650 width=800 />
+         </div>
+
+     - The schema corresponding to this annotation example is:
+
+         ```text
+         schema = ['开票日期', '名称', '纳税人识别号', '地址、电话', '开户行及账号', '金额', '税额', '价税合计', 'No', '税率']
+         ```
+
+- Relation extraction
+
+     - Step 1. Label the subject and object
+
+        <div align="center">
+            <img src=https://user-images.githubusercontent.com/40840292/218974459-4bf989fc-0e40-4dea-b309-346364cca1b5.png height=400 width=1000 />
+        </div>
+
+     - Step 2. Relation line, the direction of the arrow is from the subject to the object
+
+        <div align="center">
+            <img src=https://user-images.githubusercontent.com/40840292/218975474-0cf933bc-7c1e-4e7d-ada5-685ee5265f61.png height=450 width=1000 />
+        </div>
+
+        <div align="center">
+            <img src=https://user-images.githubusercontent.com/40840292/218975743-dc718068-6d58-4352-8eb2-8973549dd971.png height=400 width=1000 />
+        </div>
+
+     - Step 3. Add corresponding relation label
+
+        <div align="center">
+            <img src=https://user-images.githubusercontent.com/40840292/218976095-ff5a84e8-302c-4789-98df-139a8cef8d5a.png height=360 width=1000 />
+        </div>
+
+        <div align="center">
+            <img src=https://user-images.githubusercontent.com/40840292/218976368-a4556441-46ca-4372-b68b-e00b45f59260.png height=360 width=1000 />
+        </div>
+
+     - Step 4. Finish labeling
+
+        <div align="center">
+            <img src=https://user-images.githubusercontent.com/40840292/218976853-4903f2ec-b669-4c63-8c21-5f7184fc03db.png height=450 width=1000 />
+        </div>
+
+
+     - The schema corresponding to this annotation example is:
+
+        ```text
+        schema = {
+            '名称及规格': [
+                '金额',
+                '单位',
+                '数量'
+            ]
+        }
+        ```
+
+- Document classification
+
+     - Callout example
+
+         <div align="center">
+             <img src=https://user-images.githubusercontent.com/40840292/199879238-b8b41d4a-7e77-47cd-8def-2fc8ba89442f.png height=650 width=800 />
+         </div>
+
+     - The schema corresponding to this annotation example is:
+
+        ```text
+        schema = '文档类别[发票，报关单]'
+        ```
+
+
+<a name="25"></a>
+
+#### 2.5 Data Export
+
+Check the marked image ID, select the exported file type as ``JSON``, and export the data:
+
+<div align="center">
+     <img src=https://user-images.githubusercontent.com/40840292/199890897-b33ede99-97d8-4d44-877a-2518a87f8b67.png height=200 width=1200 />
+</div>
+
+
+<a name="26"></a>
+
+#### 2.6 Data Conversion
+
+After renaming the exported file to ``label_studio.json``, put it into the ``./document/data`` directory, and put the corresponding label image into the ``./document/data/images`` directory (The file name of the picture must be the same as the one uploaded to label studio). Through the [label_studio.py](./label_studio.py) script, it can be converted to the data format of UIE.
+
+- Path example
+
+```shell
+./document/data/
+├── images # image directory
+│ ├── b0.jpg # Original picture (the file name must be the same as the one uploaded to label studio)
+│ └── b1.jpg
+└── label_studio.json # Annotation file exported from label studio
+```
+
+- Extraction task
+
+```shell
+python label_studio.py \
+     --label_studio_file ./document/data/label_studio.json \
+     --save_dir ./document/data \
+     --splits 0.8 0.1 0.1 \
+     --task_type ext
+```
+
+- Document classification tasks
+
+```shell
+python label_studio.py \
+     --label_studio_file ./document/data/label_studio.json \
+     --save_dir ./document/data \
+     --splits 0.8 0.1 0.1 \
+     --task_type cls \
+     --prompt_prefix "document category" \
+     --options "invoice" "customs declaration"
+```
+
+<a name="27"></a>
+
+#### 2.7 More Configuration
+
+- ``label_studio_file``: Data labeling file exported from label studio.
+- ``save_dir``: The storage directory of the training data, which is stored in the ``data`` directory by default.
+- ``negative_ratio``: The maximum negative ratio. This parameter is only valid for extraction tasks. Properly constructing negative examples can improve the model effect. The number of negative examples is related to the actual number of labels, the maximum number of negative examples = negative_ratio * number of positive examples. This parameter is only valid for the training set, and the default is 5. In order to ensure the accuracy of the evaluation indicators, the verification set and test set are constructed with all negative examples by default.
+- ``splits``: The proportion of training set and validation set when dividing the data set. The default is [0.8, 0.1, 0.1], which means that the data is divided into training set, verification set and test set according to the ratio of ``8:1:1``.
+- ``task_type``: Select the task type, there are two types of tasks: extraction and classification.
+- ``options``: Specify the category label of the classification task, this parameter is only valid for the classification type task. Defaults to ["positive", "negative"].
+- ``prompt_prefix``: Declare the prompt prefix information of the classification task, this parameter is only valid for the classification type task. Defaults to "Sentimental Tendency".
+- ``is_shuffle``: Whether to randomly shuffle the data set, the default is True.
+- ``seed``: random seed, default is 1000.
+- ``separator``: The separator between entity category/evaluation dimension and classification label. This parameter is only valid for entity/evaluation dimension classification tasks. The default is"##".
+- ``schema_lang``: Select the language of the schema, which will be the construction method of the training data prompt, optional `ch` and `en`. Defaults to `ch`.
+- ``ocr_lang``: Select the language for OCR, optional `ch` and `en`. Defaults to `ch`.
+- ``layout_analysis``: Whether to use PPStructure to analyze the layout of the document. This parameter is only valid for document type labeling tasks. The default is False.
+
+Note:
+- By default the [label_studio.py](./label_studio.py) script will divide the data proportionally into train/dev/test datasets
+- Each time the [label_studio.py](./label_studio.py) script is executed, the existing data file with the same name will be overwritten
+- In the model training phase, we recommend constructing some negative examples to improve the model performance, and we have built-in this function in the data conversion phase. The proportion of automatically constructed negative samples can be controlled by `negative_ratio`; the number of negative samples = negative_ratio * the number of positive samples.
+- For files exported from label_studio, each piece of data in the default file is correctly labeled manually.
+
+
+## References
+- **[Label Studio](https://labelstud.io/)**

applications/information_extraction/label_studio_text.md

@@ -0,0 +1,287 @@
+# 文本抽取任务Label Studio使用指南
+
+ **目录**
+
+- [1. 安装](#1)
+- [2. 文本抽取任务标注](#2)
+    - [2.1 项目创建](#21)
+    - [2.2 数据上传](#22)
+    - [2.3 标签构建](#23)
+    - [2.4 任务标注](#24)
+    - [2.5 数据导出](#25)
+    - [2.6 数据转换](#26)
+    - [2.7 更多配置](#27)
+
+<a name="1"></a>
+
+## 1. 安装
+**以下标注示例用到的环境配置：**
+
+- Python 3.8+
+- label-studio == 1.6.0
+- paddleocr >= 2.6.0.1
+
+在终端(terminal)使用pip安装label-studio：
+
+```shell
+pip install label-studio==1.6.0
+```
+
+安装完成后，运行以下命令行：
+```shell
+label-studio start
+```
+
+在浏览器打开[http://localhost:8080/](http://127.0.0.1:8080/)，输入用户名和密码登录，开始使用label-studio进行标注。
+
+<a name="2"></a>
+
+## 2. 文本抽取任务标注
+
+<a name="21"></a>
+
+#### 2.1 项目创建
+
+点击创建（Create）开始创建一个新的项目，填写项目名称、描述，然后选择``Object Detection with Bounding Boxes``。
+
+- 填写项目名称、描述
+
+<div align="center">
+    <img src=https://user-images.githubusercontent.com/40840292/199661377-d9664165-61aa-4462-927d-225118b8535b.png height=230 width=1200 />
+</div>
+
+- **命名实体识别、关系抽取、事件抽取、实体/评价维度分类**任务选择``Relation Extraction`。
+
+<div align="center">
+    <img src=https://user-images.githubusercontent.com/40840292/199661638-48a870eb-a1df-4db5-82b9-bc8e985f5190.png height=350 width=1200 />
+</div>
+
+- **文本分类、句子级情感倾向分类**任务选择``Text Classification``。
+
+<div align="center">
+    <img src=https://user-images.githubusercontent.com/40840292/212617773-34534e68-4544-4b24-8f39-ae7f9573d397.png height=420 width=1200 />
+</div>
+
+- 添加标签(也可跳过后续在Setting/Labeling Interface中配置)
+
+<div align="center">
+    <img src=https://user-images.githubusercontent.com/40840292/199662737-ed996a2c-7a24-4077-8a36-239c4bfb0a16.png height=380 width=1200 />
+</div>
+
+图中展示了实体类型标签的构建，其他类型标签的构建可参考[2.3标签构建](#23)
+
+<a name="22"></a>
+
+#### 2.2 数据上传
+
+先从本地上传txt格式文件，选择``List of tasks``，然后选择导入本项目。
+
+<div align="center">
+    <img src=https://user-images.githubusercontent.com/40840292/199667670-1b8f6755-b41f-41c4-8afc-06bb051690b6.png height=210 width=1200 />
+</div>
+
+<a name="23"></a>
+
+#### 2.3 标签构建
+
+- Span类型标签
+
+<div align="center">
+    <img src=https://user-images.githubusercontent.com/40840292/199667941-04e300c5-3cd7-4b8e-aaf5-561415414891.png height=480 width=1200 />
+</div>
+
+- Relation类型标签
+
+<div align="center">
+    <img src=https://user-images.githubusercontent.com/40840292/199725229-f5e998bf-367c-4449-b83a-c799f1e3de00.png height=620 width=1200 />
+</div>
+
+Relation XML模板：
+
+```xml
+  <Relations>
+    <Relation value="歌手"/>
+    <Relation value="发行时间"/>
+    <Relation value="所属专辑"/>
+  </Relations>
+```
+
+- 分类类别标签
+
+<div align="center">
+    <img src=https://user-images.githubusercontent.com/40840292/199724082-ee82dceb-dab0-496d-a930-a8ecb284d8b2.png height=370 width=1200 />
+</div>
+
+
+<a name="24"></a>
+
+#### 2.4 任务标注
+
+- 实体抽取
+
+标注示例：
+
+<div align="center">
+    <img src=https://user-images.githubusercontent.com/40840292/199879957-aeec9d17-d342-4ea0-a840-457b49f6066e.png height=140 width=1000 />
+</div>
+
+该标注示例对应的schema为：
+
+```text
+schema = [
+    '时间',
+    '选手',
+    '赛事名称',
+    '得分'
+]
+```
+
+- 关系抽取
+
+<div align="center">
+    <img src=https://user-images.githubusercontent.com/40840292/199879866-03c1ecac-1828-4f35-af70-9ae61701c303.png height=230 width=1200 />
+</div>
+
+对于关系抽取，其P的类型设置十分重要，需要遵循以下原则
+
+“{S}的{P}为{O}”需要能够构成语义合理的短语。比如对于三元组(S, 父子, O)，关系类别为父子是没有问题的。但按照UIE当前关系类型prompt的构造方式，“S的父子为O”这个表达不是很通顺，因此P改成孩子更好，即“S的孩子为O”。**合理的P类型设置，将显著提升零样本效果**。
+
+该标注示例对应的schema为：
+
+```text
+schema = {
+    '作品名': [
+        '歌手',
+        '发行时间',
+        '所属专辑'
+    ]
+}
+```
+
+- 事件抽取
+
+<div align="center">
+    <img src=https://user-images.githubusercontent.com/40840292/199879776-75abbade-9bea-44dc-ac36-322fecdc03e0.png height=220 width=1200 />
+</div>
+
+该标注示例对应的schema为：
+
+```text
+schema = {
+    '地震触发词': [
+        '时间',
+        '震级'
+    ]
+}
+```
+
+- 句子级分类
+
+<div align="center">
+    <img src=https://user-images.githubusercontent.com/40840292/199879672-c3f286fe-a217-4888-950f-d4ee45b19f5a.png height=210 width=1000 />
+</div>
+
+
+该标注示例对应的schema为：
+
+```text
+schema = '情感倾向[正向，负向]'
+```
+
+- 实体/评价维度分类
+
+<div align="center">
+    <img src=https://user-images.githubusercontent.com/40840292/199879586-8c6e4826-a3b0-49e0-9920-98ca062dccff.png height=240 width=1200 />
+</div>
+
+该标注示例对应的schema为：
+
+```text
+schema = {
+    '评价维度': [
+        '观点词',
+        '情感倾向[正向，负向]'
+    ]
+}
+```
+
+<a name="25"></a>
+
+#### 2.5 数据导出
+
+勾选已标注文本ID，选择导出的文件类型为``JSON``，导出数据：
+
+<div align="center">
+    <img src=https://user-images.githubusercontent.com/40840292/199891344-023736e2-6f9d-454b-b72a-dec6689f8436.png height=180 width=1200 />
+</div>
+
+<a name="26"></a>
+
+#### 2.6 数据转换
+
+将导出的文件重命名为``label_studio.json``后，放入``./data``目录下。通过[label_studio.py](./label_studio.py)脚本可转为UIE的数据格式。
+
+- 抽取式任务
+
+```shell
+python label_studio.py \
+    --label_studio_file ./data/label_studio.json \
+    --save_dir ./data \
+    --splits 0.8 0.1 0.1 \
+    --task_type ext
+```
+
+- 句子级分类任务
+
+在数据转换阶段，我们会自动构造用于模型训练的prompt信息。例如句子级情感分类中，prompt为``情感倾向[正向,负向]``，可以通过`prompt_prefix`和`options`参数进行配置。
+
+```shell
+python label_studio.py \
+    --label_studio_file ./data/label_studio.json \
+    --task_type cls \
+    --save_dir ./data \
+    --splits 0.8 0.1 0.1 \
+    --prompt_prefix "情感倾向" \
+    --options "正向" "负向"
+```
+
+- 实体/评价维度分类任务
+
+在数据转换阶段，我们会自动构造用于模型训练的prompt信息。例如评价维度情感分类中，prompt为``XXX的情感倾向[正向,负向]``，可以通过`prompt_prefix`和`options`参数进行声明。
+
+```shell
+python label_studio.py \
+    --label_studio_file ./data/label_studio.json \
+    --task_type ext \
+    --save_dir ./data \
+    --splits 0.8 0.1 0.1 \
+    --prompt_prefix "情感倾向" \
+    --options "正向" "负向" \
+    --separator "##"
+```
+
+<a name="27"></a>
+
+#### 2.7 更多配置
+
+- ``label_studio_file``: 从label studio导出的数据标注文件。
+- ``save_dir``: 训练数据的保存目录，默认存储在``data``目录下。
+- ``negative_ratio``: 最大负例比例，该参数只对抽取类型任务有效，适当构造负例可提升模型效果。负例数量和实际的标签数量有关，最大负例数量 = negative_ratio * 正例数量。该参数只对训练集有效，默认为5。为了保证评估指标的准确性，验证集和测试集默认构造全负例。
+- ``splits``: 划分数据集时训练集、验证集所占的比例。默认为[0.8, 0.1, 0.1]表示按照``8:1:1``的比例将数据划分为训练集、验证集和测试集。
+- ``task_type``: 选择任务类型，可选有抽取和分类两种类型的任务。
+- ``options``: 指定分类任务的类别标签，该参数只对分类类型任务有效。默认为["正向", "负向"]。
+- ``prompt_prefix``: 声明分类任务的prompt前缀信息，该参数只对分类类型任务有效。默认为"情感倾向"。
+- ``is_shuffle``: 是否对数据集进行随机打散，默认为True。
+- ``seed``: 随机种子，默认为1000.
+- ``schema_lang``：选择schema的语言，将会应该训练数据prompt的构造方式，可选有`ch`和`en`。默认为`ch`。
+- ``separator``: 实体类别/评价维度与分类标签的分隔符，该参数只对实体/评价维度分类任务有效。默认为"##"。
+
+备注：
+- 默认情况下 [label_studio.py](./label_studio.py) 脚本会按照比例将数据划分为 train/dev/test 数据集
+- 每次执行 [label_studio.py](./label_studio.py) 脚本，将会覆盖已有的同名数据文件
+- 在模型训练阶段我们推荐构造一些负例以提升模型效果，在数据转换阶段我们内置了这一功能。可通过`negative_ratio`控制自动构造的负样本比例；负样本数量 = negative_ratio * 正样本数量。
+- 对于从label_studio导出的文件，默认文件中的每条数据都是经过人工正确标注的。
+
+
+## References
+- **[Label Studio](https://labelstud.io/)**

applications/information_extraction/label_studio_text_en.md

@@ -0,0 +1,288 @@
+# Label Studio User Guide - Text Information Extraction
+
+**Table of contents**
+
+- [1. Installation](#1)
+- [2. Text Extraction Task Annotation](#2)
+     - [2.1 Project Creation](#21)
+     - [2.2 Data Upload](#22)
+     - [2.3 Label Construction](#23)
+     - [2.4 Task Annotation](#24)
+     - [2.5 Data Export](#25)
+     - [2.6 Data Conversion](#26)
+     - [2.7 More Configuration](#27)
+
+<a name="1"></a>
+
+## 1. Installation
+
+**Environmental configuration used in the following annotation examples:**
+
+- Python 3.8+
+- label-studio == 1.6.0
+- paddleocr >= 2.6.0.1
+
+Use pip to install label-studio in the terminal:
+
+```shell
+pip install label-studio==1.6.0
+```
+
+Once the installation is complete, run the following command line:
+```shell
+label-studio start
+```
+
+Open [http://localhost:8080/](http://127.0.0.1:8080/) in the browser, enter the user name and password to log in, and start using label-studio for labeling.
+
+<a name="2"></a>
+
+## 2. Text extraction task annotation
+
+<a name="21"></a>
+
+#### 2.1 Project Creation
+
+Click Create to start creating a new project, fill in the project name, description, and select ``Object Detection with Bounding Boxes``.
+
+- Fill in the project name, description
+
+<div align="center">
+     <img src=https://user-images.githubusercontent.com/40840292/199661377-d9664165-61aa-4462-927d-225118b8535b.png height=230 width=1200 />
+</div>
+
+- For **Named Entity Recognition, Relation Extraction, Event Extraction, Opinion Extraction** tasks please select ``Relation Extraction`.
+
+<div align="center">
+     <img src=https://user-images.githubusercontent.com/40840292/199661638-48a870eb-a1df-4db5-82b9-bc8e985f5190.png height=350 width=1200 />
+</div>
+
+- For **Text classification, Sentence-level sentiment classification** tasks please select ``Text Classification``.
+
+<div align="center">
+     <img src=https://user-images.githubusercontent.com/40840292/212617773-34534e68-4544-4b24-8f39-ae7f9573d397.png height=420 width=1200 />
+</div>
+
+- Define labels
+
+<div align="center">
+     <img src=https://user-images.githubusercontent.com/40840292/199662737-ed996a2c-7a24-4077-8a36-239c4bfb0a16.png height=380 width=1200 />
+</div>
+
+The figure shows the construction of entity type tags, and the construction of other types of tags can refer to [2.3 Label Construction](#23)
+
+<a name="22"></a>
+
+#### 2.2 Data upload
+
+First upload the txt format file locally, select ``List of tasks``, and then choose to import this project.
+
+<div align="center">
+     <img src=https://user-images.githubusercontent.com/40840292/199667670-1b8f6755-b41f-41c4-8afc-06bb051690b6.png height=210 width=1200 />
+</div>
+
+<a name="23"></a>
+
+#### 2.3 Label construction
+
+- Entity label
+
+<div align="center">
+     <img src=https://user-images.githubusercontent.com/40840292/199667941-04e300c5-3cd7-4b8e-aaf5-561415414891.png height=480 width=1200 />
+</div>
+
+- Relation label
+
+<div align="center">
+     <img src=https://user-images.githubusercontent.com/40840292/199725229-f5e998bf-367c-4449-b83a-c799f1e3de00.png height=620 width=1200 />
+</div>
+
+Relation XML template:
+
+```xml
+   <Relations>
+     <Relation value="Singer"/>
+     <Relation value="Published"/>
+     <Relation value="Album"/>
+   </Relations>
+```
+
+- Classification label
+
+<div align="center">
+     <img src=https://user-images.githubusercontent.com/40840292/199724082-ee82dceb-dab0-496d-a930-a8ecb284d8b2.png height=370 width=1200 />
+</div>
+
+
+<a name="24"></a>
+
+#### 2.4 Task annotation
+
+- Entity extraction
+
+Callout example:
+
+<div align="center">
+     <img src=https://user-images.githubusercontent.com/40840292/199879957-aeec9d17-d342-4ea0-a840-457b49f6066e.png height=140 width=1000 />
+</div>
+
+The schema corresponding to this annotation example is:
+
+```text
+schema = [
+    '时间',
+    '选手',
+    '赛事名称',
+    '得分'
+]
+```
+
+- Relation extraction
+
+<div align="center">
+     <img src=https://user-images.githubusercontent.com/40840292/199879866-03c1ecac-1828-4f35-af70-9ae61701c303.png height=230 width=1200 />
+</div>
+
+For relation extraction, the type setting of P is very important, and the following principles need to be followed
+
+"{P} of {S} is {O}" needs to be able to form a semantically reasonable phrase. For example, for a triple (S, father and son, O), there is no problem with the relation category being father and son. However, according to the current structure of the UIE relation type prompt, the expression "the father and son of S is O" is not very smooth, so it is better to change P to child, that is, "child of S is O". **A reasonable P type setting will significantly improve the zero-shot performance**.
+
+The schema corresponding to this annotation example is:
+
+```text
+schema = {
+    '作品名': [
+        '歌手',
+        '发行时间',
+        '所属专辑'
+    ]
+}
+```
+
+- Event extraction
+
+<div align="center">
+     <img src=https://user-images.githubusercontent.com/40840292/199879776-75abbade-9bea-44dc-ac36-322fecdc03e0.png height=220 width=1200 />
+</div>
+
+The schema corresponding to this annotation example is:
+
+```text
+schema = {
+    '地震触发词': [
+        '时间',
+        '震级'
+    ]
+}
+```
+
+- Sentence level classification
+
+<div align="center">
+     <img src=https://user-images.githubusercontent.com/40840292/199879672-c3f286fe-a217-4888-950f-d4ee45b19f5a.png height=210 width=1000 />
+</div>
+
+
+The schema corresponding to this annotation example is:
+
+```text
+schema = '情感倾向[正向，负向]'
+```
+
+- Opinion Extraction
+
+<div align="center">
+     <img src=https://user-images.githubusercontent.com/40840292/199879586-8c6e4826-a3b0-49e0-9920-98ca062dccff.png height=240 width=1200 />
+</div>
+
+The schema corresponding to this annotation example is:
+
+```text
+schema = {
+    '评价维度': [
+        '观点词',
+        '情感倾向[正向，负向]'
+    ]
+}
+```
+
+<a name="25"></a>
+
+#### 2.5 Data Export
+
+Check the marked text ID, select the exported file type as ``JSON``, and export the data:
+
+<div align="center">
+     <img src=https://user-images.githubusercontent.com/40840292/199891344-023736e2-6f9d-454b-b72a-dec6689f8436.png height=180 width=1200 />
+</div>
+
+<a name="26"></a>
+
+#### 2.6 Data conversion
+
+Rename the exported file to ``label_studio.json`` and put it in the ``./data`` directory. Through the [label_studio.py](./label_studio.py) script, it can be converted to the data format of UIE.
+
+- Extraction task
+
+```shell
+python label_studio.py\
+     --label_studio_file ./data/label_studio.json \
+     --save_dir ./data \
+     --splits 0.8 0.1 0.1 \
+     --task_type ext
+```
+
+- Sentence-level classification tasks
+
+In the data conversion stage, we will automatically construct prompt information for model training. For example, in sentence-level sentiment classification, the prompt is ``Sentiment Classification [positive, negative]``, which can be configured through `prompt_prefix` and `options` parameters.
+
+```shell
+python label_studio.py\
+     --label_studio_file ./data/label_studio.json \
+     --task_type cls \
+     --save_dir ./data \
+     --splits 0.8 0.1 0.1 \
+     --prompt_prefix "Sentiment Classification" \
+     --options "positive" "negative"
+```
+
+- Opinion Extraction
+
+In the data conversion stage, we will automatically construct prompt information for model training. For example, in the emotional classification of the evaluation dimension, the prompt is ``Sentiment Classification of xxx [positive, negative]``, which can be declared through the `prompt_prefix` and `options` parameters.
+
+```shell
+python label_studio.py\
+     --label_studio_file ./data/label_studio.json \
+     --task_type ext \
+     --save_dir ./data \
+     --splits 0.8 0.1 0.1 \
+     --prompt_prefix "Sentiment Classification" \
+     --options "positive" "negative" \
+     --separator "##"
+```
+
+<a name="27"></a>
+
+#### 2.7 More Configuration
+
+- ``label_studio_file``: Data labeling file exported from label studio.
+- ``save_dir``: The storage directory of the training data, which is stored in the ``data`` directory by default.
+- ``negative_ratio``: The maximum negative ratio. This parameter is only valid for extraction tasks. Properly constructing negative examples can improve the model effect. The number of negative examples is related to the actual number of labels, the maximum number of negative examples = negative_ratio * number of positive examples. This parameter is only valid for the training set, and the default is 5. In order to ensure the accuracy of the evaluation indicators, the verification set and test set are constructed with all negative examples by default.
+- ``splits``: The proportion of training set and validation set when dividing the data set. The default is [0.8, 0.1, 0.1], which means that the data is divided into training set, verification set and test set according to the ratio of ``8:1:1``.
+- ``task_type``: Select the task type, there are two types of tasks: extraction and classification.
+- ``options``: Specify the category label of the classification task, this parameter is only valid for the classification type task. Defaults to ["positive", "negative"].
+- ``prompt_prefix``: Declare the prompt prefix information of the classification task, this parameter is only valid for the classification type task. Defaults to "Sentimental Tendency".
+- ``is_shuffle``: Whether to randomly shuffle the data set, the default is True.
+- ``seed``: random seed, default is 1000.
+- ``schema_lang``: Select the language of the schema, which will be the construction method of the training data prompt, optional `ch` and `en`. Defaults to `ch`.
+- ``separator``: The separator between entity category/evaluation dimension and classification label. This parameter is only valid for entity/evaluation dimension classification tasks. The default is"##".
+
+Note:
+- By default the [label_studio.py](./label_studio.py) script will divide the data proportionally into train/dev/test datasets
+- Each time the [label_studio.py](./label_studio.py) script is executed, the existing data file with the same name will be overwritten
+- In the model training phase, we recommend constructing some negative examples to improve the model performance, and we have built-in this function in the data conversion phase. The proportion of automatically constructed negative samples can be controlled by `negative_ratio`; the number of negative samples = negative_ratio * the number of positive samples.
+- For files exported from label_studio, each piece of data in the default file is correctly labeled manually.
+
+
+## References
+- **[Label Studio](https://labelstud.io/)**

applications/information_extraction/taskflow_doc.md

@@ -0,0 +1,310 @@
+# UIE Taskflow使用指南
+
+**目录**
+- [1. 功能简介](#1)
+- [2. 文档信息抽取](#2)
+  - [2.1 实体抽取](#21)
+  - [2.2 关系抽取](#22)
+  - [2.3 跨任务使用](#23)
+  - [2.4 输入说明](#24)
+  - [2.5 使用技巧](#25)
+  - [2.6 结果可视化](#26)
+  - [2.7 更多配置](#27)
+
+<a name="1"></a>
+
+## 1. 功能简介
+
+```paddlenlp.Taskflow```提供文本及文档的通用信息抽取、评价观点抽取等能力，可抽取多种类型的信息，包括但不限于命名实体识别（如人名、地名、机构名等）、关系（如电影的导演、歌曲的发行时间等）、事件（如某路口发生车祸、某地发生地震等）、以及评价维度、观点词、情感倾向等信息。用户可以使用自然语言自定义抽取目标，无需训练即可统一抽取输入文本或文档中的对应信息。**实现开箱即用，并满足各类信息抽取需求**
+
+<a name="2"></a>
+
+## 2. 文档信息抽取
+
+本章节主要介绍Taskflow的文档抽取功能，以下示例图片[下载链接](https://bj.bcebos.com/paddlenlp/taskflow/information_extraction/cases.zip)。
+
+<a name="21"></a>
+
+#### 2.1 实体抽取
+
+实体抽取，又称命名实体识别（Named Entity Recognition，简称NER），是指识别文本中具有特定意义的实体。在开放域信息抽取中，抽取的类别没有限制，用户可以自己定义。
+
+- 报关单
+
+<div align="center">
+    <img src=https://user-images.githubusercontent.com/40840292/206112148-82e26dad-4a77-40e3-bc11-f877047aeb87.png height=700 width=450 hspace='10'/>
+</div>
+
+```python
+>>> from pprint import pprint
+>>> from paddlenlp import Taskflow
+>>> schema = ["收发货人", "进口口岸", "进口日期", "运输方式", "征免性质", "境内目的地", "运输工具名称", "包装种类", "件数", "合同协议号"]
+>>> ie = Taskflow("information_extraction", schema=schema, model="uie-x-base")
+>>> pprint(ie({"doc": "./cases/custom.jpeg"}))
+[{'件数': [{'bbox': [[826, 1062, 926, 1121]],
+        'end': 312,
+        'probability': 0.9832498761402597,
+        'start': 308,
+        'text': '1142'}],
+'包装种类': [{'bbox': [[1214, 1066, 1310, 1121]],
+            'end': 314,
+            'probability': 0.9995648138860567,
+            'start': 312,
+            'text': '纸箱'}],
+'合同协议号': [{'bbox': [[151, 1077, 258, 1117]],
+            'end': 319,
+            'probability': 0.9984179437542124,
+            'start': 314,
+            'text': '33035'}],
+'境内目的地': [{'bbox': [[1966, 872, 2095, 923]],
+            'end': 275,
+            'probability': 0.9975541483111243,
+            'start': 272,
+            'text': '上海市'}],
+'征免性质': [{'bbox': [[1583, 770, 1756, 821]],
+            'end': 242,
+            'probability': 0.9950633161231508,
+            'start': 238,
+            'text': '一般征税'}],
+'收发货人': [{'bbox': [[321, 533, 841, 580]],
+            'end': 95,
+            'probability': 0.4772132061042136,
+            'start': 82,
+            'text': '上海新尚实国际贸易有限公司'},
+        {'bbox': [[306, 584, 516, 624]],
+            'end': 150,
+            'probability': 0.33807074572195006,
+            'start': 140,
+            'text': '31222609K9'}],
+'运输工具名称': [{'bbox': [[1306, 672, 1516, 712], [1549, 668, 1645, 712]],
+            'end': 190,
+            'probability': 0.6692050414718089,
+            'start': 174,
+            'text': 'E. R. TIANAN004E'}],
+'运输方式': [{'bbox': [[1070, 664, 1240, 715]],
+            'end': 174,
+            'probability': 0.9994416347044179,
+            'start': 170,
+            'text': '永路运输'}],
+'进口口岸': [{'bbox': [[1070, 566, 1346, 617]],
+            'end': 120,
+            'probability': 0.9945697196994345,
+            'start': 111,
+            'text': '洋山港区-2248'}],
+'进口日期': [{'bbox': [[1726, 569, 1933, 610]],
+            'end': 130,
+            'probability': 0.9804819494073627,
+            'start': 120,
+            'text': '2017-02-24'}]}]
+```
+
+- 证件
+
+
+<div align="center">
+    <img src=https://user-images.githubusercontent.com/40840292/206114081-8c82e2a2-0c88-4ca3-9651-b12c94266be9.png height=400 width=700 hspace='10'/>
+</div>
+
+```python
+>>> from pprint import pprint
+>>> from paddlenlp import Taskflow
+>>> schema = ["Name", "Date of birth", "Issue date"]
+>>> ie = Taskflow("information_extraction", schema=schema, model="uie-x-base", ocr_lang="en", schema_lang="en")
+>>> pprint(ie({"doc": "./cases/license.jpeg"}))
+```
+
+<a name="22"></a>
+
+#### 2.2 关系抽取
+
+关系抽取（Relation Extraction，简称RE），是指从文本中识别实体并抽取实体之间的语义关系，进而获取三元组信息，即<主体，谓语，客体>。
+
+- 表格
+
+<div align="center">
+    <img src=https://user-images.githubusercontent.com/40840292/206115688-30de315a-8fd4-4125-a3c3-8cb05c6e39e5.png height=180 width=600 hspace='10'/>
+</div>
+
+```python
+>>> from pprint import pprint
+>>> from paddlenlp import Taskflow
+>>> schema = {"姓名": ["招聘单位", "报考岗位"]}
+>>>> ie = Taskflow("information_extraction", schema=schema, model="uie-x-base")
+>>> pprint(ie({"doc": "./cases/table.png"}))
+```
+
+<a name="23"></a>
+
+#### 2.3 跨任务使用
+
+- 实体、关系多任务抽取
+
+对文档进行实体+关系抽取，schema构造如下：
+
+```text
+schema = [
+    "Total GBP",
+    "No.",
+    "Date",
+    "Customer No.",
+    "Subtotal without VAT",
+    {
+        "Description": [
+            "Quantity",
+            "Amount"
+        ]
+    }
+]
+```
+
+<div align="center">
+    <img src=https://user-images.githubusercontent.com/40840292/206120861-13b475dc-9a78-43bc-9dec-91f331db2ddf.png height=400 width=650 hspace='10'/>
+</div>
+
+```python
+>>> from pprint import pprint
+>>> from paddlenlp import Taskflow
+
+>>> schema = ["Total GBP", "No.", "Date", "Customer No.", "Subtotal without VAT", {"Description": ["Quantity", "Amount"]}]
+>>> ie = Taskflow("information_extraction", schema=schema, model="uie-x-base", ocr_lang="en", schema_lang="en")
+>>> pprint(ie({"doc": "./cases/delivery_note.png"}))
+```
+
+<a name="24"></a>
+
+#### 2.4 输入说明
+
+- 输入格式
+
+文档抽取UIE-X支持图片路径、http图片链接、base64的输入形式，支持图片和PDF两种文档格式。文本抽取可以通过`text`指定输入文本。
+
+```python
+[
+    {'text': '2月8日上午北京冬奥会自由式滑雪女子大跳台决赛中中国选手谷爱凌以188.25分获得金牌！'},
+    {'doc': './cases/custom.jpg'},
+    {'doc': 'https://user-images.githubusercontent.com/40840292/203457719-84a70241-607e-4bb1-ab4c-3d9beee9e254.jpeg'}
+]
+```
+
+**NOTE**: 多页PDF输入目前只抽取第一页的结果，UIE-X比较适合单证文档（如票据、单据等）的信息提取，目前还不适合过长或多页的文档。
+
+- 使用自己的layout / OCR作为输入
+
+```python
+layout = [
+    ([68.0, 12.0, 167.0, 70.0], '名次'),
+    ([464.0, 13.0, 559.0, 67.0], '球员'),
+    ([833.0, 15.0, 1054.0, 64.0], '总出场时间'),
+    ......
+]
+ie({"doc": doc_path, 'layout': layout})
+```
+
+<a name="25"></a>
+
+#### 2.5 使用技巧
+
+- 使用PP-Structure版面分析功能
+
+OCR中识别出来的文字会按照左上到右下进行排序，对于分栏、表格内有多行文本等情况我们推荐使用版面分析功能``layout_analysis=True``以优化文字排序并增强抽取效果。以下例子仅举例版面分析功能的使用场景，实际场景一般需要标注微调。
+
+<div align="center">
+    <img src=https://user-images.githubusercontent.com/40840292/206139057-aedec98f-683c-4648-999d-81ce5ea04a86.png height=250 width=500 hspace='10'/>
+</div>
+
+```python
+>>> from pprint import pprint
+>>> from paddlenlp import Taskflow
+
+>>> schema = "中标候选人名称"
+>>> ie = Taskflow("information_extraction", schema=schema, model="uie-x-base", layout_analysis=True)
+>>> pprint(ie({"doc": "https://gimg2.baidu.com/image_search/src=http%3A%2F%2Fwww.xuyiwater.com%2Fwp-content%2Fuploads%2F2021%2F06%2F1-4.jpg&refer=http%3A%2F%2Fwww.xuyiwater.com&app=2002&size=f9999,10000&q=a80&n=0&g=0n&fmt=auto?sec=1672994926&t=2a4a3fedf6999a34ccde190f97bcfa47"}))
+```
+
+<div align="center">
+    <img src=https://user-images.githubusercontent.com/40840292/206137978-3a69e7e2-dc2e-4d11-98b7-25911b0375a0.png height=350 width=600 hspace='10'/>
+</div>
+
+```python
+>>> schema = "抗血小板药物的用药指征"
+>>> ie.set_schema(schema)
+>>> pprint(ie({"doc": "./cases/drug.webp"}))
+```
+
+<a name="26"></a>
+
+#### 2.6 结果可视化
+
+- OCR识别结果可视化：
+
+```python
+>>> from paddlenlp.utils.doc_parser import DocParser
+
+>>> doc_parser = DocParser(ocr_lang="en")
+>>> doc_path = "./cases/business_card.png"
+>>> parsed_doc = doc_parser.parse({"doc": doc_path})
+>>> doc_parser.write_image_with_results(
+        doc_path,
+        layout=parsed_doc['layout'],
+        save_path="ocr_result.png")
+```
+
+<div align="center">
+    <img src=https://user-images.githubusercontent.com/40840292/206168103-0a37eab0-bb36-4eec-bd51-b3f85838b40c.png height=350 width=600 hspace='10'/>
+</div>
+
+- 抽取结果可视化：
+
+```python
+>>> from pprint import pprint
+>>> from paddlenlp import Taskflow
+>>> from paddlenlp.utils.doc_parser import DocParser
+
+>>> doc_path = "./cases/business_card.png"
+>>> schema = ["人名", "职位", "号码", "邮箱地址", "网址", "地址", "邮编"]
+>>> ie = Taskflow("information_extraction", schema=schema, model="uie-x-base", ocr_lang="en")
+
+>>> results = ie({"doc": doc_path})
+
+>>> DocParser.write_image_with_results(
+        doc_path,
+        result=results[0],
+        save_path="image_show.png")
+```
+
+<div align="center">
+    <img src=https://user-images.githubusercontent.com/40840292/206168852-c32c34c4-f245-4116-a244-390e55c13383.png height=350 width=600 hspace='10'/>
+</div>
+
+<a name="27"></a>
+
+#### 2.7 更多配置
+
+```python
+>>> from paddlenlp import Taskflow
+
+>>> ie = Taskflow('information_extraction',
+                  schema="",
+                  schema_lang="ch",
+                  ocr_lang="ch",
+                  batch_size=16,
+                  model='uie-x-base',
+                  layout_analysis=False,
+                  position_prob=0.5,
+                  precision='fp32',
+                  use_fast=False)
+```
+
+* `schema`：定义任务抽取目标，可参考开箱即用中不同任务的调用示例进行配置。
+* `schema_lang`：设置schema的语言，默认为`ch`, 可选有`ch`和`en`。因为中英schema的构造有所不同，因此需要指定schema的语言。
+* `ocr_lang`：选择PaddleOCR的语言，`ch`可在中英混合的图片中使用，`en`在英文图片上的效果更好，默认为`ch`。
+* `batch_size`：批处理大小，请结合机器情况进行调整，默认为16。
+* `model`：选择任务使用的模型，默认为`uie-base`，可选有`uie-base`, `uie-medium`, `uie-mini`, `uie-micro`, `uie-nano`和`uie-medical-base`, `uie-base-en`，`uie-x-base`。
+* `layout_analysis`：是否使用PP-Structure对文档进行布局分析以优化布局信息的排序，默认为False。
+* `position_prob`：模型对于span的起始位置/终止位置的结果概率在0~1之间，返回结果去掉小于这个阈值的结果，默认为0.5，span的最终概率输出为起始位置概率和终止位置概率的乘积。
+* `precision`：选择模型精度，默认为`fp32`，可选有`fp16`和`fp32`。`fp16`推理速度更快，支持GPU和NPU硬件环境。如果选择`fp16`，在GPU硬件环境下，请先确保机器正确安装NVIDIA相关驱动和基础软件，**确保CUDA>=11.2，cuDNN>=8.1.1**，初次使用需按照提示安装相关依赖。其次，需要确保GPU设备的CUDA计算能力（CUDA Compute Capability）大于7.0，典型的设备包括V100、T4、A10、A100、GTX 20系列和30系列显卡等。更多关于CUDA Compute Capability和精度支持情况请参考NVIDIA文档：[GPU硬件与支持精度对照表](https://docs.nvidia.com/deeplearning/tensorrt/archives/tensorrt-840-ea/support-matrix/index.html#hardware-precision-matrix)。
+* `use_fast`: 使用C++实现的高性能分词算子FastTokenizer进行文本预处理加速。需要通过`pip install fast-tokenizer-python`安装FastTokenizer库后方可使用。默认为`False`。更多使用说明可参考[FastTokenizer文档](../../fast_tokenizer)。
+
+## References
+- **[PaddleOCR](https://github.com/PaddlePaddle/PaddleOCR)**
+- **[PP-Structure](https://github.com/PaddlePaddle/PaddleOCR/tree/release/2.6/ppstructure)**

applications/information_extraction/taskflow_doc_en.md

@@ -0,0 +1,305 @@
+# UIE Taskflow User Guide
+
+**Table of contents**
+- [1. Introduction](#1)
+- [2. Document Information Extraction](#2)
+   - [2.1 Entity Extraction](#21)
+   - [2.2 Relation Extraction](#22)
+   - [2.3 Multi-Task Extraction](#23)
+   - [2.4 Input Format](#24)
+   - [2.5 Tips](#25)
+   - [2.6 Visualization](#26)
+   - [2.7 More Configuration](#27)
+
+<a name="1"></a>
+
+## 1. Introduction
+
+```paddlenlp.Taskflow``` provides general information extraction of text and documents, evaluation opinion extraction and other capabilities, and can extract various types of information, including but not limited to named entities (such as person name, place name, organization name, etc.), relations (such as the director of the movie, the release time of the song, etc.), events (such as a car accident at a certain intersection, an earthquake in a certain place, etc.), and information such as product reviews, opinions, and sentiments. Users can use natural language to customize the extraction target, and can uniformly extract the corresponding information in the input text or document without training.
+
+<a name="2"></a>
+
+## 2. Document Information Extraction
+
+This section introduces the document extraction capability of Taskflow with the following example picture [download link](https://bj.bcebos.com/paddlenlp/taskflow/information_extraction/cases.zip).
+
+<a name="21"></a>
+
+#### 2.1 Entity Extraction
+
+Entity extraction, also known as Named Entity Recognition (NER for short), refers to identifying entities with specific meanings in text. UIE adopts the open-domain approach where the entity category is not fixed and the users can define them by through natural language.
+
+- Example: Customs Declaration Form
+
+<div align="center">
+    <img src=https://user-images.githubusercontent.com/40840292/206112148-82e26dad-4a77-40e3-bc11-f877047aeb87.png height=700 width=450 hspace='10'/>
+</div>
+
+```python
+>>> from pprint import pprint
+>>> from paddlenlp import Taskflow
+>>> schema = ["收发货人", "进口口岸", "进口日期", "运输方式", "征免性质", "境内目的地", "运输工具名称", "包装种类", "件数", "合同协议号"]
+>>> ie = Taskflow("information_extraction", schema=schema, model="uie-x-base")
+>>> pprint(ie({"doc": "./cases/custom.jpeg"}))
+[{'件数': [{'bbox': [[826, 1062, 926, 1121]],
+        'end': 312,
+        'probability': 0.9832498761402597,
+        'start': 308,
+        'text': '1142'}],
+'包装种类': [{'bbox': [[1214, 1066, 1310, 1121]],
+            'end': 314,
+            'probability': 0.9995648138860567,
+            'start': 312,
+            'text': '纸箱'}],
+'合同协议号': [{'bbox': [[151, 1077, 258, 1117]],
+            'end': 319,
+            'probability': 0.9984179437542124,
+            'start': 314,
+            'text': '33035'}],
+'境内目的地': [{'bbox': [[1966, 872, 2095, 923]],
+            'end': 275,
+            'probability': 0.9975541483111243,
+            'start': 272,
+            'text': '上海市'}],
+'征免性质': [{'bbox': [[1583, 770, 1756, 821]],
+            'end': 242,
+            'probability': 0.9950633161231508,
+            'start': 238,
+            'text': '一般征税'}],
+'收发货人': [{'bbox': [[321, 533, 841, 580]],
+            'end': 95,
+            'probability': 0.4772132061042136,
+            'start': 82,
+            'text': '上海新尚实国际贸易有限公司'},
+        {'bbox': [[306, 584, 516, 624]],
+            'end': 150,
+            'probability': 0.33807074572195006,
+            'start': 140,
+            'text': '31222609K9'}],
+'运输工具名称': [{'bbox': [[1306, 672, 1516, 712], [1549, 668, 1645, 712]],
+            'end': 190,
+            'probability': 0.6692050414718089,
+            'start': 174,
+            'text': 'E. R. TIANAN004E'}],
+'运输方式': [{'bbox': [[1070, 664, 1240, 715]],
+            'end': 174,
+            'probability': 0.9994416347044179,
+            'start': 170,
+            'text': '永路运输'}],
+'进口口岸': [{'bbox': [[1070, 566, 1346, 617]],
+            'end': 120,
+            'probability': 0.9945697196994345,
+            'start': 111,
+            'text': '洋山港区-2248'}],
+'进口日期': [{'bbox': [[1726, 569, 1933, 610]],
+            'end': 130,
+            'probability': 0.9804819494073627,
+            'start': 120,
+            'text': '2017-02-24'}]}]
+```
+
+- Example: Driver's License
+
+<div align="center">
+    <img src=https://user-images.githubusercontent.com/40840292/206114081-8c82e2a2-0c88-4ca3-9651-b12c94266be9.png height=400 width=700 hspace='10'/>
+</div>
+
+```python
+>>> from pprint import pprint
+>>> from paddlenlp import Taskflow
+>>> schema = ["Name", "Date of birth", "Issue date"]
+>>> ie = Taskflow("information_extraction", schema=schema, model="uie-x-base", ocr_lang="en", schema_lang="en")
+>>> pprint(ie({"doc": "./cases/license.jpeg"}))
+```
+
+<a name="22"></a>
+
+#### 2.2 Relation Extraction
+
+Relation Extraction refers to identifying entities from text and extracting the semantic relationship between entities, and then obtaining triple information, namely <subject, predicate, object>.
+
+- Example: Extracting relations from a table
+
+<div align="center">
+    <img src=https://user-images.githubusercontent.com/40840292/206115688-30de315a-8fd4-4125-a3c3-8cb05c6e39e5.png height=180 width=600 hspace='10'/>
+</div>
+
+```python
+>>> from pprint import pprint
+>>> from paddlenlp import Taskflow
+>>> schema = {"姓名": ["招聘单位", "报考岗位"]}
+>>>> ie = Taskflow("information_extraction", schema=schema, model="uie-x-base")
+>>> pprint(ie({"doc": "./cases/table.png"}))
+```
+
+<a name="23"></a>
+
+#### 2.3 Multi-Task Extraction
+
+To extract entities and relation from documents simultaneously, you may set the schema structure as following:
+
+```text
+schema = [
+    "Total GBP",
+    "No.",
+    "Date",
+    "Customer No.",
+    "Subtotal without VAT",
+    {
+        "Description": [
+            "Quantity",
+            "Amount"
+        ]
+    }
+]
+```
+
+<div align="center">
+    <img src=https://user-images.githubusercontent.com/40840292/206120861-13b475dc-9a78-43bc-9dec-91f331db2ddf.png height=400 width=650 hspace='10'/>
+</div>
+
+```python
+>>> from pprint import pprint
+>>> from paddlenlp import Taskflow
+
+>>> schema = ["Total GBP", "No.", "Date", "Customer No.", "Subtotal without VAT", {"Description": ["Quantity", "Amount"]}]
+>>> ie = Taskflow("information_extraction", schema=schema, model="uie-x-base", ocr_lang="en", schema_lang="en")
+>>> pprint(ie({"doc": "./cases/delivery_note.png"}))
+```
+
+<a name="24"></a>
+
+#### 2.4 Input Format
+
+For document information extraction, UIE-X supports image paths, http image links, base64 input form, and image and PDF document formats. In the input dict, `text` indicates text input and `doc` refer to the document input.
+
+```python
+[
+    {'text': '2月8日上午北京冬奥会自由式滑雪女子大跳台决赛中中国选手谷爱凌以188.25分获得金牌！'},
+    {'doc': './cases/custom.jpg'},
+    {'doc': 'https://user-images.githubusercontent.com/40840292/203457719-84a70241-607e-4bb1-ab4c-3d9beee9e254.jpeg'}
+]
+```
+
+**NOTE**: Multi-page PDF input currently only extracts the results of the first page. UIE-X is more suitable for information extraction of document documents (such as bills, receipts, etc.), but it is not suitable for documents that are too long or multi-page.
+
+- Using custom OCR input
+
+```python
+layout = [
+    ([68.0, 12.0, 167.0, 70.0], '名次'),
+    ([464.0, 13.0, 559.0, 67.0], '球员'),
+    ([833.0, 15.0, 1054.0, 64.0], '总出场时间'),
+    ......
+]
+ie({"doc": doc_path, 'layout': layout})
+```
+
+<a name="25"></a>
+
+#### 2.5 Tips
+
+- Using PP-Structure layout analysis function
+
+The text recognized in OCR will be sorted from top left to bottom right. For cases such as column division and multiple lines of text in the table, we recommend using the layout analysis function ``layout_analysis=True`` to optimize text sorting and enhance the extraction effect. The following example is only an example of the usage scenario of the layout analysis function, and the actual scenario generally needs to be marked and fine-tuned.
+
+<div align="center">
+    <img src=https://user-images.githubusercontent.com/40840292/206139057-aedec98f-683c-4648-999d-81ce5ea04a86.png height=250 width=500 hspace='10'/>
+</div>
+
+```python
+>>> from pprint import pprint
+>>> from paddlenlp import Taskflow
+
+>>> schema = "中标候选人名称"
+>>> ie = Taskflow("information_extraction", schema=schema, model="uie-x-base", layout_analysis=True)
+>>> pprint(ie({"doc": "https://gimg2.baidu.com/image_search/src=http%3A%2F%2Fwww.xuyiwater.com%2Fwp-content%2Fuploads%2F2021%2F06%2F1-4.jpg&refer=http%3A%2F%2Fwww.xuyiwater.com&app=2002&size=f9999,10000&q=a80&n=0&g=0n&fmt=auto?sec=1672994926&t=2a4a3fedf6999a34ccde190f97bcfa47"}))
+```
+
+<div align="center">
+    <img src=https://user-images.githubusercontent.com/40840292/206137978-3a69e7e2-dc2e-4d11-98b7-25911b0375a0.png height=350 width=600 hspace='10'/>
+</div>
+
+```python
+>>> schema = "抗血小板药物的用药指征"
+>>> ie.set_schema(schema)
+>>> pprint(ie({"doc": "./cases/drug.webp"}))
+```
+
+<a name="26"></a>
+
+#### 2.6 Visualization
+
+- Visualization of OCR recognition results:
+
+```python
+>>> from paddlenlp.utils.doc_parser import DocParser
+
+>>> doc_parser = DocParser(ocr_lang="en")
+>>> doc_path = "./cases/business_card.png"
+>>> parsed_doc = doc_parser.parse({"doc": doc_path})
+>>> doc_parser.write_image_with_results(
+        doc_path,
+        layout=parsed_doc['layout'],
+        save_path="ocr_result.png")
+```
+
+<div align="center">
+    <img src=https://user-images.githubusercontent.com/40840292/206168103-0a37eab0-bb36-4eec-bd51-b3f85838b40c.png height=350 width=600 hspace='10'/>
+</div>
+
+- Visualization of extraction results:
+
+```python
+>>> from pprint import pprint
+>>> from paddlenlp import Taskflow
+>>> from paddlenlp.utils.doc_parser import DocParser
+
+>>> doc_path = "./cases/business_card.png"
+>>> schema = ["人名", "职位", "号码", "邮箱地址", "网址", "地址", "邮编"]
+>>> ie = Taskflow("information_extraction", schema=schema, model="uie-x-base", ocr_lang="en")
+
+>>> results = ie({"doc": doc_path})
+
+>>> DocParser.write_image_with_results(
+        doc_path,
+        result=results[0],
+        save_path="image_show.png")
+```
+
+<div align="center">
+    <img src=https://user-images.githubusercontent.com/40840292/206168852-c32c34c4-f245-4116-a244-390e55c13383.png height=350 width=600 hspace='10'/>
+</div>
+
+<a name="27"></a>
+
+#### 2.7 More Configuration
+
+```python
+>>> from paddlenlp import Taskflow
+
+>>> ie = Taskflow('information_extraction',
+                  schema="",
+                  schema_lang="ch",
+                  ocr_lang="ch",
+                  batch_size=16,
+                  model='uie-x-base',
+                  layout_analysis=False,
+                  position_prob=0.5,
+                  precision='fp32',
+                  use_fast=False)
+```
+
+* `schema`: Define the task extraction target, which can be configured by referring to the calling examples of different tasks in the out-of-the-box.
+* `schema_lang`: Set the language of the schema, the default is `ch`, optional `ch` and `en`. Because the structure of the Chinese and English schemas is different, the language of the schema needs to be specified.
+* `ocr_lang`: Select the language of PaddleOCR, `ch` can be used in mixed Chinese and English images, `en` works better on English images, the default is `ch`.
+* `batch_size`: batch size, please adjust according to the machine situation, the default is 16.
+* `model`: select the model used by the task, the default is `uie-base`, optional `uie-base`, `uie-medium`, `uie-mini`, `uie-micro`, `uie-nano` ` and `uie-medical-base`, `uie-base-en`, `uie-x-base`.
+* `layout_analysis`: Whether to use PP-Structure to analyze the layout of the document to optimize the sorting of layout information, the default is False.
+* `position_prob`: The result probability of the model for the start position/end position of the span is between 0 and 1, and the returned result removes the results less than this threshold, the default is 0.5, and the final probability output of the span is the start position probability and end position The product of the position probabilities.
+* `precision`: select the model precision, the default is `fp32`, optional `fp16` and `fp32`. `fp16` inference is faster, support GPU and NPU hardware. If you choose `fp16` and GPU hardware, please ensure that the machine is correctly installed with NVIDIA-related drivers and basic software. **Ensure that CUDA>=11.2, cuDNN>=8.1.1**. For the first time use, you need to follow the prompts to install the relevant dependencies. Secondly, it is necessary to ensure that the CUDA Compute Capability of the GPU device is greater than 7.0. Typical devices include V100, T4, A10, A100, GTX 20 series and 30 series graphics cards, etc. For more information about CUDA Compute Capability and precision support, please refer to NVIDIA documentation: [GPU Hardware and Supported Precision Comparison Table](https://docs.nvidia.com/deeplearning/tensorrt/archives/tensorrt-840-ea/support-matrix/index.html#hardware-precision-matrix).
+* `use_fast`: Use the high-performance word segmentation operator FastTokenizer implemented in C++ to accelerate text preprocessing. The FastTokenizer library needs to be installed through `pip install fast-tokenizer-python` before it can be used. Defaults to `False`. For more usage instructions, please refer to [FastTokenizer Documentation](../../fast_tokenizer).
+
+## References
+- **[PaddleOCR](https://github.com/PaddlePaddle/PaddleOCR)**
+- **[PP-Structure](https://github.com/PaddlePaddle/PaddleOCR/tree/release/2.6/ppstructure)**

applications/information_extraction/taskflow_text.md

@@ -0,0 +1,502 @@
+# 文本抽取任务UIE Taskflow使用指南
+
+**目录**
+- [1. 功能简介](#1)
+- [2. 应用示例](#2)
+- [3. 文本信息抽取](#3)
+  - [3.1 实体抽取](#31)
+  - [3.2 关系抽取](#32)
+  - [3.3 事件抽取](#33)
+  - [3.4 评论观点抽取](#34)
+  - [3.5 情感分类](#35)
+  - [3.6 跨任务抽取](#36)
+  - [3.7 模型选择](#37)
+  - [3.8 更多配置](#38)
+
+<a name="1"></a>
+
+## 1. 功能简介
+
+```paddlenlp.Taskflow```提供纯文本的通用信息抽取、评价观点抽取等能力，可抽取多种类型的信息，包括但不限于命名实体识别（如人名、地名、机构名等）、关系（如电影的导演、歌曲的发行时间等）、事件（如某路口发生车祸、某地发生地震等）、以及评价维度、观点词、情感倾向等信息。用户可以使用自然语言自定义抽取目标，无需训练即可统一抽取输入文本中的对应信息。**实现开箱即用，并满足各类信息抽取需求**
+
+<a name="2"></a>
+
+## 2. 应用示例
+
+UIE不限定行业领域和抽取目标，以下是一些通过Taskflow实现开箱即用的行业示例：
+
+- 医疗场景-专病结构化
+
+![image](https://user-images.githubusercontent.com/40840292/169017581-93c8ee44-856d-4d17-970c-b6138d10f8bc.png)
+
+- 法律场景-判决书抽取
+
+![image](https://user-images.githubusercontent.com/40840292/169017863-442c50f1-bfd4-47d0-8d95-8b1d53cfba3c.png)
+
+- 金融场景-收入证明、招股书抽取
+
+![image](https://user-images.githubusercontent.com/40840292/169017982-e521ddf6-d233-41f3-974e-6f40f8f2edbc.png)
+
+- 公安场景-事故报告抽取
+
+![image](https://user-images.githubusercontent.com/40840292/169018340-31efc1bf-f54d-43f7-b62a-8f7ce9bf0536.png)
+
+- 旅游场景-宣传册、手册抽取
+
+![image](https://user-images.githubusercontent.com/40840292/169018113-c937eb0b-9fd7-4ecc-8615-bcdde2dac81d.png)
+
+<a name="3"></a>
+
+## 3. 文本信息抽取
+
+<a name="31"></a>
+
+#### 3.1 实体抽取
+
+  实体抽取，又称命名实体识别（Named Entity Recognition，简称NER），是指识别文本中具有特定意义的实体。在开放域信息抽取中，抽取的类别没有限制，用户可以自己定义。
+
+  - 例如抽取的目标实体类型是"时间"、"选手"和"赛事名称", schema构造如下：
+
+    ```text
+    ['时间', '选手', '赛事名称']
+    ```
+
+    调用示例：
+
+    ```python
+    >>> from pprint import pprint
+    >>> from paddlenlp import Taskflow
+
+    >>> schema = ['时间', '选手', '赛事名称'] # Define the schema for entity extraction
+    >>> ie = Taskflow('information_extraction', schema=schema)
+    >>> pprint(ie("2月8日上午北京冬奥会自由式滑雪女子大跳台决赛中中国选手谷爱凌以188.25分获得金牌！")) # Better print results using pprint
+    [{'时间': [{'end': 6,
+              'probability': 0.9857378532924486,
+              'start': 0,
+              'text': '2月8日上午'}],
+      '赛事名称': [{'end': 23,
+                'probability': 0.8503089953268272,
+                'start': 6,
+                'text': '北京冬奥会自由式滑雪女子大跳台决赛'}],
+      '选手': [{'end': 31,
+              'probability': 0.8981548639781138,
+              'start': 28,
+              'text': '谷爱凌'}]}]
+    ```
+
+  - 例如抽取的目标实体类型是"肿瘤的大小"、"肿瘤的个数"、"肝癌级别"和"脉管内癌栓分级", schema构造如下：
+
+    ```text
+    ['肿瘤的大小', '肿瘤的个数', '肝癌级别', '脉管内癌栓分级']
+    ```
+
+    在上例中我们已经实例化了一个`Taskflow`对象，这里可以通过`set_schema`方法重置抽取目标。
+
+    调用示例：
+
+    ```python
+    >>> schema = ['肿瘤的大小', '肿瘤的个数', '肝癌级别', '脉管内癌栓分级']
+    >>> ie.set_schema(schema)
+    >>> pprint(ie("（右肝肿瘤）肝细胞性肝癌（II-III级，梁索型和假腺管型），肿瘤包膜不完整，紧邻肝被膜，侵及周围肝组织，未见脉管内癌栓（MVI分级：M0级）及卫星子灶形成。（肿物1个，大小4.2×4.0×2.8cm）。"))
+    [{'肝癌级别': [{'end': 20,
+                'probability': 0.9243267447402701,
+                'start': 13,
+                'text': 'II-III级'}],
+      '肿瘤的个数': [{'end': 84,
+                'probability': 0.7538413804059623,
+                'start': 82,
+                'text': '1个'}],
+      '肿瘤的大小': [{'end': 100,
+                'probability': 0.8341128043459491,
+                'start': 87,
+                'text': '4.2×4.0×2.8cm'}],
+      '脉管内癌栓分级': [{'end': 70,
+                  'probability': 0.9083292325934664,
+                  'start': 67,
+                  'text': 'M0级'}]}]
+    ```
+
+  - 例如抽取的目标实体类型是"person"和"organization"，schema构造如下：
+
+    ```text
+    ['person', 'organization']
+    ```
+
+    英文模型调用示例：
+
+    ```python
+    >>> from pprint import pprint
+    >>> from paddlenlp import Taskflow
+    >>> schema = ['Person', 'Organization']
+    >>> ie_en = Taskflow('information_extraction', schema=schema, model='uie-base-en')
+    >>> pprint(ie_en('In 1997, Steve was excited to become the CEO of Apple.'))
+    [{'Organization': [{'end': 53,
+                        'probability': 0.9985840259877357,
+                        'start': 48,
+                        'text': 'Apple'}],
+      'Person': [{'end': 14,
+                  'probability': 0.999631971804547,
+                  'start': 9,
+                  'text': 'Steve'}]}]
+    ```
+
+<a name="32"></a>
+
+#### 3.2 关系抽取
+
+  关系抽取（Relation Extraction，简称RE），是指从文本中识别实体并抽取实体之间的语义关系，进而获取三元组信息，即<主体，谓语，客体>。
+
+  - 例如以"竞赛名称"作为抽取主体，抽取关系类型为"主办方"、"承办方"和"已举办次数", schema构造如下：
+
+    ```text
+    {
+      '竞赛名称': [
+        '主办方',
+        '承办方',
+        '已举办次数'
+      ]
+    }
+    ```
+
+    调用示例：
+
+    ```python
+    >>> schema = {'竞赛名称': ['主办方', '承办方', '已举办次数']} # Define the schema for relation extraction
+    >>> ie.set_schema(schema) # Reset schema
+    >>> pprint(ie('2022语言与智能技术竞赛由中国中文信息学会和中国计算机学会联合主办，百度公司、中国中文信息学会评测工作委员会和中国计算机学会自然语言处理专委会承办，已连续举办4届，成为全球最热门的中文NLP赛事之一。'))
+    [{'竞赛名称': [{'end': 13,
+                'probability': 0.7825402622754041,
+                'relations': {'主办方': [{'end': 22,
+                                      'probability': 0.8421710521379353,
+                                      'start': 14,
+                                      'text': '中国中文信息学会'},
+                                      {'end': 30,
+                                      'probability': 0.7580801847701935,
+                                      'start': 23,
+                                      'text': '中国计算机学会'}],
+                              '已举办次数': [{'end': 82,
+                                        'probability': 0.4671295049136148,
+                                        'start': 80,
+                                        'text': '4届'}],
+                              '承办方': [{'end': 39,
+                                      'probability': 0.8292706618236352,
+                                      'start': 35,
+                                      'text': '百度公司'},
+                                      {'end': 72,
+                                      'probability': 0.6193477885474685,
+                                      'start': 56,
+                                      'text': '中国计算机学会自然语言处理专委会'},
+                                      {'end': 55,
+                                      'probability': 0.7000497331473241,
+                                      'start': 40,
+                                      'text': '中国中文信息学会评测工作委员会'}]},
+                'start': 0,
+                'text': '2022语言与智能技术竞赛'}]}]
+    ```
+
+  - 例如以"person"作为抽取主体，抽取关系类型为"Company"和"Position", schema构造如下：
+
+    ```text
+    {
+      'Person': [
+        'Company',
+        'Position'
+      ]
+    }
+    ```
+
+    英文模型调用示例：
+
+    ```python
+    >>> schema = [{'Person': ['Company', 'Position']}]
+    >>> ie_en.set_schema(schema)
+    >>> pprint(ie_en('In 1997, Steve was excited to become the CEO of Apple.'))
+    [{'Person': [{'end': 14,
+                  'probability': 0.999631971804547,
+                  'relations': {'Company': [{'end': 53,
+                                            'probability': 0.9960158209451642,
+                                            'start': 48,
+                                            'text': 'Apple'}],
+                                'Position': [{'end': 44,
+                                              'probability': 0.8871063806420736,
+                                              'start': 41,
+                                              'text': 'CEO'}]},
+                  'start': 9,
+                  'text': 'Steve'}]}]
+    ```
+
+<a name="33"></a>
+
+#### 3.3 事件抽取
+
+  事件抽取 (Event Extraction, 简称EE)，是指从自然语言文本中抽取预定义的事件触发词(Trigger)和事件论元(Argument)，组合为相应的事件结构化信息。
+
+  - 例如抽取的目标是"地震"事件的"地震强度"、"时间"、"震中位置"和"震源深度"这些信息，schema构造如下：
+
+    ```text
+    {
+      '地震触发词': [
+        '地震强度',
+        '时间',
+        '震中位置',
+        '震源深度'
+      ]
+    }
+    ```
+
+    触发词的格式统一为`触发词`或``XX触发词`，`XX`表示具体事件类型，上例中的事件类型是`地震`，则对应触发词为`地震触发词`。
+
+    调用示例：
+
+    ```python
+    >>> schema = {'地震触发词': ['地震强度', '时间', '震中位置', '震源深度']} # Define the schema for event extraction
+    >>> ie.set_schema(schema) # Reset schema
+    >>> ie('中国地震台网正式测定：5月16日06时08分在云南临沧市凤庆县(北纬24.34度，东经99.98度)发生3.5级地震，震源深度10千米。')
+    [{'地震触发词': [{'text': '地震', 'start': 56, 'end': 58, 'probability': 0.9987181623528585, 'relations': {'地震强度': [{'text': '3.5级', 'start': 52, 'end': 56, 'probability': 0.9962985320905915}], '时间': [{'text': '5月16日06时08分', 'start': 11, 'end': 22, 'probability': 0.9882578028575182}], '震中位置': [{'text': '云南临沧市凤庆县(北纬24.34度，东经99.98度)', 'start': 23, 'end': 50, 'probability': 0.8551415716584501}], '震源深度': [{'text': '10千米', 'start': 63, 'end': 67, 'probability': 0.999158304648045}]}}]}]
+    ```
+
+  - 英文模型**暂不支持事件抽取**，如有需要可使用英文事件数据集进行定制。
+
+<a name="34"></a>
+
+#### 3.4 评论观点抽取
+
+  评论观点抽取，是指抽取文本中包含的评价维度、观点词。
+
+  - 例如抽取的目标是文本中包含的评价维度及其对应的观点词和情感倾向，schema构造如下：
+
+    ```text
+    {
+      '评价维度': [
+        '观点词',
+        '情感倾向[正向，负向]'
+      ]
+    }
+    ```
+
+    调用示例：
+
+    ```python
+    >>> schema = {'评价维度': ['观点词', '情感倾向[正向，负向]']} # Define the schema for opinion extraction
+    >>> ie.set_schema(schema) # Reset schema
+    >>> pprint(ie("店面干净，很清静，服务员服务热情，性价比很高，发现收银台有排队")) # Better print results using pprint
+    [{'评价维度': [{'end': 20,
+                'probability': 0.9817040258681473,
+                'relations': {'情感倾向[正向，负向]': [{'probability': 0.9966142505350533,
+                                              'text': '正向'}],
+                              '观点词': [{'end': 22,
+                                      'probability': 0.957396472711558,
+                                      'start': 21,
+                                      'text': '高'}]},
+                'start': 17,
+                'text': '性价比'},
+              {'end': 2,
+                'probability': 0.9696849569741168,
+                'relations': {'情感倾向[正向，负向]': [{'probability': 0.9982153274927796,
+                                              'text': '正向'}],
+                              '观点词': [{'end': 4,
+                                      'probability': 0.9945318044652538,
+                                      'start': 2,
+                                      'text': '干净'}]},
+                'start': 0,
+                'text': '店面'}]}]
+    ```
+
+  - 英文模型schema构造如下：
+
+    ```text
+    {
+      'Aspect': [
+        'Opinion',
+        'Sentiment classification [negative, positive]'
+      ]
+    }
+    ```
+
+    调用示例：
+
+    ```python
+    >>> schema = [{'Aspect': ['Opinion', 'Sentiment classification [negative, positive]']}]
+    >>> ie_en.set_schema(schema)
+    >>> pprint(ie_en("The teacher is very nice."))
+    [{'Aspect': [{'end': 11,
+                  'probability': 0.4301476415932193,
+                  'relations': {'Opinion': [{'end': 24,
+                                            'probability': 0.9072940447883724,
+                                            'start': 15,
+                                            'text': 'very nice'}],
+                                'Sentiment classification [negative, positive]': [{'probability': 0.9998571920670685,
+                                                                                  'text': 'positive'}]},
+                  'start': 4,
+                  'text': 'teacher'}]}]
+    ```
+
+<a name="35"></a>
+
+#### 3.5 情感分类
+
+  - 句子级情感倾向分类，即判断句子的情感倾向是“正向”还是“负向”，schema构造如下：
+
+    ```text
+    '情感倾向[正向，负向]'
+    ```
+
+    调用示例：
+
+    ```python
+    >>> schema = '情感倾向[正向，负向]' # Define the schema for sentence-level sentiment classification
+    >>> ie.set_schema(schema) # Reset schema
+    >>> ie('这个产品用起来真的很流畅，我非常喜欢')
+    [{'情感倾向[正向，负向]': [{'text': '正向', 'probability': 0.9988661643929895}]}]
+    ```
+
+    英文模型schema构造如下：
+
+    ```text
+    'Sentiment classification [negative, positive]'
+    ```
+
+    英文模型调用示例：
+
+    ```python
+    >>> schema = 'Sentiment classification [negative, positive]'
+    >>> ie_en.set_schema(schema)
+    >>> ie_en('I am sorry but this is the worst film I have ever seen in my life.')
+    [{'Sentiment classification [negative, positive]': [{'text': 'negative', 'probability': 0.9998415771287057}]}]
+    ```
+
+<a name="36"></a>
+
+#### 3.6 跨任务抽取
+
+  - 例如在法律场景同时对文本进行实体抽取和关系抽取，schema可按照如下方式进行构造：
+
+    ```text
+    [
+      "法院",
+      {
+          "原告": "委托代理人"
+      },
+      {
+          "被告": "委托代理人"
+      }
+    ]
+    ```
+
+    调用示例：
+
+    ```python
+    >>> schema = ['法院', {'原告': '委托代理人'}, {'被告': '委托代理人'}]
+    >>> ie.set_schema(schema)
+    >>> pprint(ie("北京市海淀区人民法院\n民事判决书\n(199x)建初字第xxx号\n原告：张三。\n委托代理人李四，北京市 A律师事务所律师。\n被告：B公司，法定代表人王五，开发公司总经理。\n委托代理人赵六，北京市 C律师事务所律师。")) # Better print results using pprint
+    [{'原告': [{'end': 37,
+              'probability': 0.9949814024296764,
+              'relations': {'委托代理人': [{'end': 46,
+                                      'probability': 0.7956844697990384,
+                                      'start': 44,
+                                      'text': '李四'}]},
+              'start': 35,
+              'text': '张三'}],
+      '法院': [{'end': 10,
+              'probability': 0.9221074192336651,
+              'start': 0,
+              'text': '北京市海淀区人民法院'}],
+      '被告': [{'end': 67,
+              'probability': 0.8437349536631089,
+              'relations': {'委托代理人': [{'end': 92,
+                                      'probability': 0.7267121388225029,
+                                      'start': 90,
+                                      'text': '赵六'}]},
+              'start': 64,
+              'text': 'B公司'}]}]
+    ```
+
+<a name="37"></a>
+
+#### 3.7 模型选择
+
+- 多模型选择，满足精度、速度要求
+
+  | 模型 |  结构  | 语言 |
+  | :---: | :--------: | :--------: |
+  | `uie-base` (默认)| 12-layers, 768-hidden, 12-heads | 中文 |
+  | `uie-base-en` | 12-layers, 768-hidden, 12-heads | 英文 |
+  | `uie-medical-base` | 12-layers, 768-hidden, 12-heads | 中文 |
+  | `uie-medium`| 6-layers, 768-hidden, 12-heads | 中文 |
+  | `uie-mini`| 6-layers, 384-hidden, 12-heads | 中文 |
+  | `uie-micro`| 4-layers, 384-hidden, 12-heads | 中文 |
+  | `uie-nano`| 4-layers, 312-hidden, 12-heads | 中文 |
+  | `uie-m-large`| 24-layers, 1024-hidden, 16-heads | 中、英文 |
+  | `uie-m-base`| 12-layers, 768-hidden, 12-heads | 中、英文 |
+
+
+- `uie-nano`调用示例：
+
+  ```python
+  >>> from paddlenlp import Taskflow
+
+  >>> schema = ['时间', '选手', '赛事名称']
+  >>> ie = Taskflow('information_extraction', schema=schema, model="uie-nano")
+  >>> ie("2月8日上午北京冬奥会自由式滑雪女子大跳台决赛中中国选手谷爱凌以188.25分获得金牌！")
+  [{'时间': [{'text': '2月8日上午', 'start': 0, 'end': 6, 'probability': 0.6513581678349247}], '选手': [{'text': '谷爱凌', 'start': 28, 'end': 31, 'probability': 0.9819330659468051}], '赛事名称': [{'text': '北京冬奥会自由式滑雪女子大跳台决赛', 'start': 6, 'end': 23, 'probability': 0.4908131110420939}]}]
+  ```
+
+- `uie-m-base`和`uie-m-large`支持中英文混合抽取，调用示例：
+
+  ```python
+  >>> from pprint import pprint
+  >>> from paddlenlp import Taskflow
+
+  >>> schema = ['Time', 'Player', 'Competition', 'Score']
+  >>> ie = Taskflow('information_extraction', schema=schema, model="uie-m-base", schema_lang="en")
+  >>> pprint(ie(["2月8日上午北京冬奥会自由式滑雪女子大跳台决赛中中国选手谷爱凌以188.25分获得金牌！", "Rafael Nadal wins French Open Final!"]))
+  [{'Competition': [{'end': 23,
+                    'probability': 0.9373889907291257,
+                    'start': 6,
+                    'text': '北京冬奥会自由式滑雪女子大跳台决赛'}],
+    'Player': [{'end': 31,
+                'probability': 0.6981119555336441,
+                'start': 28,
+                'text': '谷爱凌'}],
+    'Score': [{'end': 39,
+              'probability': 0.9888507878270296,
+              'start': 32,
+              'text': '188.25分'}],
+    'Time': [{'end': 6,
+              'probability': 0.9784080036931151,
+              'start': 0,
+              'text': '2月8日上午'}]},
+  {'Competition': [{'end': 35,
+                    'probability': 0.9851549932171295,
+                    'start': 18,
+                    'text': 'French Open Final'}],
+    'Player': [{'end': 12,
+                'probability': 0.9379371275888104,
+                'start': 0,
+                'text': 'Rafael Nadal'}]}]
+  ```
+
+<a name="38"></a>
+
+#### 3.8 更多配置
+
+```python
+>>> from paddlenlp import Taskflow
+
+>>> ie = Taskflow('information_extraction',
+                  schema="",
+                  schema_lang="ch",
+                  batch_size=16,
+                  model='uie-base',
+                  position_prob=0.5,
+                  precision='fp32',
+                  use_fast=False)
+```
+
+* `schema`：定义任务抽取目标，可参考开箱即用中不同任务的调用示例进行配置。
+* `schema_lang`：设置schema的语言，默认为`ch`, 可选有`ch`和`en`。因为中英schema的构造有所不同，因此需要指定schema的语言。该参数只对`uie-x-base`,`uie-m-base`和`uie-m-large`模型有效。
+* `batch_size`：批处理大小，请结合机器情况进行调整，默认为16。
+* `model`：选择任务使用的模型，默认为`uie-base`，可选有`uie-base`, `uie-medium`, `uie-mini`, `uie-micro`, `uie-nano`和`uie-medical-base`, `uie-base-en`，`uie-x-base`。
+* `position_prob`：模型对于span的起始位置/终止位置的结果概率在0~1之间，返回结果去掉小于这个阈值的结果，默认为0.5，span的最终概率输出为起始位置概率和终止位置概率的乘积。
+* `precision`：选择模型精度，默认为`fp32`，可选有`fp16`和`fp32`。`fp16`推理速度更快，支持GPU和NPU硬件环境。如果选择`fp16`，在GPU硬件环境下，请先确保机器正确安装NVIDIA相关驱动和基础软件，**确保CUDA>=11.2，cuDNN>=8.1.1**，初次使用需按照提示安装相关依赖。其次，需要确保GPU设备的CUDA计算能力（CUDA Compute Capability）大于7.0，典型的设备包括V100、T4、A10、A100、GTX 20系列和30系列显卡等。更多关于CUDA Compute Capability和精度支持情况请参考NVIDIA文档：[GPU硬件与支持精度对照表](https://docs.nvidia.com/deeplearning/tensorrt/archives/tensorrt-840-ea/support-matrix/index.html#hardware-precision-matrix)。
+* `use_fast`: 使用C++实现的高性能分词算子FastTokenizer进行文本预处理加速。需要通过`pip install fast-tokenizer-python`安装FastTokenizer库后方可使用。默认为`False`。更多使用说明可参考[FastTokenizer文档](../../fast_tokenizer)。

applications/information_extraction/taskflow_text_en.md

@@ -0,0 +1,312 @@
+# UIE Taskflow User Guide - Text Information Extraction
+
+**Table of contents**
+- [1. Introduction](#1)
+- [2. Examples](#2)
+- [3. Text Information Extraction](#3)
+   - [3.1 Entity Extraction](#31)
+   - [3.2 Relation Extraction](#32)
+   - [3.3 Event Extraction](#33)
+   - [3.4 Opinion Extraction](#34)
+   - [3.5 Sentiment Classification](#35)
+   - [3.6 Multi-task Extraction](#36)
+   - [3.7 Available Models](#37)
+   - [3.8 More Configuration](#38)
+
+<a name="1"></a>
+
+## 1. Introduction
+```paddlenlp.Taskflow``` provides general information extraction of text and documents, evaluation opinion extraction and other capabilities, and can extract various types of information, including but not limited to named entities (such as person name, place name, organization name, etc.), relations (such as the director of the movie, the release time of the song, etc.), events (such as a car accident at a certain intersection, an earthquake in a certain place, etc.), and information such as product reviews, opinions, and sentiments. Users can use natural language to customize the extraction target, and can uniformly extract the corresponding information in the input text or document without training.
+
+<a name="2"></a>
+
+## 2. Examples
+
+UIE does not limit industry fields and extraction targets. The following are some industry examples implemented out of the box by Taskflow:
+
+- Medical scenarios - specialized disease structure
+
+![image](https://user-images.githubusercontent.com/40840292/169017581-93c8ee44-856d-4d17-970c-b6138d10f8bc.png)
+
+- Legal scene - Judgment extraction
+
+![image](https://user-images.githubusercontent.com/40840292/169017863-442c50f1-bfd4-47d0-8d95-8b1d53cfba3c.png)
+
+- Financial scenarios - proof of income, extraction of prospectus
+
+![image](https://user-images.githubusercontent.com/40840292/169017982-e521ddf6-d233-41f3-974e-6f40f8f2edbc.png)
+
+- Public security scene - accident report extraction
+
+![image](https://user-images.githubusercontent.com/40840292/169018340-31efc1bf-f54d-43f7-b62a-8f7ce9bf0536.png)
+
+- Tourism scene - brochure, manual extraction
+
+![image](https://user-images.githubusercontent.com/40840292/169018113-c937eb0b-9fd7-4ecc-8615-bcdde2dac81d.png)
+
+<a name="3"></a>
+
+## 3. Text information extraction
+
+<a name="31"></a>
+
+#### 3.1 Entity Extraction
+
+   Entity extraction, also known as Named Entity Recognition (NER for short), refers to identifying entities with specific meanings in text. In the open domain information extraction, the extracted categories are not limited, and users can define them by themselves.
+
+   - For example, the extracted target entity types are "person" and "organization", and the schema defined as follows:
+
+     ```text
+     ['person', 'organization']
+     ```
+
+     Example:
+
+     ```python
+     >>> from pprint import pprint
+     >>> from paddlenlp import Taskflow
+     >>> schema = ['Person', 'Organization']
+     >>> ie_en = Taskflow('information_extraction', schema=schema, model='uie-base-en')
+     >>> pprint(ie_en('In 1997, Steve was excited to become the CEO of Apple.'))
+     [{'Organization': [{'end': 53,
+                         'probability': 0.9985840259877357,
+                         'start': 48,
+                         'text': 'Apple'}],
+       'Person': [{'end': 14,
+                   'probability': 0.999631971804547,
+                   'start': 9,
+                   'text': 'Steve'}]}]
+     ```
+
+<a name="32"></a>
+
+#### 3.2 Relationship Extraction
+
+   Relation Extraction refers to identifying entities from text and extracting the semantic relationship between entities, and then obtaining triple information, namely <subject, predicate, object>.
+
+   - For example, if "person" is used as the extraction subject, and the extraction relationship types are "Company" and "Position", the schema structure is as follows:
+
+     ```text
+     {
+       'Person': [
+         'Company',
+         'Position'
+       ]
+     }
+     ```
+
+     Example:
+
+     ```python
+     >>> schema = [{'Person': ['Company', 'Position']}]
+     >>> ie_en.set_schema(schema)
+     >>> pprint(ie_en('In 1997, Steve was excited to become the CEO of Apple.'))
+     [{'Person': [{'end': 14,
+                   'probability': 0.999631971804547,
+                   'relations': {'Company': [{'end': 53,
+                                             'probability': 0.9960158209451642,
+                                             'start': 48,
+                                             'text': 'Apple'}],
+                                 'Position': [{'end': 44,
+                                               'probability': 0.8871063806420736,
+                                               'start': 41,
+                                               'text': 'CEO'}]},
+                   'start': 9,
+                   'text': 'Steve'}]}]
+     ```
+
+<a name="33"></a>
+
+#### 3.3 Event extraction
+
+   Event Extraction refers to extracting predefined event trigger words (Trigger) and event arguments (Argument) from natural language texts, and combining them into corresponding event structured information.
+
+   - The English model** does not support event extraction**, if necessary, it can be customized using the English event dataset.
+
+<a name="34"></a>
+
+#### 3.4 Opinion Extraction
+
+   Opinion extraction refers to the extraction of evaluation dimensions and opinion words contained in the text.
+
+   - For example, the target of extraction is the evaluation dimension contained in the text and its corresponding opinion words and emotional tendencies. The schema structure is as follows:
+
+     ```text
+     {
+       'Aspect': [
+         'Opinion',
+         'Sentiment classification [negative, positive]'
+       ]
+     }
+     ```
+
+     Example:
+
+     ```python
+     >>> schema = [{'Aspect': ['Opinion', 'Sentiment classification [negative, positive]']}]
+     >>> ie_en.set_schema(schema)
+     >>> pprint(ie_en("The teacher is very nice."))
+     [{'Aspect': [{'end': 11,
+                   'probability': 0.4301476415932193,
+                   'relations': {'Opinion': [{'end': 24,
+                                             'probability': 0.9072940447883724,
+                                             'start': 15,
+                                             'text': 'very nice'}],
+                                 'Sentiment classification [negative, positive]': [{'probability': 0.9998571920670685,
+                                                                                   'text': 'positive'}]},
+                   'start': 4,
+                   'text': 'teacher'}]}]
+     ```
+
+<a name="35"></a>
+
+#### 3.5 Sentiment Classification
+
+   - Sentence-level sentiment classification, that is, to judge whether the emotional orientation of a sentence is "positive" or "negative". The schema structure is as follows:
+
+     ```text
+     'Sentiment classification [negative, positive]'
+     ```
+
+     Example:
+
+     ```python
+     >>> schema = 'Sentiment classification [negative, positive]'
+     >>> ie_en.set_schema(schema)
+     >>> ie_en('I am sorry but this is the worst film I have ever seen in my life.')
+     [{'Sentiment classification [negative, positive]': [{'text': 'negative', 'probability': 0.9998415771287057}]}]
+     ```
+
+#### 3.6 Multi-Task Extraction
+
+   - For example, in the legal scene, entity extraction and relation extraction are performed on the text at the same time, and the schema can be constructed as follows:
+
+    ```text
+    [
+      "法院",
+      {
+          "原告": "委托代理人"
+      },
+      {
+          "被告": "委托代理人"
+      }
+    ]
+    ```
+
+    Example:
+
+    ```python
+    >>> schema = ['法院', {'原告': '委托代理人'}, {'被告': '委托代理人'}]
+    >>> ie.set_schema(schema)
+    >>> pprint(ie("北京市海淀区人民法院\n民事判决书\n(199x)建初字第xxx号\n原告：张三。\n委托代理人李四，北京市 A律师事务所律师。\n被告：B公司，法定代表人王五，开发公司总经理。\n委托代理人赵六，北京市 C律师事务所律师。")) # Better print results using pprint
+    [{'原告': [{'end': 37,
+              'probability': 0.9949814024296764,
+              'relations': {'委托代理人': [{'end': 46,
+                                      'probability': 0.7956844697990384,
+                                      'start': 44,
+                                      'text': '李四'}]},
+              'start': 35,
+              'text': '张三'}],
+      '法院': [{'end': 10,
+              'probability': 0.9221074192336651,
+              'start': 0,
+              'text': '北京市海淀区人民法院'}],
+      '被告': [{'end': 67,
+              'probability': 0.8437349536631089,
+              'relations': {'委托代理人': [{'end': 92,
+                                      'probability': 0.7267121388225029,
+                                      'start': 90,
+                                      'text': '赵六'}]},
+              'start': 64,
+              'text': 'B公司'}]}]
+    ```
+
+<a name="37"></a>
+
+#### 3.7 Available Model
+
+- A variety of models to different accuracy and speed requirements
+
+   | Model | Structure | Language |
+   | :---: | :--------: | :--------: |
+   | `uie-base` (default)| 12-layers, 768-hidden, 12-heads | Chinese |
+   | `uie-base-en` | 12-layers, 768-hidden, 12-heads | English |
+   | `uie-medical-base` | 12-layers, 768-hidden, 12-heads | Chinese |
+   | `uie-medium`| 6-layers, 768-hidden, 12-heads | Chinese |
+   | `uie-mini`| 6-layers, 384-hidden, 12-heads | Chinese |
+   | `uie-micro`| 4-layers, 384-hidden, 12-heads | Chinese |
+   | `uie-nano`| 4-layers, 312-hidden, 12-heads | Chinese |
+   | `uie-m-large`| 24-layers, 1024-hidden, 16-heads | Chinese and English |
+   | `uie-m-base`| 12-layers, 768-hidden, 12-heads | Chinese and English |
+
+
+- `uie-nano` call example:
+
+  ```python
+  >>> from paddlenlp import Taskflow
+
+  >>> schema = ['时间', '选手', '赛事名称']
+  >>> ie = Taskflow('information_extraction', schema=schema, model="uie-nano")
+  >>> ie("2月8日上午北京冬奥会自由式滑雪女子大跳台决赛中中国选手谷爱凌以188.25分获得金牌！")
+  [{'时间': [{'text': '2月8日上午', 'start': 0, 'end': 6, 'probability': 0.6513581678349247}], '选手': [{'text': '谷爱凌', 'start': 28, 'end': 31, 'probability': 0.9819330659468051}], '赛事名称': [{'text': '北京冬奥会自由式滑雪女子大跳台决赛', 'start': 6, 'end': 23, 'probability': 0.4908131110420939}]}]
+  ```
+
+- `uie-m-base` and `uie-m-large` support extraction of both Chinese and English, call example:
+
+  ```python
+  >>> from pprint import pprint
+  >>> from paddlenlp import Taskflow
+
+  >>> schema = ['Time', 'Player', 'Competition', 'Score']
+  >>> ie = Taskflow('information_extraction', schema=schema, model="uie-m-base", schema_lang="en")
+  >>> pprint(ie(["2月8日上午北京冬奥会自由式滑雪女子大跳台决赛中中国选手谷爱凌以188.25分获得金牌！", "Rafael Nadal wins French Open Final!"]))
+  [{'Competition': [{'end': 23,
+                    'probability': 0.9373889907291257,
+                    'start': 6,
+                    'text': '北京冬奥会自由式滑雪女子大跳台决赛'}],
+    'Player': [{'end': 31,
+                'probability': 0.6981119555336441,
+                'start': 28,
+                'text': '谷爱凌'}],
+    'Score': [{'end': 39,
+              'probability': 0.9888507878270296,
+              'start': 32,
+              'text': '188.25分'}],
+    'Time': [{'end': 6,
+              'probability': 0.9784080036931151,
+              'start': 0,
+              'text': '2月8日上午'}]},
+  {'Competition': [{'end': 35,
+                    'probability': 0.9851549932171295,
+                    'start': 18,
+                    'text': 'French Open Final'}],
+    'Player': [{'end': 12,
+                'probability': 0.9379371275888104,
+                'start': 0,
+                'text': 'Rafael Nadal'}]}]
+  ```
+
+<a name="38"></a>
+
+#### 3.8 More Configuration
+
+```python
+>>> from paddlenlp import Taskflow
+
+>>> ie = Taskflow('information_extraction',
+                  schema="",
+                  schema_lang="ch",
+                  batch_size=16,
+                  model='uie-base',
+                  position_prob=0.5,
+                  precision='fp32',
+                  use_fast=False)
+```
+
+* `schema`: Define the task extraction target, which can be configured by referring to the calling examples of different tasks in the out-of-the-box.
+* `schema_lang`: Set the language of the schema, the default is `ch`, optional `ch` and `en`. Because the structure of the Chinese and English schemas is different, the language of the schema needs to be specified. This parameter is only valid for `uie-x-base`, `uie-m-base` and `uie-m-large` models.
+* `batch_size`: batch size, please adjust according to the machine situation, the default is 16.
+* `model`: select the model used by the task, the default is `uie-base`, optional `uie-base`, `uie-medium`, `uie-mini`, `uie-micro`, `uie-nano` and `uie-medical-base`, `uie-base-en`, `uie-x-base`.
+* `position_prob`: The result probability of the model for the start position/end position of the span is between 0 and 1, and the returned result removes the results less than this threshold, the default is 0.5, and the final probability output of the span is the start position probability and end position The product of the position probabilities.
+* `precision`: select the model precision, the default is `fp32`, optional `fp16` and `fp32`. `fp16` inference is faster, support GPU and NPU hardware. If you choose `fp16` and GPU hardware, please ensure that the machine is correctly installed with NVIDIA-related drivers and basic software. **Ensure that CUDA>=11.2, cuDNN>=8.1.1**. For the first time use, you need to follow the prompts to install the relevant dependencies. Secondly, it is necessary to ensure that the CUDA Compute Capability of the GPU device is greater than 7.0. Typical devices include V100, T4, A10, A100, GTX 20 series and 30 series graphics cards, etc. For more information about CUDA Compute Capability and precision support, please refer to NVIDIA documentation: [GPU Hardware and Supported Precision Comparison Table](https://docs.nvidia.com/deeplearning/tensorrt/archives/tensorrt-840-ea/support-matrix/index.html#hardware-precision-matrix).
+* `use_fast`: Use the high-performance word segmentation operator FastTokenizer implemented in C++ to accelerate text preprocessing. The FastTokenizer library needs to be installed through `pip install fast-tokenizer-python` before it can be used. Defaults to `False`. For more usage instructions, please refer to [FastTokenizer Documentation](../../fast_tokenizer).

applications/information_extraction/text/README.md

@@ -0,0 +1,289 @@
+简体中文 | [English](README_en.md)
+
+# 文本信息抽取
+
+**目录**
+- [1. 文本信息抽取应用](#1)
+- [2. 快速开始](#2)
+  - [2.1 代码结构](#代码结构)
+  - [2.2 数据标注](#数据标注)
+  - [2.3 模型微调](#模型微调)
+  - [2.4 模型评估](#模型评估)
+  - [2.5 定制模型一键预测](#定制模型一键预测)
+  - [2.6 实验指标](#实验指标)
+  - [2.7 封闭域蒸馏](#封闭域蒸馏)
+
+<a name="1"></a>
+
+## 1. 文本信息抽取应用
+
+本项目提供基于UIE微调的纯文本抽取端到端应用方案，打通**数据标注-模型训练-模型调优-预测部署全流程**，可快速实现文档信息抽取产品落地。
+
+信息抽取通俗地说就是从给定的文本/图片等输入数据中抽取出结构化信息的过程。在信息抽取的落地过程中通常面临领域多变、任务多样、数据稀缺等许多挑战。针对信息抽取领域的难点和痛点，PaddleNLP信息抽取应用UIE统一建模的思想，提供了文档信息抽取产业级应用方案，支持**文档/图片/表格和纯文本场景下实体、关系、事件、观点等不同任务信息抽取**。该应用**不限定行业领域和抽取目标**，可实现从产品原型研发、业务POC阶段到业务落地、迭代阶段的无缝衔接，助力开发者实现特定领域抽取场景的快速适配与落地。
+
+**文本信息抽取应用亮点：**
+
+- **覆盖场景全面🎓：** 覆盖文本信息抽取各类主流任务，支持多语言，满足开发者多样信息抽取落地需求。
+- **效果领先🏃：** 以在纯文本具有突出效果的UIE系列模型作为训练基座，提供多种尺寸的预训练模型满足不同需求，具有广泛成熟的实践应用性。
+- **简单易用⚡：** 通过Taskflow实现三行代码可实现无标注数据的情况下进行快速调用，一行命令即可开启信息抽取训练，轻松完成部署上线，降低信息抽取技术落地门槛。
+- **高效调优✊：** 开发者无需机器学习背景知识，即可轻松上手数据标注及模型训练流程。
+
+<a name="2"></a>
+
+## 2. 快速开始
+
+对于简单的抽取目标可以直接使用```paddlenlp.Taskflow```实现零样本（zero-shot）抽取，对于细分场景我们推荐使用定制功能（标注少量数据进行模型微调）以进一步提升效果。
+
+<a name="代码结构"></a>
+
+### 2.1 代码结构
+
+```shell
+.
+├── utils.py          # 数据处理工具
+├── finetune.py       # 模型微调、压缩脚本
+├── evaluate.py       # 模型评估脚本
+└── README.md
+```
+
+<a name="数据标注"></a>
+
+### 2.2 数据标注
+
+我们推荐使用 [Label Studio](https://labelstud.io/) 进行文本信息抽取数据标注，本项目打通了从数据标注到训练的通道，也即Label Studio导出数据可以通过 [label_studio.py](../label_studio.py) 脚本轻松将数据转换为输入模型时需要的形式，实现无缝衔接。标注方法的详细介绍请参考 [Label Studio数据标注指南](../label_studio_text.md)。
+
+这里我们提供预先标注好的`军事关系抽取数据集`的文件，可以运行下面的命令行下载数据集，我们将展示如何使用数据转化脚本生成训练/验证/测试集文件，并使用UIE模型进行微调。
+
+下载军事关系抽取数据集：
+
+```shell
+wget https://bj.bcebos.com/paddlenlp/datasets/military.tar.gz
+tar -xvf military.tar.gz
+mv military data
+rm military.tar.gz
+```
+
+生成训练/验证集文件：
+```shell
+python ../label_studio.py \
+    --label_studio_file ./data/label_studio.json \
+    --save_dir ./data \
+    --splits 0.76 0.24 0 \
+    --negative_ratio 3 \
+    --task_type ext
+```
+
+更多不同类型任务（含实体抽取、关系抽取、文档分类等）的标注规则及参数说明，请参考[Label Studio数据标注指南](../label_studio_text.md)。
+
+
+<a name="模型微调"></a>
+
+### 2.3 模型微调
+
+推荐使用 [Trainer API ](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/docs/trainer.md) 对模型进行微调。只需输入模型、数据集等就可以使用 Trainer API 高效快速地进行预训练、微调和模型压缩等任务，可以一键启动多卡训练、混合精度训练、梯度累积、断点重启、日志显示等功能，Trainer API 还针对训练过程的通用训练配置做了封装，比如：优化器、学习率调度等。
+
+使用下面的命令，使用 `uie-base` 作为预训练模型进行模型微调，将微调后的模型保存至`$finetuned_model`：
+
+单卡启动：
+
+```shell
+python finetune.py  \
+    --device gpu \
+    --logging_steps 10 \
+    --save_steps 100 \
+    --eval_steps 100 \
+    --seed 1000 \
+    --model_name_or_path uie-base \
+    --output_dir ./checkpoint/model_best \
+    --train_path data/train.txt \
+    --dev_path data/dev.txt  \
+    --max_seq_len 512  \
+    --per_device_train_batch_size  16 \
+    --per_device_eval_batch_size 16 \
+    --num_train_epochs 20 \
+    --learning_rate 1e-5 \
+    --do_train \
+    --do_eval \
+    --do_export \
+    --export_model_dir ./checkpoint/model_best \
+    --overwrite_output_dir \
+    --disable_tqdm True \
+    --metric_for_best_model eval_f1 \
+    --load_best_model_at_end  True \
+    --save_total_limit 1
+```
+
+如果在GPU环境中使用，可以指定gpus参数进行多卡训练：
+
+```shell
+python -u -m paddle.distributed.launch --gpus "0,1" finetune.py \
+    --device gpu \
+    --logging_steps 10 \
+    --save_steps 100 \
+    --eval_steps 100 \
+    --seed 1000 \
+    --model_name_or_path uie-base \
+    --output_dir ./checkpoint/model_best \
+    --train_path data/train.txt \
+    --dev_path data/dev.txt  \
+    --max_seq_len 512  \
+    --per_device_train_batch_size  8 \
+    --per_device_eval_batch_size 8 \
+    --num_train_epochs 20 \
+    --learning_rate 1e-5 \
+    --do_train \
+    --do_eval \
+    --do_export \
+    --export_model_dir ./checkpoint/model_best \
+    --overwrite_output_dir \
+    --disable_tqdm True \
+    --metric_for_best_model eval_f1 \
+    --load_best_model_at_end  True \
+    --save_total_limit 1
+```
+
+该示例代码中由于设置了参数 `--do_eval`，因此在训练完会自动进行评估。
+
+可配置参数说明：
+* `device`: 训练设备，可选择 'cpu'、'gpu'、'npu' 其中的一种；默认为 GPU 训练。
+* `logging_steps`: 训练过程中日志打印的间隔 steps 数，默认10。
+* `save_steps`: 训练过程中保存模型 checkpoint 的间隔 steps 数，默认100。
+* `eval_steps`: 训练过程中保存模型 checkpoint 的间隔 steps 数，默认100。
+* `seed`：全局随机种子，默认为 42。
+* `model_name_or_path`：进行 few shot 训练使用的预训练模型。默认为 "uie-x-base"。
+* `output_dir`：必须，模型训练或压缩后保存的模型目录；默认为 `None` 。
+* `train_path`：训练集路径；默认为 `None` 。
+* `dev_path`：开发集路径；默认为 `None` 。
+* `max_seq_len`：文本最大切分长度，输入超过最大长度时会对输入文本进行自动切分，默认为512。
+* `per_device_train_batch_size`:用于训练的每个 GPU 核心/CPU 的batch大小，默认为8。
+* `per_device_eval_batch_size`:用于评估的每个 GPU 核心/CPU 的batch大小，默认为8。
+* `num_train_epochs`: 训练轮次，使用早停法时可以选择 100；默认为10。
+* `learning_rate`：训练最大学习率，UIE-X 推荐设置为 1e-5；默认值为3e-5。
+* `label_names`：训练数据标签label的名称，UIE-X 设置为'start_positions' 'end_positions'；默认值为None。
+* `do_train`:是否进行微调训练，设置该参数表示进行微调训练，默认不设置。
+* `do_eval`:是否进行评估，设置该参数表示进行评估，默认不设置。
+* `do_export`:是否进行导出，设置该参数表示进行静态图导出，默认不设置。
+* `export_model_dir`:静态图导出地址，默认为None。
+* `overwrite_output_dir`： 如果 `True`，覆盖输出目录的内容。如果 `output_dir` 指向检查点目录，则使用它继续训练。
+* `disable_tqdm`： 是否使用tqdm进度条。
+* `metric_for_best_model`：最优模型指标,UIE-X 推荐设置为 `eval_f1`，默认为None。
+* `load_best_model_at_end`：训练结束后是否加载最优模型，通常与`metric_for_best_model`配合使用，默认为False。
+* `save_total_limit`：如果设置次参数，将限制checkpoint的总数。删除旧的checkpoints `输出目录`，默认为None。
+
+<a name="模型评估"></a>
+
+### 2.4 模型评估
+
+通过运行以下命令进行模型评估：
+
+```shell
+python evaluate.py \
+    --model_path ./checkpoint/model_best \
+    --test_path ./data/dev.txt \
+    --device gpu \
+    --batch_size 16 \
+    --max_seq_len 512
+```
+
+通过运行以下命令对 UIE-M 进行模型评估：
+
+```
+python evaluate.py \
+    --model_path ./checkpoint/model_best \
+    --test_path ./data/dev.txt \
+    --batch_size 16 \
+    --device gpu \
+    --max_seq_len 512 \
+    --multilingual
+```
+
+评估方式说明：采用单阶段评价的方式，即关系抽取、事件抽取等需要分阶段预测的任务对每一阶段的预测结果进行分别评价。验证/测试集默认会利用同一层级的所有标签来构造出全部负例。
+
+可开启`debug`模式对每个正例类别分别进行评估，该模式仅用于模型调试：
+
+```shell
+python evaluate.py \
+    --model_path ./checkpoint/model_best \
+    --test_path ./data/dev.txt \
+    --debug
+```
+
+输出打印示例：
+
+```text
+[2022-11-21 12:48:41,794] [    INFO] - -----------------------------
+[2022-11-21 12:48:41,795] [    INFO] - Class Name: 武器名称
+[2022-11-21 12:48:41,795] [    INFO] - Evaluation Precision: 0.96667 | Recall: 0.96667 | F1: 0.96667
+[2022-11-21 12:48:44,093] [    INFO] - -----------------------------
+[2022-11-21 12:48:44,094] [    INFO] - Class Name: X的产国
+[2022-11-21 12:48:44,094] [    INFO] - Evaluation Precision: 1.00000 | Recall: 0.99275 | F1: 0.99636
+[2022-11-21 12:48:46,474] [    INFO] - -----------------------------
+[2022-11-21 12:48:46,475] [    INFO] - Class Name: X的研发单位
+[2022-11-21 12:48:46,475] [    INFO] - Evaluation Precision: 0.77519 | Recall: 0.64935 | F1: 0.70671
+[2022-11-21 12:48:48,800] [    INFO] - -----------------------------
+[2022-11-21 12:48:48,801] [    INFO] - Class Name: X的类型
+[2022-11-21 12:48:48,801] [    INFO] - Evaluation Precision: 1.00000 | Recall: 1.00000 | F1: 1.00000
+```
+
+可配置参数说明：
+
+- `device`: 评估设备，可选择 'cpu'、'gpu'、'npu' 其中的一种；默认为 GPU 评估。
+- `model_path`: 进行评估的模型文件夹路径，路径下需包含模型权重文件`model_state.pdparams`及配置文件`model_config.json`。
+- `test_path`: 进行评估的测试集文件。
+- `batch_size`: 批处理大小，请结合机器情况进行调整，默认为16。
+- `max_seq_len`: 文本最大切分长度，输入超过最大长度时会对输入文本进行自动切分，默认为512。
+- `debug`: 是否开启debug模式对每个正例类别分别进行评估，该模式仅用于模型调试，默认关闭。
+- `multilingual`: 是否是跨语言模型，默认关闭。
+- `schema_lang`: 选择schema的语言，可选有`ch`和`en`。默认为`ch`，英文数据集请选择`en`。
+
+<a name="定制模型一键预测"></a>
+
+### 2.5 定制模型一键预测
+
+`paddlenlp.Taskflow`装载定制模型，通过`task_path`指定模型权重文件的路径，路径下需要包含训练好的模型权重文件`model_state.pdparams`。
+
+```python
+>>> from pprint import pprint
+>>> from paddlenlp import Taskflow
+
+>>> schema = {"武器名称": ["产国", "类型", "研发单位"]}
+# 设定抽取目标和定制化模型权重路径
+>>> my_ie = Taskflow("information_extraction", schema=schema, task_path='./checkpoint/model_best')
+>>> pprint(my_ie("威尔哥（Virgo）减速炸弹是由瑞典FFV军械公司专门为瑞典皇家空军的攻击机实施低空高速轰炸而研制，1956年开始研制，1963年进入服役，装备于A32“矛盾”、A35“龙”、和AJ134“雷”攻击机，主要用于攻击登陆艇、停放的飞机、高炮、野战火炮、轻型防护装甲车辆以及有生力量。"))
+[{'武器名称': [{'end': 14,
+            'probability': 0.9998632702221926,
+            'relations': {'产国': [{'end': 18,
+                                  'probability': 0.9998815094394331,
+                                  'start': 16,
+                                  'text': '瑞典'}],
+                          '研发单位': [{'end': 25,
+                                    'probability': 0.9995875123178521,
+                                    'start': 18,
+                                    'text': 'FFV军械公司'}],
+                          '类型': [{'end': 14,
+                                  'probability': 0.999877336059086,
+                                  'start': 12,
+                                  'text': '炸弹'}]},
+            'start': 0,
+            'text': '威尔哥（Virgo）减速炸弹'}]}]
+```
+
+<a name="实验指标"></a>
+
+### 2.6 实验指标
+
+军事关系抽取数据集实验指标：
+
+  |  |  Precision  | Recall | F1 Score |
+  | :---: | :--------: | :--------: | :--------: |
+  | 0-shot | 0.64634| 0.53535 | 0.58564 |
+  | 5-shot | 0.89474 | 0.85000 | 0.87179 |
+  | 10-shot | 0.92793 | 0.85833 | 0.89177 |
+  | full-set | 0.93103 | 0.90000 | 0.91525 |
+
+
+<a name="封闭域蒸馏"></a>
+
+### 2.7 封闭域蒸馏
+
+在一些工业应用场景中对性能的要求较高，模型若不能有效压缩则无法实际应用。因此，我们基于数据蒸馏技术构建了UIE Slim数据蒸馏系统。其原理是通过数据作为桥梁，将UIE模型的知识迁移到封闭域信息抽取小模型，以达到精度损失较小的情况下却能达到大幅度预测速度提升的效果。详细介绍请参考[UIE Slim 数据蒸馏](./data_distill/README.md)

applications/information_extraction/text/README_en.md

@@ -0,0 +1,278 @@
+# Text information extraction
+
+**Table of contents**
+- [1. Text Information Extraction Application](#1)
+- [2. Quick Start](#2)
+   - [2.1 Code Structure](#21)
+   - [2.2 Data Annotation](#22)
+   - [2.3 Finetuning](#23)
+   - [2.4 Evaluation](#24)
+   - [2.5 Inference](#25)
+   - [2.6 Experiments](#26)
+   - [2.7 Closed Domain Distillation](#27)
+
+<a name="1"></a>
+
+## 1. Text Information Extraction Application
+
+This project provides an end-to-end application solution for plain text extraction based on UIE fine-tuning and goes through the full lifecycle of **data labeling, model training and model deployment**. We hope this guide can help you apply Information Extraction techniques in your own products or models.a
+
+Information Extraction (IE) is the process of extracting structured information from given input data such as text, pictures or scanned document. While IE brings immense value, applying IE techniques is never easy with challenges such as domain adaptation, heterogeneous structures, lack of labeled data, etc. This PaddleNLP Information Extraction Guide builds on the foundation of our work in [Universal Information Extraction](https://arxiv.org/abs/2203.12277) and provides an industrial-level solution that not only supports **extracting entities, relations, events and opinions from plain text**, but also supports **cross-modal extraction out of documents, tables and pictures.** Our method features a flexible prompt, which allows you to specify extraction targets with simple natural language. We also provide a few different domain-adapated models specialized for different industry sectors.
+
+**Highlights:**
+- **Comprehensive Coverage🎓:** Covers various mainstream tasks of information extraction for plain text and document scenarios, supports multiple languages
+- **State-of-the-Art Performance🏃:** Strong performance from the UIE model series models in plain text and multimodal datasets. We also provide pretrained models of various sizes to meet different needs
+- **Easy to use⚡:** three lines of code to use our `Taskflow` for out-of-box Information Extraction capabilities. One line of command to model training and model deployment
+- **Efficient Tuning✊:** Developers can easily get started with the data labeling and model training process without a background in Machine Learning.
+
+<a name="2"></a>
+
+## 2. Quick start
+
+For quick start, you can directly use ```paddlenlp.Taskflow``` out-of-the-box, leveraging the zero-shot performance. For production use cases, we recommend labeling a small amount of data for model fine-tuning to further improve the performance.
+
+<a name="21"></a>
+
+### 2.1 Code structure
+
+```shell
+.
+├── utils.py # data processing tools
+├── finetune.py # model fine-tuning, compression script
+├── evaluate.py # model evaluation script
+└── README.md
+```
+
+<a name="22"></a>
+
+### 2.2 Data labeling
+
+We recommend using [Label Studio](https://labelstud.io/) for data labeling. We provide an end-to-end pipeline for the labeling -> training process. You can export the labeled data in Label Studio through [label_studio.py](../label_studio.py) script to export and convert the data into the required input form for the model. For a detailed introduction to labeling methods, please refer to [Label Studio Data Labeling Guide](../label_studio_text_en.md).
+
+Here we provide a pre-labeled example dataset `Military Relationship Extraction Dataset`, which you can download with the following command. We will show how to use the data conversion script to generate training/validation/test set files for fine-tuning .
+
+Download the military relationship extraction dataset:
+
+```shell
+wget https://bj.bcebos.com/paddlenlp/datasets/military.tar.gz
+tar -xvf military.tar.gz
+mv military data
+rm military.tar.gz
+```
+
+Generate training/validation set files:
+```shell
+python ../label_studio.py \
+     --label_studio_file ./data/label_studio.json \
+     --save_dir ./data \
+     --splits 0.76 0.24 0 \
+     --negative_ratio 3 \
+     --task_type ext
+```
+
+For more labeling rules and parameter descriptions for different types of tasks (including entity extraction, relationship extraction, document classification, etc.), please refer to [Label Studio Data Labeling Guide](../label_studio_text_en.md).
+
+
+<a name="23"></a>
+
+### 2.3 Finetuning
+
+Use the following command to fine-tune the model using `uie-base` as the pre-trained model, and save the fine-tuned model to `$finetuned_model`:
+
+Single GPU:
+
+```shell
+python finetune.py  \
+    --device gpu \
+    --logging_steps 10 \
+    --save_steps 100 \
+    --eval_steps 100 \
+    --seed 1000 \
+    --model_name_or_path uie-base \
+    --output_dir ./checkpoint/model_best \
+    --train_path data/train.txt \
+    --dev_path data/dev.txt  \
+    --max_seq_len 512  \
+    --per_device_train_batch_size  16 \
+    --per_device_eval_batch_size 16 \
+    --num_train_epochs 20 \
+    --learning_rate 1e-5 \
+    --do_train \
+    --do_eval \
+    --do_export \
+    --export_model_dir ./checkpoint/model_best \
+    --overwrite_output_dir \
+    --disable_tqdm True \
+    --metric_for_best_model eval_f1 \
+    --load_best_model_at_end  True \
+    --save_total_limit 1
+```
+
+Multiple GPUs：
+
+```shell
+python -u -m paddle.distributed.launch --gpus "0,1" finetune.py \
+    --device gpu \
+    --logging_steps 10 \
+    --save_steps 100 \
+    --eval_steps 100 \
+    --seed 1000 \
+    --model_name_or_path uie-base \
+    --output_dir ./checkpoint/model_best \
+    --train_path data/train.txt \
+    --dev_path data/dev.txt  \
+    --max_seq_len 512  \
+    --per_device_train_batch_size  8 \
+    --per_device_eval_batch_size 8 \
+    --num_train_epochs 20 \
+    --learning_rate 1e-5 \
+    --do_train \
+    --do_eval \
+    --do_export \
+    --export_model_dir ./checkpoint/model_best \
+    --overwrite_output_dir \
+    --disable_tqdm True \
+    --metric_for_best_model eval_f1 \
+    --load_best_model_at_end  True \
+    --save_total_limit 1
+```
+
+Parameters:
+
+* `device`: Training device, one of 'cpu', 'gpu' and 'npu' can be selected; the default is GPU training.
+* `logging_steps`: The interval steps of log printing during training, the default is 10.
+* `save_steps`: The number of interval steps to save the model checkpoint during training, the default is 100.
+* `eval_steps`: The number of interval steps to save the model checkpoint during training, the default is 100.
+* `seed`: global random seed, default is 42.
+* `model_name_or_path`: The pre-trained model used for few shot training. Defaults to "uie-x-base".
+* `output_dir`: required, the model directory saved after model training or compression; the default is `None`.
+* `train_path`: training set path; defaults to `None`.
+* `dev_path`: Development set path; defaults to `None`.
+* `max_seq_len`: The maximum segmentation length of the text. When the input exceeds the maximum length, the input text will be automatically segmented. The default is 512.
+* `per_device_train_batch_size`: The batch size of each GPU core//NPU core/CPU used for training, the default is 8.
+* `per_device_eval_batch_size`: Batch size per GPU core/NPU core/CPU for evaluation, default is 8.
+* `num_train_epochs`: Training rounds, 100 can be selected when using early stopping method; the default is 10.
+* `learning_rate`: The maximum learning rate for training, UIE-X recommends setting it to 1e-5; the default value is 3e-5.
+* `label_names`: the name of the training data label, UIE-X is set to 'start_positions' 'end_positions'; the default value is None.
+* `do_train`: Whether to perform fine-tuning training, setting this parameter means to perform fine-tuning training, and it is not set by default.
+* `do_eval`: Whether to evaluate, setting this parameter means to evaluate, the default is not set.
+* `do_export`: Whether to export, setting this parameter means to export static images, and it is not set by default.
+* `export_model_dir`: Static map export address, the default is None.
+* `overwrite_output_dir`: If `True`, overwrite the contents of the output directory. If `output_dir` points to a checkpoint directory, use it to continue training.
+* `disable_tqdm`: Whether to use tqdm progress bar.
+* `metric_for_best_model`: Optimal model metric, UIE-X recommends setting it to `eval_f1`, the default is None.
+* `load_best_model_at_end`: Whether to load the best model after training, usually used in conjunction with `metric_for_best_model`, the default is False.
+* `save_total_limit`: If this parameter is set, the total number of checkpoints will be limited. Remove old checkpoints `output directory`, defaults to None.
+
+<a name="24"></a>
+
+### 2.4 Evaluation
+
+Model evaluation:
+
+```shell
+python evaluate.py \
+    --model_path ./checkpoint/model_best \
+    --test_path ./data/dev.txt \
+    --batch_size 16 \
+    --max_seq_len 512
+```
+
+Model evaluation for UIE-M：
+
+```
+python evaluate.py \
+    --model_path ./checkpoint/model_best \
+    --test_path ./data/dev.txt \
+    --batch_size 16 \
+    --max_seq_len 512 \
+    --multilingual
+```
+
+We adopt the single-stage method for evaluation, which means tasks that require multiple stages (e.g. relation extraction, event extraction) are evaluated separately for each stage. By default, the validation/test set uses all labels at the same level to construct the negative examples.
+
+The `debug` mode can be turned on to evaluate each positive category separately. This mode is only used for model debugging:
+
+```shell
+python evaluate.py \
+    --model_path ./checkpoint/model_best \
+    --test_path ./data/dev.txt \
+    --debug
+```
+
+Output print example:
+
+```text
+[2022-11-21 12:48:41,794] [    INFO] - -----------------------------
+[2022-11-21 12:48:41,795] [    INFO] - Class Name: 武器名称
+[2022-11-21 12:48:41,795] [    INFO] - Evaluation Precision: 0.96667 | Recall: 0.96667 | F1: 0.96667
+[2022-11-21 12:48:44,093] [    INFO] - -----------------------------
+[2022-11-21 12:48:44,094] [    INFO] - Class Name: X的产国
+[2022-11-21 12:48:44,094] [    INFO] - Evaluation Precision: 1.00000 | Recall: 0.99275 | F1: 0.99636
+[2022-11-21 12:48:46,474] [    INFO] - -----------------------------
+[2022-11-21 12:48:46,475] [    INFO] - Class Name: X的研发单位
+[2022-11-21 12:48:46,475] [    INFO] - Evaluation Precision: 0.77519 | Recall: 0.64935 | F1: 0.70671
+[2022-11-21 12:48:48,800] [    INFO] - -----------------------------
+[2022-11-21 12:48:48,801] [    INFO] - Class Name: X的类型
+[2022-11-21 12:48:48,801] [    INFO] - Evaluation Precision: 1.00000 | Recall: 1.00000 | F1: 1.00000
+```
+
+Parameters:
+
+- `device`: Evaluation device, one of 'cpu', 'gpu' and 'npu' can be selected; the default is GPU evaluation.
+- `model_path`: The path of the model folder for evaluation, which must contain the model weight file `model_state.pdparams` and the configuration file `model_config.json`.
+- `test_path`: The test set file for evaluation.
+- `batch_size`: batch size, please adjust according to the machine situation, the default is 16.
+- `max_seq_len`: The maximum segmentation length of the text. When the input exceeds the maximum length, the input text will be automatically segmented. The default is 512.
+- `debug`: Whether to enable the debug mode to evaluate each positive category separately. This mode is only used for model debugging and is disabled by default.
+- `multilingual`: Whether it is a multilingual model, it is turned off by default.
+- `schema_lang`: select the language of the schema, optional `ch` and `en`. The default is `ch`, please select `en` for the English dataset.
+
+<a name="25"></a>
+
+### 2.5 Inference
+Same with the pretrained models, you can use `paddlenlp.Taskflow` to load your custom model by specifying the path of the model weight file through `task_path`
+
+```python
+>>> from pprint import pprint
+>>> from paddlenlp import Taskflow
+
+>>> schema = {"武器名称": ["产国", "类型", "研发单位"]}
+# Set the extraction target and the fine-tuned model path
+>>> my_ie = Taskflow("information_extraction", schema=schema, task_path='./checkpoint/model_best')
+>>> pprint(my_ie("威尔哥（Virgo）减速炸弹是由瑞典FFV军械公司专门为瑞典皇家空军的攻击机实施低空高速轰炸而研制，1956年开始研制，1963年进入服役，装备于A32“矛盾”、A35“龙”、和AJ134“雷”攻击机，主要用于攻击登陆艇、停放的飞机、高炮、野战火炮、轻型防护装甲车辆以及有生力量。"))
+[{'武器名称': [{'end': 14,
+            'probability': 0.9998632702221926,
+            'relations': {'产国': [{'end': 18,
+                                  'probability': 0.9998815094394331,
+                                  'start': 16,
+                                  'text': '瑞典'}],
+                          '研发单位': [{'end': 25,
+                                    'probability': 0.9995875123178521,
+                                    'start': 18,
+                                    'text': 'FFV军械公司'}],
+                          '类型': [{'end': 14,
+                                  'probability': 0.999877336059086,
+                                  'start': 12,
+                                  'text': '炸弹'}]},
+            'start': 0,
+            'text': '威尔哥（Virgo）减速炸弹'}]}]
+```
+
+<a name="26"></a>
+
+### 2.6 Experiments
+
+  |  |  Precision  | Recall | F1 Score |
+  | :---: | :--------: | :--------: | :--------: |
+  | 0-shot | 0.64634| 0.53535 | 0.58564 |
+  | 5-shot | 0.89474 | 0.85000 | 0.87179 |
+  | 10-shot | 0.92793 | 0.85833 | 0.89177 |
+  | full-set | 0.93103 | 0.90000 | 0.91525 |
+
+
+<a name="27"></a>
+
+### 2.7 Closed Domain Distillation
+
+Some industrial application scenarios have high inference performance requirements and the model cannot go into production without being effectively compressed. We built the [UIE Slim Data Distillation](./data_distill/README_en.md) with knowledge distillation techniques. The principle is to use the data as a bridge to transfer the knowledge of the UIE model to the smaller closed-domain information extraction model in order to achieve speedup inference significantly with minimal loss to accuracy.

applications/information_extraction/text/data_distill/README.md

@@ -0,0 +1,156 @@
+# UIE Slim 数据蒸馏
+
+在UIE强大的抽取能力背后，同样需要较大的算力支持计算。在一些工业应用场景中对性能的要求较高，若不能有效压缩则无法实际应用。因此，我们基于数据蒸馏技术构建了UIE Slim数据蒸馏系统。其原理是通过数据作为桥梁，将UIE模型的知识迁移到封闭域信息抽取小模型，以达到精度损失较小的情况下却能达到大幅度预测速度提升的效果。
+
+#### UIE数据蒸馏三步
+
+- **Step 1**: 使用UIE模型对标注数据进行finetune，得到Teacher Model。
+
+- **Step 2**: 用户提供大规模无标注数据，需与标注数据同源。使用Taskflow UIE对无监督数据进行预测。
+
+- **Step 3**: 使用标注数据以及步骤2得到的合成数据训练出封闭域Student Model。
+
+## UIE Finetune
+
+参考[UIE关系抽取微调](../README.md)完成模型微调，得到``../checkpoint/model_best``。
+
+## 离线蒸馏
+
+#### 通过训练好的UIE定制模型预测无监督数据的标签
+
+```shell
+python data_distill.py \
+    --data_path ../data \
+    --save_dir student_data \
+    --task_type relation_extraction \
+    --synthetic_ratio 10 \
+    --model_path ../checkpoint/model_best
+```
+
+**NOTE**：schema需要根据标注数据在`data_distill.py`中进行配置，且schema需要包含标注数据中的所有标签类型。
+
+可配置参数说明：
+
+- `data_path`: 标注数据（`doccano_ext.json`）及无监督文本（`unlabeled_data.txt`）路径。
+- `model_path`: 训练好的UIE定制模型路径。
+- `save_dir`: 学生模型训练数据保存路径。
+- `synthetic_ratio`: 控制合成数据的比例。最大合成数据数量=synthetic_ratio*标注数据数量。
+- `platform`: 标注数据的所使用的标注平台，可选有`doccano`，`label_studio`，默认为`label_studio`。
+- `task_type`: 选择任务类型，可选有`entity_extraction`，`relation_extraction`，`event_extraction`和`opinion_extraction`。因为是封闭域抽取，不同任务的后处理逻辑不同，因此需指定任务类型。
+- `seed`: 随机种子，默认为1000。
+
+#### 老师模型评估
+
+UIE微调阶段针对UIE训练格式数据评估模型效果（该评估方式非端到端评估，非关系抽取或事件抽取的标准评估方式），可通过以下评估脚本进行端到端评估。
+
+```shell
+python evaluate_teacher.py \
+    --task_type relation_extraction \
+    --test_path ./student_data/dev_data.json \
+    --label_maps_path ./student_data/label_maps.json \
+    --model_path ../checkpoint/model_best
+```
+
+可配置参数说明：
+
+- `model_path`: 训练好的UIE定制模型路径。
+- `test_path`: 测试数据集路径。
+- `label_maps_path`: 学生模型标签字典。
+- `batch_size`: 批处理大小，默认为8。
+- `max_seq_len`: 最大文本长度，默认为256。
+- `task_type`: 选择任务类型，可选有`entity_extraction`，`relation_extraction`，`event_extraction`和`opinion_extraction`。因为是封闭域信息抽取的评估，需指定任务类型。
+
+
+#### 学生模型训练
+
+```shell
+python train.py \
+    --task_type relation_extraction \
+    --train_path student_data/train_data.json \
+    --dev_path student_data/dev_data.json \
+    --label_maps_path student_data/label_maps.json \
+    --num_epochs 50 \
+    --encoder ernie-3.0-mini-zh
+```
+
+可配置参数说明：
+
+- `train_path`: 训练集文件路径。
+- `dev_path`: 验证集文件路径。
+- `batch_size`: 批处理大小，默认为16。
+- `learning_rate`: 学习率，默认为3e-5。
+- `save_dir`: 模型存储路径，默认为`./checkpoint`。
+- `max_seq_len`: 最大文本长度，默认为256。
+- `weight_decay`: 表示AdamW优化器中使用的 weight_decay 的系数。
+- `warmup_proportion`: 学习率warmup策略的比例，如果0.1，则学习率会在前10%训练step的过程中从0慢慢增长到learning_rate, 而后再缓慢衰减，默认为0.0。
+- `num_epochs`: 训练轮数，默认为100。
+- `seed`: 随机种子，默认为1000。
+- `encoder`: 选择学生模型的模型底座，默认为`ernie-3.0-mini-zh`。
+- `task_type`: 选择任务类型，可选有`entity_extraction`，`relation_extraction`，`event_extraction`和`opinion_extraction`。因为是封闭域信息抽取，需指定任务类型。
+- `logging_steps`: 日志打印的间隔steps数，默认10。
+- `eval_steps`: evaluate的间隔steps数，默认200。
+- `device`: 选用什么设备进行训练，可选cpu或gpu。
+- `init_from_ckpt`: 可选，模型参数路径，热启动模型训练；默认为None。
+
+#### 学生模型评估
+
+```shell
+python evaluate.py \
+    --model_path ./checkpoint/model_best \
+    --test_path student_data/dev_data.json \
+    --task_type relation_extraction \
+    --label_maps_path student_data/label_maps.json \
+    --encoder ernie-3.0-mini-zh
+```
+
+可配置参数说明：
+
+- `model_path`: 训练好的UIE定制模型路径。
+- `test_path`: 测试数据集路径。
+- `label_maps_path`: 学生模型标签字典。
+- `batch_size`: 批处理大小，默认为8。
+- `max_seq_len`: 最大文本长度，默认为256。
+- `encoder`: 选择学生模型的模型底座，默认为`ernie-3.0-mini-zh`。
+- `task_type`: 选择任务类型，可选有`entity_extraction`，`relation_extraction`，`event_extraction`和`opinion_extraction`。因为是封闭域信息抽取的评估，需指定任务类型。
+
+## Taskflow部署学生模型
+
+- 通过Taskflow一键部署封闭域信息抽取模型，`task_path`为学生模型路径。
+
+```python
+>>> from pprint import pprint
+>>> from paddlenlp import Taskflow
+
+>>> my_ie = Taskflow("information_extraction", model="uie-data-distill-gp", task_path="checkpoint/model_best/") # Schema is fixed in closed-domain information extraction
+>>> pprint(my_ie("威尔哥（Virgo）减速炸弹是由瑞典FFV军械公司专门为瑞典皇家空军的攻击机实施低空高速轰炸而研制，1956年开始研制，1963年进入服役，装备于A32“矛盾”、A35“龙”、和AJ134“雷”攻击机，主要用于攻击登陆艇、停放的飞机、高炮、野战火炮、轻型防护装甲车辆以及有生力量。"))
+[{'武器名称': [{'end': 14,
+            'probability': 0.9976037,
+            'relations': {'产国': [{'end': 18,
+                                  'probability': 0.9988706,
+                                  'relations': {},
+                                  'start': 16,
+                                  'text': '瑞典'}],
+                          '研发单位': [{'end': 25,
+                                    'probability': 0.9978277,
+                                    'relations': {},
+                                    'start': 18,
+                                    'text': 'FFV军械公司'}],
+                          '类型': [{'end': 14,
+                                  'probability': 0.99837446,
+                                  'relations': {},
+                                  'start': 12,
+                                  'text': '炸弹'}]},
+            'start': 0,
+            'text': '威尔哥（Virgo）减速炸弹'}]}]
+```
+
+
+# References
+
+- **[GlobalPointer](https://kexue.fm/search/globalpointer/)**
+
+- **[GPLinker](https://kexue.fm/archives/8888)**
+
+- **[JunnYu/GPLinker_pytorch](https://github.com/JunnYu/GPLinker_pytorch)**
+
+- **[CBLUE](https://github.com/CBLUEbenchmark/CBLUE)**

applications/information_extraction/text/data_distill/README_en.md

@@ -0,0 +1,154 @@
+# UIE Slim data distillation
+
+While UIE has powerful zero-shot extraction capabilities, its prompting structure requires significant compute to serve in real time. Some industrial application scenarios have high inference performance requirements and the model cannot go into production without being effectively compressed. We built the UIE Slim Data Distillation with knowledge distillation techniques. The principle is to use the data as a bridge to transfer the knowledge of the UIE model to the smaller closed-domain information extraction model in order to achieve speedup inference significantly with minimal loss to accuracy.
+
+#### Three steps of UIE data distillation
+
+- **Step 1**: Finetune the UIE model on the labeled data to get the Teacher Model.
+
+- **Step 2**: Process the user-provided unlabeled data and run inference with Taskflow UIE.
+
+- **Step 3**: Use the labeled data and the inference results obtained in step 2 to train a closed-domain Student Model.
+
+## UIE Finetune
+
+Refer to [UIE relationship extraction fine-tuning](../README.md) to complete the model fine-tuning and get ``../checkpoint/model_best``.
+
+## Offline Distillation
+
+#### Predict the label of unsupervised data through the trained UIE custom model
+
+```shell
+python data_distill.py \
+     --data_path ../data \
+     --save_dir student_data \
+     --task_type relation_extraction \
+     --synthetic_ratio 10 \
+     --model_path ../checkpoint/model_best
+```
+
+**NOTE**: The schema needs to be configured in `data_distill.py` according to the label data, and the schema needs to contain all label types in the label data.
+
+Description of configurable parameters:
+
+- `data_path`: Path to labeled data (`doccano_ext.json`) and unsupervised text (`unlabeled_data.txt`).
+- `model_path`: The path of the trained UIE custom model.
+- `save_dir`: The path to save the training data of the student model.
+- `synthetic_ratio`: Controls the ratio of synthetic data. The maximum number of synthetic data=synthetic_ratio*number of labeled data.
+- `platform`: The labeling platform used to label data, optional are `doccano`, `label_studio`, the default is `label_studio`.
+- `task_type`: Select the task type, optional are `entity_extraction`, `relation_extraction`, `event_extraction` and `opinion_extraction`. Because it is a closed-domain extraction, the post-processing logic of different tasks is different, so the task type needs to be specified.
+- `seed`: random seed, default is 1000.
+
+#### Teacher model evaluation
+
+In the UIE fine-tuning stage, the model performance is evaluated on UIE training format data, which is not a standard end-to-end evaluation method for relation extraction or event extraction. The end-to-end evaluation can be performed through the following evaluation script.
+
+```shell
+python evaluate_teacher.py \
+     --task_type relation_extraction \
+     --test_path ./student_data/dev_data.json\
+     --label_maps_path ./student_data/label_maps.json \
+     --model_path ../checkpoint/model_best
+```
+
+Description of configurable parameters:
+
+- `model_path`: The path of the trained UIE custom model.
+- `test_path`: test dataset path.
+- `label_maps_path`: dictionary of student model labels.
+- `batch_size`: batch size, default is 8.
+- `max_seq_len`: Maximum text length, default is 256.
+- `task_type`: Select the task type, optional are `entity_extraction`, `relation_extraction`, `event_extraction` and `opinion_extraction`. Because it is an evaluation of closed-domain information extraction, the task type needs to be specified.
+
+
+#### Student model training
+
+```shell
+python train.py\
+     --task_type relation_extraction \
+     --train_path student_data/train_data.json \
+     --dev_path student_data/dev_data.json \
+     --label_maps_path student_data/label_maps.json \
+     --num_epochs 50 \
+     --encoder ernie-3.0-mini-zh
+```
+
+Description of configurable parameters:
+
+- `train_path`: training set file path.
+- `dev_path`: Validation set file path.
+- `batch_size`: batch size, default is 16.
+- `learning_rate`: Learning rate, default is 3e-5.
+- `save_dir`: model storage path, the default is `./checkpoint`.
+- `max_seq_len`: Maximum text length, default is 256.
+- `weight_decay`: Indicates the coefficient of weight_decay used in the AdamW optimizer.
+- `warmup_proportion`: The proportion of the learning rate warmup strategy. If it is 0.1, the learning rate will slowly increase from 0 to learning_rate during the first 10% training step, and then slowly decay. The default is 0.0.
+- `num_epochs`: The number of training epochs, the default is 100.
+- `seed`: random seed, default is 1000.
+- `encoder`: select the model base of the student model, the default is `ernie-3.0-mini-zh`.
+- `task_type`: Select the task type, optional are `entity_extraction`, `relation_extraction`, `event_extraction` and `opinion_extraction`. Because it is closed-domain information extraction, the task type needs to be specified.
+- `logging_steps`: The interval steps of log printing, the default is 10.
+- `eval_steps`: The interval steps of evaluate, the default is 200.
+- `device`: What device to choose for training, optional cpu or gpu.
+- `init_from_ckpt`: optional, model parameter path, hot start model training; default is None.
+
+#### Student model evaluation
+
+```shell
+python evaluate.py \
+     --model_path ./checkpoint/model_best \
+     --test_path student_data/dev_data.json \
+     --task_type relation_extraction \
+     --label_maps_path student_data/label_maps.json \
+     --encoder ernie-3.0-mini-zh
+```
+
+Description of configurable parameters:
+
+- `model_path`: The path of the trained UIE custom model.
+- `test_path`: test dataset path.
+- `label_maps_path`: dictionary of student model labels.
+- `batch_size`: batch size, default is 8.
+- `max_seq_len`: Maximum text length, default is 256.
+- `encoder`: select the model base of the student model, the default is `ernie-3.0-mini-zh`.
+- `task_type`: Select the task type, optional are `entity_extraction`, `relation_extraction`, `event_extraction` and `opinion_extraction`. Because it is an evaluation of closed-domain information extraction, the task type needs to be specified.
+
+## Student model deployment
+
+- Fast deployment of the closed-domain information extraction model through Taskflow, `task_path` is the path of the student model.
+
+```python
+>>> from pprint import pprint
+>>> from paddlenlp import Taskflow
+
+>>> my_ie = Taskflow("information_extraction", model="uie-data-distill-gp", task_path="checkpoint/model_best/") # Schema is fixed in closed-domain information extraction
+>>> pprint(my_ie("Virgo deceleration bomb was developed by the Swedish FFV Ordnance Company specially for the attack aircraft of the Swedish Royal Air Force to carry out low-altitude and high-speed bombing. It was developed in 1956 and entered service in 1963. It is equipped on the A32 "Contradiction", A35 "Dragon", and AJ134 "Thunder" attack aircraft are mainly used to attack landing craft, parked aircraft, anti-aircraft artillery, field artillery, light armored vehicles and active forces."))
+[{'weapon name': [{'end': 14,
+             'probability': 0.9976037,
+             'relations': {'country of origin': [{'end': 18,
+                                   'probability': 0.9988706,
+                                   'relations': {},
+                                   'start': 16,
+                                   'text': 'Sweden'}],
+                           'R&D unit': [{'end': 25,
+                                     'probability': 0.9978277,
+                                     'relations': {},
+                                     'start': 18,
+                                     'text': 'FFV Ordnance Company'}],
+                           'type': [{'end': 14,
+                                   'probability': 0.99837446,
+                                   'relations': {},
+                                   'start': 12,
+                                   'text': 'bomb'}]},
+             'start': 0,
+             'text': 'Virgo slowing bomb'}]}]
+```
+
+
+# References
+
+- **[GlobalPointer](https://kexue.fm/search/globalpointer/)**
+
+- **[GPLinker](https://kexue.fm/archives/8888)**
+
+- **[JunnYu/GPLinker_pytorch](https://github.com/JunnYu/GPLinker_pytorch**

applications/information_extraction/text/data_distill/criterion.py

@@ -1,3 +1,4 @@
+# coding=utf-8
 # Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
@@ -18,15 +19,12 @@ import paddle.nn as nn
 
 
 class Criterion(nn.Layer):
-    '''Criterion for GPNet'''
+    """Criterion for GPNet"""
 
     def __init__(self, mask_zero=True):
         self.mask_zero = mask_zero
 
-    def _sparse_multilabel_categorical_crossentropy(self,
-                                                    y_true,
-                                                    y_pred,
-                                                    mask_zero=False):
+    def _sparse_multilabel_categorical_crossentropy(self, y_true, y_pred, mask_zero=False):
         """Sparse multi-label categorical cross entropy
         reference to "https://kexue.fm/archives/7359".
         """
@@ -52,10 +50,7 @@ class Criterion(nn.Layer):
         shape = y_pred.shape
         y_true = y_true[..., 0] * shape[2] + y_true[..., 1]
         # bs, nclass, seqlen * seqlen
-        y_pred = paddle.reshape(y_pred,
-                                shape=[shape[0], -1,
-                                       np.prod(shape[2:])])
+        y_pred = paddle.reshape(y_pred, shape=[shape[0], -1, np.prod(shape[2:])])
 
-        loss = self._sparse_multilabel_categorical_crossentropy(
-            y_true, y_pred, self.mask_zero)
+        loss = self._sparse_multilabel_categorical_crossentropy(y_true, y_pred, self.mask_zero)
         return loss.sum(axis=1).mean()

applications/information_extraction/text/data_distill/data_collator.py

@@ -0,0 +1,86 @@
+# coding=utf-8
+# Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from dataclasses import dataclass
+from typing import Dict, List, Optional, Union
+
+import paddle
+
+from paddlenlp.transformers.tokenizer_utils_base import (
+    PaddingStrategy,
+    PretrainedTokenizerBase,
+)
+
+ignore_list = ["offset_mapping", "text"]
+
+
+@dataclass
+class DataCollator:
+    tokenizer: PretrainedTokenizerBase
+    padding: Union[bool, str, PaddingStrategy] = True
+    max_length: Optional[int] = None
+    label_maps: Optional[dict] = None
+    task_type: Optional[str] = None
+
+    def __call__(self, features: List[Dict[str, Union[List[int], paddle.Tensor]]]) -> Dict[str, paddle.Tensor]:
+        labels = [feature["labels"] for feature in features] if "labels" in features[0].keys() else None
+        new_features = [{k: v for k, v in f.items() if k not in ["labels"] + ignore_list} for f in features]
+
+        batch = self.tokenizer.pad(
+            new_features,
+            padding=self.padding,
+        )
+
+        batch = [paddle.to_tensor(batch[k]) for k in batch.keys()]
+
+        if labels is None:  # for test
+            if "offset_mapping" in features[0].keys():
+                batch.append([feature["offset_mapping"] for feature in features])
+            if "text" in features[0].keys():
+                batch.append([feature["text"] for feature in features])
+            return batch
+
+        bs = batch[0].shape[0]
+        if self.task_type == "entity_extraction":
+            # Ensure the dimension is greater or equal to 1
+            max_ent_num = max(max([len(lb["ent_labels"]) for lb in labels]), 1)
+            num_ents = len(self.label_maps["entity2id"])
+            batch_entity_labels = paddle.zeros(shape=[bs, num_ents, max_ent_num, 2], dtype="int64")
+            for i, lb in enumerate(labels):
+                for eidx, (l, eh, et) in enumerate(lb["ent_labels"]):
+                    batch_entity_labels[i, l, eidx, :] = paddle.to_tensor([eh, et])
+
+            batch.append([batch_entity_labels])
+        else:
+            # Ensure the dimension is greater or equal to 1
+            max_ent_num = max(max([len(lb["ent_labels"]) for lb in labels]), 1)
+            max_spo_num = max(max([len(lb["rel_labels"]) for lb in labels]), 1)
+            num_ents = len(self.label_maps["entity2id"])
+            if "relation2id" in self.label_maps.keys():
+                num_rels = len(self.label_maps["relation2id"])
+            else:
+                num_rels = len(self.label_maps["sentiment2id"])
+            batch_entity_labels = paddle.zeros(shape=[bs, num_ents, max_ent_num, 2], dtype="int64")
+            batch_head_labels = paddle.zeros(shape=[bs, num_rels, max_spo_num, 2], dtype="int64")
+            batch_tail_labels = paddle.zeros(shape=[bs, num_rels, max_spo_num, 2], dtype="int64")
+
+            for i, lb in enumerate(labels):
+                for eidx, (l, eh, et) in enumerate(lb["ent_labels"]):
+                    batch_entity_labels[i, l, eidx, :] = paddle.to_tensor([eh, et])
+                for spidx, (sh, st, p, oh, ot) in enumerate(lb["rel_labels"]):
+                    batch_head_labels[i, p, spidx, :] = paddle.to_tensor([sh, oh])
+                    batch_tail_labels[i, p, spidx, :] = paddle.to_tensor([st, ot])
+            batch.append([batch_entity_labels, batch_head_labels, batch_tail_labels])
+        return batch

applications/information_extraction/text/data_distill/data_distill.py

@@ -0,0 +1,128 @@
+# coding=utf-8
+# Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import argparse
+import json
+import math
+import os
+import random
+
+from tqdm import tqdm
+from utils import anno2distill, schema2label_maps, set_seed, synthetic2distill
+
+from paddlenlp import Taskflow
+from paddlenlp.utils.log import logger
+
+
+def do_data_distill():
+    set_seed(args.seed)
+
+    # Generate closed-domain label maps
+    if not os.path.exists(args.save_dir):
+        os.mkdir(args.save_dir)
+    label_maps = schema2label_maps(args.task_type, schema=args.schema)
+    label_maps_path = os.path.join(args.save_dir, "label_maps.json")
+
+    # Save closed-domain label maps file
+    with open(label_maps_path, "w", encoding="utf-8") as fp:
+        fp.write(json.dumps(label_maps, ensure_ascii=False))
+
+    # Load doccano file and convert to distill format
+    sample_index = json.loads(
+        open(os.path.join(args.data_path, "sample_index.json"), "r", encoding="utf-8").readline()
+    )
+
+    train_ids = sample_index["train_ids"]
+    dev_ids = sample_index["dev_ids"]
+    test_ids = sample_index["test_ids"]
+
+    if args.platform == "label_studio":
+        with open(os.path.join(args.data_path, "label_studio.json"), "r", encoding="utf-8") as fp:
+            json_lines = json.loads(fp.read())
+    elif args.platform == "doccano":
+        json_lines = []
+        with open(os.path.join(args.data_path, "doccano_ext.json"), "r", encoding="utf-8") as fp:
+            for line in fp:
+                json_lines.append(json.loads(line))
+    else:
+        raise ValueError("Unsupported annotation platform!")
+
+    train_lines = [json_lines[i] for i in train_ids]
+    train_lines = anno2distill(train_lines, args.task_type, label_maps, args.platform)
+
+    dev_lines = [json_lines[i] for i in dev_ids]
+    dev_lines = anno2distill(dev_lines, args.task_type, label_maps, args.platform)
+
+    test_lines = [json_lines[i] for i in test_ids]
+    test_lines = anno2distill(test_lines, args.task_type, label_maps, args.platform)
+
+    # Load trained UIE model
+    uie = Taskflow("information_extraction", schema=args.schema, task_path=args.model_path)
+
+    if args.synthetic_ratio > 0:
+        # Generate synthetic data
+        texts = open(os.path.join(args.data_path, "unlabeled_data.txt"), "r", encoding="utf-8").readlines()
+
+        actual_ratio = math.ceil(len(texts) / len(train_lines))
+        if actual_ratio <= args.synthetic_ratio or args.synthetic_ratio == -1:
+            infer_texts = texts
+        else:
+            idxs = random.sample(range(0, len(texts)), args.synthetic_ratio * len(train_lines))
+            infer_texts = [texts[i] for i in idxs]
+
+        infer_results = []
+        for text in tqdm(infer_texts, desc="Predicting: ", leave=False):
+            infer_results.extend(uie(text))
+
+        train_synthetic_lines = synthetic2distill(infer_texts, infer_results, args.task_type)
+
+        # Concat origin and synthetic data
+        train_lines.extend(train_synthetic_lines)
+
+    def _save_examples(save_dir, file_name, examples):
+        count = 0
+        save_path = os.path.join(save_dir, file_name)
+        with open(save_path, "w", encoding="utf-8") as f:
+            for example in examples:
+                f.write(json.dumps(example, ensure_ascii=False) + "\n")
+                count += 1
+        logger.info("Save %d examples to %s." % (count, save_path))
+
+    _save_examples(args.save_dir, "train_data.json", train_lines)
+    _save_examples(args.save_dir, "dev_data.json", dev_lines)
+    _save_examples(args.save_dir, "test_data.json", test_lines)
+
+
+if __name__ == "__main__":
+    # yapf: disable
+    parser = argparse.ArgumentParser()
+
+    parser.add_argument("--data_path", default="../data", type=str, help="The directory for labeled data with doccano format and the large scale unlabeled data.")
+    parser.add_argument("--model_path", type=str, default="../checkpoint/model_best", help="The path of saved model that you want to load.")
+    parser.add_argument("--save_dir", default="./distill_task", type=str, help="The path of data that you wanna save.")
+    parser.add_argument("--synthetic_ratio", default=10, type=int, help="The ratio of labeled and synthetic samples.")
+    parser.add_argument("--task_type", choices=['relation_extraction', 'event_extraction', 'entity_extraction', 'opinion_extraction'], default="entity_extraction", type=str, help="Select the training task type.")
+    parser.add_argument("--seed", type=int, default=1000, help="Random seed for initialization")
+    parser.add_argument("--platform", choices=['doccano', 'label_studio'], type=str, default="label_studio", help="Select the annotation platform.")
+
+    args = parser.parse_args()
+    # yapf: enable
+
+    # Define your schema here
+    schema = {"武器名称": ["产国", "类型", "研发单位"]}
+
+    args.schema = schema
+
+    do_data_distill()

applications/information_extraction/text/data_distill/deploy/simple_serving/README.md

@@ -0,0 +1,58 @@
+# 基于PaddleNLP SimpleServing 的服务化部署
+
+## 目录
+- [环境准备](#环境准备)
+- [Server服务启动](#Server服务启动)
+- [Client请求启动](#Client请求启动)
+- [服务化自定义参数](#服务化自定义参数)
+
+## 环境准备
+使用有SimpleServing功能的PaddleNLP版本(或者最新的develop版本)
+
+```shell
+pip install paddlenlp >= 2.4.4
+```
+
+
+## Server服务启动
+
+```bash
+paddlenlp server server:app --workers 1 --host 0.0.0.0 --port 8189
+```
+
+## Client请求启动
+
+```bash
+python client.py
+```
+
+## 服务化自定义参数
+
+### Server 自定义参数
+#### schema替换
+```python
+# Default schema
+schema = {"武器名称": ["产国", "类型", "研发单位"]}
+```
+
+#### 设置模型路径
+```
+# Default task_path
+uie = Taskflow('information_extraction', model='uie-data-distill-gp', task_path='../../checkpoint/model_best/', schema=schema)
+```
+
+#### 多卡服务化预测
+PaddleNLP SimpleServing 支持多卡负载均衡预测，主要在服务化注册的时候，注册两个Taskflow的task即可，下面是示例代码
+```
+uie1 = Taskflow('information_extraction', model='uie-data-distill-gp', task_path='../../checkpoint/model_best/', schema=schema, device_id=0)
+uie2 = Taskflow('information_extraction', model='uie-data-distill-gp', task_path='../../checkpoint/model_best/', schema=schema, device_id=1)
+service.register_taskflow('uie', [uie1, uie2])
+```
+
+### Client 自定义参数
+
+```python
+# Changed to input texts you wanted
+texts = ['威尔哥（Virgo）减速炸弹是由瑞典FFV军械公司专门为瑞典皇家空军的攻击机实施低空高速轰炸而研制，1956年开始研制，1963年进入服役，装备于A32“矛盾”、A35“龙”、和AJ134“雷”攻击机，主要用于攻击登陆艇、停放的飞机、高炮、野战火炮、轻型防护装甲车辆以及有生力量。']
+
+```

applications/information_extraction/text/data_distill/deploy/simple_serving/README_en.md

@@ -0,0 +1,64 @@
+# Service deployment based on PaddleNLP SimpleServing
+
+- [Environment Preparation](#1)
+- [Server](#2)
+- [Client](#3)
+- [Service Custom Parameters](#4)
+
+<a name="1"></a>
+
+## Environment Preparation
+Use the PaddleNLP version with SimpleServing function (or the latest develop version)
+
+```shell
+pip install paddlenlp >= 2.4.4
+```
+
+<a name="2"></a>
+
+## Server
+
+```bash
+paddlenlp server server:app --workers 1 --host 0.0.0.0 --port 8189
+```
+
+<a name="3"></a>
+
+## Client
+
+```bash
+python client.py
+```
+
+<a name="4"></a>
+
+## Service Custom Parameters
+
+### Server Custom Parameters
+
+#### schema replacement
+```python
+# Default schema
+schema = {"Weapon Name": ["Country of Production", "Type", "R&D Unit"]}
+```
+
+#### Set model path
+```
+# Default task_path
+uie = Taskflow('information_extraction', model='uie-data-distill-gp', task_path='../../checkpoint/model_best/', schema=schema)
+```
+
+#### Doka Service Prediction
+PaddleNLP SimpleServing supports multi-card load balancing prediction, mainly during service registration, just register two Taskflow tasks, the following is the sample code
+```
+uie1 = Taskflow('information_extraction', task_path='../../checkpoint/model_best/', schema=schema, device_id=0)
+uie2 = Taskflow('information_extraction', task_path='../../checkpoint/model_best/', schema=schema, device_id=1)
+service. register_taskflow('uie', [uie1, uie2])
+```
+
+### Client Custom Parameters
+
+```python
+# Changed to input texts you wanted
+texts = ['威尔哥（Virgo）减速炸弹是由瑞典FFV军械公司专门为瑞典皇家空军的攻击机实施低空高速轰炸而研制，1956年开始研制，1963年进入服役，装备于A32“矛盾”、A35“龙”、和AJ134“雷”攻击机，主要用于攻击登陆艇、停放的飞机、高炮、野战火炮、轻型防护装甲车辆以及有生力量。']
+```

applications/information_extraction/text/data_distill/deploy/simple_serving/client.py

@@ -0,0 +1,29 @@
+# Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import json
+
+import requests
+
+url = "http://0.0.0.0:8189/taskflow/uie"
+
+headers = {"Content-Type": "application/json"}
+texts = [
+    "威尔哥（Virgo）减速炸弹是由瑞典FFV军械公司专门为瑞典皇家空军的攻击机实施低空高速轰炸而研制，1956年开始研制，1963年进入服役，装备于A32“矛盾”、A35“龙”、和AJ134“雷”攻击机，主要用于攻击登陆>艇、停放的飞机、高炮、野战火炮、轻型防护装甲车辆以及有生力量。"
+]
+
+data = {"data": {"text": texts}}
+r = requests.post(url=url, headers=headers, data=json.dumps(data))
+datas = json.loads(r.text)
+print(datas)

applications/information_extraction/text/data_distill/deploy/simple_serving/server.py

@@ -0,0 +1,25 @@
+# Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from paddlenlp import SimpleServer, Taskflow
+
+# The schema changed to your defined schema
+schema = {"武器名称": ["产国", "类型", "研发单位"]}
+# The task path changed to your best model path
+uie = Taskflow(
+    "information_extraction", model="uie-data-distill-gp", schema=schema, task_path="../../checkpoint/model_best/"
+)
+# If you want to define the finetuned uie service
+app = SimpleServer()
+app.register_taskflow("taskflow/uie", uie)

applications/information_extraction/text/data_distill/evaluate.py

@@ -0,0 +1,96 @@
+# coding=utf-8
+# Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import argparse
+import os
+
+import paddle
+from metric import get_eval
+from tqdm import tqdm
+from utils import create_dataloader, get_label_maps, postprocess, reader
+
+from paddlenlp.datasets import load_dataset
+from paddlenlp.layers import (
+    GlobalPointerForEntityExtraction,
+    GPLinkerForRelationExtraction,
+)
+from paddlenlp.transformers import AutoModel, AutoTokenizer
+from paddlenlp.utils.log import logger
+
+
+@paddle.no_grad()
+def evaluate(model, dataloader, label_maps, task_type="relation_extraction"):
+    model.eval()
+    all_preds = ([], []) if task_type in ["opinion_extraction", "relation_extraction", "event_extraction"] else []
+    for batch in tqdm(dataloader, desc="Evaluating: ", leave=False):
+        input_ids, attention_masks, offset_mappings, texts = batch
+        logits = model(input_ids, attention_masks)
+        batch_outputs = postprocess(logits, offset_mappings, texts, label_maps, task_type)
+        if isinstance(batch_outputs, tuple):
+            all_preds[0].extend(batch_outputs[0])  # Entity output
+            all_preds[1].extend(batch_outputs[1])  # Relation output
+        else:
+            all_preds.extend(batch_outputs)
+    eval_results = get_eval(all_preds, dataloader.dataset.raw_data, task_type)
+    model.train()
+    return eval_results
+
+
+def do_eval():
+    label_maps = get_label_maps(args.task_type, args.label_maps_path)
+
+    tokenizer = AutoTokenizer.from_pretrained(args.encoder)
+    encoder = AutoModel.from_pretrained(args.encoder)
+    if args.task_type == "entity_extraction":
+        model = GlobalPointerForEntityExtraction(encoder, label_maps)
+    else:
+        model = GPLinkerForRelationExtraction(encoder, label_maps)
+
+    if args.model_path:
+        state_dict = paddle.load(os.path.join(args.model_path, "model_state.pdparams"))
+        model.set_dict(state_dict)
+
+    test_ds = load_dataset(reader, data_path=args.test_path, lazy=False)
+
+    test_dataloader = create_dataloader(
+        test_ds,
+        tokenizer,
+        max_seq_len=args.max_seq_len,
+        batch_size=args.batch_size,
+        label_maps=label_maps,
+        mode="test",
+        task_type=args.task_type,
+    )
+
+    eval_result = evaluate(model, test_dataloader, label_maps, task_type=args.task_type)
+    logger.info("Evaluation precision: " + str(eval_result))
+
+
+if __name__ == "__main__":
+    # yapf: disable
+    parser = argparse.ArgumentParser()
+
+    parser.add_argument("--model_path", type=str, default=None, help="The path of saved model that you want to load.")
+    parser.add_argument("--test_path", type=str, default=None, help="The path of test set.")
+    parser.add_argument("--encoder", default="ernie-3.0-mini-zh", type=str, help="Select the pretrained encoder model for GP.")
+    parser.add_argument("--label_maps_path", default="./ner_data/label_maps.json", type=str, help="The file path of the labels dictionary.")
+    parser.add_argument("--batch_size", type=int, default=16, help="Batch size per GPU/CPU for training.")
+    parser.add_argument("--max_seq_len", type=int, default=128, help="The maximum total input sequence length after tokenization.")
+    parser.add_argument("--task_type", choices=['relation_extraction', 'event_extraction', 'entity_extraction', 'opinion_extraction'], default="entity_extraction", type=str, help="Select the training task type.")
+
+    args = parser.parse_args()
+    # yapf: enable
+
+    do_eval()

applications/information_extraction/text/data_distill/evaluate_teacher.py

@@ -0,0 +1,95 @@
+# coding=utf-8
+# Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import argparse
+
+import paddle
+from metric import get_eval
+from tqdm import tqdm
+from utils import create_dataloader, get_label_maps, reader, synthetic2distill
+
+from paddlenlp import Taskflow
+from paddlenlp.datasets import load_dataset
+from paddlenlp.transformers import AutoTokenizer
+from paddlenlp.utils.log import logger
+
+
+@paddle.no_grad()
+def evaluate(uie, dataloader, task_type="relation_extraction"):
+    all_preds = ([], []) if task_type in ["opinion_extraction", "relation_extraction", "event_extraction"] else []
+
+    infer_results = []
+    all_texts = []
+    for batch in tqdm(dataloader, desc="Evaluating: ", leave=False):
+        _, _, _, texts = batch
+        all_texts.extend(texts)
+        infer_results.extend(uie(texts))
+
+    infer_results = synthetic2distill(all_texts, infer_results, task_type)
+
+    for res in infer_results:
+        if task_type == "entity_extraction":
+            all_preds.append(res["entity_list"])
+        else:
+            all_preds[0].append(res["entity_list"])
+            all_preds[1].append(res["spo_list"])
+
+    eval_results = get_eval(all_preds, dataloader.dataset.raw_data, task_type)
+    return eval_results
+
+
+def do_eval():
+    # Load trained UIE model
+    uie = Taskflow("information_extraction", schema=args.schema, batch_size=args.batch_size, task_path=args.model_path)
+
+    label_maps = get_label_maps(args.task_type, args.label_maps_path)
+
+    tokenizer = AutoTokenizer.from_pretrained("ernie-3.0-base-zh")
+
+    test_ds = load_dataset(reader, data_path=args.test_path, lazy=False)
+
+    test_dataloader = create_dataloader(
+        test_ds,
+        tokenizer,
+        max_seq_len=args.max_seq_len,
+        batch_size=args.batch_size,
+        label_maps=label_maps,
+        mode="test",
+        task_type=args.task_type,
+    )
+
+    eval_result = evaluate(uie, test_dataloader, task_type=args.task_type)
+    logger.info("Evaluation precision: " + str(eval_result))
+
+
+if __name__ == "__main__":
+    # yapf: disable
+    parser = argparse.ArgumentParser()
+
+    parser.add_argument("--model_path", type=str, default=None, help="The path of saved model that you want to load.")
+    parser.add_argument("--test_path", type=str, default=None, help="The path of test set.")
+    parser.add_argument("--label_maps_path", default="./ner_data/label_maps.json", type=str, help="The file path of the labels dictionary.")
+    parser.add_argument("--batch_size", type=int, default=8, help="Batch size per GPU/CPU for training.")
+    parser.add_argument("--max_seq_len", type=int, default=256, help="The maximum total input sequence length after tokenization.")
+    parser.add_argument("--task_type", choices=['relation_extraction', 'event_extraction', 'entity_extraction', 'opinion_extraction'], default="entity_extraction", type=str, help="Select the training task type.")
+
+    args = parser.parse_args()
+    # yapf: enable
+
+    schema = {"武器名称": ["产国", "类型", "研发单位"]}
+
+    args.schema = schema
+
+    do_eval()

applications/information_extraction/text/data_distill/metric.py

@@ -0,0 +1,72 @@
+# coding=utf-8
+# Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+
+def get_eval(all_preds, raw_data, task_type):
+    if task_type == "entity_extraction":
+        ex, ey, ez = 1e-10, 1e-10, 1e-10
+        for ent_preds, data in zip(all_preds, raw_data):
+            pred_ent_set = set([tuple(p.values()) for p in ent_preds])
+            gold_ent_set = set([tuple(g.values()) for g in data["entity_list"]])
+            ex += len(pred_ent_set & gold_ent_set)
+            ey += len(pred_ent_set)
+            ez += len(gold_ent_set)
+        ent_f1 = round(2 * ex / (ey + ez), 5) if ex != 1e-10 else 0.0
+        ent_precision = round(ex / ey, 5) if ey != 1e-10 else 0.0
+        ent_recall = round(ex / ez, 5) if ez != 1e-10 else 0.0
+
+        return {
+            "entity_f1": ent_f1,
+            "entity_precision": ent_precision,
+            "entity_recall": ent_recall,
+        }
+    else:
+        all_ent_preds, all_rel_preds = all_preds
+
+        ex, ey, ez = 1e-10, 1e-10, 1e-10
+        for ent_preds, data in zip(all_ent_preds, raw_data):
+            pred_ent_set = set([tuple(p.values()) for p in ent_preds])
+            gold_ent_set = set([tuple(g.values()) for g in data["entity_list"]])
+            ex += len(pred_ent_set & gold_ent_set)
+            ey += len(pred_ent_set)
+            ez += len(gold_ent_set)
+        ent_f1 = round(2 * ex / (ey + ez), 5) if ex != 1e-10 else 0.0
+        ent_precision = round(ex / ey, 5) if ey != 1e-10 else 0.0
+        ent_recall = round(ex / ez, 5) if ez != 1e-10 else 0.0
+
+        rx, ry, rz = 1e-10, 1e-10, 1e-10
+
+        for rel_preds, raw_data in zip(all_rel_preds, raw_data):
+            pred_rel_set = set([tuple(p.values()) for p in rel_preds])
+            if task_type == "opinion_extraction":
+                gold_rel_set = set([tuple(g.values()) for g in raw_data["aso_list"]])
+            else:
+                gold_rel_set = set([tuple(g.values()) for g in raw_data["spo_list"]])
+            rx += len(pred_rel_set & gold_rel_set)
+            ry += len(pred_rel_set)
+            rz += len(gold_rel_set)
+
+        rel_f1 = round(2 * rx / (ry + rz), 5) if rx != 1e-10 else 0.0
+        rel_precision = round(rx / ry, 5) if ry != 1e-10 else 0.0
+        rel_recall = round(rx / rz, 5) if rz != 1e-10 else 0.0
+
+        return {
+            "entity_f1": ent_f1,
+            "entity_precision": ent_precision,
+            "entity_recall": ent_recall,
+            "relation_f1": rel_f1,
+            "relation_precision": rel_precision,
+            "relation_recall": rel_recall,
+        }

applications/information_extraction/text/data_distill/train.py

@@ -0,0 +1,190 @@
+# coding=utf-8
+# Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import argparse
+import os
+import time
+
+import paddle
+from criterion import Criterion
+from evaluate import evaluate
+from utils import (
+    create_dataloader,
+    criteria_map,
+    get_label_maps,
+    reader,
+    save_model_config,
+    set_seed,
+)
+
+from paddlenlp.datasets import load_dataset
+from paddlenlp.layers import (
+    GlobalPointerForEntityExtraction,
+    GPLinkerForRelationExtraction,
+)
+from paddlenlp.transformers import AutoModel, AutoTokenizer, LinearDecayWithWarmup
+from paddlenlp.utils.log import logger
+
+
+def do_train():
+    paddle.set_device(args.device)
+    rank = paddle.distributed.get_rank()
+    if paddle.distributed.get_world_size() > 1:
+        paddle.distributed.init_parallel_env()
+    set_seed(args.seed)
+
+    label_maps = get_label_maps(args.task_type, args.label_maps_path)
+
+    train_ds = load_dataset(reader, data_path=args.train_path, lazy=False)
+    dev_ds = load_dataset(reader, data_path=args.dev_path, lazy=False)
+    tokenizer = AutoTokenizer.from_pretrained(args.encoder)
+
+    train_dataloader = create_dataloader(
+        train_ds,
+        tokenizer,
+        max_seq_len=args.max_seq_len,
+        batch_size=args.batch_size,
+        label_maps=label_maps,
+        mode="train",
+        task_type=args.task_type,
+    )
+
+    dev_dataloader = create_dataloader(
+        dev_ds,
+        tokenizer,
+        max_seq_len=args.max_seq_len,
+        batch_size=args.batch_size,
+        label_maps=label_maps,
+        mode="dev",
+        task_type=args.task_type,
+    )
+
+    encoder = AutoModel.from_pretrained(args.encoder)
+    if args.task_type == "entity_extraction":
+        model = GlobalPointerForEntityExtraction(encoder, label_maps)
+    else:
+        model = GPLinkerForRelationExtraction(encoder, label_maps)
+
+    model_config = {"task_type": args.task_type, "label_maps": label_maps, "encoder": args.encoder}
+
+    num_training_steps = len(train_dataloader) * args.num_epochs
+    lr_scheduler = LinearDecayWithWarmup(args.learning_rate, num_training_steps, args.warmup_proportion)
+
+    # Generate parameter names needed to perform weight decay.
+    # All bias and LayerNorm parameters are excluded.
+    decay_params = [p.name for n, p in model.named_parameters() if not any(nd in n for nd in ["bias", "norm"])]
+    optimizer = paddle.optimizer.AdamW(
+        learning_rate=lr_scheduler,
+        parameters=model.parameters(),
+        weight_decay=args.weight_decay,
+        apply_decay_param_fun=lambda x: x in decay_params,
+    )
+
+    if args.init_from_ckpt and os.path.isfile(args.init_from_ckpt):
+        state_dict = paddle.load(args.init_from_ckpt)
+        model.set_dict(state_dict)
+
+    if paddle.distributed.get_world_size() > 1:
+        model = paddle.DataParallel(model)
+
+    criterion = Criterion()
+
+    global_step, best_f1 = 1, 0.0
+    tr_loss, logging_loss = 0.0, 0.0
+    tic_train = time.time()
+    for epoch in range(1, args.num_epochs + 1):
+        for batch in train_dataloader:
+            input_ids, attention_masks, labels = batch
+
+            logits = model(input_ids, attention_masks)
+
+            loss = sum([criterion(o, l) for o, l in zip(logits, labels)]) / 3
+
+            loss.backward()
+
+            tr_loss += loss.item()
+
+            lr_scheduler.step()
+            optimizer.step()
+            optimizer.clear_grad()
+
+            if global_step % args.logging_steps == 0 and rank == 0:
+                time_diff = time.time() - tic_train
+                loss_avg = (tr_loss - logging_loss) / args.logging_steps
+                logger.info(
+                    "global step %d, epoch: %d, loss: %.5f, speed: %.2f step/s"
+                    % (global_step, epoch, loss_avg, args.logging_steps / time_diff)
+                )
+                logging_loss = tr_loss
+                tic_train = time.time()
+
+            if global_step % args.eval_steps == 0 and rank == 0:
+                save_dir = os.path.join(args.save_dir, "model_%d" % global_step)
+                if not os.path.exists(save_dir):
+                    os.makedirs(save_dir)
+                save_param_path = os.path.join(save_dir, "model_state.pdparams")
+                paddle.save(model.state_dict(), save_param_path)
+                save_model_config(save_dir, model_config)
+                logger.disable()
+                tokenizer.save_pretrained(save_dir)
+                logger.enable()
+
+                eval_result = evaluate(model, dev_dataloader, label_maps, task_type=args.task_type)
+                logger.info("Evaluation precision: " + str(eval_result))
+
+                f1 = eval_result[criteria_map[args.task_type]]
+                if f1 > best_f1:
+                    logger.info(f"best F1 performance has been updated: {best_f1:.5f} --> {f1:.5f}")
+                    best_f1 = f1
+                    save_dir = os.path.join(args.save_dir, "model_best")
+                    if not os.path.exists(save_dir):
+                        os.makedirs(save_dir)
+                    save_param_path = os.path.join(save_dir, "model_state.pdparams")
+                    paddle.save(model.state_dict(), save_param_path)
+                    save_model_config(save_dir, model_config)
+                    logger.disable()
+                    tokenizer.save_pretrained(save_dir)
+                    logger.enable()
+                tic_train = time.time()
+
+            global_step += 1
+
+
+if __name__ == "__main__":
+    # yapf: disable
+    parser = argparse.ArgumentParser()
+
+    parser.add_argument("--train_path", default=None, type=str, help="The path of train set.")
+    parser.add_argument("--dev_path", default=None, type=str, help="The path of dev set.")
+    parser.add_argument("--batch_size", default=16, type=int, help="Batch size per GPU/CPU for training.")
+    parser.add_argument("--learning_rate", default=3e-5, type=float, help="The initial learning rate for Adam.")
+    parser.add_argument("--save_dir", default='./checkpoint', type=str, help="The output directory where the model checkpoints will be written.")
+    parser.add_argument("--max_seq_len", default=256, type=int, help="The maximum input sequence length.")
+    parser.add_argument("--label_maps_path", default="./ner_data/label_maps.json", type=str, help="The file path of the labels dictionary.")
+    parser.add_argument("--weight_decay", default=0.0, type=float, help="Weight decay rate for L2 regularizer.")
+    parser.add_argument("--warmup_proportion", default=0.0, type=float, help="Linear warmup proportion over the training process.")
+    parser.add_argument("--num_epochs", default=100, type=int, help="Number of epoches for training.")
+    parser.add_argument("--seed", default=1000, type=int, help="Random seed for initialization")
+    parser.add_argument("--encoder", default="ernie-3.0-mini-zh", type=str, help="Select the pretrained encoder model for GP.")
+    parser.add_argument("--task_type", choices=['relation_extraction', 'event_extraction', 'entity_extraction', 'opinion_extraction'], default="entity_extraction", type=str, help="Select the training task type.")
+    parser.add_argument("--logging_steps", default=10, type=int, help="The interval steps to logging.")
+    parser.add_argument("--eval_steps", default=200, type=int, help="The interval steps to evaluate model performance.")
+    parser.add_argument('--device', choices=['cpu', 'gpu'], default="gpu", help="Select which device to train model, defaults to gpu.")
+    parser.add_argument("--init_from_ckpt", default=None, type=str, help="The path of model parameters for initialization.")
+
+    args = parser.parse_args()
+    # yapf: enable
+
+    do_train()

applications/information_extraction/text/data_distill/utils.py

@@ -0,0 +1,554 @@
+# coding=utf-8
+# Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import copy
+import json
+import os
+import random
+
+import numpy as np
+import paddle
+from data_collator import DataCollator
+
+from paddlenlp.taskflow.utils import SchemaTree
+from paddlenlp.utils.log import logger
+
+criteria_map = {
+    "entity_extraction": "entity_f1",
+    "opinion_extraction": "relation_f1",  # (Aspect, Sentiment, Opinion)
+    "relation_extraction": "relation_f1",  # (Subject, Predicate, Object)
+    "event_extraction": "relation_f1",  # (Trigger, Role, Argument)
+}
+
+
+def set_seed(seed):
+    paddle.seed(seed)
+    random.seed(seed)
+    np.random.seed(seed)
+
+
+def reader(data_path):
+    with open(data_path, "r", encoding="utf-8") as f:
+        for line in f:
+            json_line = json.loads(line)
+            yield json_line
+
+
+def save_model_config(save_dir, model_config):
+    model_config_file = os.path.join(save_dir, "model_config.json")
+    with open(model_config_file, "w", encoding="utf-8") as fp:
+        fp.write(json.dumps(model_config, ensure_ascii=False, indent=2))
+
+
+def map_offset(ori_offset, offset_mapping):
+    """
+    map ori offset to token offset
+    """
+    for index, span in enumerate(offset_mapping):
+        if span[0] <= ori_offset < span[1]:
+            return index
+    return -1
+
+
+def get_label_maps(task_type="relation_extraction", label_maps_path=None):
+    with open(label_maps_path, "r", encoding="utf-8") as fp:
+        label_maps = json.load(fp)
+    if task_type == "entity_extraction":
+        entity2id = label_maps["entity2id"]
+        id2entity = {idx: t for t, idx in entity2id.items()}
+        label_maps["id2entity"] = id2entity
+    else:
+        entity2id = label_maps["entity2id"]
+        relation2id = (
+            label_maps["relation2id"]
+            if task_type in ["relation_extraction", "event_extraction"]
+            else label_maps["sentiment2id"]
+        )
+        id2entity = {idx: t for t, idx in entity2id.items()}
+        id2relation = {idx: t for t, idx in relation2id.items()}
+        label_maps["id2entity"] = id2entity
+        label_maps["id2relation"] = id2relation
+    return label_maps
+
+
+def create_dataloader(
+    dataset, tokenizer, max_seq_len=128, batch_size=1, label_maps=None, mode="train", task_type="relation_extraction"
+):
+    def tokenize_and_align_train_labels(example):
+        tokenized_inputs = tokenizer(
+            example["text"],
+            max_length=max_seq_len,
+            padding=False,
+            truncation=True,
+            return_attention_mask=True,
+            return_token_type_ids=False,
+            return_offsets_mapping=True,
+        )
+        offset_mapping = tokenized_inputs["offset_mapping"]
+
+        ent_labels = []
+        for e in example["entity_list"]:
+            _start, _end = e["start_index"], e["start_index"] + len(e["text"]) - 1
+            start = map_offset(_start, offset_mapping)
+            end = map_offset(_end, offset_mapping)
+            if start == -1 or end == -1:
+                continue
+            label = label_maps["entity2id"][e["type"]]
+            ent_labels.append([label, start, end])
+
+        outputs = {
+            "input_ids": tokenized_inputs["input_ids"],
+            "attention_mask": tokenized_inputs["attention_mask"],
+            "labels": {"ent_labels": ent_labels, "rel_labels": []},
+        }
+
+        if task_type in ["relation_extraction", "event_extraction"]:
+            rel_labels = []
+            for r in example["spo_list"]:
+                _sh, _oh = r["subject_start_index"], r["object_start_index"]
+                _st, _ot = _sh + len(r["subject"]) - 1, _oh + len(r["object"]) - 1
+                sh = map_offset(_sh, offset_mapping)
+                st = map_offset(_st, offset_mapping)
+                oh = map_offset(_oh, offset_mapping)
+                ot = map_offset(_ot, offset_mapping)
+                if sh == -1 or st == -1 or oh == -1 or ot == -1:
+                    continue
+                p = label_maps["relation2id"][r["predicate"]]
+                rel_labels.append([sh, st, p, oh, ot])
+            outputs["labels"]["rel_labels"] = rel_labels
+        elif task_type == "opinion_extraction":
+            rel_labels = []
+            for r in example["aso_list"]:
+                _ah, _oh = r["aspect_start_index"], r["opinion_start_index"]
+                _at, _ot = _ah + len(r["aspect"]) - 1, _oh + len(r["opinion"]) - 1
+                ah = map_offset(_ah, offset_mapping)
+                at = map_offset(_at, offset_mapping)
+                oh = map_offset(_oh, offset_mapping)
+                ot = map_offset(_ot, offset_mapping)
+                if ah == -1 or at == -1 or oh == -1 or ot == -1:
+                    continue
+
+                s = label_maps["sentiment2id"][r["sentiment"]]
+                rel_labels.append([ah, at, s, oh, ot])
+            outputs["labels"]["rel_labels"] = rel_labels
+        return outputs
+
+    def tokenize(example):
+        tokenized_inputs = tokenizer(
+            example["text"],
+            max_length=max_seq_len,
+            padding=False,
+            truncation=True,
+            return_attention_mask=True,
+            return_offsets_mapping=True,
+            return_token_type_ids=False,
+        )
+        tokenized_inputs["text"] = example["text"]
+        return tokenized_inputs
+
+    if mode == "train":
+        dataset = dataset.map(tokenize_and_align_train_labels)
+    else:
+        dataset_copy = copy.deepcopy(dataset)
+        dataset = dataset.map(tokenize)
+
+    data_collator = DataCollator(tokenizer, label_maps=label_maps, task_type=task_type)
+
+    shuffle = True if mode == "train" else False
+    batch_sampler = paddle.io.BatchSampler(dataset=dataset, batch_size=batch_size, shuffle=shuffle)
+    dataloader = paddle.io.DataLoader(
+        dataset=dataset, batch_sampler=batch_sampler, collate_fn=data_collator, num_workers=0, return_list=True
+    )
+    if mode != "train":
+        dataloader.dataset.raw_data = dataset_copy
+    return dataloader
+
+
+def postprocess(batch_outputs, offset_mappings, texts, label_maps, task_type="relation_extraction"):
+    if task_type == "entity_extraction":
+        batch_ent_results = []
+        for entity_output, offset_mapping, text in zip(batch_outputs[0].numpy(), offset_mappings, texts):
+            entity_output[:, [0, -1]] -= np.inf
+            entity_output[:, :, [0, -1]] -= np.inf
+            ent_list = []
+            for l, start, end in zip(*np.where(entity_output > 0.0)):
+                start, end = (offset_mapping[start][0], offset_mapping[end][-1])
+                ent = {"text": text[start:end], "type": label_maps["id2entity"][l], "start_index": start}
+                ent_list.append(ent)
+            batch_ent_results.append(ent_list)
+        return batch_ent_results
+    else:
+        batch_ent_results = []
+        batch_rel_results = []
+        for entity_output, head_output, tail_output, offset_mapping, text in zip(
+            batch_outputs[0].numpy(),
+            batch_outputs[1].numpy(),
+            batch_outputs[2].numpy(),
+            offset_mappings,
+            texts,
+        ):
+            entity_output[:, [0, -1]] -= np.inf
+            entity_output[:, :, [0, -1]] -= np.inf
+            ents = set()
+            ent_list = []
+            for l, start, end in zip(*np.where(entity_output > 0.0)):
+                ents.add((start, end))
+                start, end = (offset_mapping[start][0], offset_mapping[end][-1])
+                ent = {"text": text[start:end], "type": label_maps["id2entity"][l], "start_index": start}
+                ent_list.append(ent)
+            batch_ent_results.append(ent_list)
+
+            rel_list = []
+            for sh, st in ents:
+                for oh, ot in ents:
+                    p1s = np.where(head_output[:, sh, oh] > 0.0)[0]
+                    p2s = np.where(tail_output[:, st, ot] > 0.0)[0]
+                    ps = set(p1s) & set(p2s)
+                    for p in ps:
+                        if task_type in ["relation_extraction", "event_extraction"]:
+                            rel = {
+                                "subject": text[offset_mapping[sh][0] : offset_mapping[st][1]],
+                                "predicate": label_maps["id2relation"][p],
+                                "object": text[offset_mapping[oh][0] : offset_mapping[ot][1]],
+                                "subject_start_index": offset_mapping[sh][0],
+                                "object_start_index": offset_mapping[oh][0],
+                            }
+                        else:
+                            rel = {
+                                "aspect": text[offset_mapping[sh][0] : offset_mapping[st][1]],
+                                "sentiment": label_maps["id2relation"][p],
+                                "opinion": text[offset_mapping[oh][0] : offset_mapping[ot][1]],
+                                "aspect_start_index": offset_mapping[sh][0],
+                                "opinion_start_index": offset_mapping[oh][0],
+                            }
+                        rel_list.append(rel)
+            batch_rel_results.append(rel_list)
+        return (batch_ent_results, batch_rel_results)
+
+
+def build_tree(schema, name="root"):
+    """
+    Build the schema tree.
+    """
+    schema_tree = SchemaTree(name)
+    for s in schema:
+        if isinstance(s, str):
+            schema_tree.add_child(SchemaTree(s))
+        elif isinstance(s, dict):
+            for k, v in s.items():
+                if isinstance(v, str):
+                    child = [v]
+                elif isinstance(v, list):
+                    child = v
+                else:
+                    raise TypeError(
+                        "Invalid schema, value for each key:value pairs should be list or string"
+                        "but {} received".format(type(v))
+                    )
+                schema_tree.add_child(build_tree(child, name=k))
+        else:
+            raise TypeError("Invalid schema, element should be string or dict, " "but {} received".format(type(s)))
+    return schema_tree
+
+
+def schema2label_maps(task_type, schema=None):
+    if schema and isinstance(schema, dict):
+        schema = [schema]
+
+    label_maps = {}
+    if task_type == "entity_extraction":
+        entity2id = {}
+        for s in schema:
+            entity2id[s] = len(entity2id)
+
+        label_maps["entity2id"] = entity2id
+    elif task_type == "opinion_extraction":
+        schema = ["观点词", {"评价维度": ["观点词", "情感倾向[正向,负向]"]}]
+        logger.info("Opinion extraction does not support custom schema, the schema is default to %s." % schema)
+        label_maps["entity2id"] = {"评价维度": 0, "观点词": 1}
+        label_maps["sentiment2id"] = {"正向": 0, "负向": 1}
+    else:
+        entity2id = {}
+        relation2id = {}
+        schema_tree = build_tree(schema)
+        schema_list = schema_tree.children[:]
+        while len(schema_list) > 0:
+            node = schema_list.pop(0)
+
+            if node.name not in entity2id.keys() and len(node.children) != 0:
+                entity2id[node.name] = len(entity2id)
+
+            for child in node.children:
+                if child.name not in relation2id.keys():
+                    relation2id[child.name] = len(relation2id)
+                schema_list.append(child)
+
+        entity2id["object"] = len(entity2id)
+        label_maps["entity2id"] = entity2id
+        label_maps["relation2id"] = relation2id
+
+    label_maps["schema"] = schema
+    return label_maps
+
+
+def anno2distill(json_lines, task_type, label_maps=None, platform="label_studio"):
+    if platform == "label_studio":
+        return label_studio2distill(json_lines, task_type, label_maps)
+    else:
+        return doccano2distill(json_lines, task_type, label_maps)
+
+
+def label_studio2distill(json_lines, task_type, label_maps=None):
+    """Convert label-studio to distill format"""
+    if task_type == "opinion_extraction":
+        outputs = []
+        for json_line in json_lines:
+            id2ent = {}
+            text = json_line["data"]["text"]
+            output = {"text": text}
+            entity_list = []
+            aso_list = []
+            annos = json_line["annotations"][0]["result"]
+            for anno in annos:
+                if anno["type"] == "labels":
+                    ent_text = text[anno["value"]["start"] : anno["value"]["end"]]
+                    ent_type_gather = anno["value"]["labels"][0].split("##")
+                    if len(ent_type_gather) == 2:
+                        ent_type, ent_senti = ent_type_gather
+                    else:
+                        ent_type = ent_type_gather[0]
+                        ent_senti = None
+                    ent = {"text": ent_text, "type": ent_type, "start_index": anno["value"]["start"]}
+                    id2ent[anno["id"]] = ent
+                    id2ent[anno["id"]]["sentiment"] = ent_senti
+                    entity_list.append(ent)
+                else:
+                    _aspect = id2ent[anno["from_id"]]
+                    if _aspect["sentiment"]:
+                        _opinion = id2ent[anno["to_id"]]
+                        rel = {
+                            "aspect": _aspect["text"],
+                            "sentiment": _aspect["sentiment"],
+                            "opinion": _opinion["text"],
+                            "aspect_start_index": _aspect["start_index"],
+                            "opinion_start_index": _opinion["start_index"],
+                        }
+                        aso_list.append(rel)
+                    output["aso_list"] = aso_list
+            output["entity_list"] = entity_list
+            output["aso_list"] = aso_list
+            outputs.append(output)
+    else:
+        outputs = []
+        for json_line in json_lines:
+            id2ent = {}
+            text = json_line["data"]["text"]
+            output = {"text": text}
+            entity_list = []
+            spo_list = []
+            annos = json_line["annotations"][0]["result"]
+            for anno in annos:
+                if anno["type"] == "labels":
+                    ent_text = text[anno["value"]["start"] : anno["value"]["end"]]
+                    ent_label = anno["value"]["labels"][0]
+                    ent_type = "object" if ent_label not in label_maps["entity2id"].keys() else ent_label
+                    ent = {"text": ent_text, "type": ent_type, "start_index": anno["value"]["start"]}
+                    id2ent[anno["id"]] = ent
+                    entity_list.append(ent)
+                else:
+                    _subject = id2ent[anno["from_id"]]
+                    _object = id2ent[anno["to_id"]]
+                    rel = {
+                        "subject": _subject["text"],
+                        "predicate": anno["labels"][0],
+                        "object": _object["text"],
+                        "subject_start_index": _subject["start_index"],
+                        "object_start_index": _object["start_index"],
+                    }
+                    spo_list.append(rel)
+            output["entity_list"] = entity_list
+            output["spo_list"] = spo_list
+            outputs.append(output)
+    return outputs
+
+
+def doccano2distill(json_lines, task_type, label_maps=None):
+    """Convert doccano to distill format"""
+    if task_type == "opinion_extraction":
+        outputs = []
+        for json_line in json_lines:
+            id2ent = {}
+            text = json_line["text"]
+            output = {"text": text}
+            entity_list = []
+            entities = json_line["entities"]
+            for entity in entities:
+                ent_text = text[entity["start_offset"] : entity["end_offset"]]
+                ent_type_gather = entity["label"].split("##")
+                if len(ent_type_gather) == 2:
+                    ent_type, ent_senti = ent_type_gather
+                else:
+                    ent_type = ent_type_gather[0]
+                    ent_senti = None
+                ent = {"text": ent_text, "type": ent_type, "start_index": entity["start_offset"]}
+                id2ent[entity["id"]] = ent
+                id2ent[entity["id"]]["sentiment"] = ent_senti
+                entity_list.append(ent)
+            output["entity_list"] = entity_list
+            aso_list = []
+            relations = json_line["relations"]
+            for relation in relations:
+                _aspect = id2ent[relation["from_id"]]
+                if _aspect["sentiment"]:
+                    _opinion = id2ent[relation["to_id"]]
+                    rel = {
+                        "aspect": _aspect["text"],
+                        "sentiment": _aspect["sentiment"],
+                        "opinion": _opinion["text"],
+                        "aspect_start_index": _aspect["start_index"],
+                        "opinion_start_index": _opinion["start_index"],
+                    }
+                    aso_list.append(rel)
+            output["aso_list"] = aso_list
+            outputs.append(output)
+    else:
+        outputs = []
+        for json_line in json_lines:
+            id2ent = {}
+            text = json_line["text"]
+            output = {"text": text}
+            entity_list = []
+            entities = json_line["entities"]
+            for entity in entities:
+                ent_text = text[entity["start_offset"] : entity["end_offset"]]
+                if entity["label"] not in label_maps["entity2id"].keys():
+                    if task_type == "entity_extraction":
+                        logger.warning(
+                            "Found undefined label type. The setting of schema should contain all the label types in annotation file export from annotation platform."
+                        )
+                        continue
+                    else:
+                        ent_type = "object"
+                else:
+                    ent_type = entity["label"]
+                ent = {"text": ent_text, "type": ent_type, "start_index": entity["start_offset"]}
+                id2ent[entity["id"]] = ent
+                entity_list.append(ent)
+            output["entity_list"] = entity_list
+            spo_list = []
+            relations = json_line["relations"]
+            for relation in relations:
+                _subject = id2ent[relation["from_id"]]
+                _object = id2ent[relation["to_id"]]
+                rel = {
+                    "subject": _subject["text"],
+                    "predicate": relation["type"],
+                    "object": _object["text"],
+                    "subject_start_index": _subject["start_index"],
+                    "object_start_index": _object["start_index"],
+                }
+                spo_list.append(rel)
+            output["spo_list"] = spo_list
+            outputs.append(output)
+    return outputs
+
+
+def synthetic2distill(texts, infer_results, task_type, label_maps=None):
+    """Convert synthetic data to distill format"""
+    if task_type == "opinion_extraction":
+        outputs = []
+        for i, line in enumerate(infer_results):
+            pred = line
+            output = {"text": texts[i]}
+
+            entity_list = []
+            aso_list = []
+            for key1 in pred.keys():
+                for s in pred[key1]:
+                    ent = {"text": s["text"], "type": key1, "start_index": s["start"]}
+                    entity_list.append(ent)
+
+                    if (
+                        "relations" in s.keys()
+                        and "观点词" in s["relations"].keys()
+                        and "情感倾向[正向,负向]" in s["relations"].keys()
+                    ):
+                        for o in s["relations"]["观点词"]:
+                            rel = {
+                                "aspect": s["text"],
+                                "sentiment": s["relations"]["情感倾向[正向,负向]"][0]["text"],
+                                "opinion": o["text"],
+                                "aspect_start_index": s["start"],
+                                "opinion_start_index": o["start"],
+                            }
+                            aso_list.append(rel)
+
+                            ent = {"text": o["text"], "type": "观点词", "start_index": o["start"]}
+                            entity_list.append(ent)
+            output["entity_list"] = entity_list
+            output["aso_list"] = aso_list
+            outputs.append(output)
+    else:
+        outputs = []
+        for i, line in enumerate(infer_results):
+            pred = line
+            output = {"text": texts[i]}
+
+            entity_list = []
+            spo_list = []
+            for key1 in pred.keys():
+                for s in pred[key1]:
+                    ent = {"text": s["text"], "type": key1, "start_index": s["start"]}
+                    entity_list.append(ent)
+                    if "relations" in s.keys():
+                        for key2 in s["relations"].keys():
+                            for o1 in s["relations"][key2]:
+                                if "start" in o1.keys():
+                                    rel = {
+                                        "subject": s["text"],
+                                        "predicate": key2,
+                                        "object": o1["text"],
+                                        "subject_start_index": s["start"],
+                                        "object_start_index": o1["start"],
+                                    }
+                                    spo_list.append(rel)
+
+                                    if "relations" not in o1.keys():
+                                        ent = {"text": o1["text"], "type": "object", "start_index": o1["start"]}
+                                        entity_list.append(ent)
+                                    else:
+                                        ent = {"text": o1["text"], "type": key2, "start_index": o1["start"]}
+                                        entity_list.append(ent)
+                                        for key3 in o1["relations"].keys():
+                                            for o2 in o1["relations"][key3]:
+                                                ent = {
+                                                    "text": o2["text"],
+                                                    "type": "object",
+                                                    "start_index": o2["start"],
+                                                }
+                                                entity_list.append(ent)
+
+                                                rel = {
+                                                    "subject": o1["text"],
+                                                    "predicate": key3,
+                                                    "object": o2["text"],
+                                                    "subject_start_index": o1["start"],
+                                                    "object_start_index": o2["start"],
+                                                }
+                                                spo_list.append(rel)
+            output["entity_list"] = entity_list
+            output["spo_list"] = spo_list
+            outputs.append(output)
+    return outputs

applications/information_extraction/text/deploy/simple_serving/README.md

@@ -0,0 +1,58 @@
+# 基于PaddleNLP SimpleServing 的服务化部署
+
+## 目录
+- [环境准备](#环境准备)
+- [Server服务启动](#Server服务启动)
+- [Client请求启动](#Client请求启动)
+- [服务化自定义参数](#服务化自定义参数)
+
+## 环境准备
+使用有SimpleServing功能的PaddleNLP版本(或者最新的develop版本)
+
+```shell
+pip install paddlenlp >= 2.4.4
+```
+
+
+## Server服务启动
+
+```bash
+paddlenlp server server:app --workers 1 --host 0.0.0.0 --port 8189
+```
+
+## Client请求启动
+
+```bash
+python client.py
+```
+
+## 服务化自定义参数
+
+### Server 自定义参数
+#### schema替换
+```python
+# Default schema
+schema = {"武器名称": ["产国", "类型", "研发单位"]}
+```
+
+#### 设置模型路径
+```
+# Default task_path
+uie = Taskflow('information_extraction', task_path='../../checkpoint/model_best/', schema=schema)
+```
+
+#### 多卡服务化预测
+PaddleNLP SimpleServing 支持多卡负载均衡预测，主要在服务化注册的时候，注册两个Taskflow的task即可，下面是示例代码
+```
+uie1 = Taskflow('information_extraction', task_path='../../checkpoint/model_best/', schema=schema, device_id=0)
+uie2 = Taskflow('information_extraction', task_path='../../checkpoint/model_best/', schema=schema, device_id=1)
+service.register_taskflow('uie', [uie1, uie2])
+```
+
+### Client 自定义参数
+
+```python
+# Changed to input texts you wanted
+texts = ['威尔哥（Virgo）减速炸弹是由瑞典FFV军械公司专门为瑞典皇家空军的攻击机实施低空高速轰炸而研制，1956年开始研制，1963年进入服役，装备于A32“矛盾”、A35“龙”、和AJ134“雷”攻击机，主要用于攻击登陆>艇、停放的飞机、高炮、野战火炮、轻型防护装甲车辆以及有生力量。']
+
+```

applications/information_extraction/text/deploy/simple_serving/README_en.md

@@ -0,0 +1,64 @@
+# Service deployment based on PaddleNLP SimpleServing
+
+- [Environment Preparation](#1)
+- [Server](#2)
+- [Client](#3)
+- [Service Custom Parameters](#4)
+
+<a name="1"></a>
+
+## Environment Preparation
+Use the PaddleNLP version with SimpleServing function (or the latest develop version)
+
+```shell
+pip install paddlenlp >= 2.4.4
+```
+
+<a name="2"></a>
+
+## Server
+
+```bash
+paddlenlp server server:app --workers 1 --host 0.0.0.0 --port 8189
+```
+
+<a name="3"></a>
+
+## Client
+
+```bash
+python client.py
+```
+
+<a name="4"></a>
+
+## Service Custom Parameters
+
+### Server Custom Parameters
+
+#### schema replacement
+```python
+# Default schema
+schema = {"Weapon Name": ["Country of Production", "Type", "R&D Unit"]}
+```
+
+#### Set model path
+```
+# Default task_path
+uie = Taskflow('information_extraction', task_path='../../checkpoint/model_best/', schema=schema)
+```
+
+#### Doka Service Prediction
+PaddleNLP SimpleServing supports multi-card load balancing prediction, mainly during service registration, just register two Taskflow tasks, the following is the sample code
+```
+uie1 = Taskflow('information_extraction', task_path='../../checkpoint/model_best/', schema=schema, device_id=0)
+uie2 = Taskflow('information_extraction', task_path='../../checkpoint/model_best/', schema=schema, device_id=1)
+service. register_taskflow('uie', [uie1, uie2])
+```
+
+### Client Custom Parameters
+
+```python
+# Changed to input texts you wanted
+texts = ['威尔哥（Virgo）减速炸弹是由瑞典FFV军械公司专门为瑞典皇家空军的攻击机实施低空高速轰炸而研制，1956年开始研制，1963年进入服役，装备于A32“矛盾”、A35“龙”、和AJ134“雷”攻击机，主要用于攻击登陆艇、停放的飞机、高炮、野战火炮、轻型防护装甲车辆以及有生力量。']
+```

applications/information_extraction/text/deploy/simple_serving/client.py

@@ -0,0 +1,29 @@
+# Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import json
+
+import requests
+
+url = "http://0.0.0.0:8189/taskflow/uie"
+
+headers = {"Content-Type": "application/json"}
+texts = [
+    "威尔哥（Virgo）减速炸弹是由瑞典FFV军械公司专门为瑞典皇家空军的攻击机实施低空高速轰炸而研制，1956年开始研制，1963年进入服役，装备于A32“矛盾”、A35“龙”、和AJ134“雷”攻击机，主要用于攻击登陆>艇、停放的飞机、高炮、野战火炮、轻型防护装甲车辆以及有生力量。"
+]
+
+data = {"data": {"text": texts}}
+r = requests.post(url=url, headers=headers, data=json.dumps(data))
+datas = json.loads(r.text)
+print(datas)

applications/information_extraction/text/deploy/simple_serving/server.py

@@ -0,0 +1,23 @@
+# Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from paddlenlp import SimpleServer, Taskflow
+
+# The schema changed to your defined schema
+schema = {"武器名称": ["产国", "类型", "研发单位"]}
+# The task path changed to your best model path
+uie = Taskflow("information_extraction", schema=schema, task_path="../../checkpoint/model_best/")
+# If you want to define the finetuned uie service
+app = SimpleServer()
+app.register_taskflow("taskflow/uie", uie)

applications/information_extraction/text/evaluate.py

@@ -0,0 +1,142 @@
+# Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import argparse
+from functools import partial
+
+import paddle
+from utils import convert_example, create_data_loader, reader
+
+from paddlenlp.data import DataCollatorWithPadding
+from paddlenlp.datasets import MapDataset, load_dataset
+from paddlenlp.metrics import SpanEvaluator
+from paddlenlp.transformers import UIE, UIEM, AutoTokenizer
+from paddlenlp.utils.ie_utils import get_relation_type_dict, unify_prompt_name
+from paddlenlp.utils.log import logger
+
+
+@paddle.no_grad()
+def evaluate(model, metric, data_loader, multilingual=False):
+    """
+    Given a dataset, it evals model and computes the metric.
+    Args:
+        model(obj:`paddle.nn.Layer`): A model to classify texts.
+        metric(obj:`paddle.metric.Metric`): The evaluation metric.
+        data_loader(obj:`paddle.io.DataLoader`): The dataset loader which generates batches.
+        multilingual(bool): Whether is the multilingual model.
+    """
+    model.eval()
+    metric.reset()
+    for batch in data_loader:
+        if multilingual:
+            start_prob, end_prob = model(batch["input_ids"], batch["position_ids"])
+        else:
+            start_prob, end_prob = model(
+                batch["input_ids"], batch["token_type_ids"], batch["position_ids"], batch["attention_mask"]
+            )
+
+        start_ids = paddle.cast(batch["start_positions"], "float32")
+        end_ids = paddle.cast(batch["end_positions"], "float32")
+        num_correct, num_infer, num_label = metric.compute(start_prob, end_prob, start_ids, end_ids)
+        metric.update(num_correct, num_infer, num_label)
+    precision, recall, f1 = metric.accumulate()
+    model.train()
+    return precision, recall, f1
+
+
+def do_eval():
+    paddle.set_device(args.device)
+
+    if args.model_path in ["uie-m-base", "uie-m-large"]:
+        args.multilingual = True
+    tokenizer = AutoTokenizer.from_pretrained(args.model_path)
+    if args.multilingual:
+        model = UIEM.from_pretrained(args.model_path)
+    else:
+        model = UIE.from_pretrained(args.model_path)
+
+    test_ds = load_dataset(reader, data_path=args.test_path, max_seq_len=args.max_seq_len, lazy=False)
+    class_dict = {}
+    relation_data = []
+    if args.debug:
+        for data in test_ds:
+            class_name = unify_prompt_name(data["prompt"])
+            # Only positive examples are evaluated in debug mode
+            if len(data["result_list"]) != 0:
+                p = "的" if args.schema_lang == "ch" else " of "
+                if p not in data["prompt"]:
+                    class_dict.setdefault(class_name, []).append(data)
+                else:
+                    relation_data.append((data["prompt"], data))
+
+        relation_type_dict = get_relation_type_dict(relation_data, schema_lang=args.schema_lang)
+    else:
+        class_dict["all_classes"] = test_ds
+
+    trans_fn = partial(
+        convert_example, tokenizer=tokenizer, max_seq_len=args.max_seq_len, multilingual=args.multilingual
+    )
+
+    for key in class_dict.keys():
+        if args.debug:
+            test_ds = MapDataset(class_dict[key])
+        else:
+            test_ds = class_dict[key]
+        test_ds = test_ds.map(trans_fn)
+
+        data_collator = DataCollatorWithPadding(tokenizer)
+
+        test_data_loader = create_data_loader(test_ds, mode="test", batch_size=args.batch_size, trans_fn=data_collator)
+
+        metric = SpanEvaluator()
+        precision, recall, f1 = evaluate(model, metric, test_data_loader, args.multilingual)
+        logger.info("-----------------------------")
+        logger.info("Class Name: %s" % key)
+        logger.info("Evaluation Precision: %.5f | Recall: %.5f | F1: %.5f" % (precision, recall, f1))
+
+    if args.debug and len(relation_type_dict.keys()) != 0:
+        for key in relation_type_dict.keys():
+            test_ds = MapDataset(relation_type_dict[key])
+            test_ds = test_ds.map(trans_fn)
+            test_data_loader = create_data_loader(
+                test_ds, mode="test", batch_size=args.batch_size, trans_fn=data_collator
+            )
+
+            metric = SpanEvaluator()
+            precision, recall, f1 = evaluate(model, metric, test_data_loader)
+            logger.info("-----------------------------")
+            if args.schema_lang == "ch":
+                logger.info("Class Name: X的%s" % key)
+            else:
+                logger.info("Class Name: %s of X" % key)
+            logger.info("Evaluation Precision: %.5f | Recall: %.5f | F1: %.5f" % (precision, recall, f1))
+
+
+if __name__ == "__main__":
+    # yapf: disable
+    parser = argparse.ArgumentParser()
+
+    parser.add_argument("--model_path", type=str, default=None, help="The path of saved model that you want to load.")
+    parser.add_argument("--test_path", type=str, default=None, help="The path of test set.")
+    parser.add_argument("--batch_size", type=int, default=16, help="Batch size per GPU/CPU for training.")
+    parser.add_argument("--device", type=str, default="gpu", choices=["gpu", "cpu", "npu"], help="Device selected for evaluate.")
+    parser.add_argument("--max_seq_len", type=int, default=512, help="The maximum total input sequence length after tokenization.")
+    parser.add_argument("--debug", action='store_true', help="Precision, recall and F1 score are calculated for each class separately if this option is enabled.")
+    parser.add_argument("--multilingual", action='store_true', help="Whether is the multilingual model.")
+    parser.add_argument("--schema_lang", choices=["ch", "en"], default="ch", help="Select the language type for schema.")
+
+    args = parser.parse_args()
+    # yapf: enable
+
+    do_eval()

applications/information_extraction/text/finetune.py

@@ -0,0 +1,243 @@
+# Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import json
+import os
+from dataclasses import dataclass, field
+from functools import partial
+from typing import List, Optional
+
+import paddle
+from utils import convert_example, reader
+
+from paddlenlp.data import DataCollatorWithPadding
+from paddlenlp.datasets import load_dataset
+from paddlenlp.metrics import SpanEvaluator
+from paddlenlp.trainer import (
+    CompressionArguments,
+    PdArgumentParser,
+    Trainer,
+    get_last_checkpoint,
+)
+from paddlenlp.transformers import UIE, UIEM, AutoTokenizer, export_model
+from paddlenlp.utils.ie_utils import compute_metrics, uie_loss_func
+from paddlenlp.utils.log import logger
+
+
+@dataclass
+class DataArguments:
+    """
+    Arguments pertaining to what data we are going to input our model for training and eval.
+    Using `PdArgumentParser` we can turn this class into argparse arguments to be able to
+    specify them on the command line.
+    """
+
+    train_path: str = field(
+        default=None, metadata={"help": "The name of the dataset to use (via the datasets library)."}
+    )
+
+    dev_path: str = field(
+        default=None, metadata={"help": "The name of the dataset to use (via the datasets library)."}
+    )
+
+    max_seq_length: Optional[int] = field(
+        default=512,
+        metadata={
+            "help": "The maximum total input sequence length after tokenization. Sequences longer "
+            "than this will be truncated, sequences shorter will be padded."
+        },
+    )
+
+    dynamic_max_length: Optional[List[int]] = field(
+        default=None,
+        metadata={"help": "dynamic max length from batch, it can be array of length, eg: 16 32 64 128"},
+    )
+
+
+@dataclass
+class ModelArguments:
+    """
+    Arguments pertaining to which model/config/tokenizer we are going to fine-tune from.
+    """
+
+    model_name_or_path: Optional[str] = field(
+        default="uie-base",
+        metadata={
+            "help": "Path to pretrained model, such as 'uie-base', 'uie-tiny', "
+            "'uie-medium', 'uie-mini', 'uie-micro', 'uie-nano', 'uie-base-en', "
+            "'uie-m-base', 'uie-m-large', or finetuned model path."
+        },
+    )
+    export_model_dir: Optional[str] = field(
+        default=None,
+        metadata={"help": "Path to directory to store the exported inference model."},
+    )
+    multilingual: bool = field(default=False, metadata={"help": "Whether the model is a multilingual model."})
+
+
+def main():
+    parser = PdArgumentParser((ModelArguments, DataArguments, CompressionArguments))
+    model_args, data_args, training_args = parser.parse_args_into_dataclasses()
+    training_args.label_names = ["start_positions", "end_positions"]
+
+    if model_args.model_name_or_path in ["uie-m-base", "uie-m-large"]:
+        model_args.multilingual = True
+    elif os.path.exists(os.path.join(model_args.model_name_or_path, "model_config.json")):
+        with open(os.path.join(model_args.model_name_or_path, "model_config.json")) as f:
+            init_class = json.load(f)["init_class"]
+        if init_class == "UIEM":
+            model_args.multilingual = True
+
+    # Log model and data config
+    training_args.print_config(model_args, "Model")
+    training_args.print_config(data_args, "Data")
+
+    paddle.set_device(training_args.device)
+
+    # Log on each process the small summary:
+    logger.warning(
+        f"Process rank: {training_args.local_rank}, device: {training_args.device}, world_size: {training_args.world_size}, "
+        + f"distributed training: {bool(training_args.local_rank != -1)}, 16-bits training: {training_args.fp16}"
+    )
+
+    # Detecting last checkpoint.
+    last_checkpoint = None
+    if os.path.isdir(training_args.output_dir) and training_args.do_train and not training_args.overwrite_output_dir:
+        last_checkpoint = get_last_checkpoint(training_args.output_dir)
+        if last_checkpoint is None and len(os.listdir(training_args.output_dir)) > 0:
+            raise ValueError(
+                f"Output directory ({training_args.output_dir}) already exists and is not empty. "
+                "Use --overwrite_output_dir to overcome."
+            )
+        elif last_checkpoint is not None and training_args.resume_from_checkpoint is None:
+            logger.info(
+                f"Checkpoint detected, resuming training at {last_checkpoint}. To avoid this behavior, change "
+                "the `--output_dir` or add `--overwrite_output_dir` to train from scratch."
+            )
+
+    tokenizer = AutoTokenizer.from_pretrained(model_args.model_name_or_path)
+    if model_args.multilingual:
+        model = UIEM.from_pretrained(model_args.model_name_or_path)
+    else:
+        model = UIE.from_pretrained(model_args.model_name_or_path)
+
+    train_ds = load_dataset(reader, data_path=data_args.train_path, max_seq_len=data_args.max_seq_length, lazy=False)
+    dev_ds = load_dataset(reader, data_path=data_args.dev_path, max_seq_len=data_args.max_seq_length, lazy=False)
+
+    trans_fn = partial(
+        convert_example,
+        tokenizer=tokenizer,
+        max_seq_len=data_args.max_seq_length,
+        multilingual=model_args.multilingual,
+        dynamic_max_length=data_args.dynamic_max_length,
+    )
+
+    train_ds = train_ds.map(trans_fn)
+    dev_ds = dev_ds.map(trans_fn)
+
+    if training_args.device == "npu":
+        data_collator = DataCollatorWithPadding(tokenizer, padding="longest")
+    else:
+        data_collator = DataCollatorWithPadding(tokenizer)
+
+    trainer = Trainer(
+        model=model,
+        criterion=uie_loss_func,
+        args=training_args,
+        data_collator=data_collator,
+        train_dataset=train_ds if training_args.do_train or training_args.do_compress else None,
+        eval_dataset=dev_ds if training_args.do_eval or training_args.do_compress else None,
+        tokenizer=tokenizer,
+        compute_metrics=compute_metrics,
+    )
+
+    trainer.optimizer = paddle.optimizer.AdamW(
+        learning_rate=training_args.learning_rate, parameters=model.parameters()
+    )
+    checkpoint = None
+    if training_args.resume_from_checkpoint is not None:
+        checkpoint = training_args.resume_from_checkpoint
+    elif last_checkpoint is not None:
+        checkpoint = last_checkpoint
+
+    # Training
+    if training_args.do_train:
+        train_result = trainer.train(resume_from_checkpoint=checkpoint)
+        metrics = train_result.metrics
+        trainer.save_model()
+        trainer.log_metrics("train", metrics)
+        trainer.save_metrics("train", metrics)
+        trainer.save_state()
+
+    # Evaluate and tests model
+    if training_args.do_eval:
+        eval_metrics = trainer.evaluate()
+        trainer.log_metrics("eval", eval_metrics)
+
+    # export inference model
+    if training_args.do_export:
+        # You can also load from certain checkpoint
+        # trainer.load_state_dict_from_checkpoint("/path/to/checkpoint/")
+        if training_args.device == "npu":
+            # npu will transform int64 to int32 for internal calculation.
+            # To reduce useless transformation, we feed int32 inputs.
+            input_spec_dtype = "int32"
+        else:
+            input_spec_dtype = "int64"
+        if model_args.multilingual:
+            input_spec = [
+                paddle.static.InputSpec(shape=[None, None], dtype=input_spec_dtype, name="input_ids"),
+                paddle.static.InputSpec(shape=[None, None], dtype=input_spec_dtype, name="position_ids"),
+            ]
+        else:
+            input_spec = [
+                paddle.static.InputSpec(shape=[None, None], dtype=input_spec_dtype, name="input_ids"),
+                paddle.static.InputSpec(shape=[None, None], dtype=input_spec_dtype, name="token_type_ids"),
+                paddle.static.InputSpec(shape=[None, None], dtype=input_spec_dtype, name="position_ids"),
+                paddle.static.InputSpec(shape=[None, None], dtype=input_spec_dtype, name="attention_mask"),
+            ]
+        if model_args.export_model_dir is None:
+            model_args.export_model_dir = os.path.join(training_args.output_dir, "export")
+        export_model(model=trainer.model, input_spec=input_spec, path=model_args.export_model_dir)
+    if training_args.do_compress:
+
+        @paddle.no_grad()
+        def custom_evaluate(self, model, data_loader):
+            metric = SpanEvaluator()
+            model.eval()
+            metric.reset()
+            for batch in data_loader:
+                if model_args.multilingual:
+                    logits = model(input_ids=batch["input_ids"], position_ids=batch["position_ids"])
+                else:
+                    logits = model(
+                        input_ids=batch["input_ids"],
+                        token_type_ids=batch["token_type_ids"],
+                        position_ids=batch["position_ids"],
+                        attention_mask=batch["attention_mask"],
+                    )
+                start_prob, end_prob = logits
+                start_ids, end_ids = batch["start_positions"], batch["end_positions"]
+                num_correct, num_infer, num_label = metric.compute(start_prob, end_prob, start_ids, end_ids)
+                metric.update(num_correct, num_infer, num_label)
+            precision, recall, f1 = metric.accumulate()
+            logger.info("f1: %s, precision: %s, recall: %s" % (f1, precision, f1))
+            model.train()
+            return f1
+
+        trainer.compress(custom_evaluate=custom_evaluate)
+
+
+if __name__ == "__main__":
+    main()

applications/information_extraction/text/utils.py

@@ -0,0 +1,234 @@
+# Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import json
+import random
+from typing import List, Optional
+
+import numpy as np
+import paddle
+
+from paddlenlp.utils.log import logger
+
+
+def set_seed(seed):
+    paddle.seed(seed)
+    random.seed(seed)
+    np.random.seed(seed)
+
+
+def create_data_loader(dataset, mode="train", batch_size=1, trans_fn=None):
+    """
+    Create dataloader.
+    Args:
+        dataset(obj:`paddle.io.Dataset`): Dataset instance.
+        mode(obj:`str`, optional, defaults to obj:`train`): If mode is 'train', it will shuffle the dataset randomly.
+        batch_size(obj:`int`, optional, defaults to 1): The sample number of a mini-batch.
+        trans_fn(obj:`callable`, optional, defaults to `None`): function to convert a data sample to input ids, etc.
+    Returns:
+        dataloader(obj:`paddle.io.DataLoader`): The dataloader which generates batches.
+    """
+    if trans_fn:
+        dataset = dataset.map(trans_fn)
+
+    shuffle = True if mode == "train" else False
+    if mode == "train":
+        sampler = paddle.io.DistributedBatchSampler(dataset=dataset, batch_size=batch_size, shuffle=shuffle)
+    else:
+        sampler = paddle.io.BatchSampler(dataset=dataset, batch_size=batch_size, shuffle=shuffle)
+    dataloader = paddle.io.DataLoader(dataset, batch_sampler=sampler, return_list=True)
+    return dataloader
+
+
+def map_offset(ori_offset, offset_mapping):
+    """
+    map ori offset to token offset
+    """
+    for index, span in enumerate(offset_mapping):
+        if span[0] <= ori_offset < span[1]:
+            return index
+    return -1
+
+
+def reader(data_path, max_seq_len=512):
+    """
+    read json
+    """
+    with open(data_path, "r", encoding="utf-8") as f:
+        for line in f:
+            json_line = json.loads(line)
+            content = json_line["content"].strip()
+            prompt = json_line["prompt"]
+            # Model Input is aslike: [CLS] Prompt [SEP] Content [SEP]
+            # It include three summary tokens.
+            if max_seq_len <= len(prompt) + 3:
+                raise ValueError("The value of max_seq_len is too small, please set a larger value")
+            max_content_len = max_seq_len - len(prompt) - 3
+            if len(content) <= max_content_len:
+                yield json_line
+            else:
+                result_list = json_line["result_list"]
+                json_lines = []
+                accumulate = 0
+                while True:
+                    cur_result_list = []
+                    for result in result_list:
+                        if result["end"] - result["start"] > max_content_len:
+                            logger.warning(
+                                "result['end'] - result ['start'] exceeds max_content_len, which will result in no valid instance being returned"
+                            )
+                        if (
+                            result["start"] + 1 <= max_content_len < result["end"]
+                            and result["end"] - result["start"] <= max_content_len
+                        ):
+                            max_content_len = result["start"]
+                            break
+
+                    cur_content = content[:max_content_len]
+                    res_content = content[max_content_len:]
+
+                    while True:
+                        if len(result_list) == 0:
+                            break
+                        elif result_list[0]["end"] <= max_content_len:
+                            if result_list[0]["end"] > 0:
+                                cur_result = result_list.pop(0)
+                                cur_result_list.append(cur_result)
+                            else:
+                                cur_result_list = [result for result in result_list]
+                                break
+                        else:
+                            break
+
+                    json_line = {"content": cur_content, "result_list": cur_result_list, "prompt": prompt}
+                    json_lines.append(json_line)
+
+                    for result in result_list:
+                        if result["end"] <= 0:
+                            break
+                        result["start"] -= max_content_len
+                        result["end"] -= max_content_len
+                    accumulate += max_content_len
+                    max_content_len = max_seq_len - len(prompt) - 3
+                    if len(res_content) == 0:
+                        break
+                    elif len(res_content) < max_content_len:
+                        json_line = {"content": res_content, "result_list": result_list, "prompt": prompt}
+                        json_lines.append(json_line)
+                        break
+                    else:
+                        content = res_content
+
+                for json_line in json_lines:
+                    yield json_line
+
+
+def get_dynamic_max_length(examples, default_max_length: int, dynamic_max_length: List[int]) -> int:
+    """get max_length by examples which you can change it by examples in batch"""
+    cur_length = len(examples[0]["input_ids"])
+    max_length = default_max_length
+    for max_length_option in sorted(dynamic_max_length):
+        if cur_length <= max_length_option:
+            max_length = max_length_option
+            break
+    return max_length
+
+
+def convert_example(
+    example, tokenizer, max_seq_len, multilingual=False, dynamic_max_length: Optional[List[int]] = None
+):
+    """
+    example: {
+        title
+        prompt
+        content
+        result_list
+    }
+    """
+    if dynamic_max_length is not None:
+        temp_encoded_inputs = tokenizer(
+            text=[example["prompt"]],
+            text_pair=[example["content"]],
+            truncation=True,
+            max_seq_len=max_seq_len,
+            return_attention_mask=True,
+            return_position_ids=True,
+            return_dict=False,
+            return_offsets_mapping=True,
+        )
+        max_length = get_dynamic_max_length(
+            examples=temp_encoded_inputs, default_max_length=max_seq_len, dynamic_max_length=dynamic_max_length
+        )
+        # always pad to max_length
+        encoded_inputs = tokenizer(
+            text=[example["prompt"]],
+            text_pair=[example["content"]],
+            truncation=True,
+            max_seq_len=max_length,
+            pad_to_max_seq_len=True,
+            return_attention_mask=True,
+            return_position_ids=True,
+            return_dict=False,
+            return_offsets_mapping=True,
+        )
+        start_ids = [0.0 for x in range(max_length)]
+        end_ids = [0.0 for x in range(max_length)]
+    else:
+        encoded_inputs = tokenizer(
+            text=[example["prompt"]],
+            text_pair=[example["content"]],
+            truncation=True,
+            max_seq_len=max_seq_len,
+            pad_to_max_seq_len=True,
+            return_attention_mask=True,
+            return_position_ids=True,
+            return_dict=False,
+            return_offsets_mapping=True,
+        )
+        start_ids = [0.0 for x in range(max_seq_len)]
+        end_ids = [0.0 for x in range(max_seq_len)]
+
+    encoded_inputs = encoded_inputs[0]
+    offset_mapping = [list(x) for x in encoded_inputs["offset_mapping"]]
+    bias = 0
+    for index in range(1, len(offset_mapping)):
+        mapping = offset_mapping[index]
+        if mapping[0] == 0 and mapping[1] == 0 and bias == 0:
+            bias = offset_mapping[index - 1][1] + 1  # Includes [SEP] token
+        if mapping[0] == 0 and mapping[1] == 0:
+            continue
+        offset_mapping[index][0] += bias
+        offset_mapping[index][1] += bias
+    for item in example["result_list"]:
+        start = map_offset(item["start"] + bias, offset_mapping)
+        end = map_offset(item["end"] - 1 + bias, offset_mapping)
+        start_ids[start] = 1.0
+        end_ids[end] = 1.0
+    if multilingual:
+        tokenized_output = {
+            "input_ids": encoded_inputs["input_ids"],
+            "position_ids": encoded_inputs["position_ids"],
+            "start_positions": start_ids,
+            "end_positions": end_ids,
+        }
+    else:
+        tokenized_output = {
+            "input_ids": encoded_inputs["input_ids"],
+            "token_type_ids": encoded_inputs["token_type_ids"],
+            "position_ids": encoded_inputs["position_ids"],
+            "attention_mask": encoded_inputs["attention_mask"],
+            "start_positions": start_ids,
+            "end_positions": end_ids,
+        }
+    return tokenized_output

applications/neural_search/README.md

@@ -16,7 +16,12 @@
 车头如何放置车牌    后牌照怎么装
 ```
 
-语义检索系统的关键就在于，采用语义而非关键词方式进行召回，达到更精准、更广泛得召回相似结果的目的。想快速体验搜索的效果，请参考[Pipelines的实现](https://github.com/PaddlePaddle/PaddleNLP/tree/develop/pipelines)
+语义检索系统的关键就在于，采用语义而非关键词方式进行召回，达到更精准、更广泛得召回相似结果的目的。想快速体验搜索的效果，请参考[Pipelines的语义检索实现](https://github.com/PaddlePaddle/PaddleNLP/tree/develop/pipelines/examples/semantic-search)
+
+<div align="center">
+    <img src="https://user-images.githubusercontent.com/12107462/190302765-663ba441-9dd3-470a-8fee-f7a6f81da615.gif" width="500px">
+</div>
+
 
 ## 2. 产品功能介绍
 
@@ -28,6 +33,7 @@
     + 手把手搭建起检索系统
     + 无需标注数据也能构建检索系统
     + 提供 训练、预测、ANN 引擎一站式能力
+    + Pipelines 快速实现语义检索系统
 
 + 效果好
     + 针对多种数据场景的专业方案
@@ -110,7 +116,7 @@
 
 （2）采用文献的 query,title,keywords 三个字段内容，构造无标签数据集，进行无监督召回训练SimCSE；
 
-（3）使用文献的的query, title, keywords，构造带正标签的数据集，不包含负标签样本，基于 In-batch Negatives 策略进行训练；
+（3）使用文献的query, title, keywords，构造带正标签的数据集，不包含负标签样本，基于 In-batch Negatives 策略进行训练；
 
 （4）在排序阶段，使用点击（作为正样本）和展现未点击（作为负样本）数据构造排序阶段的训练集，进行精排训练。
 
@@ -308,7 +314,7 @@ pip install -r requirements.txt
 
 ## 4. Neural Search 快速体验实践
 
-PaddleNLP已经基于ERNIE 1.0训练了一个基线模型，如果想快速搭建Neural Search的完整系统。
+PaddleNLP已经基于ERNIE 1.0训练了一个基线模型，如果想快速搭建Neural Search的完整系统，有两种方法，第一种是请参考下面的实现，包含了服务化的完整流程，另一种是使用Pipelines加载，Pipelines已经支持Neural Search训练的模型的载入，可以使用Pipelines的快速的基于Neural Search模型实现检索系统，详情请参考文档[Pipelines-Neural-Search](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/pipelines/examples/semantic-search/Neural_Search.md)。
 
 ### 4.1. 召回
 

applications/neural_search/ranking/cross_encoder/data.py

@@ -15,16 +15,12 @@ import numpy as np
 import paddle
 
 
-def convert_example(example,
-                    tokenizer,
-                    max_seq_length=512,
-                    is_test=False,
-                    is_pair=False):
+def convert_example(example, tokenizer, max_seq_length=512, is_test=False, is_pair=False):
     """
     Builds model inputs from a sequence or a pair of sequence for sequence classification tasks
-    by concatenating and adding special tokens. And creates a mask from the two sequences passed 
+    by concatenating and adding special tokens. And creates a mask from the two sequences passed
     to be used in a sequence-pair classification task.
-        
+
     A BERT sequence has the following format:
 
     - single sequence: ``[CLS] X [SEP]``
@@ -34,9 +30,9 @@ def convert_example(example,
 
     Args:
         example(obj:`list[str]`): List of input data, containing text and label if it have label.
-        tokenizer(obj:`PretrainedTokenizer`): This tokenizer inherits from :class:`~paddlenlp.transformers.PretrainedTokenizer` 
+        tokenizer(obj:`PretrainedTokenizer`): This tokenizer inherits from :class:`~paddlenlp.transformers.PretrainedTokenizer`
             which contains most of the methods. Users should refer to the superclass for more information regarding methods.
-        max_seq_len(obj:`int`): The maximum total input sequence length after tokenization. 
+        max_seq_len(obj:`int`): The maximum total input sequence length after tokenization.
             Sequences longer than this will be truncated, sequences shorter will be padded.
         is_test(obj:`False`, defaults to `False`): Whether the example contains label or not.
 
@@ -52,9 +48,7 @@ def convert_example(example,
     else:
         text = example["text"]
         text_pair = None
-    encoded_inputs = tokenizer(text=text,
-                               text_pair=text_pair,
-                               max_seq_len=max_seq_length)
+    encoded_inputs = tokenizer(text=text, text_pair=text_pair, max_seq_len=max_seq_length)
     input_ids = encoded_inputs["input_ids"]
     token_type_ids = encoded_inputs["token_type_ids"]
 
@@ -66,51 +60,40 @@ def convert_example(example,
 
 def read_text_pair(data_path):
     """Reads data."""
-    with open(data_path, 'r', encoding='utf-8') as f:
+    with open(data_path, "r", encoding="utf-8") as f:
         for line in f:
             data = line.rstrip().split("\t")
             if len(data) != 3:
                 continue
-            yield {'text_a': data[0], 'text_b': data[1]}
+            yield {"text_a": data[0], "text_b": data[1]}
 
 
 def read_data(data_path):
     """Reads data."""
-    with open(data_path, 'r', encoding='utf-8', errors='ignore') as f:
+    with open(data_path, "r", encoding="utf-8", errors="ignore") as f:
         for i, line in enumerate(f):
             # Skip column name
-            if (i == 0):
+            if i == 0:
                 continue
-            data = line.rstrip('\n').split("\t")
-            if (len(data) != 3):
+            data = line.rstrip("\n").split("\t")
+            if len(data) != 3:
                 print(data)
                 continue
             query = data[0]
             title = data[1]
             label = data[-1]
             # breakpoint()
-            yield {'text_a': query, 'text_b': title, 'label': int(label)}
+            yield {"text_a": query, "text_b": title, "label": int(label)}
 
 
-def create_dataloader(dataset,
-                      mode='train',
-                      batch_size=1,
-                      batchify_fn=None,
-                      trans_fn=None):
+def create_dataloader(dataset, mode="train", batch_size=1, batchify_fn=None, trans_fn=None):
     if trans_fn:
         dataset = dataset.map(trans_fn)
 
-    shuffle = True if mode == 'train' else False
-    if mode == 'train':
-        batch_sampler = paddle.io.DistributedBatchSampler(dataset,
-                                                          batch_size=batch_size,
-                                                          shuffle=shuffle)
+    shuffle = True if mode == "train" else False
+    if mode == "train":
+        batch_sampler = paddle.io.DistributedBatchSampler(dataset, batch_size=batch_size, shuffle=shuffle)
     else:
-        batch_sampler = paddle.io.BatchSampler(dataset,
-                                               batch_size=batch_size,
-                                               shuffle=shuffle)
+        batch_sampler = paddle.io.BatchSampler(dataset, batch_size=batch_size, shuffle=shuffle)
 
-    return paddle.io.DataLoader(dataset=dataset,
-                                batch_sampler=batch_sampler,
-                                collate_fn=batchify_fn,
-                                return_list=True)
+    return paddle.io.DataLoader(dataset=dataset, batch_sampler=batch_sampler, collate_fn=batchify_fn, return_list=True)

applications/neural_search/ranking/cross_encoder/deploy/cpp/http_client.py

@@ -14,24 +14,19 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-import sys
 import time
-import numpy as np
-import requests
-import json
 
+import numpy as np
 from paddle_serving_client.httpclient import HttpClient
 from scipy.special import expit
+
 from paddlenlp.transformers import AutoTokenizer
-import paddlenlp as ppnlp
 
 
 def convert_example(example, tokenizer, max_seq_length=512):
 
     query, title = example["query"], example["title"]
-    encoded_inputs = tokenizer(text=query,
-                               text_pair=title,
-                               max_seq_len=max_seq_length)
+    encoded_inputs = tokenizer(text=query, text_pair=title, max_seq_len=max_seq_length)
     input_ids = encoded_inputs["input_ids"]
     token_type_ids = encoded_inputs["token_type_ids"]
 
@@ -39,34 +34,29 @@ def convert_example(example, tokenizer, max_seq_length=512):
 
 
 # 启动python客户端
-endpoint_list = ['127.0.0.1:8600']
+endpoint_list = ["127.0.0.1:8600"]
 client = HttpClient()
-client.load_client_config('serving_client')
+client.load_client_config("serving_client")
 client.connect(endpoint_list)
 feed_names = client.feed_names_
 fetch_names = client.fetch_names_
 
 # 创建tokenizer
-tokenizer = AutoTokenizer.from_pretrained('rocketqa-base-cross-encoder')
+tokenizer = AutoTokenizer.from_pretrained("rocketqa-base-cross-encoder")
 max_seq_len = 64
 
 # 数据预处理
-list_data = [{
-    "query": "加强科研项目管理有效促进医学科研工作",
-    "title": "科研项目管理策略科研项目,项目管理,实施,必要性,策略"
-}]
+list_data = [{"query": "加强科研项目管理有效促进医学科研工作", "title": "科研项目管理策略科研项目,项目管理,实施,必要性,策略"}]
 
 input_ids, token_type_ids = [], []
 for example in list_data:
-    input_id, token_type_id = convert_example(example,
-                                              tokenizer,
-                                              max_seq_length=max_seq_len)
+    input_id, token_type_id = convert_example(example, tokenizer, max_seq_length=max_seq_len)
     input_ids.append(input_id)
     token_type_ids.append(token_type_id)
 
 feed_dict = {}
-feed_dict['input_ids'] = np.array(input_ids)
-feed_dict['token_type_ids'] = np.array(token_type_ids)
+feed_dict["input_ids"] = np.array(input_ids)
+feed_dict["token_type_ids"] = np.array(token_type_ids)
 # batch设置为True表示的是批量预测
 b_start = time.time()
 result = client.predict(feed=feed_dict, fetch=fetch_names, batch=True)

applications/neural_search/ranking/cross_encoder/deploy/cpp/rpc_client.py

@@ -14,22 +14,19 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-import sys
 import time
-import numpy as np
 
+import numpy as np
 from paddle_serving_client import Client
 from scipy.special import expit
+
 from paddlenlp.transformers import AutoTokenizer
-import paddlenlp as ppnlp
 
 
 def convert_example(example, tokenizer, max_seq_length=512):
 
     query, title = example["query"], example["title"]
-    encoded_inputs = tokenizer(text=query,
-                               text_pair=title,
-                               max_seq_len=max_seq_length)
+    encoded_inputs = tokenizer(text=query, text_pair=title, max_seq_len=max_seq_length)
 
     input_ids = encoded_inputs["input_ids"]
     token_type_ids = encoded_inputs["token_type_ids"]
@@ -38,39 +35,34 @@ def convert_example(example, tokenizer, max_seq_length=512):
 
 
 # 启动python客户端
-endpoint_list = ['127.0.0.1:8600']
+endpoint_list = ["127.0.0.1:8600"]
 client = Client()
-client.load_client_config('serving_client')
+client.load_client_config("serving_client")
 client.connect(endpoint_list)
 feed_names = client.feed_names_
 fetch_names = client.fetch_names_
 
 # 创建tokenizer
-tokenizer = AutoTokenizer.from_pretrained('rocketqa-base-cross-encoder')
+tokenizer = AutoTokenizer.from_pretrained("rocketqa-base-cross-encoder")
 max_seq_len = 64
 
 # 数据预处理
-list_data = [{
-    "query": "加强科研项目管理有效促进医学科研工作",
-    "title": "科研项目管理策略科研项目,项目管理,实施,必要性,策略"
-}]
+list_data = [{"query": "加强科研项目管理有效促进医学科研工作", "title": "科研项目管理策略科研项目,项目管理,实施,必要性,策略"}]
 
 input_ids, token_type_ids = [], []
 for example in list_data:
-    input_id, token_type_id = convert_example(example,
-                                              tokenizer,
-                                              max_seq_length=max_seq_len)
+    input_id, token_type_id = convert_example(example, tokenizer, max_seq_length=max_seq_len)
     input_ids.append(input_id)
     token_type_ids.append(token_type_id)
 
 feed_dict = {}
-feed_dict['input_ids'] = np.array(input_ids)
-feed_dict['token_type_ids'] = np.array(token_type_ids)
+feed_dict["input_ids"] = np.array(input_ids)
+feed_dict["token_type_ids"] = np.array(token_type_ids)
 # batch设置为True表示的是批量预测
 b_start = time.time()
 result = client.predict(feed=feed_dict, fetch=fetch_names, batch=True)
 # breakpoint()
-sim_score = expit(result['predict'])[:, 1]
+sim_score = expit(result["predict"])[:, 1]
 b_end = time.time()
 print("time to cost :{} seconds".format(b_end - b_start))
 print(sim_score)

applications/neural_search/ranking/cross_encoder/deploy/python/predict.py

@@ -16,73 +16,51 @@ import argparse
 import os
 import sys
 
-from scipy.special import softmax
-from scipy.special import expit
 import numpy as np
-import pandas as pd
-
 import paddle
 from paddle import inference
-import paddle.nn.functional as F
-from paddlenlp.data import Stack, Tuple, Pad
-from paddlenlp.datasets import load_dataset
-from paddlenlp.utils.log import logger
-from paddlenlp.transformers import AutoTokenizer
+from scipy.special import softmax
 
-sys.path.append('.')
+from paddlenlp.data import Pad, Tuple
+from paddlenlp.datasets import load_dataset
+from paddlenlp.transformers import AutoTokenizer
+from paddlenlp.utils.log import logger
+
+sys.path.append(".")
 
 # yapf: disable
 parser = argparse.ArgumentParser()
-parser.add_argument("--model_dir", type=str, required=True,
-    help="The directory to static model.")
-
-parser.add_argument("--input_file", type=str, required=True,
-    help="The test set file.")
-parser.add_argument("--max_seq_length", default=128, type=int,
-    help="The maximum total input sequence length after tokenization. Sequences "
-    "longer than this will be truncated, sequences shorter will be padded.")
-parser.add_argument("--batch_size", default=32, type=int,
-    help="Batch size per GPU/CPU for training.")
-parser.add_argument('--device', choices=['cpu', 'gpu', 'xpu'], default="gpu",
-    help="Select which device to train model, defaults to gpu.")
-
-parser.add_argument('--use_tensorrt', default=False, type=eval, choices=[True, False],
-    help='Enable to use tensorrt to speed up.')
-parser.add_argument("--precision", default="fp32", type=str, choices=["fp32", "fp16", "int8"],
-    help='The tensorrt precision.')
-
-parser.add_argument('--cpu_threads', default=10, type=int,
-    help='Number of threads to predict when using cpu.')
-parser.add_argument('--enable_mkldnn', default=False, type=eval, choices=[True, False],
-    help='Enable to use mkldnn to speed up when using cpu.')
-
-parser.add_argument("--benchmark", type=eval, default=False,
-    help="To log some information about environment and running.")
-parser.add_argument("--save_log_path", type=str, default="./log_output/",
-    help="The file path to save log.")
+parser.add_argument("--model_dir", type=str, required=True, help="The directory to static model.")
+parser.add_argument("--input_file", type=str, required=True, help="The test set file.")
+parser.add_argument("--max_seq_length", default=128, type=int, help="The maximum total input sequence length after tokenization. Sequences longer than this will be truncated, sequences shorter will be padded.")
+parser.add_argument("--batch_size", default=32, type=int, help="Batch size per GPU/CPU for training.")
+parser.add_argument('--device', choices=['cpu', 'gpu', 'xpu'], default="gpu", help="Select which device to train model, defaults to gpu.")
+parser.add_argument('--use_tensorrt', default=False, type=eval, choices=[True, False], help='Enable to use tensorrt to speed up.')
+parser.add_argument("--precision", default="fp32", type=str, choices=["fp32", "fp16", "int8"], help='The tensorrt precision.')
+parser.add_argument('--cpu_threads', default=10, type=int, help='Number of threads to predict when using cpu.')
+parser.add_argument('--enable_mkldnn', default=False, type=eval, choices=[True, False], help='Enable to use mkldnn to speed up when using cpu.')
+parser.add_argument("--benchmark", type=eval, default=False, help="To log some information about environment and running.")
+parser.add_argument("--save_log_path", type=str, default="./log_output/", help="The file path to save log.")
 parser.add_argument('--model_name_or_path', default="rocketqa-base-cross-encoder", help="The pretrained model used for training")
-
 args = parser.parse_args()
 # yapf: enable
 
 
 def read_text_pair(data_path):
     """Reads data."""
-    with open(data_path, 'r', encoding='utf-8') as f:
+    with open(data_path, "r", encoding="utf-8") as f:
         for line in f:
             data = line.rstrip().split("\t")
             if len(data) != 3:
                 continue
-            yield {'query': data[0], 'title': data[1]}
+            yield {"query": data[0], "title": data[1]}
 
 
 def convert_example(example, tokenizer, max_seq_length=512, is_test=False):
 
     query, title = example["query"], example["title"]
 
-    encoded_inputs = tokenizer(text=query,
-                               text_pair=title,
-                               max_seq_len=max_seq_length)
+    encoded_inputs = tokenizer(text=query, text_pair=title, max_seq_len=max_seq_length)
 
     input_ids = encoded_inputs["input_ids"]
     token_type_ids = encoded_inputs["token_type_ids"]
@@ -95,16 +73,17 @@ def convert_example(example, tokenizer, max_seq_length=512, is_test=False):
 
 
 class Predictor(object):
-
-    def __init__(self,
-                 model_dir,
-                 device="gpu",
-                 max_seq_length=128,
-                 batch_size=32,
-                 use_tensorrt=False,
-                 precision="fp32",
-                 cpu_threads=10,
-                 enable_mkldnn=False):
+    def __init__(
+        self,
+        model_dir,
+        device="gpu",
+        max_seq_length=128,
+        batch_size=32,
+        use_tensorrt=False,
+        precision="fp32",
+        cpu_threads=10,
+        enable_mkldnn=False,
+    ):
         self.max_seq_length = max_seq_length
         self.batch_size = batch_size
 
@@ -118,19 +97,19 @@ class Predictor(object):
 
         if device == "gpu":
             # set GPU configs accordingly
-            # such as intialize the gpu memory, enable tensorrt
+            # such as initialize the gpu memory, enable tensorrt
             config.enable_use_gpu(100, 0)
             precision_map = {
                 "fp16": inference.PrecisionType.Half,
                 "fp32": inference.PrecisionType.Float32,
-                "int8": inference.PrecisionType.Int8
+                "int8": inference.PrecisionType.Int8,
             }
             precision_mode = precision_map[precision]
 
             if args.use_tensorrt:
-                config.enable_tensorrt_engine(max_batch_size=batch_size,
-                                              min_subgraph_size=30,
-                                              precision_mode=precision_mode)
+                config.enable_tensorrt_engine(
+                    max_batch_size=batch_size, min_subgraph_size=30, precision_mode=precision_mode
+                )
         elif device == "cpu":
             # set CPU configs accordingly,
             # such as enable_mkldnn, set_cpu_math_library_num_threads
@@ -146,15 +125,12 @@ class Predictor(object):
 
         config.switch_use_feed_fetch_ops(False)
         self.predictor = paddle.inference.create_predictor(config)
-        self.input_handles = [
-            self.predictor.get_input_handle(name)
-            for name in self.predictor.get_input_names()
-        ]
-        self.output_handle = self.predictor.get_output_handle(
-            self.predictor.get_output_names()[0])
+        self.input_handles = [self.predictor.get_input_handle(name) for name in self.predictor.get_input_names()]
+        self.output_handle = self.predictor.get_output_handle(self.predictor.get_output_names()[0])
 
         if args.benchmark:
             import auto_log
+
             pid = os.getpid()
             self.autolog = auto_log.AutoLogger(
                 model_name=args.model_name_or_path,
@@ -166,11 +142,10 @@ class Predictor(object):
                 pids=pid,
                 process_name=None,
                 gpu_ids=0,
-                time_keys=[
-                    'preprocess_time', 'inference_time', 'postprocess_time'
-                ],
+                time_keys=["preprocess_time", "inference_time", "postprocess_time"],
                 warmup=0,
-                logger=logger)
+                logger=logger,
+            )
 
     def predict(self, data, tokenizer):
         """
@@ -178,7 +153,7 @@ class Predictor(object):
 
         Args:
             data (obj:`List(str)`): The batch data whose each element is a raw text.
-            tokenizer(obj:`PretrainedTokenizer`): This tokenizer inherits from :class:`~paddlenlp.transformers.PretrainedTokenizer` 
+            tokenizer(obj:`PretrainedTokenizer`): This tokenizer inherits from :class:`~paddlenlp.transformers.PretrainedTokenizer`
                 which contains most of the methods. Users should refer to the superclass for more information regarding methods.
             label_map(obj:`dict`): The label id (key) to label str (value) map.
 
@@ -190,17 +165,12 @@ class Predictor(object):
 
         examples = []
         for text in data:
-            input_ids, segment_ids = convert_example(
-                text,
-                tokenizer,
-                max_seq_length=self.max_seq_length,
-                is_test=True)
+            input_ids, segment_ids = convert_example(text, tokenizer, max_seq_length=self.max_seq_length, is_test=True)
             examples.append((input_ids, segment_ids))
 
         batchify_fn = lambda samples, fn=Tuple(
             Pad(axis=0, pad_val=tokenizer.pad_token_id, dtype="int64"),  # input
-            Pad(axis=0, pad_val=tokenizer.pad_token_id, dtype="int64"
-                ),  # segment
+            Pad(axis=0, pad_val=tokenizer.pad_token_id, dtype="int64"),  # segment
         ): fn(samples)
 
         if args.benchmark:
@@ -213,7 +183,7 @@ class Predictor(object):
         sim_score = self.output_handle.copy_to_cpu()
         if args.benchmark:
             self.autolog.times.stamp()
-        sim_score = expit(sim_score)[:, 1]
+        sim_score = softmax(sim_score)[:, 1]
 
         if args.benchmark:
             self.autolog.times.end(stamp=True)
@@ -223,27 +193,29 @@ class Predictor(object):
 
 if __name__ == "__main__":
     # Define predictor to do prediction.
-    predictor = Predictor(args.model_dir, args.device, args.max_seq_length,
-                          args.batch_size, args.use_tensorrt, args.precision,
-                          args.cpu_threads, args.enable_mkldnn)
+    predictor = Predictor(
+        args.model_dir,
+        args.device,
+        args.max_seq_length,
+        args.batch_size,
+        args.use_tensorrt,
+        args.precision,
+        args.cpu_threads,
+        args.enable_mkldnn,
+    )
 
     tokenizer = AutoTokenizer.from_pretrained(args.model_name_or_path)
 
-    test_ds = load_dataset(read_text_pair,
-                           data_path=args.input_file,
-                           lazy=False)
+    test_ds = load_dataset(read_text_pair, data_path=args.input_file, lazy=False)
 
-    data = [{'query': d['query'], 'title': d['title']} for d in test_ds]
+    data = [{"query": d["query"], "title": d["title"]} for d in test_ds]
 
-    batches = [
-        data[idx:idx + args.batch_size]
-        for idx in range(0, len(data), args.batch_size)
-    ]
+    batches = [data[idx : idx + args.batch_size] for idx in range(0, len(data), args.batch_size)]
 
     results = []
     for batch_data in batches:
         results.extend(predictor.predict(batch_data, tokenizer))
     for idx, text in enumerate(data):
-        print('Data: {} \t prob: {}'.format(text, results[idx]))
+        print("Data: {} \t prob: {}".format(text, results[idx]))
     if args.benchmark:
         predictor.autolog.report()

applications/neural_search/ranking/cross_encoder/deploy/python/rpc_client.py

@@ -17,12 +17,9 @@ import numpy as np
 from paddle_serving_server.pipeline import PipelineClient
 
 client = PipelineClient()
-client.connect(['127.0.0.1:8089'])
+client.connect(["127.0.0.1:8089"])
 
-list_data = [{
-    "query": "加强科研项目管理有效促进医学科研工作",
-    "title": "科研项目管理策略科研项目,项目管理,实施,必要性,策略"
-}]
+list_data = [{"query": "加强科研项目管理有效促进医学科研工作", "title": "科研项目管理策略科研项目,项目管理,实施,必要性,策略"}]
 feed = {}
 for i, item in enumerate(list_data):
     feed[str(i)] = str(item)

applications/neural_search/ranking/cross_encoder/deploy/python/web_service.py

@@ -12,20 +12,16 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-import numpy as np
-import sys
 import json
 
-from scipy.special import expit
-from paddle_serving_server.web_service import WebService, Op
+from paddle_serving_server.web_service import Op, WebService
+from scipy.special import softmax
 
 
 def convert_example(example, tokenizer, max_seq_length=512):
 
     query, title = example["query"], example["title"]
-    encoded_inputs = tokenizer(text=query,
-                               text_pair=title,
-                               max_seq_len=max_seq_length)
+    encoded_inputs = tokenizer(text=query, text_pair=title, max_seq_len=max_seq_length)
 
     input_ids = encoded_inputs["input_ids"]
     token_type_ids = encoded_inputs["token_type_ids"]
@@ -34,44 +30,40 @@ def convert_example(example, tokenizer, max_seq_length=512):
 
 
 class ErnieOp(Op):
-
     def init_op(self):
         from paddlenlp.transformers import AutoTokenizer
-        self.tokenizer = AutoTokenizer.from_pretrained(
-            'rocketqa-base-cross-encoder')
+
+        self.tokenizer = AutoTokenizer.from_pretrained("rocketqa-base-cross-encoder")
 
     def preprocess(self, input_dicts, data_id, log_id):
-        from paddlenlp.data import Stack, Tuple, Pad
+        from paddlenlp.data import Pad, Tuple
 
-        (_, input_dict), = input_dicts.items()
+        ((_, input_dict),) = input_dicts.items()
         print("input dict", input_dict)
         batch_size = len(input_dict.keys())
         examples = []
         for i in range(batch_size):
-            example = json.loads(input_dict[str(i)].replace("\'", "\""))
+            example = json.loads(input_dict[str(i)].replace("'", '"'))
             input_ids, segment_ids = convert_example(example, self.tokenizer)
             examples.append((input_ids, segment_ids))
         batchify_fn = lambda samples, fn=Tuple(
-            Pad(axis=0, pad_val=self.tokenizer.pad_token_id, dtype="int64"
-                ),  # input
-            Pad(axis=0, pad_val=self.tokenizer.pad_token_type_id, dtype="int64"
-                ),  # segment
+            Pad(axis=0, pad_val=self.tokenizer.pad_token_id, dtype="int64"),  # input
+            Pad(axis=0, pad_val=self.tokenizer.pad_token_type_id, dtype="int64"),  # segment
         ): fn(samples)
         input_ids, segment_ids = batchify_fn(examples)
         feed_dict = {}
-        feed_dict['input_ids'] = input_ids
-        feed_dict['token_type_ids'] = segment_ids
+        feed_dict["input_ids"] = input_ids
+        feed_dict["token_type_ids"] = segment_ids
         return feed_dict, False, None, ""
 
     def postprocess(self, input_dicts, fetch_dict, data_id, log_id):
         new_dict = {}
-        sim_score = expit(fetch_dict["predict"])[:, 1]
+        sim_score = softmax(fetch_dict["predict"])[:, 1]
         new_dict["predict"] = str(sim_score)
         return new_dict, None, ""
 
 
 class ErnieService(WebService):
-
     def get_pipeline_response(self, read_op):
         ernie_op = ErnieOp(name="ernie", input_ops=[read_op])
         return ernie_op

applications/neural_search/ranking/cross_encoder/evaluate.py

@@ -12,29 +12,24 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-from functools import partial
 import argparse
 import os
 import random
-import time
+from functools import partial
 
 import numpy as np
-import pandas as pd
-from tqdm import tqdm
 import paddle
 import paddle.nn.functional as F
-from paddlenlp.data import Stack, Tuple, Pad
-from paddlenlp.datasets import load_dataset
-from paddlenlp.transformers import LinearDecayWithWarmup, AutoModelForSequenceClassification, AutoTokenizer
+from data import convert_example, create_dataloader, read_data
 
-from data import create_dataloader
-from data import convert_example, read_data
+from paddlenlp.data import Pad, Stack, Tuple
+from paddlenlp.datasets import load_dataset
+from paddlenlp.transformers import AutoModelForSequenceClassification, AutoTokenizer
 
 # yapf: disable
 parser = argparse.ArgumentParser()
 parser.add_argument("--test_file", type=str, required=True, help="The full path of test file")
-parser.add_argument("--max_seq_length", default=128, type=int, help="The maximum total input sequence length after tokenization. "
-    "Sequences longer than this will be truncated, sequences shorter will be padded.")
+parser.add_argument("--max_seq_length", default=128, type=int, help="The maximum total input sequence length after tokenization. Sequences longer than this will be truncated, sequences shorter will be padded.")
 parser.add_argument("--batch_size", default=32, type=int, help="Batch size per GPU/CPU for training.")
 parser.add_argument('--model_name_or_path', default="rocketqa-base-cross-encoder", help="The pretrained model used for training")
 parser.add_argument("--init_from_ckpt", type=str, default=None, help="The path of checkpoint to be loaded.")
@@ -79,7 +74,6 @@ def evaluate(model, metric, data_loader, phase="dev"):
 
 def main():
     paddle.set_device(args.device)
-    rank = paddle.distributed.get_rank()
     if paddle.distributed.get_world_size() > 1:
         paddle.distributed.init_parallel_env()
 
@@ -87,35 +81,26 @@ def main():
 
     dev_ds = load_dataset(read_data, data_path=args.test_file, lazy=False)
 
-    model = AutoModelForSequenceClassification.from_pretrained(
-        args.model_name_or_path, num_classes=2)
+    model = AutoModelForSequenceClassification.from_pretrained(args.model_name_or_path, num_classes=2)
     tokenizer = AutoTokenizer.from_pretrained(args.model_name_or_path)
 
-    trans_func_eval = partial(convert_example,
-                              tokenizer=tokenizer,
-                              max_seq_length=args.max_seq_length,
-                              is_pair=True)
+    trans_func_eval = partial(convert_example, tokenizer=tokenizer, max_seq_length=args.max_seq_length, is_pair=True)
 
     batchify_fn_eval = lambda samples, fn=Tuple(
-        Pad(axis=0, pad_val=tokenizer.pad_token_id, dtype="int64"
-            ),  # pair_input
-        Pad(axis=0, pad_val=tokenizer.pad_token_type_id, dtype="int64"
-            ),  # pair_segment
-        Stack(dtype="int64")  # label
+        Pad(axis=0, pad_val=tokenizer.pad_token_id, dtype="int64"),  # pair_input
+        Pad(axis=0, pad_val=tokenizer.pad_token_type_id, dtype="int64"),  # pair_segment
+        Stack(dtype="int64"),  # label
     ): [data for data in fn(samples)]
 
-    dev_data_loader = create_dataloader(dev_ds,
-                                        mode='dev',
-                                        batch_size=args.batch_size,
-                                        batchify_fn=batchify_fn_eval,
-                                        trans_fn=trans_func_eval)
+    dev_data_loader = create_dataloader(
+        dev_ds, mode="dev", batch_size=args.batch_size, batchify_fn=batchify_fn_eval, trans_fn=trans_func_eval
+    )
 
     if args.init_from_ckpt and os.path.isfile(args.init_from_ckpt):
         state_dict = paddle.load(args.init_from_ckpt)
         model.set_dict(state_dict)
     else:
-        raise ValueError(
-            "Please set --params_path with correct pretrained model file")
+        raise ValueError("Please set --params_path with correct pretrained model file")
 
     metric = paddle.metric.Auc()
     evaluate(model, metric, dev_data_loader, "dev")

applications/neural_search/ranking/cross_encoder/export_model.py

@@ -15,10 +15,8 @@
 import argparse
 import os
 
-import numpy as np
 import paddle
-import paddle.nn.functional as F
-from paddlenlp.data import Stack, Tuple, Pad
+
 from paddlenlp.transformers import AutoModelForSequenceClassification, AutoTokenizer
 
 # yapf: disable
@@ -31,27 +29,24 @@ args = parser.parse_args()
 
 if __name__ == "__main__":
     tokenizer = AutoTokenizer.from_pretrained(args.model_name_or_path)
-    model = AutoModelForSequenceClassification.from_pretrained(
-        args.model_name_or_path, num_classes=2)
+    model = AutoModelForSequenceClassification.from_pretrained(args.model_name_or_path, num_classes=2)
 
     if args.params_path and os.path.isfile(args.params_path):
         state_dict = paddle.load(args.params_path)
         model.set_dict(state_dict)
         print("Loaded parameters from %s" % args.params_path)
     else:
-        raise ValueError(
-            "Please set --params_path with correct pretrained model file")
+        raise ValueError("Please set --params_path with correct pretrained model file")
     model.eval()
 
     # Convert to static graph with specific input description
     model = paddle.jit.to_static(
         model,
         input_spec=[
-            paddle.static.InputSpec(shape=[None, None],
-                                    dtype="int64"),  # input_ids
-            paddle.static.InputSpec(shape=[None, None],
-                                    dtype="int64")  # segment_ids
-        ])
+            paddle.static.InputSpec(shape=[None, None], dtype="int64"),  # input_ids
+            paddle.static.InputSpec(shape=[None, None], dtype="int64"),  # segment_ids
+        ],
+    )
     # Save in static graph model.
     save_path = os.path.join(args.output_path, "inference")
     paddle.jit.save(model, save_path)

applications/neural_search/ranking/cross_encoder/export_to_serving.py

@@ -14,6 +14,7 @@
 
 import argparse
 import paddle_serving_client.io as serving_io
+
 # yapf: disable
 parser = argparse.ArgumentParser()
 parser.add_argument("--dirname", type=str, required=True,
@@ -29,7 +30,7 @@ parser.add_argument("--client_path", type=str, default='./serving_client',
 parser.add_argument("--feed_alias_names", type=str, default=None,
                     help='set alias names for feed vars, split by comma \',\', you should run --show_proto to check the number of feed vars')
 parser.add_argument("--fetch_alias_names", type=str, default=None,
-                    help='set alias names for feed vars, split by comma \',\', you should run --show_proto to check the number of fetch vars')
+                    help='set alias names for fetch vars, split by comma \',\', you should run --show_proto to check the number of fetch vars')
 parser.add_argument("--show_proto", type=bool, default=False,
                     help='If yes, you can preview the proto and then determine your feed var alias name and fetch var alias name.')
 # yapf: enable
@@ -44,4 +45,5 @@ if __name__ == "__main__":
         params_filename=args.params_filename,
         show_proto=args.show_proto,
         feed_alias_names=args.feed_alias_names,
-        fetch_alias_names=args.fetch_alias_names)
+        fetch_alias_names=args.fetch_alias_names,
+    )

applications/neural_search/ranking/cross_encoder/predict.py

@@ -18,18 +18,16 @@ from functools import partial
 
 import paddle
 import paddle.nn.functional as F
-from paddlenlp.datasets import load_dataset
-from paddlenlp.data import Stack, Tuple, Pad
-from tqdm import tqdm
-from paddlenlp.transformers import AutoTokenizer, AutoModel, AutoModelForSequenceClassification
+from data import convert_example, create_dataloader, read_text_pair
 
-from data import convert_example, read_text_pair, create_dataloader
+from paddlenlp.data import Pad, Tuple
+from paddlenlp.datasets import load_dataset
+from paddlenlp.transformers import AutoModelForSequenceClassification, AutoTokenizer
 
 # yapf: disable
 parser = argparse.ArgumentParser()
 parser.add_argument("--params_path", type=str, required=True, default="checkpoints/model_900/model_state.pdparams", help="The path to model parameters to be loaded.")
-parser.add_argument("--max_seq_length", type=int, default=128, help="The maximum total input sequence length after tokenization. "
-    "Sequences longer than this will be truncated, sequences shorter will be padded.")
+parser.add_argument("--max_seq_length", type=int, default=128, help="The maximum total input sequence length after tokenization. Sequences longer than this will be truncated, sequences shorter will be padded.")
 parser.add_argument("--batch_size", type=int, default=32, help="Batch size per GPU/CPU for training.")
 parser.add_argument("--test_set", type=str, required=True, help="The full path of test_set.")
 parser.add_argument("--topk", type=int, default=10, help="The Topk texts.")
@@ -57,35 +55,28 @@ if __name__ == "__main__":
     paddle.set_device(args.device)
 
     test_ds = load_dataset(read_text_pair, data_path=args.test_set, lazy=False)
-    model = AutoModelForSequenceClassification.from_pretrained(
-        args.model_name_or_path, num_classes=2)
+    model = AutoModelForSequenceClassification.from_pretrained(args.model_name_or_path, num_classes=2)
     tokenizer = AutoTokenizer.from_pretrained(args.model_name_or_path)
 
-    trans_func = partial(convert_example,
-                         tokenizer=tokenizer,
-                         max_seq_length=args.max_seq_length,
-                         is_test=True,
-                         is_pair=True)
+    trans_func = partial(
+        convert_example, tokenizer=tokenizer, max_seq_length=args.max_seq_length, is_test=True, is_pair=True
+    )
 
     batchify_fn = lambda samples, fn=Tuple(
         Pad(axis=0, pad_val=tokenizer.pad_token_id, dtype="int64"),  # input
-        Pad(axis=0, pad_val=tokenizer.pad_token_type_id, dtype="int64"
-            ),  # segment
+        Pad(axis=0, pad_val=tokenizer.pad_token_type_id, dtype="int64"),  # segment
     ): [data for data in fn(samples)]
 
-    test_data_loader = create_dataloader(test_ds,
-                                         mode='predict',
-                                         batch_size=args.batch_size,
-                                         batchify_fn=batchify_fn,
-                                         trans_fn=trans_func)
+    test_data_loader = create_dataloader(
+        test_ds, mode="predict", batch_size=args.batch_size, batchify_fn=batchify_fn, trans_fn=trans_func
+    )
 
     if args.params_path and os.path.isfile(args.params_path):
         state_dict = paddle.load(args.params_path)
         model.set_dict(state_dict)
         print("Loaded parameters from %s" % args.params_path)
     else:
-        raise ValueError(
-            "Please set --params_path with correct pretrained model file")
+        raise ValueError("Please set --params_path with correct pretrained model file")
     results = predict(model, test_data_loader)
     test_ds = load_dataset(read_text_pair, data_path=args.test_set, lazy=False)
     text_pairs = []
@@ -93,7 +84,6 @@ if __name__ == "__main__":
         text_pair = test_ds[idx]
         text_pair["pred_prob"] = prob
         text_pairs.append(text_pair)
-    text_pairs = sorted(text_pairs, key=lambda x: x['pred_prob'],
-                        reverse=True)[:args.topk]
+    text_pairs = sorted(text_pairs, key=lambda x: x["pred_prob"], reverse=True)[: args.topk]
     for item in text_pairs:
         print(item)