Ascend Extension for PyTorch
Overview
This repository develops the PyTorch Extension named torch_npu to adapt Ascend NPU to PyTorch so that developers who use the PyTorch can obtain powerful compute capabilities of Ascend AI Processors.
Ascend is a full-stack AI computing infrastructure for industry applications and services based on Huawei Ascend processors and software. For more information about Ascend, see Ascend Community.
Installation
From Binary
Provide users with wheel package to quickly install torch_npu. Before installing torch_npu, complete the installation of CANN according to Ascend Auxiliary Software. To obtain the CANN installation package, refer to the CANN Installation.
- Install PyTorch
Install PyTorch through pip.
For Aarch64:
pip3 install torch==2.1.0
For x86:
pip3 install torch==2.1.0+cpu --index-url https://download.pytorch.org/whl/cpu
- Install torch-npu dependencies
Run the following command to install dependencies.
pip3 install pyyaml
pip3 install setuptools
If the installation fails, use the download link or visit the PyTorch official website to download the installation package of the corresponding version.
| OS arch | Python version | link |
|---|---|---|
| x86 | Python3.8 | link |
| x86 | Python3.9 | link |
| x86 | Python3.10 | link |
| x86 | Python3.11 | link |
| aarch64 | Python3.8 | link |
| aarch64 | Python3.9 | link |
| aarch64 | Python3.10 | link |
| aarch64 | Python3.11 | link |
- Install torch-npu
pip3 install torch-npu==2.1.0.post12
From Source
In some special scenarios, users may need to compile torch-npu by themselves.Select a branch in table Ascend Auxiliary Software and a Python version in table PyTorch and Python Version Matching Table first. The docker image is recommended for compiling torch-npu through the following steps(It is recommended to mount the working path only and avoid the system path to reduce security risks), the generated .whl file path is ./dist/. Note that gcc version has the following constraints if you try to compile without using docker image: we recommend the use gcc 10.2 for ARM and gcc 9.3.1 for X86.
-
Clone torch-npu
git clone https://github.com/ascend/pytorch.git -b v2.1.0-7.0.0 --depth 1 -
Build Docker Image
cd pytorch/ci/docker/{arch} # {arch} for X86 or ARM docker build -t manylinux-builder:v1 . -
Enter Docker Container
docker run -it -v /{code_path}/pytorch:/home/pytorch manylinux-builder:v1 bash # {code_path} is the torch_npu source code path -
Compile torch-npu
Take Python 3.8 as an example.
cd /home/pytorch bash ci/build.sh --python=3.8
Tips
If you would like to compile with new C++ ABI, then first run this command, at this point, the recommended compilation environment is same to community torch package: glibc 2.28, gcc 8.5.0
export _GLIBCXX_USE_CXX11_ABI=1
Meanwhile, we support configuring -fabi-version using the following variables，require consistency with the community torch package
export _ABI_VERSION=13
Getting Started
Prerequisites
Initialize CANN environment variable by running the command as shown below.
# Default path, change it if needed.
source /usr/local/Ascend/ascend-toolkit/set_env.sh
Quick Verification
You can quickly experience Ascend NPU by the following simple examples.
import torch
import torch_npu
x = torch.randn(2, 2).npu()
y = torch.randn(2, 2).npu()
z = x.mm(y)
print(z)
User Manual
Refer to API of Ascend Extension for PyTorch for more detailed informations.
PyTorch and Python Version Matching Table
| PyTorch Version | Python Version |
|---|---|
| PyTorch1.11.0 | Python3.7.x(>=3.7.5),Python3.8.x,Python3.9.x,Python3.10.x |
| PyTorch2.1.0 | Python3.8.x,Python3.9.x,Python3.10.x,Python3.11.x |
| PyTorch2.2.0 | Python3.8.x,Python3.9.x,Python3.10.x |
| PyTorch2.3.1 | Python3.8.x,Python3.9.x,Python3.10.x,Python3.11.x |
| PyTorch2.4.0 | Python3.8.x,Python3.9.x,Python3.10.x,Python3.11.x |
| PyTorch2.5.1 | Python3.9.x,Python3.10.x,Python3.11.x |
Ascend Auxiliary Software
PyTorch Extension versions follow the naming convention {PyTorch version}-{Ascend version}, where the former represents the PyTorch version compatible with the PyTorch Extension, and the latter is used to match the CANN version. The detailed matching is as follows:
| CANN Version | Supported PyTorch Version | Supported Extension Version | Github Branch |
|---|---|---|---|
| CANN 8.1.RC1 | 2.5.1 | 2.5.1 | v2.5.1-7.0.0 |
| 2.4.0 | 2.4.0.post4 | v2.4.0-7.0.0 | |
| 2.3.1 | 2.3.1.post6 | v2.3.1-7.0.0 | |
| 2.1.0 | 2.1.0.post12 | v2.1.0-7.0.0 | |
| CANN 8.0.0 | 2.4.0 | 2.4.0.post2 | v2.4.0-6.0.0 |
| 2.3.1 | 2.3.1.post4 | v2.3.1-6.0.0 | |
| 2.1.0 | 2.1.0.post10 | v2.1.0-6.0.0 | |
| CANN 8.0.RC3 | 2.4.0 | 2.4.0 | v2.4.0-6.0.rc3 |
| 2.3.1 | 2.3.1.post2 | v2.3.1-6.0.rc3 | |
| 2.1.0 | 2.1.0.post8 | v2.1.0-6.0.rc3 | |
| CANN 8.0.RC2 | 2.3.1 | 2.3.1 | v2.3.1-6.0.rc2 |
| 2.2.0 | 2.2.0.post2 | v2.2.0-6.0.rc2 | |
| 2.1.0 | 2.1.0.post6 | v2.1.0-6.0.rc2 | |
| 1.11.0 | 1.11.0.post14 | v1.11.0-6.0.rc2 | |
| CANN 8.0.RC1 | 2.2.0 | 2.2.0 | v2.2.0-6.0.rc1 |
| 2.1.0 | 2.1.0.post4 | v2.1.0-6.0.rc1 | |
| 1.11.0 | 1.11.0.post11 | v1.11.0-6.0.rc1 | |
| CANN 7.0.0 | 2.1.0 | 2.1.0 | v2.1.0-5.0.0 |
| 2.0.1 | 2.0.1.post1 | v2.0.1-5.0.0 | |
| 1.11.0 | 1.11.0.post8 | v1.11.0-5.0.0 | |
| CANN 7.0.RC1 | 2.1.0 | 2.1.0.rc1 | v2.1.0-5.0.rc3 |
| 2.0.1 | 2.0.1 | v2.0.1-5.0.rc3 | |
| 1.11.0 | 1.11.0.post4 | v1.11.0-5.0.rc3 | |
| CANN 6.3.RC3.1 | 1.11.0 | 1.11.0.post3 | v1.11.0-5.0.rc2.2 |
| CANN 6.3.RC3 | 1.11.0 | 1.11.0.post2 | v1.11.0-5.0.rc2.1 |
| CANN 6.3.RC2 | 2.0.1 | 2.0.1.rc1 | v2.0.1-5.0.rc2 |
| 1.11.0 | 1.11.0.post1 | v1.11.0-5.0.rc2 | |
| 1.8.1 | 1.8.1.post2 | v1.8.1-5.0.rc2 | |
| CANN 6.3.RC1 | 1.11.0 | 1.11.0 | v1.11.0-5.0.rc1 |
| 1.8.1 | 1.8.1.post1 | v1.8.1-5.0.rc1 | |
| CANN 6.0.1 | 1.5.0 | 1.5.0.post8 | v1.5.0-3.0.0 |
| 1.8.1 | 1.8.1 | v1.8.1-3.0.0 | |
| 1.11.0 | 1.11.0.rc2（beta) | v1.11.0-3.0.0 | |
| CANN 6.0.RC1 | 1.5.0 | 1.5.0.post7 | v1.5.0-3.0.rc3 |
| 1.8.1 | 1.8.1.rc3 | v1.8.1-3.0.rc3 | |
| 1.11.0 | 1.11.0.rc1（beta) | v1.11.0-3.0.rc3 | |
| CANN 5.1.RC2 | 1.5.0 | 1.5.0.post6 | v1.5.0-3.0.rc2 |
| 1.8.1 | 1.8.1.rc2 | v1.8.1-3.0.rc2 | |
| CANN 5.1.RC1 | 1.5.0 | 1.5.0.post5 | v1.5.0-3.0.rc1 |
| 1.8.1 | 1.8.1.rc1 | v1.8.1-3.0.rc1 | |
| CANN 5.0.4 | 1.5.0 | 1.5.0.post4 | 2.0.4.tr5 |
| CANN 5.0.3 | 1.8.1 | 1.5.0.post3 | 2.0.3.tr5 |
| CANN 5.0.2 | 1.5.0 | 1.5.0.post2 | 2.0.2.tr5 |
Hardware support
The Ascend training device includes the following models, all of which can be used as training environments for PyTorch models
| Product series | Product model |
|---|---|
| Atlas Training series products | Atlas 800（model: 9000） |
| Atlas 800（model：9010） | |
| Atlas 900 PoD（model：9000） | |
| Atlas 300T（model：9000） | |
| Atlas 300T Pro（model：9000） | |
| Atlas A2 Training series products | Atlas 800T A2 |
| Atlas 900 A2 PoD | |
| Atlas 200T A2 Box16 | |
| Atlas 300T A2 |
The Ascend inference device includes the following models, all of which can be used as inference environments for large models
| Product series | Product model |
|---|---|
| Atlas 800I A2 Inference product | Atlas 800I A2 |
Suggestions and Communication
Everyone is welcome to contribute to the community. If you have any questions or suggestions, you can submit Github Issues. We will reply to you as soon as possible. Thank you very much.
Branch Maintenance Policies
The version branches of AscendPyTorch have the following maintenance phases:
| Status | Duration | Description |
|---|---|---|
| Planning | 1-3 months | Plan features. |
| Development | 6-12 months | Develop new features and fix issues, regularly release new versions. Different strategies are adopted for different versions of PyTorch, with a regular branch development cycle of 6 months and a long-term support branch development cycle of 12 months. |
| Maintained | 1 year/3.5 years | Regular Release branch for 1 year, Long Term Support branch maintenance for 3.5 years. Fix major issues, do not incorporate new features, and release patch versions based on the impact of fixed bugs. |
| End Of Life (EOL) | N/A | Do not accept any modification to a branch. |
PyTorch Maintenance Policies
| PyTorch | Maintenance Policies | Status | Launch Date | Subsequent Status | EOL Date |
|---|---|---|---|---|---|
| 2.5.1 | Regular Release | Development | 2024/11/08 | Expected to enter maintenance status from April 8, 2025 | |
| 2.4.0 | Regular Release | Development | 2024/10/15 | Expected to enter maintenance status from June 15, 2025 | |
| 2.3.1 | Regular Release | Development | 2024/06/06 | Expected to enter maintenance status from June 7, 2025 | |
| 2.2.0 | Regular Release | Maintained | 2024/04/01 | Expected to enter maintenance free status from September 10, 2025 | |
| 2.1.0 | Long Term Support | Development | 2023/10/15 | Expected to enter maintenance status from September 15, 2025 | |
| 2.0.1 | Regular Release | EOL | 2023/7/19 | 2024/3/14 | |
| 1.11.0 | Long Term Support | Maintained | 2023/4/19 | Expected to enter maintenance free status from September 10, 2025 | |
| 1.8.1 | Long Term Support | EOL | 2022/4/10 | 2023/4/10 | |
| 1.5.0 | Long Term Support | EOL | 2021/7/29 | 2022/7/29 |
Reference Documents
For more detailed information on installation guides, model migration, training/inference tutorials, and API lists, please refer to the PyTorch Ascend Adapter on the HiAI Community.
| Document Name | Document Link |
|---|---|
| AscendPyTorch Installation Guide | link |
| AscendPyTorch Network Model Migration and Training | link |
| AscendPyTorch Operator Adaptation | link |
| AscendPyTorch API List (PyTorch and Custom Interfaces) | link |
License
Ascend Extension for Pytorch has a BSD-style license, as found in the LICENSE file.
2 maintainers
Metadata
2 maintainers
Release files for torch-npu 2.1.0.post12
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Built distributions (wheels)
Total release size: 122.0 MB
Release files / torch_npu-2.1.0.post12-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
| Download URL | torch_npu-2.1.0.post12-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl |
|---|---|
| Size | 16.3 MB |
| Tags | CPython 3.11 Linux glibc 2.17+ x86-64 |
|
SHA-256 checksum How to use checksums |
7fee5c7d24b2990ee694367666a24683c3c1b6f80bc55109eb19e0c1b6e01ac3
|
|
BLAKE2b-256 checksum How to use checksums |
a64e4eefeb22ca70f68971f32789623de7f49d3f119b9ce1f18cb866b658318e
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/5.0.0 CPython/3.10.10
|
Release files / torch_npu-2.1.0.post12-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
| Download URL | torch_npu-2.1.0.post12-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl |
|---|---|
| Size | 14.2 MB |
| Tags | CPython 3.11 Linux glibc 2.17+ ARM64 |
|
SHA-256 checksum How to use checksums |
d37f105bb9b37ad4042dffbb2e3c304de50722f7e8961d78b6a09e3e2fa14958
|
|
BLAKE2b-256 checksum How to use checksums |
902e4db70e12c5b465dababbd82e2e1fdd929f80518da5d7782c434a009d2776
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/5.0.0 CPython/3.10.10
|
Release files / torch_npu-2.1.0.post12-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
| Download URL | torch_npu-2.1.0.post12-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl |
|---|---|
| Size | 16.3 MB |
| Tags | CPython 3.10 Linux glibc 2.17+ x86-64 |
|
SHA-256 checksum How to use checksums |
330707ca8e5cfb96f99703b4c774ad3ebf12172c98dd02f188b227f5b1fd4ace
|
|
BLAKE2b-256 checksum How to use checksums |
234d87d7c08d88b944db126106e5b0da8af1754723a30e26e641c6515f678938
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/5.0.0 CPython/3.10.10
|
Release files / torch_npu-2.1.0.post12-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
| Download URL | torch_npu-2.1.0.post12-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl |
|---|---|
| Size | 14.2 MB |
| Tags | CPython 3.10 Linux glibc 2.17+ ARM64 |
|
SHA-256 checksum How to use checksums |
b7fdc7c7cd9cd7389d2c73f2a7ccd34bb6bc7f47debdddab49bbce3581ee2b7f
|
|
BLAKE2b-256 checksum How to use checksums |
ab626ef0dfcc565de7892dd70a503180e27a6debc4a6c98bfc73b969e6af5b16
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/5.0.0 CPython/3.10.10
|
Release files / torch_npu-2.1.0.post12-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
| Download URL | torch_npu-2.1.0.post12-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl |
|---|---|
| Size | 16.3 MB |
| Tags | CPython 3.9 Linux glibc 2.17+ x86-64 |
|
SHA-256 checksum How to use checksums |
32e0098ab1c28a2720e02cd0421d1198a693205abf597ef7b066a7d34752aff4
|
|
BLAKE2b-256 checksum How to use checksums |
da230dd8243f738179cf9e7ca9547f5102ab31f0f480c64dbd5f10fc0ed63538
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/5.0.0 CPython/3.10.10
|
Release files / torch_npu-2.1.0.post12-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
| Download URL | torch_npu-2.1.0.post12-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl |
|---|---|
| Size | 14.2 MB |
| Tags | CPython 3.9 Linux glibc 2.17+ ARM64 |
|
SHA-256 checksum How to use checksums |
eb17f72106ce029d0f252ea286ebf0cd3e6a7dd180559121b8d50a751ec104cd
|
|
BLAKE2b-256 checksum How to use checksums |
6b8b79cadbe941f560a3d53424f609e63a3a51ab22d360a03eac2cd51901198f
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/5.0.0 CPython/3.10.10
|
Release files / torch_npu-2.1.0.post12-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
| Download URL | torch_npu-2.1.0.post12-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl |
|---|---|
| Size | 16.3 MB |
| Tags | CPython 3.8 Linux glibc 2.17+ x86-64 |
|
SHA-256 checksum How to use checksums |
f1d61eaa67082640f66ad2eaadf453283a539014c5ca473ca6d43b17f202847d
|
|
BLAKE2b-256 checksum How to use checksums |
b916389f3beaf9a8411e4827b9aaa81007fc72e4e514d805afa2714a23b611d8
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/5.0.0 CPython/3.10.10
|
Release files / torch_npu-2.1.0.post12-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
| Download URL | torch_npu-2.1.0.post12-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl |
|---|---|
| Size | 14.2 MB |
| Tags | CPython 3.8 Linux glibc 2.17+ ARM64 |
|
SHA-256 checksum How to use checksums |
a8bf57be297628d2c3e878d7b027945070231a39893e0dc6d96c8f16beff5e93
|
|
BLAKE2b-256 checksum How to use checksums |
f6fee2421ab98a04805d9c51d6ebdd1cb31c6f0aeab0ce39899923c9c23d8944
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/5.0.0 CPython/3.10.10
|
