No project description provided
Welcome to Pyccel
Pyccel stands for Python extension language using accelerators.
The aim of Pyccel is to provide a simple way to generate automatically, parallel low level code. The main uses would be:
- Convert a Python code (or project) into a Fortran or C code.
- Accelerate Python functions by converting them to Fortran or C functions.
Pyccel can be viewed as:
- Python-to-Fortran/C converter
- a compiler for a Domain Specific Language with Python syntax
Pyccel comes with a selection of extensions allowing you to convert calls to some specific Python packages to Fortran/C. The following packages will be covered (partially):
mpi4py(not available yet)
h5py(not available yet)
Pyccel's acceleration capabilities lead to much faster code. Comparisons of Python vs Pyccel or other tools can be found in the benchmarks repository. The results for the master branch currently show the following performance on python 3.10:
If you are eager to try Pyccel out, we recommend reading our quick-start guide
Table of contents
Pyccel Installation Methods
Pyccel can be installed on virtually any machine that provides Python 3, the pip package manager, a C/Fortran compiler, and an Internet connection. Some advanced features of Pyccel require additional non-Python libraries to be installed, for which we provide detailed instructions below.
Alternatively, Pyccel can be deployed through a Linux Docker image that contains all dependencies, and which can be setup with any version of Pyccel. For more information, please read the section on Pyccel container images.
It is possible to use Pyccel with anaconda but this is generally not advised as anaconda modifies paths used for finding executables, shared libraries and other objects. Support is provided for anaconda on linux/macOS.
On Windows support is limited to examples which do not use external libraries. This is because we do not know of a way to reliably avoid DLL hell. As a result DLLs managed by conda are always loaded before DLLs related to the compiler.
First of all, Pyccel requires a working Fortran/C compiler
For Fortran it supports
For C it supports
In order to perform fast linear algebra calculations, Pyccel uses the following libraries:
Finally, Pyccel supports distributed-memory parallel programming through the Message Passing Interface (MPI) standard; hence it requires an MPI library like
We recommend using GFortran/GCC and Open-MPI.
Pyccel also depends on several Python3 packages, which are automatically downloaded by pip, the Python Package Installer, during the installation process. In addition to these, unit tests require additional packages which are installed as optional dependencies with pip, while building the documentation requires Sphinx.
To install all requirements on a Linux Ubuntu machine, just use APT, the Advanced Package Tool:
sudo apt update sudo apt install gcc sudo apt install gfortran sudo apt install libblas-dev liblapack-dev sudo apt install libopenmpi-dev openmpi-bin sudo apt install libomp-dev libomp5
Install all requirements using the DNF software package manager:
su dnf check-update dnf install gcc dnf install gfortran dnf install blas-devel lapack-devel dnf install openmpi-devel dnf install libgomp exit
Similar commands work on Linux openSUSE, just replace
Mac OS X
On an Apple Macintosh machine we recommend using Homebrew:
brew update brew install gcc brew install openblas brew install lapack brew install open-mpi brew install libomp
This requires that the Command Line Tools (CLT) for Xcode are installed.
Support for Windows is still experimental, and the installation of all requirements is more cumbersome. We recommend using Chocolatey to speed up the process, and we provide commands that work in a git-bash sh. In an Administrator prompt install git-bash (if needed), a Python3 distribution, and a GCC compiler:
choco install git choco install python3 choco install mingw
Download x64 BLAS and LAPACK DLLs from https://icl.cs.utk.edu/lapack-for-windows/lapack/:
WEB_ADDRESS=https://icl.cs.utk.edu/lapack-for-windows/libraries/VisualStudio/3.7.0/Dynamic-MINGW/Win64 LIBRARY_DIR=/c/ProgramData/chocolatey/lib/mingw/tools/install/mingw64/lib curl $WEB_ADDRESS/libblas.dll -o $LIBRARY_DIR/libblas.dll curl $WEB_ADDRESS/liblapack.dll -o $LIBRARY_DIR/liblapack.dll
Generate static MS C runtime library from corresponding dynamic link library:
cd "$LIBRARY_DIR" cp $SYSTEMROOT/SysWOW64/vcruntime140.dll . gendef vcruntime140.dll dlltool -d vcruntime140.def -l libmsvcr140.a -D vcruntime140.dll cd -
Download MS MPI runtime and SDK, then install MPI:
WEB_ADDRESS=https://github.com/microsoft/Microsoft-MPI/releases/download/v10.1.1 curl -L $WEB_ADDRESS/msmpisetup.exe -o msmpisetup.exe curl -L $WEB_ADDRESS/msmpisdk.msi -o msmpisdk.msi ./msmpisetup.exe msiexec //i msmpisdk.msi
At this point, close and reopen your terminal to refresh all environment variables!
In Administrator git-bash, generate
mpi.mod for GFortran according to https://abhilashreddy.com/writing/3/mpi_instructions.html:
cd "$MSMPI_INC" sed -i 's/mpifptr.h/x64\/mpifptr.h/g' mpi.f90 sed -i 's/mpifptr.h/x64\/mpifptr.h/g' mpif.h gfortran -c -D_WIN64 -D INT_PTR_KIND\(\)=8 -fno-range-check mpi.f90 cd -
cd "$MSMPI_LIB64" cp $SYSTEMROOT/SysWOW64/msmpi.dll . gendef msmpi.dll dlltool -d msmpi.def -l libmsmpi.a -D msmpi.dll cd -
On Windows it is important that all locations containing DLLs are on the PATH. If you have added any variables to locations which are not on the PATH then you need to add them:
echo $PATH export PATH=$LIBRARY_DIR;$PATH
As of Python 3.8 it is also important to tell Python which directories contain trusted DLLs. In order to use Pyccel this should include all folders containing DLLs used by your chosen compiler. The function which communicates this to Python is:
import os os.add_dll_directory(C://ProgramData/chocolatey/lib/mingw/tools/install/mingw64/lib') os.add_dll_directory('C://ProgramData/chocolatey/lib/mingw/tools/install/mingw64/bin')
These commands must be run every time a Python instance is opened which will import a Pyccel-generated library.
If you use Pyccel often and aren't scared of debugging any potential DLL confusion from other libraries. You can use a
.pth file to run the necessary commands automatically. The location where the
.pth file should be installed is described in the python docs. Once the site is located you can run:
echo "import os; os.add_dll_directory('C://ProgramData/chocolatey/lib/mingw/tools/install/mingw64/lib'); os.add_dll_directory('C://ProgramData/chocolatey/lib/mingw/tools/install/mingw64/bin')" > $SITE_PATH/dll_path.pth
(The command may need adapting for your installation locations)
On Windows and/or Anaconda Python, use
pip instead of
pip3 for the Installation of Pyccel below.
Simply run, for a user-specific installation:
pip3 install --user pyccel
sudo pip3 install pyccel
for a system-wide installation.
git clone firstname.lastname@example.org:pyccel/pyccel.git cd pyccel pip3 install --user .
git clone email@example.com:pyccel/pyccel.git cd pyccel pip3 install --user -e .[test]
this will install a python library Pyccel and a binary called
Any required Python packages will be installed automatically from PyPI.
On a read-only system
If the folder where Pyccel is saved is read only, it may be necessary to run an additional command after installation or updating:
This step is necessary in order to pickle header files. If this command is not run then Pyccel will still run correctly but may be slower when using OpenMP or other supported external packages. A warning, reminding the user to execute this command, will be printed to the screen when pyccelising files which rely on these packages if the pickling step has not been executed.
In order to run the unit tests and to get a coverage report, a few additional Python packages should be installed:
pip install --user -e .[test]
Most of the unit tests can also be run in parallel.
To test your Pyccel installation please run the script
tests/run\_tests\_py3.sh (Unix), or
Continuous testing runs on GitHub actions: https://github.com/pyccel/pyccel/actions?query=branch%3Amaster
Pyccel Container Images
Pyccel container images are available through both Docker Hub (<docker.io>) and the GitHub Container Registry (<ghcr.io>).
- are based on
- use distro packaged python3, GCC, GFortran, BLAS and OpenMPI
- support all Pyccel releases except the legacy "0.1"
Image tags match Pyccel releases.
In order to implement your Pyccel-accelerated code, you can use a host based volume during the Pyccel container creation.
docker pull pyccel/pyccel:v1.0.0 docker run -it -v $PWD:/data:rw pyccel/pyccel:v1.0.0 bash
If you are using SELinux, you will need to set the right context for your host based volume.
Alternatively you may have docker or podman set the context using
-v $PWD:/data:rwz instead of
-v $PWD:/data:rw .
Release history Release notifications | RSS feed
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.