Adept

_{Automatic Differentiation Engine for Tensor Processing}

Русский | English

Dependencies | Build | Usage example| Developer rules

A research and educational project for developing a platform for training and inference machine learning models, with a GPU backend implemented on the Vulkan API.

Supported platforms:

• Linux x86_64
• Android aarch64

CPU:

• Armv7+: NEON_WITHOUT_AES, NEON, NEON_BF16, SVE, SVE2, SVE_256, SVE2_128;

  Tested on: Qualcomm Snapdragon 870

• x86: SSE2, SSSE3, SSE4, AVX2, AVX3, AVX3_DL, AVX3_ZEN4, AVX3_SPR;

  Tested on: 13th Gen Intel(R) Core(TM) i9-13900HX, AMD Ryzen™ AI 9 HX 370w

GPU:

• Vulkan (NVIDIA, AMD, Intel, Qualcomm, etc.)

  Tested on: NVIDIA RTX 4070 8Gb, Radeon™ 890M × 24, Adreno (TM) 650, Intel(R) Graphics (RPL-S)

Dependencies

• OpenBLAS - operations with 2D matrices
• Highway - cross-platform SIMD intrinsics
• hash_flat_map - fast flat_map container
• oneTBB - parallel algorithms
• Vulkan SDK - low-level API and development tools for GPU
• Python 3.12 - the Python language

Optional dependencies

• pybind11 - binding C++ code to Python
• protobuf - Protocol Buffers library for ONNX support
• ONNX - only the protobuf description of the Open Neural Network Exchange format is used
• OpenMPI - library for distributed message processing

Development environment

The project is developed using C++20(gcc-13.3.0) and the CMake build system.

Instructions for setting up the development environment can be found in the Wiki.

Installation

pip install https://storage.yandexcloud.net/adept-releases/adept-0.2.0-cp312-cp312-manylinux_2_34_x86_64.whl

Getting the source code:

git clone --recursive https://gitverse.ru/adept-platform/adept.git
cd adept
# if you are updating an existing checkout
git submodule sync
git submodule update --init --recursive

Building:

WITH_PYTHON_BINDINGS=1 ./build_dependencies.sh
mkdir -p build && cd build
cmake .. -DWITH_TESTS=ON -DWITH_PYTHON_BINDINGS=ON -DCMAKE_BUILD_TYPE=Release
cmake --build . --target mlp tests adeptpy
cd ../
./build_python_wheel.sh

To build with Vulkan support, use the CMake configuration flag:

cmake -DWITH_VULKAN=ON ...

To build for the Android platform, use the following flags:

BUILD_ANDROID=1 ./build_dependencies.sh
cmake -DBUILD_ANDROID=ON ...

Python interface usage example:

Notebook implementing MNIST classification using LeNet.

Example of a simple perceptron:

class MLP(Module):
    def __init__(self):
        super().__init__("MLP")
        self.l1 = Linear(28 * 28, 512, device, dtype)
        self.l2 = Linear(512, 256, device, dtype)
        self.l3 = Linear(256, 10, device, dtype)

    def forward(self, x):
        out = relu(self.l1(x))
        out = relu(self.l2(out))
        out = self.l3(out)
        return out
...

dataset = MNISTDataset(train_images_file, train_labels_file, dtype, device, True)
dataloader = DataLoader(dataset, batch_size)

mlp = MLP()
optimizer = SGD(mlp.parameters(), lr)

for _ in tqdm(range(epochs), unit="epoch"):
    pbar = tqdm(dataloader, unit="batch")
    for b_i, batch in enumerate(pbar):
        x, y = batch
        out = mlp.forward(Variable(x))

        loss = cross_entropy_with_logits(out.squeeze(1), Variable(y).squeeze(1))
        if b_i % 64 == 0:
            pbar.set_postfix(loss=loss.data().float_at([0, 0]))

        loss.backward()
        optimizer.step()
        optimizer.zero_grad()
    optimizer.set_lr(optimizer.lr() * lr_decay)

More detailed examples can be found in the apps directory.