OSA: OPEN-SOURCE ADVISOR

Built with:

Overview

OSA (Open-Source-Advisor) is a LLM-based tool for improving the quality of scientific open source projects and helping create them from scratch. It automates the generation of README, different levels of documentation, CI/CD scripts, etc. It also generates advices and recommendations for the repository.

How it works?

Here is a short video:

Table of contents

• Core features
• Installation
• Getting started
• Examples
• Documentation
• Chat with developers: OSA_helpdesk
• Publications about OSA
• Contributing
• License
• Acknowledgments
• Citation

Core features

1. README file generation: Automates the creation of a clear and structured README file for a repository, including projects based on research papers.

2. Documentation generation: Automatically generates docstrings for Python code.

3. Automatic implementation of changes: Clones the repository, creates a branch, commits and pushes changes, and creates a pull request with proposed changes.

4. Various LLMs: Use OSA with an LLM accessible via API (e.g., OpenAI, VseGPT, Ollama), a local server, or try an osa_bot hosted on ITMO servers.

5. Workflow Generator: Automatically generates customizable CI/CD workflows for Python repositories, including unit tests, code formatting, PEP 8 compliance checks, and PyPI publication.

6. Repository-quality score: Evaluates a repository against formal criteria such as a non-empty README and license file. Its standalone artifact is a provenance-bearing report.json; run it independently with python -m osa_tool.tools.repository_quality or as part of paper analysis.

7. Standalone paper claims pipeline: Extracts technical claims from PDF papers through the reusable paper_claims operation and batch utilities. This pipeline is available as a separate module and is not registered in the scheduler yet.

8. Paper repository analysis: The canonical osa-tool --paper-analysis mode extracts or resumes typed paper claims and verifies them against a repository. Formal repository-quality scoring is optional through --include-repository-quality; the focused python -m osa_tool.tools.paper_analysis wrapper runs the same pipeline. With --delete-dir, both entry points remove only a remote clone created by that invocation, never a user-supplied local repository.

Installation

Install Open-Source-Advisor using one of the following methods:

Using PyPi:

pip install osa_tool

Install optional features when needed:

# PDF-to-claims extraction and evaluation utilities
pip install "osa_tool[paper-claims]"

The core package supports Python 3.11 and later. The paper-claims PDF conversion workflow currently requires Python 3.11--3.14 because its Marker dependency stack is not available for Python 3.15+.

Build from source:

1. Clone the Open-Source-Advisor repository:

git clone https://github.com/aimclub/OSA

1. Navigate to the project directory:

cd Open-Source-Advisor

1. Install the project dependencies:

Using pip  

pip install -r requirements.txt

Using poetry  

poetry install 

For a source checkout with both optional feature sets:

poetry install --all-extras

Using docker  

docker build --build-arg GIT_USER_NAME="your-user-name" --build-arg GIT_USER_EMAIL="your-user-email" -f docker/Dockerfile -t {image-name} .

Getting started

Prerequisites

OSA requires Python 3.11 or higher.

The .env file is required to specify the LLM API key (OPENAI_API_KEY or AUTHORIZATION_KEY) and optionally a Git token. The Git token (GIT_TOKEN) may be omitted if you plan to work with a public repository without creating a fork (using the --no-fork option).

Alternatively, instead of GIT_TOKEN, you can use GITHUB_TOKEN, GITLAB_TOKEN, or GITVERSE_TOKEN for GitHub, GitLab, and Gitverse respectively.

When running osa-tool from CLI, you need to set the GIT_TOKEN and API key first:

# Linux / macOS (bash/zsh)
export OPENAI_API_KEY=<your_api_key>
export GIT_TOKEN=<your_git_token>

# Windows (PowerShell)
setx OPENAI_API_KEY "<your_api_key>"
setx GIT_TOKEN "<your_git_token>"

Tokens

Usage

Run Open-Source-Advisor using the following command:

Using pip  

python -m osa_tool.run -r {repository} [--api {api}] [--base-url {base_url}] [--model {model_name}] [--attachment {article}] [--convert-notebooks {notebook_paths}]

Using docker  

docker run --env-file .env {image-name} -r {repository} [--api {api}] [--base-url {base_url}] [--model {model_name}] [--attachment {article}] [--convert-notebooks {notebook_paths}]

The --attachment option enables you to choose a README template for a repository based on an article. You can provide either a link to a PDF file of the article or a path to a local PDF file after the --attachment option. If you are using Docker, ensure that you upload the PDF file to the OSA folder before building the image, then, specify the path as /app/OSA/... or just use volume mounting to access the file.

The --generate-workflows option is intended to create customizable CI/CD pipelines for Python repositories. For detailed documentation, see the Workflow Generator README.

Configuration

Also OSA supports custom configuration via TOML files. Use the --config-file option to specify a path to custom configuration file. If no custom configuration file is provided, OSA will use the default configuration.

By default, OSA uses a single model for all tasks (specified via --model). If you want to use different models for different types of tasks, disable the --use-single-model flag and specify models for each task type ( --model-docstring, --model-readme, --model-validation, --model-general).

To learn how to work with the interactive CLI and view descriptions of all available keys, visit the CLI usage guide.

Examples

Examples of generated README files are available in examples.

URL of the GitHub/GitLab/Gitverse repository, LLM API service provider (optional) and Specific LLM model to use (optional) are required to use the generator.

Supported LLM providers are available as part of the ProtoLLM ecosystem. See the connectors directory for the full list.

Local ITMO model:

python -m osa_tool.run -r https://github.com/aimclub/OSA --base-url [ITMO_MODEL_URL]

For this API provider itmo model url must be specified in dotenv (ITMO_MODEL_URL=) or in the --base-url argument.

OpenAI:

python -m osa_tool.run -r https://github.com/aimclub/OSA --api openai

VseGPT:

python -m osa_tool.run -r https://github.com/aimclub/OSA --api openai --base-url https://api.vsegpt.ru/v1 --model openai/gpt-3.5-turbo

Openrouter:

python -m osa_tool.run -r https://github.com/aimclub/OSA --api openai --base-url https://openrouter.ai/api/v1 --model qwen/qwen3-30b-a3b-instruct-2507

Ollama:

python -m osa_tool.run -r https://github.com/aimclub/OSA --api ollama --base-url http://[YOUR_OLLAMA_IP]:11434 --model gemma3:27b

Documentation

Detailed description of OSA API is available here.

Chat with developers: OSA_helpdesk

In our Telegram chat OSA_helpdesk you can ask questions about working with OSA and find the latest news about the project.

Publications about OSA

In English:

• Automate Your Coding with OSA – ITMO-Made AI Assistant for Researchers

In Russian:

• OSA: ИИ-помощник для разработчиков научного open source кода

Contributing

• Report Issues: Submit bugs found or log feature requests for the Open-Source-Advisor project.

License

This project is protected under the BSD 3-Clause "New" or "Revised" License. For more details, refer to the LICENSE file.

Acknowledgments

The project is supported as ITMO University Research Project in AI Initiative (RPAII).

OSA is tested by the members of ITMO OpenSource community. Useful content from community is available in Open-source-ops

Also, we thank Readme-ai for their code that we used as a foundation for our own version of README generator.

Citation

If you use this software, please cite it as below.

Simple format

Nikitin N. et al. An LLM-Powered Tool for Enhancing Scientific Open-Source Repositories // Championing Open-source DEvelopment in ML Workshop@ ICML25.

BibTeX format

    @inproceedings{nikitinllm,
    title={An LLM-Powered Tool for Enhancing Scientific Open-Source Repositories},
    author={Nikitin, Nikolay and Getmanov, Andrey and Popov, Zakhar and 
        Ulyanova Ekaterina and Aksenkin, Yaroslav and 
        Sokolov, Ilya and Boukhanovsky, Alexander},
    booktitle={Championing Open-source DEvelopment in ML Workshop@ ICML25}}