OSA: OPEN-SOURCE ADVISOR

Built with:

Overview

OSA (Open-Source-Advisor) is a LLM-based tool for improving the quality of scientific open source projects and helping create them from scratch. It automates the generation of README, different levels of documentation, CI/CD scripts, etc. It also generates advices and recommendations for the repository.

How it works?

Here is a short video:

Table of contents

• Core features
• Installation
• Getting started
• Examples
• Documentation
• Chat with developers: OSA_helpdesk
• Publications about OSA
• Contributing
• License
• Acknowledgments
• Citation

Core features

1. README file generation: Automates the creation of a clear and structured README file for a repository, including projects based on research papers.

2. Documentation generation: Automatically generates docstrings for Python code.

3. Automatic implementation of changes: Clones the repository, creates a branch, commits and pushes changes, and creates a pull request with proposed changes.

4. Various LLMs: Use OSA with an LLM accessible via API (e.g., OpenAI, VseGPT, Ollama), a local server, or try an osa_bot hosted on ITMO servers.

5. Workflow Generator: Automatically generates customizable CI/CD workflows for Python repositories, including unit tests, code formatting, PEP 8 compliance checks, and PyPI publication.

6. Repository-quality score: Evaluates a repository against formal criteria such as a non-empty README and license file. The score is available as part of paper analysis with --include-repository-quality.

7. Paper repository analysis: The canonical osa-tool --paper-analysis mode extracts or resumes typed paper claims and verifies them against a repository. Formal repository-quality scoring is optional through --include-repository-quality. With --delete-dir, the command removes only a remote clone created by that invocation, never a user-supplied local repository.

Installation

Install Open-Source-Advisor using one of the following methods:

Using PyPi:

pip install osa_tool

Install optional features when needed:

# PDF-to-claims extraction and evaluation utilities
pip install "osa_tool[paper-claims]"

# Add the optional local Marker parser
pip install "osa_tool[paper-claims,paper-claims-marker]"

The core package and remote PDF parsers support Python 3.11 and later. The optional Marker parser requires Python 3.11--3.14 because its dependency stack is not available for Python 3.15+.

Build from source:

1. Clone the Open-Source-Advisor repository:

git clone https://github.com/aimclub/OSA

1. Navigate to the project directory:

cd Open-Source-Advisor

1. Install the project dependencies:

Using pip  

pip install -r requirements.txt

Using poetry  

poetry install 

For a source checkout with both optional feature sets:

poetry install --all-extras

Using docker  

docker build --build-arg GIT_USER_NAME="your-user-name" --build-arg GIT_USER_EMAIL="your-user-email" -f docker/Dockerfile -t {image-name} .

Getting started

Prerequisites

OSA requires Python 3.11 or higher.

OSA can be configured in two common ways:

# Subscription-backed Host LLM provider (api = "host")
export HOST_LLM_COMMAND=codex
export GIT_TOKEN=<your_git_token>

# OpenAI-compatible API provider (--api openai)
export OPENAI_API_KEY=<your_api_key>
export GIT_TOKEN=<your_git_token>

The Git token (GIT_TOKEN) may be omitted if you plan to work with a public repository without creating a fork (using the --no-fork option). Instead of GIT_TOKEN, you can use GITHUB_TOKEN, GITLAB_TOKEN, or GITVERSE_TOKEN for GitHub, GitLab, and Gitverse respectively. AUTHORIZATION_KEY is only needed when using Gigachat.

Tokens

Usage

Run Open-Source-Advisor using the following command:

Using pip  

python -m osa_tool.run -r {repository} [--api {api}] [--base-url {base_url}] [--model {model_name}] [--attachment {article}] [--convert-notebooks {notebook_paths}]

Using docker  

docker run --env-file .env {image-name} -r {repository} [--api {api}] [--base-url {base_url}] [--model {model_name}] [--attachment {article}] [--convert-notebooks {notebook_paths}]

Docker runs default to the OpenAI-compatible API-key provider because the container cannot inherit a logged-in host command from the user's machine. Pass --api host only when the container has access to a host command or bridge.

The --attachment option enables you to choose a README template for a repository based on an article. You can provide either a link to a PDF file of the article or a path to a local PDF file after the --attachment option. If you are using Docker, ensure that you upload the PDF file to the OSA folder before building the image, then, specify the path as /app/OSA/... or just use volume mounting to access the file.

The --generate-workflows option is intended to create customizable CI/CD pipelines for Python repositories. For detailed documentation, see the Workflow Generator README.

Configuration

Also OSA supports custom configuration via TOML files. Use the --config-file option to specify a path to custom configuration file. If no custom configuration file is provided, OSA will use the default configuration.

OSA uses the shared [llm] model settings unless a task-specific profile is configured. Task profiles fall back to the shared settings when they do not override a value. The --use-single-model flag forces all tasks to use the shared [llm] settings; omit it to allow task-specific profiles and model flags. Supported task model flags include --model-docstring, --model-readme, --model-general, --model-repository-quality, --model-paper-claims, and --model-paper-verification. There is no --model-validation option in the current CLI.

To learn how to work with the interactive CLI and view descriptions of all available keys, visit the CLI usage guide.

Examples

Examples of generated README files are available in examples.

URL of the GitHub/GitLab/Gitverse repository, LLM API service provider (optional) and Specific LLM model to use (optional) are required to use the generator.

OSA supports the subscription-backed Host LLM provider directly. HOST_LLM_COMMAND currently expects the Codex CLI protocol; use the bridge or an API-backed provider when custom generation controls are required. Other LLM providers are available as part of the ProtoLLM ecosystem. See the connectors directory for the full list.

Subscription-backed Host LLM:

export HOST_LLM_COMMAND=codex
python -m osa_tool.run -r https://github.com/aimclub/OSA --api host

OpenAI-compatible API key:

export OPENAI_API_KEY=<your_api_key>
python -m osa_tool.run -r https://github.com/aimclub/OSA --api openai --base-url https://api.openai.com/v1 --model gpt-4o

Local ITMO model:

python -m osa_tool.run -r https://github.com/aimclub/OSA --base-url [ITMO_MODEL_URL]

For this API provider itmo model url must be specified in dotenv (ITMO_MODEL_URL=) or in the --base-url argument.

OpenAI:

python -m osa_tool.run -r https://github.com/aimclub/OSA --api openai

VseGPT:

python -m osa_tool.run -r https://github.com/aimclub/OSA --api openai --base-url https://api.vsegpt.ru/v1 --model openai/gpt-3.5-turbo

Openrouter:

python -m osa_tool.run -r https://github.com/aimclub/OSA --api openai --base-url https://openrouter.ai/api/v1 --model qwen/qwen3-30b-a3b-instruct-2507

Ollama:

python -m osa_tool.run -r https://github.com/aimclub/OSA --api ollama --base-url http://[YOUR_OLLAMA_IP]:11434 --model gemma3:27b

Documentation

Detailed description of OSA API is available here.

Chat with developers: OSA_helpdesk

In our Telegram chat OSA_helpdesk you can ask questions about working with OSA and find the latest news about the project.

Publications about OSA

In English:

• Automate Your Coding with OSA – ITMO-Made AI Assistant for Researchers

In Russian:

• OSA: ИИ-помощник для разработчиков научного open source кода

Contributing

• Report Issues: Submit bugs found or log feature requests for the Open-Source-Advisor project.

License

This project is protected under the BSD 3-Clause "New" or "Revised" License. For more details, refer to the LICENSE file.

Acknowledgments

The project is supported as ITMO University Research Project in AI Initiative (RPAII).

OSA is tested by the members of ITMO OpenSource community. Useful content from community is available in Open-source-ops

Also, we thank Readme-ai for their code that we used as a foundation for our own version of README generator.

Citation

If you use this software, please cite it as below.

Simple format

Nikitin N. et al. An LLM-Powered Tool for Enhancing Scientific Open-Source Repositories // Championing Open-source DEvelopment in ML Workshop@ ICML25.

BibTeX format

    @inproceedings{nikitinllm,
    title={An LLM-Powered Tool for Enhancing Scientific Open-Source Repositories},
    author={Nikitin, Nikolay and Getmanov, Andrey and Popov, Zakhar and 
        Ulyanova Ekaterina and Aksenkin, Yaroslav and 
        Sokolov, Ilya and Boukhanovsky, Alexander},
    booktitle={Championing Open-source DEvelopment in ML Workshop@ ICML25}}