Skip to main content

Plot how multiple ranks evolved over processing steps - draw a rankflow.

Project description

RankFlow

pypi version badge python version badge license badge monthly downloads badge

Library for plotting multiple ranks evolved over processing steps - drawing a rankflow.

RankFlow

RankFlow is a Python package that allows you to create rank flow plots (bump charts), helping visualize the changes in ranking of nodes.

Initially it was applied to re-ranking visualization of nodes (parts of documents, document chunks) during the retrieval and re-ranking processes within a Retrieval Augmented Generation (RAG) retriever, but the usage is not limited to RAG.

⭐️ Please star the repository if you find it useful.

Installation

pip install rankflow

Usage

plot from pandas DataFrame

Start with creating pandas DataFrame with ranks for each document at each step.

import pandas as pd
import matplotlib.pyplot as plt
from rankflow import RankFlow

data = {"Doc 1": [2, 1, 3, 2], "Doc 2": [1, 2, 1, 3], "Doc 3": [3, 3, 2, 1]}
df = pd.DataFrame(data, index=["Step_1", "Step_2", "Step_3", "Step_4"])

This creates the following DataFrame:

NOTE: The rows of the DataFrame are the steps and the columns are the documents. The values are the ranks of the documents at each step. Remember to define proper column names and index values since they will be used as labels in the plot.

When the DataFrame is ready, then it is time to create RankFlow object and call plot() method.

rf = RankFlow(df=df)
rf.plot()

# save the plot to png
plt.savefig("rankflow.png")

plt.show()

Here is the expected output:

plot from numpy array

You can also create RankFlow object without using pandas DataFrame. You can pass numpy array with ranks for each document at each step and provide labels for steps and documents.

import matplotlib.pyplot as plt
from rankflow import RankFlow
import numpy as np

my_step_labels: list[str] = [
    "Hybrid Search",
    "Cross-encoder",
    "Graph-reranker",
    "Booster",
]
my_chunk_labels: list[str] = [
    "Doc 0",
    "Doc 1",
    "Doc 2",
    "Doc 3",
    "Doc 4",
    "Doc 5",
    "Doc 6",
    "Doc 7",
    "Doc 8",
    "Doc 9",
]
my_ranks = np.array(
    [
        [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
        [3, 0, 2, 4, 1, 6, 7, 9, 5, 8],
        [2, 3, 0, 4, 6, 1, 7, 8, 5, 9],
        [5, 3, 2, 1, 0, 4, 6, 7, 8, 9],
    ]
)

rf = RankFlow(
    ranks=my_ranks,
    step_labels=my_step_labels,
    chunk_labels=my_chunk_labels,
    fig_size=(6, 6),
    title_font_size=24,
)
_ = rf.plot()
plt.show()

This should produce the following plot:

RankFlow

Further reading

There is and blog article describing usage of this package in RAG retriever: RankFlow plot for retriever visual evaluation that might be helpful if you are wondering how to efficienty track rank changes in your retriever and finally visualize them.

License

MIT © Krystian Safjan.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

rankflow-0.1.4.tar.gz (242.9 kB view details)

Uploaded Source

Built Distribution

rankflow-0.1.4-py3-none-any.whl (6.3 kB view details)

Uploaded Python 3

File details

Details for the file rankflow-0.1.4.tar.gz.

File metadata

  • Download URL: rankflow-0.1.4.tar.gz
  • Upload date:
  • Size: 242.9 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: pdm/2.16.1 CPython/3.12.4 Darwin/23.5.0

File hashes

Hashes for rankflow-0.1.4.tar.gz
Algorithm Hash digest
SHA256 077dfd879be8feebb686a5f12908092dbfd0d3a5989cc7e9da0bdb815e88ae8a
MD5 b77c274c5b86031623599be6350215bf
BLAKE2b-256 a2c81c8a76969dee9224cfa5bc28c41f6b66b52bff8b6aadd95df35676ad98e7

See more details on using hashes here.

File details

Details for the file rankflow-0.1.4-py3-none-any.whl.

File metadata

  • Download URL: rankflow-0.1.4-py3-none-any.whl
  • Upload date:
  • Size: 6.3 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: pdm/2.16.1 CPython/3.12.4 Darwin/23.5.0

File hashes

Hashes for rankflow-0.1.4-py3-none-any.whl
Algorithm Hash digest
SHA256 c7935542c55fd394c50f379e716ce5ae579ddecad76b7f08f9fed59742220e47
MD5 11a7c8facd4d5dce30c39a0216089047
BLAKE2b-256 9e967c21d9232e220cad555c57034aab0ccf67a3dd704d1a5399907572e60f18

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page