Parse Torrents using PTN and Rank them according to your preferences!
Project description
Rank Torrent Name (RTN) is a Python library designed to parse and rank torrent names based on customizable criteria. It allows users to define their preferences for filtering and ranking torrents, providing a detailed analysis of each torrent's metadata. RTN is perfect for automating the selection of torrents based on quality, resolution, audio, and more.
RTN is mean't to be used as a Version and Ranking System for parsing and scoring scraped torrent results.
Features
- Advanced Torrent Parsing: Utilizes PTN for parsing and enriches metadata with custom-defined patterns.
- Customizable Ranking: Define detailed preferences for ranking torrents based on attributes like resolution, audio quality, and others.
- Flexible Filtering: Easily specify requirements, exclusions, and preferences for torrent selection.
- Comprehensive Ranking Model: Includes a default ranking model that can be customized or extended according to your needs.
- Levenshtein Ratio Comparison: Compares parsed titles with original titles to ensure accuracy.
Installation
pip install rank-torrent-name
or you can add it to your project through Poetry
as well,
poetry add rank-torrent-name
Quick Start
Setting Up Your Preferences
- Create a Settings Model: Begin by defining your preferences in a
SettingsModel
. This includes specifying the required patterns, exclusions, preferences, and custom ranks for various torrent attributes.
from RTN.models import SettingsModel, CustomRank
settings = SettingsModel(
require=["4K", "1080p"],
exclude=["/CAM/i", "TS"],
preferred=["HDR", "/BluRay/"],
custom_ranks={
"uhd": CustomRank(enable=False, fetch=True, rank=120),
"fhd": CustomRank(enable=False, fetch=True, rank=90),
"hd": CustomRank(enable=False, fetch=True, rank=80),
"sd": CustomRank(enable=False, fetch=True, rank=-120),
"bluray": CustomRank(enable=False, fetch=True, rank=80),
"hdr": CustomRank(enable=False, fetch=True, rank=40),
"hdr10": CustomRank(enable=False, fetch=True, rank=50),
"dolby_video": CustomRank(enable=False, fetch=True, rank=-100),
"dts_x": CustomRank(enable=False, fetch=True, rank=0),
"dts_hd": CustomRank(enable=False, fetch=True, rank=0),
"dts_hd_ma": CustomRank(enable=False, fetch=True, rank=0),
"atmos": CustomRank(enable=False, fetch=True, rank=0),
"truehd": CustomRank(enable=False, fetch=True, rank=0),
"ddplus": CustomRank(enable=False, fetch=True, rank=0),
"aac": CustomRank(enable=False, fetch=True, rank=70),
"ac3": CustomRank(enable=False, fetch=True, rank=50),
"remux": CustomRank(enable=False, fetch=True, rank=-1000),
"webdl": CustomRank(enable=False, fetch=True, rank=90),
"repack": CustomRank(enable=False, fetch=True, rank=5),
"proper": CustomRank(enable=False, fetch=True, rank=4),
"dubbed": CustomRank(enable=False, fetch=True, rank=4),
"subbed": CustomRank(enable=False, fetch=True, rank=2),
"av1": CustomRank(enable=False, fetch=True, rank=0),
}
)
:warning: You don't need to set CAM and TS as these are already disregarded by default. This is just an example.
We cover a lot already, so users are able to add their own custom regex patterns without worrying about the basic patterns.
Understanding Fetch and Enable:
fetch
: Determines if RTN should consider a torrent for downloading based on the attribute. True means RTN will fetch torrents matching this criterion.enable
: Controls whether the custom rank value is used in the overall ranking calculation. Disabling it reverts to using the ranking model you set instead. This is useful for toggling custom ranks on and off from a users perspective.rank
: Sets the rank at which that item is graded with.
For example, if we detect a title is 4K or 2160p then we use the uhd
ranking, and add +120 points. The same goes for the rest of the strings in custom_ranks
.
Settings can be easily adjusted at runtime if needed. To enable or disable a specific rank dynamically:
settings.custom_ranks["hdr"].enable = False # To disable HDR ranking
Ranking Torrents
- Rank a Torrent: Feed a torrent title to RTN to parse it and calculate its rank based on your settings.
from RTN import RTN
from RTN.models import DefaultRanking
rtn = RTN(settings=settings, ranking_model=DefaultRanking())
torrent = rtn.rank("Example.Movie.2020.1080p.BluRay.x264-Example", "infohash123456")
- Inspecting the Torrent Object: The returned
Torrent
object includes parsed data and a rank. Access its properties to understand its quality:
print(f"Title: {torrent.data.parsed_title}, Rank: {torrent.rank}")
Sorting Torrents
- Sort Multiple Torrents: If you have multiple torrents, RTN can sort them based on rank, helping you select the best one.
torrents = [rtn.rank(title, "infohash") for title in torrent_titles]
sorted_torrents = RTN.sort(torrents)
Torrent Object
A Torrent
object encapsulates metadata about a torrent, such as its title, parsed information, and rank. Here's an example structure:
Torrent(
raw_title="Example.Movie.2020.1080p.BluRay.x264-Example",
infohash="infohash123456",
data=ParsedData(parsed_title='Example Movie', ...),
fetch=True,
rank=150,
lev_ratio=0.95
)
Understanding SettingsModel and RankingModel
SettingsModel and RankingModel play crucial roles in RTN, offering users flexibility in filtering and ranking torrents according to specific needs. Here's what each model offers and why they are important:
SettingsModel
SettingsModel
is where you define your filtering criteria, including patterns to require, exclude, and prefer in torrent names. This model allows for dynamic configuration of torrent selection based on user-defined patterns and preferences.
Key functionalities:
- Filtering Torrents: Determine which torrents to consider or ignore based on matching patterns.
- Prioritizing Torrents: Indicate preferred attributes that give certain torrents higher precedence.
- Custom Ranks Usage: Decide how specific attributes influence the overall ranking, enabling or disabling custom ranks.
Example usage:
from RTN.models import SettingsModel, CustomRank
settings = SettingsModel(
require=["1080p", "4K"],
exclude=["CAM"],
preferred=["HDR", "/SenSiTivE/"],
custom_ranks={
"uhd": CustomRank(enable=True, fetch=True, rank=200),
"hdr": CustomRank(enable=True, fetch=True, rank=100),
}
)
As shown above with "/SenSiTivE/", you are able to set explicit case sensitivity as well for entering patterns for require
, exclude
and preferred
attributes. We default to ignore case sensitivity.
RankingModel
While SettingsModel
focuses on the selection and preference of torrents, RankingModel
(such as BaseRankingModel
or its extensions) is designed to compute the ranking scores based on those preferences. This model allows for the creation of a nuanced scoring system that evaluates each torrent's quality and attributes, translating user preferences into a quantifiable score.
Key functionalities:
- Scoring Torrent Attributes: Assign scores to various torrent attributes like resolution, audio quality, etc.
- Customizable Ranking Logic: Extend
BaseRankingModel
to tailor ranking criteria and values, enhancing the decision-making process in selecting torrents.
Example usage:
from RTN.models import BaseRankingModel
class MyRankingModel(BaseRankingModel):
uhd = 200 # Ultra HD content
hdr = 100 # HDR content
# Define more attributes and scores as needed
Why Both Models are Necessary
SettingsModel
and RankingModel
work together to provide a comprehensive approach to torrent ranking:
- SettingsModel specifies what to look for in torrents, defining the search and preference criteria.
- RankingModel quantifies those preferences, assigning scores to make informed decisions on which torrents are of higher quality and relevance.
This separation allows for flexible configuration and a powerful, customizable ranking system tailored to individual user preferences.
BaseRankingModel
Here is the default BaseRankingModel that RTN uses, and it's attributes.
class BaseRankingModel(BaseModel):
"""
A base class for ranking models used in the context of media quality and attributes.
The ranking values are used to determine the quality of a media item based on its attributes.
Attributes:
uhd (int): The ranking value for Ultra HD (4K) resolution.
fhd (int): The ranking value for Full HD (1080p) resolution.
hd (int): The ranking value for HD (720p) resolution.
sd (int): The ranking value for SD (480p) resolution.
bluray (int): The ranking value for Blu-ray quality.
hdr (int): The ranking value for HDR quality.
hdr10 (int): The ranking value for HDR10 quality.
dolby_video (int): The ranking value for Dolby video quality.
dts_x (int): The ranking value for DTS:X audio quality.
dts_hd (int): The ranking value for DTS-HD audio quality.
dts_hd_ma (int): The ranking value for DTS-HD Master Audio audio quality.
atmos (int): The ranking value for Dolby Atmos audio quality.
truehd (int): The ranking value for Dolby TrueHD audio quality.
ddplus (int): The ranking value for Dolby Digital Plus audio quality.
ac3 (int): The ranking value for AC3 audio quality.
aac (int): The ranking value for AAC audio quality.
remux (int): The ranking value for remux attribute.
webdl (int): The ranking value for web-dl attribute.
repack (int): The ranking value for repack attribute.
proper (int): The ranking value for proper attribute.
dubbed (int): The ranking value for dubbed attribute.
subbed (int): The ranking value for subbed attribute.
av1 (int): The ranking value for AV1 attribute.
"""
# resolution
uhd: int = 0
fhd: int = 0
hd: int = 0
sd: int = 0
# quality
bluray: int = 0
hdr: int = 0
hdr10: int = 0
dolby_video: int = 0
# audio
dts_x: int = 0
dts_hd: int = 0
dts_hd_ma: int = 0
atmos: int = 0
truehd: int = 0
ddplus: int = 0
ac3: int = 0
aac: int = 0
# other
remux: int = 0
webdl: int = 0
repack: int = 5
proper: int = 4
# extras
dubbed: int = 4
subbed: int = 2
av1: int = 0
Keep in mind that these are explicitly set within RTN and are needed in order for RTN to work. You can add new attributes, but it will be up to you to handle them.
Create as many SettingsModel
and RankingModel
as you like to use anywhere in your code. They are mean't to be used as a way to version settings for your users.
Extras
Torrent Parser
You can also parse a torrent title similar to how PTN works. This is an enhanced version of PTN that combines RTN's parsing as well. This also includes enhanced episode parsing as well that covers a much better range of titles.
Using the example above:
from RTN import parse
parsed = parse("Example.Movie.2020.1080p.BluRay.x264-Example")
print(parsed.data.raw_title) # Output: "Example.Movie.2020.1080p.BluRay.x264-Example"
print(parsed.data.parsed_title) # Output: "Example Movie"
print(parsed.data.year) # Output: [2020]
:warning: We also set coherent_types to
True
from the PTN data that get's combined with RTN parsed metadata. This just ensures that all the types are uniform. Everything is either a list of string or int's, or it's a boolean.
Checking Title Similarity
Sometimes, you might just want to check if two titles match closely enough, without going through the entire ranking process. RTN provides a simple function, title_match, for this purpose:
from RTN import title_match
# Check if two titles are similar above a threshold of 0.9
match = title_match("Correct Movie Title 2020", "Correct Movie Title (2020)")
print(match) # Output: True if similarity is above 0.9, otherwise False
>>> True
This functionality is especially useful when you have a list of potential titles and want to find the best match for a given reference title.
Trash Check
Maybe you just want to use our own garbage collector to weed out bad titles in your current scraping setup?
from RTN import check_trash
if check_trash(raw_title):
# You can safely remove any title or item from being scraped if this returns True!
...
Movie Check
Now you can check if a raw torrent title is a movie
or a show
type!
from RTN.parser import get_type, parse
parsed_data = parse("Joker.2019.PROPER.mHD.10Bits.1080p.BluRay.DD5.1.x265-TMd", remove_trash = False)
print(parsed_data.type)
>>> "movie"
Alternatively, if you prefer a boolean, you can use is_movie
instead, like so:
from RTN.parser import is_movie, parse
parsed_data = parse("Joker.2019.PROPER.mHD.10Bits.1080p.BluRay.DD5.1.x265-TMd", remove_trash = False)
print(is_movie(parsed_data))
>>> True
Real World Example
Here is a crude example of how you could use RTN in scraping.
from RTN import RTN, Torrent, DefaultRanking
from RTN.exceptions import GarbageTorrent
# Assuming 'settings' is defined somewhere and passed correctly.
rtn = RTN(settings=settings, ranking_model=DefaultRanking())
...
# Define some function for scraping for results from some API.
if response.ok:
torrents = set()
for stream in response.streams:
if not stream.infohash or not title_match(correct_title, stream.title):
# Skip results that don't match the query.
# We want to do this first to weed out torrents
# that are below the 90% match criteria. (Default is 90%)
continue
try:
torrent: Torrent = rtn.rank(stream.title, stream.infohash)
except GarbageTorrent:
# One thing to note is that as we parse titles, we also get rid of garbage.
# Feel free to add your own logic when this happens!
# You can bypass this by setting `remove_trash` to `False` in `rank` or `parse`.
pass
if torrent and torrent.fetch:
# If torrent.fetch is True, then it's a good torrent,
# as considered by your ranking profile and settings model.
torrents.add(torrent)
# Sort the list of torrents based on their rank in descending order
sorted_torrents = sorted(list(torrents), key=lambda x: x.rank, reverse=True)
return sorted_torrents
...
# Example usage
for torrent in sorted_torrents:
print(f"Title: {torrent.data.parsed_title}, Infohash: {torrent.infohash}, Rank: {torrent.rank}")
ParsedData Structure
Here is all of the attributes of data
from the Torrent
object, along with their default values.
This is accessible at torrent.data
in the Torrent
object. Ex: torrent.data.resolution
class ParsedData(BaseModel):
"""Parsed data model for a torrent title."""
raw_title: str
parsed_title: str
fetch: bool = False
is_4k: bool = False
is_multi_audio: bool = False
is_multi_subtitle: bool = False
is_complete: bool = False
year: int = 0
resolution: List[str] = []
quality: List[str] = []
season: List[int] = []
episode: List[int] = []
codec: List[str] = []
audio: List[str] = []
subtitles: List[str] = []
language: List[str] = []
bitDepth: List[int] = []
hdr: str = ""
proper: bool = False
repack: bool = False
remux: bool = False
upscaled: bool = False
remastered: bool = False
directorsCut: bool = False
extended: bool = False
This will continue to grow though as we expand on functionality, so keep checking back for this list!
:warning: Don't see something you want in the list? Submit a Feature Request to have it added!
Performance Benchmarks
Here, we dive into the heart of RTN's efficiency, showcasing how it performs under various loads. Whether you're parsing a single title or ranking thousands, understanding these benchmarks will help you optimize your use of RTN.
Benchmark Categories
We categorize benchmarks into two main processes:
- Parsing: Measures the time to parse a title and return a
ParsedData
object. - Ranking: A comprehensive process that includes parsing and then evaluates the title based on defined criteria. This represents a more "real-world" scenario and is crucial for developers looking to integrate RTN effectively.
Benchmark Results
To facilitate comparison, we've compiled the results into a single table:
Operation | Items Count | Mean Time | Standard Deviation |
---|---|---|---|
Parsing Benchmark (Single item) | 1 | 583 us | 10 us |
Batch Parse Benchmark (Small batch) | 10 | 6.24 ms | 0.16 ms |
Batch Parse Benchmark (Large batch) | 1000 | 1.57 s | 0.06 s |
Batch Parse Benchmark (XLarge batch) | 2000 | 3.62 s | 0.11 s |
Ranking Benchmark (Single item) | 1 | 616 us | 11 us |
Batch Rank Benchmark (Small batch) | 10 | 24.6 ms | 2.4 ms |
Batch Rank Benchmark (Large batch) | 1000 | 3.13 s | 0.15 s |
Batch Rank Benchmark (XLarge batch) | 2000 | 6.27 s | 0.13 s |
:warning: Test Bench consisted of R9 5900X CPU and 64GB DDR4 RAM - Your mileage may vary.
To run your own benchmark, you can clone the repo and run make benchmark
from inside the root of the repository.
Benchmark Settings
- Small batch parsing used a
chunk_size
of10
. - Large batch parsing handled
chunk_size
of200
. - XLarge batch parsing handled
chunk_size
of500
. - Small batch ranking operated with the default
max_workers
of4
and used achunk_size
of10
. - Large batch ranking escalated concurrency with
max_workers
of8
and handled achunk_size
of200
. - XLarge batch ranking escalated concurrency with
max_workers
of16
and handled achunk_size
of500
.
This data shows RTN's robust capability to efficiently process both small and extensive datasets.
Optimizing RTN Performance
The performance benchmarks provided give a glimpse into how RTN handles different loads, from parsing single titles to ranking thousands. For developers looking to integrate RTN into their applications efficiently, here are some tips on tweaking performance:
1. Adjusting Chunk Size for Batch Parsing
The batch_parse
function allows you to parse titles in batches, significantly reducing processing time for large datasets. However, the optimal chunk_size
can vary depending on the dataset size and your system's resources.
- For smaller datasets, a lower
chunk_size
might suffice, keeping overhead low. - For larger datasets, increasing
chunk_size
can reduce the number of batches processed and potentially lower overall processing time.
Experiment with different chunk_size
values to find the sweet spot for your particular use case.
2. Tuning Concurrency in Batch Ranking
The batch_rank
function uses multiple threads to rank torrents in parallel, which can significantly speed up processing for large numbers of torrents.
- The default
max_workers
value is set to4
, but this might not be optimal for all systems. - Systems with higher CPU core counts might benefit from increasing
max_workers
, allowing more torrents to be processed simultaneously. - However, setting
max_workers
too high can lead to diminishing returns and increased overhead. Monitor your system's resource utilization to find an optimal setting.
3. Leveraging ThreadPoolExecutor
Both batch_parse
and batch_rank
utilize ThreadPoolExecutor
for parallel processing. Adjusting the max_workers
parameter can help manage how many threads are used for these operations, impacting performance and resource utilization.
4. Custom Settings and Ranking Models
Customizing SettingsModel
and RankingModel
allows you to tailor the parsing and ranking criteria to your needs, potentially streamlining the processing by focusing only on relevant data.
- Evaluate which torrent attributes are essential for your application and adjust your settings model accordingly.
- Consider disabling unnecessary custom ranks or attributes in the ranking model to simplify the ranking process.
Example: Tweaking Performance for Large Datasets
Suppose you're processing a dataset of 10,000 torrent titles. You might start with a default chunk_size
of 50
and max_workers
of 4
. Through experimentation, you find that increasing chunk_size
to 500
and max_workers
to 8
cuts your processing time in half.
from RTN import RTN, SettingsModel, DefaultRanking, batch_parse
# Setup
settings = SettingsModel()
ranking_model = DefaultRanking()
rtn = RTN(settings=settings, ranking_model=ranking_model)
# Optimized batch parsing
optimized_titles = ["Title 1", "Title 2", ..., "Title 10000"]
parsed_data = batch_parse(optimized_titles, chunk_size=500, max_workers=8)
By monitoring performance and adjusting parameters based on your specific requirements and system capabilities, you can significantly enhance RTN's efficiency in your projects.
Contributing
Contributions to RTN are welcomed! Feel free to submit pull requests or open issues to suggest features or report bugs. As we grow, more features will be coming to RTN, there's already a lot planned!
License
RTN is released under the MIT License. See the LICENSE file for more details.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for rank_torrent_name-1.0.4-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 8db5fbd53a08206fe92afb86912b81848e39588c1e348f25ee56276317d5ffac |
|
MD5 | 054968fae3f655e806a536a0735d76c6 |
|
BLAKE2b-256 | 03d0705df9d494d3a7c003971a25abb0c7e47febd093c081a7ee1d8462076ce2 |