ml-research-trends 0.1.1

A Python toolkit for collecting academic papers from Semantic Scholar and analyzing/visualizing publication trends in machine learning research

ML Research Trends

A Python toolkit for collecting academic papers from Semantic Scholar and analyzing/visualizing publication trends in machine learning research.

Features

• Data Collection: Collect papers from Semantic Scholar API based on keywords and date ranges
• Trend Analysis: Analyze publication trends, citation patterns, and temporal distributions
• Visualization: Create publication trend plots, embedding visualizations, and landmark timelines
• Embedding Analysis: Generate and visualize paper abstract embeddings using transformer models
• Flexible API: Easy-to-use functions for data collection, analysis, and visualization

Installation

pip install ml-research-trends

Or with uv:

uv add ml-research-trends

Requirements

• Python >= 3.11
• Semantic Scholar API key (set as environment variable SEMANTIC_SCHOLAR_API_KEY)

Quick Start

from ml_research_trends import (
    collect_topic_data,
    summarize_topic_data,
    analyze_topic_trends,
    plot_topic_counts_by_year,
)

# Collect papers on a topic
df = collect_topic_data(
    keywords=["retrieval augmented generation", "RAG"],
    max_results_per_keyword=100,
    min_year=2020,
)

# Summarize the data
summary = summarize_topic_data(df)
print(f"Total papers: {summary['total_papers']}")

# Analyze trends
trends = analyze_topic_trends(df)

# Visualize publication trends
plot_topic_counts_by_year(trends, topic_name="RAG")

API Reference

Data Collection & Analysis

• collect_topic_data() - Collect papers from Semantic Scholar
• summarize_topic_data() - Generate summary statistics
• analyze_topic_trends() - Analyze per-year trends

Visualization

• plot_topic_counts_by_year() - Plot publication counts over time
• embed_and_plot_abstracts() - Create embedding visualizations
• plot_landmark_timeline() - Plot timelines with landmark events

Examples

The package includes several example topics:

• RAG (Retrieval-Augmented Generation): Track the evolution of RAG research
• Chain-of-Thought Reasoning: Analyze reasoning capabilities in LLMs
• Multimodal Vision-Language Models: Study multimodal learning trends
• Transformer Architectures: Examine transformer model developments

Configuration

Set your Semantic Scholar API key:

export SEMANTIC_SCHOLAR_API_KEY="your_api_key_here"

Or create a .env file:

SEMANTIC_SCHOLAR_API_KEY=your_api_key_here

Dependencies

• pandas: Data manipulation
• requests: API calls
• matplotlib/seaborn: Plotting
• sentence-transformers: Text embeddings
• torch: Machine learning
• scikit-learn: Dimensionality reduction
• umap-learn: Embedding visualization
• plotly: Interactive plots

License

MIT License