pubmed-scraper-rashi 0.1.3

Fetch PubMed papers with non-academic authors

PubMed Scraper

A command-line tool to fetch research papers from PubMed based on a user-specified query. The tool identifies papers with at least one author affiliated with a pharmaceutical or biotech company and outputs the results as a CSV.

Features

• Fetches papers using the PubMed E-utilities API
• Filters authors affiliated with biotech/pharmaceutical companies
• Saves results as CSV or prints to console
• Supports full PubMed query syntax
• Easy CLI usage
• Installable as a Python package

---

Installation (from TestPyPI)

pip install --index-url https://test.pypi.org/simple/ pubmed-scraper

---

From Source

1. Clone the repository:

   git clone https://github.com/Rashijain2709/pubmed_scraper.git
   cd pubmed_scraper
   

Tools Used

Tool	Purpose
[Poetry]	Dependency management and packaging
[Requests]	Making HTTP requests to the PubMed API
[argparse]	CLI argument parsing
[Python Typing]	Type hints and type safety
[pytest]	Unit testing

---

Project Structure

pubmed_scraper/
│
├── cli/                      # CLI entry point logic
│   └── main.py               # Contains CLI commands (argparse/typer/click)
│
├── src/
│   └── pubmed_scraper/       # Installable Python package
│       ├── __init__.py
│       ├── api.py            # Handles PubMed API requests
│       ├── parser.py         # Parses XML or JSON responses
│       ├── writer.py         # Writes results to CSV
│
├── tests/                    # Unit tests
│   └── test_parser.py
│
├── pyproject.toml            # Poetry project configuration
├── README.md                 # Project documentation
└── .gitignore                # (Recommended) Git ignore patterns

---

Poetry Project Execution Guide

This guide helps you set up, test, and publish your PubMed scraper project using Poetry and TestPyPI.

---

1. Initial Setup

Install required dependencies and dev tools:

poetry add requests

poetry add pytest --dev

poetry install

---

2. Check and Lock Dependencies

poetry check

poetry lock

---

3. Run Tests

poetry run pytest

---

4. Configure TestPyPI

poetry config repositories.test-pypi https://test.pypi.org/legacy/

# Verify the configuration (Windows)
poetry config --list | findstr pypi

# On macOS/Linux
poetry config --list | grep pypi

---

5. Build the Package

# Optional: Clean previous builds
rm -r dist/*   # Use 'rmdir /S /Q dist' on Windows

# Build the package
poetry build

---

6. Publish to TestPyPI

# Publish the package to TestPyPI
poetry publish --build -r test-pypi

---

7. Run Your Program

# Run the scraper with a query and output file
poetry run get-papers-list "cancer immunotherapy" --file papers.csv --debug

# Another example
poetry run get-papers-list "cancer therapy" --file results.csv --debug

---

Optional: Clean and Rebuild Everything

rm -r dist/*
poetry build
poetry publish --build -r test-pypi

---