Skip to main content

A FastAPI web server for creating RSS and ActivityPub feeds for scholarly literature with the magic of adversarial interoperability

Project description

paper-feeds

PyPI - Version Coverage Status

A FastAPI web server for creating RSS feeds for scholarly journals with the magic of adversarial interoperability

Many journals still have RSS feeds. Some don't though, as they try and squeeze everyone onto their platforms to monetize our engagement data.

This is a simple web app for creating feeds (currently RSS, soon ActivityPub and Atom) for academic papers by collecting metadata from multiple data sources. It intended to be a publicly- and self-hostable toolkit for subscribing to and curating scholarly literature!

Dependencies are kept minimal, as is deployment - No webpack, no complex build, no postgres, just pip install and press play :).

usage

(to be completed when main docs are, for now here's something brief)

After creating and activating a virtual environment...

pip install paper-feeds
python -m paper_feeds
# then open http://localhost:8000 in your browser

Note: we are still working out the packaging here, so you may need to clone the repository and run the server from the repo root until we can figure that out :)

And see CONTRIBUTING.md for more information on setting up a development environment

progress

Everything is just getting started! things will break and change! To be moved to docs when made. Help wanted on all, open an issue <3

API:

  • Query Crossref for journal
  • Write journal metadata to db
  • Paginate papers by journal
  • Store papers in db
  • Populate papers when feed created
  • Periodic database updates
  • Cache Feed output
  • Scheduled update of feed metadata
  • Backfill Abstracts and other additional data
  • Feed statistics

Frontend

  • Search for journal
  • Display list of journals
  • Pages for each journal
  • Create new feed button
  • Copy feed link
  • Export feeds
  • Show existing feeds, stats, threads

Feed Types

  • Journals
  • Authors (via ORCID)
  • Keywords

Feed Formats

  • RSS
    • RSS feed from papers by issn
    • Linked Data-enriched RSS feeds (see crossref's advice)
    • HTML formatting for item details
  • Activitypub
    • Actors for feeds
    • LD-enriched ActivityStreams actions
    • Bot-Actor for instance
    • DOI mention detection & crossref events data
    • Hashtag -> keyword detection
    • Create threads under feed actor with mention

Data Sources

  • Crossref
    • Journals
    • Papers
    • Events
  • OpenAlex
  • ORCID
  • PubPeer
  • RetractionWatch
  • Hypothes.is

Meta

  • Docs
    • We need em! Sphinx & RTD!
    • Move this list to there
    • Scope
    • Design
    • Usage
    • Configuration
  • Tests
    • Basic CI

Credits

  • El Duvelle, whose need for RSS feeds inspired this project
  • @lambdaloop (list PRs)
  • @roaldarbol (list PRs)

References

See also

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

paper_feeds-0.0.5.tar.gz (52.8 kB view details)

Uploaded Source

Built Distribution

paper_feeds-0.0.5-py3-none-any.whl (63.9 kB view details)

Uploaded Python 3

File details

Details for the file paper_feeds-0.0.5.tar.gz.

File metadata

  • Download URL: paper_feeds-0.0.5.tar.gz
  • Upload date:
  • Size: 52.8 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/1.6.1 CPython/3.11.3 Darwin/23.3.0

File hashes

Hashes for paper_feeds-0.0.5.tar.gz
Algorithm Hash digest
SHA256 80c51d7fff7970cd8471a72199041cbf97b66a32d37f72c762007a2ca20d7d32
MD5 d8436b97a08f735f19596d4526dc08c2
BLAKE2b-256 b5f7267f7338076e56586784028ec830b5c71c1cac192d82e46a0e23a19e3c99

See more details on using hashes here.

File details

Details for the file paper_feeds-0.0.5-py3-none-any.whl.

File metadata

  • Download URL: paper_feeds-0.0.5-py3-none-any.whl
  • Upload date:
  • Size: 63.9 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/1.6.1 CPython/3.11.3 Darwin/23.3.0

File hashes

Hashes for paper_feeds-0.0.5-py3-none-any.whl
Algorithm Hash digest
SHA256 72c0c26439422bbd8b91b46ca8c103aceb1fcb76fca1a6f928119729e1bd00fc
MD5 ac4d3ea5d6f7900ad4ae9b0cc6feb8a9
BLAKE2b-256 c6aef67ba3dbc1c76ca21bb03e04224917095f93d25df38caf609e6c4a86866c

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page