Skip to main content

A modular toolkit for analytics.

Project description

travis-img # newslynx-core

NewsLynx Core is an expandable open-source toolkit for building modular content analytics workflows. It provides a fully RESTful API as well as a comprehensive python client and command line interface.

NewsLynx Core was built to power `newslynx-app <http://github.com/newslynx/newslynx-app>`__ but is capable of powering a diverse range of potential applications, as well, including:

  • A Mention.net-like pipeline for your personal or company blog.

  • A Flexible timeseries store for content metrics which will automatically summarize and compare your data, as well as enable the additional of custom, computed metrics.

  • A framework for configuring, scheduling, and monitoring arbitrary python jobs via API.

  • A content-extraction API.

Installation

For most applications, refer to our installation guide. If you’d like to setup a development environment, following the instructions below for MacOS X. If you’d like to spin up a Virtual Machine, check out our automation guide.

Dependencies

Postgres

NOTE We recommend using Postgres APP. However, if you prefer the brew distribution, make sure to install it with plpythonu.

$ brew install postgresql --build-from-source --with-python

(Re)create a postgresql database

# If you already have a database called `newslynx`, delete it first
$ dropdb newslynx
$ createdb newslynx

Redis

Install redis:

$ brew install redis

Open another tab in your shell and run:

$ redis-server

Installation / Initialization

NOTE we recommend that you install newslynx in a virtual environment.

First clone this repository and move into it’s root directory

$ git clone https://github.com/newslynx/newslynx-core.git
$ cd newslynx-core

OPTIONAL Set your configuration. If you don’t do this, we will fallback on the app’s default configuration file.

Initialize the database, super user, and install default sous chefs, tags, and recipes.

Run this command when working with newslynx-app

$ make app_install

Initialize a bare install.

[Expert mode] don’t install the app’s default sous chefs, tags, or recipes.

$ make bare_install

Start the server

  • In debug mode: newslynx debug

  • Debug mode with errors: newslynx debug --raise-errors

  • Production gunicorn server: bin/run

Start the task workers

$ bin/start_workers

Stop the task workers

$ bin/stop_workers

Start the cron daemon

$ newslynx cron

Testing

newslynx is a complicated applcation, to simplify testing…

Makefile

This repository contains a Makefile which provides access to the following commands:

  • make clean

    • clean out cruft generated by pip.

  • make clean_sc

    • clean out cruft generated by pip in ~/.newslynx/sous-chefs

  • make install

    • installs newslynx

    • will first uninstall newslynx if a version already exists.

  • make app_install

    • Installs newslynx, initializes the Super User, and adds the Recipes, Tags, and Sous Chefs needed to run newslynx-app

  • make bare_install

    • Installs newslynx and initializes the Super User

  • make test_install

    • removes all files under ~/.newslynx

    • runs make app_install

    • runs pip install -r test-requirements.txt

  • distribute

    • creates a new version of newslynx on PyPI

Docs

Application Structure

License

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

newslynx-1.2.2.tar.gz (228.2 kB view details)

Uploaded Source

Built Distribution

newslynx-1.2.2.macosx-10.10-intel.exe (420.2 kB view details)

Uploaded Source

File details

Details for the file newslynx-1.2.2.tar.gz.

File metadata

  • Download URL: newslynx-1.2.2.tar.gz
  • Upload date:
  • Size: 228.2 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No

File hashes

Hashes for newslynx-1.2.2.tar.gz
Algorithm Hash digest
SHA256 9ea46dc5637b3bd11370a679569796cbf8b1642463d694058b51e3c434832365
MD5 83f94fb45a76b8b93351a3b6ff19a358
BLAKE2b-256 db34237db4ef03edd879126a01f476e5eb002f02f7735d7602bdf0951b891e7d

See more details on using hashes here.

File details

Details for the file newslynx-1.2.2.macosx-10.10-intel.exe.

File metadata

File hashes

Hashes for newslynx-1.2.2.macosx-10.10-intel.exe
Algorithm Hash digest
SHA256 24976b719a1c3746f0a2e2cc7936b8fccc63f2592e2dfb57fc59ff30f1a07e16
MD5 afa79dd9edca8289bd42149760608091
BLAKE2b-256 33a5a9b3e6852be9f942db5a5368c98e89ab399b7341ab8cdef4cba1737fdc5e

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page