A modular toolkit for analytics.
Project description
NewsLynx Core is an expandable open-source toolkit for building modular content analytics workflows. It provides a fully RESTful API as well as a comprehensive python client and command line interface.
NewsLynx Core was built to power `newslynx-app <http://github.com/newslynx/newslynx-app>`__ but is capable of powering a diverse range of potential applications, as well, including:
A Mention.net-like pipeline for your personal or company blog.
A Flexible timeseries store for content metrics which will automatically summarize and compare your data, as well as enable the additional of custom, computed metrics.
A framework for configuring, scheduling, and monitoring arbitrary python jobs via API.
A content-extraction API.
Installation
For most applications, refer to our installation guide. If you’d like to setup a development environment, following the instructions below for MacOS X. If you’d like to spin up a Virtual Machine, check out our automation guide.
Dependencies
Postgres
NOTE We recommend using Postgres APP. However, if you prefer the brew distribution, make sure to install it with plpythonu.
$ brew install postgresql --build-from-source --with-python
(Re)create a postgresql database
# If you already have a database called `newslynx`, delete it first
$ dropdb newslynx
$ createdb newslynx
Redis
Install redis:
$ brew install redis
Open another tab in your shell and run:
$ redis-server
Installation / Initialization
NOTE we recommend that you install newslynx in a virtual environment.
First clone this repository and move into it’s root directory
$ git clone https://github.com/newslynx/newslynx-core.git
$ cd newslynx-core
OPTIONAL Set your configuration. If you don’t do this, we will fallback on the app’s default configuration file.
Initialize a bare install.
[Expert mode] don’t install the app’s default sous chefs, tags, or recipes.
$ make bare_install
Start the server
In debug mode: newslynx debug
Debug mode with errors: newslynx debug --raise-errors
Production gunicorn server: bin/run
Start the task workers
$ bin/start_workers
Stop the task workers
$ bin/stop_workers
Start the cron daemon
$ newslynx cron
Testing
newslynx is a complicated applcation, to simplify testing…
Makefile
This repository contains a Makefile which provides access to the following commands:
make clean
clean out cruft generated by pip.
make clean_sc
clean out cruft generated by pip in ~/.newslynx/sous-chefs
make install
installs newslynx
will first uninstall newslynx if a version already exists.
make app_install
Installs newslynx, initializes the Super User, and adds the Recipes, Tags, and Sous Chefs needed to run newslynx-app
make bare_install
Installs newslynx and initializes the Super User
make test_install
removes all files under ~/.newslynx
runs make app_install
runs pip install -r test-requirements.txt
distribute
creates a new version of newslynx on PyPI
Docs
Application Structure
License
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for newslynx-1.2.2.macosx-10.10-intel.exe
Algorithm | Hash digest | |
---|---|---|
SHA256 | 24976b719a1c3746f0a2e2cc7936b8fccc63f2592e2dfb57fc59ff30f1a07e16 |
|
MD5 | afa79dd9edca8289bd42149760608091 |
|
BLAKE2b-256 | 33a5a9b3e6852be9f942db5a5368c98e89ab399b7341ab8cdef4cba1737fdc5e |