A modular toolkit for analytics.
Project description
NewsLynx Core is an expandable open-source toolkit for building modular content analytics workflows. It provides a fully RESTful API as well as a comprehensive python client and command line interface.
NewsLynx Core was built to power `newslynx-app <http://github.com/newslynx/newslynx-app>`__ but is capable of powering a diverse range of potential applications, as well, including:
A Mention.net-like pipeline for your personal or company blog.
A Flexible timeseries store for content metrics which will automatically summarize and compare your data, as well as enable the additional of custom, computed metrics.
A framework for configuring, scheduling, and monitoring arbitrary python jobs via API.
A content-extraction API.
Installation
For most applications, refer to our installation guide. If you’d like to setup a development environment, following the instructions below for MacOS X. If you’d like to spin up a Virtual Machine, check out our automation guide.
Dependencies
Postgres
NOTE We recommend using Postgres APP. However, if you prefer the brew distribution, make sure to install it with plpythonu.
$ brew install postgresql --build-from-source --with-python
(Re)create a postgresql database
# If you already have a database called `newslynx`, delete it first
$ dropdb newslynx
$ createdb newslynx
Redis
Install redis:
$ brew install redis
Open another tab in your shell and run:
$ redis-server
Installation / Initialization
NOTE we recommend that you install newslynx in a virtual environment.
First clone this repository and move into it’s root directory
$ git clone https://github.com/newslynx/newslynx-core.git
$ cd newslynx-core
OPTIONAL Set your configuration. If you don’t do this, we will fallback on the app’s default configuration file.
Initialize a bare install.
[Expert mode] don’t install the app’s default sous chefs, tags, or recipes.
$ make bare_install
Start the server
In debug mode: newslynx debug
Debug mode with errors: newslynx debug --raise-errors
Production gunicorn server: bin/run
Start the task workers
$ bin/start_workers
Stop the task workers
$ bin/stop_workers
Start the cron daemon
$ newslynx cron
Testing
newslynx is a complicated applcation, to simplify testing…
Makefile
This repository contains a Makefile which provides access to the following commands:
make clean
clean out cruft generated by pip.
make clean_sc
clean out cruft generated by pip in ~/.newslynx/sous-chefs
make install
installs newslynx
will first uninstall newslynx if a version already exists.
make app_install
Installs newslynx, initializes the Super User, and adds the Recipes, Tags, and Sous Chefs needed to run newslynx-app
make bare_install
Installs newslynx and initializes the Super User
make test_install
removes all files under ~/.newslynx
runs make app_install
runs pip install -r test-requirements.txt
distribute
creates a new version of newslynx on PyPI
Docs
Application Structure
License
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file newslynx-1.2.2.tar.gz
.
File metadata
- Download URL: newslynx-1.2.2.tar.gz
- Upload date:
- Size: 228.2 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9ea46dc5637b3bd11370a679569796cbf8b1642463d694058b51e3c434832365 |
|
MD5 | 83f94fb45a76b8b93351a3b6ff19a358 |
|
BLAKE2b-256 | db34237db4ef03edd879126a01f476e5eb002f02f7735d7602bdf0951b891e7d |
File details
Details for the file newslynx-1.2.2.macosx-10.10-intel.exe
.
File metadata
- Download URL: newslynx-1.2.2.macosx-10.10-intel.exe
- Upload date:
- Size: 420.2 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 24976b719a1c3746f0a2e2cc7936b8fccc63f2592e2dfb57fc59ff30f1a07e16 |
|
MD5 | afa79dd9edca8289bd42149760608091 |
|
BLAKE2b-256 | 33a5a9b3e6852be9f942db5a5368c98e89ab399b7341ab8cdef4cba1737fdc5e |