Skip to main content

A flask blueprint providing an API for accessing and searching an ElasticSearch index created from source datapackages

Project description

apies

Travis Coveralls PyPI - Python Version

apies is a flask blueprint providing an API for accessing and searching an ElasticSearch index created from source datapackages.

endpoints

/get

/search/count

/search/<doctypes>

configuration

Flask configuration for this blueprint:

    from apies import apies_blueprint
    import elasticsearch

    app.register_blueprint(
        apies_blueprint(['path/to/datapackage.json', Package(), ...],
                        elasticsearch.Elasticsearch(...), 
                        {'doc-type-1': 'index-for-doc-type-1', ...}, 
                        'index-for-documents',
                        dont_highlight=['fields', 'not.to', 'highlight'],
                        text_field_rules=lambda schema_field: [], # list of tuples: ('exact'/'inexact'/'natural', <field-name>)
                        multi_match_type='most_fields',
                        multi_match_operator='and'),
        url_prefix='/search/'
    )

local development

You can start a local development server by following these steps:

  1. Install Dependencies:

    a. Install Docker locally

    b. Install Python dependencies:

    $ pip install dataflows dataflows-elasticsearch
    $ pip install -e .
    
  2. Go to the sample/ directory

  3. Start ElasticSearch locally:

    $ ./start_elasticsearch.sh
    

    This script will wait and poll the server until it's up and running. You can test it yourself by running:

    $ curl -s http://localhost:9200
         {
         "name" : "99cd2db44924",
         "cluster_name" : "docker-cluster",
         "cluster_uuid" : "nF9fuwRyRYSzyQrcH9RCnA",
         "version" : {
             "number" : "7.4.2",
             "build_flavor" : "default",
             "build_type" : "docker",
             "build_hash" : "2f90bbf7b93631e52bafb59b3b049cb44ec25e96",
             "build_date" : "2019-10-28T20:40:44.881551Z",
             "build_snapshot" : false,
             "lucene_version" : "8.2.0",
             "minimum_wire_compatibility_version" : "6.8.0",
             "minimum_index_compatibility_version" : "6.0.0-beta1"
         },
         "tagline" : "You Know, for Search"
         }
    
  4. Load data into the database

    $ DATAFLOWS_ELASTICSEARCH=localhost:9200 python load_fixtures.py
    

    You can test that data was loaded:

    $ curl -s http://localhost:9200/jobs-job/_count?pretty
         {
         "count" : 1757,
         "_shards" : {
             "total" : 1,
             "successful" : 1,
             "skipped" : 0,
             "failed" : 0
         }
         }
    
  5. Start the sample server

    $ python server.py 
     * Serving Flask app "server" (lazy loading)
     * Environment: production
     WARNING: Do not use the development server in a production environment.
     Use a production WSGI server instead.
     * Debug mode: off
     * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)
    
  6. Now you can hit the server's endpoints, for example:

         $ curl -s 'localhost:5000/api/search/jobs?q=engineering&size=2' | jq
         127.0.0.1 - - [26/Jun/2019 10:45:31] "GET /api/search/jobs?q=engineering&size=2 HTTP/1.1" 200 -
         {
             "search_counts": {
                 "_current": {
                 "total_overall": 617
                 }
             },
             "search_results": [
                 {
                 "score": 18.812,
                 "source": {
                     "# Of Positions": "5",
                     "Additional Information": "TO BE APPOINTED TO ANY CIVIL <em>ENGINEERING</em> POSITION IN BRIDGES, CANDIDATES MUST POSSESS ONE YEAR OF CIVIL <em>ENGINEERING</em> EXPERIENCE IN BRIDGE DESIGN, BRIDGE CONSTRUCTION, BRIDGE MAINTENANCE OR BRIDGE INSPECTION.",
                     "Agency": "DEPARTMENT OF TRANSPORTATION",
                     "Business Title": "Civil Engineer 2",
                     "Civil Service Title": "CIVIL ENGINEER",
                     "Division/Work Unit": "<em>Engineering</em> Review & Support",
             ...
         }
    

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

apies-1.0.0.tar.gz (11.8 kB view hashes)

Uploaded Source

Built Distribution

apies-1.0.0-py2.py3-none-any.whl (10.9 kB view hashes)

Uploaded Python 2 Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page