Skip to main content

This module defines a Flask App that serves URLs based on the contents of a diskcache.

Requirements

  • Python >= 3.4

  • Flask >= 1.0.0

Apache + Gunicorn configuration

This service runs under Gunicorn behind an Apache proxy, which is used to verify X.509 certificates. The certificate information is then passed to the application in the header where the subject is authenticated against a grid-mapfile.

Add the following to the Apache system config file, e.g. /etc/sysconfig/httpd on CentOS 7

OPENSSL_ALLOW_PROXY_CERTS=1

Edit the appropriate 443 virtual host configuration to add

# Using Apache as a proxy to gunicorn
  ProxyPass "/LDR" "http://127.0.0.1:8080/" retry=0
  ProxyPassReverse "/LDR" "http://127.0.0.1:8080/"

  <Location "/LDR/">
       SSLRequireSSL
       SSLVerifyClient optional
       SSLVerifyDepth 10
       SSLOptions +ExportCertData +StrictRequire +LegacyDNStringFormat
       SSLOptions +StdEnvVars
       RequestHeader set SSL_CLIENT_S_DN "%{SSL_CLIENT_S_DN}s"
       RequestHeader set SSL_CLIENT_I_DN "%{SSL_CLIENT_I_DN}s"
  </Location>

Start a gunicron instance with

/usr/bin/gunicorn-3.6 --bind=127.0.0.1:8080 --workers=5 "gwdatafind_server:create_app()"

Sample configuration file can be put into /etc/gwdatafind-server.ini

[LDRDataFindServer]
# frame cache file
framecachefile = <path_to>/ascii_cache.dat
framecachetimeout = 10
# grid-mapfile
gridmapcachefile = <path_to>/grid-mapfile
gridmapcachetimeout = 60
# optional parameters
site_exclude_pattern = ^X$
frametype_exclude_pattern = .+EXCLUDE.+
frametype_include_pattern = .+_TEST_\d+
filter_preference = """{'^file': ['preferred', 'hdfs']}"""

Client API

The GWDataFind Server runs as a daemon providing a RESTful interface via Gunicorn + Apache. There is also a client application gw_data_find for command line usage.

The URLs supported areliwted below. They all support HTTP GET,HEAD, and OPTIONS methods.

URL (add HTTP[S]://<host>[:<port>]/LDR to beginning

Function

/services/data/v1/

Show URLs (debugging)

/services/data/v1/<ext>.json

Show observatories

/services/data/v1/<ext>/<site>.json

Show tags (frame types)

/services/data/v1/<ext>/<site>/<tag>/latest.json

Show latest

/services/data/v1/<ext>/<site>/<tag>/<start>,<end>.json

Show URLs

/services/data/v1/<ext>/<site>/<tag>/<start>,<end>/<urltype>.json

Show URLs of one type (file,URL)

/services/data/v1/<ext>/<site>/<tag>/segments.json

Show all available segments

/services/data/v1/<ext>/<site>/<tag>/segments/<start>,<end>.json

Show avalable segments in a time interval

/services/data/v1/<ext>/<site>/<tag>/latest.json

Show latest URL

/services/data/v1/<ext>/<site>/<tag>/<filename>.json

Show a single URL

Metadata

Release files for gwdatafind-server 1.1.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for gwdatafind-server 1.1.0
File Size Uploaded
gwdatafind-server-1.1.0.tar.gz 31.1 kB Details

Built distribution (wheel)

Table of built distributions (wheels) for gwdatafind-server 1.1.0
File Interpreter ABI Platform
gwdatafind_server-1.1.0-py3-none-any.whl Python 3 none any Details

Total release size: 58.2 kB

Release files / gwdatafind-server-1.1.0.tar.gz

Download URL gwdatafind-server-1.1.0.tar.gz
Size 31.1 kB
Tags Source
SHA-256 checksum
How to use checksums
1062dc551de95ee7509c18385d9c819517207e515ec203c8fd424d77faffc6aa
BLAKE2b-256 checksum
How to use checksums
41629f0521de6bbc31b545b1cd2a67b11569255768d54c9d3f73e7cec21a96a1
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/3.8.0 pkginfo/1.8.2 readme-renderer/33.0 requests/2.27.1 requests-toolbelt/0.9.1 urllib3/1.26.8 tqdm/4.63.0 importlib-metadata/4.11.2 keyring/23.5.0 rfc3986/2.0.0 colorama/0.4.4 CPython/3.8.10

Release files / gwdatafind_server-1.1.0-py3-none-any.whl

Download URL gwdatafind_server-1.1.0-py3-none-any.whl
Size 27.0 kB
Tags Python 3
SHA-256 checksum
How to use checksums
1997a2e0cf9e4f999143867dbe1bde3d13e6a2ed21230321018622db1deb2c96
BLAKE2b-256 checksum
How to use checksums
32b64a051e175283357c78447aee681fe4f0d5c1f95ee428bcbd22826a7fc906
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/3.8.0 pkginfo/1.8.2 readme-renderer/33.0 requests/2.27.1 requests-toolbelt/0.9.1 urllib3/1.26.8 tqdm/4.63.0 importlib-metadata/4.11.2 keyring/23.5.0 rfc3986/2.0.0 colorama/0.4.4 CPython/3.8.10

Release history Release notifications | RSS feed

2.4.0

2 release files

2.3.1

2 release files

2.3.0

2 release files

2.2.0

2 release files

2.1.0

2 release files

1.5.0

2 release files

1.4.0

2 release files

1.3.0

2 release files

1.2.2

2 release files

1.2.1

2 release files

1.2.0

2 release files

This release

1.1.0 This release

2 release files

1.0.1

2 release files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page