Skip to main content

Slycat web client utilties for interacting with the Slycat data analysis and visualization server.

Project description

Slycat Web Client

The Slycat web client provides a Python package for interacting with the Slycat web server.

Slycat is a web based data analysis and visualization platform created at Sandia National Labs. You can read about it at https://slycat.readthedocs.io/en/latest/. Slycat is open source and can be downloaded from https://github.com/sandialabs/slycat.

The Slycat web client provides a Python package which can be used to interact with the Slycat web server. The web client provides Python routines to query the Slycat server and create Slycat data analysis models.

Installation

pip install slycat-web-client

If you are working behind a proxy, you might also need, e.g.

pip install slycat-web-client --proxy your-proxy:your-port

If you are getting SSL certificate errors, you can use:

pip install slycat-web-client --trusted-host pypi.org --trusted-host files.pythonhosted.org

Be aware that the last option is insecure. The better approach is to fix your SSL certificate and/or point Python to a copy of the certificate. This can be done using:

pip config set global.cert path-to-your-certificate

Note: that for the Slycat web client to work, you must have a Slycat server running.
See https://slycat.readthedocs.io/en/latest/ for details on setting up a server.

Basic Use

The Slycat web client can be imported from within a Python file using

import slycat.web.client

Some examples using the web client can be found in the slycat/web/client source directory. These can be run using, e.g.

$ python list_markings.py

or

$ python -m slycat.web.client.list_markings

In addition, there are two main entry points defined for the Slycat Dial-A-Cluster plugin

$ dac_tdms

and

$ dac_gen

These are described in greater detail below.

User Authentication

The Slycat server requires user authentication. The slycat.web.client module provides the options for the authentication process.

For example, to use standard password authentication for a Slycat server running on https://localhost:9000 without a security certificate, use:

$ python -m slycat.web.client.list_markings.py --user slycat --port 9000 --no-verify

Or, to access a Kerberos authenticated server running at slycat.sandia.gov, use:

$ python -m slycat.web.client.list_markings.py --host https://slycat.sandia.gov --kerberos

Kerberos

The --kerberos option relies on a working Kerberos installation on your system. Sometimes this will fail. If you get an error related to Kerberos credentials (e.g. "Couldn't find Kerberos ticket," or "User not Kerberos authenticated"), try:

$ kinit

Then re-run the original command.

Proxies/Certificates

If you are separated from the Slycat server by a proxy, or have not set up a security certificate, you will have to use the slycat.web.client proxy settings. The proxy settings are available using the flags:

  • --http-proxy
  • --https-proxy
  • --verify
  • --no-verify

The proxy flags are by default set to "no proxy". If you have proxies set in the environment variables, they will be ignored. The proxy flags are used as follows (for example):

$ python -m slycat.web.client.list_markings.py --http-proxy http://your.http.proxy --https-proxy https://your.https.proxy

The verify flag can be used to pass a security certificate as a command line argument and the --no-verify flag can be used to ignore the security certificates altogether.

General Utilities

The simplest examples of interacting with the Slycat server issue requests for markings and projects, e.g.

$ python -m slycat.web.client.list_markings.py
$ python -m slcyat.web.client.list_projects.py

To examine a particular model or project, use

$ python -m slycat.web.client.get_model.py mid
$ python -m slycat.web.client.get_project.py pid

where mid and pid are the hash identifiers for a Slycat model or project residing on the Slycat server. These IDs can be extracted from the URL in the Slycat web browser client, or by using Info -> Model Details from the browser.

Creating Models

The slycat.web.client provides a command line option for creating Slycat models. For example, to create a sample CCA model using random data, use:

$ python -m slycat.web.client.cca_random.py

To create a sample CCA model from a CSV file, use:

$ python -m slycat.web.client.cca_csv.py slycat-data/cars.csv --input Cylinders Displacement Weight Year --output MPG Horsepower Acceleration

where "slycat-data/cars.csv" is from the slycat-data git repository at https://github.com/sandialabs/slycat-data.

Note that when a model is created, the URL is given in the console and can be copied into a web browser to display the model. The model ID can also be extracted from this URL (it is the hash at the end of the URL).

Dial-A-Cluster (DAC) Models

Dial-A-Cluster models can be loaded using different formats. The first format is the generic dial-a-cluster format, described more fully in the Slycat user manual.

To upload a DAC generic .zip file, use

$ dac_gen dac-gen.zip

This will create a model from a single .zip file containing the appropriate folders with the pre-computed distance or PCA matrices.

Dial-A-Cluster TDMS Models

To upload a DAC TDMS model, use

$ dac_tdms data-file.TDM

This will create a model from a single .TDM file. You can also use .TDMS files and .zip archives containing .tdms files. The options available for the creation of the models are the same as the options available using the DAC model creation wizard in the browser. To see the options use the "--help" flag when calling the script.

In addition, there are two methods for uploading batches of multiple DAC TDMS models. The first method uses the dac_tdms_batch script. This script takes as input a part-num match (see also dac_run_chart) and creates the corresponding batches. For example,

$ dac_tdms_batch root-data-directory part-num

The second method uses the dac_tdms_batch_file script. To use this script, you must first create a file containing the options for each model. The file has the following format.

Line 1 contains the authentication information for the Slycat server that you would pass to the dac_tdms script, but separated by commas. For example,

--user,user-name,--kerberos

If authentication information is unnecessary, just leave the line blank.

Line 2 contains the project information for the project that will contain the DAC models to be created, e.g.

--project-name,Batch TDMS Models

Line 2 can also be left blank. It will default to "Batch TDMS Models". Lines 3 and beyond contain the model information for each model, such as

model-data-file-1.tdms,--model-name,Model 1,--project-name,Batch TDMS Models
model-data-file-2.tdms,--model-name,Model 2,--project-name,Batch TDMS Models

Note that you must supply a model file (or multiple files) in accordance with the dac_tdms script for each model. Also note that if you want to put models into different projects, you can override the original project given in line 2, by using the "--project-name" flag again, e.g.

model-data-file-n.tdms,--model-name,Model n,--project-name,Special Project

After the batch file has been created, you can call the TDMS batch processor to create your models using:

$ dac_tdms_batch tdms-batch-file.txt

where tdms-batch-file.txt is the .txt file containing the lines just described.

Depending on how many models are being created, it is helpful to use the "--log_file" flag to specify a log file for recording any errors in the upload process.

Dial-A-Cluster Run Chart Models

To create Dial-A-Cluster run chart model, use the dac_run_chart.py script. From the command line:

$ python -m slycat.web.client.dac_run_chart root-data-directory part-num run-charts.zip

where root-data-directory is the root directory containing the data directories indexed by part number, and part-num is the prefix for the data directories and associated structure of sub directories. By specifying the (mandatory) output zip file you get a .zip file that can be loaded into dial-a-cluster through the wizard (or the dac_gen script). You can later delete this file if it is not needed.

Parameter Space Models

A Paraneter Space model can be created from .csv file using the ps_csv script. From the command line, use:

$ python -m slycat.web.client.ps_csv slycat-data/cars.csv --input-columns Cylinders Displacement Weight Year --output-columns MPG Horsepower Acceleration

To push up a .csv file from a python script, use the slycat-web-client module. For example:

import slycat.web.client.ps_csv as ps_csv

# parameter space file
CARS_FILE = ['../../slycat-data/cars.csv']

# input/output columns for cars data file
CARS_INPUT = ['--input-columns', 'Model', 'Cylinders', 'Displacement', 'Weight', 'Year']
CARS_OUTPUT = ['--output-columns', 'MPG', 'Horsepower', 'Acceleration']

# create PS model from cars file
ps_parser = ps_csv.parser()
arguments = ps_parser.parse_args(CARS_FILE + CARS_INPUT + CARS_OUTPUT)
ps_csv.create_model(arguments, ps_csv.log)     

API

You should be able to find the API for slycat.web.client at https://slycat.readthedocs.io/en/latest/.

Contact

Shawn Martin -- smartin@sandia.gov

Distributed under the Sandia license. See LICENSE file for more information.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

slycat-web-client-4.2.0.tar.gz (47.0 kB view hashes)

Uploaded Source

Built Distribution

slycat_web_client-4.2.0-py3-none-any.whl (54.7 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page