Skip to main content

Data Science Work Space for Jupyter & CDSW

Project description

Data Science WorkSpace (dsws)

DSWS is Python package providing conventions for configuration and use of existing python packages. Intent is to make disparate source data platforms consistanly accessible in python ad-hoc and workflow analyses independent of data source or REPL/RunTime environment.

Main Features

DSWS offers no functionlity beyond what’s in dependency libraries. Instead, it provides conventions to consistantly and concisely use existing libraries. Conventions include:

  • Configurations for data access are instantiated at the beginning of a session, specific to analysis.
  • Configurations include not just source data connection, but also environement preferences in the connection, like default database and resource queue.
  • Common method syntax for connection and query.
  • For ipython environements, include line and cell magic for executing sql.
  • Consistant approach for profile startup scripts.
  • Ability to transition to configured CLI from within python session.

Environments

DSWS originated to address connectivity complexity of connecting to popular hadoop distributions (HDP, CDH) from edge node. Since the original release, it now includes conventions to connect from cluster edge node virtual environement, local device virtual environement, cdsw, jupyter notebooks, or quickstart/sandbox. DSWS is intended to provide a consistant user experience within any of these evnironements.

Included Libraries & CLIs

Exisiting libraries, classes, configs, and type.

package default name class type Incl Exp Roadmap
impyla impyla ws_impyla.py conn X
pyodbc pyodbc ws_pyodbc.py conn X
iopro iopro ws_iopro.py conn X
ibis ibis_impala ws_ibis_impala.py client X
pyspark spark (sql) ws_spark.py sess X
SQLAlchemy sa ws_sqlalchemy.py conn X
dask-yarn dy ws_daskyarn.py sess X
phoenixdb pheonix ws_pheonix.py conn X
pydruid druid ws_druid.py conn X
pyflink flink ws_flink.py conn X
pandasql ps ws_ps.py X

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for dsws, version 0.17
Filename, size File type Python version Upload date Hashes
Filename, size dsws-0.17-py3-none-any.whl (18.1 kB) File type Wheel Python version py3 Upload date Hashes View
Filename, size dsws-0.17.tar.gz (9.1 kB) File type Source Python version None Upload date Hashes View

Supported by

Pingdom Pingdom Monitoring Google Google Object Storage and Download Analytics Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page