Skip to main content

Data Science Work Space for Jupyter & CDSW

Project description

Data Science WorkSpace (dsws)

DSWS is Python package providing conventions for configuration and use of existing python packages. Intent is to make disparate source data platforms consistanly accessible in python ad-hoc and workflow analyses independent of data source or REPL/RunTime environment.

Main Features

DSWS offers no functionlity beyond what’s in dependency libraries. Instead, it provides conventions to consistantly and concisely use existing libraries. Conventions include:

  • Configurations for data access are instantiated at the beginning of a session, specific to analysis.
  • Configurations include not just source data connection, but also environement preferences in the connection, like default database and resource queue.
  • Common method syntax for connection and query.
  • For ipython environements, include line and cell magic for executing sql.
  • Consistant approach for profile startup scripts.
  • Ability to transition to configured CLI from within python session.


DSWS originated to address connectivity complexity of connecting to popular hadoop distributions (HDP, CDH) from edge node. Since the original release, it now includes conventions to connect from cluster edge node virtual environement, local device virtual environement, cdsw, jupyter notebooks, or quickstart/sandbox. DSWS is intended to provide a consistant user experience within any of these evnironements.

Included Libraries & CLIs

Exisiting libraries, classes, configs, and type.

package default name class type Incl Exp Roadmap
impyla impyla conn X
pyodbc pyodbc conn X
iopro iopro conn X
ibis ibis_impala client X
pyspark spark (sql) sess X
SQLAlchemy sa conn X
dask-yarn dy sess X
phoenixdb pheonix conn X
pydruid druid conn X
pyflink flink conn X
pandasql ps X

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

dsws-0.17.tar.gz (9.1 kB view hashes)

Uploaded source

Built Distribution

dsws-0.17-py3-none-any.whl (18.1 kB view hashes)

Uploaded py3

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Huawei Huawei PSF Sponsor Microsoft Microsoft PSF Sponsor NVIDIA NVIDIA PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page