Skip to main content
Help us improve Python packaging – donate today!

UNKNOWN

Project Description

sqoopy
======

Python CLI to generate custom [sqoop](http://sqoop.apache.org/) import statements.
Modified from [https://github.com/wikimedia/sqoopy/](https://github.com/wikimedia/sqoopy/).


## Installation

You can install `sqoopy` via `pip`:

```bash
$ pip install sqoopy
```

## Usage

`sqoopy` will generate custom [sqoop](http://sqoop.apache.org/) import statements given a few simple options:

```bash
usage: sqoopy [-h] [-c CONNECT] [-d TARGET_DIR] [-t TABLES]

Python CLI to generate custom sqoop import statements.

optional arguments:
-h, --help show this help message and exit
-c CONNECT, --connect CONNECT
A jdbc connection string.
-d TARGET_DIR, --target-dir TARGET_DIR
The directory to send output to. If sending to s3, use
"{table}" to insert the table name into the directory.
EG: s3://my-bucket/{table}/
-t TABLES, --tables TABLES
(Optional) comma-separated list of tables that need to
be inspected. If not supplied, all tables will be
imported.
```

For example, here is a simple command that will generate an import command given the [test database](tests/test.db):

```bash
$ sqoopy --connect=sqlite:///tests/test.db --target-dir=s3://foo-bar/
```

This will output:

```
sqoop import --connect=sqlite:///tests/test.db --table=test --target-dir=s3://foo-bar/
```

If you'd like to programmatically add the table name to the `target-dir`, use `{table}`:

```
$ sqoopy --connect=sqlite:///tests/test.db --target-dir=s3://foo-bar/{table}
```

This will output:

```
sqoop import --connect=sqlite:///tests/test.db --table=test --target-dir=s3://foo-bar/test/
```

If you'd like to use a custom list of tables to import, use the `--tables` argument, with each table name separated by a comma:

```
$ sqoopy --connect=sqlite:///tests/test.db --target-dir=s3://foo-bar/{table} --tables=test
```

Finally, you can also passthrough any other `sqoop import` arguments:

```bash
$ sqoopy --connect=sqlite:///tests/test.db --target-dir=s3://foo-bar/{table} --tables=test --split-by id --num-mappers 4
```

This will output:

```
sqoop import --connect=sqlite:///tests/test.db --table=test --target-dir=s3://foo-bar/test/ --split-by id --num-mappers 4
```

## Tests

You can run tests by first installing `nose`:

```
$ pip install nose
$ nosetests
```

Release history Release notifications

This version
History Node

0.0.75

History Node

0.0.74

History Node

0.0.73

History Node

0.0.72

History Node

0.0.9.1

History Node

0.0.9.0

History Node

0.0.8.9

History Node

0.0.8.8

History Node

0.0.8.7

History Node

0.0.8.6

History Node

0.0.8.5

History Node

0.0.8.3

History Node

0.0.8.2

History Node

0.0.8

History Node

0.0.7.1

History Node

0.0.7

History Node

0.0.5

History Node

0.0.4

History Node

0.0.3

History Node

0.0.2

History Node

0.0.1

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Filename, size & hash SHA256 hash help File type Python version Upload date
sqoopy-0.0.75.macosx-10.10-intel.exe (108.8 kB) Copy SHA256 hash SHA256 Windows Installer any Jan 8, 2016
sqoopy-0.0.75.tar.gz (5.2 kB) Copy SHA256 hash SHA256 Source None Jan 8, 2016

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging CloudAMQP CloudAMQP RabbitMQ AWS AWS Cloud computing Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page