sqoopy 0.0.4

UNKNOWN

sqoopy
======

Python CLI to generate custom [sqoop](http://sqoop.apache.org/) import statements.
Modified from [https://github.com/wikimedia/sqoopy/](https://github.com/wikimedia/sqoopy/).


## Installation

You can install `sqoopy` via `pip`:

```bash
$ pip install sqoopy
```

## Usage

`sqoopy` will generate custom [sqoop](http://sqoop.apache.org/) import statements given a few simple options:

```bash
usage: sqoopy [-h] [-c CONNECT] [-d TARGET_DIR] [-t TABLES]

Python CLI to generate custom sqoop import statements.

optional arguments:
-h, --help show this help message and exit
-c CONNECT, --connect CONNECT
A jdbc connection string.
-d TARGET_DIR, --target-dir TARGET_DIR
The directory to send output to. If sending to s3, use
"{table}" to insert the table name into the directory.
EG: s3://my-bucket/{table}/
-t TABLES, --tables TABLES
(Optional) comma-separated list of tables that need to
be inspected. If not supplied, all tables will be
imported.
```

For example, here is a simple command that will generate an import command given the [test database](tests/test.db):

```bash
$ sqoopy --connect=sqlite:///tests/test.db --target-dir=s3://foo-bar/
```

This will output:

```
sqoop import --connect=sqlite:///tests/test.db --table=test --target-dir=s3://foo-bar/
```

If you'd like to programmatically add the table name to the `target-dir`, use `{table}`:

```
$ sqoopy --connect=sqlite:///tests/test.db --target-dir=s3://foo-bar/{table}
```

This will output:

```
sqoop import --connect=sqlite:///tests/test.db --table=test --target-dir=s3://foo-bar/test/
```

If you'd like to use a custom list of tables to import, use the `--tables` argument, with each table name separated by a comma:

```
$ sqoopy --connect=sqlite:///tests/test.db --target-dir=s3://foo-bar/{table} --tables=test
```

Finally, you can also passthrough any other `sqoop import` arguments:

```bash
$ sqoopy --connect=sqlite:///tests/test.db --target-dir=s3://foo-bar/{table} --tables=test --split-by id --num-mappers 4
```

This will output:

```
sqoop import --connect=sqlite:///tests/test.db --table=test --target-dir=s3://foo-bar/test/ --split-by id --num-mappers 4
```

## Tests

You can run tests by first installing `nose`:

```
$ pip install nose
$ nosetests
```