This is a pre-production deployment of Warehouse. Changes made here affect the production instance of PyPI (pypi.python.org).
Help us improve Python packaging - Donate today!

An intent to write SQL like programs and to translate them in Python, SQL, Hive, PIG.

Project Description

Links:

Description

Writing a map/reduce job (using PIG for example), usually requires to switch from local files to remote files (on Hadoop). On way to work is extract a small sample of the data which will be processed by a map/reduce job. The job is then locally developped. And when it works, it is run on a parallized environment.

The goal of this extension is allow the implementation of this job using Python syntax as follows:

def myjob(input):
    iter = input.select (input.age, input.nom, age2 = input.age2*input.age2)
    wher = iter.where( (iter.age > 60).Or(iter.age < 25))
    return where

input = IterRow (None, [ {"nom": 10}, {"jean": 40} ] )
output = myjob(input)

When the job is ready, it can be translated into a PIG job:

input = LOAD '...' USING PigStorage('\t') AS (nom, age);
iter = FOREACH input GENERATE age, nom, age*age AS age2 ;
wher = FILTER iter BY age > 60 or age < 25 ;
STORE wher INTO '...' USING PigStorage();

It should also be translated into SQL.

Functionalities

  • not yet ready

Design

This project contains various helper about logging functions, unit tests and help generation.
  • a source folder: src
  • a unit test folder: _unittests, go to this folder and run run_unittests.py
  • a _doc folder: _doc, it will contains the documentation
  • a file setup.py to build and to install the module
  • a file make_help.py to build the sphinx documentation
Release History

Release History

This version
History Node

0.1.141

History Node

0.1.130

Download Files

Download Files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

File Name & Checksum SHA256 Checksum Help Version File Type Upload Date
pysqllike-0.1.141-py3-none-any.whl (24.0 kB) Copy SHA256 Checksum SHA256 3.5 Wheel Jan 1, 2017
pysqllike-0.1.141.tar.gz (18.6 kB) Copy SHA256 Checksum SHA256 Source Jan 1, 2017

Supported By

WebFaction WebFaction Technical Writing Elastic Elastic Search Pingdom Pingdom Monitoring Dyn Dyn DNS Sentry Sentry Error Logging CloudAMQP CloudAMQP RabbitMQ Heroku Heroku PaaS Kabu Creative Kabu Creative UX & Design Fastly Fastly CDN DigiCert DigiCert EV Certificate Rackspace Rackspace Cloud Servers DreamHost DreamHost Log Hosting