Skip to main content

Utilities that allow a batch of CSV files to be passed as a single SQLITE db

Project description

# DancingPeasant

## Overview

This is a living / functional experiment in moving bioinformatics workflows away from multiple CSV files and towards SQL (specifically SQLite).
DancingPeasant consists of two main classes: BaseFile and Interface

BaseFile is a base class that enables packaging of several csv-type files into one SQLite database. It supports creation, addition of tables, versioning and history.
Interface is a base class that provides access (currently insert / select / update) to the tables in BaseFile.

## Installation

DancingPeasant depends only on Python core libraries and it's on PIP. so just:

$ pip install DancingPeasant

## Example usage

You should make your classes derive from BaseFile and Interface. Suppose you want to make a new db-based file format called a foofile (*.ff). First you need to extend the BaseClass:

Somewhere within your "foo" package you should make a file called "" with the following contents:

# Always version your files!
__FOO_DB_VERSION__ = "1.0.0"

from dancingPeasant.baseFile import BaseFile
from dancingPeasant.exceptions import *

class FooDB(BaseFile):
def __init__(self, verbosity=0):
BaseFile.__init__(self, verbosity)

def createNewFile(self,
fileName, # name of the new file
force=False, # should we check to see if this is a wise move?
"""Create a new Foo database file"""
# make a basic file

# add Foo specific tables
self._addTable("bars", # the name of the table
"length" : "INT", # Specify column names and
"diameter" : "INT", # standard SQL syntax for the type
"material" : "TEXT", # As may as you like

# Keep adding tables

To read and write from this file you'll need to extend the Interface class.
Make a file called with the following contents:

from dancingPeasant.exceptions import *
from dancingPeasant.interface import Interface
from dancingPeasant.interface import Condition
from foo.db import FooDB # <--- NOTE: foo is the name of your package and FooDB is the name of your file class

class FooInterface(Interface):
"""Use this interface for importing data stored in csv
files into the TrackM DB"""

def __init__(self,
dbFileName, # file name to connect to
verbosity=-1 # turn off all DP chatter
Interface.__init__(self, dbFileName, verbosity)
self.db = FooDB(verbosity=verbosity) # <-- This line is important!

def addBars(self, bars):
"""Add bars to the database"""
# connect to the database

# bars is an array of tuples. All ordered in the same way
# EX:
# [(10, 1, "iron"), (20, 15, "wax"), ... ]

# disconnect once done

def getBars(self, length):
"""Get bars longer tha a set length"""

# connect to the database

# set the select condition
C = Condition("length", ">", length)

# access the database and get rows
rows ='bars', ["material", "diameter"], C)

# disconnect once done

# do something with the results

This is just a starter. Hopefully I'll add some more examples here soon but I'm still busy writing it and trying to learn how to use it.
In the meantime check out the unit tests in dancngpeasant and the various interfaces and dbs used in TrackM and StoreM for more examples on how I use this class.

## Help

If you experience any problems using DancingPeasant, open an [issue]( on GitHub and tell us about it.

## Licence

GPL3, with love.

Project home page, info on the source tree, documentation, issues and how to contribute, see

## Copyright

Copyright (c) 2014 Michael Imelfort. See LICENSE.txt for further details.

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for DancingPeasant, version 0.1.6
Filename, size File type Python version Upload date Hashes
Filename, size DancingPeasant-0.1.6.tar.gz (22.4 kB) File type Source Python version None Upload date Hashes View

Supported by

Pingdom Pingdom Monitoring Google Google Object Storage and Download Analytics Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page