This is a pre-production deployment of Warehouse, however changes made here WILL affect the production instance of PyPI.
Help us improve Python packaging - Donate today!
Project Description

This package tries to do the best to support the development of sample data generators. A sample data generator is a pluggable tool to create data needed for tests.

Detailed Documentation

Pluggable sample data framework

Creating a good testing environment is the most important way to create high quality software.

But most of the time it is a pain !

This package tries to do the best to support the development of sample data generators. A sample data generator is a pluggable tool to create data needed for tests.

There are several goals to this framework:

  • provide the developers with an automatic setup that is close to real-world use.
  • provide the users with an easy setup for evaluation with plausible data
  • provide the user a management interface for different sample generators

The framework is pluggable and allows the creators of generator extensions to provide their own plugins that generate sample data for those extensions.

Generators

A generator generates sample data.

>>> from zope import interface
>>> from zope import component
>>> from z3c.sampledata.interfaces import ISampleDataPlugin
>>> @interface.implementer(ISampleDataPlugin)
... class GeneratePrincipals(object):
...     dependencies = []
...     schema = None
...     def generate(self, context, param={}, dataSource=None, seed=None):
...         print(self.__class__.__name__)
...         if dataSource is not None:
...             for data in dataSource:
...                 print('- %s'%data['login'])
>>> principalPlugin = GeneratePrincipals()

For our tests we provide another generator :

>>> @interface.implementer(ISampleDataPlugin)
... class GenerateSite(object):
...     dependencies = []
...     schema = None
...     def generate(self, context, param={}, dataSource=None, seed=None):
...         if 'sitename' in param:
...             print('This is site %r'%param['sitename'])
...         else:
...             print(self.__class__.__name__)
...         return 'I am from the site'
>>> sitePlugin = GenerateSite()

Generator Manager

A generator manager groups a collection of generators. The manager allows to :

  • define dependencies between generators
  • define data connections between dependent generators
  • provide default configuration data
>>> from z3c.sampledata import Manager
>>> manager = Manager('manager', '')
Generator Plugin

For the manager our sample generators must be registered as named utilities.

>>> component.provideUtility(sitePlugin,
...                          ISampleDataPlugin,'z3c.sampledata.site')
>>> component.provideUtility(principalPlugin,
...                          ISampleDataPlugin,'z3c.sampledata.principals')
Generating Sample Data

Now we can add generators to the manager.

>>> manager.add('z3c.sampledata.principals',
...             dependsOn=['z3c.sampledata.site',],
...             contextFrom='z3c.sampledata.site')

In addition to the “hardwired” dependencies defined by the dependencies property in each generator it is possible to add dependencies in the generator manager.

A manager provides it’s generators.

>>> list(manager.generators.keys())
['z3c.sampledata.principals']

We can tell the manager to generate all samples. There is no need to add the sample generator ‘z3c.sampledata.site’, it is added automatically because of the dependency of ‘z3c.sampledata.principals’.

>>> infos = manager.generate(context=None, param={}, seed='something')
GenerateSite
GeneratePrincipals
>>> [info.name for info in infos]
['z3c.sampledata.site', 'z3c.sampledata.principals']

Parameters for the sample generators

To have more control over the sample generation process it is possible to setup parameters for the generators.

>>> manager = Manager('manager', '')
>>> manager.add('z3c.sampledata.site',
...             param={'sitename':'samplesite'})
>>> manager.add('z3c.sampledata.principals',
...             dependsOn=['z3c.sampledata.site',],
...             contextFrom='z3c.sampledata.site')
>>> infos = manager.generate(context=None, param={}, seed='something')
This is site 'samplesite'
GeneratePrincipals

It is also possible to overwrite the parameters from the configuration.

>>> infos = manager.generate(context=None,
...                          param={'z3c.sampledata.site':
...                                 {'sitename':'managers site'}},
...                          seed='something')
This is site 'managers site'
GeneratePrincipals

Cycles in the generator definition

>>> manager = Manager('manager', '')
>>> manager.add('z3c.sampledata.principals',
...             dependsOn=['z3c.sampledata.site',],
...             contextFrom='z3c.sampledata.site')
>>> manager.add('z3c.sampledata.site',
...             dependsOn=['z3c.sampledata.principals',])
>>> infos = manager.generate(context=None, param={}, seed='something')
Traceback (most recent call last):
...
CyclicDependencyError: cyclic dependency at 'z3c.sampledata.principals'

A test for a complex dependency.

>>> @interface.implementer(ISampleDataPlugin)
... class Generator(object):
...     name = 'generator'
...     dependencies = []
...     schema = None
...     def generate(self, context, param={}, dataSource=None, seed=None):
...         return 'I am a generator'
>>> component.provideUtility(Generator(), ISampleDataPlugin,'g.1')
>>> component.provideUtility(Generator(), ISampleDataPlugin,'g.2')
>>> component.provideUtility(Generator(), ISampleDataPlugin,'g.3')
>>> manager = Manager('manager', '')
>>> manager.add('g.1')
>>> manager.add('g.2', contextFrom='g.1')
>>> manager.add('g.3', dependsOn=['g.2', 'g.1'], contextFrom='g.1')
>>> infos = manager.generate(context=None, param={}, seed=None)
>>> [info.name for info in infos]
['g.1', 'g.2', 'g.3']

Sample Data Source

A sample data generator usually gets its sample data from a data source. Mostly it is necessary to have different data sources for different uses.

As an example, it is always a pain if the sample data for the tests use the same data as the UI uses later to provide data for the customer to click around.

>>> manager = Manager('manager', '')
>>> manager.addSource('z3c.datasource.principals',
...                   data=[{'login':'jukart', 'password':'trakuj'},
...                         {'login':'srichter', 'password':'rethcirs'}])
>>> manager.add('z3c.sampledata.principals',
...             dataSource='z3c.datasource.principals',
...             dependsOn=['z3c.sampledata.site',],
...             contextFrom='z3c.sampledata.site')
>>> infos = manager.generate(context=None, param={}, seed='something')
GenerateSite
GeneratePrincipals
- jukart
- srichter

It is also possible to use adapters to act as a data source.

>>> manager = Manager('manager', '')
>>> class IPrincipalDataSource(interface.Interface):
...     pass
>>> def principalDataFactory(object):
...      return [{'login':'jukart', 'password':'trakuj'},
...              {'login':'srichter', 'password':'rethcirs'}]
>>> component.provideAdapter(
...                          factory=principalDataFactory,
...                          adapts=(ISampleDataPlugin,),
...                          provides=IPrincipalDataSource,
...                          name='testprincipals')
>>> manager.addSource('z3c.datasource.principals',
...                   adapterName='testprincipals',
...                   adaptTo=IPrincipalDataSource)
>>> manager.add('z3c.sampledata.principals',
...             dataSource='z3c.datasource.principals',
...             dependsOn=['z3c.sampledata.site',],
...             contextFrom='z3c.sampledata.site')
>>> infos = manager.generate(context=None, param={}, seed='something')
GenerateSite
GeneratePrincipals
- jukart
- srichter

How to setup configuration for the generator manager

Configuration can be done using ZCML:

<configure xmlns="http://namespaces.zope.org/zope">

  <configure
      xmlns:zcml="http://namespaces.zope.org/zcml"
      zcml:condition="have devmode">

    <utility
        factory=".SampleSite"
        provides="z3c.sampledata.interfaces.ISampleDataPlugin"
        name="z3c.site"
        />

    <utility
        factory=".SamplePrincipals"
        provides="z3c.sampledata.interfaces.ISampleDataPlugin"
        name="z3c.principals"
        />

    <SampleManager
      name="Site with principals"
      >
      <generator name="z3c.site" />
      <generator
        name="z3c.principal"
        dependsOn="z3c.site"
        contextFrom="z3c.site" />
    </SampleManager>

  </configure>

</configure>

Data Sources

This package implements the base functionality for data generators. A data generator is used to provide the raw data for a sample generator. Raw data can be read from text files in different ways.

>>> from z3c.sampledata.data import DataGenerator
>>> generator = DataGenerator(55)

The generator can read data lines from files.

>>> generator.readLines('testlines.txt')
[u'Line 1', u'Another line']

The generator can read data from CSV files.

>>> generator.readCSV('testlines.csv')
[['Line 1', 'Col 2'], ['Another line', 'Another Col']]

The generator can read a list of files from a path :

>>> import os
>>> generator.files(os.path.dirname(__file__))
['...README.txt', ...]

CHANGES

2.0.0 (2015-11-09)

  • Standardize namespace __init__

2.0.0a1 (2013-03-04)

  • Added support for Python 3.3.
  • Dropped browser support completely, since it relied on really old o-wrap techniques. Also, it contained Lovely-specif project language from about 7 years ago. ;-)
  • Replaced deprecated zope.interface.implements usage with equivalent zope.interface.implementer decorator.
  • Dropped support for Python 2.4 and 2.5.

1.0.0 (2013-02-26)

  • Moved depndencies for the special SampleDataLayer setup to the “testing” extra.
  • Added MANIFEST.in and tox.ini.
  • Fixed file headers.
  • Browser views are only available using the “browser” extra.
  • Change zope.app.pagetemplate dependency to zope.browserpage.

0.4.0 (2010-08-30)

  • Use current packages.

0.3.1 (2010-08-30)

  • Update dependency meta, clean imports.

0.3.0 (2010-06-28)

  • Configured test runner to run functional tests.
  • Removed ZPKG and ZCML slugs.
  • Now requires zope.app.pagetemplate >= 3.6.0 as zope.formlib.namedtemplate has been moved there an there is no longer a BBB import zope.formlib >= 4.0
  • Fixed tests to run with zope.app.authentication >= 3.7.0.
  • Fixed tests to run with zope.publisher >= 3.12.0.
  • Using python’s doctest module instead of deprecated zope.testing.doctestunit.

0.2.0 (2010-06-25)

  • adjust zope.app.session to zope.session in ftesting.zcml
  • Adjusted zope.app.securitypolicy to zope.securitpolicy in ftesting.zcml.
  • Fixed tests after zope packages refactoring. Updated imports and dependencies.

0.1.0 (2008-02-14)

  • Initial release.
Release History

Release History

2.0.0

This version

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

2.0.0a1

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

1.0.0

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.4.0

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.3.1

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.3.0

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.2.0

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.1.0

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

Download Files

Download Files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

File Name & Checksum SHA256 Checksum Help Version File Type Upload Date
z3c.sampledata-2.0.0.tar.gz (21.6 kB) Copy SHA256 Checksum SHA256 Source Nov 9, 2015

Supported By

WebFaction WebFaction Technical Writing Elastic Elastic Search Pingdom Pingdom Monitoring Dyn Dyn DNS Sentry Sentry Error Logging CloudAMQP CloudAMQP RabbitMQ Heroku Heroku PaaS Kabu Creative Kabu Creative UX & Design Fastly Fastly CDN DigiCert DigiCert EV Certificate Rackspace Rackspace Cloud Servers DreamHost DreamHost Log Hosting