This is a pre-production deployment of Warehouse, however changes made here WILL affect the production instance of PyPI.
Latest Version Dependencies status unknown Test status unknown Test coverage unknown
Project Description

File System Overlay (FSO) allows side-effect free unit testing of file I/O operations. It does this by creating a caching overlay over the local file system, allowing read-through access, but storing modifications in memory. These in-memory changes can be inspected, to validate unit tests, and when the test completes, all changes to the file system will be vaporized and blown out to sea (to quote Dr. Stanley Goodspeed: http://www.youtube.com/watch?v=K-uEbYq9kNU&t=6m29s).

TL;DR

Install:

$ pip install fso

Use:

import unittest, fso

class MyTest(unittest.TestCase):

  def setUp(self):
    self.fso = fso.push()

  def tearDown(self):
    fso.pop()

  def test_fs_changes(self):

    self.assertFalse(os.path.exists('/etc/foobar.conf'))

    with open('/etc/foobar.conf', 'wb') as fp:
      fp.write('some-data')

    self.assertTrue(os.path.exists('/etc/foobar.conf'))
    self.assertEqual(open('/etc/foobar.conf', 'rb').read(), 'some-data')

    # BUT, when testing ends, /etc/foobar.conf will not exist! *awesome*! :)

    # you can also check that the expected changes are there (noting
    # that all paths are absolutized, dereferenced, and normalized):
    self.assertEqual(self.fso.changes, [
      'add:/etc/foobar.conf',
      ])

Overview

Traditionally, testing I/O operations on the file system requires modifying the implementation so that there is a pluggable layer of file operations that gets replaced with mocks when performing tests (http://stackoverflow.com/questions/2655697/python-unittest-howto).

This is, IMHO, a terrible approach, since it means that the real code is not being executed, and may well hide some very real bugs.

As an alternative, the FSO package switches out the implementation of the low-level file system calls, and caches changes in-memory, never actually modifying the file system.

Although this is a very “pure” approach, there are some gotchas… So, currently, only basic file operations are supported (such as creating and deleting files and directories) – if you are doing more complex things such as opening unix domain sockets and working with block-special devices, FSO may not be up to the task. But, if you don’t mind, please help identify those holes by either reporting issues or providing patches… any contributions will be merged and very much appreciated!

Supported Operations

Currently, only the following I/O functions have replacements implemented:

  • builtin.open
  • os.symlink
  • os.stat
  • os.lstat
  • os.unlink
  • os.remove
  • os.listdir
  • os.mkdir
  • os.makedirs
  • os.rmdir
  • os.path.exists
  • os.path.lexists
  • os.access
  • os.path.islink

Most other I/O operations are built on top of these, so they implicitly work with FSO. However, because they use whatever instrumented functions are currently in the global scope, this means that they are not compatible with multiple levels of FSO overlays. Since that is not the typical FSO use-case, this is deemed an acceptable trade-off.

Examples of I/O operations that are supported, but only when using a single active FSO layer:

  • os.walk
  • os.path.isdir
  • os.path.isfile

Known Limitations

  • File permissions are currently NOT enforced (and might be overkill), and overlayed directories report mode 0700, and overlayed files and symlinks report mode 0600.
  • File open modes of ‘U’ and ‘rU’ are silently treated as ‘r’.
  • The following stat attributes are not available/managed: * st_ino * st_dev * st_nlink * st_uid * st_gid * st_atime * st_mtime * st_ctime
  • Since changes are explicitly stored in-memory, changes that exceed the local machine’s memory will cause problems.
  • The following categories of filesystem entries will not work: * sockets * block special device files * character special device files * FIFOs (named pipes)

Usage

FSO supports context managers! In most cases, this is actually recommend. The reason is that some unit testing frameworks, such as nose, do not report errors very well if an FSO layer is still active. Using the context manager will ensure that the FSO is uninstalled before they need to report the errors. Example:

import unittest, fso

class TestWithContextManager(unittest.TestCase):

  def test_with_cm(self):

    self.assertFalse(os.path.exists('no-such-file'))

    with fso.push() as overlay:

      self.assertFalse(os.path.exists('no-such-file'))

      with open('no-such-file', 'wb') as fp:
        fp.write('created')

      os.unlink('/etc/hosts')
      os.mkdir('/tmp/my-test-directory')

      self.assertTrue(os.path.exists('no-such-file'))
      self.assertEqual(overlay.changes, [
        'del:/etc/hosts',
        'add:/path/to/cwd/no-such-file',
        'add:/tmp/my-test-directory',
        ])

    self.assertFalse(os.path.exists('no-such-file'))
    self.assertFalse(os.path.exists('/etc/my-test-directory'))
    self.assertTrue(os.path.exists('/etc/hosts'))
Release History

Release History

0.3.1

This version

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.3

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.2

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.1.5

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.1.4

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.1.3

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.1.1

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.1.0

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.0.1

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

Download Files

Download Files

TODO: Brief introduction on what you do with files - including link to relevant help section.

File Name & Checksum SHA256 Checksum Help Version File Type Upload Date
fso-0.3.1.tar.gz (27.3 kB) Copy SHA256 Checksum SHA256 Source Aug 18, 2015

Supported By

WebFaction WebFaction Technical Writing Elastic Elastic Search Pingdom Pingdom Monitoring Dyn Dyn DNS HPE HPE Development Sentry Sentry Error Logging CloudAMQP CloudAMQP RabbitMQ Heroku Heroku PaaS Kabu Creative Kabu Creative UX & Design Fastly Fastly CDN DigiCert DigiCert EV Certificate Rackspace Rackspace Cloud Servers DreamHost DreamHost Log Hosting