raincoat 0.8.5

"Raincoat has your code covered when you can't stay DRY."

Raincoat has you covered when you can’t stay DRY. When the time comes where you HAVE to copy code from a third party, Raincoat will let you know when this code is changed so that you can update your local copy.

The problem

Lets say you’re using a lib named umbrella which provides a function named use_umbrella and it reads as such :

def use_umbrella(umbrella):

    # Prepare umbrella
    umbrella.remove_pouch()
    umbrella.open()

    # Use umbrella
    while rain_detector.still_raining():
        umbrella.keep_over_me()

    # Put umbrella away
    umbrella.close()
    while not umbrella.is_wet():
        time.sleep(1)
    umbrella.put_pouch()

This function does what it says it does, but it’s not ideally splitted, depending on your needs. For example, maybe at some point you realize you need each of the 3 separate parts to be a function of its own. Or maybe you can’t call time.sleep in your app. Or do something else with the umbrella when it’s open like dance with it.

It’s also possible that you can’t really make a pull request because your needs are specific, or you don’t have the time (that’s sad but, hey, I know it happens) or any other personnal reason. So what do you do ? There’s no real alternative. You copy and paste the code, modify it to fit your needs and use your modified version. And whenever there’s a change to the upstream function, chances are you’ll never know.

The solution

Enter Raincoat.

You have made your own private copy of umbrella.use_umbrella (umbrella being at the time at version 14.5.7) and it looks like this :

def dance_with_umbrella(umbrella):
        """
    I'm siiiiiinging in the rain !
    """
    # Prepare umbrella
    umbrella.remove_pouch()
    umbrella.open()

    # Use umbrella
    while rain_detector.still_raining():
        Dancer.sing_in_the_rain(umbrella)

    # Put umbrella away
    umbrella.close()
    while not umbrella.is_wet()
        time.sleep(1)
    umbrella.put_pouch()

Now simply add a comment somewhere (preferably just after the docstring) that says something like:

def dance_with_umbrella(umbrella):
    """
    I'm siiiiiinging in the rain !
    """
    # This code was adapted from the original umbrella.use_umbrella function
    # (we just changed the part inside the middle while loop)
    # Raincoat: pypi package: umbrella==14.5.7 path: umbrella/__init__.py element: use_umbrella

    ...

Now, if you run raincoat in your project (At this stage, I assume you’ve installed it with pip install raincoat)

$ raincoat

It will:

• Grep the code for all # Raincoat: comments and for each comment:

• Look at the currently installed version of the lib (say, umbrella 16.0.3) (or, if not found, the latest version)

• Compare with the version in the Raincoat comment (here, 14.5.7)

• If they are different, download and pip install the specified version in a temp dir (using cached wheel as pip does by default, this should be quite fast in most cases)

• Locate the code using the provided path for both the downloaded and the currently installed versions

• Diff it

• Tell you if there’s a difference (and mention the location of the original Raincoat comment)

Whether there is something to change or not, you’ve now verified your code with umbrella 16.0.3, so you can update manually the umbrella comment.

# Raincoat: pypi package: umbrella==16.0.3 path: umbrella/__init__.py element: use_umbrella"

Raincoat can be used like a linter, you can integrate it in CI, make it a tox target…

Note that if you omit the last argument, Raincoat will analyze the whole module:

# Raincoat: pypi package: umbrella==16.0.3 path: umbrella/__init__.py

And beyond !

Actually, the base principle of Raincoat can be extended to many other subjects than PyPI packages. To fit this, Raincoat was written with a modular achitecture allowing other kinds of Raincoat comments.

For now Raincoat comes with:

• PyPI: The module presented above

• Django: A module that checks if a given bug in Django for which you may have had to write a workaround is fixed in your (or the latest) version of Django. Syntax is :

# Raincoat: django ticket: #26976

• PyGitHub : Same as the PyPI module but using Github. It’s useful if your upstream is a python package that’s not on PyPI, like, say, the Python Standard Library itself. Say you want to know if the element Maildir._lookup in the file Lib/mailbox.py changed on the master branch since commit 43ba8861. What you can do is:

# Raincoat: pygithub repo: python/cpython@43ba8861 branch: master path: Lib/mailbox.py element: Maildir._lookup

Of course, feel free to code your own !

Caveats and Gotchas

• The 2 elements you provide in path should be the location of the file when the package is installed (in most case, this should match the location of the file in the project repo) and the object defined in this file. This object can be a variable, a class, a function or a method.

• Your own customized (copied/pasted) version of the function will not be analyzed. In fact, you don’t even have to place the Raincoat comment in the function that uses it.

• You may realize that raincoat works best if you can use some kind of pip cache.

• Raincoat does not run files (either your files or the package file). Package files are parsed and the AST is analyzed.

• If for any reason, several code objects are identically named in the file you analyze, there’s no guarantee you’ll get any specific one.

• The Django module uses the public GitHub API and does a few calls. This should not be a concern most of the time, but you may experience rate-limiting issues if Raincoat is launched from an IP that does a lot of calls to the GitHub API (e.g. Travis). In this case, from your Travis settings, set the environment variable RAINCOAT_GITHUB_TOKEN to username:github_token, github_token being a token generated here with all checkboxes unchecked.

• So few people use Raincoat for now that you should expect a few bumps down the road. This being said, fire issues and pull requetes at will and I’ll do my best to answer them in a timely manner.

Todos

Things I’d like to add at some point

• An option to update a comment automatically

• A way to say you want your customized function to be diffed too (in case it’s a close copy and you want to keep track of what you’ve modified)

• A way to access the original function without the process of downloading the whole package and installing it for nothing. We just want a single file of it.

• A smart way to make raincoat not need a pip cache (a cache of its own, or something)

Acknowledgments

This code is open-sourced and maintained by me (Joachim Jablon) during both my free time and my time working at PeopleDoc, based on an idea and a first implemention made at Smart Impulse. Kudos to these 2 companies.