Skip to main content

Filesystem events monitoring

Project description

https://travis-ci.org/gorakhargosh/watchdog.svg?branch=master

Python API and shell utilities to monitor file system events.

Works on Python 2.7 and 3.4+. If you want to use an old version of Python, you should stick with watchdog < 0.10.0.

Example API Usage

A simple program that uses watchdog to monitor directories specified as command-line arguments and logs events generated:

import sys
import time
import logging
from watchdog.observers import Observer
from watchdog.events import LoggingEventHandler

if __name__ == "__main__":
    logging.basicConfig(level=logging.INFO,
                        format='%(asctime)s - %(message)s',
                        datefmt='%Y-%m-%d %H:%M:%S')
    path = sys.argv[1] if len(sys.argv) > 1 else '.'
    event_handler = LoggingEventHandler()
    observer = Observer()
    observer.schedule(event_handler, path, recursive=True)
    observer.start()
    try:
        while True:
            time.sleep(1)
    except KeyboardInterrupt:
        observer.stop()
    observer.join()

Shell Utilities

Watchdog comes with an optional utility script called watchmedo. Please type watchmedo --help at the shell prompt to know more about this tool.

Here is how you can log the current directory recursively for events related only to *.py and *.txt files while ignoring all directory events:

watchmedo log \
    --patterns="*.py;*.txt" \
    --ignore-directories \
    --recursive \
    .

You can use the shell-command subcommand to execute shell commands in response to events:

watchmedo shell-command \
    --patterns="*.py;*.txt" \
    --recursive \
    --command='echo "${watch_src_path}"' \
    .

Please see the help information for these commands by typing:

watchmedo [command] --help

About watchmedo Tricks

watchmedo can read tricks.yaml files and execute tricks within them in response to file system events. Tricks are actually event handlers that subclass watchdog.tricks.Trick and are written by plugin authors. Trick classes are augmented with a few additional features that regular event handlers don’t need.

An example tricks.yaml file:

tricks:
- watchdog.tricks.LoggerTrick:
    patterns: ["*.py", "*.js"]
- watchmedo_webtricks.GoogleClosureTrick:
    patterns: ['*.js']
    hash_names: true
    mappings_format: json                  # json|yaml|python
    mappings_module: app/javascript_mappings
    suffix: .min.js
    compilation_level: advanced            # simple|advanced
    source_directory: app/static/js/
    destination_directory: app/public/js/
    files:
      index-page:
      - app/static/js/vendor/jquery*.js
      - app/static/js/base.js
      - app/static/js/index-page.js
      about-page:
      - app/static/js/vendor/jquery*.js
      - app/static/js/base.js
      - app/static/js/about-page/**/*.js

The directory containing the tricks.yaml file will be monitored. Each trick class is initialized with its corresponding keys in the tricks.yaml file as arguments and events are fed to an instance of this class as they arrive.

Tricks will be included in the 0.5.0 release. I need community input about them. Please file enhancement requests at the issue tracker.

Installation

Install from PyPI using pip:

$ python -m pip install watchdog

# or to install the watchmedo utility:
$ python -m pip install watchdog[watchmedo]

Install from source:

$ python -m pip install -e .

# or to install the watchmedo utility:
$ python -m pip install -e ".[watchmedo]"

Installation Caveats

The watchmedo script depends on PyYAML which links with LibYAML, which brings a performance boost to the PyYAML parser. However, installing LibYAML is optional but recommended. On Mac OS X, you can use homebrew to install LibYAML:

$ brew install libyaml

On Linux, use your favorite package manager to install LibYAML. Here’s how you do it on Ubuntu:

$ sudo aptitude install libyaml-dev

On Windows, please install PyYAML using the binaries they provide.

Documentation

You can browse the latest release documentation online.

Contribute

Fork the repository on GitHub and send a pull request, or file an issue ticket at the issue tracker. For general help and questions use the official mailing list or ask on stackoverflow with tag python-watchdog.

Create and activate your virtual environment, then:

python -m pip install pytest pytest-cov
python -m pip install -e .[watchmedo]
python -m pytest tests

If you are making a substantial change, add an entry to the “Unreleased” section of the changelog.

Supported Platforms

  • Linux 2.6 (inotify)
  • Mac OS X (FSEvents, kqueue)
  • FreeBSD/BSD (kqueue)
  • Windows (ReadDirectoryChangesW with I/O completion ports; ReadDirectoryChangesW worker threads)
  • OS-independent (polling the disk for directory snapshots and comparing them periodically; slow and not recommended)

Note that when using watchdog with kqueue, you need the number of file descriptors allowed to be opened by programs running on your system to be increased to more than the number of files that you will be monitoring. The easiest way to do that is to edit your ~/.profile file and add a line similar to:

ulimit -n 1024

This is an inherent problem with kqueue because it uses file descriptors to monitor files. That plus the enormous amount of bookkeeping that watchdog needs to do in order to monitor file descriptors just makes this a painful way to monitor files and directories. In essence, kqueue is not a very scalable way to monitor a deeply nested directory of files and directories with a large number of files.

About using watchdog with editors like Vim

Vim does not modify files unless directed to do so. It creates backup files and then swaps them in to replace the files you are editing on the disk. This means that if you use Vim to edit your files, the on-modified events for those files will not be triggered by watchdog. You may need to configure Vim appropriately to disable this feature.

About using watchdog with CIFS

When you want to watch changes in CIFS, you need to explicitly tell watchdog to use PollingObserver, that is, instead of letting watchdog decide an appropriate observer like in the example above, do:

from watchdog.observers.polling import PollingObserver as Observer

Dependencies

  1. Python 2.7, 3.4 or above.
  2. pathtools
  3. XCode (only on Mac OS X)
  4. PyYAML (only for watchmedo script)
  5. argh (only for watchmedo script)

Licensing

Watchdog is licensed under the terms of the Apache License, version 2.0.

Copyright 2011 Yesudeep Mangalapilly.

Copyright 2012 Google, Inc.

Project source code is available at Github. Please report bugs and file enhancement requests at the issue tracker.

Why Watchdog?

Too many people tried to do the same thing and none did what I needed Python to do:

Changelog

0.10.2

2020-02-08 • full history

  • Fixed the build_ext command on macOS Catalina (#628)
  • Fixed the installation of macOS requirements on non-macOS OSes (#635)
  • Refactored dispatch() method of FileSystemEventHandler, PatternMatchingEventHandler and RegexMatchingEventHandler
  • [bsd] Improved tests support on non Windows/Linux platforms (#633, #639)
  • [bsd] Added FreeBSD CI support (#532)
  • [bsd] Restored full support (#638, #641)
  • Thanks to our beloved contributors: @BoboTiG, @evilham, @danilobellini

0.10.1

2020-01-30 • full history

  • Fixed Python 2.7 to 3.6 installation when the OS locale is set to POSIX (#615)
  • Fixed the build_ext command on macOS (#618, #620)
  • Moved requirements to setup.cfg (#617)
  • [mac] Removed old C code for Python 2.5 in the fsevents C implementation
  • [snapshot] Added EmptyDirectorySnapshot (#613)
  • Thanks to our beloved contributors: @Ajordat, @tehkirill, @BoboTiG

0.10.0

2020-01-26 • full history

Breaking Changes

  • Dropped support for Python 2.6, 3.2 and 3.3
  • Emitters that failed to start are now removed
  • [snapshot] Removed the deprecated walker_callback argument, use stat instead
  • [watchmedo] The utility is no more installed by default but via the extra watchdog[watchmedo]

Other Changes

  • Fixed several Python 3 warnings
  • Identify synthesized events with is_synthetic attribute (#369)
  • Use os.scandir() to improve memory usage (#503)
  • [bsd] Fixed flavors of FreeBSD detection (#529)
  • [bsd] Skip unprocessable socket files (#509)
  • [inotify] Fixed events containing non-ASCII characters (#516)
  • [inotify] Fixed the way OSError are re-raised (#377)
  • [inotify] Fixed wrong source path after renaming a top level folder (#515)
  • [inotify] Removed delay from non-move events (#477)
  • [mac] Fixed a bug when calling FSEventsEmitter.stop() twice (#466)
  • [mac] Support for unscheduling deleted watch (#541)
  • [mac] Fixed missing field initializers and unused parameters in watchdog_fsevents.c
  • [snapshot] Don’t walk directories without read permissions (#408)
  • [snapshot] Fixed a race condition crash when a directory is swapped for a file (#513)
  • [snasphot] Fixed an AttributeError about forgotten path_for_inode attr (#436)
  • [snasphot] Added the ignore_device=False parameter to the ctor (597)
  • [watchmedo] Fixed the path separator used (#478)
  • [watchmedo] Fixed the use of yaml.load() for yaml.safe_load() (#453)
  • [watchmedo] Handle all available signals (#549)
  • [watchmedo] Added the --debug-force-polling argument (#404)
  • [windows] Fixed issues when the observed directory is deleted (#570 and #601)
  • [windows] WindowsApiEmitter made easier to subclass (#344)
  • [windows] Use separate ctypes DLL instances
  • [windows] Generate sub created events only if recursive=True (#454)
  • Thanks to our beloved contributors: @BoboTiG, @LKleinNux, @rrzaripov, @wildmichael, @TauPan, @segevfiner, @petrblahos, @QuantumEnergyE, @jeffwidman, @kapsh, @nickoala, @petrblahos, @julianolf, @tonybaloney, @mbakiev, @pR0Ps, javaguirre, @skurfer, @exarkun, @joshuaskelly, @danilobellini, @Ajordat

0.9.0

2018-08-28 • full history

  • Deleting the observed directory now emits a DirDeletedEvent event
  • [bsd] Improved the platform detection (#378)
  • [inotify] Fixed a crash when the root directory being watched by was deleted (#374)
  • [inotify] Handle systems providing uClibc
  • [linux] Fixed a possible DirDeletedEvent duplication when deleting a directory
  • [mac] Fixed unicode path handling fsevents2.py (#298)
  • [watchmedo] Added the --debug-force-polling argument (#336)
  • [windows] Fixed the FILE_LIST_DIRECTORY constant (#376)
  • Thanks to our beloved contributors: @vulpeszerda, @hpk42, @tamland, @senden9, @gorakhargosh, @nolsto, @mafrosis, @DonyorM, @anthrotype, @danilobellini, @pierregr, @ShinNoNoir, @adrpar, @gforcada, @pR0Ps, @yegorich, @dhke

0.8.3

2015-02-11 • full history

  • Fixed the use of the root logger (#274)
  • [inotify] Refactored libc loading and improved error handling in inotify_c.py
  • [inotify] Fixed a possible unbound local error in inotify_c.py
  • Thanks to our beloved contributors: @mmorearty, @tamland, @tony, @gorakhargosh

0.8.2

2014-10-29 • full history

  • Event emitters are no longer started on schedule if Observer is not already running
  • [mac] Fixed usued arguments to pass clang compilation (#265)
  • [snapshot] Fixed a possible race condition crash on directory deletion (#281)
  • [windows] Fixed an error when watching the same folder again (#270)
  • Thanks to our beloved contributors: @tamland, @apetrone, @Falldog, @theospears

0.8.1

2014-07-28 • full history

  • Fixed anon_inode descriptors leakage (#249)
  • [inotify] Fixed thread stop dead lock (#250)
  • Thanks to our beloved contributors: @Witos, @adiroiban, @tamland

0.8.0

2014-07-02 • full history

  • Fixed argh deprecation warnings (#242)
  • [snapshot] Methods returning internal stats info were replaced by mtime(), inode() and path() methods
  • [snapshot] Deprecated the walker_callback argument
  • [watchmedo] Fixed auto-restart to terminate all children processes (#225)
  • [watchmedo] Added the --no-parallel argument (#227)
  • [windows] Fixed the value of INVALID_HANDLE_VALUE (#123)
  • [windows] Fixed octal usages to work with Python 3 as well (#223)
  • Thanks to our beloved contributors: @tamland, @Ormod, @berdario, @cro, @BernieSumption, @pypingou, @gotcha, @tommorris, @frewsxcv

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for watchdog, version 0.10.2
Filename, size File type Python version Upload date Hashes
Filename, size watchdog-0.10.2.tar.gz (95.5 kB) File type Source Python version None Upload date Hashes View

Supported by

Pingdom Pingdom Monitoring Google Google Object Storage and Download Analytics Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page