File Tree Subs

Allows to synchronize a destination file tree from a source file tree while allowing certain substitutions to take place.

File Tree Subs uses doit under the hood to keep track of changes, so that files are only changed if necessary.

See the following three examples for typical use cases of filetreesubs. I'm personally using it to preprocess the output of Nikola, a static blog/site generator, to insert a sidebar into all generated HTML pages, and a tag cloud into the sidebar and the tag overview page.

To install, use pip install filetreesubs.

Example

Assume you have the following file tree:

input/
    index.html
    team.html
    products.html
    menu.inc
    testimonials.inc

In the .html files, you put placeholder strings INSERT_MENU_HERE for where the content of input/menu.inc should be inserted, and INSERT_TESTIMONIALS for where the content of input/testimonials.inc should be inserted. Also, you want COPYRIGHT_YEAR to be replaced by 2017. The result should be a tree like this, without the .inc files:

output/
    index.html
    team.html
    products.html

with the placeholder string replaced. To do this with filetreesubs, create a config file filetreesubs-config.yaml:

# Source directory
source: input
# Destination directory
destination: output
substitutes:
  # The following is a regular expression to match the filenames:
  '.*\.html':
    # The strings to replace
    'INSERT_MENU_HERE':
      # With what to replace them
      file: menu.inc
    'INSERT_TESTIMONIALS':
      file: testimonials.inc
    'COPYRIGHT_YEAR':
      text: '2017'

Then running filetreesubs will synchronize output/ so that it contains the files from input/, except menu.inc, and makes sure the substitutions take place.

Example: Sidebar in Nikola

You can find an example site for Nikola using the sidebar plugin in the Github repository felixfontein/filetreesubs-nikola-demo.

A more complex, but less explicit example can be found in my blog, which also includes a tag cloud (rendered by the static_tag_cloud pugin) into the sidebar.

Example: Substitution chains

Assume that in the above example, you want to use INSERT_TESTIMONIALS also in menu.inc itself. Running the above example, this substitution will not be done, also if you extend the regular expression matching all HTML files to .* to match all files.

To apply substitutions to included files, you need to use substitution chains. Append the following to the configuration above:

substitute_chains:
- template: menu.inc
  substitutes:
    'INSERT_TESTIMONIALS':
      file: testimonials.inc

This will apply the substitution for INSERT_TESTIMONIALS also to menu.inc.

Example: Creating index files

Assume that you have folder structure:

input/
    index.html
    images/
        logo.jpeg
        2017/
            happynewyear-2017.jpeg

You want to upload the output to a web server so it is available under http://example.com, but if someone accesses http://example.com/images/ or http://example.com/images/2017/, you don't want the persons to see a file listing or some error page, but show them a nice message to check out the home page. You can use filetreesubs for this. Add the following to the configuration:

create_index_filename: index.html
create_index_content: |
  <!DOCTYPE html>
  <html>
    <head>
      <title>There's nothing to see here.</title>
      <meta http-equiv="refresh" content="10; url=..">
    </head>
    <body>
      There's nothing to see here. Go <a href="..">here</a> instead.
      You will be automatically redirected there in 10 seconds.
    </body>
  </html>

Then in every folder not containing a file index.html, a file index.html will be created with the specified content.

Configuration file format

The configuration file is in YAML format. By default, the configuration is assumed to be in filetreesubs-config.yaml in the current directory. If you want to specify a different configuration file name, you can simply specify it on the command line:

filetreesubs my-config-file.yaml

The following commented YAML file shows all available options:

# The source directory. Specify a path here.
source: input

# The destination directory. Specify a path here.
destination: output

# The substitutions to make
substitutes:
  # For every substitution, you need to specify a regex pattern
  # matching the file name. Use '.*' to match everything, and
  # '.*\.html' to match all files ending with '.html'.
  '.*':
    # Now you can specify a number of strings which shall be replaced
    'STRING TO REPLACE':
      # In this case, we want to replace the string by the contents
      # of the file menu.inc. Note that menu.inc won't be copied
      # to the destination directory anymore.
      file: menu.inc
    'ANOTHER_REPLACEMENT_STRING':
      # In this case, we want to replace the string by another string
      # we explicitly specify here.
      text: '(replacement text)'
  # Now we can specify more filename matching patterns ...
  '.*\.html':
    # ... and more replacements
    'YET_ANOTHER_STRING':
      text: '(some more)'

# To do substitutions in files like menu.inc, we need substitution
# chains.
substitute_chains:
# Each substitution chain consists of the name of the file to
# substitute in, like menu.inc:
- template: menu.inc
  # As well as a list of substitutions, using the same syntax as above:
  substitutes:
    # The string to replace:
    'INCLUDE_INCLUDE':
      # What to replace it with
      file: include.inc
    'INCLUDE_STRING':
      text: '...'
# You can have as many substitution chains as you want
- template: include.inc
  substitutes:
    'ONE_MORE':
      text: '(...)'

# To create index files (when not already existing), you must
# specify the name of these files:
create_index_filename: index.html

# This allows to specify the content of index files.
create_index_content: |
  <!DOCTYPE html>
  <html lang="en">
    <head>
      <title>there's nothing to see here.</title>
      <meta name="robots" content="noindex">
      <meta http-equiv="refresh" content="0; url=..">
    </head>
    <body style="background-color:black; color:white;">
      <div style="position:absolute; top:0; left:0; right:0; bottom:0;">
        <div style="width:100%; height:100%; display:table;">
            there's nothing to see here. go <a href=".." style="color:#AAA;">here</a> instead.
          </div>
        </div>
      </div>
    </body>
  </html>

# By default, filetreesubs assumes that all text files it processes
# are UTF-8 encoded. If that's not the case, you can change another
# encoding here.
encoding: utf-8

# In case you need to do so, you can insert configurations for doit
# directly here. See `here <http://pydoit.org/configuration.html#configuration-at-dodo-py>`__
# for possible configurations.
doit_config:
  # The following option sets the filename for the dependency database.
  # If you want to execute different filetreesubs commands concurrently
  # from a folder, you need to specify different dependency database
  # names per project config.
  dep_file: '.doit-myproject.db'

Release files for filetreesubs 1.2.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for filetreesubs 1.2.0
File	Size	Uploaded
filetreesubs-1.2.0.tar.gz	12.6 kB	May 7, 2023	Details

Built distribution (wheel)

Table of built distributions (wheels) for filetreesubs 1.2.0
File	Interpreter	ABI	Platform	Reset
filetreesubs-1.2.0-py3-none-any.whl 13.2 kB May 7, 2023	Python 3	none	any	Details

Total release size: 25.8 kB

Release files / filetreesubs-1.2.0.tar.gz

Download URL	filetreesubs-1.2.0.tar.gz
Size	12.6 kB
Tags	Source
SHA-256 checksum How to use checksums	`897a2ea2b46d1fa0524caf1080da175acf4688fda5949d28f91a6e9e732f7985`
BLAKE2b-256 checksum How to use checksums	`a3c38189944954e39bc62095204c466ad94ad964deb589efa8999366888afe32`
Upload date	May 7, 2023
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`python-httpx/0.23.3`

Release files / filetreesubs-1.2.0-py3-none-any.whl

Download URL	filetreesubs-1.2.0-py3-none-any.whl
Size	13.2 kB
Tags	Python 3
SHA-256 checksum How to use checksums	`4e1649d6ef4a767e103bfbce97ce4f136b02a13c0f176502ceb7c852f2fa59b8`
BLAKE2b-256 checksum How to use checksums	`c69f83a9ab50c6f9e0ff36e3f47c5f004473bdce2bcc5d58225f1c0f68e2931b`
Upload date	May 7, 2023
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`python-httpx/0.23.3`