fmtree 0.3.2

Scrape File System and output in different formats

fmtree

Documentation

https://fmtree-dev.github.io/fmtree/

Sample Output

GithubMarkdownContentFormatter

- OSCP
- [Notes](./Notes)
  - Tools
    - [Python](./Notes/Tools/Python.md)
    - [nmap](./Notes/Tools/nmap.md)
    - [Netcat](./Notes/Tools/Netcat.md)
    - [Metasploit](./Notes/Tools/Metasploit.md)
  - [common](./Notes/common.md)
  - [FileTransfer](./Notes/FileTransfer.md)
  - [Service](./Notes/Service.md)
  - [Bash](./Notes/Bash.md)

Render in MarkDown

• OSCP
  • Notes
    • Tools
      • Python
      • nmap
      • Netcat
      • Metasploit
    • common
    • FileTransfer
    • Service
    • Bash

TreeCommandFormatter

OSCP
└── Notes
    ├── Tools
    │   ├── Python.md
    │   ├── nmap.md
    │   ├── Netcat.md
    │   └── Metasploit.md
    ├── common.md
    ├── FileTransfer.md
    ├── README.md
    ├── Service.md
    └── Bash.md

Sample Code

import sys
import pathlib2
from fmtree.core.scraper import Scraper
from fmtree.core.format import TreeCommandFormatter, GithubMarkdownContentFormatter
from fmtree.core.filter import MarkdownFilter
from fmtree.core.sorter import Sorter


path_ = pathlib2.Path('/OSCP')
scraper = Scraper(path_, scrape_now=False, keep_empty_dir=False)

# add filter
scraper.add_filter(filter_=MarkdownFilter())

# run scraper
scraper.run()

# GNU Tree Format
formatter = TreeCommandFormatter(scraper.get_tree())
stringio = formatter.generate()
print(stringio.getvalue())

# sort
sorter_ = Sorter()
tree = sorter_(scraper.get_tree())

# GitHub Content Format
formatter = GithubMarkdownContentFormatter(tree)
stringio = formatter.generate()
print(stringio.getvalue())
formatter.to_stream(sys.stdout)

Visualizer

fmtree.visualizer.visualize

A command line one-liner to visualize a directory like gnu tree

python -m fmtree.visualizer.visualize -h                                                      
usage: fmtree visualizer argument parser [-h] [--debug] [-i INPUT] [--stdout] [--stderr]
                                         [-o OUTPUT] [--tree] [--markdown] [--html]
                                         [--ext EXT [EXT ...]] [-d DEPTH]

optional arguments:
  -h, --help            show this help message and exit
  --debug               debug mode
  -i INPUT, --input INPUT
                        input path (path to format)
  --stdout              output to stdout
  --stderr              output to stderr
  -o OUTPUT, --output OUTPUT
                        output file path
  --tree                nu tree style output
  --markdown            markdown style output
  --html                html list style output
  --ext EXT [EXT ...]
  -d DEPTH, --depth DEPTH
                        Directory depth to parse

Sample Usage

python -m fmtree.visualizer.visualize -i '/home/user/images' --depth 5 --ext .jpg .png --stdout --debug --html

fmtree.visualizer.image_dir

A command line one-liner that produce a html for visualizing an image directory.

It produces a html file that display all images in a nested image directory.

Just open the html file in a browser.

python -m visualizer.image_dir --help          
usage: Fmtree Visualizer Parser [-h] [-o OUTPUT] [-q]
                                [-d DEPTH] [--cdn]
                                [--show_all]
                                input

positional arguments:
  input                 input path

optional arguments:
  -h, --help            show this help message and exit
  -o OUTPUT, --output OUTPUT
                        output directory to save html
  -q, --quiet           whether to print out the
                        directory
  -d DEPTH, --depth DEPTH
                        Directory depth to parse
  --cdn                 Use CDN for libraries, requires
                        internet access, minimize html
                        size
  --show_all            Show All Images By Default

Sample Usage

python -m fmtree.visualizer.image_dir --cdn /home/user/images

Inspiration

This is a one-liner bash script that puts all images into an html, but with lots of limitations:

• no styling (width problem)
• doesn't support nested image directory
• not interactive (cannot collapse image etc.)

for file in $(ls);do echo "<img src=\"./$file\"/>" >> index.html; done

Image

Duplicate Image Search

This is a module used to detect duplicate images, and optionally export the data, remove duplicates or extract unique images.

The python script is in ./fmtree/image/dup.py

Usage

python -m fmtree.image.dup ~/Downloads/images --output_dir ./out --remove_inplace --output_json dup.json -m perceptual-hash