py-tree-sitter

This module provides Python bindings to the tree-sitter parsing library.

Installation

This package currently only works with Python 3. There are no library dependencies, but you do need to have a C compiler installed.

pip3 install tree_sitter

Usage

Setup

First you'll need a Tree-sitter language implementation for each language that you want to parse. You can clone some of the existing language repos or create your own:

git clone https://github.com/tree-sitter/tree-sitter-go
git clone https://github.com/tree-sitter/tree-sitter-javascript
git clone https://github.com/tree-sitter/tree-sitter-python

Use the Language.build_library method to compile these into a library that's usable from Python. This function will return immediately if the library has already been compiled since the last time its source code was modified:

from tree_sitter import Language, Parser

Language.build_library(
  # Store the library in the `build` directory
  'build/my-languages.so',

  # Include one or more languages
  [
    'vendor/tree-sitter-go',
    'vendor/tree-sitter-javascript',
    'vendor/tree-sitter-python'
  ]
)

Load the languages into your app as Language objects:

GO_LANGUAGE = Language('build/my-languages.so', 'go')
JS_LANGUAGE = Language('build/my-languages.so', 'javascript')
PY_LANGUAGE = Language('build/my-languages.so', 'python')

Basic Parsing

Create a Parser and configure it to use one of the languages:

parser = Parser()
parser.set_language(PY_LANGUAGE)

Parse some source code:

tree = parser.parse(bytes("""
def foo():
    if bar:
        baz()
""", "utf8"))

Inspect the resulting Tree:

root_node = tree.root_node
assert root_node.type == 'module'
assert root_node.start_point == (1, 0)
assert root_node.end_point == (3, 13)

function_node = root_node.children[0]
assert function_node.type == 'function_definition'
assert function_node.child_by_field_name('name').type == 'identifier'

function_name_node = function_node.children[1]
assert function_name_node.type == 'identifier'
assert function_name_node.start_point == (1, 4)
assert function_name_node.end_point == (1, 7)

assert root_node.sexp() == "(module "
    "(function_definition "
        "name: (identifier) "
        "parameters: (parameters) "
        "body: (block "
            "(if_statement "
                "condition: (identifier) "
                "consequence: (block "
                    "(expression_statement (call "
                        "function: (identifier) "
                        "arguments: (argument_list))))))))"

Walking Syntax Trees

If you need to traverse a large number of nodes efficiently, you can use a TreeCursor:

cursor = tree.walk()

assert cursor.node.type == 'module'

assert cursor.goto_first_child()
assert cursor.node.type == 'function_definition'

assert cursor.goto_first_child()
assert cursor.node.type == 'def'

# Returns `False` because the `def` node has no children
assert not cursor.goto_first_child()

assert cursor.goto_next_sibling()
assert cursor.node.type == 'identifier'

assert cursor.goto_next_sibling()
assert cursor.node.type == 'parameters'

assert cursor.goto_parent()
assert cursor.node.type == 'function_definition'

Editing

When a source file is edited, you can edit the syntax tree to keep it in sync with the source:

tree.edit(
    start_byte=5,
    old_end_byte=5,
    new_end_byte=5 + 2,
    start_point=(0, 5),
    old_end_point=(0, 5),
    new_end_point=(0, 5 + 2),
)

Then, when you're ready to incorporate the changes into a new syntax tree, you can call Parser.parse again, but pass in the old tree:

new_tree = parser.parse(new_source, tree)

This will run much faster than if you were parsing from scratch.

Pattern-matching

You can search for patterns in a syntax tree using a tree query:

query = PY_LANGUAGE.query("""
(function_definition
  name: (identifier) @function.def)

(call
  function: (identifier) @function.call)
""")

captures = query.captures(tree.root_node)
assert len(captures) == 2
assert captures[0][0] == function_name_node
assert captures[0][1] == "function.def"

Release files for tree-sitter 0.1.1

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for tree-sitter 0.1.1
File	Size	Uploaded
tree_sitter-0.1.1.tar.gz	93.7 kB	May 4, 2020	Details

Built distribution (wheel)

Table of built distributions (wheels) for tree-sitter 0.1.1
File	Interpreter	ABI	Platform	Reset
tree_sitter-0.1.1-cp37-cp37m-macosx_10_14_x86_64.whl 96.9 kB May 4, 2020	CPython 3.7	CPython 3.7 pymalloc	macOS 10.14+ x86-64	Details

Total release size: 190.5 kB

Release files / tree_sitter-0.1.1.tar.gz

Download URL	tree_sitter-0.1.1.tar.gz
Size	93.7 kB
Tags	Source
SHA-256 checksum How to use checksums	`704edf00b83522fecbcf7957e84c452848aaa0bc5f6575ca9721f43ce835e834`
BLAKE2b-256 checksum How to use checksums	`a6510fa7c5e7951d2b01bcd4dcaa8a52780741a058bba92c660329fc8403799d`
Upload date	May 4, 2020
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`twine/1.13.0 pkginfo/1.5.0.1 requests/2.21.0 setuptools/45.1.0 requests-toolbelt/0.9.1 tqdm/4.31.1 CPython/3.7.6`

Release files / tree_sitter-0.1.1-cp37-cp37m-macosx_10_14_x86_64.whl

Download URL	tree_sitter-0.1.1-cp37-cp37m-macosx_10_14_x86_64.whl
Size	96.9 kB
Tags	CPython 3.7 CPython 3.7 pymalloc macOS 10.14+ x86-64
SHA-256 checksum How to use checksums	`14e556f9369445d27ae0e6d678299a3f0f5a54dc0b75c365d7beb43c183d95bc`
BLAKE2b-256 checksum How to use checksums	`5c02d293b2652294b5434df644b2fd5bf47bab5589b5c4a60709384108db2447`
Upload date	May 4, 2020
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`twine/1.13.0 pkginfo/1.5.0.1 requests/2.21.0 setuptools/45.1.0 requests-toolbelt/0.9.1 tqdm/4.31.1 CPython/3.7.6`