InScript Lexer - Rust Implementation (v3.7.2)

Real, compilable Rust tokenizer for InScript language.

Features

✅ Zero-copy token references (fast, memory efficient) ✅ State machine lexer (no backtracking) ✅ 30+ token types (keywords, operators, literals) ✅ Accurate line/column tracking ✅ Comprehensive error reporting ✅ Python FFI via PyO3 ✅ Full test coverage

Build

cargo build --release

Token Types Supported

Literals

  • Numbers (integers and floats)
  • Strings (single and double quoted)
  • Identifiers

Keywords

  • if, else, while, for, fn, return
  • true, false, nil
  • and, or, not

Operators

  • Arithmetic: +, -, *, /, %
  • Comparison: ==, !=, <, <=, >, >=
  • Assignment: =
  • Logical: and, or, not

Delimiters

  • Parentheses: ( )
  • Braces: { }
  • Brackets: [ ]
  • Punctuation: , . ; : ->

Rust Usage

use inscript_lexer::Lexer;

fn main() {
    let code = "let x = 42;";
    let lexer = Lexer::new(code);
    
    for token in lexer {
        match token {
            Ok(t) => println!("{:?}: {}", t.token_type, t.value),
            Err(e) => eprintln!("Error: {}", e),
        }
    }
}

Python Usage (after compilation)

from inscript_lexer import PyLexer, tokenize_string

# Method 1: Using PyLexer class
lexer = PyLexer("let x = 42;")
tokens = lexer.tokens()
print(f"Found {lexer.count()} tokens")

# Method 2: Direct tokenization
tokens = tokenize_string("if true then x else y")
for token in tokens:
    print(f"{token.token_type}: {token.value}")

Performance

Expected Benchmarks

  • Simple tokens: < 100ns per token
  • Complex code: < 10µs per 100 tokens
  • Memory: O(1) per token

Optimizations

  • Inline hot paths
  • No allocations in common cases
  • SIMD-friendly string scanning
  • Zero-copy string references

Testing

# Run tests
cargo test

# Run with output
cargo test -- --nocapture

# Benchmark
cargo bench

Accuracy

✅ Correct line/column tracking ✅ Handles escape sequences ✅ Detects unterminated strings ✅ Error recovery

License

MIT

Project links

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Newer release available (3.9.6.45)

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for Souvik_Sarkar from gravatar.com Souvik_Sarkar

Credits

Author: Shreyasi Sarkar

License

MIT License (MIT)

Requires

Python >=3.10

Provides Extra

game lsp all

Tags

game scripting language gamedev gdscript

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

inscript_lang-3.9.6.43.tar.gz (337.1 kB view details)

Uploaded Source

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

inscript_lang-3.9.6.43-cp312-cp312-win_amd64.whl (690.7 kB view details)

Uploaded CPython 3.12Windows x86-64

inscript_lang-3.9.6.43-cp312-cp312-manylinux_2_28_x86_64.whl (774.6 kB view details)

Uploaded CPython 3.12manylinux: glibc 2.28+ x86-64

inscript_lang-3.9.6.43-cp312-cp312-manylinux_2_28_aarch64.whl (742.9 kB view details)

Uploaded CPython 3.12manylinux: glibc 2.28+ ARM64

inscript_lang-3.9.6.43-cp312-cp312-macosx_10_13_universal2.whl (1.1 MB view details)

Uploaded CPython 3.12macOS 10.13+ universal2 (ARM64, x86-64)

inscript_lang-3.9.6.43-cp311-cp311-win_amd64.whl (687.2 kB view details)

Uploaded CPython 3.11Windows x86-64

inscript_lang-3.9.6.43-cp311-cp311-manylinux_2_28_x86_64.whl (771.4 kB view details)

Uploaded CPython 3.11manylinux: glibc 2.28+ x86-64

inscript_lang-3.9.6.43-cp311-cp311-manylinux_2_28_aarch64.whl (741.5 kB view details)

Uploaded CPython 3.11manylinux: glibc 2.28+ ARM64

inscript_lang-3.9.6.43-cp311-cp311-macosx_10_12_universal2.whl (1.1 MB view details)

Uploaded CPython 3.11macOS 10.12+ universal2 (ARM64, x86-64)

inscript_lang-3.9.6.43-cp310-cp310-win_amd64.whl (687.4 kB view details)

Uploaded CPython 3.10Windows x86-64

inscript_lang-3.9.6.43-cp310-cp310-manylinux_2_28_x86_64.whl (771.8 kB view details)

Uploaded CPython 3.10manylinux: glibc 2.28+ x86-64

inscript_lang-3.9.6.43-cp310-cp310-manylinux_2_28_aarch64.whl (741.8 kB view details)

Uploaded CPython 3.10manylinux: glibc 2.28+ ARM64

inscript_lang-3.9.6.43-cp310-cp310-macosx_10_12_universal2.whl (1.1 MB view details)

Uploaded CPython 3.10macOS 10.12+ universal2 (ARM64, x86-64)

File details

Details for the file inscript_lang-3.9.6.43.tar.gz.

File metadata

  • Download URL: inscript_lang-3.9.6.43.tar.gz
  • Upload date:
  • Size: 337.1 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.1.0 CPython/3.13.12

File hashes

Hashes for inscript_lang-3.9.6.43.tar.gz
Algorithm Hash digest
SHA256 e2b6b9d5a76eae9ae351b097fd4b137c0343b75e7fb641bc8c91f1c6853b54e9
MD5 12b2d6a3f9742a7ba1df335f5f67f50a
BLAKE2b-256 35638a0ef8bba9158b22a8b663d2c910bf5f6b46e14d2cfa88c1e7834e986408

See more details on using hashes here.

File details

Details for the file inscript_lang-3.9.6.43-cp312-cp312-win_amd64.whl.

File metadata

File hashes

Hashes for inscript_lang-3.9.6.43-cp312-cp312-win_amd64.whl
Algorithm Hash digest
SHA256 9e5ea3b0c2e445d5465250462cbc3debdd5dcec3e4fddcf3e8c714ea546b7a20
MD5 620d7d37e9b970b59886de9b91e27eb1
BLAKE2b-256 2c9a0037690d07a64eed924dfa41b39ee5a2b3893273a85891bf49fd960490f9

See more details on using hashes here.

File details

Details for the file inscript_lang-3.9.6.43-cp312-cp312-manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for inscript_lang-3.9.6.43-cp312-cp312-manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 a6a717fac0fda8a77c8e78ab46e77cbb2e91a213dbfdbfc8d06ad6938fe21a8a
MD5 22fa46c96ef52d0d6f3779a0095bdbe4
BLAKE2b-256 54d0d4c47274a7d424e0f147ea22dae8ec57666e3a89bb2061a73292b9115832

See more details on using hashes here.

File details

Details for the file inscript_lang-3.9.6.43-cp312-cp312-manylinux_2_28_aarch64.whl.

File metadata

File hashes

Hashes for inscript_lang-3.9.6.43-cp312-cp312-manylinux_2_28_aarch64.whl
Algorithm Hash digest
SHA256 3133208149dc1a930f5ce642b747d964ca5da07b20bb444373b9ae8750ad34d9
MD5 cf490c88249c61d48196ba01dd7a88e1
BLAKE2b-256 38db33e7df9c78fac966c0d53202a3a720a244be023ad65c3f5761ca47df344d

See more details on using hashes here.

File details

Details for the file inscript_lang-3.9.6.43-cp312-cp312-macosx_10_13_universal2.whl.

File metadata

File hashes

Hashes for inscript_lang-3.9.6.43-cp312-cp312-macosx_10_13_universal2.whl
Algorithm Hash digest
SHA256 92f7347a44fa269b7132e8145891bebcb5cefe926a32d1de0f4080efc7313c8e
MD5 448c503cc50f10b4f2642d718542a0db
BLAKE2b-256 baf9ee82d416f7a204c537ec3a90ec18fae0353e8a5d44c2831f5befa9429b1d

See more details on using hashes here.

File details

Details for the file inscript_lang-3.9.6.43-cp311-cp311-win_amd64.whl.

File metadata

File hashes

Hashes for inscript_lang-3.9.6.43-cp311-cp311-win_amd64.whl
Algorithm Hash digest
SHA256 41592b3460986092847a95dad545d623d4fb4e6667e59d0724ad241ef96ac944
MD5 ba2a896ba642906d2263e5e756b28d7c
BLAKE2b-256 5b0b39810b60cfee0a2a5940b64cc8a4ddca167b1b5666aface677fe7e2d885d

See more details on using hashes here.

File details

Details for the file inscript_lang-3.9.6.43-cp311-cp311-manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for inscript_lang-3.9.6.43-cp311-cp311-manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 447df75da909e1ee8876b5e6a1b8500bfeca03162517cb7a4b2ce5370112a923
MD5 7551293d5587cc0409955e76d7ae61c1
BLAKE2b-256 48b3a127851680b6537b4cf44e2e71db1cebc0552d645f8bd60e87c407e16969

See more details on using hashes here.

File details

Details for the file inscript_lang-3.9.6.43-cp311-cp311-manylinux_2_28_aarch64.whl.

File metadata

File hashes

Hashes for inscript_lang-3.9.6.43-cp311-cp311-manylinux_2_28_aarch64.whl
Algorithm Hash digest
SHA256 b2c34ee30a7d76510e4d20ed537172af7f909935e2ca8ee392af30425a7b8686
MD5 36788e92eb765cf9a34f18764164a41b
BLAKE2b-256 70d7eefcfe10cc2cf95629bb4c289acb5f4965778472ce96b4710e7cf64ac863

See more details on using hashes here.

File details

Details for the file inscript_lang-3.9.6.43-cp311-cp311-macosx_10_12_universal2.whl.

File metadata

File hashes

Hashes for inscript_lang-3.9.6.43-cp311-cp311-macosx_10_12_universal2.whl
Algorithm Hash digest
SHA256 f04e8c4b04c05407ca3bb10c6d954a039f72ad6adcb47bab8953bbfb88d72044
MD5 8219568e427797c203c1685a450ea4e4
BLAKE2b-256 9e3b8b0a3408bcfbf14762bd99546dc72e9b70d9561d6608bde199b22da1f603

See more details on using hashes here.

File details

Details for the file inscript_lang-3.9.6.43-cp310-cp310-win_amd64.whl.

File metadata

File hashes

Hashes for inscript_lang-3.9.6.43-cp310-cp310-win_amd64.whl
Algorithm Hash digest
SHA256 cdfb04b4706dbfb4cc1d8691bc1576a3b98bda343cab2cede5098e141168790b
MD5 1f231ef20445ae0cc574f2a22f25854d
BLAKE2b-256 aaa0cda5575803cbe38157c98dceb610a5779694961b508a1679b916d193aac3

See more details on using hashes here.

File details

Details for the file inscript_lang-3.9.6.43-cp310-cp310-manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for inscript_lang-3.9.6.43-cp310-cp310-manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 5889242f8e7cfaed717ed83490050396be743eb839e5e3c596456651cac96218
MD5 7f0a91f017f622196b8edba4e5d0c688
BLAKE2b-256 525e6c63d3ba4b68ba2e68fc0bdbc8f854de466e79e49ff32aeccd59837ee671

See more details on using hashes here.

File details

Details for the file inscript_lang-3.9.6.43-cp310-cp310-manylinux_2_28_aarch64.whl.

File metadata

File hashes

Hashes for inscript_lang-3.9.6.43-cp310-cp310-manylinux_2_28_aarch64.whl
Algorithm Hash digest
SHA256 0451c5508c38f77b354bf0fc28b252a0b1c42503e3309d50ca926bcca159de09
MD5 0ba395433a5422dedf4ba57108b97374
BLAKE2b-256 016c0a17b462997da33758a4168ec4ba2f343ef310a80ccb20eeb98aaa622712

See more details on using hashes here.

File details

Details for the file inscript_lang-3.9.6.43-cp310-cp310-macosx_10_12_universal2.whl.

File metadata

File hashes

Hashes for inscript_lang-3.9.6.43-cp310-cp310-macosx_10_12_universal2.whl
Algorithm Hash digest
SHA256 79aa4e2fc1d7df88ecbf1fc7310cc79cb186d270a3b40936cf774a27702a02ab
MD5 93a98e076ff6233fa444d53981dffc4b
BLAKE2b-256 976e9b37418e8d7c9942a1ada27273aaf8eddcaf0622a360766642aef5cc3494

See more details on using hashes here.

Release history Release notifications | RSS feed

3.9.6.45

13 files

3.9.6.44

13 files

This release

3.9.6.43 This release

13 files

3.9.6.39

13 files

3.9.6.38

13 files

3.9.6.24

13 files

3.9.6.23

13 files

3.9.6.10.16

13 files

3.0.0

2 files

2.13.0

2 files

2.12.0

2 files

2.11.0

2 files

2.10.0

2 files

2.8.0

2 files

2.5.0

2 files

2.4.0

2 files

2.3.0

2 files

2.2.0

2 files

2.1.2

2 files

2.1.1

2 files

2.1.0

2 files

2.0.3

2 files

2.0.2

2 files

2.0.1

2 files

2.0.0

2 files

1.9.14

2 files

1.9.13

2 files

1.9.12

2 files

1.9.11

2 files

1.9.10

2 files

1.9.9

2 files

1.9.8

2 files

1.9.7

2 files

1.9.6

2 files

1.9.5

2 files

1.9.4

2 files

1.9.2

2 files

1.8.4

2 files

1.8.3

2 files

1.8.2

2 files

1.8.1

2 files

1.7.4

2 files

1.7.3

2 files

1.7.1

2 files

1.6.0

2 files

1.1.0

2 files

1.0.23

2 files

1.0.0

2 files

0.6.0

2 files

0.5.0

1 file

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page