InScript Lexer - Rust Implementation (v3.7.2)

Real, compilable Rust tokenizer for InScript language.

Features

✅ Zero-copy token references (fast, memory efficient) ✅ State machine lexer (no backtracking) ✅ 30+ token types (keywords, operators, literals) ✅ Accurate line/column tracking ✅ Comprehensive error reporting ✅ Python FFI via PyO3 ✅ Full test coverage

Build

cargo build --release

Token Types Supported

Literals

  • Numbers (integers and floats)
  • Strings (single and double quoted)
  • Identifiers

Keywords

  • if, else, while, for, fn, return
  • true, false, nil
  • and, or, not

Operators

  • Arithmetic: +, -, *, /, %
  • Comparison: ==, !=, <, <=, >, >=
  • Assignment: =
  • Logical: and, or, not

Delimiters

  • Parentheses: ( )
  • Braces: { }
  • Brackets: [ ]
  • Punctuation: , . ; : ->

Rust Usage

use inscript_lexer::Lexer;

fn main() {
    let code = "let x = 42;";
    let lexer = Lexer::new(code);
    
    for token in lexer {
        match token {
            Ok(t) => println!("{:?}: {}", t.token_type, t.value),
            Err(e) => eprintln!("Error: {}", e),
        }
    }
}

Python Usage (after compilation)

from inscript_lexer import PyLexer, tokenize_string

# Method 1: Using PyLexer class
lexer = PyLexer("let x = 42;")
tokens = lexer.tokens()
print(f"Found {lexer.count()} tokens")

# Method 2: Direct tokenization
tokens = tokenize_string("if true then x else y")
for token in tokens:
    print(f"{token.token_type}: {token.value}")

Performance

Expected Benchmarks

  • Simple tokens: < 100ns per token
  • Complex code: < 10µs per 100 tokens
  • Memory: O(1) per token

Optimizations

  • Inline hot paths
  • No allocations in common cases
  • SIMD-friendly string scanning
  • Zero-copy string references

Testing

# Run tests
cargo test

# Run with output
cargo test -- --nocapture

# Benchmark
cargo bench

Accuracy

✅ Correct line/column tracking ✅ Handles escape sequences ✅ Detects unterminated strings ✅ Error recovery

License

MIT

Project links

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Newer release available (3.9.6.45)

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for Souvik_Sarkar from gravatar.com Souvik_Sarkar

Credits

Author: Shreyasi Sarkar

License

MIT License (MIT)

Requires

Python >=3.10

Provides Extra

game lsp all

Tags

game scripting language gamedev gdscript

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

inscript_lang-3.9.6.44.tar.gz (337.8 kB view details)

Uploaded Source

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

inscript_lang-3.9.6.44-cp312-cp312-win_amd64.whl (691.4 kB view details)

Uploaded CPython 3.12Windows x86-64

inscript_lang-3.9.6.44-cp312-cp312-manylinux_2_28_x86_64.whl (775.3 kB view details)

Uploaded CPython 3.12manylinux: glibc 2.28+ x86-64

inscript_lang-3.9.6.44-cp312-cp312-manylinux_2_28_aarch64.whl (743.5 kB view details)

Uploaded CPython 3.12manylinux: glibc 2.28+ ARM64

inscript_lang-3.9.6.44-cp312-cp312-macosx_10_13_universal2.whl (1.1 MB view details)

Uploaded CPython 3.12macOS 10.13+ universal2 (ARM64, x86-64)

inscript_lang-3.9.6.44-cp311-cp311-win_amd64.whl (687.8 kB view details)

Uploaded CPython 3.11Windows x86-64

inscript_lang-3.9.6.44-cp311-cp311-manylinux_2_28_x86_64.whl (772.1 kB view details)

Uploaded CPython 3.11manylinux: glibc 2.28+ x86-64

inscript_lang-3.9.6.44-cp311-cp311-manylinux_2_28_aarch64.whl (742.1 kB view details)

Uploaded CPython 3.11manylinux: glibc 2.28+ ARM64

inscript_lang-3.9.6.44-cp311-cp311-macosx_10_12_universal2.whl (1.1 MB view details)

Uploaded CPython 3.11macOS 10.12+ universal2 (ARM64, x86-64)

inscript_lang-3.9.6.44-cp310-cp310-win_amd64.whl (688.0 kB view details)

Uploaded CPython 3.10Windows x86-64

inscript_lang-3.9.6.44-cp310-cp310-manylinux_2_28_x86_64.whl (772.4 kB view details)

Uploaded CPython 3.10manylinux: glibc 2.28+ x86-64

inscript_lang-3.9.6.44-cp310-cp310-manylinux_2_28_aarch64.whl (742.4 kB view details)

Uploaded CPython 3.10manylinux: glibc 2.28+ ARM64

inscript_lang-3.9.6.44-cp310-cp310-macosx_10_12_universal2.whl (1.1 MB view details)

Uploaded CPython 3.10macOS 10.12+ universal2 (ARM64, x86-64)

File details

Details for the file inscript_lang-3.9.6.44.tar.gz.

File metadata

  • Download URL: inscript_lang-3.9.6.44.tar.gz
  • Upload date:
  • Size: 337.8 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.1.0 CPython/3.13.12

File hashes

Hashes for inscript_lang-3.9.6.44.tar.gz
Algorithm Hash digest
SHA256 ca9080a062a375355ff5004c30d4f3ccdb4d7ef29268c7424f7f16dd9a019ccd
MD5 b3d7ceaf2b97bbbe5bb99bf1f6d751e6
BLAKE2b-256 9657c8015fb8f388c7fff63eba3c6c072708f1950793daeb2655555688968d58

See more details on using hashes here.

File details

Details for the file inscript_lang-3.9.6.44-cp312-cp312-win_amd64.whl.

File metadata

File hashes

Hashes for inscript_lang-3.9.6.44-cp312-cp312-win_amd64.whl
Algorithm Hash digest
SHA256 e5a7dc090edc903262c6e3bd5eccc24a1818ac211ed4466649ff121016d126af
MD5 6bb283b3574589cfc7a2ae622b76bad9
BLAKE2b-256 cd088173c01708244d37378132a493084aefb0612c360ec9e009ea789f326ad8

See more details on using hashes here.

File details

Details for the file inscript_lang-3.9.6.44-cp312-cp312-manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for inscript_lang-3.9.6.44-cp312-cp312-manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 534c1a997f12b68a81420705ef1068e26c1948809f89c9e68492436dfaa2b80d
MD5 dd8797bf57d284321db6cc2c79d05531
BLAKE2b-256 32fcd76987ea0f47d4094e7921baf1b3895283de3443291054c608a944861683

See more details on using hashes here.

File details

Details for the file inscript_lang-3.9.6.44-cp312-cp312-manylinux_2_28_aarch64.whl.

File metadata

File hashes

Hashes for inscript_lang-3.9.6.44-cp312-cp312-manylinux_2_28_aarch64.whl
Algorithm Hash digest
SHA256 ccaeee0409ed080c3168dd97c88ced3f251ecd2b832fa6fefb5874b5e9fa10d4
MD5 336e061e49afeff9d00dddabb0d11a7d
BLAKE2b-256 753832ba750b396cd5b51be70d6bd0abf6662e804257185f1a3fcdadf8c3e5f1

See more details on using hashes here.

File details

Details for the file inscript_lang-3.9.6.44-cp312-cp312-macosx_10_13_universal2.whl.

File metadata

File hashes

Hashes for inscript_lang-3.9.6.44-cp312-cp312-macosx_10_13_universal2.whl
Algorithm Hash digest
SHA256 f85028505fe328ef621c88a75ae61a69140e98b256f321fb10bf4fcdb92051ee
MD5 bcda6aa3bd8f4432b523cbfa860aed60
BLAKE2b-256 bfe5136c45b77e618514c4cab2239d38dd275b8b4f46060dcd7e8d0e510a1bec

See more details on using hashes here.

File details

Details for the file inscript_lang-3.9.6.44-cp311-cp311-win_amd64.whl.

File metadata

File hashes

Hashes for inscript_lang-3.9.6.44-cp311-cp311-win_amd64.whl
Algorithm Hash digest
SHA256 9807b2cfad52b6ad7fe5b4e2cd4d8d832ac5c54ebe823696f6cd79bea1ae6023
MD5 2c03e7fd888a4cb9e19268c6316721c5
BLAKE2b-256 0ac35f07cd1015d2b9cc524b7dc6919b392234acfa0a8013bb7b0a81bc7f6f32

See more details on using hashes here.

File details

Details for the file inscript_lang-3.9.6.44-cp311-cp311-manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for inscript_lang-3.9.6.44-cp311-cp311-manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 3e9021a8da3c0c40470870cedb76f03fc30c1eaa9b6414fdeb6a0f65fa9adff3
MD5 cec85307f14c075748dc6734921ea6a8
BLAKE2b-256 c9d4d27d8a53aea863b0f8b0b6482560412c734f95d23c4eae871052193e492f

See more details on using hashes here.

File details

Details for the file inscript_lang-3.9.6.44-cp311-cp311-manylinux_2_28_aarch64.whl.

File metadata

File hashes

Hashes for inscript_lang-3.9.6.44-cp311-cp311-manylinux_2_28_aarch64.whl
Algorithm Hash digest
SHA256 bf4af59e00642b1a1e80f634fa5632a0771856f4c154243828275da31d1f8f81
MD5 eea02b93551ee2941d66673b0c98ee23
BLAKE2b-256 80c5483cd71ee17364a7b13c63f890bb6e285e34c498e5b20ac280995f21b839

See more details on using hashes here.

File details

Details for the file inscript_lang-3.9.6.44-cp311-cp311-macosx_10_12_universal2.whl.

File metadata

File hashes

Hashes for inscript_lang-3.9.6.44-cp311-cp311-macosx_10_12_universal2.whl
Algorithm Hash digest
SHA256 9bef16b9817377fbc1317924899e651a14abd959573d5bf88c3652fde7190e0e
MD5 845108cf48fbfe8ed5c447e2f17d32da
BLAKE2b-256 0cb7ea43630cee881a2ed403cd5c55226ef3271a5ccba538a73c6804b54b14bf

See more details on using hashes here.

File details

Details for the file inscript_lang-3.9.6.44-cp310-cp310-win_amd64.whl.

File metadata

File hashes

Hashes for inscript_lang-3.9.6.44-cp310-cp310-win_amd64.whl
Algorithm Hash digest
SHA256 97b44f9dcb444f093d494463f6b23562c78f62946b2277f43b7922d35e6ac081
MD5 c5221b888c84239b7e35b2c0df40a34e
BLAKE2b-256 a4f3d1630f83985de29ba7a5df0a4bcd9e57c8fe90531cc4ef36d302c0d65204

See more details on using hashes here.

File details

Details for the file inscript_lang-3.9.6.44-cp310-cp310-manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for inscript_lang-3.9.6.44-cp310-cp310-manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 0f1f730c203fdd4a8ca2a7269185524c506771e4fcfc238357b503f8c09776c5
MD5 2ff988800320e1ded32b8244bca6575c
BLAKE2b-256 83462ba0fb9c14a39de468c2fc7f902568c1fe5f4d42f6f47ef178b6cbd61dd8

See more details on using hashes here.

File details

Details for the file inscript_lang-3.9.6.44-cp310-cp310-manylinux_2_28_aarch64.whl.

File metadata

File hashes

Hashes for inscript_lang-3.9.6.44-cp310-cp310-manylinux_2_28_aarch64.whl
Algorithm Hash digest
SHA256 3589d89ab372546e18cd8d382e6932e8517ba46b75ab01bdd7e63da2d8c83605
MD5 8e0c965fc9d4814a2fd1625e0589320e
BLAKE2b-256 20a3859f147cd14bf3c787e2f08abab4dd0b4ec121995dcd1a621c22d801ec88

See more details on using hashes here.

File details

Details for the file inscript_lang-3.9.6.44-cp310-cp310-macosx_10_12_universal2.whl.

File metadata

File hashes

Hashes for inscript_lang-3.9.6.44-cp310-cp310-macosx_10_12_universal2.whl
Algorithm Hash digest
SHA256 3eede99ba43f0646f5cff1a4bf3d4a413633a9f98d0a466fcc90f3842d2e7f99
MD5 2eeee996ee1b2f188af9468a89f4f7e5
BLAKE2b-256 4c1f79c5958d2f3c908fd3fac6a3198a394af2238f4dcc9ba0453c5c7b082db6

See more details on using hashes here.

Release history Release notifications | RSS feed

3.9.6.45

13 files

This release

3.9.6.44 This release

13 files

3.9.6.43

13 files

3.9.6.39

13 files

3.9.6.38

13 files

3.9.6.24

13 files

3.9.6.23

13 files

3.9.6.10.16

13 files

3.0.0

2 files

2.13.0

2 files

2.12.0

2 files

2.11.0

2 files

2.10.0

2 files

2.8.0

2 files

2.5.0

2 files

2.4.0

2 files

2.3.0

2 files

2.2.0

2 files

2.1.2

2 files

2.1.1

2 files

2.1.0

2 files

2.0.3

2 files

2.0.2

2 files

2.0.1

2 files

2.0.0

2 files

1.9.14

2 files

1.9.13

2 files

1.9.12

2 files

1.9.11

2 files

1.9.10

2 files

1.9.9

2 files

1.9.8

2 files

1.9.7

2 files

1.9.6

2 files

1.9.5

2 files

1.9.4

2 files

1.9.2

2 files

1.8.4

2 files

1.8.3

2 files

1.8.2

2 files

1.8.1

2 files

1.7.4

2 files

1.7.3

2 files

1.7.1

2 files

1.6.0

2 files

1.1.0

2 files

1.0.23

2 files

1.0.0

2 files

0.6.0

2 files

0.5.0

1 file

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page