InScript Lexer - Rust Implementation (v3.7.2)

Real, compilable Rust tokenizer for InScript language.

Features

✅ Zero-copy token references (fast, memory efficient) ✅ State machine lexer (no backtracking) ✅ 30+ token types (keywords, operators, literals) ✅ Accurate line/column tracking ✅ Comprehensive error reporting ✅ Python FFI via PyO3 ✅ Full test coverage

Build

cargo build --release

Token Types Supported

Literals

  • Numbers (integers and floats)
  • Strings (single and double quoted)
  • Identifiers

Keywords

  • if, else, while, for, fn, return
  • true, false, nil
  • and, or, not

Operators

  • Arithmetic: +, -, *, /, %
  • Comparison: ==, !=, <, <=, >, >=
  • Assignment: =
  • Logical: and, or, not

Delimiters

  • Parentheses: ( )
  • Braces: { }
  • Brackets: [ ]
  • Punctuation: , . ; : ->

Rust Usage

use inscript_lexer::Lexer;

fn main() {
    let code = "let x = 42;";
    let lexer = Lexer::new(code);
    
    for token in lexer {
        match token {
            Ok(t) => println!("{:?}: {}", t.token_type, t.value),
            Err(e) => eprintln!("Error: {}", e),
        }
    }
}

Python Usage (after compilation)

from inscript_lexer import PyLexer, tokenize_string

# Method 1: Using PyLexer class
lexer = PyLexer("let x = 42;")
tokens = lexer.tokens()
print(f"Found {lexer.count()} tokens")

# Method 2: Direct tokenization
tokens = tokenize_string("if true then x else y")
for token in tokens:
    print(f"{token.token_type}: {token.value}")

Performance

Expected Benchmarks

  • Simple tokens: < 100ns per token
  • Complex code: < 10µs per 100 tokens
  • Memory: O(1) per token

Optimizations

  • Inline hot paths
  • No allocations in common cases
  • SIMD-friendly string scanning
  • Zero-copy string references

Testing

# Run tests
cargo test

# Run with output
cargo test -- --nocapture

# Benchmark
cargo bench

Accuracy

✅ Correct line/column tracking ✅ Handles escape sequences ✅ Detects unterminated strings ✅ Error recovery

License

MIT

Project links

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Latest release

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for Souvik_Sarkar from gravatar.com Souvik_Sarkar

Credits

Author: Shreyasi Sarkar

License

MIT License (MIT)

Requires

Python >=3.10

Provides Extra

game lsp all

Tags

game scripting language gamedev gdscript

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

inscript_lang-3.9.6.45.tar.gz (338.8 kB view details)

Uploaded Source

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

inscript_lang-3.9.6.45-cp312-cp312-win_amd64.whl (692.5 kB view details)

Uploaded CPython 3.12Windows x86-64

inscript_lang-3.9.6.45-cp312-cp312-manylinux_2_28_x86_64.whl (776.4 kB view details)

Uploaded CPython 3.12manylinux: glibc 2.28+ x86-64

inscript_lang-3.9.6.45-cp312-cp312-manylinux_2_28_aarch64.whl (744.6 kB view details)

Uploaded CPython 3.12manylinux: glibc 2.28+ ARM64

inscript_lang-3.9.6.45-cp312-cp312-macosx_10_13_universal2.whl (1.1 MB view details)

Uploaded CPython 3.12macOS 10.13+ universal2 (ARM64, x86-64)

inscript_lang-3.9.6.45-cp311-cp311-win_amd64.whl (688.9 kB view details)

Uploaded CPython 3.11Windows x86-64

inscript_lang-3.9.6.45-cp311-cp311-manylinux_2_28_x86_64.whl (773.2 kB view details)

Uploaded CPython 3.11manylinux: glibc 2.28+ x86-64

inscript_lang-3.9.6.45-cp311-cp311-manylinux_2_28_aarch64.whl (743.2 kB view details)

Uploaded CPython 3.11manylinux: glibc 2.28+ ARM64

inscript_lang-3.9.6.45-cp311-cp311-macosx_10_12_universal2.whl (1.1 MB view details)

Uploaded CPython 3.11macOS 10.12+ universal2 (ARM64, x86-64)

inscript_lang-3.9.6.45-cp310-cp310-win_amd64.whl (689.1 kB view details)

Uploaded CPython 3.10Windows x86-64

inscript_lang-3.9.6.45-cp310-cp310-manylinux_2_28_x86_64.whl (773.5 kB view details)

Uploaded CPython 3.10manylinux: glibc 2.28+ x86-64

inscript_lang-3.9.6.45-cp310-cp310-manylinux_2_28_aarch64.whl (743.5 kB view details)

Uploaded CPython 3.10manylinux: glibc 2.28+ ARM64

inscript_lang-3.9.6.45-cp310-cp310-macosx_10_12_universal2.whl (1.1 MB view details)

Uploaded CPython 3.10macOS 10.12+ universal2 (ARM64, x86-64)

File details

Details for the file inscript_lang-3.9.6.45.tar.gz.

File metadata

  • Download URL: inscript_lang-3.9.6.45.tar.gz
  • Upload date:
  • Size: 338.8 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.1.0 CPython/3.13.12

File hashes

Hashes for inscript_lang-3.9.6.45.tar.gz
Algorithm Hash digest
SHA256 5d6d80bcc3f3a64e8cf352dd3b5fcf91bec5824d50c2aed5064a71482aa14036
MD5 400e218bedfca8f91ae5138785633ca8
BLAKE2b-256 c3b84d8e294c796f5ecd8738d33e44387a00624fa5b64928a1d6c819144349bb

See more details on using hashes here.

File details

Details for the file inscript_lang-3.9.6.45-cp312-cp312-win_amd64.whl.

File metadata

File hashes

Hashes for inscript_lang-3.9.6.45-cp312-cp312-win_amd64.whl
Algorithm Hash digest
SHA256 1d8aca4bd8cb7e0c7d4c7cf3bb31796adebbf02473f753993a35c219af2ef12b
MD5 3c0c48466ba72a7854cb411feb296ad5
BLAKE2b-256 2c5a25a56b244a771d7c5d0684e4c4faaf7be8d4dc64dda82d46450bd12d56bb

See more details on using hashes here.

File details

Details for the file inscript_lang-3.9.6.45-cp312-cp312-manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for inscript_lang-3.9.6.45-cp312-cp312-manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 0628e0e16ae275f4fb974a237955c4d58dfafed6043cc065ea430443f5d3a3d6
MD5 4927f8003d672724538adad2a95f3c8a
BLAKE2b-256 65b3755f98336530d80b28f77ba2f2c3f0fb44cab30cf594705df39b3ace714d

See more details on using hashes here.

File details

Details for the file inscript_lang-3.9.6.45-cp312-cp312-manylinux_2_28_aarch64.whl.

File metadata

File hashes

Hashes for inscript_lang-3.9.6.45-cp312-cp312-manylinux_2_28_aarch64.whl
Algorithm Hash digest
SHA256 44fdc7c06f4f7c7072230f9be3f07769f7fdc9a2b88fb52136f7b63c39fb4903
MD5 f70bf2ec642c2e1cbaf3a3b9e8fd53a3
BLAKE2b-256 6db6d46fc26f7a4f5e23aa308d4ec696b54c99b3c22f110a94f64df53ce3daec

See more details on using hashes here.

File details

Details for the file inscript_lang-3.9.6.45-cp312-cp312-macosx_10_13_universal2.whl.

File metadata

File hashes

Hashes for inscript_lang-3.9.6.45-cp312-cp312-macosx_10_13_universal2.whl
Algorithm Hash digest
SHA256 d26f3e8c90a318c76b5fee79ed6cd512a725c908638c04f0eac83d105a74cfe9
MD5 bb02b828960beac1f8d9cde246378153
BLAKE2b-256 9dc6477200960d41043402149febb77d4d13de7166831b1266924d04e04e9a3a

See more details on using hashes here.

File details

Details for the file inscript_lang-3.9.6.45-cp311-cp311-win_amd64.whl.

File metadata

File hashes

Hashes for inscript_lang-3.9.6.45-cp311-cp311-win_amd64.whl
Algorithm Hash digest
SHA256 254e8a7d05c0766fda13fe3b698c53e1f6fe56cd782dd0f82ac2caad41084fe2
MD5 fa33be4424f9463013d1553eb2850062
BLAKE2b-256 680288586f74f642b6582aca4d9c350072ec3032daabfb3637c39057f8de1f72

See more details on using hashes here.

File details

Details for the file inscript_lang-3.9.6.45-cp311-cp311-manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for inscript_lang-3.9.6.45-cp311-cp311-manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 af253b6f0a3776f3789d372ced1b5b61b5fb0b6fbd6cb608e962b40c3f590e46
MD5 ff0cc4603e5ab8a3d87e0c86eccaa4c0
BLAKE2b-256 1138d9defeb3b41861b96be26f58b44073844738cc0231546bc76822a6c2a23e

See more details on using hashes here.

File details

Details for the file inscript_lang-3.9.6.45-cp311-cp311-manylinux_2_28_aarch64.whl.

File metadata

File hashes

Hashes for inscript_lang-3.9.6.45-cp311-cp311-manylinux_2_28_aarch64.whl
Algorithm Hash digest
SHA256 16aeb82adef29e0d36879d904c5331fa6000ac9365bd08065b4d7d465e3c46f0
MD5 79f9f7812cbcdaabf73a00876d085d8e
BLAKE2b-256 980cbcd71973e33d71ba60cd813cc17f7eca91b24025109bd954747e0d148db5

See more details on using hashes here.

File details

Details for the file inscript_lang-3.9.6.45-cp311-cp311-macosx_10_12_universal2.whl.

File metadata

File hashes

Hashes for inscript_lang-3.9.6.45-cp311-cp311-macosx_10_12_universal2.whl
Algorithm Hash digest
SHA256 9b609634927487ac53e2e4bb4ab0af5f7bfe757183828b63f3f2caa9192f113c
MD5 0f3f9fc1041ed28c278b55b27711632d
BLAKE2b-256 968b2051a74d8b4de03fba7380958bff425b517001489834e0f0e4db1b79218b

See more details on using hashes here.

File details

Details for the file inscript_lang-3.9.6.45-cp310-cp310-win_amd64.whl.

File metadata

File hashes

Hashes for inscript_lang-3.9.6.45-cp310-cp310-win_amd64.whl
Algorithm Hash digest
SHA256 c4971783b133c9827d8f70092567177357f7bebe227dd6ff5944c8b85a1b6711
MD5 ac853879c0e69cffade576ee2c46a292
BLAKE2b-256 d0164e4d27daf2de9dfe15c71254593f3076dacc7443e97b8f03fc51c05ddf76

See more details on using hashes here.

File details

Details for the file inscript_lang-3.9.6.45-cp310-cp310-manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for inscript_lang-3.9.6.45-cp310-cp310-manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 a9f2d4097e2cd32d1eb284097450c2901f29113b9560e3994cb2f45da1427410
MD5 4beb0997c42d9242ec52a841217312eb
BLAKE2b-256 a5865715d5f4f9e714f0535843699b4470cc529f4f72141866624f6db4a31805

See more details on using hashes here.

File details

Details for the file inscript_lang-3.9.6.45-cp310-cp310-manylinux_2_28_aarch64.whl.

File metadata

File hashes

Hashes for inscript_lang-3.9.6.45-cp310-cp310-manylinux_2_28_aarch64.whl
Algorithm Hash digest
SHA256 212bc0997cda068786b3cc5c84c954b814e4d82edd5215662dfd52445145b340
MD5 8f0494350b0d3e04d7198e62bd33cbac
BLAKE2b-256 4da4e6501e17de5dd133facb20d3775d6b3afec41bdf47ea6801e0360f385698

See more details on using hashes here.

File details

Details for the file inscript_lang-3.9.6.45-cp310-cp310-macosx_10_12_universal2.whl.

File metadata

File hashes

Hashes for inscript_lang-3.9.6.45-cp310-cp310-macosx_10_12_universal2.whl
Algorithm Hash digest
SHA256 d38703b389a9e90184afaffaa9d8fe11c87b4dc266890f85e144bf0449d3ecf3
MD5 2508ac9326e225c9b2b5a6fb41856801
BLAKE2b-256 7f6f00369c0ee967f362d356ec97243252daeedd4441172429ebcf520d43eddf

See more details on using hashes here.

Release history Release notifications | RSS feed

This release

3.9.6.45 This release

13 files

3.9.6.44

13 files

3.9.6.43

13 files

3.9.6.39

13 files

3.9.6.38

13 files

3.9.6.24

13 files

3.9.6.23

13 files

3.9.6.10.16

13 files

3.0.0

2 files

2.13.0

2 files

2.12.0

2 files

2.11.0

2 files

2.10.0

2 files

2.8.0

2 files

2.5.0

2 files

2.4.0

2 files

2.3.0

2 files

2.2.0

2 files

2.1.2

2 files

2.1.1

2 files

2.1.0

2 files

2.0.3

2 files

2.0.2

2 files

2.0.1

2 files

2.0.0

2 files

1.9.14

2 files

1.9.13

2 files

1.9.12

2 files

1.9.11

2 files

1.9.10

2 files

1.9.9

2 files

1.9.8

2 files

1.9.7

2 files

1.9.6

2 files

1.9.5

2 files

1.9.4

2 files

1.9.2

2 files

1.8.4

2 files

1.8.3

2 files

1.8.2

2 files

1.8.1

2 files

1.7.4

2 files

1.7.3

2 files

1.7.1

2 files

1.6.0

2 files

1.1.0

2 files

1.0.23

2 files

1.0.0

2 files

0.6.0

2 files

0.5.0

1 file

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page