pylox

My first implementation of Lox, written in Python.

This implementation is actually a superset of the Lox programming language defined in the Crafting Interpreters book. Meaning, every valid lox program will run on this interpreter too, but programs that would throw an error in the book's implementation, might work in mine.

For example, this one supports single quotes:

$ lox
> print 'Single quotes!';
Single quotes!

Installation and usage

• Install via pip:

pip install mypylox

• Run it interactively:

$ lox
> print "Hi!";
Hi!
> var num = 2.5 + 2.5;
> print num;
5.0

• Run a file:

$ cat myfile.lox
var name = "Tushar";
var age = "21";

print "My name is " + name + " and I'm " + age + " years old.";

$ lox myfile.lox
My name is Tushar and I'm 21 years old.

Progress

What has already been implemented:

• Entire lexer implementation
• Expression parsing
• Syntax Errors
• Expression execution
• Print statements
• Variable declaration, and global scope
• Interactive REPL
• Variable assignment
• Proper runtime errors
• Synchronization and reporting multiple parse errors
• Local scope, enclosing scope, blocks and nesting
• if-else statements
• while loops
• for loops
• Logical and and or operators
• Function declarations, calls, first class functions and callbacks
• Return values
• Closures
• Compile time variable resolution and binding
• Class declarations, objects, and properties
• Class properties
• Methods
• this attribute
• Constructors
• Inheritance

Changes / Extra features

There's a small number of changes from the reference language:

• Defining a variable as itself inside a block does not result in a parse error. Instead, it uses the value from outer scope to define itself in local scope. This lets us simplify the resolver code: We don't need a separate declare and define method anymore, and each scope is a set, not a dictionary.

• You can access methods directly on a class, you don't need to necessarily create an object to do so. Doing this returns an "unbound method". The main reason to do this is for better DX, and to allow dir(C) to work. It is not yet decided if unbound methods are just not usable at all, or if some sort of binding mechanism will be implemented to allow for that. [TODO]

Here's the full set of extra features, and their progress:

• Much better error messages
• Allowing single quotes for strings
• String escapes: \n, \t, \', \", \\ and \↵
• New operators:
  • Modulo %
  • Integer division \
  • Power **
• Augmented versions of all operators: +=, **=, etc.
• New data types:
  • int: 42
  • list: [42, 56]
  • dictionary: {42: "Forty two"}
• Indexing support: for lists, dictionaries and strings
• Comparison operators work on strings
• break and continue semantics in loops
• Exceptions, try / except blocks and raise statements
• Added builtin functions:
  • input
  • format (Python equivalent, for string interpolation)
  • min, max and abs
  • map, filter and reduce that take lists and return new lists
  • dir to print out an object's attributes
• An import system
• A standard library
  • random module
  • io module string and binary reads/writes to files
  • http library to make a web server and client

Examples of all of these will be available in the examples folder.

Ideas to tinker with

The following are ideas that I'm not 100% sure about, but would like to try. Some of these will not be compatible with the original lox language, so if I implement those, I'll do that in a separate branch.

• No nil. Variables will always have to be initialized with a value, and functions will either never return anything, or return in every case. This will be checked at compile time.
• Declaration scope vs. Block scope. The idea being, if we don't allow nil, block scope might cause an issue, but no nil + declaration scope might work.
• Adding whitespace tokens to the lexer and AST, and adding a code formatting module into the standard library.
• Built-in benchmarking library.

Testing / Development

Get the project:

git clone https://github.com/tusharsadhwani/pylox
cd pylox

Get the dev requirements:

pip install -r requirements-dev.txt

Run tests:

pytest

Type check the code:

mypy .