Javascript for refined palates: a Python 3 to ES6 Javascript translator
Project description
It is based on previous work by Andrew Schaaf.
- author:
Alberto Berti
- contact:
- license:
GNU General Public License version 3 or later
Introduction
JavaScripthon is a small and simple Python 3.5+ translator to JavaScript which aims to be able to translate most of the Python’s core semantics without providing a full python-in-js environment, as most existing translators do. It tries to emit code which is simple to read and check. It does so by switching to ES6 construct when possible/required. This allows to simplify the needs of polyfills for many of the expected Python behaviors.
It is designed to be the first step in a pipeline that translates your Python code into something that a browser can understand. Usually it is used with tools like BabelJS and Webpack to prepare the final bundle that will be served to the browser. The steps from the source code to the bundle are the following:
JavaScripthon converts your Python 3.5+ code to ES6 JavaScript modules;
the BabelJS loader (configured inside Webpack or standalone) translates the ES6 JavaScript to ES5 so that the browser can understand it;
Webpack parses the resulting source code and packages your source code with its dependencies by analyzing import statements and emits a bundle.js ready to be served to the browser.
Along this process the corresponding source maps are read and integrated at every step, allowing you to place breakpoints on your original Python source files when working with the developer tools of your browser.
An example of such setup is provided in the examples directory.
In addition to that, you can choose to do most these steps without using external JS tools. It comes with an embedded js interpreter that loads a standalone version of BabelJS and converts your code to ES5 JavaScript without the need to install anything else. In fact most of the the test you can find in tests/test_evaljs.py use the embedded interpreter to dual evaluate the source code (one time in Python, one time in JavaScript) and simply check that the results are the same.
Thanks to that, JavaScripthon can also be used as a server-side library to translate single functions or classes that you want your browser to load and evaluate.
The interface with the JS world is completely flat, just import the modules or use the expected globals (window, document, etc…) as you would do in JavaScript.
Brief list of the supported Python semantics
The fact that JavaScripthon doesn’t reinvent the wheel by reimplementing in Python many of the features available with JavaScript translators/transpilers allows it to be lean while implementing quite a decent set of the core Python semantics. These are, briefly:
Misc
list slices;
list’s append();
dict’s copy(), update();
len();
print();
str();
type(instance);
yield and yield from;
async and await;
import and from...import to use any JS module (see import statements);
callable();
hasattr(), getattr(), setattr();
template literals with tmpl('a string with ${substitution}');
template literals and tagged_templates (see Strings);
names starting with d_ and dd_ will have that part replaced with $ and $$, respectively;
names ending with an underscore will have it removed. Useful for example with the AVA ES6 test runner which has a check named is;
__instancecheck__ to [Symbol.hasInstance];
int to parseInt;
float to parseFloat;
dictionary keys are unambiguous when ES6 translation is enabled. For example the following code gets translated correctly:
a = 'foo' d = {a: 1} print(d[a])
prints 1 in both Python and JavaScript, while it prints undefined when translated and evaluated in JavaScript without ES6.
Comparisons (see section Simple stuff for the details)
most of the basics;
isinstance() and issubclass();
element in container for use with lists, objects, strings and the new ES6 collections like Map, Set and so on;
identity checks: foo is bar;
chained comparisons like x < y <= z;
Statements (see section Simple stuff and for statement for the details)
if...elif...else;
while loop;
for over list, over range, over plain js objects, over iterables (JS iterables);
try...except...finally with pythonesque behavior (see try…except…finally statement section for the details);
assert statement;
Functions (see Functions section)
standard functions, generator functions, async functions;
parameters defaults;
keyword parameters;
parameters accumulators (*args and **kwargs), with some restrictions;
functions in methods are usually converted to “arrow functions” (the new ES6 syntax like (foo, bar) => foo * bar;) because they automatically keep this from the enclosing scope. Appending _fn to a function declaration will force the translation to a normal function;
Classes (see Classes section)
single inheritance;
Exception classes for use with except statement;
class decorators and method decorators;
property descriptors;
special handling of property and classmethod descriptors;
async methods, generator methods;
non-function body members (i.e. member_of_class_Foo = bar);
License
This package is covered by the GNU General Public License version 3 or later. The code produced by it (i.e. the transpiled JavaScript) is your code, and you are free to choose whatever license you like. The only runtime that exists is the file snippets.py from which some utility functions are picked when necessary and transpiled together with your code. While it’s distributed with the same license as the other source code, in its transpiled form will have the license you choose.
So, to summarize, the license of the this tool is GPL, but it doesn’t extends to the products of this tool, on which you are free to decide.
Installation
Python 3.5 is required because Python’s AST has changed between 3.4 and 3.5 and as of now supporting multiple Python versions is not one of my priorities.
To install the package execute the following command:
$ pip install javascripthon
or, if you want install it from sources:
$ git clone https://github.com/azazel75/metapensiero.pj $ pip install -r metapensiero.pj/requirements.txt $ pip install metapensiero.pj
Usage
To compile or transpile a python source module, use the commandline:
$ python -m metapensiero.pj source.py
or:
$ python -m metapensiero.pj -5 source.py
to transpile.
A pj console script is also automatically installed:
$ pj --help
usage: pj [-h] [--disable-es6] [--disable-stage3] [-5] [--transform-runtime]
[-o OUTPUT] [-d] [--pdb] [-s STRING] [-e]
[file [file ...]]
A Python 3.5+ to ES6 JavaScript compiler
positional arguments:
file Python source file(s) or directory(ies) to convert.
When it is a directory it will be converted
recursively
optional arguments:
-h, --help show this help message and exit
--disable-es6 Disable ES6 features during conversion (Ignored if
--es5 is specified)
--disable-stage3 Disable ES7 stage3 features during conversion
-5, --es5 Also transpile to ES5 using BabelJS.
--transform-runtime Add trasform runtime as plugin during transpile
-o OUTPUT, --output OUTPUT
Output file/directory where to save the generated code
-d, --debug Enable error reporting
--pdb Enter post-mortem debug when an error occurs
-s STRING, --string STRING
Convert a string, useful for small snippets. If the
string is '-' will be read from the standard input.
-e, --eval Evaluate the string supplied with the -s using the
embedded interpreter and return the last result. This
will convert the input string with all the extensions
enabled (comparable to adding the '-5' option) and so
it will take some time because of BabelJS load times.
This offers many ways to test the framework, both the string conversion and the evaluation using the embedded JavaScript interpreter are very handy. For example:
$ pj -s '"foo" if True else "bar"'
(true ? "foo" : "bar");
and evaluating the same statement:
$ pj -s '"foo" if True else "bar"' -e
foo
You can even try more fancy ES6 features, like destructuring assignment:
$ pj -s "a, b, c = (2, 3, 5) \na+b+c" -e
10
You can use metapensiero.pj in python code as well.
from metapensiero.pj.__main__ import transform_string
transform_string("print()")
Reporting Bugs
The main development repository is the one on gitlab, the one on github is just a mirror so please report issues and feature requests there.
Conversions Rosetta Stone
Here is a brief list of examples of the conversions the tool applies, just some, but not all.
Simple stuff
Python |
JavaScript |
---|---|
|
|
|
|
|
|
|
|
|
|
|
|
Then there are special cases. Here you can see some of these conversions. JavaScripthon cannot do a full trace of the sources, so some shortcuts are taken about the conversion of some core, specific Python’s semantics. For example Python’s self is always converted to JavaScript’s this, no matter where it’s found. Or len(foo) is always translated to foo.length. Albeit this an API specific of just some objects (Strings, Arrays, etc…), it is considered wide adopted and something the user may consider obvious.
The rules of thumb to treat things especially are:
Is it possible to think of a conversion that covers most of the use cases?
Is ts possible to find a convention widely used on the Python world to express this special case?
Python |
JavaScript |
---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
for statement
The for statement by default is translated as if the object of the cycle is a list but has two special cases:
Python |
JavaScript |
notes |
---|---|---|
|
|
With this kind of loop if you use dict(a_dict, True) the check on hasOwnProperty() will not be added, so the loop will include inherited (and enumerable) properties. |
|
|
|
|
|
|
|
|
This will loop over all the iterables, like instances of Array, Map, Set, etc. but not over normal objects. |
Functions
Functions are very well supported. This should be obvious, you can say. Really it is not so simple, if we mean functions in their broader meaning, including the async functions and generator functions.
Python |
JavaScript |
notes |
---|---|---|
|
|
Normal functions |
|
|
Generator functions. They return an iterable and to correctly loop over it you should use the iterable(...) call, so that the Python’s for...in will be converted into a for...of |
|
|
Async functions. They make use of the new Promise class, which is also available. |
Function’s args and call parameters
Parmeters defaults and keyword parameters are supported and so is *foo accumulator, which is translated into the ES6 rest expression (...foo).
The only caveat is that JS support for keyword args sucks, so you will have to remember to fill in all the arguments before specifying keywords.
On function definitions, **kwargs is supported if it’s alone, i.e. without either keyword arguments or *args.
Python |
JavaScript |
---|---|
|
|
|
|
|
|
|
|
|
|
|
|
Classes
Classes are translated to ES6 classes as much as they can support. This means:
no direct support multi-class inheritance, you have to come up with your own solution for now. Many established frameworks support this in a way or another so just use those facilities for now. I’ve read of some attempts, see for example the suggestion on Mozilla developer or the other about simple mixins on Exploring ES6.
external implementation for class-level non assignment members. Assignment members are those on the body of a class which are defined with: a_label = an_expression like:
class Foo: bar = 'zoo' # or any kind of expression
These members are removed from the translated body and submitted to a snippet of code that will run after class creation in JS land. This serves two purposes: if the value is simple, i.e. it isn’t an instance of Object, it will be setup as a data descriptor, and it will work mostly like you are used to in Python. The most noticeable caveat is that it will not be accessible through the class as it is in Python, you will have to access the class’ prototype, so in the case above i mean Foo.prototype.bar.
The other purpose is to check for accessor descriptors. If the value on the right side of the assignment implements a get function, it will be installed as a property as-is, and its get and set members will be used to manage the value with the bar name.
external implementation for method decorators whose name is different from property or classmethod (more on these later on), because these are already supported by the ES6 class notation.
external implementation for class decorators. One caveat here is that the return value of the decorator has always to be a function with a prototype: unfortunately a new statement seems not to be delegable in any way. So for example a class decorator implemented like the following:
def test_class_deco(): counter = 0 def deco(cls): def wrapper(self, *args): counter += 1 # side effect return cls(*args) return wrapper @deco class Foo: pass
will never work. This will work instead:
def deco(cls): def wrapper(self, *args): counter += 1 # side effect return cls.prototype.constructor.call(self, *args) wrapper.prototype = cls.prototype return wrapper
So either return the original class or setup the wrapper appropriately.
Methods can be functions or async-functions although the latters aren’t officially supported yet by the JavaScript specification. You can disable them adding a --disable-stage3 to the command line utility.
Python`s super() calls are converted accordingly to the type of their surrounding method: super().__init__(foo) becomes super(foo) in constructors.
Functions inside methods are translated to arrow functions so that they keep the this of the surrounding method.
@property and @a_property.setter are translated to ES6 properties.
Methods decorated with @classmethod are translated to static methods.
Special methods __str__ and __len__ are translated to toString() method and get length() property, respectively.
Arrow method expression to retain the this at method level aren’t implemented yet.
Python |
JavaScript |
---|---|
|
|
Only direct descendants of Exception are treated especially, but just for them to be meaningful in JS land and to be detectable with instanceof in catch statements.
Python |
JavaScript |
---|---|
|
|
try...except...finally statement
The conversion of this statement is mostly obvious with the only exception of the except part: it translates to a catch part containing one if statement for each non catchall except. If a catchall except isn’t present, the error will be re-thrown, to mimic Python’s behavior.
Python |
JavaScript |
---|---|
|
|
import statements
import and from ... import statements are converted to ES6 imports, and the declaration of an __all__ member on the module top level is translated to ES6 named exports.
Python |
JavaScript |
---|---|
|
|
About JS default export and import
If you want to export something as default export in your modules, declare a __default__ member and assign to it the string of the symbol you want to export. To clarify:
foo = 42
bar = "hello"
__all__ = ['foo', 'bar'] # foo and bar will be exported as named exports
__default__ = 'bar' # bar will also be exported as the *default*
This becomes:
var bar, foo;
foo = 42;
bar = "hello";
export {foo, bar};
export default bar;
For what concerns the import, you can import the default export of a module using the default name, as defined by the ES6 spec. However, as there were some issues reported to me with bundlers not supporting the named import of the default export, a special import statement using __default__ as name has been added that directly translates to the more common form of ES6 default import. So:
from foo import default as bar
from foo import __default__ as zoo
Translates to:
import {default as bar} from 'foo';
import zoo from 'foo';
The two imports should work the same, see exploring js section and the linked spec. But if you encounter problems with the former use the latter instead. Keep in mind that you cannot mix the __default__ import with others (i.e. it needs to be on a line of its own) and that you always need to specify an ... as name ... part.
Strings
Javascripthon supports converting Python 3.6+ f-strings to ES6 template literals. The expression in the braces gets converted, but neither conversion nor format_spec are supported: f"Value of {a}" becomes `Value of ${a}` and f"Value of {self.foo}" becomes `Value of ${this.foo}`.
You can also write raw template literals by using the function tmpl() it does only a conversion of the string markers, from those used in Python’s literal string notation to template literal notation.
There is also the way to express tagged templates, template literals that are parsed using a provided function. This is done by using the function __. So for example:
__('A template ${string} with foo', bar)
gets translated to:
bar`A template ${string} with foo`
bar will be executed with the value of ${string} as a parameter, see the link for template literals for help.
Verbatim JS
You can intermix Python and JS by using the JS('Your-JS-code-here') marker function. It will not be touched by the ES6 transcoder but if you choose to also transpile it to ES5, il will be considered by Babel.
Examples
Execute make inside the examples directory.
Testing
To run the tests you should run the following at the package root:
python setup.py test
How to contribute
So you like this project and want to contribute? Good!
These are the terse guidelines:
There are some TODO points in the readme, or even the issue #6 is quite simple to fix. Feel free to pick what you like. The guidelines are to follow PEP8 for coding where possible, so use CamelCase for classes and snake_case for variables, functions and members, and UPPERCASE for constants. An exception to this rules are the function names inside ``metapensiero.pj.transformations`` subpackage. Those are matched against names of the AST objects coming from the ``ast`` module in standard lib, so they have to to match even in case. Try to keep lines lengths under 79 chars, more or less ;-) The workflow is to fork the project, do your stuff, maybe add a test for it and then submit a pull request. Have fun
Contributing
Any contribution is welcome, drop me a line or file a pull request.
External contributions
BrainBacon has made a JavaScripthon loader for WebPack;
icarito has contributed support for JavaScripthon to the python-webpack-loader for WebPack (Every valid JS package has at least two implementations! ROTFL);
icarito has also integrated JavaScripthon with Nuxt.js and Vue.js;
chfw has integrated JavaScripthon into pyecharts to allow Python function translation.
Todo
This is a brief list of what needs to be done:
refactor the comprehensions conversion to use the snippets facility;
refactor snippets rendering to write them as a module and import them in the module when tree conversion is enabled;
convert dict() calls to ES6 Map object creation;
convert set literals to ES6 Set objects. Also, update “foo in bar” to use bar.has(foo) for sets;
Done
Stuff that was previously in the todo:
translate import statements to ES6;
translate __all__ definition to ES6 module exports;
write a command line interface to expose the api;
make try…except work again and implement try…finally;
convert async and await to the same proposed features for js (see BabelJS documentation);
convert argument defaults on functions to ES6;
convert call keyword arguments;
convert *iterable syntax to ES6 destructuring;
use arrow functions for functions created in functions;
properties to ES6 properties (getter and setter);
take advantage of new duckpy features to use a JS execution context that lasts multiple calls. This way the BabelJS bootstrap affects only the initial execution;
class and method decorators;
implement yield, yield from and generator functions;
update “foo in bar” to use bar.has(foo) for maps;
External documentation
A good documentation and explanation of ES6 features can be found on the book Exploring ES6 by Axel Rauschmayer (donate if you can).
An extensive documentation about Python’s AST objects, very handy.
Tools
Have a look at ECMAScript 6 Tools by Addy Osmani.
To debug source maps have a look at source-map-visualization and its package on npm.
Still i found these links to be helpful:
Here is an example of the latter tool showing code generated by JavaScripthon, have fun!
Notes
A benchmark of ES6 features and discussion about it on hacker’s news.
A compatibility table of ES6 features showing completeness of support feature by feature.
A story about ES6 craziest stuff… symbols
Changes
0.13 (2024-07-04)
allow attributes to be used as indices (i.e. a[b.c]) (thanks to Todd Parsons);
allow unaryop as indices ( i.e. negative integers ) (thanks to Todd Parsons);
allow subscripts as indices (i.e. a[b[0]]) (thanks to Todd Parsons);
0.12 (2022-07-19)
remove macropy
fix evaluation from commandline
deprecate Python 3.5 and 3.6
tested on Python 3.10
0.11 (2020-03-30)
update test infrastructure to work with latest pytest;
add support for Python 3.7 and 3.8 (thanks to Richard Höchenberger).
do not crash when translating source with assignment typehints ( with the help of Sirenfal)
0.10 (2018-05-12)
use Macropy3 version 1.1.0b2 to avoid issues with Django
0.9 (2018-04-19)
add a --source-name options to be used together with --inline-map when using -s;
move main repository to gitlab.com/metapensiero;
add support for default export and import;
add documentation for the JS() marker function;
refactor of the JS AST nodes;
fix path splitting and joining on Windows (thanks to Roman Yakubuk);
0.8 (2017-11-16)
add support for except sections with more than one exception type and arbitrary exception variable name. Thanks to @devanlai;
dict keys conversion fixes;
enable --inline-map when translating a string with -s;
0.7 (2017-09-08)
translate dicts unambiguously, using “computed member name form” for keys that aren’t strings;
use macropy package to deal with some of the translation details;
translate int() and float();
fix a bug that prevented BabelJS translation when keyword arguments; are present;
0.6 (2017-05-09)
allow to define template literals and tagged templates;
define package scopes in imports prepending names with __;
translate issubclass();
translate lambdas as arrow functions;
translate Python 3.6+ f-strings to ES6 template literals;
Add translation for __instancecheck__ to [Symbol.hasInstance];
Sort imports alphabetically;
0.5 (2016-11-23)
translate tmpl("A string with js ${interpolation}") to ES6 template literals;
preliminary support to translate names like d_foo and dd_bar to $foo and $$bar;
addded translation of the assert statement;
fixed a bug in try...except...finally statement when there’s no except section;
added translation for foo is not bar that seems to have dedicated ast node;
if the function is defined in a method but starts with fn_ do not convert it to an arrow function. Useful to not maintain this;
added translation for callable and hasattr/getattr/setattr;
updated for loops to support more than one target, so now its possible to write loops like for k, v in iterable(a_map):;
updated documentation;
added a new cli option -s to translate source from the command line or the standard input;
fixed a pair of bugs on sourcemaps;
added a new cli option --eval to also evaluate the produced JavaScript using the embedded interpreter;
added a new cli option --dump-ast to print out the ast tree of the passed in string;
added sorting to the rendered snippets/decorators/assignments so that their order does not change at every ricompilation;
do not re-declare variables declare in outer scopes;
0.4 (2016-11-15)
updated BabelJS to version 6.18.1;
allow to import modules with dashes inside by using dunder-inside-words notation (foo__bar becomes foo-bar);
reuse JavaScript interpreter context to speedup translation;
update in operator to support ES6 collections;
added support for method and class decorators;
added support for class properties and descriptors;
add for loop over JS iterables;
allow to loop over inherited properties;
fix a bug on type() translation;
support for range() steps;
add support for generator functions and yield and yield from expressions;
optionally load babel-polyfill before evaluating code;
fix a bug on sourcemaps having wrong references when there are documentation elements;
translate __get__() and __set__() to to JS equivalents;
implement dict(foo).update(bar) and dict(foo).copy;
documentation improvements;
0.3 (2016-04-08)
updates to the documentation ( with some fixes made by Hugo Herter, Daniel Kopitchinski and ironmaniiith)
Translate str(x) into x.toString()
Add support for properties and classmethods
Translate __len__ and __str__ methods to get length() and toString()
Add support for slices syntax to .slice()
Fixed two bugs in sourcemaps generation
Fixed a bug in the inport ... from translation
Correctly include BabelJS minimized code
Fix transpiling of stage3 features
0.2 (2016-03-29)
use arrow functions to retain this were possible
translate async/await
refactoring of the for loops
add ability to subtranslate pieces of Python code or objects. Used to template the creation of Exception sublasses
add support for param defaults and keyword arguments
updated documentation
0.1 (2016-03-21)
First cut of the features
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file javascripthon-0.13.tar.gz
.
File metadata
- Download URL: javascripthon-0.13.tar.gz
- Upload date:
- Size: 544.3 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/5.1.1 CPython/3.10.14
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | aef945c3c544f3c527b6497a01a3e057d2049b9a2f660f99ad0cf1da7995bfdb |
|
MD5 | e79649235a8574fca27f42aafa23f142 |
|
BLAKE2b-256 | 50a8cec12d3d666d1e27e28a9d45d17ec583cfbd0f741304b759050d7b91fade |
File details
Details for the file javascripthon-0.13-py3-none-any.whl
.
File metadata
- Download URL: javascripthon-0.13-py3-none-any.whl
- Upload date:
- Size: 526.8 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/5.1.1 CPython/3.10.14
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 5a9bda2c4f2b8e6f569eb228a0b97a111d413e2b5644cd77fcc0f52d34a0c3ad |
|
MD5 | b18d291661acbe4025ddb78289bb4e99 |
|
BLAKE2b-256 | 9013915796fd1e6abab2389f287405f281f069182ac259e588268cc7f936f046 |