This is a pre-production deployment of Warehouse, however changes made here WILL affect the production instance of PyPI.
Project Description
(image courtesy of fossBytes)

It is based on previous work by Andrew Schaaf.

author: Alberto Berti alberto@metapensiero.it GNU General Public License version 3 or later

## Introduction

JavaScripthon is a small and simple Python 3.5+ translator to JavaScript which aims to be able to translate most of the Python’s core semantics without providing a full python-in-js environment, as most existing translators do. It tries to emit code which is simple to read and check. It does so by switching to ES6 construct when possible/required. This allows to simplify the needs of polyfills for many of the expected Python behaviors.

It is designed to be the first step in a pipeline that translates your Pyhton code into something that a browser can understand. Usually it is used with tools like BabelJS and Webpack to prepare the final bundle that will be served to the browser. The steps from the source code to the bundle are the following:

1. JavaScripthon converts your Python 3.5+ code to ES6 JavaScript modules;
2. the BabelJS loader (configured inside Webpack or standalone) translates the ES6 JavaScript to ES5 so that the browser can understand it;
3. Webpack parses the resulting source code and packages your source code with its dependencies by analyzing import statements and emits a bundle.js ready to be served to the browser.

Along this process the corresponding source maps are read and integrated at every step, allowing you to place breakpoints on your original Python source files when working with the developer tools of your browser.

An example of such setup is provided in the examples directory.

In addition to that, you can choose to do most these steps without using external JS tools. It comes with an embedded js interpreter that loads a standalone version of BabelJS and converts your code to ES5 JavaScript without the need to install anything else. In fact most of the the test you can find in tests/test_evaljs.py use the embedded interpreter to dual evaluate the source code (one time in Python, one time in JavaScript) and simply check that the results are the same.

Thanks to that, JavaScripthon can also be used as a server-side library to translate single functions or classes that you want your browser to load and evaluate.

The interface with the JS world is completely flat, just import the modules or use the expected globals (window, document, etc…) as you would do in JavaScript.

## Brief list of the supported Python semantics

The fact that JavaScripthon doesn’t reinvent the wheel by reimplementing in Python many of the features available with JavaScript translators/transpilers allows it to be lean while implementing quite a decent set of the core Python semanticts. These are, briefly:

• Misc
• list slices;
• list’s append();
• dict’s copy(), update();
• len();
• print();
• str();
• type(instance);
• yield and yield from;
• async and await;
• import and from...import to use any JS module (see import statements);
• callable();
• hasattr(), getattr(), setattr();
• template literals with tmpl('a string with ${substitution}'); • names starting with d_ and dd_ will have that part replaced with$ and $$, respectively; • names ending with an underscore will have it removed. Useful for example with the AVA ES6 test runner which has a check named is; • Comparisons (see section Simple stuff for the details) • most of the basics; • isinstance(); • element in container for use with lists, objects, strings and the new ES6 collections like Map, Set and so on; • identity checks: foo is bar; • ternary comparisons like x < y <= z; • Statements (see section Simple stuff and for statement for the details) • if...elif...else; • while loop; • for over list, over range, over plain js objects, over iterables (JS iterables); • try...except...finally with pythonesque behavior (see try…except…finally statement section for the details); • assert statement; • Functions (see Functions section) • standard functions, generator functions, async functions; • parameters defaults; • keyword parameters; • parameters accumulators (*args and **kwargs), with some restrictions; • functions in methods are usually converted to “arrow functions” (the new ES6 syntax like (foo, bar) => foo * bar;) because they automatically keep this from the enclosing scope. Appending _fn to a function declaration will force the translation to a normal function; • Classes (see Classes section) • single inheritance; • Exception classes for use with except statement; • class decorators and method decorators; • property descriptors; • special handling of property and classmethod descriptors; • async methods, generator methods; • non-function body members (i.e. member_of_class_Foo = bar); ## Installation Python 3.5 is required because Python’s AST has changed between 3.4 and 3.5 and as of now supporting multiple Python versions is not one of my priorities. To install the package execute the following command:  pip install javascripthon  ## Usage To compile or transpile a python source module, use the commandline:  python -m metapensiero.pj source.py  or:  python -m metapensiero.pj -5 source.py  to transpile. A pj console script is also automatically installed:  pj --help usage: pj [-h] [--disable-es6] [--disable-stage3] [-5] [--transform-runtime] [-o OUTPUT] [-d] [--pdb] [-s STRING] [-e] [file [file ...]] A Python 3.5+ to ES6 JavaScript compiler positional arguments: file Python source file(s) or directory(ies) to convert. When it is a directory it will be converted recursively optional arguments: -h, --help show this help message and exit --disable-es6 Disable ES6 features during conversion (Ignored if --es5 is specified) --disable-stage3 Disable ES7 stage3 features during conversion -5, --es5 Also transpile to ES5 using BabelJS. --transform-runtime Add trasform runtime as plugin during transpile -o OUTPUT, --output OUTPUT Output file/directory where to save the generated code -d, --debug Enable error reporting --pdb Enter post-mortem debug when an error occurs -s STRING, --string STRING Convert a string, useful for small snippets. If the string is '-' will be read from the standard input. -e, --eval Evaluate the string supplied with the -s using the embedded interpreter and return the last result. This will convert the input string with all the extensions enabled (comparable to adding the '-5' option) and so it will take some time because of BabelJS load times.  This offers many ways to test the framework, both the string conversion and the evaluation using the embedded JavaScript interpreter are very handy. For example:  pj -s '"foo" if True else "bar"' (true ? "foo" : "bar");  and evaluating the same statement:  pj -s '"foo" if True else "bar"' -e foo  You can even try more fancy ES6 features, like destructuring assignment:  pj -s "a, b, c = (2, 3, 5) \na+b+c" -e 10  ## Conversions Rosetta Stone Here is a brief list of examples of the conversions the tool applies, just some, but not all. ### Simple stuff <caption>Most are obvious</caption> Python JavaScript x < y <= z < 5  ((x < y) && (y <= z) && (z < 5))  def foo(): return [True, False, None, 1729, "foo", r"foo\bar", {}]  function foo() { return [true, false, null, 1729, "foo", "foo\\bar", {}]; }  while len(foo) > 0: print(foo.pop())  while ((foo.length > 0)) { console.log(foo.pop()); }  if foo > 0: .... elif foo < 0: .... else: ....  if ((foo > 0)) { .... } else { if ((foo < 0)) { .... } else { .... } }  str(x)  x.toString()  yield foo yield from foo  yield foo yield* foo  Then there are special cases. Here you can see some of these conversions. JavaScripthon cannot do a full trace of the sources, so some shortcuts are taken about the conversion of some core, specific Python’s semantics. For example Python’s self is always converted to JavaScript’s this, no matter where it’s found. Or len(foo) is always translated to foo.length. Albeit this an API specific of just some objects (Strings, Arrays, etc…), it is considered wide adopted and something the user may consider obvious. The rules of thumb to treat things especially are: • Is it possible to think of a conversion that covers most of the use cases? • Is ts possible to find a convention widely used on the Python world to express this special case? <caption>There are special cases</caption> Python JavaScript ==  ===  !=  !==  2**3  Math.pow(2, 3)  'docstring'  /* docstring */  self  this  len(...)  (...).length  print(...)  console.log(...)  isinstance(x, y) isinstance(x, (y, z))  (x instanceof y) (x instanceof y || x instanceof z)  typeof(x)  (typeof x)  type(x)  Object.getPrototypeOf(x)  FirstCharCapitalized(...) new(any_function(...))  new FirstCharCapitalized(...) new any_function(...)  foo in bar  var _pj; function _pj_snippets(container) { function in_es6(left, right) { if (((right instanceof Array) || ((typeof right) === "string"))) { return (right.indexOf(left) > (- 1)); } else { if (((right instanceof Map) || (right instanceof Set) || (right instanceof WeakMap) || (right instanceof WeakSet))) { return right.has(left); } else { return (left in right); } } } container["in_es6"] = in_es6; return container; } _pj = {}; _pj_snippets(_pj); _pj.in_es6(foo, bar);  foo[3:] foo[:3]  foo.slice(3); foo.slice(0, 3);  list(foo).append(bar)  foo.push(bar);  dict(foo).update(bar)  Object.assign(foo, bar);  dict(foo).copy()  Object.assign({}, foo);  ### for statement The for statement by default is translated as if the object of the cycle is a list but has two special cases: <caption>for loops</caption> Python JavaScript notes for el in dict(a_dict): print(el)  var _pj_a = a_dict; for (var el in _pj_a) { if (_pj_a.hasOwnProperty(el)) { console.log(el); } }  With this kind of loop if you use dict(a_dict, True) the check on hasOwnProperty() will not be added, so the loop will include inherited (and enumerable) properties. for el in an_array: print(el)  for (var el, _pj_c = 0, _pj_a = an_array, _pj_b = _pj_a.length; (_pj_c < _pj_b); _pj_c += 1) { el = _pj_a[_pj_c]; console.log(el); }  for i in range(5): print(i)  for (var i = 0, _pj_a = 5; (i < _pj_a); i += 1) { console.log(i); }  for el in iterable(a_set): print(el)  var _pj_a = a_set; for (var el of _pj_a) { console.log(el); }  This will loop over all the iterables, like instances of Array, Map, Set, etc. but not over normal objects. ### Functions Functions are very well supported. This should be obvious, you can say. Really it is not so simple, if we mean functions in their broader meaning, including the async functions and generator functions. <caption>The various types of functions at play</caption> Python JavaScript notes def foo(a, b, c): pass  function foo(a, b, c) { }  Normal functions def foo(a, b, c): for i in range(a, b, c): yield i for i in iterable(foo(0, 5, 2)): print(i)  function* foo(a, b, c) { for ... { // loop control omitted for brevity yield i; } } for (var i of foo(0, 5, 2)) { console.log(i); }  Generator functions. They return an iterable and to correctly loop over it you should use the iterable(...) call, so that the Python’s for...in will be converted into a for...of async def foo(a, b, c): await some_promise_based_async  async function foo(a, b, c) { await some_promised_base_async; }  Async functions. They make use of the new Promise class, which is also available. #### Function’s args and call parameters Parmeters defaults and keyword parameters are supported and so is *foo accumulator, which is translated into the ES6 rest expression (...foo). The only caveat is that JS support for keyword args sucks, so you will have to remember to fill in all the arguments before specifying keywords. On function definitions, **kwargs is supported if it’s alone, i.e. without either keyword arguments or *args. <caption>function’s args and call parameters</caption> Python JavaScript def foo(a=2, b=3, *args): pass  function foo(a = 2, b = 3, ...args) { }  def bar(c, d, *, zoo=2): pass  function bar(c, d, {zoo = 2}={}) { }  foo(5, *a_list)  foo(5, ...a_list);  bar('a', 'b', zoo=5, another='c')  bar("a", "b", {zoo: 5, another: "c"});  def zoo(e, **kwargs): print(kwargs['bar'])  function zoo(e, kwargs = {}) { console.log(kwargs['bar']) }  zoo(4, bar=6)  zoo(4, {bar: 6})  ### Classes Classes are translated to ES6 classes as much as they can support. This means: • no direct support multi-class inheritance, you have to come up with your own solution for now. Many established frameworks support this in a way or another so just use those facilities for now. I’ve read of some attempts, see for example the suggestion on Mozilla developer or the other about simple mixins on Exploring ES6. • external implementation for class-level non assignment members. Assignement members are those on the body of a class which are defined with: a_label = an_expression like: class Foo: bar = 'zoo' # or any kind of expression  These members are removed from the translated body and submitted to a snippet of code that will run after class creation in JS land. This serves two purposes: if the value is simple, i.e. it isn’t an instance of Object, it will be setup as a data descriptor, and it will work mostly like you are used to in Python. The most noticeable caveat is that it will not be accessible through the class as it is in Python, you will have to access the class’ prototype, so in the case above i mean Foo.prototype.bar. The other purpose is to check for accessor descriptors. If the value on the right side of the assignment implements a get function, it will be installed as a property as-is, and its get and set members will be used to manage the value with the bar name. • external implementation for method decorators whose name is different from property or classmethod (more on these later on), because these are already supported by the ES6 class notation. • external implementation for class decorators. One caveat here is that the return value of the decorator has always to be a function with a prototype: unfortunately a new statement seems not to be delegable in any way. So for example a class decorator implemented like the following: def test_class_deco(): counter = 0 def deco(cls): def wrapper(self, *args): counter += 1 # side effect return cls(*args) return wrapper @deco class Foo: pass  will never work. This will work instead: def deco(cls): def wrapper(self, *args): counter += 1 # side effect return cls.prototype.constructor.call(self, *args) wrapper.prototype = cls.prototype return wrapper  So either return the original class or setup the wrapper appropriately. Methods can be functions or async-functions although the latters aren’t officially supported yet by the JavaScript specification. You can disable them adding a --disable-stage3 to the command line utility. Pythons super() calls are converted accordingly to the type of their surrounding method: super().__init__(foo) becomes super(foo) in constructors. Functions inside methods are translated to arrow functions so that they keep the this of the surrounding method. @property and @a_property.setter are translated to ES6 properties. Methods decorated with @classmethod are translated to static methods. Special methods __str__ and __len__ are translated to toString() method and get length() property, respectively. Arrow method expression to retain the this at method level aren’t implemented yet. <caption>Classes</caption> Python JavaScript class Foo(bar): def __init__(self, zoo): super().__init__(zoo) def meth(self, zoo): super().meth(zoo) def cool(a, b, c): print(self.zoo) async def something(self, a_promise): result = await a_promise def generator_method(self): yield something @property def foo(self): return self._foo @foo.setter def foo(self, value): self._foo = value @classmethod def bar(self, val): do_something() def __len__(self): return 1 def __str__(self): return 'Foo instance'  class Foo extends bar { constructor(zoo) { super(zoo); } meth(zoo) { super.meth(zoo); var cool; cool = (a, b, c) => { console.log(this.zoo); }; } async something(a_promise) { var result; result = await a_promise; } * generator_method() { yield something; } get foo() { return this._foo; } set foo(value) { self._foo = value; } static bar(val) { do_something() } get length() { return 1; } toString() { return "Foo instance"; } }  Only direct descendants of Exception are threated especially, but just for them to be meaningful in JS land and to be detectable with instanceof in catch statements. <caption>Exceptions</caption> Python JavaScript class MyError(Exception): pass raise MyError("An error occurred")  function MyError(message) { this.name = "MyError"; this.message = (message || "Custom error MyError"); if (((typeof Error.captureStackTrace) === "function")) { Error.captureStackTrace(this, this.constructor); } else { this.stack = new Error(message).stack; } } MyError.prototype = Object.create(Error.prototype); MyError.prototype.constructor = MyError; throw new MyError("An error occurred");  ### try...except...finally statement The conversion of this statement is mostly obvious with the only exception of the except part: it translates to a catch part containing one if statement for each non catchall except. If a catchall except isn’t present, the error will be re-thrown, to mimic Python’s behavior. <caption>try...catch...finally statement</caption> Python JavaScript try: foo.bar() except MyError: recover() except MyOtherError: recover_bad() finally: foo.on_end()  try { foo.bar(); } catch(e) { if ((e instanceof MyError)) { recover(); } else { if ((e instanceof MyOtherError)) { recover_bad() } else { throw e; } } } finally { foo.on_end(); }  ### import statements import and from ... import statements are converted to ES6 imports, and the declaration of an __all__ member on the module top level is translated to ES6 exports. <caption>import and exports</caption> Python JavaScript import foo, bar import foo.bar as b from foo.bar import hello as h, bye as bb from ..foo.zoo import bar from . import foo from .foo import bar from foo__bar import zoo from __globals__ import test_name # this should not trigger variable definition test_name = 2 # this instead should do it test_foo = True __all__ = ['test_name', 'test_foo']  var test_foo; import * as foo from 'foo'; import * as bar from 'bar'; import * as b from 'foo/bar'; import {hello as h, bye as bb} from 'foo/bar'; import {bar} from '../foo/zoo'; import * as foo from './foo'; import {bar} from './foo'; import {zoo} from 'foo-bar'; test_name = 2; test_foo = true; export {test_name}; export {test_foo};  ## Examples Execute make inside the examples directory. ## Testing To run the tests you should run the following at the package root: python setup.py test  ## How to contribute So you like this project and want to contribute? Good! These are the terse guidelines: There are some TODO points in the readme, or even the issue #6 is quite simple to fix. Feel free to pick what you like. The guidelines are to follow PEP8 for coding where possible, so use CamelCase for classes and snake_case for variables, functions and members, and UPPERCASE for constants. An exception to this rules are the function names inside metapensiero.pj.transformations subpackage. Those are matched against names of the ast objects coming from the ast module in standard lib, so they have to to match even in case. Try to keep lines lengths under 79 chars, more or less ;-) The workflow is to fork the project, do your stuff, maybe add a test for it and then submit a pull request. Have fun ` ## Build status ## Contributing Any contribution is welcome, drop me a line or file a pull request. ## Todo This is a brief list of what needs to be done: • refactor the comprehensions conversion to use the snippets facility; • refactor snippets rendering to write them as a module and import them in the module when tree conversion is enabled; • convert dict() calls to ES6 Map object creation; • convert set literals to ES6 Set objects. Also, update “foo in bar” to use bar.has(foo) for sets; • multi-line strings to ES6 template strings (does this make any sense?); ## Done Stuff that was previously in the todo: • translate import statements to ES6; • translate __all__ definition to ES6 module exports; • write a command line interface to expose the api; • make try…except work again and implement try…finally; • convert async and await to the same proposed features for js (see BabelJS documentation); • convert argument defaults on functions to ES6; • convert call keyword arguments; • convert *iterable syntax to ES6 destructuring; • use arrow functions for functions created in functions; • properties to ES6 properties (getter and setter); • take advantage of new duckpy features to use a JS execution context that lasts multiple calls. This way the BabelJS bootstrap affects only the initial execution; • class and method decorators; • implement yield, yield from and generator functions; • update “foo in bar” to use bar.has(foo) for maps; ## External documentation A good documentation and explanation of ES6 features can be found on the book Exploring ES6 by Axel Rauschmayer (donate if you can). An extensive documentation about Python’s ast objects, very handy. ## Tools Have a look at ECMAScript 6 Tools by Addy Osmani. To debug source maps have a look at source-map-visualization and its package on npm. Still i found these links to be helpful: Here is an example of the latter tool showing code generated by JavaScripthon, have fun! ## Notes • A post about proposed solutions to use ES6 classes with Backbone. See also the bug open on github. • A story about ES6 crazyest stuff… symbols ## Changes ### 0.5 (2016-11-23) • translate tmpl("A string with js {interpolation}") to ES6 template literals; • preliminary support to translate names like d_foo and dd_bar to foo and$$bar;
• addded translation of the assert statement;
• fixed a bug in try...except...finally statement when there’s no except section;
• added translation for foo is not bar that seems to have dedicated ast node;
• if the function is defined in a method but starts with fn_ do not convert it to an arrow function. Useful to not maintain this;
• added translation for callable and hasattr/getattr/setattr;
• updated for loops to support more than one target, so now its possible to write loops like for k, v in iterable(a_map):;
• updated documentation;
• added a new cli option -s to translate source from the command line or the standard input;
• fixed a pair of bugs on sourcemaps;
• added a new cli option --eval to also evaluate the produced JavaScript using the embedded interpreter;
• added a new cli option --dump-ast to print out the ast tree of the passed in string;
• added sorting to the rendered snippets/decorators/assignments so that their order does not change at every ricompilation;
• do not re-declare variables declare in outer scopes;

### 0.4 (2016-11-15)

• updated BabelJS to version 6.18.1;
• allow to import modules with dashes inside by using dunder-inside-words notation (foo__bar becomes foo-bar);
• reuse JavaScript interpreter context to speedup translation;
• update in operator to support ES6 collections;
• added support for method and class decorators;
• added support for class properties and descriptors;
• add for loop over JS iterables;
• allow to loop over inherited properties;
• fix a bug on type() translation;
• support for range() steps;
• add support for generator functions and yield and yield from expressions;
• optionally load babel-polyfill before evaluating code;
• fix a bug on sourcemaps having wrong references when there are documentation elements;
• translate __get__() and __set__() to to JS equivalents;
• implement dict(foo).update(bar) and dict(foo).copy;
• documentation improvements;

### 0.3 (2016-04-08)

• updates to the documentation ( with some fixes made by Hugo Herter, Daniel Kopitchinski and ironmaniiith)
• Translate str(x) into x.toString()
• Add support for properties and classmethods
• Translate __len__ and __str__ methods to get length() and toString()
• Add support for slices syntax to .slice()
• Fixed two bugs in sourcemaps generation
• Fixed a bug in the inport ... from translation
• Correctly include BabelJS minimized code
• Fix transpiling of stage3 features

### 0.2 (2016-03-29)

• use arrow functions to retain this were possible
• translate async/await
• refactoring of the for loops
• add ability to subtranslate pieces of Python code or objects. Used to template the creation of Exception sublasses
• add support for param defaults and keyword arguments
• updated documentation

### 0.1 (2016-03-21)

• First cut of the features
Release History

## Release History

0.5

This version

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

0.4

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

0.3

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

0.2

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

0.1

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.