py3to2 2008.11.19

backport 3.0 opcodes to python2.6 so it can natively run 3.0 scripts w/ 2.6 extension modules

################################################################################
this is a python2.6 interpreter extended w/ python3.0 opcodes, allowing it to
run python3.0 scripts. it should b fully compatible w/ cpython2.6 & its
extensions.

the intended purpose is to allow developers to migrate python2.6 scripts to
python3.0 while retaining backwards compatibility w/ existing extension modules.

REVISION DATE: 2008.11.19

AUTHOR:
kai zhu
[e-mail]

REQUIREMENTS:
- posix/unix os (Windows currently unsupported)
- w/ python2.6 & python3.0 installed

INSTALLATION
$ python2.6 setup.py build
$ python2.6 setup.py install

the above will build & install 3 files:
- enhanced python2.6 interpreter: bin/py3to2
- initialization script: lib/python2.6/site-packages/py3to2_init.py
- python3.0 bytecode compiler: lib/python2.6/site-packages/py3to2.py

MAGIC
simply add the MAGIC LINE:

from __future__ import py3k_syntax

to make py3to2 aware that a script is using python3.0 syntax

USAGE:
start up the py3to2 interpreter by typing "py3to2" in a shell:
$ ./py3to2

Python 2.6.py3to2 (r26:66714, Nov 18 2008, 00:56:43)
[GCC 3.4.6 20060404 (Red Hat 3.4.6-10)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>

try out this simple python3.0 script:
################################################################
# PEP3132 Extended Iterable Unpacking
# copy this to file test_pep3132.py

from __future__ import py3k_syntax

a,*b = 1,2,3
assert a == 1 and b == [2,3]
print(a,b)
################################################################

>>> import test_pep3132
created <read/write pipes 4/5>
py3k server starting...
...py3k server started w/ ...

1 [2, 3]
>>>

here's another python3.0 script using scipy (python2.6) extension module:
################################################################
# u must have scipy installed for this script to work
# copy this to file test_pep3132_scipy.py

from __future__ import py3k_syntax

import scipy
a,*b = scipy.array([1,2,3])
assert a == 1 and b == [2,3]
print(a,b)
################################################################

>>>
>>> import test_pep3132_scipy
1 [2, 3]
>>>

another simple, but more thorough test script, test_py3k,
is included w/ this distribution:
>>>
>>> import test_py3k
testing PEP3104 Access to Names in Outer Scopes
testing PEP3105 Make print a function
testing PEP3107 Function Annotations
testing PEP3112 Bytes literals in Python 3000
testing PEP3113 Removal of Tuple Parameter Unpacking
testing PEP3114 Renaming iterator.next() to .__next__()
testing PEP3115 Metaclasses in Python 3000
testing PEP3127 Integer Literal Support and Syntax
testing PEP3129 Class Decorators
testing PEP3132 Extended Iterable Unpacking
testing PEP3135 New Super
testing pseudomethod example 0
testing pseudomethod example 1
testing pseudomethod example 2
testing pseudomethod example 3
>>>

FEATURES:
PEP3102 Keyword-Only Arguments
PEP3104 Access to Names in Outer Scopes
PEP3105 Make print a function
PEP3107 Function Annotations
PEP3111 Simple input built-in in Python 3000
PEP3112 Bytes literals in Python 3000
PEP3113 Removal of Tuple Parameter Unpacking
PEP3114 Renaming iterator.next() to .__next__()
PEP3115 Metaclasses in Python 3000
PEP3127 Integer Literal Support and Syntax
PEP3129 Class Decorators
PEP3132 Extended Iterable Unpacking
PEP3135 New Super

LIMITATIONS (FEATURES NOT FULLY SUPPORTED):
from a migration standpoint, py3to2 is almost feature complete in terms of
python3.0's language syntax, except for:
- unicode support (str vs bytes). future support for utf8 is pending...

language issue aside, python3.0 scripts will still behave differently b/c of
internal differences between python2.6 & python3.0:
- exception handling. py3to2 implements python3.0 syntax for raising &
catching exceptions. but the underlying behavior is still python2.6
- builtin functions / types. alot of these have been changed to behave
differently under python3.0

################################################################################
MECHANISM

py3to2 has 3 components:
- py3to2
python interpreter. can evaluate python2.6 bytecode containing additional
python3.0 opcode instructions

- py3to2_init.py
initialization script. sets up import hook for recognizing python3.0 scripts

- py3to2.py
bytecode compiler. the compilation has 2 steps:
- a persistent python3.0 process is created for compiling scripts into
python3.0 code
- py3to2.py converts the python3.0 code into python2.6 format

################################################################################
MANIFEST
./patch/ - patched files
./py3to2.diff - summary of patches (may be out-of-date)

PYTHON2.6 COMPATIBILITY TEST
output from amd opteron x86_64 machine on redhat linux (3.4.6-10):
$ python setup.py dev --maketest
...
324 tests OK.
36 tests skipped:
test_aepack test_al test_applesingle test_bsddb185 test_bsddb3
test_cd test_cl test_codecmaps_cn test_codecmaps_hk
test_codecmaps_jp test_codecmaps_kr test_codecmaps_tw test_curses
test_dl test_gdbm test_gl test_imageop test_imgfile test_kqueue
test_linuxaudiodev test_macos test_macostools test_normalization
test_ossaudiodev test_pep277 test_py3kwarn test_scriptpackages
test_socketserver test_startfile test_sunaudiodev test_timeout
test_urllib2net test_urllibnet test_winreg test_winsound
test_zipfile64
1 skip unexpected on linux2:
test_gdbm