token translator framework
Project description
tokentranslator
Experimental project. Can be used to create equations or proposals translators by defining replacers for each term (token) with use of gui web interface. Currently contain support for Wolfram->sympy (or cpp), Tex-> sympy (or cpp). Also can extract arguments from equation/proposal. Has experimental sampling proposal generator i.e. for proposal can create subset of it's arguments at which this proposal is true.
Examples of translation:
Tex to sympy (cpp):
U'=a*(\\frac{d^2U}{dx^2}+ \\frac{d^2U}{dy^2})
translated to:
(sympy)
sympy.diff(U(t), t)=a*(diff(U,x, 2)+diff(U,y, 2))
(cpp)
result[idx + 0]=params[0]*((DXM2 * (source[0][idx + 1 * Block0StrideX * Block0CELLSIZE + 0] - 2.0 * source[0][idx + 0 * Block0StrideX * Block0CELLSIZE + 0] + source[0][idx-1 * Block0StrideX * Block0CELLSIZE + 0]))+(DYM2 * (source[0][idx + 1 * Block0StrideY * Block0CELLSIZE + 0] - 2.0 * source[0][idx + 0 * Block0StrideY * Block0CELLSIZE + 0] + source[0][idx-1 * Block0StrideY * Block0CELLSIZE + 0])))
U'=a*(\\frac{d^2U}{dx^2}+ \\frac{d^2U}{dy^2})+sin(x)+A.transpose().conj()
translated to
(sympy)
sympy.diff(U(t), t)=a*(diff(U,x, 2)+diff(U,y, 2))+sympy.sin(x)+A.transpose().conj()
(cpp)
result[idx + 0]=params[0]*((DXM2 * (source[0][idx + 1 * Block0StrideX * Block0CELLSIZE + 0] - 2.0 * source[0][idx + 0 * Block0StrideX * Block0CELLSIZE + 0] + source[0][idx-1 * Block0StrideX * Block0CELLSIZE + 0]))+(DYM2 * (source[0][idx + 1 * Block0StrideY * Block0CELLSIZE + 0] - 2.0 * source[0][idx + 0 * Block0StrideY * Block0CELLSIZE + 0] + source[0][idx-1 * Block0StrideY * Block0CELLSIZE + 0])))+sin((Block0OffsetX+idxX*DX))+A.transpose().conj()
requirements
linux, python3
usage
parsing equations (defalut from Wolfram)
from tokentranslator.db_models.model_main import TokenizerDB
from tokentranslator.env.equation_net.equation import Equation
model = TokenizerDB()
eq = Equation("U'=a*(D[U,{x, 2}]+D[U,{y,2}])", db=model)
eq.parser.parse()
# set default params (like dimension, bounds type (Dirichlet or Neumann) an so on):
eq.replacer.cpp.editor.set_default()
eq.replacer.cpp.make_cpp()
print('\noriginal:')
eq.show_original()
print("\nparsed tree:")
eq.show_cyk_out()
print('\ncpp:')
eq.replacer.cpp.show_cpp()
print("\nsympy:")
eq.replacer.sympy.make_sympy()
eq.replacer.sympy.show_sympy()
parsing equations (from TeX)
# there is currently two dialect databases for equations:
# 'env/equation_net/data/terms/input/tex_dialect.db' for tex
# 'env/equation_net/data/terms/input/demo_dialect.db' for wolfram
# default is wolfram
# to change to tex use commands:
model.save_path("eqs", "env/equation_net/data/terms/input/tex_dialect.db")
model.change_dialect_db("eqs")
# this change "eqs" path for all parsers wherever they run.
# show current dialect.db path:
model.get_path_of_dialect_db("eqs")
# if it's ended with tex_dialect.db then tex input used
# remained is same as above:
eq = Equation("U'=a*(\\frac{d^2U}{dx^2}+ \\frac{d^2U}{dy^2})", db=model)
eq.parser.show_patterns()
eq.parser.parse()
print('\noriginal:')
eq.show_original()
print("\nparsed tree:")
eq.show_cyk_out()
print('\ncpp:')
eq.replacer.cpp.show_cpp()
print("\nsympy:")
eq.replacer.sympy.make_sympy()
eq.replacer.sympy.show_sympy()
# find vars:
from proposalsampler.sampling.vars.vars_extractor import Extractor
import proposalsampler.sampling.vars.vars_maps as vms
vars_extractor = Extractor("eqs")
net_vars = vms.get_args(str(["s"]), clause.net_out.copy(), vars_extractor)
print(net_vars)
parsing proposals:
from tokentranslator.db_models.model_main import TokenizerDB
from tokentranslator.env.clause.clause_main import Clause
model = TokenizerDB()
# switch to clauses db:
model.change_dialect_db("cs")
clause = Clause("bilinear(f)=>Eq(f(x,y,)=g(x,y,)+h(x,y,))Eq"
+ "\\where (g: simmetric(g)) \\and (h: simplectic(h))", db=model)
clause.parser.parse()
# there is currently no dialect to translate clause to, so just check it's generated tree:
clause.show_cyk_out()
# this tree will be used for proposal sampling.
# !for equation parser to work don't forget change db back:
model.change_dialect_db("eqs")
# even if You in other session!
# find vars:
from tokentranslator.translator.sampling.vars.vars_extractor import Extractor
import tokentranslator.translator.sampling.vars.vars_maps as vms
vars_extractor = Extractor("cs")
net_vars = vms.get_args(str(["s"]), clause.net_out.copy(), vars_extractor)
print(net_vars)
installation and running
pip install tokentranslator
Tests:
see tests/test_list.txt
References:
Parser:
Cocke–Younger–Kasami algorithm: https://en.wikipedia.org/wiki/CYK_algorithm
Acknowledgments:
Used software:
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
tokentranslator-0.0.1.3.1.tar.gz
(404.2 kB
view hashes)
Built Distribution
Close
Hashes for tokentranslator-0.0.1.3.1.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | dbc43f09fbaf410fc2be3307a60cf8aae9faec4b58db86e9198cdf6cda34b956 |
|
MD5 | 328f09a3bfdd1fd056ea0689dd910987 |
|
BLAKE2b-256 | 006fcb7a78e357e750af652fb830e62351ebb351b6207f5a47f493754438d001 |
Close
Hashes for tokentranslator-0.0.1.3.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 007274b29026caef28c538cbf68e58cb475b285d9e9462326222280c6fa3d3a2 |
|
MD5 | 4010d0ea31d0e0e6a2f2dd51ef976e11 |
|
BLAKE2b-256 | 60599f789b45a966182820e48d9e7065270b68f0e6c4be182e57c45caea75c2f |