token translator framework
Project description
tokentranslator
Experimental project. Can be used to create equations or proposals translators by defining replacers for each term (token) with use of a gui web interface. Currently contain support for Wolfram->sympy (or cpp), Tex-> sympy (or cpp). Also can extract arguments from an equation/proposal. Has an experimental sampling proposal generator i.e. for proposal can create subset of it's arguments at which this proposal is true.
Examples of translation:
Tex to sympy (cpp):
U'=a*(\\frac{d^2U}{dx^2}+ \\frac{d^2U}{dy^2})
translated to:
(sympy)
sympy.diff(U(t), t)=a*(diff(U,x, 2)+diff(U,y, 2))
(cpp)
result[idx + 0]=params[0]*((DXM2 * (source[0][idx + 1 * Block0StrideX * Block0CELLSIZE + 0] - 2.0 * source[0][idx + 0 * Block0StrideX * Block0CELLSIZE + 0] + source[0][idx-1 * Block0StrideX * Block0CELLSIZE + 0]))+(DYM2 * (source[0][idx + 1 * Block0StrideY * Block0CELLSIZE + 0] - 2.0 * source[0][idx + 0 * Block0StrideY * Block0CELLSIZE + 0] + source[0][idx-1 * Block0StrideY * Block0CELLSIZE + 0])))
U'=a*(\\frac{d^2U}{dx^2}+ \\frac{d^2U}{dy^2})+sin(x)+A.transpose().conj()
translated to
(sympy)
sympy.diff(U(t), t)=a*(diff(U,x, 2)+diff(U,y, 2))+sympy.sin(x)+A.transpose().conj()
(cpp)
result[idx + 0]=params[0]*((DXM2 * (source[0][idx + 1 * Block0StrideX * Block0CELLSIZE + 0] - 2.0 * source[0][idx + 0 * Block0StrideX * Block0CELLSIZE + 0] + source[0][idx-1 * Block0StrideX * Block0CELLSIZE + 0]))+(DYM2 * (source[0][idx + 1 * Block0StrideY * Block0CELLSIZE + 0] - 2.0 * source[0][idx + 0 * Block0StrideY * Block0CELLSIZE + 0] + source[0][idx-1 * Block0StrideY * Block0CELLSIZE + 0])))+sin((Block0OffsetX+idxX*DX))+A.transpose().conj()
requirements
pip install -r requirements.txt
installation and running
pip install tokentranslator
# for tex as default dialect for equations:
pip install -v --install-option="--dialect=tex" tokentranslator
# for wolfram as default dialect for equations:
pip install -v --install-option="--dialect=wolfram" tokentranslator
Tests:
see tests/test_list.txt
GUI:
There is also a GUI for this project at
https://github.com/tokentranslator-group/tokentranslator-gui
usage
parsing equations (defalut from Wolfram)
from tokentranslator.db_models.model_main import TokenizerDB
from tokentranslator.env.equation_net.equation import Equation
model = TokenizerDB()
eq = Equation("U'=a*(D[U,{x, 2}]+D[U,{y,2}])", db=model)
eq.parser.parse()
# set default params (like dimension, bounds type (Dirichlet or Neumann) an so on):
eq.replacer.cpp.editor.set_default()
eq.replacer.cpp.make_cpp()
print('\noriginal:')
eq.show_original()
print("\nparsed tree:")
eq.show_cyk_out()
print('\ncpp:')
eq.replacer.cpp.show_cpp()
print("\nsympy:")
eq.replacer.sympy.make_sympy()
eq.replacer.sympy.show_sympy()
parsing equations (from TeX)
# there is currently two dialect databases for equations:
# 'db_models/data/eqs/tex_dialect.db' for tex
# 'db_models/data/eqs/demo_dialect.db' for wolfram
# default is wolfram
# to change to default tex use commands:
model.change_eqs_to_tex()
# to default wolfram:
model.change_eqs_to_wolfram()
# to some other dialect db:
# (here to tex but manually)
model.save_path("eqs", "db_models/data/eqs/tex_dialect.db")
model.change_dialect_db("eqs")
# this change "eqs" path for all parsers wherever they run.
# show current dialect.db path:
model.get_path_of_dialect_db("eqs")
# if it's ended with tex_dialect.db then tex input used
# remained is same as above:
eq = Equation("U'=a*(\\frac{d^2U}{dx^2}+ \\frac{d^2U}{dy^2})", db=model)
eq.parser.show_patterns()
eq.parser.parse()
print('\noriginal:')
eq.show_original()
print("\nparsed tree:")
eq.show_cyk_out()
print('\ncpp:')
eq.replacer.cpp.show_cpp()
print("\nsympy:")
eq.replacer.sympy.make_sympy()
eq.replacer.sympy.show_sympy()
# find vars:
from proposalsampler.sampling.vars.vars_extractor import Extractor
import proposalsampler.sampling.vars.vars_maps as vms
vars_extractor = Extractor("eqs")
net_vars = vms.get_args(str(["s"]), clause.net_out.copy(), vars_extractor)
print(net_vars)
parsing proposals:
from tokentranslator.db_models.model_main import TokenizerDB
from tokentranslator.env.clause.clause_main import Clause
model = TokenizerDB()
# switch to clauses db:
model.change_dialect_db("cs")
clause = Clause("bilinear(f)=>Eq(f(x,y,)=g(x,y,)+h(x,y,))Eq"
+ "\\where (g: simmetric(g)) \\and (h: simplectic(h))", db=model)
clause.parser.parse()
# there is currently no dialect to translate clause to, so just check it's generated tree:
clause.show_cyk_out()
# this tree will be used for proposal sampling.
# !for equation parser to work don't forget change db back:
model.change_dialect_db("eqs")
# even if You in other session!
# find vars:
from tokentranslator.translator.sampling.vars.vars_extractor import Extractor
import tokentranslator.translator.sampling.vars.vars_maps as vms
vars_extractor = Extractor("cs")
net_vars = vms.get_args(str(["s"]), clause.net_out.copy(), vars_extractor)
print(net_vars)
References:
Parser:
Cocke–Younger–Kasami algorithm: https://en.wikipedia.org/wiki/CYK_algorithm
Acknowledgments:
Used software:
networkx: https://networkx.github.io/
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for tokentranslator-0.0.1.3.3.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 337b9df777473744053ff48f934ab5c13fe336b960be1392c60ff8a380837206 |
|
MD5 | f117987e04a79523ea0504d7803ffc68 |
|
BLAKE2b-256 | 3db163fdcf4812680d689eca885154a5ede24f75463c165b600db58945303e70 |
Hashes for tokentranslator-0.0.1.3.3-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9055b4dbbb7acea9122b96434acc49d410b7db33fad5ea1ac9e24b2c131ff424 |
|
MD5 | 335b0f810efc69400798cd33e744dc1a |
|
BLAKE2b-256 | 3381e8aecdb2bbf68bd55a6160c8dfe919576431d1ffda2c949440723ac3d77a |