No project description provided
Project description
Just Your Average Function
💡 Don't forget to check out the docs for more in-depth info.
Look at this innocent-looking piece of code:
import jyafn as fn
@fn.func
def reduce_sum(mat: fn.tensor[2, 2]) -> fn.scalar:
return np.sum(mat)
I know: it looks a bit funny but what if I told you that
- This compiles to machine code.
- You can still call it as a regular Python function.
- You can export it, load it and call it from Go.
Neat, huh? It's basically tf.function
+ onnx
in a single package!
A quick example
Let's write a silly function in jyafn
:
@fn.func
def a_fun(a: fn.scalar, b: fn.scalar) -> fn.scalar:
return 2.0 * a + b + 1.0
It's so silly that if you call it like you normally would, a_fun(2, 3)
, you get what you expect, 8
. But that is not the fun part. The fun part is that you can export this function to a file:
with open("a_fun.jyafn", "wb") as f:
f.write(a_fun.dump())
And now you can pass this file anywhere and it will work. Let's call it, for example, from Go:
// Read exported data:
code, err := os.ReadFile("a_fun.jyafn")
if err != nil {
log.Fatal(err)
}
// Load the function:
fn, err := jyafn.LoadFunction(code)
if err != nil {
log.Fatal(err)
}
// Call the function:
result, err := jyafn.Call[float64](
fn,
struct {
a float64
b float64
}{a: 2.0, b: 3.0},
)
if err != nil {
log.Fatal(err)
}
fmt.Println(result, "==", 8.0)
How to use it
For all cases, unfortuately you will need gcc
or clang
installed (they are not build dependencies!), since we need an assembler and a linker to finish QBE's job. Also, jyafn
is guaranteed not to work in Windows. For your specific programming language, see below:
Python
Download from GitHub
You can use the following snippet of code to install jyafn
from GitHub releases, using the gh
GitHub CLI:
PY=cp311 && \
V="0.1.0" && \
LATEST=$(gh release list -R FindHotel/jyafn | head -n1 | awk '{print $1}') && \
FILE=jyafn_python-$V-$PY-$PY-manylinux_2_17_x86_64.manylinux2014_x86_64.whl && \
rm -f $FILE && \
gh release download -R FindHotel/jyafn -p $FILE && \
pip -m pip install --force-reinstall $FILE
Remeber to substitute foryour python version. In the above example, we are using cp311
(Python 3.11).
Build from source
Clone the repo, then
make install
This should do the trick. You can set the specific target Python version like so:
make install py=3.12
The default version is 3.11 at the moment.
At the moment, the Python version depends on the Rust compiler to work. It will compile jyafn
from source. As such, you will need cargo
, Rust's package manager, as well as maturin
, the tool for building Python packages from Rust code. Maturin can be easily installed with pip
:
pip install maturin
Go
You can use this as a Go module:
import "github.com/viodotcom/jyafn/jyafn-go/pkg/jyafn"
Please note that this package depends on CGO under the hood.
FAQ
There is something going on!
Yes, there is definitely something going on. What you see is basically a mini-JIT (just-in-time compiler). Your Python instructions (add this! multiply that!) are recorded in a computational graph, which is then compiled to machine code thanks to QBE, which does all the heavy-lifting, "compilery stuff". This code is exposed as a function pointer in the running program.
Isn't loading arbitrary code just a security hazard?
Yes, but the code produced by jyafn
is far from arbitrary. First, it's pure: it doesn't cause any outside mutations of any kind. Second, since it is based on a computational graph, which is acyclic, it is guaranteed to finish (and the size of the code limits how much time it takes to run). Third, no machine code is exchanged: that code is only valid for the duration of the process in which it resides. It's the computational graph and not the code that is exchanged.
So, I can just write anything in it and it will work?
No, far from it:
- Your python function can only have constant-sized loops.
- The libraries you are using must be able to work with generic Python objects (i.e., support the magic dunder methods,
__add__
,__sub__
, etc...). Thankfully,numpy
does just that for itsndarray
s. if-else
is still a bit wonky (you can use thechoose
method instead). This can be solved in the future, if there is demand.- By now, support is focused on DS uses. So, floats and bools are supported, but not other primitive types.
- Once in a while, a new function out of a litany of them, might not be supported. However, implementing it is very easy.
Is it faster than pure Python?
You bet! There is a benchmark in ./jyafn-python/tests/simple_graph.py
at which you can take a look. One example showed a 10x speedup over vanilla CPython.
Which programming languages are supported?
By now, Go, Rust, Python and C. You can use the cjyafn
library to port jyafn
to your language.
What is the current status of the project?
It's mature enough for a test. Guinea pigs wanted!
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
Built Distributions
Hashes for jyafn-0.1.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | df7d1b9ac29045a817e1e93f06c2db2a9b64755c740aab784154f65321496886 |
|
MD5 | 0e124dedbb327470609b54fe213c51bb |
|
BLAKE2b-256 | 149777afcff394a678ec31467a4ded0794ed70997812655f1312fd2ac208dad4 |
Hashes for jyafn-0.1.1-cp312-cp312-macosx_11_0_arm64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 906fb1e65cbd12cede3a92a841d6c2283d93675d256178e7b304a88c1f83da3f |
|
MD5 | 2aa6541220105851d57a9bb8f77580dc |
|
BLAKE2b-256 | 7ee658720196c5f5f8b8a2f4d5000310a5a40bc4e83fca6f553529ae8eb455ad |
Hashes for jyafn-0.1.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 59371416315909ab8352a5035cf2b29e2a81aacd2a86edb0c76914d9fcd4ba24 |
|
MD5 | e266e4e9a792e4b5314a9571d4a65d6a |
|
BLAKE2b-256 | 18a72ecfdd791c873a01b3a0c87b02ffd1dabab6dfe1fde8d6e30c48b448b3c0 |
Hashes for jyafn-0.1.1-cp311-cp311-macosx_11_0_arm64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e9785d0ab5e7f5d2f7e1ba9054a9b19633f45fe08b5164182e0a94eb0f60c55a |
|
MD5 | ab244aeeb08e0d445832337b99e6eda8 |
|
BLAKE2b-256 | 41c9b87751c19ba1b6b8f396b9e652d4e043ab845f00698a426bfe5de4388512 |
Hashes for jyafn-0.1.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c3200f4a81c7821b7244160155760ae97e41dff9b54ae9a0de724b439ded10f3 |
|
MD5 | 7aa1d75bf2aeaf9251e0a18f370a3253 |
|
BLAKE2b-256 | 10db1f6494d78cb23a70c9a2295f2ce5c9ee1b8bb6deaf955d838a114d9a2c24 |
Hashes for jyafn-0.1.1-cp310-cp310-macosx_11_0_arm64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 965cf178d1bb5b59250124c244bf61c4d496f69617a9b05d6baec462e23b0304 |
|
MD5 | f93b4385d93781d538c5c292c03d9fbc |
|
BLAKE2b-256 | 1d186a9e50305087e283933cd8a09b6e37370162a9c047ceb1bf5129c83bc288 |
Hashes for jyafn-0.1.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 5857b0b0dbd96056773116bcf48555719cf7534dabec4b24c4a032ca7f8d8204 |
|
MD5 | 431652d03dd95d5520f9cb2785b8de73 |
|
BLAKE2b-256 | ddb079a63e4094a2d5f591f465f6d27fe8e6d5bcf30a3c25cdd67cc97a5e0587 |
Hashes for jyafn-0.1.1-cp39-cp39-macosx_11_0_arm64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a8ae58fbd68dc075e4da883483d8aa1736582d49b0def1af7edeaa342fc09af2 |
|
MD5 | 27eeed3afd75a883813538983ee401aa |
|
BLAKE2b-256 | 20ffa81029905a15398d04e6de2ac39bd543bad766800e93bbf2ecde772f18ad |
Hashes for jyafn-0.1.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | cf90090c1256a21a47b6ec31168435b1fd3e76c685b4c085fa7ab9044f2a7aed |
|
MD5 | 62488c312a48e9e2444ddc73792faf96 |
|
BLAKE2b-256 | ffcd0ce5bf01e777afb6c1e5954c4daabd981aa44c502c8bc3a3c71245a957a5 |
Hashes for jyafn-0.1.1-cp38-cp38-macosx_11_0_arm64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 6d08a57b0e34595de3a141e662b2837863cb1009e973112bcaebbd992a164a8f |
|
MD5 | 64aa268e4a7768a16fe9c692a4c5c8f1 |
|
BLAKE2b-256 | f974d9e5966cb94258ab4021b42e66ab59f9cdd588e94b2d4f9408a13416d18f |
Hashes for jyafn-0.1.1-1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 6346e0b5f278ecdca516f8ec46614592ea594d411dc690ec6794ce6deb2ef35d |
|
MD5 | cbc6cc46d10314985fac1b2128840c73 |
|
BLAKE2b-256 | 16d8d10f57139ad50df328df9e1f17442e457464a1fbfdc04a65e6111ba4a243 |
Hashes for jyafn-0.1.1-1-cp312-cp312-macosx_11_0_arm64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 437c81721cb2526970ab6a025e9cedae049b95e834e8bad3ceda090b3d602f1e |
|
MD5 | 31f5315d9c7dee1ed8c5144d8b7d7e0a |
|
BLAKE2b-256 | 3ba7d928fc70adf4a9429817eda18ae05016c2b6dac4b3ba8c218db7b42dba0f |
Hashes for jyafn-0.1.1-1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | bbca476c3f9fb102dd6e9da1019ad1bd47832bde8b8c58e243e548e13ac0a27d |
|
MD5 | c248ec24423c13c4de340790f4f430bc |
|
BLAKE2b-256 | 6e52780b9d57a5bd0f1ec2ca6c36527ff9e84d0d0d9f178276ad97aacbde8d7e |
Hashes for jyafn-0.1.1-1-cp311-cp311-macosx_11_0_arm64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | dcae8b88d5aafb2ef1d9b4e169b277a792b25c4be5863179ef8762c864e5f23f |
|
MD5 | 01a878788311c1ba080d0ebd0365d131 |
|
BLAKE2b-256 | 8397e3d7787cb3683f2e0cb93bcda766acc5060c87607dc5eeb2cd662483c45e |
Hashes for jyafn-0.1.1-1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f010c9fd521f9547002604ea372cf9aaece442f96cf639c8a7d012aeb543b72a |
|
MD5 | c2155622c4ba2b0ff6aff3ea963a5883 |
|
BLAKE2b-256 | 5e5eb48b48585e8fc17c0b3c127e89cd2390141764cd422f3ee38b7e6d311145 |
Hashes for jyafn-0.1.1-1-cp310-cp310-macosx_11_0_arm64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | abedd381bed94ee5754e76b130d2999787ff99dcdf6ea8ab19839da9c5a9902c |
|
MD5 | 35d2e4bbb8d820c35b7cbc4a5396d726 |
|
BLAKE2b-256 | 5b745211b964f439c15fe6beefb998402fa971b360e276b4527a5278e27b54d4 |
Hashes for jyafn-0.1.1-1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 954c3fd674804a4c1a716693e503eee47028708cacb41ccb9ae4d650be01ac00 |
|
MD5 | f76fbbb667d10260cb53ebc081fc0b6e |
|
BLAKE2b-256 | 05f0b1ec4538bdd515a496a5c87803d5c34a9cd927e7bd43c8c775b504f67aa6 |
Hashes for jyafn-0.1.1-1-cp39-cp39-macosx_11_0_arm64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 2f0670a4aec47b9120a21a83e80fdb8b21141aeb47f1214c4af49429bbfbbd5f |
|
MD5 | 3ec62915c62aba37bf87fcc3e2a04f85 |
|
BLAKE2b-256 | be4e1f7541dc3113f7f47da24f3715381987c7d61435de06c845944dfb8a919c |
Hashes for jyafn-0.1.1-1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 631f569d802f45d2455db3b785bec823ec1fc8965d26f3879cb22920ff1d932d |
|
MD5 | 13ac0207584f4dc5cba1dd16a6bc5f78 |
|
BLAKE2b-256 | da15b274c772d8582c1a47939fb38e775a1207ba108505c0537fe7a5dde13ed8 |
Hashes for jyafn-0.1.1-1-cp38-cp38-macosx_11_0_arm64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | dfe2f0db8051020120218dd0b764fe15027154600ec2835825f95b3181b9f1de |
|
MD5 | 377d4c059dcaffbc1fee71b8ed86e271 |
|
BLAKE2b-256 | 75349ce03a0429def75af039a6be8a96da93a05d943e30b6fa65882b0302c209 |