llparse 0.1.3

A Parody of llparse written for writing C Parsers with Python

pyllparse

A python parody of the typescript library llparse.

I take no credit for the orginal work done by indutny and the other node-js contributors involved and I was originally very nervous about making this python library that I made public...

Links to the original library

• https://llparse.org
• https://github.com/nodejs/llparse

Unlike the typescript library all 3 of llparse's libraries were combined in this version of the code for the sake of portability... I ended up mentioning how I did this a while back and I also had a concept for a C parser as well but I just didn't like it and I ended up using this instead but I also learned typescript as a bonus and It was alot of fun for me. I don't plan to make this into a real pypi library yet but I also didn't want to take away from the magic of the original source code that I borrowed from...

Looking back on my dead work 2 years later

I had this idea lay somewhat dormant for 2 years and now seeing that I wanted to write a new socks5 server parser writing it in python just made a little bit more sense to me, this way I don't have to stress over typescript or needing to relearn things I learned over 2 years ago. I have now made this project public on pypi for anybody who wanted to dig into it and access the code or generate your own parsers for C.

If your better at typescript. I reccommend sticking to the typescript version, no pressure. This library aims to not compete with llparse but to act as an alternative for users who prefer python over typescript/node-js, simillar to puppeteer or pyppeteer. Personally maybe we should make a rust crate for a rust version or at least someone will do it.

Warnings

• It should be safe enough to generate code and stress-test it but if something doesn't seem right to you, be sure to throw an issue and share a typescript version of something that works in typescript but not in python, this way something unintended by the typescript developers can be solved.
• Some code may look unfinished or may need further internal refactoring/polishing and with the other things I want to go out and do such as contributions to aiohttp and it's other libraries, cyares, winloop, aiothreading, aiocallback & deprecated-params and I also have an irl Part-time Job, ideas and pull requests are welcomed without hesitation. I maintain other projects listed above to prevent myself from experiencing burnout.
• Pytest testsuite has not been made yet. If this is a concern to you, please throw an issue on github. (Pytests and new workflows soon!)
• Some code might be spaghetti-code (because It's been 2 years since I touched much of the code if not at all) e.g. tests module should be moved away from the library into the github repo.
• There's some research portions such as tools to help generate things for cython or tools to make mirrors of llhttp's code such as it's native c code (I was looking to see if it could be theoretically autogenerated). Feel free to use them but do it at your own risk. They maybe incompleted or not throughly stress-tested.

New Features

• Throw me an issue if typescript llparse introduces something new that you want for me or another contributor to try and implement just seeing llparse add new features is exciting to me.

• If you want a feature that typescript llparse doesn't have, be sure to try making a pull request over there as well and not just here, there's a good chance they will appericate you for helping over there too and your helping make llhttp better by doing so. :)

Why Did I Translate llparse to python?

• I wanted to work with a langauge I was more familiar with
• Better educate myself and others on how these great libraries like llhttp are made
• Write faster C code that could do more than just a simple split function or a regex...
• Make it easy for me or someone else to find a problem and solve it in typescript after testing it in python
• Typescript takes 2 commands to run a script with node-js it while python only takes one cutting the time required tremendously...
• The orginal project was MIT licensed.
• I wanted to write my own C Parser tool with llhttp styled callbacks of my own using a language I was the most comfortable with using.
• I didn't like Lemon Parser or Yacc all that much and a good ide for handling them in Visual Studio Code with error checking to my knowlegde does not exist.
• The closest thing I got to what I wanted was a project named NMFU shorthand for no memory for you and even I had problems with writing things using that library...

This was the Code that inspired me to try and make a new pyi writer branch for cython and if it wasn't for llhttp existing as well as it's magical experience I would've never done what I did.

Unlike llparse in typescript, this version has more integrated and experimental features like automatically building the api seen in llhttp and I've added a few other things like the dot compiler from llparse_dot and I also made a brand new cython compiler for it making easy and simple to make pxd files to port your projects to cython

How to use

# The good old http_parser was borrowed from llparse.org to demonstrate this for you :)
from llparse import LLParse

p = LLParse("http_parser")
method = p.node("method")
beforeUrl = p.node("before_url")
urlSpan = p.span(p.code.span("on_url"))
url = p.node("url")
http = p.node("http")

# Add custom uint8_t property to the state
p.property("i8", "method")

# Store method inside a custom property
onMethod = p.invoke(p.code.store("method"), beforeUrl)

# Invoke custom C function
complete = p.invoke(
    p.code.match("on_complete"),
    {
        #  Restart
        0: method
    },
    p.error(4, "`on_complete` error"),
)

method.select(
    {
        "HEAD": 0,
        "GET": 1,
        "POST": 2,
        "PUT": 3,
        "DELETE": 4,
        "OPTIONS": 5,
        "CONNECT": 6,
        "TRACE": 7,
        "PATCH": 8,
    },
    onMethod,
).otherwise(p.error(5, "Expected method"))

beforeUrl.match(" ", beforeUrl).otherwise(urlSpan.start(url))

url.peek(" ", urlSpan.end(http)).skipTo(url)

http.match(" HTTP/1.1\r\n\r\n", complete).otherwise(
    p.error(6, "Expected HTTP/1.1 and two newlines")
)

c = p.build(method)
print(c.c)
open("http_parser.c", "w").write(c.c)
open("http_parser.h", "w").write(c.header)

Video Showcasing this library

• https://youtu.be/YQOzJ2BghQw