generate selector schemas classes from dsl-like language based on python
Project description
Selector schema codegen
ssc_codegen - generator of parsers for various programming languages (for html priority) using python-DSL configurations with built-in declarative language.
Designed to port parsers to various programming languages and libs
Motivation
interesting in practice write DSL-like language- decrease boilerplate code for web-parsers
- write once - convert to other mainstream http parser libs
- minimal operations for easy add another libs and languages in future
- include css, xpath, attributes operations, regex, minimal string formatting operations
- pre validate css/xpath queries and logic before generate code
- standardisation: generate classes with minimal dependencies and documented parsed signature
Install
pipx (recommended for CLI usage)
pipx install ssc_codegen
pip
pip install ssc_codegen
Supported libs and languages
language | lib | xpath | css | formatter |
---|---|---|---|---|
python | bs4 | NO | YES | black |
- | parsel | YES | YES | - |
- | selectolax (modest) | NO | YES | - |
- | scrapy (based on parsel, but class init argument - Response) | YES | YES | - |
dart | universal_html | NO | YES | dart format |
Quickstart
see example and read code with comments
Recommendations
- usage css selector: they can be guaranteed converted to xpath (if target language not support CSS selectors)
- usage simple operations for more compatibility other libraries.
- Some libraries may not fully support selector specifications
- for example,
#product_description+ p
selector inparsel
works fine, but not works inselectolax
,dart
libs
- there is a xpath to css converter for simple queries without guarantees of functionality.
For example, in css there is no analogue of
contains
from xpath, etc.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
ssc_codegen-0.2.0.tar.gz
(18.8 kB
view hashes)
Built Distribution
Close
Hashes for ssc_codegen-0.2.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 109a53dd2a50980e87483d1c5c9a326416233d7fc574f25dfd803f86f88255e7 |
|
MD5 | 7e893829d5495262a0030f227ee52121 |
|
BLAKE2b-256 | 3adce4bd662c3a5c6b81075f11621ac83af4102ccfb4078d17d338924bf390b3 |