sfb
sfb helps SQL testing and estimating the cost of services that depend on scan volume.
Description
- Check SQL syntax
- Estimate query costs for free
- per Run
- per Month
- Replace query parameters automatically
- Be useful on continuous integration
- Use dryrun include Google BigQuery API Client Libraries
Install
$ pip install sfb
Requirements
- Python >= 3.6
- Jupyter Notebook
- Google Colaboratory
- google-cloud-bigquery >= 2.6.1
- pyyaml >= 5.4.1
Usage
Estimate Query Costs
# If runs with no arguments, execute files in './sql/*.sql'.
$ sfb
{
"Succeeded": [
{
"SQL File": "/home/admin/project/sfb_test/sql/covid19_open_data.covid19_open_data.sql",
"Total Bytes Processed": "1.9 GiB",
"Estimated Cost($)": {
"per Run": 0.009414,
"per Month": 0.28242
},
"Frequency": "Daily"
},
{
...
}
],
"Failed": [
{
"SQL File": "/home/admin/project/sfb_test/sql/test_failure_badrequest_01.sql",
"Errors": [
{
"message": "Unrecognized name: names; Did you mean name? at [9:5]",
"domain": "global",
"reason": "invalidQuery",
"location": "q",
"locationType": "parameter"
}
]
},
{
...
}
]
}
# Others
$ sfb -f ./sql/*.sql
$ sfb -q "select * from test;"
$ echo "select * from test;" | sfb | jq
$ find ./sql -type f | sfb
Arguments
$ sfb -h
usage: sfb [-h] [-f [FILE [FILE ...]] | -q QUERY] [-c CONFIG] [-s {BigQuery}]
[-p PROJECT] [-v] [-d]
optional arguments:
-h, --help show this help message and exit
-f [FILE [FILE ...]], --file [FILE [FILE ...]]
sql filepath
-q QUERY, --query QUERY
query string
-c CONFIG, --config CONFIG
config filepath
-s {BigQuery}, --source {BigQuery}
source type
-p PROJECT, --project PROJECT
GCP project
-v, --verbose verbose results
-d, --debug run as debug mode
Directory (Optional)
$ tree .
.
├── config
│ └── sfb.yaml
├── log
│ └── sfb.log (if runs as debug mode)
└── sql
└── [SQL files here]
Configuration
$ cat ./config/sfb.yaml
# Default settings
Globals:
Service: BigQuery
Location: US
Frequency: Daily
QueryFiles:
[your_sql_file_name]:
Frequency: Weekly
Parameters:
- name: ds_start_date
type: DATE
value: '2020-01-01'
- name: ds_end_date
type: DATE
value: '2020-01-31'
...
Type
Name of query parameter type. Select one of types below.
- STRING
- INT64
- FLOAT64
- NUMERIC
- BOOL
- TIMESTAMP
- DATETIME
- DATE
Frequency
For calculating monthly cost estimation.
- Hourly
- (cost_per_run) * 30(days) * 24(h)
- Daily
- (cost_per_run) * 30(days)
- Weekly
- (cost_per_run) * 4(weeks)
- Monthly
- cost_per_run
Metadata
Release files for sfb 0.1.4
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| sfb-0.1.4.tar.gz | 6.3 kB | Without provenance | Details for sfb-0.1.4.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| sfb-0.1.4-py3-none-any.whl | Python 3 | none | any | Details for sfb-0.1.4-py3-none-any.whl |
Total release size: 16.7 kB
Release files / sfb-0.1.4.tar.gz
| Download URL | sfb-0.1.4.tar.gz |
|---|---|
| Size | 6.3 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
a25a054ddd263ffd47e34a38cf27ded5610054115d6eaedd5b28f40c968b906c
|
|
BLAKE2b-256 checksum How to use checksums |
07c435e202b304834d218a48e310b39bd0dc2b0de35d3537477493f388670f72
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.3.0 pkginfo/1.7.0 requests/2.23.0 setuptools/41.2.0 requests-toolbelt/0.9.1 tqdm/4.56.0 CPython/3.7.5
|
Release files / sfb-0.1.4-py3-none-any.whl
| Download URL | sfb-0.1.4-py3-none-any.whl |
|---|---|
| Size | 10.3 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
fcfcf0398350be24f8c61cf353190d21c29b0bc54696b0c9cf6aad66465f00a2
|
|
BLAKE2b-256 checksum How to use checksums |
d2535b38862529ee46b200b3584a5950070a50f49cce2ea8ceb509b5ea328424
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.3.0 pkginfo/1.7.0 requests/2.23.0 setuptools/41.2.0 requests-toolbelt/0.9.1 tqdm/4.56.0 CPython/3.7.5
|
Security for sfb 0.1.4
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.