Skip to main content

Syncany-SQL

Tests GitHub Repo stars

简单易用的SQL执行引擎。

  • 可在本地运行MySQL语法结构的SQL
  • 支持查询常用mysql、mongodb、postgresql、sqlserver、oracle、elasticsearch、influxdb、clickhouse、sqlite数据库及execl、csv、json和普通文本文件
  • 支持本地临时数据表逻辑做中间结果保存
  • 数据库数据加载使用简单条件过滤及IN条件查询
  • 因由本地完成Join匹配所以支持不同库表、不同主机及不同类型数据库间Join关联查询
  • Group By分组聚合计算及Order By排序也由本地执行,保证数据库安全性
  • 数据写Insert Into支持 ”仅插入 I“、”仅更新 U“、”存在更新否则插入 UI“、”存在更新否则插入其余删除 UDI“、”删除后插入 DI“四种模式
  • 大数据量支持批次执行,有Group By或Having条件过滤自动执行Reduce合并结果
  • 支持流式执行
  • 支持使用常用MySQL函数
  • 支持分区窗口函数查询
  • 支持使用pyeval执行python代码(调用的就是python的eval函数)

安装

pip3 install syncanysql

Docker

docker pull sujin190/syncany-sql

查询Nginx日志

-- 查询访问量最高的三个IP
SELECT seg0 AS ip, COUNT(*) AS cnt FROM `file://data/access.log?sep= ` GROUP BY seg0 ORDER BY cnt DESC LIMIT 3;

查询JSON文件

SELECT
    a.site_id,
    b.name AS site_name,
    IF(c.site_amount > 0, c.site_amount, 0) AS site_amount,
    MAX(a.timeout_at) AS timeout_at,
    MAX(a.vip_timeout_at) AS vip_timeout_at,
    now() as `created_at?`
FROM
    (SELECT
        YIELD_ARRAY(sites) AS site_id,
            IF(vip_type = '2', GET_VALUE(rules, 0, 'timeout_time'), '') AS timeout_at,
            IF(vip_type = '1', GET_VALUE(rules, 0, 'timeout_time'), '') AS vip_timeout_at
    FROM
        `data/demo.json`
    WHERE
        start_date >= '2021-01-01') a
        JOIN
    `data/sites.json` b ON a.site_id = b.site_id
        JOIN
    (SELECT
        site_id, SUM(amount) AS site_amount
    FROM
        `data/orders.json`
    WHERE
        status <= 0
    GROUP BY site_id) c ON a.site_id = c.site_id
GROUP BY a.site_id;

Python API

from syncanysql import ScriptEngine

with ScriptEngine() as engine:
    engine.execute('''
        INSERT INTO `top_ips` SELECT
            ip, cnt
        FROM
            (SELECT
                seg0 AS ip, COUNT(*) AS cnt
            FROM
                `file:///var/log/nginx/access.log?sep= `
            GROUP BY seg0) a
        ORDER BY cnt DESC
        LIMIT 3;
    ''')
    print(engine.pop_memory_datas("top_ips"))

License

Syncany-SQL uses the MIT license, see LICENSE file for the details.

Metadata

Release files for syncanysql 0.1.29

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for syncanysql 0.1.29
File Size Uploaded Provenance
syncanysql-0.1.29.tar.gz 76.1 kB Without provenance Details

Release files / syncanysql-0.1.29.tar.gz

Download URL syncanysql-0.1.29.tar.gz
Size 76.1 kB
Tags Source
SHA-256 checksum
How to use checksums
799863b99b1e7267ae1489fb1de8e9f36f86d71680060e998fd0595214ff6b49
BLAKE2b-256 checksum
How to use checksums
56f29339353aeeed6862640593200b53ca98844253a6f74788fecdb8962eeb0e
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.2.0 CPython/3.13.1

Security for syncanysql 0.1.29

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.1.29 This release

1 release file No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page