exonware-xwquery 0.0.1.7

Universal query language for data structures - 50+ operations, 35+ format converters

🚀 xwquery: Universal Query Language for Python

Company: eXonware.com
Author: Eng. Muhammad AlShehri
Email: [e-mail]
Version: 0.0.1.7

🎯 What is xwquery?

xwquery is the universal query language for Python that works across all data structures. Write once, query anywhere - from simple lists to complex graphs, from JSON to XML, from nodes to entities.

Think of it as SQL for everything.

⚡ Quick Start

Installation

pip install exonware-xwquery

Basic Usage

from exonware.xwquery import XWQuery

# Query any data structure
data = {'users': [
    {'name': 'Alice', 'age': 30, 'city': 'NYC'},
    {'name': 'Bob', 'age': 25, 'city': 'LA'},
    {'name': 'Charlie', 'age': 35, 'city': 'NYC'}
]}

# Use familiar SQL-like syntax
result = XWQuery.execute("""
    SELECT name, age 
    FROM users 
    WHERE age > 25 AND city = 'NYC'
""", data)

print(result)
# [{'name': 'Alice', 'age': 30}, {'name': 'Charlie', 'age': 35}]

🎯 Perfect For:

• 📊 Data Querying - Query any Python data structure with SQL-like syntax
• 🔄 Format Conversion - Convert between 35+ query formats (SQL, GraphQL, Cypher, etc.)
• 🔀 Data Transformation - Transform and filter data with powerful operations
• 🔗 Universal Interface - One query language for nodes, data, schemas, and entities
• 🚀 Performance - Optimized execution for different data structure types

🚀 Key Features

50 Query Operations

✅ Core CRUD: SELECT, INSERT, UPDATE, DELETE, CREATE, ALTER, DROP
✅ Filtering: WHERE, FILTER, BETWEEN, LIKE, IN, HAS, TERM, RANGE
✅ Aggregation: SUM, COUNT, AVG, MIN, MAX, GROUP BY, HAVING
✅ Graph Operations: MATCH, PATH, OUT, IN_TRAVERSE, RETURN
✅ Advanced: JOIN, UNION, WITH, MERGE, WINDOW, PIPE

30+ Grammar-Based Formats (Powered by xwsyntax)

✅ SQL: Full SQL parsing and generation with bidirectional support
✅ Graph Queries: Cypher, Gremlin, SPARQL, GraphQL, GQL
✅ Document Queries: MongoDB (MQL), Elasticsearch, CQL
✅ Time Series: PromQL, Flux, LogQL
✅ Data Queries: JQ, JMESPath, JSONiq, XPath, XQuery, JSON
✅ Others: Datalog, LINQ, N1QL, PartiQL, HiveQL, HQL, Pig, KQL
✅ All with parse + generate + validation support!

🆕 Database-Grade Query Optimization

✅ Query Planning: Logical and physical execution plans
✅ Cost Estimation: Smart cost-based optimization decisions
✅ Statistics Management: Table/column statistics for better plans
✅ Optimization Rules: Predicate pushdown, index selection, join reordering
✅ Query Caching: LRU cache with TTL and memory management
✅ Multiple Optimization Levels: From BASIC to AGGRESSIVE

Type-Aware Execution

# Automatically optimizes based on data structure
linear_data = [1, 2, 3, 4, 5]
tree_data = {'a': 1, 'b': 2, 'c': 3}
graph_data = {'nodes': [...], 'edges': [...]}

# Same query, optimized execution for each type!
XWQuery.execute("SELECT * WHERE value > 2", linear_data)  # Sequential scan
XWQuery.execute("SELECT * WHERE key BETWEEN 'a' AND 'c'", tree_data)  # Tree traversal
XWQuery.execute("MATCH (n)-[r]->(m)", graph_data)  # Graph algorithm

📚 XWQuery Script Language

Comprehensive Syntax

-- Data retrieval
SELECT name, email, age FROM users WHERE age >= 18;

-- Aggregation
SELECT 
    department,
    COUNT(*) as employee_count,
    AVG(salary) as avg_salary
FROM employees
GROUP BY department
HAVING avg_salary > 50000;

-- Graph pattern matching
MATCH (u:User)-[:FRIENDS_WITH]->(f:User)
WHERE u.age > 25
RETURN u.name, f.name;

-- Data transformation
PIPE users
|> FILTER age > 18
|> EXTEND full_name = CONCAT(first_name, ' ', last_name)
|> PROJECT user_id, full_name, email
|> ORDER BY full_name;

-- Complex joins
SELECT u.name, o.total_amount
FROM users u
INNER JOIN orders o ON u.id = o.user_id
WHERE o.order_date >= '2024-01-01';

-- Window functions
SELECT 
    product_id,
    price,
    AVG(price) OVER (PARTITION BY category) as avg_category_price
FROM products;

Format Conversion

# Parse SQL, convert to GraphQL
sql_query = "SELECT id, name FROM users WHERE age > 25"
graphql = XWQuery.convert(sql_query, from_format='sql', to_format='graphql')
# Result: "query { users(filter: {age: {gt: 25}}) { id name } }"

# Parse Cypher, convert to SQL
cypher_query = "MATCH (u:User)-[:WORKS_AT]->(c:Company) RETURN u.name, c.name"
sql = XWQuery.convert(cypher_query, from_format='cypher', to_format='sql')
# Result: "SELECT u.name, c.name FROM users u JOIN companies c ON ..."

# Universal intermediate representation
any_query = XWQuery.parse(query_string)  # Parses to actions tree
target_format = any_query.to_format('mongodb')  # Convert to any format

🔄 Integration with exonware Stack

Query Nodes (xwnode)

from exonware.xwnode import XWNode
from exonware.xwquery import XWQuery

node = XWNode.from_native({'users': [...]})
result = XWQuery.execute("SELECT * FROM users WHERE active = true", node)

Query Data (xwdata)

from exonware.xwdata import XWData
from exonware.xwquery import XWQuery

# Load, query, convert format
data = XWData.load('users.json')
filtered = XWQuery.execute("SELECT * WHERE age > 18", data)
filtered.save('adults.xml')  # Save in different format!

Query Entities (xwentity)

from exonware.xwentity import XWEntity
from exonware.xwquery import XWQuery

class User(XWEntity):
    name: str
    age: int
    email: str

# Schema-validated queries
users = XWQuery.execute("SELECT * FROM User WHERE age > 18")
# ✅ Validates 'age' exists in schema
# ✅ Ensures age is int type
# ✅ Returns typed User entities

🎓 Examples

Example 1: Simple Filtering

products = [
    {'id': 1, 'name': 'Laptop', 'price': 999, 'category': 'Electronics'},
    {'id': 2, 'name': 'Mouse', 'price': 29, 'category': 'Electronics'},
    {'id': 3, 'name': 'Desk', 'price': 299, 'category': 'Furniture'},
]

result = XWQuery.execute("""
    SELECT name, price 
    FROM products 
    WHERE category = 'Electronics' AND price < 500
""", products)
# [{'name': 'Mouse', 'price': 29}]

Example 2: Aggregation

orders = [
    {'user_id': 1, 'amount': 100},
    {'user_id': 1, 'amount': 200},
    {'user_id': 2, 'amount': 150},
    {'user_id': 2, 'amount': 300},
]

result = XWQuery.execute("""
    SELECT 
        user_id, 
        COUNT(*) as order_count,
        SUM(amount) as total_spent,
        AVG(amount) as avg_order
    FROM orders
    GROUP BY user_id
    HAVING total_spent > 200
""", orders)
# [
#   {'user_id': 1, 'order_count': 2, 'total_spent': 300, 'avg_order': 150},
#   {'user_id': 2, 'order_count': 2, 'total_spent': 450, 'avg_order': 225}
# ]

Example 3: Graph Traversal

graph = {
    'nodes': [
        {'id': 1, 'type': 'User', 'name': 'Alice'},
        {'id': 2, 'type': 'User', 'name': 'Bob'},
        {'id': 3, 'type': 'Post', 'title': 'Hello World'}
    ],
    'edges': [
        {'from': 1, 'to': 2, 'type': 'FRIENDS_WITH'},
        {'from': 1, 'to': 3, 'type': 'AUTHORED'}
    ]
}

result = XWQuery.execute("""
    MATCH (u:User)-[:FRIENDS_WITH]->(friend:User)
    WHERE u.name = 'Alice'
    RETURN u.name, friend.name
""", graph)
# [{'u.name': 'Alice', 'friend.name': 'Bob'}]

Example 4: Format Conversion

# SQL to MongoDB
sql = "SELECT name, email FROM users WHERE age > 25"
mongo = XWQuery.convert(sql, to_format='mongodb')
# db.users.find({age: {$gt: 25}}, {name: 1, email: 1})

# GraphQL to SQL  
graphql = "query { users(filter: {active: true}) { id name email } }"
sql = XWQuery.convert(graphql, to_format='sql')
# SELECT id, name, email FROM users WHERE active = true

# Cypher to SPARQL
cypher = "MATCH (a:Person)-[:KNOWS]->(b:Person) RETURN a.name, b.name"
sparql = XWQuery.convert(cypher, to_format='sparql')
# SELECT ?aName ?bName WHERE { ?a rdf:type :Person . ?a :KNOWS ?b . ... }

🏗️ Architecture

Query Execution Flow

User Query String
       ↓
  [Parser] - Parse to Actions Tree
       ↓
[Capability Checker] - Validate operation compatibility
       ↓
 [Executor Registry] - Get appropriate executor
       ↓
[Type-Aware Execution] - Optimized for data structure type
       ↓
    Result

Supported Node Types

• LINEAR: Arrays, lists, queues, stacks
• TREE: Hash maps, B-trees, AVL trees, tries
• GRAPH: Adjacency lists, graph structures
• MATRIX: Bitmaps, sparse matrices
• HYBRID: Combined structures

Each query operation automatically adapts to the node type for optimal performance.

📖 Documentation

• Complete Query Syntax - All 50 operations detailed
• Query Optimization Guide - 🆕 Database-grade optimization
• Format Converters - 35+ format conversion guide
• API Reference - Complete API documentation
• Examples - Practical usage examples
• Integration Guide - Using with xwnode, xwdata, xwentity

🚀 Project Phases

xwquery follows a structured 5-phase development approach designed to deliver enterprise-grade functionality.

Current Phase: 🧪 Version 0 - Experimental Stage

• Focus: Core query engine, format converters, execution optimization
• Status: 🟢 ACTIVE - Foundation complete with 50 operations

Development Roadmap:

• Version 1 (Q1 2026): Production Ready - Performance optimization
• Version 2 (Q2 2026): Query Optimization - Smart query planning
• Version 3 (Q3 2026): Distributed Queries - Parallel execution
• Version 4 (Q4 2026): Mars Standard Implementation - Universal interoperability

🔧 Development

# Install in development mode
pip install -e .

# Run tests
python tests/runner.py

# Run specific test types
python tests/runner.py --core
python tests/runner.py --unit
python tests/runner.py --integration

🌟 Why xwquery?

Before xwquery:

# Different syntax for different data structures
list_result = [x for x in data if x['age'] > 25]  # List comprehension
dict_result = {k: v for k, v in data.items() if v > 25}  # Dict comprehension
df_result = df[df['age'] > 25]  # Pandas
collection.find({'age': {'$gt': 25}})  # MongoDB
# ... and so on

With xwquery:

# One syntax for everything
result = XWQuery.execute("SELECT * WHERE age > 25", data)
# Works with: lists, dicts, nodes, dataframes, graphs, entities, etc.

Key Benefits:

1. Universal Interface - One query language for all data structures
2. Format Agnostic - Query JSON, save as XML, load CSV, export as YAML
3. Type-Aware - Automatically optimizes for data structure type
4. Production-Grade - Built on proven patterns and best practices
5. Extensible - Easy to add custom operations and formats
6. Performance - Optimized execution for each node type

🤝 Contributing

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Add tests
5. Run the test suite
6. Submit a pull request

📄 License

MIT License - see LICENSE file for details.

---

Built with ❤️ by eXonware.com - Making universal data querying effortless

---

🔗 Part of the exonware Ecosystem

• xwsystem - Core system utilities
• xwnode - Node-based data structures
• xwquery - Universal query language ⭐ You are here
• xwdata - Format-agnostic data manipulation
• xwschema - Schema validation
• xwaction - Action definitions
• xwentity - Entity management
• xwstorage - Storage abstraction
• xwbase - Firebase-like backend