title: Tellaro Query Language (TQL) class: repo-spec audience: in-repo status: current owner: tql-team last_verified: '2026-08-10' verification_note: 'CONTRADICTED: listed geoip as a mutator name (Python has geoip_lookup/geo only), claimed stdev is a std alias when neither stdev nor stddev parses in either runtime, pointed at two test paths that moved under unit/ and integration/, and recommended black/flake8 which are not dependencies. All 26 example queries re-parsed; the opensearch extra and keyword-only execute_opensearch calls verified correct.' verified_by: doc-refresh@2026-08-10 source_refs:

• tellaro-query-language:src/tql/init.py
• tellaro-query-language:pyproject.toml
• tellaro-query-language:src/tql/mutators/init.py
• tellaro-query-language:src/tql/parser_components/grammar.py
• tellaro-query-language:tql/src/parser/grammar.pest

Tellaro Query Language (TQL)

A flexible, human-friendly query language for searching and filtering structured data across files, databases, and search engines.

TQL provides a unified, readable syntax for expressing complex queries that works seamlessly with:

• Files: Query JSON, JSONL, CSV files directly with CLI or Python API
• OpenSearch/Elasticsearch: Convert TQL to DSL queries automatically
• In-Memory Data: Filter Python dictionaries and lists
• Statistical Analysis: Built-in aggregations and grouping

# Query JSON files directly
results = tql.query("logs.jsonl", "status = 200 AND response_time > 500")

# Query OpenSearch with automatic DSL translation
results = tql.execute_opensearch(client, "events-*",
    "user.role = 'admin' AND timestamp > '2024-01-01'")

# Aggregate data with stats
results = tql.query("sales.json", "region = 'west' | stats sum(revenue) by product")

🚀 Quick Start

Installation

# Install from PyPI (Python package)
pip install tellaro-query-language

# Install with OpenSearch support
pip install tellaro-query-language[opensearch]

# Or install Rust CLI (300x faster for large files)
cargo install tellaro-query-language

Query Files with CLI

TQL includes a blazing-fast command-line interface for querying files:

# Query JSON/JSONL files
tql 'status = "active"' users.json
tql 'age > 25 AND city = "NYC"' data.jsonl

# Query CSV files (auto-detects headers)
tql 'price > 100 AND category = "electronics"' products.csv

# Statistical aggregations
tql '| stats count() by status' events.jsonl
tql 'status = 200 | stats average(response_time) by endpoint' logs.jsonl

# Process folders recursively
tql 'level = "ERROR"' logs/ --pattern "*.jsonl" --recursive

# Pipe data from stdin
cat data.jsonl | tql 'score > 90'

Performance: The Rust CLI processes 50MB files in milliseconds vs. seconds for Python implementations.

Query Files with Python API

from tql import TQL

tql = TQL()

# Query JSON files directly
results = tql.query("data.json", "user.role = 'admin' AND status = 'active'")

# Query with field transformations
results = tql.query("logs.jsonl", "email | lowercase contains '@example.com'")

# Statistical analysis
results = tql.query("sales.json", "| stats sum(revenue), avg(price) by category")

Query In-Memory Data

from tql import TQL

tql = TQL()
data = [
    {'name': 'Alice', 'age': 30, 'city': 'NYC'},
    {'name': 'Bob', 'age': 25, 'city': 'LA'},
    {'name': 'Charlie', 'age': 35, 'city': 'NYC'}
]

# Simple queries
results = tql.query(data, 'age > 27')
# Returns: [{'name': 'Alice', 'age': 30, 'city': 'NYC'},
#           {'name': 'Charlie', 'age': 35, 'city': 'NYC'}]

# Logical operators
results = tql.query(data, 'age >= 30 AND city = "NYC"')
# Returns: [{'name': 'Alice', ...}, {'name': 'Charlie', ...}]

# Field transformations
results = tql.query(data, 'name | lowercase = "alice"')

Query OpenSearch

from opensearchpy import OpenSearch
from tql import TQL

# Initialize OpenSearch client
client = OpenSearch(
    hosts=['localhost:9200'],
    http_auth=('admin', 'admin'),
    use_ssl=True,
    verify_certs=False
)

# Initialize TQL with field mappings
mappings = {
    'user.name': {'type': 'keyword'},
    'user.email': {'type': 'text'},
    'timestamp': {'type': 'date'}
}
tql = TQL(mappings)

# Execute queries with automatic DSL translation
results = tql.execute_opensearch(
    opensearch_client=client,
    index='users-*',
    query='user.name = "admin" AND status = "active"'
)

# Complex queries with mutators and post-processing
results = tql.execute_opensearch(
    opensearch_client=client,
    index='logs-*',
    query='email | lowercase contains "@example.com" AND level = "ERROR"'
)
# TQL automatically applies post-processing for mutators

🎯 Core Features

🔍 Unified Query Syntax

Write one query, run it anywhere - files, OpenSearch, in-memory data:

# Same query works everywhere
query = 'status = "active" AND age > 25'

# Query files
tql.query("users.json", query)

# Query OpenSearch
tql.execute_opensearch(client, "users-*", query)

# Query Python data
tql.query(python_list, query)

📁 First-Class File Support

Query files as easily as databases:

# JSON/JSONL files
tql.query("logs.jsonl", "level = 'ERROR'")

# CSV files with automatic header detection
tql.query("products.csv", "price > 100 AND stock < 10")

# Folders with glob patterns
tql.query("logs/2024/*.jsonl", "status = 500", recursive=True)

# Streaming for large files (CLI)
$ tql 'status = 200' large-file.jsonl  # Processes without loading to memory

🔄 25+ Field Mutators

Transform data inline before comparison:

# String transformations
'email | lowercase | trim = "admin@example.com"'
'name | uppercase = "JOHN DOE"'

# Encoding/decoding
'data | b64decode | lowercase = "secret"'
'password | md5 = "5f4dcc3b5aa765d61d8327deb882cf99"'

# Network operations
'ip | is_private = true'           # Check if IP is RFC 1918
'domain | defang = "hxxp://evil[.]com"'  # Security analysis

# DNS lookups
'hostname | nslookup contains "8.8.8.8"'

# GeoIP enrichment
'ip | geoip.country_name = "United States"'

# List operations
'scores | avg > 80'
'prices | sum between [100, 500]'

📊 Statistical Aggregations

Analyze data with built-in stats functions:

# Simple aggregations
tql.query(data, '| stats count(), sum(revenue), avg(price)')

# Grouped analysis
tql.query(data, '| stats count() by status, region')

# Top N analysis
tql.query(data, '| stats sum(sales, top 10) by product')

# Combined filtering and stats
tql.query(data, 'region = "west" | stats avg(revenue) by category')

🔧 OpenSearch Integration

Seamless OpenSearch/Elasticsearch integration:

• Automatic DSL Translation: TQL queries → OpenSearch Query DSL
• Smart Field Mapping: Handles keyword vs text fields automatically
• Post-Processing: Apply mutators that OpenSearch can't handle
• Pagination Support: Handle large result sets efficiently

# TQL handles field mapping automatically
mappings = {'user.email': {'type': 'text', 'fields': {'keyword': {'type': 'keyword'}}}}
tql = TQL(mappings)

# Exact match uses .keyword automatically
query = 'user.email = "admin@example.com"'  # Uses user.email.keyword

# Mutators trigger post-processing when needed
query = 'user.email | lowercase contains "admin"'  # Post-processes results

📖 Syntax Guide

Comparison Operators

# Equality
'status = "active"'           # Exact match (alias: eq)
'status != "inactive"'        # Not equal (alias: ne)

# Numeric comparisons
'age > 25'                    # Greater than
'age >= 18'                   # Greater or equal
'age < 65'                    # Less than
'age <= 100'                  # Less or equal

# String operations
'email contains "@example.com"'      # Substring
'name startswith "John"'            # Prefix
'filename endswith ".pdf"'          # Suffix
'email regexp "^\\w+@\\w+\\.\\w+$"'  # Regex

# Range and membership
'age between [18, 65]'              # Inclusive range
'status in ["active", "pending"]'   # Value in list
'priority range [1, 5]'             # Alias for between

# Existence checks
'field exists'                      # Field is present
'field not exists'                  # Field is missing
'field is null'                     # Field is null
'field is not null'                 # Field is not null

# Network operations
'ip cidr "192.168.0.0/16"'          # IP in CIDR range

Logical Operators

# AND (all conditions must be true)
'age > 25 AND city = "NYC"'
'status = "active" AND role in ["admin", "moderator"]'

# OR (any condition must be true)
'city = "NYC" OR city = "LA"'
'status = "admin" OR role = "superuser"'

# NOT (negates condition)
'NOT (age < 18)'
'NOT status = "deleted"'

# Complex expressions with parentheses
'(age > 25 AND city = "NYC") OR (status = "vip" AND score > 90)'

Collection Operators

# ANY - at least one array element matches
'ANY tags = "premium"'
'ANY user.roles = "admin"'

# ALL - every array element matches
'ALL scores >= 80'
'ALL status = "active"'

# NONE - no array elements match
'NONE flags = "spam"'
'NONE violations.severity = "critical"'

Nested Field Access

# Dot notation for nested objects
'user.profile.email contains "@example.com"'
'metadata.tags.priority = "high"'

# Array indexing
'tags[0] = "urgent"'
'history[5].status = "completed"'

Field Mutators Reference

String Mutators

• lowercase, uppercase - Case conversion
• trim - Remove whitespace
• split(delimiter) - Split string into array
• length - Get string length
• replace(old, new) - Replace substring

Encoding Mutators

• b64encode, b64decode - Base64 encoding/decoding
• urldecode - URL decode
• hexencode, hexdecode - Hex encoding/decoding
• md5, sha256 - Cryptographic hashing

Network/Security Mutators

• refang - Convert defanged indicators (hxxp → http)
• defang - Defang URLs for safe display
• is_private - Check if IP is private (RFC 1918)
• is_global - Check if IP is globally routable

DNS Mutators

• nslookup - Resolve hostname to IP addresses

GeoIP Mutators

• geoip_lookup (alias geo) - Enrich IP with geolocation data. geoip alone is not a Python mutator name — ip | geoip() raises TQLSyntaxError. (The Rust implementation does accept geoip as a third alias; prefer geoip_lookup for queries that run under both.)
• Returns: geo.country_name, geo.city_name, geo.location, geo.continent_code, etc.

List Mutators

• any, all - Boolean aggregations
• avg, average - Calculate mean
• sum - Calculate sum
• min, max - Find min/max values

📊 Statistical Aggregations

TQL includes a powerful stats engine for data analysis:

Available Functions

# Counting
'| stats count()'              # Count all records
'| stats count(field)'         # Count non-null values
'| stats unique_count(field)'  # Count distinct values

# Numeric aggregations
'| stats sum(revenue)'         # Calculate sum
'| stats avg(price)'           # Calculate average (aliases: average, mean)
'| stats min(age), max(age)'   # Find min/max values
'| stats median(score)'        # Calculate median

# Statistical measures
'| stats std(values)'          # Standard deviation
# Full name: '| stats standard_deviation(values)'. Note that neither `stdev`
# nor `stddev` works in either runtime, despite `stddev` appearing in the Rust
# grammar -- the PEG matches the shorter `std` first, so the longer spelling is
# unreachable. The same shadowing affects `percentiles`, `percentile_rank(s)`
# and `pct_rank(s)` in Rust.
'| stats percentile(score, 95)' # Calculate percentile

# Value extraction
'| stats values(category)'     # Get unique values

Grouping and Top N

# Group by single field
'| stats count() by status'

# Group by multiple fields
'| stats sum(revenue) by region, category'

# Top N analysis
'| stats sum(sales, top 10) by product'

# Multiple aggregations
'| stats count(), sum(revenue), avg(price) by status'

Combined Filtering and Stats

# Filter then aggregate
'status = "success" AND region = "west" | stats avg(revenue) by category'

# Complex analytics
'timestamp > "2024-01-01" | stats count(), sum(bytes), avg(response_time) by endpoint'

🔌 OpenSearch Integration Guide

Setup

from opensearchpy import OpenSearch
from tql import TQL

# Create OpenSearch client
client = OpenSearch(
    hosts=['localhost:9200'],
    http_auth=('admin', 'admin'),
    use_ssl=True,
    verify_certs=False
)

# Get index mappings
response = client.indices.get_mapping(index='users-*')
mappings = response['users-2024']['mappings']['properties']

# Initialize TQL with mappings
tql = TQL(mappings)

Query Translation

TQL automatically translates queries to OpenSearch DSL:

# TQL Query
query = 'age > 25 AND status = "active"'

# Translates to OpenSearch DSL:
{
    "query": {
        "bool": {
            "must": [
                {"range": {"age": {"gt": 25}}},
                {"term": {"status.keyword": "active"}}
            ]
        }
    }
}

# Execute seamlessly
results = tql.execute_opensearch(client, 'users-*', query)

Field Mapping Intelligence

TQL automatically handles field types:

# Text field with keyword subfield
mappings = {
    'email': {
        'type': 'text',
        'fields': {
            'keyword': {'type': 'keyword'}
        }
    }
}

# Exact match - uses .keyword automatically
'email = "admin@example.com"'  # → term query on email.keyword

# Full-text search - uses text field
'email contains "example"'      # → match query on email

# Case-insensitive - triggers post-processing
'email | lowercase = "admin@example.com"'  # → fetch + filter

Post-Processing

When OpenSearch can't handle operations, TQL applies post-processing:

# Mutators that require post-processing
'email | lowercase contains "admin"'    # Post-process: case conversion
'data | b64decode contains "secret"'    # Post-process: decode
'ip | geoip.country = "US"'            # Post-process: GeoIP lookup

# TQL automatically:
# 1. Executes base query in OpenSearch
# 2. Fetches results
# 3. Applies mutators in Python
# 4. Filters results
# 5. Returns final matches

Query Analysis

Analyze queries before execution to understand performance implications:

# Analyze query health
analysis = tql.analyze_query('email | lowercase contains "admin"', context='opensearch')

print(f"Health: {analysis['health']['status']}")  # 'fair' (post-processing)
print(f"Score: {analysis['health']['score']}")    # 85
print(f"Post-processing: {analysis['mutator_health']['requires_post_processing']}")  # True

# Recommendations for optimization
for issue in analysis['health']['issues']:
    print(f"Issue: {issue['message']}")
    print(f"Fix: {issue['recommendation']}")

📚 Documentation

Comprehensive documentation is available in the docs/ directory:

User-facing TQL documentation — getting started, query basics, the operator and mutator references, OpenSearch integration, stats, the API reference and the cookbook — lives in the tellaro-docs repository under public/tql/ and is published from there.

What stays in this repo is contributor material: see docs/README.md.

⚡ Performance

Benchmarks

Unsubstantiated. These figures predate the current code and could not be reproduced: the repo ships no benchmark harness (no benches/, no [[bench]] target, no pytest-benchmark). Treat them as rough historical claims rather than measurements.

Python Implementation:

• In-memory queries: ~10,000 records/sec
• File parsing (JSON): ~5MB/sec
• OpenSearch queries: Limited by network latency

Rust CLI (300x faster):

• In-memory queries: ~3,000,000 records/sec
• File parsing (JSON): ~150MB/sec
• Large file streaming: Process 50MB in ~200ms

Optimization Tips

# Use CLI for large files (300x faster)
$ tql 'status = 200' 50MB-file.jsonl  # ✓ Fast (Rust)
$ python -m tql 'status = 200' 50MB-file.jsonl  # ✗ Slow (Python)

# Pre-compile queries for reuse
ast = tql.parse('age > 25 AND status = "active"')
results1 = tql.evaluate(ast, dataset1)
results2 = tql.evaluate(ast, dataset2)

# Use OpenSearch for large datasets
tql.execute_opensearch(client, 'huge-index-*', query)  # Leverages OpenSearch's speed

# Minimize post-processing
'email.keyword = "admin@example.com"'  # ✓ Fast (OpenSearch only)
'email | lowercase = "admin@example.com"'  # ✗ Slower (post-processing)

🛠️ Development

Installation

# Clone repository
git clone https://github.com/tellaro/tellaro-query-language.git
cd tellaro-query-language

# Install dependencies (recommended)
uv sync

# Or with pip
pip install -e .

Testing

# Run all tests
uv run tql-tests

# Run specific test file
uv run pytest tests/unit/test_parser.py -v

# Run with coverage
uv run tql-cov

# Run integration tests (requires OpenSearch)
cp .env.example .env  # Configure OpenSearch connection
uv run pytest tests/integration/test_opensearch_integration.py -v

Code Quality

# Format + lint in one step
uv run tql-lint-all

# Format code (ruff replaces black, isort and flake8 -- none is a dependency)
uv run ruff format src tests

# Type checking
uv run pyright

# Linting
uv run ruff check src tests

# Security checks
uv run bandit -c bandit.yml -r src/

🗺️ Roadmap

✅ Implemented Features

• ✅ Core query engine with all operators
• ✅ 25+ field mutators (string, encoding, network, DNS, GeoIP, list)
• ✅ Statistical aggregations with grouping
• ✅ File support (JSON, JSONL, CSV)
• ✅ OpenSearch/Elasticsearch backend
• ✅ Intelligent post-processing
• ✅ Rust CLI for performance
• ✅ Mutator caching for GeoIP/DNS
• ✅ Query health analysis

🚧 In Progress

• 🚧 OpenSearch stats aggregation translation
• 🚧 Additional hash functions (SHA1, SHA512)
• 🚧 JSON parsing mutator
• 🚧 Timestamp conversion mutators

📋 Planned Features

• 📋 ElasticSearch backend support
• 📋 PostgreSQL/MySQL backends
• 📋 Query optimization engine
• 📋 Custom mutator plugins
• 📋 GraphQL-style field selection
• 📋 Parallel record evaluation
• 📋 Incremental file processing

🔮 Future Considerations

• 🔮 Time-series specific operators
• 🔮 Machine learning integration
• 🔮 Distributed query execution
• 🔮 Query caching layer

🤝 Contributing

Contributions are welcome! Please see CONTRIBUTING.md for guidelines.

How to Contribute

1. Fork the repository
2. Create a feature branch (git checkout -b feature/amazing-feature)
3. Make your changes
4. Run tests (uv run tql-tests)
5. Run linters (uv run tql-lint)
6. Commit changes (git commit -m 'Add amazing feature')
7. Push to branch (git push origin feature/amazing-feature)
8. Open a Pull Request

📄 License

Tellaro Query Language (TQL) is source-available software with specific usage terms:

✅ Permitted Uses:

• Personal use (individual, non-commercial)
• Organizational use (within your company/organization)
• Integration into your applications and services
• Internal tools and automation

❌ Restricted Uses:

• Creating derivative query language products
• Commercial redistribution or resale
• Offering TQL-based commercial services to third parties
• Using source code to build competing products

For commercial licensing inquiries, contact: support@tellaro.io

See LICENSE for complete terms and conditions.

🔗 Related Projects

• TQL Rust - High-performance Rust implementation
• Tellaro Platform - Security operations platform using TQL

💬 Support

• Issues: GitHub Issues
• Documentation: Full Documentation
• Examples: the cookbook in tellaro-docs/public/tql/cookbook/
• Email: support@tellaro.io

🌟 Quick Examples

Security Log Analysis

# Find high-severity events from private IPs
query = '''
    source_ip | is_private = true AND
    severity in ["high", "critical"] AND
    (ANY tags = "malware" OR url | defang contains "suspicious")
'''
results = tql.query("security-logs.jsonl", query)

E-commerce Analytics

# Analyze sales by region for premium products
query = '''
    product_tier = "premium" AND
    order_date > "2024-01-01" |
    stats sum(revenue), avg(order_value), count() by region
'''
results = tql.query("sales.json", query)

System Monitoring

# Find servers with high resource usage
query = '''
    hostname | nslookup exists AND
    (cpu_usage > 80 OR memory_usage > 90) AND
    status = "production"
'''
results = tql.execute_opensearch(client, "metrics-*", query)

Made with ❤️ by the Tellaro Team