Skip to main content

Query Expressions

ekoDB's query language uses a structured JSON format with tagged unions instead of SQL strings or MongoDB-style query documents. This technical reference explains the design rationale, expression structure, and complete operator specification.

Design Rationale​

Why Not SQL?​

SQL was designed for relational databases with fixed schemas and table joins. ekoDB is a document database optimized for:

  1. Schema flexibility - Documents can have different structures in the same collection
  2. Nested data - No joins needed when related data is embedded
  3. HTTP-first API - JSON over REST, not a separate query protocol
  4. Type-safe query builders - Statically typed in Rust, TypeScript, Kotlin, Go; runtime checked in Python

SQL's limitations for document databases:

  • String concatenation creates injection vulnerabilities
  • No type checking until runtime
  • Impedance mismatch between tabular results and nested documents
  • Query parsers are complex and error-prone

Why Not MongoDB Query Language?​

MongoDB uses a JSON-like query format, but with significant limitations:

// MongoDB - operators are magic strings at runtime
{ age: { $gte: 18, $lte: 65 }, status: "active" }

Problems with this approach:

  1. No type discrimination - Cannot distinguish between a filter object and a data object
  2. Operator collision - Field names starting with $ are reserved
  3. Ambiguous semantics - Is {a: 1, b: 2} an AND or just two fields?
  4. Harder to parse - Must scan all keys to detect operators

ekoDB's Tagged Union Approach​

{
"type": "Condition",
"content": {
"field": "age",
"operator": "Gte",
"value": 18
}
}

Benefits:

  1. Unambiguous parsing - type discriminator makes structure explicit
  2. Type-safe composition - Logical operators nest cleanly with proper typing
  3. Minimal reserved field names - Only id and ttl are reserved (vs MongoDB's $ prefix for all operators)
  4. Better error messages - Invalid queries caught during JSON schema validation
  5. Optimizable - Query planner can analyze structure before execution
  6. Query intent is obvious - Looking at the JSON, you immediately know what's being filtered. Compare:
    • ekoDB: {"type": "Condition", "content": {"field": "age", "operator": "Gte", "value": 18}}
    • MongoDB: {"age": {"$gte": 18}} (is $gte an operator or a field name?)
  7. Fine-grained propagation control - Query options include bypass_cache and bypass_ripple for direct control over the data propagation waterfall. Most databases don't expose this level of control at the query level.

Trade-offs:

  • More verbose - Requires type and content wrappers
  • Unfamiliar syntax - Not SQL or MongoDB query language
  • More JSON payload - Slightly larger wire format (mitigated by HTTP compression)

Why we chose verbosity: Explicit query intent and type safety outweigh brevity. While the syntax may be unfamiliar initially, it's far more obvious what you're querying for—there's no ambiguity between data and query operators. Client libraries abstract the verbosity with fluent APIs like QueryBuilder().gte("age", 18).

Comparison to Other NoSQL Databases​

DatabaseQuery FormatType SafetyComposabilityReserved Fields
ekoDBTagged unions✅ Strong✅ Explicit nesting⚠️ id, ttl only
MongoDBDollar operators❌ Runtime only⚠️ Implicit✅ All $ operators
CouchDBMap-Reduce❌ None⚠️ JavaScriptN/A
DynamoDBCondition expressions⚠️ Partial❌ String-based✅ Reserved words
FirestoreMethod chaining✅ Client-side only✅ Good⚠️ Minimal

ekoDB's approach is most similar to Firestore's, but with explicit JSON representation that works identically across all clients and the REST API.


Expression Structure​

ekoDB uses a tagged union (also called discriminated union or sum type) for query expressions. The type field discriminates between different expression variants at parse time.

{
"type": "Condition" | "Logical",
"content": { ... }
}

Type Discrimination​

The parser uses the type field to determine how to interpret content:

// Rust deserialization (simplified)
enum QueryExpression {
Condition { field: String, operator: Operator, value: Value },
Logical { operator: LogicOp, expressions: Vec<QueryExpression> }
}

This enables:

  • Compile-time type checking in typed languages (Rust, TypeScript, Kotlin, Go)
  • Validation before execution - malformed queries rejected immediately
  • Query plan optimization - structure is known before accessing data

Two Expression Types​

  1. Condition - Single field comparison (leaf node in expression tree)
  2. Logical - Combine multiple expressions (branch node in expression tree)

Expressions form a tree structure where:

  • Leaf nodes are Condition expressions
  • Branch nodes are Logical expressions containing child expressions
  • Tree depth is unlimited (for complex business logic)

Visual Example: Query Expression Tree​

Here's how the query age >= 18 AND (status = "active" OR role = "admin") is represented as a tree:

Legend:

  • Logical expressions (AND, OR, NOT) = branch nodes that combine child expressions
  • Condition expressions (age >= 18, status = "active") = leaf nodes with field comparisons

The parser evaluates this tree recursively:

  1. Start at root AND node
  2. Evaluate left child: age >= 18 (Condition)
  3. Evaluate right child: OR node
    • Evaluate status = "active" (Condition)
    • Evaluate role = "admin" (Condition)
  4. Combine results with AND logic

Comparison Operators​

All comparison operators follow this structure:

{
"type": "Condition",
"content": {
"field": "field_name",
"operator": "OperatorName",
"value": "comparison_value"
}
}

Available Operators​

OperatorDescriptionAliasesExample Value
EqEqualsEqual, Equals"active", 25
NeNot equalsNotEqual, NotEquals"deleted", 0
GtGreater thanGreaterThan18, 100.5
GteGreater than or equalGreaterThanOrEqual18, 0
LtLess thanLessThan65, 1000
LteLess than or equalLessThanOrEqual100, 999.99
InValue in array-["active", "pending"]
NotInValue not in array-["deleted", "archived"]

String Operators​

Special operators for string/text matching:

OperatorDescriptionCase SensitiveExample
ContainsSubstring matchYes"@gmail.com"
StartsWithPrefix matchYes"Dr. "
EndsWithSuffix matchYes".com"
String Matching

All string operators are case-sensitive. To perform case-insensitive searches, normalize your data (e.g., lowercase) on insert.


Logical Operators​

Combine multiple expressions with logical operators:

Empty logical filters are rejected

A Logical expression must contain at least one child in expressions. An empty array is rejected with HTTP 400 — an empty And would otherwise match every record and an empty Or none, which is almost always a malformed filter. To match all records, omit the filter entirely.

AND - All conditions must match​

{
"type": "Logical",
"content": {
"operator": "And",
"expressions": [
{
"type": "Condition",
"content": {
"field": "age",
"operator": "Gte",
"value": 18
}
},
{
"type": "Condition",
"content": {
"field": "status",
"operator": "Eq",
"value": "active"
}
}
]
}
}

OR - At least one condition must match​

{
"type": "Logical",
"content": {
"operator": "Or",
"expressions": [
{
"type": "Condition",
"content": {
"field": "role",
"operator": "Eq",
"value": "admin"
}
},
{
"type": "Condition",
"content": {
"field": "role",
"operator": "Eq",
"value": "moderator"
}
}
]
}
}

NOT - Negate a condition​

{
"type": "Logical",
"content": {
"operator": "Not",
"expressions": [
{
"type": "Condition",
"content": {
"field": "status",
"operator": "Eq",
"value": "deleted"
}
}
]
}
}

Complex Nested Logic​

You can nest logical operators for complex queries:

{
"type": "Logical",
"content": {
"operator": "And",
"expressions": [
{
"type": "Condition",
"content": {
"field": "age",
"operator": "Gte",
"value": 18
}
},
{
"type": "Logical",
"content": {
"operator": "Or",
"expressions": [
{
"type": "Condition",
"content": {
"field": "country",
"operator": "Eq",
"value": "US"
}
},
{
"type": "Condition",
"content": {
"field": "country",
"operator": "Eq",
"value": "CA"
}
}
]
}
}
]
}
}

This finds records where: age >= 18 AND (country = "US" OR country = "CA")


Field Paths​

A field can name a value inside a nested object or array, not only a top-level field:

  • Dot notation reaches into objects: address.city is the city of the address object.
  • A numeric segment reaches into an array: items.0.name is the name of the first element of items.

Both work in filter conditions and in sort. In text search fields, dot paths reach into objects.

{
"filter": {
"type": "Logical",
"content": {
"operator": "And",
"expressions": [
{
"type": "Condition",
"content": { "field": "address.city", "operator": "Eq", "value": "Austin" }
},
{
"type": "Condition",
"content": { "field": "items.0.sku", "operator": "Eq", "value": "A-100" }
}
]
}
},
"sort": [{ "field": "address.zip", "ascending": true }]
}
Field names

Do not put ., , or : in a field name. A . is always read as a path separator, so a field named unit.price cannot be addressed in a filter, sort or search. , separates the entries of search field lists and comma-joined projections, and : separates a search field from its weight, so a name containing either cannot be named there. Use _ or camelCase instead (unit_price, not unit.price).


Query Options​

Beyond filters, ekoDB provides fine-grained control over sorting, pagination, and data propagation at the query level:

{
"filter": { ... },
"sort": [
{
"field": "created_at",
"ascending": false
},
{
"field": "name",
"ascending": true
}
],
"limit": 10,
"skip": 20,
"bypass_cache": false,
"bypass_ripple": true
}

Options Reference​

OptionTypeDescription
filterQueryExpressionFilter condition (see above)
sortArraySort by one or more fields
limitIntegerMax results to return (default 1,000 on REST; see Limits and Paging)
skipIntegerResults to skip (pagination; capped, see Limits and Paging)
bypass_cacheBooleanSkip cache lookup (read directly from storage)
bypass_rippleBooleanDon't propagate writes to ripple peers (for bulk imports)

Propagation Control (Unique to ekoDB)​

Unlike most databases, ekoDB exposes direct control over the data propagation waterfall at the query level:

bypass_cache - Control read path:

  • false (default): Check cache first, then storage if cache miss
  • true: Skip cache entirely, read directly from durable storage
  • Use case: When you need guaranteed fresh data and can tolerate slightly higher latency

bypass_ripple - Control write replication:

  • false (default): Writes propagate to all configured ripple peers
  • true: Writes stay local, skip replication
  • Use case: Bulk data imports, maintenance operations, temporary data

Most databases treat caching and replication as infrastructure concerns outside the query language. ekoDB makes these first-class query options for maximum control.

Limits and Paging​

Every read is bounded, so one query cannot pull a whole large collection into memory. The bounds are instance settings, read with GET /api/config and changed with PUT /api/config (admin; see Configuration: Queries).

SituationWhat happens
REST find with no limitAt most default_query_limit records are returned: 1,000 by default, settable from 1 to 10,000.
limit above max_query_limitRefused with 400: Limit value N exceeds the maximum of M. Page through results with `limit` and `skip` instead. The limit is not lowered for you.
skip above max_query_skipRefused with 400: Skip value N exceeds the maximum of M. ...
limit: 0Asks for every matching record, up to max_unpaginated_records. A read that would return more is refused with an error telling you to paginate; it is never cut short silently.
WebSocket Query with no limitThere is no 1,000 default on WebSocket: an omitted limit asks for every matching record, with the same max_unpaginated_records bound as limit: 0.

max_query_limit, max_query_skip and max_unpaginated_records are unset by default (GET /api/config shows null), and the instance derives each one from its memory: max_query_limit and max_query_skip fall between 1,000 and 100,000, and max_unpaginated_records is at least 1,000. A small instance therefore accepts smaller pages than a large one, so read the values from GET /api/config rather than assuming them.

Practical rules:

  • Always pass an explicit limit, and over WebSocket in particular.
  • Page large collections by key, not by skip. skip is capped and costs more the deeper it goes. Instead, sort on a field whose values are unique and increasing (a timestamp or a sequence number you write), and ask for the next page with a filter greater than the last value you received:
{
"filter": {
"type": "Condition",
"content": { "field": "created_seq", "operator": "Gt", "value": 4211 }
},
"sort": [{ "field": "created_seq", "ascending": true }],
"limit": 500
}

Repeat with the last created_seq of each page until a page comes back with fewer than limit records.

Runnable examples: client_query_builder.ts (limits and sorting through the query builder) and client_simple_websocket.ts (queries over WebSocket).


Usage Examples​

For practical examples of using query expressions:


Common Query Patterns​

Find All Active Records​

{
"filter": {
"type": "Condition",
"content": {
"field": "status",
"operator": "Eq",
"value": "active"
}
}
}

Range Query (Between Values)​

{
"filter": {
"type": "Logical",
"content": {
"operator": "And",
"expressions": [
{
"type": "Condition",
"content": {
"field": "price",
"operator": "Gte",
"value": 10.0
}
},
{
"type": "Condition",
"content": {
"field": "price",
"operator": "Lte",
"value": 100.0
}
}
]
}
}
}

Exclude Values (NOT IN)​

{
"filter": {
"type": "Condition",
"content": {
"field": "status",
"operator": "NotIn",
"value": ["deleted", "archived", "spam"]
}
}
}

Search Multiple Fields (OR)​

{
"filter": {
"type": "Logical",
"content": {
"operator": "Or",
"expressions": [
{
"type": "Condition",
"content": {
"field": "title",
"operator": "Contains",
"value": "urgent"
}
},
{
"type": "Condition",
"content": {
"field": "description",
"operator": "Contains",
"value": "urgent"
}
}
]
}
}
}

Complex Business Logic​

Find premium users who are either:

  • Over 30 years old, OR
  • Have made more than 10 purchases
{
"filter": {
"type": "Logical",
"content": {
"operator": "And",
"expressions": [
{
"type": "Condition",
"content": {
"field": "tier",
"operator": "Eq",
"value": "premium"
}
},
{
"type": "Logical",
"content": {
"operator": "Or",
"expressions": [
{
"type": "Condition",
"content": {
"field": "age",
"operator": "Gt",
"value": 30
}
},
{
"type": "Condition",
"content": {
"field": "purchase_count",
"operator": "Gt",
"value": 10
}
}
]
}
}
]
}
}
}

Performance Tips​

Use Indexes​

Create indexes on fields you query frequently:

use ekodb_client::QueryBuilder;

// When a field has an index, ekoDB uses it automatically — no hint required.
let query = QueryBuilder::new().eq("category", "electronics").build();
let results = client.find("products", query, None).await?;

See Indexes Documentation for more details.

Limit Results​

Always use limit to prevent loading excessive data:

{
"filter": { ... },
"limit": 100
}

The defaults and ceilings on limit and skip, and how to page a large collection, are in Limits and Paging.

Bypass Cache for Fresh Data​

When you need the absolute latest data:

{
"filter": { ... },
"bypass_cache": true
}

Use Specific Fields​

Only select fields you need (reduces bandwidth):

{
"filter": { ... },
"select_fields": ["id", "name", "email"]
}

Projection works on both transports. The POST body uses a JSON array, either to include fields (select_fields) or to drop them (exclude_fields):

{
"filter": { ... },
"exclude_fields": ["secret"]
}

GET query strings use a comma-joined value:

GET /api/find/users?select_fields=id,name,email
GET /api/find/users?exclude_fields=secret

An empty value (?select_fields=) means no projection. The primary key (id) is always returned.


Troubleshooting​

400 Bad Request - Invalid Filter Format​

❌ Wrong:

{
"filter": {
"field": "status",
"operator": "Eq",
"value": "active"
}
}

✅ Correct:

{
"filter": {
"type": "Condition",
"content": {
"field": "status",
"operator": "Eq",
"value": "active"
}
}
}

Missing type and content wrapper!

400 Bad Request - Unknown Operator​

❌ Wrong:

{
"operator": "equals" // lowercase
}

✅ Correct:

{
"operator": "Eq" // PascalCase
}

Operators are case-sensitive and use PascalCase.

Empty Results When Data Exists​

  • Check if field names are correct (case-sensitive)
  • Verify value types match (string vs number)
  • For string matching, remember it's case-sensitive

Next Steps​

Need Help?​