Functions Architecture
This document provides an in-depth technical overview of ekoDB's Functions system. For practical usage examples, see Advanced Operations - Functions.
Overview
ekoDB's Functions system is a powerful server-side execution engine that enables you to define, version, and execute complex business logic as composable operations. Unlike traditional stored procedures, ekoDB Functions are JSON-based, parameterized, and designed for modern application architectures.
Key Concepts
Function: A stored procedure containing one or more operations (Query, Insert, Update, Delete, Chat, Embed, etc.)
Operation: A single database action within a Function (referred to in the functions array)
Composability: Functions can call other Functions using CallFunction, enabling unlimited nesting and reusability
Why Functions?
Traditional Approach Problems
// ❌ Business logic scattered across application code
async function getEnrichedUserData(userId: string) {
// Query user
const user = await db.findOne("users", { id: userId });
// Query orders
const orders = await db.find("orders", { user_id: userId, status: "active" });
// Generate AI summary
const summary = await openai.complete(`Summarize: ${JSON.stringify(orders)}`);
// Update user with summary
await db.update("users", userId, { ai_summary: summary });
return { user, orders, summary };
}
Issues:
- Multiple network round-trips
- Logic duplicated in every client
- Hard to version and maintain
- No transaction guarantees
ekoDB Functions Approach
// ✅ Define once, call from anywhere
{
"label": "get_user_orders",
"name": "Get User Orders with Count",
"parameters": {
"user_id": { "type": "string", "required": true },
"status": { "type": "string", "required": false, "default": "active" }
},
"functions": [
{
"type": "Query",
"collection": "orders",
"filter": {
"type": "Logical",
"content": {
"operator": "And",
"expressions": [
{
"type": "Condition",
"content": {
"field": "user_id",
"operator": "Eq",
"value": "{{user_id}}"
}
},
{
"type": "Condition",
"content": {
"field": "status",
"operator": "Eq",
"value": "{{status}}"
}
}
]
}
},
"limit": 50
},
{
"type": "Count",
"output_field": "total_orders"
},
{
"type": "Project",
"fields": ["id", "amount", "status", "total_orders"],
"exclude": false
}
]
}
Benefits:
- ✅ Single network call
- ✅ Defined once, used everywhere
- ✅ Version controlled
- ✅ Server-side execution
- ✅ Automatic parameter validation
Architecture
Operation Types
ekoDB Functions support these step types (the type field of each step). This
is the complete catalog.
Query & Read
- FindAll - Retrieve all records in a collection
- Query - Filter, sort, and paginate (
filter/sort/limit/skipare inline on this step) - FindById - Get one record by ID
- FindOne - Find one record by key/value
- TextSearch - Full-text search
- VectorSearch - Semantic similarity (vector) search
- HybridSearch - Combined text + vector search
Write (CRUD)
- Insert - Insert one record
- BatchInsert - Insert many records
- Update - Update records matching a filter
- UpdateById - Update one record by ID
- FindOneAndUpdate - Atomic find-and-update
- UpdateWithAction - Atomic field action (increment, push, append, pop, etc.)
- Upsert - Atomic find-or-create by lookup key
- Delete - Delete records matching a filter
- DeleteById - Delete one record by ID
- BatchDelete - Delete many records by ID
In-pipeline record edits
- SetField - Set a literal value on the working record
- Increment - Atomic counter add on a stored record
- Push - Atomic array append on a stored record
- CurrentDatetime - Stamp the current UTC time onto the working record
Aggregation & shaping
- Group - Group and aggregate (11 operations: Count, Sum, Average, Min, Max, First, Last, Push, AddToSet, StandardDeviation, ApproxDistinct)
- AddFields - Add computed/derived fields (arithmetic, string, and conditional expressions)
- Project - Select or exclude fields
- Count - Add a record-count field
Control flow
- If - Branch on a
FunctionCondition - ForEach - Iterate over the working records, running sub-steps per record
- Parallel - Run sub-steps concurrently and merge their results
- TryCatch - Run steps; on error, run the catch steps (error bound to
output_error_field) - Sleep - Pause (e.g. for rate limits or retry backoff)
- Return - Build the final response object (optionally setting
status_codefor the HTTP status) - Validate - Assert a condition; fail the pipeline if it doesn't hold
- CallFunction - Call another saved function by label (composition)
Transactions
- CreateSavepoint / RollbackToSavepoint / ReleaseSavepoint - Nested-transaction savepoints for partial rollback
A saved function with transaction_config.enabled = true runs its steps as one atomic unit: every applied write is tracked and the per-transaction undo log reverts all of them on any failure. This is the recommended way to do a multi-statement unit, and it gives you atomicity, consistency, and durability (plus savepoints). It does not add isolation — each step applies to the live store immediately (apply-then-undo), so a concurrent operation can observe a transactional function's uncommitted intermediate writes. Enforced transaction isolation between concurrent transactions is on the roadmap.
AI
- Chat - LLM chat completion
- Embed - Generate embeddings for a field
Auth & crypto (build login/signup without external services)
- BcryptHash / BcryptVerify - Password hashing and verification
- JwtSign / JwtVerify - Issue and verify JWTs
- HmacSign / HmacVerify - HMAC sign and timing-safe verify
- AesEncrypt / AesDecrypt - AES-GCM encrypt/decrypt
- TotpGenerate / TotpVerify - TOTP MFA codes
- RandomToken - CSPRNG random token
- UuidGenerate - Generate a UUID
- Base64Encode / Base64Decode - Base64
- HexEncode / HexDecode - Hex
- Slugify - URL-safe slug
Key-value store
- KvGet / KvSet / KvDelete / KvExists / KvQuery - KV cache operations (TTL, prefix scan)
Caching & concurrency
- SWR - Stale-while-revalidate cached pipeline (cache check → run → cache set)
- IdempotencyClaim - Claim an idempotency key (once-only execution)
- RateLimit - Fixed-window rate limit
- LockAcquire / LockRelease - Distributed lock
External integrations
- HttpRequest - Call an external HTTP API (Stripe, SendGrid, etc.)
- EmailSend - Send an email
Parameter System
Functions support dynamic parameterization using {{parameter_name}} syntax:
{
"parameters": {
"user_id": {
"type": "string",
"required": true,
"description": "User identifier"
},
"limit": {
"type": "number",
"required": false,
"default": 10,
"description": "Result limit"
},
"status": {
"type": "string",
"required": false,
"default": "active",
"enum": ["active", "inactive", "pending"]
}
},
"functions": [
{
"type": "Query",
"collection": "users",
"filter": {
"type": "Logical",
"content": {
"operator": "And",
"expressions": [
{
"type": "Condition",
"content": {
"field": "id",
"operator": "Eq",
"value": "{{user_id}}"
}
},
{
"type": "Condition",
"content": {
"field": "status",
"operator": "Eq",
"value": "{{status}}"
}
}
]
}
},
"limit": "{{limit}}"
}
]
}
Parameter Resolution:
- User provides values when calling Function
- ekoDB validates types and requirements
- Parameters are substituted before execution
- Results can reference previous Function outputs
Result Chaining
Operations execute sequentially, with results flowing from one to the next:
{
"functions": [
{
"type": "Query",
"collection": "products",
"filter": {
"type": "Condition",
"content": {
"field": "category",
"operator": "Eq",
"value": "{{category}}"
}
}
},
{
"type": "Count",
"output_field": "product_count"
},
{
"type": "Project",
"fields": ["name", "price", "product_count"],
"exclude": false
}
]
}
How Results Flow:
- Query operation returns records
- Count adds a count field to those records
- Project selects only specified fields
- Each operation works with the current set of records
Execution Model
Sequential Execution
By default, Functions execute in sequence:
Function 1 → Function 2 → Function 3 → Result
Each Function waits for the previous to complete before starting.
Conditional Execution
Use the If Function type for conditional logic:
{
"type": "If",
"condition": {
"type": "FieldEquals",
"value": { "field": "plan", "value": "premium" }
},
"then_functions": [{ "type": "Query", "collection": "premium_features" }],
"else_functions": [{ "type": "Query", "collection": "basic_features" }]
}
condition is a FunctionCondition object (adjacently tagged with type +
value), evaluated against the working records — not a string expression. The
branches are then_functions and else_functions.
FunctionCondition variants:
- Field comparisons:
FieldEquals,FieldExists,FieldGreaterThan,FieldLessThan,FieldGreaterThanOrEqual,FieldLessThanOrEqual— e.g.{ "type": "FieldGreaterThan", "value": { "field": "stock", "value": 0 } } - Set/count:
HasRecords,CountEquals,CountGreaterThan,CountLessThan - Boolean:
And,Or,Not— e.g.{ "type": "And", "value": { "conditions": [<FunctionCondition>, ...] } }
Loop Execution
Use the ForEach operation type to iterate over current records:
{
"functions": [
{
"type": "Query",
"collection": "users",
"filter": {
"type": "Condition",
"content": {
"field": "status",
"operator": "Eq",
"value": "pending"
}
}
},
{
"type": "ForEach",
"functions": [
{
"type": "UpdateById",
"collection": "users",
"record_id": "{{id}}",
"updates": { "notified": true }
}
]
}
]
}
How it works:
- Operates on records from previous operations
- Executes nested operations for each record
- Each record's fields are available as variables (e.g.,
{{id}},{{email}})
Creating Functions
What is a Function?
A Function is a named collection of operations that can be leveraged to perform complex logic and transformations. Each function is versioned for friendly maintenance:
- Stored in ekoDB
- Called by label or ID
- Nested using
CallFunction - Parameterized with dynamic values
Function Structure
| Field | Description | Required |
|---|---|---|
label | Unique identifier | Yes |
name | Human-readable name | Yes |
description | What the function does | No |
version | Semantic version | No |
parameters | Input parameters | No |
functions | Array of operations | Yes |
tags | Categorization tags | No |
Creating a Function
Minimal Example (Required Fields Only):
{
"label": "get_user",
"name": "Get User by ID",
"functions": [
{
"type": "FindOne",
"collection": "users",
"filter": { "id": "{{user_id}}" }
}
]
}
Full Example (With Optional Fields):
{
"label": "user_onboarding",
"name": "User Onboarding Flow",
"description": "Complete new user setup with AI welcome message",
"version": "1.0.0",
"tags": ["onboarding", "users", "ai"],
"parameters": {
"email": { "type": "string", "required": true },
"name": { "type": "string", "required": true }
},
"functions": [
{
"type": "Insert",
"collection": "users",
"record": {
"email": "{{email}}",
"name": "{{name}}"
}
},
{
"type": "Chat",
"messages": [
{
"role": "system",
"content": "You are a friendly onboarding assistant"
},
{
"role": "user",
"content": "Write a welcome message for {{name}}"
}
],
"model": "gpt-4"
}
]
}
Calling a Function
From Client Library:
const result = await client.callFunction("user_onboarding", {
email: "user@example.com",
name: "John Doe",
});
From REST API:
POST /api/functions/user_onboarding
Content-Type: application/json
{
"email": "user@example.com",
"name": "John Doe"
}
Response format:
A call returns { "records": [...], "stats": {...} }. The record field values follow the same use_typed_values rule as find and query: with use_typed_values enabled they are wrapped ({"type":"String","value":"..."}), and with it disabled they are bare. A client configured for typed values therefore gets the same shape from callFunction as from a read.
This wrapping is applied only on the response to an external caller. Inside the pipeline, and when one function calls another with CallFunction, records flow as bare native values so that {{record.field}} access keeps working in the calling function.
A Return step may set status_code to control the HTTP status of the response (for example 201 after a create, or 404 when a lookup misses). The code sets the actual HTTP status; it does not appear in the response body.
Function Nesting
Functions can call other Functions for modularity using CallFunction:
{
"label": "process_order",
"functions": [
{
"type": "CallFunction",
"function_label": "validate_inventory",
"params": { "product_id": "{{product_id}}" }
},
{
"type": "CallFunction",
"function_label": "charge_payment",
"params": { "amount": "{{amount}}" }
},
{
"type": "CallFunction",
"function_label": "send_confirmation",
"params": { "order_id": "{{order_id}}" }
}
]
}
Execution:
process_order
├─ validate_inventory
│ ├─ Query products
│ └─ Update inventory
├─ charge_payment
│ └─ HttpRequest to Stripe
└─ send_confirmation
├─ Chat (AI message)
└─ HttpRequest to email API
Versioning
Why Version Functions?
- Evolution - Update logic without breaking existing callers
- Rollback - Revert to previous versions if issues arise
- Testing - Test new versions alongside old ones
- Auditing - Track changes over time
Version Strategy
{
"label": "calculate_pricing",
"version": "2.0", // Increment on breaking changes
"functions": [...]
}
Best Practices:
- Use semantic versioning:
major.minor.patch - Major version: Breaking changes
- Minor version: New features (backward compatible)
- Patch version: Bug fixes
Calling Specific Versions
// Call latest version
await client.callFunction("calculate_pricing", params);
// Call specific version (if supported)
await client.callFunction("calculate_pricing", params, { version: "1.0" });
AI Integration
Chat Functions
Execute LLM completions with context:
{
"type": "Chat",
"messages": [
{
"role": "system",
"content": "You are a data analyst"
},
{
"role": "user",
"content": "Analyze this data: {{query_results}}"
}
],
"model": "gpt-4",
"temperature": 0.7,
"max_tokens": 1000
}
Supported Models:
- OpenAI:
gpt-4,gpt-4-turbo,gpt-3.5-turbo - Anthropic:
claude-3-opus,claude-3-sonnet - Perplexity:
pplx-7b-online,pplx-70b-online
Embedding Functions
Generate vector embeddings for text:
{
"type": "Embed",
"input_field": "description",
"output_field": "description_vector",
"model": "text-embedding-ada-002"
}
Use Cases:
- Semantic search
- Document similarity
- Recommendation systems
- Clustering
RAG Workflows
Build complete RAG pipelines by combining search and AI:
{
"label": "semantic_search",
"parameters": {
"query": { "type": "string", "required": true }
},
"functions": [
{
"type": "TextSearch",
"collection": "knowledge_base",
"query_text": "{{query}}",
"limit": 5
},
{
"type": "Chat",
"messages": [
{
"role": "system",
"content": "Answer using only the provided context"
},
{
"role": "user",
"content": "Context: {{search_results}}\n\nQuestion: {{query}}"
}
],
"model": "gpt-4"
}
]
}
Note: For true vector search RAG, first generate embeddings for your documents using the Embed operation, then use VectorSearch to find similar content.
External Integrations
HTTP Requests
Call external APIs from Functions:
{
"type": "HttpRequest",
"method": "POST",
"url": "https://api.stripe.com/v1/charges",
"headers": {
"Authorization": "Bearer {{stripe_key}}",
"Content-Type": "application/x-www-form-urlencoded"
},
"body": {
"amount": "{{amount}}",
"currency": "usd",
"source": "{{token}}"
},
"output_field": "charge_result"
}
Response Handling:
The HTTP response is stored in the specified output_field (default: http_response):
- JSON responses are automatically parsed into native Objects/Arrays
- Non-JSON responses (HTML, plain text) are stored as Strings
You can reference the response in subsequent stages using the field name specified in output_field (e.g., {{charge_result}} in the example above).
Use Cases:
- Payment processing (Stripe, PayPal)
- Email services (SendGrid, Mailgun)
- SMS notifications (Twilio)
- Webhook triggers
- Third-party API integration
Performance Considerations
Execution Time
- Operation overhead: ~1-5ms per operation
- Network calls: Variable (external APIs, AI)
- Database operations: Sub-millisecond for queries
- Total Function time: Sum of all operation times
Optimization Strategies
- Batch operations instead of loops when possible
- Limit result sets with filters and projections
- Filter early - Reduce data before expensive operations
- Cache Function definitions (auto-cached by ekoDB)
- Minimize external HTTP calls
Stale-While-Revalidate (SWR) Pattern
Functions are perfect for the SWR pattern, which dramatically improves perceived performance by serving cached results while fetching fresh data in the background.
How it works:
1. Client requests Function execution
2. Return cached result immediately (if exists)
3. Execute Function in background
4. Update cache with fresh result
5. Next request gets updated data
Benefits:
- ⚡ Instant response - Users see data immediately
- 🔄 Always fresh - Background updates keep data current
- 📉 Reduced load - Fewer Function executions
- 🎯 Better UX - No loading spinners for cached data
When to use:
- ✅ Dashboard summaries that don't change frequently
- ✅ User profile data (name, email, settings)
- ✅ Product catalogs with periodic updates
- ✅ Analytics data that's expensive to compute
- ❌ Real-time data (stock prices, live scores)
- ❌ User-specific transactional data (cart, orders)
Example pattern:
// Client-side SWR with Functions
async function getUserDashboard(userId: string) {
// 1. Return cached data immediately
const cached = cache.get(`dashboard:${userId}`);
if (cached) {
// Serve stale data instantly
displayDashboard(cached);
// 2. Revalidate in background
client.callFunction("user_dashboard", { user_id: userId }).then((fresh) => {
cache.set(`dashboard:${userId}`, fresh);
displayDashboard(fresh); // Update UI with fresh data
});
return cached;
}
// 3. No cache - fetch and display
const data = await client.callFunction("user_dashboard", { user_id: userId });
cache.set(`dashboard:${userId}`, data);
return data;
}
Server-side caching:
Functions can also implement internal caching for even better performance:
{
"label": "expensive_analytics",
"functions": [
{
"type": "Query",
"collection": "events",
"filter": {
"type": "Condition",
"content": {
"field": "date",
"operator": "Eq",
"value": "{{today}}"
}
}
},
{
"type": "Group",
"by_fields": ["category"],
"functions": [{ "output_field": "count", "operation": "Count" }]
}
]
}
Call this Function with a cache key and TTL, and ekoDB can cache results server-side for all clients.
SWR Pattern Examples in all languages:
- Rust:
swr_pattern.rs - Python:
swr_pattern.py - TypeScript:
client_swr_pattern.ts - Go:
swr_pattern.go - Kotlin:
SwrPattern.kt
Edge Cache Pattern - Use ekoDB as an edge cache with external API caching:
- Rust:
client_edge_cache.rs - Python:
client_edge_cache.py - TypeScript:
client_edge_cache.ts - JavaScript:
client_edge_cache.js - Go:
client_edge_cache.go - Kotlin:
ClientEdgeCache.kt
Best Practices
DO:
- ✅ Keep Functions focused and single-purpose
- ✅ Use meaningful labels and descriptions
- ✅ Document parameters clearly
- ✅ Version Functions properly
- ✅ Test with sample data before production
DON'T:
- ❌ Create mega-Functions with 50+ operations
- ❌ Use Functions for simple single operations
- ❌ Hard-code values (use parameters)
- ❌ Nest Functions more than 3-4 levels deep
- ❌ Ignore error handling
Error Handling
Operation-Level Errors
If an operation fails:
- Execution stops immediately
- Error details are returned
- No rollback (unless in transaction)
- Caller receives error response
{
"error": true,
"operation_index": 2,
"operation_type": "Chat",
"message": "API rate limit exceeded",
"details": {...}
}
Error Recovery
To catch a failing step and run fallback logic, use TryCatch (the step's
error is bound to output_error_field):
{
"type": "TryCatch",
"try_functions": [
{ "type": "Chat", "prompt": "{{question}}", "output_field": "reply" }
],
"catch_functions": [
{
"type": "Insert",
"collection": "error_log",
"record": { "error": "{{error}}" }
}
],
"output_error_field": "error"
}
If branches on a FunctionCondition evaluated against the working records —
note condition is an object (not a string), and the branches are
then_functions / else_functions:
{
"type": "If",
"condition": { "type": "HasRecords" },
"then_functions": [
{ "type": "Insert", "collection": "audit", "record": { "event": "found" } }
],
"else_functions": []
}
Transactions
Wrap operations in transactions for atomicity:
{
"transaction": true,
"functions": [
{ "type": "Update", "collection": "inventory", "..." },
{ "type": "Insert", "collection": "orders", "..." }
]
}
If any operation fails, all changes are rolled back.
Security
Permission Model
Functions execute with the caller's permissions:
- Collection-level access control applies
- Field-level permissions enforced
- API key scoping respected
Parameter Validation
All parameters are validated before execution:
- Type checking (string, number, boolean, array, object)
- Required field validation
- Enum value checking
- Custom validation rules
Injection Prevention
ekoDB automatically:
- ✅ Sanitizes all parameter inputs
- ✅ Prevents NoSQL injection
- ✅ Escapes special characters
- ✅ Validates JSON structure
Safe:
{
"filter": { "email": "{{user_email}}" }
}
User cannot inject malicious queries.
Use Cases
1. User Authentication Flow
{
"label": "get_user_by_email",
"parameters": {
"email": { "type": "string", "required": true }
},
"functions": [
{
"type": "FindOne",
"collection": "users",
"key": "email",
"value": "{{email}}"
}
]
}
2. E-commerce Order Processing
{
"label": "create_order",
"parameters": {
"user_id": { "type": "string", "required": true },
"product_id": { "type": "string", "required": true }
},
"functions": [
{
"type": "FindById",
"collection": "products",
"record_id": "{{product_id}}"
},
{
"type": "Insert",
"collection": "orders",
"record": {
"user_id": "{{user_id}}",
"product_id": "{{product_id}}",
"status": "pending"
}
}
]
}
3. AI Content Moderation
{
"label": "moderate_content",
"parameters": {
"content": { "type": "string", "required": true }
},
"functions": [
{
"type": "Chat",
"messages": [
{
"role": "system",
"content": "Analyze if content is appropriate and respond with 'SAFE' or 'UNSAFE'"
},
{
"role": "user",
"content": "{{content}}"
}
],
"model": "gpt-4"
}
]
}
4. Data Analytics Pipeline
{
"label": "event_summary",
"parameters": {
"date": { "type": "string", "required": true }
},
"functions": [
{
"type": "Query",
"collection": "events",
"filter": {
"type": "Condition",
"content": {
"field": "date",
"operator": "Eq",
"value": "{{date}}"
}
}
},
{
"type": "Count",
"output_field": "total_events"
}
]
}
Comparison with Alternatives
vs. Stored Procedures (SQL)
| Feature | ekoDB Functions | SQL Stored Procedures |
|---|---|---|
| Language | JSON (declarative) | SQL (procedural) |
| Versioning | Built-in | Manual |
| AI Integration | Native | None |
| External APIs | Yes | Limited |
| Portability | JSON format | Database-specific |
vs. Cloud Functions (AWS Lambda, etc.)
| Feature | ekoDB Functions | Cloud Functions |
|---|---|---|
| Execution | Server-side in ekoDB | Separate compute |
| Network | Single call | Multiple calls |
| State | Direct DB access | Must connect to DB |
| Deployment | JSON definition | Code deployment |
| Cost | Included | Per-invocation |
vs. Application Code
| Feature | ekoDB Functions | Application Code |
|---|---|---|
| Location | Server-side | Client-side |
| Reusability | All clients | Single codebase |
| Performance | Low latency | Multiple round-trips |
| Versioning | Built-in | Git/deployment |
| Testing | Isolated | Full app context |
Related Documentation
- Advanced Operations - Functions - Practical usage examples
- Transactions Architecture - ACID transaction details
- White Paper - Overall ekoDB architecture
- Basic Operations - REST API reference
Next Steps
📚 Explore Function Examples
Complete working examples in all languages:
-
Function Composition Examples - Full-featured examples:
- Rust:
client_function_composition.rs - Python:
client_function_composition.py - TypeScript:
client_function_composition.ts - JavaScript:
client_function_composition.js - Go:
client_functions.go - Kotlin:
ClientFunctionComposition.kt
- Rust:
-
RAG & AI Examples - Retrieval-augmented generation:
- Rust:
rag_conversation_system.rs - Python:
rag_conversation_system.py - TypeScript:
rag_conversation_system.ts - Go:
rag_conversation_system.go - Kotlin:
RagConversationSystem.kt
- Rust:
-
SWR Pattern - Stale-while-revalidate with Functions:
- Rust:
swr_pattern.rs - Python:
swr_pattern.py - TypeScript:
client_swr_pattern.ts - Go:
swr_pattern.go - Kotlin:
SwrPattern.kt
- Rust:
Learning Path
- Learn by example - Browse the GitHub examples above
- Start simple - Create a basic Query Function
- Add parameters - Make Functions dynamic
- Compose Functions - Combine Functions for workflows
- Integrate AI - Add Chat and Embed Functions
Need help? Submit a support ticket or open an issue