Skip to main content

Functions Architecture

Technical Overview

This document provides an in-depth technical overview of ekoDB's Functions system. For practical usage examples, see Advanced Operations - Functions.

Overview

ekoDB's Functions system is a powerful server-side execution engine that enables you to define, version, and execute complex business logic as composable operations. Unlike traditional stored procedures, ekoDB Functions are JSON-based, parameterized, and designed for modern application architectures.

Key Concepts

Function: A stored procedure containing one or more operations (Query, Insert, Update, Delete, Chat, Embed, etc.)

Operation: A single database action within a Function (referred to in the functions array)

Composability: Functions can call other Functions using CallFunction, enabling unlimited nesting and reusability

Why Functions?

Traditional Approach Problems

// ❌ Business logic scattered across application code
async function getEnrichedUserData(userId: string) {
// Query user
const user = await db.findOne("users", { id: userId });

// Query orders
const orders = await db.find("orders", { user_id: userId, status: "active" });

// Generate AI summary
const summary = await openai.complete(`Summarize: ${JSON.stringify(orders)}`);

// Update user with summary
await db.update("users", userId, { ai_summary: summary });

return { user, orders, summary };
}

Issues:

  • Multiple network round-trips
  • Logic duplicated in every client
  • Hard to version and maintain
  • No transaction guarantees

ekoDB Functions Approach

// ✅ Define once, call from anywhere
{
"label": "get_user_orders",
"name": "Get User Orders with Count",
"parameters": {
"user_id": { "type": "string", "required": true },
"status": { "type": "string", "required": false, "default": "active" }
},
"functions": [
{
"type": "Query",
"collection": "orders",
"filter": {
"type": "Logical",
"content": {
"operator": "And",
"expressions": [
{
"type": "Condition",
"content": {
"field": "user_id",
"operator": "Eq",
"value": "{{user_id}}"
}
},
{
"type": "Condition",
"content": {
"field": "status",
"operator": "Eq",
"value": "{{status}}"
}
}
]
}
},
"limit": 50
},
{
"type": "Count",
"output_field": "total_orders"
},
{
"type": "Project",
"fields": ["id", "amount", "status", "total_orders"],
"exclude": false
}
]
}

Benefits:

  • ✅ Single network call
  • ✅ Defined once, used everywhere
  • ✅ Version controlled
  • ✅ Server-side execution
  • ✅ Automatic parameter validation

Architecture

Operation Types

ekoDB Functions support these step types (the type field of each step). This is the complete catalog.

Query & Read

  • FindAll - Retrieve all records in a collection
  • Query - Filter, sort, and paginate (filter/sort/limit/skip are inline on this step)
  • FindById - Get one record by ID
  • FindOne - Find one record by key/value
  • TextSearch - Full-text search
  • VectorSearch - Semantic similarity (vector) search
  • HybridSearch - Combined text + vector search

Write (CRUD)

  • Insert - Insert one record
  • BatchInsert - Insert many records
  • Update - Update records matching a filter
  • UpdateById - Update one record by ID
  • FindOneAndUpdate - Atomic find-and-update
  • UpdateWithAction - Atomic field action (increment, push, append, pop, etc.)
  • Upsert - Atomic find-or-create by lookup key
  • Delete - Delete records matching a filter
  • DeleteById - Delete one record by ID
  • BatchDelete - Delete many records by ID

In-pipeline record edits

  • SetField - Set a literal value on the working record
  • Increment - Atomic counter add on a stored record
  • Push - Atomic array append on a stored record
  • CurrentDatetime - Stamp the current UTC time onto the working record

Aggregation & shaping

  • Group - Group and aggregate (11 operations: Count, Sum, Average, Min, Max, First, Last, Push, AddToSet, StandardDeviation, ApproxDistinct)
  • AddFields - Add computed/derived fields (arithmetic, string, and conditional expressions)
  • Project - Select or exclude fields
  • Count - Add a record-count field

Control flow

  • If - Branch on a FunctionCondition
  • ForEach - Iterate over the working records, running sub-steps per record
  • Parallel - Run sub-steps concurrently and merge their results
  • TryCatch - Run steps; on error, run the catch steps (error bound to output_error_field)
  • Sleep - Pause (e.g. for rate limits or retry backoff)
  • Return - Build the final response object (optionally setting status_code for the HTTP status)
  • Validate - Assert a condition; fail the pipeline if it doesn't hold
  • CallFunction - Call another saved function by label (composition)

Transactions

  • CreateSavepoint / RollbackToSavepoint / ReleaseSavepoint - Nested-transaction savepoints for partial rollback

A saved function with transaction_config.enabled = true runs its steps as one atomic unit: every applied write is tracked and the per-transaction undo log reverts all of them on any failure. This is the recommended way to do a multi-statement unit, and it gives you atomicity, consistency, and durability (plus savepoints). It does not add isolation — each step applies to the live store immediately (apply-then-undo), so a concurrent operation can observe a transactional function's uncommitted intermediate writes. Enforced transaction isolation between concurrent transactions is on the roadmap.

AI

  • Chat - LLM chat completion
  • Embed - Generate embeddings for a field

Auth & crypto (build login/signup without external services)

  • BcryptHash / BcryptVerify - Password hashing and verification
  • JwtSign / JwtVerify - Issue and verify JWTs
  • HmacSign / HmacVerify - HMAC sign and timing-safe verify
  • AesEncrypt / AesDecrypt - AES-GCM encrypt/decrypt
  • TotpGenerate / TotpVerify - TOTP MFA codes
  • RandomToken - CSPRNG random token
  • UuidGenerate - Generate a UUID
  • Base64Encode / Base64Decode - Base64
  • HexEncode / HexDecode - Hex
  • Slugify - URL-safe slug

Key-value store

  • KvGet / KvSet / KvDelete / KvExists / KvQuery - KV cache operations (TTL, prefix scan)

Caching & concurrency

  • SWR - Stale-while-revalidate cached pipeline (cache check → run → cache set)
  • IdempotencyClaim - Claim an idempotency key (once-only execution)
  • RateLimit - Fixed-window rate limit
  • LockAcquire / LockRelease - Distributed lock

External integrations

  • HttpRequest - Call an external HTTP API (Stripe, SendGrid, etc.)
  • EmailSend - Send an email

Parameter System

Functions support dynamic parameterization using {{parameter_name}} syntax:

{
"parameters": {
"user_id": {
"type": "string",
"required": true,
"description": "User identifier"
},
"limit": {
"type": "number",
"required": false,
"default": 10,
"description": "Result limit"
},
"status": {
"type": "string",
"required": false,
"default": "active",
"enum": ["active", "inactive", "pending"]
}
},
"functions": [
{
"type": "Query",
"collection": "users",
"filter": {
"type": "Logical",
"content": {
"operator": "And",
"expressions": [
{
"type": "Condition",
"content": {
"field": "id",
"operator": "Eq",
"value": "{{user_id}}"
}
},
{
"type": "Condition",
"content": {
"field": "status",
"operator": "Eq",
"value": "{{status}}"
}
}
]
}
},
"limit": "{{limit}}"
}
]
}

Parameter Resolution:

  1. User provides values when calling Function
  2. ekoDB validates types and requirements
  3. Parameters are substituted before execution
  4. Results can reference previous Function outputs

Result Chaining

Operations execute sequentially, with results flowing from one to the next:

{
"functions": [
{
"type": "Query",
"collection": "products",
"filter": {
"type": "Condition",
"content": {
"field": "category",
"operator": "Eq",
"value": "{{category}}"
}
}
},
{
"type": "Count",
"output_field": "product_count"
},
{
"type": "Project",
"fields": ["name", "price", "product_count"],
"exclude": false
}
]
}

How Results Flow:

  • Query operation returns records
  • Count adds a count field to those records
  • Project selects only specified fields
  • Each operation works with the current set of records

Execution Model

Sequential Execution

By default, Functions execute in sequence:

Function 1 → Function 2 → Function 3 → Result

Each Function waits for the previous to complete before starting.

Conditional Execution

Use the If Function type for conditional logic:

{
"type": "If",
"condition": {
"type": "FieldEquals",
"value": { "field": "plan", "value": "premium" }
},
"then_functions": [{ "type": "Query", "collection": "premium_features" }],
"else_functions": [{ "type": "Query", "collection": "basic_features" }]
}

condition is a FunctionCondition object (adjacently tagged with type + value), evaluated against the working records — not a string expression. The branches are then_functions and else_functions.

FunctionCondition variants:

  • Field comparisons: FieldEquals, FieldExists, FieldGreaterThan, FieldLessThan, FieldGreaterThanOrEqual, FieldLessThanOrEqual — e.g. { "type": "FieldGreaterThan", "value": { "field": "stock", "value": 0 } }
  • Set/count: HasRecords, CountEquals, CountGreaterThan, CountLessThan
  • Boolean: And, Or, Not — e.g. { "type": "And", "value": { "conditions": [<FunctionCondition>, ...] } }

Loop Execution

Use the ForEach operation type to iterate over current records:

{
"functions": [
{
"type": "Query",
"collection": "users",
"filter": {
"type": "Condition",
"content": {
"field": "status",
"operator": "Eq",
"value": "pending"
}
}
},
{
"type": "ForEach",
"functions": [
{
"type": "UpdateById",
"collection": "users",
"record_id": "{{id}}",
"updates": { "notified": true }
}
]
}
]
}

How it works:

  • Operates on records from previous operations
  • Executes nested operations for each record
  • Each record's fields are available as variables (e.g., {{id}}, {{email}})

Creating Functions

What is a Function?

A Function is a named collection of operations that can be leveraged to perform complex logic and transformations. Each function is versioned for friendly maintenance:

  • Stored in ekoDB
  • Called by label or ID
  • Nested using CallFunction
  • Parameterized with dynamic values

Function Structure

FieldDescriptionRequired
labelUnique identifierYes
nameHuman-readable nameYes
descriptionWhat the function doesNo
versionSemantic versionNo
parametersInput parametersNo
functionsArray of operationsYes
tagsCategorization tagsNo

Creating a Function

Minimal Example (Required Fields Only):

{
"label": "get_user",
"name": "Get User by ID",
"functions": [
{
"type": "FindOne",
"collection": "users",
"filter": { "id": "{{user_id}}" }
}
]
}

Full Example (With Optional Fields):

{
"label": "user_onboarding",
"name": "User Onboarding Flow",
"description": "Complete new user setup with AI welcome message",
"version": "1.0.0",
"tags": ["onboarding", "users", "ai"],
"parameters": {
"email": { "type": "string", "required": true },
"name": { "type": "string", "required": true }
},
"functions": [
{
"type": "Insert",
"collection": "users",
"record": {
"email": "{{email}}",
"name": "{{name}}"
}
},
{
"type": "Chat",
"messages": [
{
"role": "system",
"content": "You are a friendly onboarding assistant"
},
{
"role": "user",
"content": "Write a welcome message for {{name}}"
}
],
"model": "gpt-4"
}
]
}

Calling a Function

From Client Library:

const result = await client.callFunction("user_onboarding", {
email: "user@example.com",
name: "John Doe",
});

From REST API:

POST /api/functions/user_onboarding
Content-Type: application/json

{
"email": "user@example.com",
"name": "John Doe"
}

Response format:

A call returns { "records": [...], "stats": {...} }. The record field values follow the same use_typed_values rule as find and query: with use_typed_values enabled they are wrapped ({"type":"String","value":"..."}), and with it disabled they are bare. A client configured for typed values therefore gets the same shape from callFunction as from a read.

This wrapping is applied only on the response to an external caller. Inside the pipeline, and when one function calls another with CallFunction, records flow as bare native values so that {{record.field}} access keeps working in the calling function.

A Return step may set status_code to control the HTTP status of the response (for example 201 after a create, or 404 when a lookup misses). The code sets the actual HTTP status; it does not appear in the response body.

Function Nesting

Functions can call other Functions for modularity using CallFunction:

{
"label": "process_order",
"functions": [
{
"type": "CallFunction",
"function_label": "validate_inventory",
"params": { "product_id": "{{product_id}}" }
},
{
"type": "CallFunction",
"function_label": "charge_payment",
"params": { "amount": "{{amount}}" }
},
{
"type": "CallFunction",
"function_label": "send_confirmation",
"params": { "order_id": "{{order_id}}" }
}
]
}

Execution:

process_order
├─ validate_inventory
│ ├─ Query products
│ └─ Update inventory
├─ charge_payment
│ └─ HttpRequest to Stripe
└─ send_confirmation
├─ Chat (AI message)
└─ HttpRequest to email API

Versioning

Why Version Functions?

  • Evolution - Update logic without breaking existing callers
  • Rollback - Revert to previous versions if issues arise
  • Testing - Test new versions alongside old ones
  • Auditing - Track changes over time

Version Strategy

{
"label": "calculate_pricing",
"version": "2.0", // Increment on breaking changes
"functions": [...]
}

Best Practices:

  • Use semantic versioning: major.minor.patch
  • Major version: Breaking changes
  • Minor version: New features (backward compatible)
  • Patch version: Bug fixes

Calling Specific Versions

// Call latest version
await client.callFunction("calculate_pricing", params);

// Call specific version (if supported)
await client.callFunction("calculate_pricing", params, { version: "1.0" });

AI Integration

Chat Functions

Execute LLM completions with context:

{
"type": "Chat",
"messages": [
{
"role": "system",
"content": "You are a data analyst"
},
{
"role": "user",
"content": "Analyze this data: {{query_results}}"
}
],
"model": "gpt-4",
"temperature": 0.7,
"max_tokens": 1000
}

Supported Models:

  • OpenAI: gpt-4, gpt-4-turbo, gpt-3.5-turbo
  • Anthropic: claude-3-opus, claude-3-sonnet
  • Perplexity: pplx-7b-online, pplx-70b-online

Embedding Functions

Generate vector embeddings for text:

{
"type": "Embed",
"input_field": "description",
"output_field": "description_vector",
"model": "text-embedding-ada-002"
}

Use Cases:

  • Semantic search
  • Document similarity
  • Recommendation systems
  • Clustering

RAG Workflows

Build complete RAG pipelines by combining search and AI:

{
"label": "semantic_search",
"parameters": {
"query": { "type": "string", "required": true }
},
"functions": [
{
"type": "TextSearch",
"collection": "knowledge_base",
"query_text": "{{query}}",
"limit": 5
},
{
"type": "Chat",
"messages": [
{
"role": "system",
"content": "Answer using only the provided context"
},
{
"role": "user",
"content": "Context: {{search_results}}\n\nQuestion: {{query}}"
}
],
"model": "gpt-4"
}
]
}

Note: For true vector search RAG, first generate embeddings for your documents using the Embed operation, then use VectorSearch to find similar content.

External Integrations

HTTP Requests

Call external APIs from Functions:

{
"type": "HttpRequest",
"method": "POST",
"url": "https://api.stripe.com/v1/charges",
"headers": {
"Authorization": "Bearer {{stripe_key}}",
"Content-Type": "application/x-www-form-urlencoded"
},
"body": {
"amount": "{{amount}}",
"currency": "usd",
"source": "{{token}}"
},
"output_field": "charge_result"
}

Response Handling:

The HTTP response is stored in the specified output_field (default: http_response):

  • JSON responses are automatically parsed into native Objects/Arrays
  • Non-JSON responses (HTML, plain text) are stored as Strings

You can reference the response in subsequent stages using the field name specified in output_field (e.g., {{charge_result}} in the example above).

Use Cases:

  • Payment processing (Stripe, PayPal)
  • Email services (SendGrid, Mailgun)
  • SMS notifications (Twilio)
  • Webhook triggers
  • Third-party API integration

Performance Considerations

Execution Time

  • Operation overhead: ~1-5ms per operation
  • Network calls: Variable (external APIs, AI)
  • Database operations: Sub-millisecond for queries
  • Total Function time: Sum of all operation times

Optimization Strategies

  1. Batch operations instead of loops when possible
  2. Limit result sets with filters and projections
  3. Filter early - Reduce data before expensive operations
  4. Cache Function definitions (auto-cached by ekoDB)
  5. Minimize external HTTP calls

Stale-While-Revalidate (SWR) Pattern

Functions are perfect for the SWR pattern, which dramatically improves perceived performance by serving cached results while fetching fresh data in the background.

How it works:

1. Client requests Function execution
2. Return cached result immediately (if exists)
3. Execute Function in background
4. Update cache with fresh result
5. Next request gets updated data

Benefits:

  • Instant response - Users see data immediately
  • 🔄 Always fresh - Background updates keep data current
  • 📉 Reduced load - Fewer Function executions
  • 🎯 Better UX - No loading spinners for cached data

When to use:

  • ✅ Dashboard summaries that don't change frequently
  • ✅ User profile data (name, email, settings)
  • ✅ Product catalogs with periodic updates
  • ✅ Analytics data that's expensive to compute
  • ❌ Real-time data (stock prices, live scores)
  • ❌ User-specific transactional data (cart, orders)

Example pattern:

// Client-side SWR with Functions
async function getUserDashboard(userId: string) {
// 1. Return cached data immediately
const cached = cache.get(`dashboard:${userId}`);
if (cached) {
// Serve stale data instantly
displayDashboard(cached);

// 2. Revalidate in background
client.callFunction("user_dashboard", { user_id: userId }).then((fresh) => {
cache.set(`dashboard:${userId}`, fresh);
displayDashboard(fresh); // Update UI with fresh data
});

return cached;
}

// 3. No cache - fetch and display
const data = await client.callFunction("user_dashboard", { user_id: userId });
cache.set(`dashboard:${userId}`, data);
return data;
}

Server-side caching:

Functions can also implement internal caching for even better performance:

{
"label": "expensive_analytics",
"functions": [
{
"type": "Query",
"collection": "events",
"filter": {
"type": "Condition",
"content": {
"field": "date",
"operator": "Eq",
"value": "{{today}}"
}
}
},
{
"type": "Group",
"by_fields": ["category"],
"functions": [{ "output_field": "count", "operation": "Count" }]
}
]
}

Call this Function with a cache key and TTL, and ekoDB can cache results server-side for all clients.

Example Code

SWR Pattern Examples in all languages:

Edge Cache Pattern - Use ekoDB as an edge cache with external API caching:

Best Practices

DO:

  • ✅ Keep Functions focused and single-purpose
  • ✅ Use meaningful labels and descriptions
  • ✅ Document parameters clearly
  • ✅ Version Functions properly
  • ✅ Test with sample data before production

DON'T:

  • ❌ Create mega-Functions with 50+ operations
  • ❌ Use Functions for simple single operations
  • ❌ Hard-code values (use parameters)
  • ❌ Nest Functions more than 3-4 levels deep
  • ❌ Ignore error handling

Error Handling

Operation-Level Errors

If an operation fails:

  1. Execution stops immediately
  2. Error details are returned
  3. No rollback (unless in transaction)
  4. Caller receives error response
{
"error": true,
"operation_index": 2,
"operation_type": "Chat",
"message": "API rate limit exceeded",
"details": {...}
}

Error Recovery

To catch a failing step and run fallback logic, use TryCatch (the step's error is bound to output_error_field):

{
"type": "TryCatch",
"try_functions": [
{ "type": "Chat", "prompt": "{{question}}", "output_field": "reply" }
],
"catch_functions": [
{
"type": "Insert",
"collection": "error_log",
"record": { "error": "{{error}}" }
}
],
"output_error_field": "error"
}

If branches on a FunctionCondition evaluated against the working records — note condition is an object (not a string), and the branches are then_functions / else_functions:

{
"type": "If",
"condition": { "type": "HasRecords" },
"then_functions": [
{ "type": "Insert", "collection": "audit", "record": { "event": "found" } }
],
"else_functions": []
}

Transactions

Wrap operations in transactions for atomicity:

{
"transaction": true,
"functions": [
{ "type": "Update", "collection": "inventory", "..." },
{ "type": "Insert", "collection": "orders", "..." }
]
}

If any operation fails, all changes are rolled back.

Security

Permission Model

Functions execute with the caller's permissions:

  • Collection-level access control applies
  • Field-level permissions enforced
  • API key scoping respected

Parameter Validation

All parameters are validated before execution:

  • Type checking (string, number, boolean, array, object)
  • Required field validation
  • Enum value checking
  • Custom validation rules

Injection Prevention

ekoDB automatically:

  • ✅ Sanitizes all parameter inputs
  • ✅ Prevents NoSQL injection
  • ✅ Escapes special characters
  • ✅ Validates JSON structure

Safe:

{
"filter": { "email": "{{user_email}}" }
}

User cannot inject malicious queries.

Use Cases

1. User Authentication Flow

{
"label": "get_user_by_email",
"parameters": {
"email": { "type": "string", "required": true }
},
"functions": [
{
"type": "FindOne",
"collection": "users",
"key": "email",
"value": "{{email}}"
}
]
}

2. E-commerce Order Processing

{
"label": "create_order",
"parameters": {
"user_id": { "type": "string", "required": true },
"product_id": { "type": "string", "required": true }
},
"functions": [
{
"type": "FindById",
"collection": "products",
"record_id": "{{product_id}}"
},
{
"type": "Insert",
"collection": "orders",
"record": {
"user_id": "{{user_id}}",
"product_id": "{{product_id}}",
"status": "pending"
}
}
]
}

3. AI Content Moderation

{
"label": "moderate_content",
"parameters": {
"content": { "type": "string", "required": true }
},
"functions": [
{
"type": "Chat",
"messages": [
{
"role": "system",
"content": "Analyze if content is appropriate and respond with 'SAFE' or 'UNSAFE'"
},
{
"role": "user",
"content": "{{content}}"
}
],
"model": "gpt-4"
}
]
}

4. Data Analytics Pipeline

{
"label": "event_summary",
"parameters": {
"date": { "type": "string", "required": true }
},
"functions": [
{
"type": "Query",
"collection": "events",
"filter": {
"type": "Condition",
"content": {
"field": "date",
"operator": "Eq",
"value": "{{date}}"
}
}
},
{
"type": "Count",
"output_field": "total_events"
}
]
}

Comparison with Alternatives

vs. Stored Procedures (SQL)

FeatureekoDB FunctionsSQL Stored Procedures
LanguageJSON (declarative)SQL (procedural)
VersioningBuilt-inManual
AI IntegrationNativeNone
External APIsYesLimited
PortabilityJSON formatDatabase-specific

vs. Cloud Functions (AWS Lambda, etc.)

FeatureekoDB FunctionsCloud Functions
ExecutionServer-side in ekoDBSeparate compute
NetworkSingle callMultiple calls
StateDirect DB accessMust connect to DB
DeploymentJSON definitionCode deployment
CostIncludedPer-invocation

vs. Application Code

FeatureekoDB FunctionsApplication Code
LocationServer-sideClient-side
ReusabilityAll clientsSingle codebase
PerformanceLow latencyMultiple round-trips
VersioningBuilt-inGit/deployment
TestingIsolatedFull app context

Next Steps

📚 Explore Function Examples

Complete working examples in all languages:

  • Function Composition Examples - Full-featured examples:

    • Rust: client_function_composition.rs
    • Python: client_function_composition.py
    • TypeScript: client_function_composition.ts
    • JavaScript: client_function_composition.js
    • Go: client_functions.go
    • Kotlin: ClientFunctionComposition.kt
  • RAG & AI Examples - Retrieval-augmented generation:

    • Rust: rag_conversation_system.rs
    • Python: rag_conversation_system.py
    • TypeScript: rag_conversation_system.ts
    • Go: rag_conversation_system.go
    • Kotlin: RagConversationSystem.kt
  • SWR Pattern - Stale-while-revalidate with Functions:

Learning Path

  1. Learn by example - Browse the GitHub examples above
  2. Start simple - Create a basic Query Function
  3. Add parameters - Make Functions dynamic
  4. Compose Functions - Combine Functions for workflows
  5. Integrate AI - Add Chat and Embed Functions

Need help? Submit a support ticket or open an issue