Ripples - Data Propagation

Ripples enable real-time data propagation across multiple ekoDB instances, providing horizontal scaling, geographic distribution, and high availability through automatic replication.

What are Ripples?

Ripples are ekoDB's built-in data propagation system that replicates database operations (inserts, updates, deletes) across connected ekoDB instances, either as application-level operations (Operations mode) or by shipping committed WAL entries (WAL mode).

Implementation status

Ripples today is an ekoDB→ekoDB system. Two modes are wired and production-ready:

Operations mode — real-time, one-way CRUD propagation with loop prevention.
WAL mode — interval-based committed-WAL shipping to replicas (see WAL Mode).

The other modes and destinations described below are roadmap, not current behavior: Snapshot, Search, Analytics, Stream, Webhook, Embedding, and Chat modes — and all foreign destinations (Elasticsearch, BigQuery, Kafka, PostgreSQL, etc.) — are scaffolded but not wired to the write path. PropagationFilter / TransformConfig are accepted but not yet enforced. Bidirectional / active-active is supported (both nodes writable, with automatic loop prevention), but there is no conflict resolution — concurrent writes to the same record on different nodes diverge (last write by arrival), so active-active is safe only when writes are partitioned by node or contention is low.

Ripples handles forward change propagation only. Base-state seeding, backups, and disaster recovery are provided by the managed deployment platform, not by Ripples: new nodes boot their data disk from a snapshot, deployments take automated disk-snapshot backups, and a deployment can be cloned from a snapshot. So a read replica is seeded from a snapshot (base state) and then kept current by WAL-mode shipping (forward changes) — see System Administration for backup/restore.

Key Features (today):

Real-Time Propagation: Operations replicate immediately as they occur (Operations mode)
WAL shipping: committed WAL entries streamed to replicas on an interval (WAL mode)
Selective Replication: Configure which nodes receive which operations
Loop Prevention: Automatic deduplication prevents infinite loops
Bidirectional Sync / Active-Active: Nodes can both send and receive (loop-prevented). Works as active-active — but there's no conflict resolution, so partition writes by node to avoid same-record divergence

How Ripples Work

When a write operation occurs on a node:

Operation Executes: Record is inserted/updated/deleted locally
Ripple Generated: Operation metadata is packaged with unique ID
Propagation: Ripple is sent to configured peer nodes
Remote Execution: Peer nodes receive and execute the operation
Loop Prevention: Request ID prevents duplicate processing

┌─────────────┐
│   Node 1    │  ← Client writes here
│  (Primary)  │
└──────┬──────┘
       │ Ripple (INSERT)
       ├───────────────────┐
       ↓                   ↓
┌─────────────┐     ┌─────────────┐
│   Node 2    │     │   Node 3    │
│  (Replica)  │     │  (Replica)  │
└─────────────┘     └─────────────┘

Ripple Propagation Modes

Ripples support multiple propagation modes for different use cases. The mode field in the configuration determines how data is formatted and transmitted.

Quick Mode Selection Guide

Mode	Status	Primary Use Case	Destination
Operations (default)	✅ Available	Multi-node database replication	ekoDB instances
WAL	✅ Available	High-volume efficient replication	ekoDB instances
Snapshot	🚧 Roadmap	Periodic backups, data warehousing	ekoDB, storage systems
Search	🚧 Roadmap	Full-text search integration	Elasticsearch, Meilisearch
Analytics	🚧 Roadmap	Business intelligence, reporting	BigQuery, Snowflake
Stream	🚧 Roadmap	Event-driven architectures	Kafka, RabbitMQ
Webhook	🚧 Roadmap	Third-party integrations	HTTP endpoints
Embedding	🚧 Roadmap	Semantic search, AI/ML	Vector databases
Chat	🚧 Roadmap	AI chatbots, LLM context	OpenAI, custom LLMs
Custom	🚧 Roadmap	Specialized integrations	Custom systems

Recommended Mode

Use Operations mode (the default) for standard multi-node replication, or WAL mode for high-volume forward replication to read replicas. The 🚧 Roadmap modes above are the intended design but are not yet wired — see the implementation-status note at the top.

Operations Mode (Default)

Purpose: Real-time CRUD operation replication between ekoDB instances

Use Case: Multi-node clusters, horizontal scaling, high availability

How it works: Each insert/update/delete is sent as an individual operation to peer nodes in real-time.

Example Configuration:

POST /api/ripples/config
{
  "name": "replica1",
  "url": "https://replica1.ekodb.net:8080",
  "api_key": "replica1-admin-key",
  "mode": "Operations",  # Default mode
  "enabled": true
}

Best for: Standard database replication and read replicas. Also supports active-active / bidirectional (both nodes writable, loop-prevented) — just partition writes by node, since there's no conflict resolution for concurrent same-record writes (see the implementation-status note).

WAL Mode

Purpose: Batch Write-Ahead Log shipping for efficient replication

Use Case: Large-scale replication with lower network overhead

How it works: A background loop on the primary ships committed WAL entries to WAL-mode replicas on the wal_ship_interval_secs interval (default 5s, set via PUT /api/config). Shipping is incremental and crash-safe — the primary tracks a persisted position and only sends entries committed since the last successful ship — and the replica deduplicates by sequence, so a re-ship after a crash is idempotent. The replica applies inserts/updates/deletes (KV entries replicate via Operations mode instead).

Scope: WAL mode ships forward changes one-way (primary→replica). Seeding a replica's base state is handled by the managed deployment platform — new nodes boot their disk from a snapshot and deployments take automated snapshot backups (see System Administration) — so the usual flow is: snapshot-seed the replica, then WAL-ship forward changes onto it.

Example Configuration:

POST /api/ripples/config
{
  "name": "wal_replica",
  "url": "https://replica.ekodb.net:8080",
  "api_key": "replica-admin-key",
  "mode": "WAL",
  "strategy": "batched",
  "options": {
    "batch_size": 5000,
    "interval_secs": 10
  },
  "enabled": true
}

Best for: High-volume replication, reducing network traffic, eventual consistency scenarios

Roadmap modes

The modes from Snapshot through Custom below (and all non-ekoDB destinations) describe the intended design and are not yet wired to the write path — see the implementation-status note above. The configuration examples show the planned shape, not behavior you can rely on today. Use Operations or WAL mode for real replication.

Snapshot Mode

Purpose: Periodic full database snapshots

Use Case: Backup nodes, reporting databases, point-in-time recovery

How it works: Sends complete snapshots of collections on a schedule rather than individual operations.

Example Configuration:

POST /api/ripples/config
{
  "name": "backup_snapshot",
  "url": "https://backup.ekodb.net:8080",
  "api_key": "backup-admin-key",
  "mode": "Snapshot",
  "strategy": "scheduled",
  "options": {
    "cron": "0 2 * * *",  # Daily at 2 AM
    "collections": ["users", "orders"]
  },
  "enabled": true
}

Best for: Backup systems, data warehousing, periodic sync to reporting databases

Search Mode

Purpose: Send data formatted for search indexing

Use Case: Elasticsearch, Meilisearch, or other search engine integration

How it works: Transforms operations into search-optimized format and sends to search indices.

Example Configuration:

POST /api/ripples/config
{
  "name": "elasticsearch_index",
  "url": "https://elasticsearch.example.com:9200",
  "api_key": "es-api-key",
  "mode": "Search",
  "destination": "elasticsearch",
  "transform": {
    "include_fields": ["title", "content", "author", "tags"],
    "exclude_fields": ["internal_id"]
  },
  "enabled": true
}

Best for: Full-text search, faceted search, search-as-you-type features

Analytics Mode

Purpose: Send data to analytics platforms

Use Case: BigQuery, Snowflake, data warehouses, BI tools

How it works: Formats data for analytics schemas and sends to analytics platforms in batches.

Example Configuration:

POST /api/ripples/config
{
  "name": "bigquery_analytics",
  "url": "https://bigquery.googleapis.com/v2/projects/my-project",
  "api_key": "bq-service-account-key",
  "mode": "Analytics",
  "destination": "bigquery",
  "filter": {
    "collections": ["events", "metrics", "user_activity"]
  },
  "options": {
    "dataset": "production_analytics",
    "batch_size": 10000
  },
  "enabled": true
}

Best for: Business intelligence, data analytics, reporting dashboards

Stream Mode

Purpose: Send data to message streams

Use Case: Kafka, RabbitMQ, event-driven architectures

How it works: Publishes operations as messages to streaming platforms for event processing.

Example Configuration:

POST /api/ripples/config
{
  "name": "kafka_stream",
  "url": "https://kafka.example.com:9092",
  "api_key": "kafka-api-key",
  "mode": "Stream",
  "destination": "kafka",
  "options": {
    "topic": "ekodb-events",
    "partition_key": "collection_name"
  },
  "enabled": true
}

Best for: Event sourcing, microservices communication, real-time data pipelines

Webhook Mode

Purpose: Trigger HTTP webhooks on data changes

Use Case: Third-party integrations, automation, notifications

How it works: Sends HTTP POST requests to configured webhook URLs when operations occur.

Example Configuration:

POST /api/ripples/config
{
  "name": "slack_notifications",
  "url": "https://hooks.slack.com/services/YOUR/WEBHOOK/URL",
  "mode": "Webhook",
  "filter": {
    "collections": ["orders"],
    "operations": ["insert"]
  },
  "transform": {
    "transform_fn": "format_slack_message"
  },
  "enabled": true
}

Best for: Slack/Discord notifications, Zapier integration, custom automation

Embedding Mode

Purpose: Generate and send vector embeddings

Use Case: Vector databases, semantic search, AI/ML pipelines

How it works: Processes text fields, generates embeddings, and sends to vector databases.

Example Configuration:

POST /api/ripples/config
{
  "name": "pinecone_vectors",
  "url": "https://your-index.pinecone.io",
  "api_key": "pinecone-api-key",
  "mode": "Embedding",
  "destination": "vectordb",
  "options": {
    "embedding_field": "content",
    "embedding_model": "text-embedding-ada-002",
    "dimension": 1536
  },
  "enabled": true
}

Best for: Semantic search, RAG (Retrieval Augmented Generation), similarity matching

Chat Mode

Purpose: Send data to chat/LLM systems

Use Case: AI assistants, chatbots, context-aware LLMs

How it works: Formats data for chat context and sends to LLM providers or chat databases.

Example Configuration:

POST /api/ripples/config
{
  "name": "openai_context",
  "url": "https://api.openai.com/v1/assistants/asst_xxx/messages",
  "api_key": "openai-api-key",
  "mode": "Chat",
  "filter": {
    "collections": ["support_tickets", "knowledge_base"]
  },
  "transform": {
    "include_fields": ["question", "answer", "category"]
  },
  "enabled": true
}

Best for: AI chatbots, customer support automation, context-aware assistants

Custom Mode

Purpose: User-defined custom propagation logic

Use Case: Specialized integrations, custom data pipelines

How it works: Allows you to define custom propagation behavior via configuration.

Example Configuration:

POST /api/ripples/config
{
  "name": "custom_pipeline",
  "url": "https://custom-endpoint.example.com/ingest",
  "api_key": "custom-api-key",
  "mode": {
    "Custom": "my_custom_handler"
  },
  "options": {
    "custom_param1": "value1",
    "custom_param2": "value2"
  },
  "enabled": true
}

Best for: Proprietary systems, specialized data transformations, unique integrations

Replication Patterns (Send/Receive/Both)

Beyond the propagation mode, nodes can be configured in different replication patterns:

Send Mode

Node sends ripples to peers but doesn't receive.

Use Case: Primary write nodes that propagate to replicas

Configure each peer separately:

# Add first replica peer
POST /api/ripples/config
{
  "name": "replica1",
  "url": "https://replica1.ekodb.net:8080",
  "api_key": "replica1-admin-key",
  "mode": "Operations",
  "enabled": true
}

# Add second replica peer
POST /api/ripples/config
{
  "name": "replica2",
  "url": "https://replica2.ekodb.net:8080",
  "api_key": "replica2-admin-key",
  "mode": "Operations",
  "enabled": true
}

Receive Mode

Node receives ripples from peers but doesn't send.

Use Case: Read replicas that stay synchronized

Note: Receive-only nodes don't need to configure peer connections. They simply process incoming ripple requests from nodes that have them configured as peers.

Both Mode (Full Mesh)

Node both sends and receives ripples.

Use Case: Multi-master deployments, peer-to-peer sync

Configure each peer separately:

# On Node 1: Add Node 2 as peer
POST /api/ripples/config
{
  "name": "node2",
  "url": "https://node2.ekodb.net:8080",
  "api_key": "node2-admin-key",
  "mode": "Operations",
  "enabled": true
}

# On Node 2: Add Node 1 as peer
POST /api/ripples/config
{
  "name": "node1",
  "url": "https://node1.ekodb.net:8080",
  "api_key": "node1-admin-key",
  "mode": "Operations",
  "enabled": true
}

None Mode

Node operates independently with no ripples.

Use Case: Isolated analytics nodes, testing environments

No configuration needed: Simply don't add any ripple peers to the node.

Real-World Use Cases

1. Geographic Distribution (Multi-Region)

Scenario: E-commerce platform with users in US, EU, and Asia

Architecture:

Primary node in each region (both mode for bidirectional sync)
Cross-region ripples for data consistency
Read replicas in each region (receive mode)

US Region                EU Region               Asia Region
┌──────────┐            ┌──────────┐            ┌──────────┐
│ Primary  │◄──────────►│ Primary  │◄──────────►│ Primary  │
│  (both)  │            │  (both)  │            │  (both)  │
└────┬─────┘            └────┬─────┘            └────┬─────┘
     │                       │                       │
     ├─────┐            ├────┴────┐            ├────┴────┐
     ↓     ↓            ↓         ↓            ↓         ↓
┌────────┐┌────────┐┌────────┐┌────────┐┌────────┐┌────────┐
│Replica1││Replica2││Replica1││Replica2││Replica1││Replica2│
│(receive)││(receive)││(receive)││(receive)││(receive)││(receive)│
└────────┘└────────┘└────────┘└────────┘└────────┘└────────┘

Benefits:

Users read from nearest region (low latency)
Writes propagate globally (eventual consistency)
Failover to another region if one goes down

2. Read Scaling (Analytics Workload)

Scenario: SaaS application with heavy reporting queries

Architecture:

Primary write node with durable WAL (send mode)
Multiple read replicas with fast WAL (receive mode)
Analytics nodes isolated from production (none mode)

Production Traffic            Analytics Traffic
       │                             │
       ↓                             ↓
┌─────────────┐               ┌─────────────┐
│   Primary   │──────────────►│  Analytics  │
│   (send)    │    WAL Export │   (none)    │
│ Durable WAL │               │  Fast WAL   │
└──────┬──────┘               └─────────────┘
       │ Ripples
       ├──────────┬──────────┐
       ↓          ↓          ↓
┌──────────┐┌──────────┐┌──────────┐
│ Replica1 ││ Replica2 ││ Replica3 │
│(receive) ││(receive) ││(receive) │
│ Fast WAL ││ Fast WAL ││ Fast WAL │
└──────────┘└──────────┘└──────────┘

Benefits:

Production writes don't slow down (durable primary)
Replicas handle read traffic (fast WAL, high throughput)
Analytics isolated (no ripple overhead, batch data loads)

Configuration Example:

# Primary (write node) - Add each replica as a peer
curl -X POST https://primary.ekodb.net/api/ripples/config \
  -H "Authorization: Bearer $ADMIN_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "replica1",
    "url": "https://replica1.ekodb.net:8080",
    "api_key": "replica1-admin-key",
    "mode": "Operations",
    "enabled": true
  }'

curl -X POST https://primary.ekodb.net/api/ripples/config \
  -H "Authorization: Bearer $ADMIN_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "replica2",
    "url": "https://replica2.ekodb.net:8080",
    "api_key": "replica2-admin-key",
    "mode": "Operations",
    "enabled": true
  }'

curl -X POST https://primary.ekodb.net/api/ripples/config \
  -H "Authorization: Bearer $ADMIN_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "replica3",
    "url": "https://replica3.ekodb.net:8080",
    "api_key": "replica3-admin-key",
    "mode": "Operations",
    "enabled": true
  }'

# Read Replicas (no peer config needed - they only receive)
# Analytics node (isolated - no peer config needed)

3. High Availability (Active-Active)

Active-active works, but there is no conflict resolution

You can run two writable nodes peering at each other — bidirectional propagation with automatic loop prevention (request-ID dedup, origin rejection, hop-count cap) is implemented and works. What's missing is conflict resolution: a received write is blind-applied (no last-write-wins, no vector clocks), so two nodes writing the same record concurrently will diverge (each keeps the other's value). This is safe when writes are partitioned by node (each node owns distinct keys/regions) or contention is low; avoid it for hot-key contention. Note also that Ripples does not auto-promote on failover — route writes with an external load balancer/health check.

Scenario: Multi-region app where each region owns its own keyspace

Architecture:

Two writable nodes, peered bidirectionally (both direction)
Writes partitioned by region/key so the same record isn't written on both
External load balancer for routing (no automatic promotion in Ripples)
Conflict handling: none — last write by arrival (partition writes to avoid same-key conflicts)

┌──────────────┐
│Load Balancer │
└──────┬───────┘
       │
   ┌───┴────┐
   ↓        ↓
┌────────┐┌────────┐
│Primary1││Primary2│
│ (both) ││ (both) │
└────┬───┘└───┬────┘
     └────┬───┘
          │ Bidirectional Ripples

Benefits:

Both nodes accept writes (active-active)
Automatic failover (no manual intervention)
Zero downtime for maintenance

Example Workflow:

# Node 1 configuration - Add Node 2 as peer
curl -X POST https://node1.ekodb.net/api/ripples/config \
  -H "Authorization: Bearer $ADMIN_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "node2",
    "url": "https://node2.ekodb.net:8080",
    "api_key": "node2-admin-key",
    "mode": "Operations",
    "enabled": true
  }'

# Node 2 configuration - Add Node 1 as peer
curl -X POST https://node2.ekodb.net/api/ripples/config \
  -H "Authorization: Bearer $ADMIN_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "node1",
    "url": "https://node1.ekodb.net:8080",
    "api_key": "node1-admin-key",
    "mode": "Operations",
    "enabled": true
  }'

# Write to Node 1
curl -X POST https://node1.ekodb.net/api/insert/orders \
  -H "Authorization: Bearer $TOKEN" \
  -d '{"data": {"product": "Widget", "amount": 100}}'

# Immediately available on Node 2 via ripple
curl -X POST https://node2.ekodb.net/api/find/orders \
  -H "Authorization: Bearer $TOKEN" \
  -d '{"filter": {"type": "Condition", "content": {"field": "product", "operator": "Eq", "value": "Widget"}}}'

4. Data Ingestion Pipeline

Scenario: IoT platform collecting sensor data from thousands of devices

Architecture:

Ingestion nodes accept writes only (send mode)
Storage nodes receive and persist (receive mode, durable WAL)
Processing nodes compute analytics (receive mode)

IoT Devices
   ├────┬────┬────┐
   ↓    ↓    ↓    ↓
┌────┐┌────┐┌────┐┌────┐
│Ing1││Ing2││Ing3││Ing4│  ← Fast WAL (write-optimized)
│send││send││send││send│
└──┬─┘└──┬─┘└──┬─┘└──┬─┘
   │  Ripples    │     │
   └──────┬──────┴─────┘
          │
    ┌─────┼─────┐
    ↓     ↓     ↓
┌────────┐┌────────┐┌────────┐
│Storage1││Storage2││Process │  ← Durable WAL (data safety)
│receive ││receive ││receive │
└────────┘└────────┘└────────┘

Benefits:

Ingestion nodes optimized for write throughput
Storage nodes ensure data durability
Processing nodes compute real-time analytics
Horizontal scaling at each layer

5. Development/Staging Environments

Scenario: Testing environment that mirrors production data

Architecture:

Production primary (send mode)
Staging replica (receive mode)
Staging accepts test writes locally (send mode disabled)

Production                  Staging
┌──────────┐               ┌──────────┐
│ Primary  │──────────────►│ Replica  │
│  (send)  │   Ripples     │(receive) │
└──────────┘               └──────────┘
                                 ↑
                                 │ Test writes
                                 │ (not rippled back)

Benefits:

Staging has real production data
Test writes don't affect production
Safe environment for testing features

Configuring Ripples

Basic Configuration

Configure ripples on a node via the ripples API. Each peer requires a separate configuration:

Client Libraries (Recommended)
Direct API

use ekodb_client::{Client, RippleConfig};

let client = Client::builder()
    .base_url("https://your-ekodb-instance.com")
    .api_key("your-admin-token")
    .build()?;

let config = RippleConfig {
    name: "peer-node-1".to_string(),
    url: "https://peer1.ekodb.net:8080".to_string(),
    api_key: "peer1-admin-key".to_string(),
    mode: "Operations".to_string(),
    enabled: true,
};

client.configure_ripple(config).await?;

import { EkoDBClient } from "@ekodb/ekodb-client";

const client = new EkoDBClient({
  baseURL: "https://your-ekodb-instance.com",
  apiKey: "your-admin-token",
});
await client.init();

await client.configureRipple({
  name: "peer-node-1",
  url: "https://peer1.ekodb.net:8080",
  api_key: "peer1-admin-key",
  mode: "Operations",
  enabled: true,
});

const { EkoDBClient } = require("@ekodb/ekodb-client");

const client = new EkoDBClient({
  baseURL: "https://your-ekodb-instance.com",
  apiKey: "your-admin-token",
});
await client.init();

await client.configureRipple({
  name: "peer-node-1",
  url: "https://peer1.ekodb.net:8080",
  api_key: "peer1-admin-key",
  mode: "Operations",
  enabled: true,
});

from ekodb_client import Client

client = Client.new(
    'https://your-ekodb-instance.com',
    'your-admin-token'
)

client.configure_ripple({
    'name': 'peer-node-1',
    'url': 'https://peer1.ekodb.net:8080',
    'api_key': 'peer1-admin-key',
    'mode': 'Operations',
    'enabled': True
})

import io.ekodb.client.EkoDBClient

val client = EkoDBClient.builder()
    .baseUrl("https://your-ekodb-instance.com")
    .apiKey("your-admin-token")
    .build()

client.configureRipple(mapOf(
    "name" to "peer-node-1",
    "url" to "https://peer1.ekodb.net:8080",
    "api_key" to "peer1-admin-key",
    "mode" to "Operations",
    "enabled" to true
))

import "github.com/ekoDB/ekodb-client-go"

client := ekodb.NewClient(
    "https://your-ekodb-instance.com",
    "your-admin-token",
)

err := client.ConfigureRipple(map[string]interface{}{
    "name":    "peer-node-1",
    "url":     "https://peer1.ekodb.net:8080",
    "api_key": "peer1-admin-key",
    "mode":    "Operations",
    "enabled": true,
})
if err != nil {
    log.Fatal(err)
}

curl -X POST https://{EKODB_API_URL}/api/ripples/config \
  -H "Authorization: Bearer {ADMIN_TOKEN}" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "peer-node-1",
    "url": "https://peer1.ekodb.net:8080",
    "api_key": "peer1-admin-key",
    "mode": "Operations",
    "enabled": true
  }'

# Response
{
  "status": "success",
  "peer_name": "peer-node-1",
  "ripple_enabled": true
}

List Configured Ripples

note

Ripple management is an admin-only operation. Use the Direct API with an admin token.

Direct API

curl https://{EKODB_API_URL}/api/ripples/config \
  -H "Authorization: Bearer {ADMIN_TOKEN}"

# Response (array of RippleConfig objects)
[
  {
    "name": "peer1",
    "url": "https://peer1.ekodb.net:8080",
    "mode": "operations",
    "enabled": true
  },
  {
    "name": "peer2",
    "url": "https://peer2.ekodb.net:8080",
    "mode": "operations",
    "enabled": true
  }
]

Remove Ripple Configuration

Not Yet Implemented

The DELETE endpoint for ripple configurations is not yet available. To disable a ripple, update its configuration with "enabled": false:

curl -X PUT https://{EKODB_API_URL}/api/ripples/config \
  -H "Authorization: Bearer {ADMIN_TOKEN}" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "peer-node-1",
    "enabled": false
  }'

Bypassing Ripples

For bulk imports or maintenance operations, you can bypass ripple propagation:

Single Operations

POST https://{EKODB_API_URL}/api/insert/users?bypass_ripple=true
Authorization: Bearer {TOKEN}
Content-Type: application/json

{
  "data": {
    "name": "John Doe",
    "email": "john@example.com"
  }
}

Batch Operations

POST https://{EKODB_API_URL}/api/batch/insert/users?bypass_ripple=true
Authorization: Bearer {TOKEN}
Content-Type: application/json

{
  "inserts": [
    {"data": {"name": "User 1"}},
    {"data": {"name": "User 2"}},
    {"data": {"name": "User 3"}}
  ]
}

When to Bypass:

Initial data seeding
Large bulk imports
Database migrations
Maintenance operations
Temporary disconnection scenarios

Loop Prevention

ekoDB implements automatic loop prevention to avoid infinite ripple cycles in multi-node deployments. Operations are automatically deduplicated across all nodes, ensuring each operation is processed exactly once regardless of network topology.

Key guarantees:

Operations are never processed more than once per node
No infinite loops in bidirectional or full-mesh deployments
Automatic deduplication with no manual configuration required
Works transparently across all replication patterns

Performance Considerations

Ripple Overhead

Minimal for typical workloads:

Negligible CPU overhead per operation
Network latency depends on topology

Batch operations scale well:

Single ripple for entire batch
Amortized overhead across records

Network Topology

Full Mesh (N nodes):

Each write ripples to (N-1) peers
Best for: Small clusters (2-5 nodes)
Network traffic: O(N²)

Hub-and-Spoke:

Primary sends to all replicas
Replicas don't send to each other
Best for: Read scaling (1 primary, N replicas)
Network traffic: O(N)

Hybrid:

Primaries in full mesh
Replicas receive only
Best for: Multi-region with local replicas
Network traffic: O(P²) + O(R) where P=primaries, R=replicas

Monitoring Ripples

Check Ripple Status

curl -X GET https://{EKODB_API_URL}/api/ripples/config \
  -H "Authorization: Bearer {ADMIN_TOKEN}"

Monitor Replication Lag

# Check WAL health for replication status
curl -X GET https://{EKODB_API_URL}/api/wal/health \
  -H "Authorization: Bearer {ADMIN_TOKEN}" \
  | jq '.last_entry_timestamp'

Health Checks

# Verify all peers are healthy
for peer in peer1 peer2 peer3; do
  echo "Checking $peer..."
  curl -s https://$peer.ekodb.net/api/health | jq '.status'
done

Best Practices

1. Configure Primary Nodes to Send to Replicas

Why: Prevents write conflicts and simplifies conflict resolution

Add each replica as a peer on the primary node:

POST /api/ripples/config
{
  "name": "replica1",
  "url": "https://replica1.ekodb.net:8080",
  "api_key": "replica1-admin-key",
  "mode": "Operations",
  "enabled": true
}

2. Bypass Ripples for Bulk Operations

Why: Reduces network overhead during large imports

# Initial data load
curl -X POST .../api/batch/insert/users?bypass_ripple=true \
  -d '{"inserts": [...]}'

# After import, sync manually via WAL

3. Monitor Ripple Health

Why: Detect failures early, prevent data divergence

# Daily health check
*/5 * * * * /scripts/check_ripple_health.sh

4. Read Replicas Need No Configuration

Why: Prevents accidental writes from replicating back

Read replicas simply receive ripples from primary nodes that have them configured as peers. No peer configuration is needed on the replica itself.

5. Isolate Analytics Nodes

Why: Prevents ripple overhead from impacting analytics queries

Simply don't add any ripple peers to analytics nodes. They will operate independently with no replication overhead.

Troubleshooting

Ripples Not Propagating

Check ripple configuration:

curl -X GET https://{EKODB_API_URL}/api/ripples/config \
  -H "Authorization: Bearer {ADMIN_TOKEN}"

Verify network connectivity:

# Test from source node to peer
curl -k https://peer.ekodb.net:8080/api/health

Check authentication:

# Ensure peer has valid admin token
curl -X POST https://peer.ekodb.net/api/auth/token \
  -d '{"api_key": "your-admin-key"}'

Replication Lag

Check WAL entries:

# Compare timestamps between primary and replica
curl -s https://primary.ekodb.net/api/wal/health | jq '.last_entry_timestamp'
curl -s https://replica.ekodb.net/api/wal/health | jq '.last_entry_timestamp'

Monitor network latency:

ping peer.ekodb.net

Duplicate Operations

Verify loop prevention:

Ensure all nodes are running and healthy
Verify ripple configuration is correct on each node
Check network connectivity between nodes

Security

Authentication

All ripple endpoints require admin authentication:

Authorization: Bearer {ADMIN_TOKEN}

TLS/SSL Required

Ripples always use HTTPS:

✓ https://peer.ekodb.net:8080  (secure)
✗ http://peer.ekodb.net:8080   (rejected)

Network Isolation

Recommended: Use private networks (VPC) for ripple traffic

Production: 10.0.0.0/16 (VPC)
Peer URLs: https://10.0.0.2:8080, https://10.0.0.3:8080

System Administration - WAL management and replication
Batch Operations - Bypass ripples for bulk operations
Basic Operations - Single-record ripple behavior
White Paper - Architecture deep dive

For questions or support with ripple configuration, contact support@ekodb.io

What are Ripples?​

How Ripples Work​

Ripple Propagation Modes​

Quick Mode Selection Guide​

Operations Mode (Default)​

WAL Mode​

Snapshot Mode​

Search Mode​

Analytics Mode​

Stream Mode​

Webhook Mode​

Embedding Mode​

Chat Mode​

Custom Mode​

Replication Patterns (Send/Receive/Both)​

Send Mode​

Receive Mode​

Both Mode (Full Mesh)​

None Mode​

Real-World Use Cases​

1. Geographic Distribution (Multi-Region)​

2. Read Scaling (Analytics Workload)​

3. High Availability (Active-Active)​

4. Data Ingestion Pipeline​

5. Development/Staging Environments​

Configuring Ripples​

Basic Configuration​

List Configured Ripples​

Remove Ripple Configuration​

Bypassing Ripples​

Single Operations​

Batch Operations​

Loop Prevention​

Performance Considerations​

Ripple Overhead​

Network Topology​

Monitoring Ripples​

Check Ripple Status​

Monitor Replication Lag​

Health Checks​

Best Practices​

1. Configure Primary Nodes to Send to Replicas​

2. Bypass Ripples for Bulk Operations​

3. Monitor Ripple Health​

4. Read Replicas Need No Configuration​

5. Isolate Analytics Nodes​

Troubleshooting​

Ripples Not Propagating​

Replication Lag​

Duplicate Operations​

Security​

Authentication​

TLS/SSL Required​

Network Isolation​

Related Documentation​

What are Ripples?

How Ripples Work

Ripple Propagation Modes

Quick Mode Selection Guide

Operations Mode (Default)

WAL Mode

Snapshot Mode

Search Mode

Analytics Mode

Stream Mode

Webhook Mode

Embedding Mode

Chat Mode

Custom Mode

Replication Patterns (Send/Receive/Both)

Send Mode

Receive Mode

Both Mode (Full Mesh)

None Mode

Real-World Use Cases

1. Geographic Distribution (Multi-Region)

2. Read Scaling (Analytics Workload)

3. High Availability (Active-Active)

4. Data Ingestion Pipeline

5. Development/Staging Environments

Configuring Ripples

Basic Configuration

List Configured Ripples

Remove Ripple Configuration

Bypassing Ripples

Single Operations

Batch Operations

Loop Prevention

Performance Considerations

Ripple Overhead

Network Topology

Monitoring Ripples

Check Ripple Status

Monitor Replication Lag

Health Checks

Best Practices

1. Configure Primary Nodes to Send to Replicas

2. Bypass Ripples for Bulk Operations

3. Monitor Ripple Health

4. Read Replicas Need No Configuration

5. Isolate Analytics Nodes

Troubleshooting

Ripples Not Propagating

Replication Lag

Duplicate Operations

Security

Authentication

TLS/SSL Required

Network Isolation

Related Documentation