Skip to main content

Batch Operations

Perform multiple create, update, or delete operations in a single request. Batch operations are optimized for high throughput and handle partial failures gracefully.

:::tip Performance Your deployment tier determines batch size limits (500-20,000 records per request). Batch operations are processed in chunks for optimal performance. :::

First, generate an access token using your API key:

curl -X POST https://{EKODB_API_URL}/api/auth/token \
  -H "Content-Type: application/json" \
  -d '{"api_key": "YOUR_API_KEY"}'

Response:

{
  "token": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9..."
}

Batch Insert​

Insert multiple records in a single request.

let records = vec![
    json!({
        "name": "User 1",
        "email": "user1@example.com"
    }),
    json!({
        "name": "User 2",
        "email": "user2@example.com"
    })
];

let result = client.batch_insert("users", records).await?;

println!("{:?}", result.successful); // ['id1', 'id2']
println!("{:?}", result.failed); // []

Disable Real-Time Sync (Ripples):

Use bypass_ripple=true to skip real-time data propagation for bulk imports. This significantly improves throughput by avoiding network overhead.

When to bypass ripples:

  • Initial data loading (millions of records)
  • Database migrations or imports
  • Maintenance operations
  • Temporary disconnection scenarios
  • Bulk data seeding in development

Performance impact: Bypassing ripples can improve batch throughput by 2-5x for large operations.

POST https://{EKODB_API_URL}/api/batch/insert/{collection}?bypass_ripple=true
Content-Type: application/json
Authorization: Bearer {YOUR_API_TOKEN}

{
  "inserts": [
    {"data": {"name": "Bulk User 1"}},
    {"data": {"name": "Bulk User 2"}}
  ]
}

Or set it in the request body:

{
  "bypass_ripple": true,
  "inserts": [
    {"data": {"name": "User 1"}},
    {"data": {"name": "User 2"}}
  ]
}

:::tip After Bulk Import After completing a large import with bypass_ripple=true, you can synchronize replicas manually using WAL exports or by temporarily reconfiguring ripples. See Ripples - Data Propagation for synchronization strategies. :::

Response on Partial Failure:

{
  "successful": ["id1", "id2"],
  "failed": [
    {
      "id": null,
      "error": "Chunk 3 failed: validation error"
    }
  ]
}

Batch Update​

Update multiple records by ID.

let updates = vec![
    json!({
        "id": "record_id_1",
        "data": {
            "status": "active"
        }
    }),
    json!({
        "id": "record_id_2",
        "data": {
            "status": "active"
        }
    })
];

let result = client.batch_update("users", updates).await?;

println!("{:?}", result.successful); // ['record_id_1', 'record_id_2']
println!("{:?}", result.failed); // []

Disable Real-Time Sync:

PUT https://{EKODB_API_URL}/api/batch/update/{collection}?bypass_ripple=true

Or in request body:

{
  "bypass_ripple": true,
  "updates": [...]
}

Response on Partial Failure:

{
  "successful": ["record_id_1"],
  "failed": [
    {
      "id": "record_id_2",
      "error": "Record not found or update failed"
    }
  ]
}

Batch Delete​

Delete multiple records by ID.

let ids = vec!["record_id_1", "record_id_2", "record_id_3"];

let result = client.batch_delete("users", ids).await?;

println!("{:?}", result.successful); // ['record_id_1', 'record_id_2', 'record_id_3']

Disable Real-Time Sync:

DELETE https://{EKODB_API_URL}/api/batch/delete/{collection}?bypass_ripple=true

Or in request body:

{
  "bypass_ripple": true,
  "deletes": [...]
}

Response on Partial Failure:

{
  "successful": ["record_id_1", "record_id_2"],
  "failed": [
    {
      "id": "record_id_3",
      "error": "Record not found or delete failed"
    }
  ]
}

Using Batch Operations Within Transactions​

All batch operations can be performed within a transaction by adding the transaction_id query parameter. This ensures atomicity across all batch operations.

Batch Insert in Transaction:

POST https://{EKODB_API_URL}/api/batch/insert/users?transaction_id=tx-001
Content-Type: application/json
Authorization: Bearer {YOUR_API_TOKEN}

{
  "inserts": [
    {"data": {"name": "User 1", "email": "user1@example.com"}},
    {"data": {"name": "User 2", "email": "user2@example.com"}}
  ]
}

Batch Update in Transaction:

PUT https://{EKODB_API_URL}/api/batch/update/users?transaction_id=tx-001
Content-Type: application/json
Authorization: Bearer {YOUR_API_TOKEN}

{
  "updates": [
    {"id": "user_1", "data": {"status": "active"}},
    {"id": "user_2", "data": {"status": "active"}}
  ]
}

Batch Delete in Transaction:

DELETE https://{EKODB_API_URL}/api/batch/delete/users?transaction_id=tx-001
Content-Type: application/json
Authorization: Bearer {YOUR_API_TOKEN}

{
  "deletes": [
    {"id": "user_1"},
    {"id": "user_2"}
  ]
}

Complete Transaction Example:

# Begin transaction
tx_id=$(curl -s -X POST https://{EKODB_API_URL}/api/transactions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer {YOUR_API_TOKEN}" \
  -d '{"isolation_level": "Serializable"}' | jq -r '.transaction_id')

# Batch insert in transaction
curl -X POST https://{EKODB_API_URL}/api/batch/insert/users?transaction_id=$tx_id \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer {YOUR_API_TOKEN}" \
  -d '{
    "inserts": [
      {"data": {"name": "User 1", "email": "user1@example.com"}},
      {"data": {"name": "User 2", "email": "user2@example.com"}}
    ]
  }'

# Batch update in transaction
curl -X PUT https://{EKODB_API_URL}/api/batch/update/users?transaction_id=$tx_id \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer {YOUR_API_TOKEN}" \
  -d '{
    "updates": [
      {"id": "user_1", "data": {"verified": true}},
      {"id": "user_2", "data": {"verified": true}}
    ]
  }'

# Commit or rollback based on results
if [ $? -eq 0 ]; then
  curl -X POST https://{EKODB_API_URL}/api/transactions/$tx_id/commit \
    -H "Authorization: Bearer {YOUR_API_TOKEN}"
else
  curl -X POST https://{EKODB_API_URL}/api/transactions/$tx_id/rollback \
    -H "Authorization: Bearer {YOUR_API_TOKEN}"
fi

:::tip Atomic Batch Operations Using transactions with batch operations ensures that either all operations succeed or all are rolled back together. See Transactions for full transaction management. :::

Response Status Codes​

Batch operations return different status codes based on the result:

  • 201 Created - All inserts successful
  • 200 OK - All updates/deletes successful
  • 207 Multi-Status - Partial success (some succeeded, some failed)
  • 400 Bad Request - All operations failed
  • 503 Service Unavailable - Server is processing too many concurrent batch operations

Error Handling​

Batch operations continue processing even if individual records fail:

# Example: Batch insert with validation error
curl -X POST https://{EKODB_API_URL}/api/batch/insert/users \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer {YOUR_API_TOKEN}" \
  -d '{
    "inserts": [
      {"data": {"name": "Valid User", "email": "valid@example.com"}},
      {"data": {"name": ""}},
      {"data": {"name": "Another Valid", "email": "valid2@example.com"}}
    ]
  }'

# Response (207 Multi-Status):
{
  "successful": ["id1", "id3"],
  "failed": [
    {
      "id": null,
      "error": "Validation error: name cannot be empty"
    }
  ]
}

Best Practices​

Choose Appropriate Batch Size​

# Small batch (< 100 records) - Fast response
POST /api/batch/insert/users
{"inserts": [...]}  # 50 records

# Medium batch (100-1,000 records) - Balanced
POST /api/batch/insert/users
{"inserts": [...]}  # 500 records

# Large batch (1,000-10,000 records) - Maximum throughput
POST /api/batch/insert/users
{"inserts": [...]}  # 5,000 records

Handle Partial Failures​

Always check both successful and failed arrays:

# Check response
response=$(curl -X POST .../api/batch/insert/users ...)

# Parse response
successful_count=$(echo $response | jq '.successful | length')
failed_count=$(echo $response | jq '.failed | length')

if [ $failed_count -gt 0 ]; then
  echo "Warning: $failed_count operations failed"
  # Log failures or retry
fi

Monitor Performance​

# Start with smaller batches to test performance
curl -X POST .../api/batch/insert/users -d '{"inserts": [...]}'  # 100 records

# Monitor response time and adjust batch size
# - Fast response (< 1s): Can increase batch size
# - Slow response (> 5s): Reduce batch size
# - 503 errors: Server overloaded, wait and retry with smaller batches

# Use appropriate timeouts for large batches
curl --max-time 60 -X POST .../api/batch/insert/users ...

# Handle rate limiting gracefully
if [ $http_code -eq 503 ]; then
  echo "Server busy, waiting 5 seconds..."
  sleep 5
  # Retry with smaller batch
fi

Use Transactions for Atomicity​

For operations that must succeed or fail together:

# Begin transaction
tx_id=$(curl -X POST .../api/transactions | jq -r '.transaction_id')

# Perform batch operations
curl -X POST .../api/batch/insert/users?transaction_id=$tx_id ...

# Commit or rollback based on results
if [ $success ]; then
  curl -X POST .../api/transactions/$tx_id/commit
else
  curl -X POST .../api/transactions/$tx_id/rollback
fi

Complete Example​

Here's a complete workflow for batch inserting users with error handling:

#!/bin/bash

# Batch insert 1000 users
response=$(curl -s -X POST https://{EKODB_API_URL}/api/batch/insert/users \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer {YOUR_API_TOKEN}" \
  -d @- << 'EOF'
{
  "inserts": [
    {"data": {"name": "User 1", "email": "user1@example.com"}},
    {"data": {"name": "User 2", "email": "user2@example.com"}},
    {"data": {"name": "User 3", "email": "user3@example.com"}}
  ]
}
EOF
)

# Parse response
successful=$(echo $response | jq -r '.successful[]')
failed=$(echo $response | jq -r '.failed[]')

# Log results
echo "Successfully inserted: $(echo $response | jq '.successful | length') records"
echo "Failed: $(echo $response | jq '.failed | length') records"

# Handle failures
if [ "$(echo $response | jq '.failed | length')" -gt 0 ]; then
  echo "Failed records:"
  echo $response | jq '.failed'
  
  # Optionally retry failed records
  # ... retry logic here ...
fi

Limits​

TierMax Batch SizeMax Concurrent Operations
Free5005
Starter2,00010
Pro10,00050
Enterprise20,000100

:::tip Concurrency If you receive a 503 Service Unavailable response, the server is processing too many concurrent batch operations. Wait and retry after a few seconds. :::

Example Code​

Direct HTTP/REST API Examples​

Raw HTTP examples demonstrating the REST API directly:

  • JavaScript: batch_operations.js
  • Python: batch_operations.py
  • Go: batch_operations.go
  • Rust: batch_operations.rs

Client Library Examples​

Production-ready examples using official client libraries:

  • Rust: client_batch_operations.rs
  • Python: client_batch_operations.py
  • TypeScript: client_batch_operations.ts
  • Go: client_batch_operations.go
  • Kotlin: ClientBatchOperations.kt
  • JavaScript: client_batch_operations.js