Event Store Selection and Implementation

·16 min read·Architecture and Designintermediate

Why choosing and building with an event store matters in modern, distributed systems

A server rack with multiple nodes representing an event store cluster, symbolizing event sourcing and distributed log storage

When you are designing a system that needs to scale, stay observable, and evolve without breaking existing behavior, the way you store state changes becomes a pivotal architectural decision. Over the past few years, I’ve worked on several projects where the database choice determined how easily we could add new features, debug production issues, and recover from failures. Event stores sit at the heart of event-driven architectures and event sourcing. They provide a reliable log of “what happened,” enabling systems to rebuild state, power projections, and feed analytics. However, the landscape of options is wide: dedicated event stores like EventStoreDB, Kafka-based logs, PostgreSQL’s logical replication, and even cloud-native services each have tradeoffs. Choosing one isn’t just about performance charts; it’s about the operational model you can sustain and the developer experience your team can handle.

This article walks through how to select an event store and how to implement a minimal but production-ready event store in a real-world context. I’ll share practical patterns, configuration pitfalls, and code you can actually run. The goal is to help you reason about consistency, scalability, and maintainability while grounding decisions in examples that mirror real projects. If you’ve ever been unsure whether to build on a dedicated store, a log stream, or a relational database, this should give you a clear framework and some code you can adapt.

Context and fit: where event stores sit in today’s architecture

Event sourcing is no longer a niche pattern. It shows up in fintech platforms for audit trails, in SaaS products for building flexible read models, and in IoT backends where device telemetry needs reliable replay. The central idea is simple: instead of mutating state, you record immutable events that describe state changes. The current state is derived by applying events in order. This gives you natural auditability, the ability to time-travel models, and the option to create new projections without rewriting past data.

In practice, teams use event stores in a few common scenarios:

  • Compliance and audit: A bank records every transaction and interest calculation as events, allowing reconstructable histories.
  • Multi-tenant SaaS: You derive tenant-specific read models from shared event streams, adding new features without backfilling old tables.
  • IoT and analytics: Sensor readings are appended as events and fanned out to stream processors for real-time aggregates.

Compared to traditional CRUD databases, event stores emphasize immutability, ordering, and predictable append behavior. Compared to general message brokers like Kafka, dedicated event stores often provide stronger guarantees around reading a stream from a specific version, partial stream reads, and built-in mechanisms for optimistic concurrency control. A relational database like PostgreSQL can serve as an event store too, especially when you already have operational expertise; however, you will need to handle ordering, replication, and storage growth carefully.

A few anchors in the ecosystem:

  • EventStoreDB offers a purpose-built event store with strong semantics for stream reads and subscriptions.
  • Apache Kafka provides a durable log that scales well for streaming but requires additional tooling for “replaying from a specific event” and managing stream version semantics.
  • PostgreSQL’s logical replication and append-only tables can serve as a pragmatic foundation for smaller teams.

For further context on why event sourcing is gaining traction, see Martin Fowler’s overview at https://martinfowler.com/eaaDev/EventSourcing.html.

Core concepts and practical implementation

An event store stores events (facts) in streams. A stream is typically a logical grouping keyed by an entity identifier, such as a user or order. Each event has a type, payload, and metadata like timestamp and correlation ID. A critical concept is the expected stream version, used for optimistic concurrency control: when writing an event, you assert the current version to prevent lost updates.

Reads usually fall into two categories:

  • Stream reads: Load all events for a given aggregate root (e.g., order) to rebuild state.
  • Catch-up subscriptions: Start from a position and continuously receive new events, which is useful for projections and integrations.

Minimal event store implementation in PostgreSQL

Below is a pragmatic event store implementation in Node.js with PostgreSQL. This mirrors a small but real system I built for a multi-tenant service needing audit and flexible projections. We keep events in an append-only table, use a sequence for ordering, and rely on transactional writes for consistency.

-- schema.sql
-- We use an append-only events table. The stream_version ensures optimistic concurrency.
CREATE TABLE events (
  id BIGSERIAL PRIMARY KEY,
  stream_id TEXT NOT NULL,
  stream_version INT NOT NULL,
  event_type TEXT NOT NULL,
  payload JSONB NOT NULL,
  metadata JSONB DEFAULT '{}'::jsonb,
  created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);

-- For fast stream reads and ordering.
CREATE INDEX idx_events_stream ON events (stream_id, stream_version);

In the application, writing events uses a stored procedure or a transaction with a check for the expected version. The function below simulates the append logic with concurrency control.

-- append_event.sql
-- Append an event to a stream with optimistic concurrency control.
-- Returns the new stream version on success, or raises an exception on conflict.
CREATE OR REPLACE FUNCTION append_event(
  p_stream_id TEXT,
  p_expected_version INT,
  p_event_type TEXT,
  p_payload JSONB,
  p_metadata JSONB DEFAULT '{}'::jsonb
) RETURNS INT AS $$
DECLARE
  next_version INT;
BEGIN
  -- Calculate the next version based on the current max version for the stream.
  SELECT COALESCE(MAX(stream_version), 0) INTO next_version
  FROM events
  WHERE stream_id = p_stream_id;

  -- Check the expected version.
  IF next_version != p_expected_version THEN
    RAISE EXCEPTION 'Concurrency conflict: expected version % but found %', p_expected_version, next_version;
  END IF;

  -- Append the event.
  INSERT INTO events (stream_id, stream_version, event_type, payload, metadata)
  VALUES (p_stream_id, next_version + 1, p_event_type, p_payload, p_metadata);

  RETURN next_version + 1;
END;
$$ LANGUAGE plpgsql;

For reads, we provide two common operations: read all events for a stream and read from a given position for a catch-up subscription.

-- read_stream.sql
-- Read all events for a stream in order.
SELECT
  id,
  stream_id,
  stream_version,
  event_type,
  payload,
  metadata,
  created_at
FROM events
WHERE stream_id = $1
ORDER BY stream_version ASC;

-- read_from_position.sql
-- Read events from a global position (useful for catch-up subscriptions).
SELECT
  id,
  stream_id,
  stream_version,
  event_type,
  payload,
  metadata,
  created_at
FROM events
WHERE id > $1
ORDER BY id ASC
LIMIT $2;

Now, a Node.js service wrapper around these operations. This example uses pg for database access and zod for runtime validation of event payloads.

// src/db.js
// Database connection pool. Use environment variables for host, user, password.
const { Pool } = require('pg');

const pool = new Pool({
  host: process.env.DB_HOST || 'localhost',
  port: parseInt(process.env.DB_PORT || '5432', 10),
  database: process.env.DB_NAME || 'eventstore',
  user: process.env.DB_USER || 'postgres',
  password: process.env.DB_PASSWORD || 'postgres',
  max: 20,
  idleTimeoutMillis: 30000,
  connectionTimeoutMillis: 2000,
});

module.exports = { pool };

// src/eventStore.js
// A minimal, production-inspired event store.
const { z } = require('zod');
const { pool } = require('./db');

const EventSchema = z.object({
  type: z.string().min(1),
  data: z.record(z.any()),
  metadata: z.record(z.any()).optional(),
});

// Write an event to a stream with expected version for optimistic concurrency.
async function appendToStream(streamId, expectedVersion, event) {
  const parsed = EventSchema.safeParse(event);
  if (!parsed.success) {
    throw new Error(`Invalid event: ${JSON.stringify(parsed.error.errors)}`);
  }

  const client = await pool.connect();
  try {
    await client.query('BEGIN');

    const res = await client.query(
      `SELECT COALESCE(MAX(stream_version), 0) as current FROM events WHERE stream_id = $1`,
      [streamId]
    );
    const current = parseInt(res.rows[0].current, 10);

    if (current !== expectedVersion) {
      throw new Error(
        `Concurrency conflict: expected version ${expectedVersion} but found ${current}`
      );
    }

    const insertRes = await client.query(
      `INSERT INTO events (stream_id, stream_version, event_type, payload, metadata)
       VALUES ($1, $2, $3, $4, $5)
       RETURNING id, stream_version`,
      [
        streamId,
        current + 1,
        parsed.data.type,
        JSON.stringify(parsed.data.data),
        JSON.stringify(parsed.data.metadata || {}),
      ]
    );

    await client.query('COMMIT');
    return {
      id: parseInt(insertRes.rows[0].id, 10),
      streamVersion: parseInt(insertRes.rows[0].stream_version, 10),
    };
  } catch (err) {
    await client.query('ROLLBACK');
    throw err;
  } finally {
    client.release();
  }
}

// Read all events for a stream and reduce to state.
async function readStream(streamId) {
  const res = await pool.query(
    `SELECT id, stream_id, stream_version, event_type, payload, metadata, created_at
     FROM events
     WHERE stream_id = $1
     ORDER BY stream_version ASC`,
    [streamId]
  );

  return res.rows.map(row => ({
    id: row.id,
    streamId: row.stream_id,
    version: row.stream_version,
    type: row.event_type,
    data: row.payload,
    metadata: row.metadata,
    createdAt: row.created_at,
  }));
}

// Catch-up subscription: start from a position and poll for new events.
// In a real system, use LISTEN/NOTIFY or a change feed for efficiency.
async function catchUp(fromId, batchSize = 100, onBatch) {
  let lastId = fromId;
  while (true) {
    const res = await pool.query(
      `SELECT id, stream_id, stream_version, event_type, payload, metadata, created_at
       FROM events
       WHERE id > $1
       ORDER BY id ASC
       LIMIT $2`,
      [lastId, batchSize]
    );

    if (res.rows.length === 0) {
      // Wait a bit before polling again.
      await new Promise(r => setTimeout(r, 500));
      continue;
    }

    const events = res.rows.map(row => ({
      id: row.id,
      streamId: row.stream_id,
      version: row.stream_version,
      type: row.event_type,
      data: row.payload,
      metadata: row.metadata,
      createdAt: row.created_at,
    }));

    await onBatch(events);
    lastId = events[events.length - 1].id;
  }
}

module.exports = { appendToStream, readStream, catchUp };

Rebuilding state: a simple aggregate

Event sourcing shines when you rebuild state from events. Here’s a small example for an order aggregate.

// src/orderAggregate.js
// Define domain events and logic to rebuild order state.
const OrderStatus = {
  Created: 'Created',
  Confirmed: 'Confirmed',
  Shipped: 'Shipped',
  Cancelled: 'Cancelled',
};

function applyEvent(state, event) {
  switch (event.type) {
    case 'OrderCreated': {
      return {
        id: event.data.orderId,
        status: OrderStatus.Created,
        items: event.data.items,
        total: event.data.total,
        version: event.version,
      };
    }
    case 'OrderConfirmed': {
      if (!state) throw new Error('Cannot confirm a non-existing order');
      return { ...state, status: OrderStatus.Confirmed, version: event.version };
    }
    case 'OrderShipped': {
      if (!state) throw new Error('Cannot ship a non-existing order');
      return { ...state, status: OrderStatus.Shipped, version: event.version };
    }
    case 'OrderCancelled': {
      if (!state) throw new Error('Cannot cancel a non-existing order');
      return { ...state, status: OrderStatus.Cancelled, version: event.version };
    }
    default:
      // Unknown event types are ignored for forward compatibility.
      return state;
  }
}

function rebuildOrder(events) {
  return events.reduce(applyEvent, null);
}

module.exports = { rebuildOrder, applyEvent };

Example usage workflow

A typical workflow in services: write an event, rebuild state to validate business rules, and project to a read model. Below is a minimal orchestration that writes and reads, demonstrating optimistic concurrency.

// src/demo.js
const { appendToStream, readStream } = require('./eventStore');
const { rebuildOrder } = require('./orderAggregate');

async function demo() {
  const streamId = 'order-123';

  // Create the order.
  const first = await appendToStream(streamId, 0, {
    type: 'OrderCreated',
    data: {
      orderId: 'order-123',
      items: [{ sku: 'book-1', qty: 1, price: 20 }],
      total: 20,
    },
    metadata: { correlationId: 'cor-42', user: 'alice' },
  });
  console.log('Created event inserted:', first);

  // Confirm the order.
  const second = await appendToStream(streamId, first.streamVersion, {
    type: 'OrderConfirmed',
    data: {},
    metadata: { correlationId: 'cor-42' },
  });
  console.log('Confirmed event inserted:', second);

  // Rebuild state.
  const events = await readStream(streamId);
  const order = rebuildOrder(events);
  console.log('Rebuilt order state:', order);

  // Attempt an out-of-order write (should fail).
  try {
    await appendToStream(streamId, 0, {
      type: 'OrderShipped',
      data: {},
    });
  } catch (err) {
    console.log('Concurrency conflict caught as expected:', err.message);
  }
}

demo().catch(console.error);

When to use a dedicated event store vs Kafka vs PostgreSQL

  • Dedicated event store (EventStoreDB): Best when you need strong stream semantics, built-in subscriptions, and efficient stream reads. Ideal for systems where replaying streams frequently and managing stream versions is core to the domain.
  • Kafka: Great for high-throughput streaming and multi-consumer ecosystems. However, building “read stream from version X” and managing idempotent consumers requires additional orchestration and storage for offsets.
  • PostgreSQL: Suitable when you want to keep operational complexity low, especially if your team is already experienced with Postgres. You’ll handle ordering via sequences and concurrency via transactions, and you’ll need strategies for compaction and retention.

A useful article that contrasts these approaches is “Event Store vs Kafka” by Event Store, available at https://eventstore.com/blog/event-store-vs-kafka/.

Honest evaluation: strengths, weaknesses, and tradeoffs

Event sourcing and event stores bring unique benefits, but they are not a silver bullet. Here’s a grounded view:

Strengths:

  • Auditability and time-travel: You can reconstruct state at any point in time.
  • Evolvable read models: New projections can be built without altering past data.
  • Resilience: Append-heavy workloads are simpler to operate than complex mutation patterns.
  • Scalability: Streams can be sharded, and consumers can catch up independently.

Weaknesses:

  • Complexity: Designing event schemas and handling versioning is non-trivial.
  • Storage growth: Append-only can lead to large volumes; compaction or snapshotting may be required.
  • Operational overhead: Subscriptions, projections, and retention policies need careful planning.
  • Learning curve: Teams must adopt patterns like aggregates, domain events, and eventual consistency.

Where it’s a good fit:

  • Systems requiring strong audit trails and compliance.
  • Domains with changing reporting or analytics needs.
  • Distributed systems where multiple consumers need consistent views of the same data.

Where it might be overkill:

  • Simple CRUD applications with few integrations and limited audit requirements.
  • Teams without capacity to manage event schema evolution and projections.
  • Projects where immediate strong consistency across multiple streams is critical and cannot be modeled effectively.

Personal experience: lessons from real projects

I’ve used event sourcing for a multi-tenant SaaS platform and a small IoT ingestion pipeline. In the SaaS project, the biggest win was being able to introduce a new billing model without touching historical data. We added a projection that recalculated invoices from past events, validated it, and switched traffic gradually. The “aha” moment came when a customer dispute was resolved by replaying events for their tenant and proving exactly what happened, which would have been difficult with a standard relational audit log.

Common mistakes I’ve seen:

  • Under-specifying event schemas: Events like “OrderUpdated” that carry a partial patch are hard to reason about. Prefer events that describe what happened, such as “OrderItemAdded.”
  • Not planning for schema evolution: Adding fields without backward compatibility can break consumers. Use versioned event types and explicit migration policies.
  • Overusing global sequence numbers: They’re useful for catch-up subscriptions but add coordination overhead. Stream-level versions are usually enough for most business logic.
  • Ignoring idempotency: Consumers must handle duplicate deliveries gracefully, especially when using Kafka or polling-based subscriptions.

Moments where it proved valuable:

  • Incident analysis: Replaying events helped pinpoint exactly where a financial discrepancy originated.
  • Feature velocity: Adding a new analytics dashboard by projecting a new read model from existing events took days instead of weeks.
  • Regulatory audits: Immutable event logs provided a natural compliance trail.

Getting started: setup, tooling, and mental model

A realistic project structure keeps concerns separate: domain events, application commands, event store interface, and projections. Here’s a line-based folder structure that works well:

src/
  domain/
    events.js         # Event definitions and serialization
    aggregates/       # Aggregate logic (e.g., orderAggregate.js)
  store/
    eventStore.js     # Interface for append/read/subscribe
    projections.js    # Read model builders
  handlers/
    commandHandlers.js # Command handlers invoking the store
  api/
    routes.js         # Express or Fastify routes
  migrations/         # SQL files to evolve schema
config/
  database.js         # Pool configuration
scripts/
  seed.js             # Seed initial events for testing
tests/
  integration/        # End-to-end flows with DB container
  unit/               # Aggregate and projection logic

Mental model:

  • Commands produce events. Commands are requests to perform business operations. They should never mutate state directly; they append events.
  • Events are facts. They must be immutable and descriptive.
  • Aggregates are rebuilt by folding events. They enforce invariants at write time by validating against current state.
  • Projections are read models. They subscribe to events and update materialized views. Keep them separate from the write model.

For development, consider:

  • Using Docker Compose to spin up PostgreSQL. This keeps your local environment consistent.
  • Managing migrations with a simple tool like node-pg-migrate or raw SQL scripts in migrations/.
  • Setting up basic logging and correlation IDs in metadata to trace flows across services.
  • Adding unit tests for aggregates and projections, and integration tests that verify concurrency conflicts and replay behavior.

Sample Docker Compose for local development:

# docker-compose.yml
version: '3.8'
services:
  db:
    image: postgres:15-alpine
    environment:
      POSTGRES_DB: eventstore
      POSTGRES_USER: postgres
      POSTGRES_PASSWORD: postgres
    ports:
      - '5432:5432'
    volumes:
      - pgdata:/var/lib/postgresql/data

volumes:
  pgdata:

What makes event sourcing stand out: developer experience and maintainability

Event sourcing emphasizes transparency. Because the event log is the source of truth, you can inspect the exact sequence of changes that led to a state. This pays dividends in debugging and onboarding. Developer experience is improved when events are well-named and structured; the code reads like a domain narrative. On the maintenance side, adding new features often means building a new projection or handler, which reduces the risk of touching existing behavior.

That said, the maintainability payoff depends on disciplined event design and tooling. If your team can invest in schema versioning, migration scripts, and robust testing, event sourcing is extremely maintainable. If not, it becomes a liability. The pragmatic path is to start small: store events in PostgreSQL, implement one or two projections, and evolve toward a dedicated event store as complexity grows.

Free learning resources and references

Summary: who should use an event store, and who might skip it

Event stores are a strong fit for teams building systems where auditability, evolvability, and reliable replay are core requirements. If you are in fintech, compliance-heavy SaaS, or distributed IoT backends, investing in event sourcing will likely pay off. Start with a relational implementation, focus on clear event names and schemas, and introduce projections gradually. When throughput and stream semantics become central, consider a dedicated store like EventStoreDB or a Kafka-based architecture with careful offset management.

You might skip event sourcing if your application is primarily CRUD with few integrations, your audit needs are light, or your team is already resource-constrained and lacks the bandwidth to manage event schema evolution. In those cases, a well-designed relational model with audit tables might be sufficient.

The ultimate takeaway: choose an event store when the benefits of immutable facts and replayable streams align with your product and operational goals. Implement the simplest version that solves your immediate problems, measure your real workload, and evolve the architecture as your needs grow. The code and patterns above are a practical starting point for that journey.

Related Articles