Skip to content

xcon-db Timeseries

xcon-db Timeseries is a lean HTAP time-series database engine for IoT. It combines a synchronous, flexible-schema write path with compressed columnar reads in a single process, on modest hardware — no cluster machinery, no reserved RAM, no external sync pipelines.

What It Does

Devices write sparse, fragmented telemetry; dashboards and analytics read dense time ranges. xcon-db serves both from one engine:

CREATE TABLE telemetry (
  speed DOUBLE FILL HOLD STALENESS 60 s,
  event TEXT   FILL NONE
) WITH (RETENTION 90 d);

INSERT INTO telemetry (entity, ts, speed) VALUES ('veh-1', 1700000000000, 52.5);

SELECT * FROM telemetry LATEST ON ts PARTITION BY entity;   -- device shadow
SELECT avg(speed) FROM telemetry
  WHERE entity = 'veh-1' AND ts BETWEEN 1700000000000 AND 1700086400000
  SAMPLE BY 5 m FILL(PREV);                                 -- downsample

Every language's PostgreSQL driver, Grafana's pg datasource and any ILP-speaking collector work out of the box — xcon-db speaks standard protocols instead of inventing its own.

Design Principles

  • The engine knows shape, not meaning — channels are nameless streams to the engine; semantics live in your catalog and code
  • Frozen type system: f64 + string — the core never grows; flexibility lives in code
  • Store the sparse truth, fill at read time — a filled value is never written to disk
  • Writes are synchronous and durable — fsync'ed (group commit) before the ack
  • Reads always combine the delta and the columnar — correctness, not an optimization
  • db-per-firm — one directory per tenant; move, back up or delete a firm by moving a directory

Architecture

 ingest (pg-wire / ILP / HTTP)
        │
        ▼
 delta log (WAL, group-commit fsync)  ──►  merge worker (throttled)
        │                                        │
        ▼                                        ▼
    memtable            +            columnar segments (mmap, per channel)
        └───────────── read path ────────────────┘
                (always both sides)
Layer Role
Delta log Append-only WAL; the write buffer is the log
Memtable In-memory image of the unmerged delta, split per channel
Merge worker Background delta→columnar folding, size+time triggered, throttled
Segments Immutable, compressed, per-channel columnar files with sparse indexes
SQL layer Time-series SQL subset over pg-wire and HTTP

Status

All engine phases are implemented and tested (delta log, columnar core, merge worker, read path, catalog, SCRAM auth, TS-SQL, pg-wire, ILP, HTTP, TLS). Known limitations:

  • No bind parameters over pg-wire (inline literals; use simple query mode)
  • No pg_catalog emulation — IDE schema browsers stay empty; SHOW TABLES works
  • One process per database directory (no lock file yet)
  • ILP carries no credentials: bind an ILP listener to one database and firewall it