Introducing JDBC Support for XTDB

XTDB is built for providing queryability at scale and for that reason we specifically designed XTDB around Apache Kafka as the immutable event log, to serve as the foundational piece in our unbundled architecture.

Until now Kafka has acted as the golden store for transactions and documents, and provided coordination between nodes in the form of serialized transaction ordering. XTDB nodes rely on these properties to deterministically ingest the stream of events and maintain local bitemporal indexes in a local Key/Value stores, such as RocksDB or LMDB.

One potential drawback of Kafka is that if you don’t already have an existing Kafka cluster established then it requires additional investment to get one running and it carries implications for future operational overhead that may not be immediately justifiable. We discussed the ease and benefits of using a managed Kafka service in our site:/blog/xtdb-confluent-cloud/[previous blog post], but procuring services from an external provider is not always an option.

On the other hand, use of Relational Data Stores is prevalent throughout most companies and these more traditional "databases" are often readily available as a service, either on-prem or in the cloud, such as Amazon RDS. If your data throughput requirements are not massive then using a typical RDS database setup with XTDB, in place of Kafka, is a reasonable choice.

We have just released xtdb-jdbc, which allows you to use XTDB nodes in a cluster without needing Kafka. Transaction and document writes are made directly into an RDS of your choosing via JDBC. This happens transparently from the regular XTDB API. XTDB nodes will then ingest directly from the JDBC tables into their local K/V stores.

We have provided initial support for:

The Kafka-like semantics of eviction and compaction of documents is preserved.

To get started, simply add the com.xtdb/xtdb-jdbc dependency to your project and fire up an XTDB node that is configured to point to an existing database. Alternatively you can run an embedded SQLite or H2 database in a standalone node.

Follow the docs to configure your XTDB node to use JDBC.

Enjoy! We’re available on the #xtdb channel on the Clojurians slack to respond to any queries / issues.