Is there options to embed data node and DB e.g. provided by SingleStore (VAR), ExtremeDB, and Informix (embedded) not mentioning IMDG like Hazelcast and Apache Ignite.
The 3 DBs above are used to build trading exchanges (crypto/futures/stock).
Motivation: Minimise overhead of IPC (lower latency) but potentially sacrifice reliability.
It is possible to run a data node and an API node on the same machine.
They can communicate over a shared memory transporter.
This provides a very low latency operation.
So by having 2 nodes in the cluster and using READ COMMITTED
and having your application colocated with data nodes using shared
memory transporter all reads are performed in the local machine
using memory communication. Thus no network protocols are involved
in the reads. The setup is still highly available. The write transactions
in this setup will still use networks to perform updates.
A setup with a single data node colocated with the API node using
shared memory isn’t highly available, but has low latency also for
updates.
Finally to achieve the very lowest latency one can increase the
CPU spinning configuration to DatabaseMachine level. With this
almost all communication internally to achieve the reads and writes
will be using direct memory communications where each thread
is immediately ready to serve read and write requests.