Unboxing Stelo V6.1: MERGE Support

Stelo Data Replicator has a well-deserved reputation for its robust, real-time data replication capabilities, and its latest version, V6.1, continues this trend. Packed with additional features, V6.1 is poised to support evolving data management strategies well into the future. With over 30 years of experience, Stelo has honed best practices for information movement, which guide us in every version update. In this blog we focus on the MERGE SQL support. This functionality allows compressed change data to be delivered using micro-batches, reducing latency and leveraging Relational Database Management System (RDBMS) optimisations like column-store indexing.

MERGE, introduced in the 2003 SQL standard, is now gaining traction in modern data management deployments. But how does MERGE operate, and which databases does it serve best?

MERGE shifts the burden of logic from the client to the server. It’s a compound data manipulation language (DML) statement designed for compound operations based on predicates—expressions that insert, update, and delete. Traditionally, deletions, updates, and insertions are handled individually. However, with MERGE, actions are determined by the database’s state at any given moment. This allows for selective decision-making; for instance, if an attempted insertion matches an existing record, MERGE will convert it into an update. This intelligence enables more nuanced responses to changing conditions.

By offloading logic to the server, MERGE simplifies programming and significantly reduces latency compared to client-side decision-making. Also, MERGE encourages the use of micro-batches, which are much faster to transfer than individual requests. Deciding how long to accumulate changes for an efficient micro-batch depends on application-based replication requirements. At Stelo, timers typically range from thirty seconds to three minutes, but clients can customise these intervals according to their needs. Extremely low-latency applications benefit most from DML support, as longer delays with micro-batches negate MERGE’s efficiency benefits. Databases like Snowflake and PostgreSQL, designed for large-scale data repositories rather than high-speed updates, are ideal for MERGE operations. Stelo’s micro-batch compression innovations further enhance efficiency, particularly over wide area networks (WAN). MERGE combined with column-store indexing, streamlines data operations and retrieval in modern databases, making V6.1 an indispensable tool for data management.

Stelo V6.1: MERGE Support

Next Post