Skip to content

Latest commit

 

History

History
143 lines (110 loc) · 5.87 KB

replication-couchdb.md

File metadata and controls

143 lines (110 loc) · 5.87 KB

CouchDB Replication

With the CouchDB replication you can replicate data in both directions with any CouchDB- or PouchDB compliant endpoint. It works with remote endpoints and also with local pouchdb instances or RxCollection that is created with the pouchdb storage.

Add the CouchDB replication plugin

To enable the CouchDB replication, you have to add the replication-couchdb plugin.

import { addRxPlugin } from 'rxdb';
import { RxDBReplicationCouchDBPlugin } from 'rxdb/plugins/replication-couchdb';
addRxPlugin(RxDBReplicationCouchDBPlugin);

RxCollection.syncCouchDB()

To replicate the collection with another instance, use RxCollection.syncCouchDB(). It basically does the same as pouchdb-sync but also adds event-handlers to make sure that change-events will be recognized in the internal event-stream.

// you need these plugins to sync
addPouchPlugin(require('pouchdb-adapter-http')); // enable syncing over http (remote database)

const replicationState = myCollection.syncCouchDB({
    remote: 'http://localhost:10102/db/', // remote database. This can be the serverURL, another RxCollection or a PouchDB-instance
    waitForLeadership: true,              // (optional) [default=true] to save performance, the sync starts on leader-instance only
    direction: {                          // direction (optional) to specify sync-directions
        pull: true, // default=true
        push: true  // default=true
    },
    options: {                             // sync-options (optional) from https://pouchdb.com/api.html#replication
        live: true,
        retry: true
    },
    query: myCollection.find().where('age').gt(18) // query (optional) only documents that match that query will be synchronised
});

Limitations

Since CouchDB only allows synchronization through HTTP1.1 long polling requests there is a limitation of 6 active synchronization connections before the browser prevents sending any further request. This limitation is at the level of browser per tab per domain (some browser, especially older ones, might have a different limit, see here).

Since this limitation is at the browser level there are several solutions:

  1. Use a proxy (ex: HAProxy) between the browser and CouchDB and configure it to use HTTP2.0, since HTTP2.0 doesn't have this limitation (RECOMMENDED)
  2. Use only a single database for all entities and set a "type" field for each of the documents
  3. Create multiple subdomains for CouchDB and use a max of 6 active synchronizations (or less) for each

RxReplicationState

The method RxCollection.syncCouchDB() returns a RxReplicationState which can be used to observe events via rxjs-observables and to cancel the replication.

change$

Emits the change-events every time some documents get replicated.

replicationState.change$.subscribe(change => console.dir(change));

docs$

Emits each replicated document-data.

replicationState.docs$.subscribe(docData => console.dir(docData));

denied$

Emits when a document failed to replicate (e.g. due to permissions).

replicationState.denied$.subscribe(docData => console.dir(docData));

active$

Emits true or false depending if the replication is transmitting data. A false value does not imply the connection has died.

replicationState.active$.subscribe(active => console.dir(active));

alive$

Emits true or false depending if the replication is alive - data is transmitting properly between databases. A false value implies the connection has died -if you're replicating to a remote database, for example. It will only emit false if there are pending changes that couldn't be replicated -it won't emit immediately after the connection dies.

replicationState.alive$.subscribe(alive => console.dir(alive));

complete$

Emits true or false depending if the replication is completed. If you do a live: true-sync (default) the replication never completes. Only one-time-replications will complete.

replicationState.complete$.subscribe(completed => console.dir(completed));

error$

If errors occur during the replication, they will get emitted here.

replicationState.error$.subscribe(error => console.dir(error));

awaitInitialReplication()

Returns a Promise that resolves when the initial replication is done and the data is equal to replication goal. This only works on non-live replications and when waitForLeadership: false

const repState = await myCollection.syncCouchDB({
  remote: 'http://localhost:8080/db',
  waitForLeadership: false,
  options: {
    live: false
  }
});
await repState.awaitInitialReplication();

cancel()

Calling this method will cancel the replication.

await replicationState.cancel(); // cancel() is async

Known problems and workarounds

  • When you do many writes in parallel and replicate them. Your replication might stuck because it reaches the maxSockets value of nodejs. You can increase it by setting
require('http').globalAgent.maxSockets = 10;
  • Especially when you replicate big attachments, you might get a stuck or slow replication. See. You can solve this be changing the batches configuration.
const replicationState = await myCollection.syncCouchDB({
    remote: 'http://...',
    options: {
        retry: true,
        batch_size: 1, // only transfer one document per batch
        batches_limit: 1 // only one batch in parrallel
    }
});
  • When you replicate attachments bigger then 1mb you might cause the replication to stuck. This is an unsolved problem with pouchdb that requires further analysis. It is recommended to not store attachments bigger then 1mb when using the replication.