Couchbase Technologies

Recovering a Couchbase node

While adding back a failed over node, recovery can be performed, by means of rebalancing.

After a node has been failed over :

  • The node can be removed from the cluster, by means of the rebalance operation.
  • The node can be recovered, and thereby added back into the cluster — again by means of the rebalance operation.

Below options are displayed by Couchbase Web Console:

Source : Couchbase Documentation

To recover a failed-over node, you must select a recovery option, before performing a rebalance. Two recovery options are provided, by the buttons at the lower right; which are Add Back: Full Recovery and Add Back: Delta Recovery. After you have selected the appropriate option, you then Rebalance.

Full Recovery

Full recovery involves removing all pre-existing data from, and assigning new data to, the node that is being recovered. Therefore, selecting Full Recovery, left-clicking on Rebalance causes the following:

  • All existing vBuckets and documents are removed from the node.
  • If GSI Indexes reside on the node, they are left unmodified during the rebalance process.
  • A new set of vBuckets and documents is assigned to the node.
  • When the node’s vBuckets are all up to date, and the rebalance process concludes, the node recommences the serving of data. If GSI Indexes reside on the node, they become active, and are updated by the Index Service as appropriate.

Delta Recovery

Delta recovery maintains and resynchronizes a node’s pre-existing data. Therefore, selecting Delta recovery , left-clicking on Rebalance causes the following:

  • No existing vBucket or document is removed from the node.
  • All existing data is loaded into memory.
  • The point at which mutations to node-data most recently stopped is determined. Then, vBuckets are duly updated from that point; based on the data-changes that have since occurred, elsewhere on the cluster.
  • If GSI Indexes reside on the node, they are left unmodified during the rebalance process.
  • When the node’s vBuckets are all up to date, and the rebalance process concludes, the node recommences the serving of data. If GSI Indexes reside on the node, they become active.

Recovery Performance

Delta recovery is faster than Full recovery; since a significant quantity of usable data already resides on the node, and therefore does not require network-transfer: only updates made since the node’s last-recorded mutation need to be accessed from other nodes in the cluster.

However, in cases where the node’s memory-footprint is extremely large, and data-size exceeds bucket memory-quotas, the memory-management overhead potentially entailed by Deltarecovery might imply Full recovery’s taking less time overall.


Source : Performing Node Recovery
See
Recover a Node with the UI


Categories

Categories