Couchbase Eventing Service was introduced in Couchbase Data Platform 5.5 release. Couchbase Data Platform 6.0 release saw the introduction of Timers. Similar to the Indexing Best Practices that we had published earlier, let’s look at some best practices when it comes to using Couchbase Eventing Service:
- On Debuggability and Diagnosability
(a) Never use Debugger in Production environments. A thread is allocated for the debugger session when it is ON. Debugger being Enabled(i.e, ON) in Production environments can lead to out of order events for a given document. (Ability to Enable/Disable Debugger in different environments is available in 6.0 release.)
(b) Application log files are rotated by default. Update the log file size based on your log rotation policy.
2. Do not deploy Functions leading to cascading writes i.e. Function A listening to bucket A writes to bucket B & Function B listening to bucket B writes to bucket A.
3. Always surround the Javascript code in the Function using a try-catch block to catch all run-time exceptions; logging of exceptions ensures that the messages are captured in the the application specific log files.
4. On Metadata Bucket:
(a) Do not Flush the Metadata bucket when Functions are deployed. The metadata bucket is used to store many system related information which is used by the Couchbase Eventing Service. Flushing of the metadata bucket may lead to loss of checkpoint information which can lead to Eventing Service losing track of sequence of mutations being processed.
(b) Avoid using ephemeral bucket as metadata bucket is possible, as it might eject items if memory usage goes beyond the quota. Ejection of items from the metadata bucket might result in unpredictable behaviors like losing timer events, duplicate events getting fired etc.
(c) We recommend having sufficient replicas for metadata bucket to avoid losing timers and causing duplicate events, in the event of one or more Data service node failures.
(d) Do not use the metadata bucket for other applications. Couchbase Eventing Service prefixes all the system information being stored in the metadata bucket with a unique key. Though this namespacing of the key helps differentiate with other applications that may insert into the same bucket, it is advised that customers do not insert new documents into this bucket from other applications, nor upsert/delete the existing documents inserted by the Couchbase Eventing Service.
5. Statistics displayed in the Administration console for a given Function should be used for infrastructure provisioning and monitoring. Function backlog should be stable and if it’s ever increasing, then either more Eventing nodes need to be deployed or more Eventing workers/node for the given Function need to be assigned.
6. If there are many timeouts for a given Function, understand the root cause and consider increasing the timeout value(In Settings for a given Function). If increasing timeout is not an option, try allocating more resources(cores) to the Eventing nodes or add more Eventing nodes to increasing the processing throughput.
7. When creating Functions for data-archival, that uses timers, study the backlog and apportion enough delta time before the actual expiry. This is to make sure that the document exists in the bucket if it is accessed inside the timer. Else, pass the document as a context in the timer(this is similar to ‘Pass By Value’).
8. Understand Feed Boundaries supported by the Eventing Service. Prefer using ‘From Now’ to save time and processing power when older mutations in a bucket need not be processed.
9. Nodes running the Couchbase Eventing Service ideally should not have other Couchbase services running on the same nodes.Couchbase Eventing Service is a compute oriented service and hence the nodes running it should be core-heavy than being RAM-heavy.
10. Couchbase Server deduplicates multiple updates to a document. Understand how Deduplication and ordering of mutations works in here.
To see a demo of how Couchbase Eventing Service processes a billion documents in under 30minutes, do check out Connect Silicon Valley’18 presentation/talk. Also, from Connect New York’18 (earlier this year).
