From f76f199c7edeaf77a0119fccd774690873795ccc Mon Sep 17 00:00:00 2001 From: Bvsk Patnaik Date: Mon, 14 Oct 2024 13:42:55 -0700 Subject: [PATCH] [#21963] YSQL: Configure clockbound agent --- .../deploy/manual-deployment/system-config.md | 67 +++++++++++++++++-- 1 file changed, 61 insertions(+), 6 deletions(-) diff --git a/docs/content/preview/deploy/manual-deployment/system-config.md b/docs/content/preview/deploy/manual-deployment/system-config.md index 28a0a83a35fe..d66e38a06c44 100644 --- a/docs/content/preview/deploy/manual-deployment/system-config.md +++ b/docs/content/preview/deploy/manual-deployment/system-config.md @@ -13,7 +13,7 @@ type: docs Perform the following configuration on each node in the cluster: -- ntp or chrony +- chrony (ntp) - ulimits - transparent hugepages @@ -21,20 +21,75 @@ Keep in mind that, although YugabyteDB is PostgreSQL compatible and runs a postg In particular, the main YugabyteDB process, the YB-TServer, is multi-threaded. As a result, you don't need to modify settings for shared memory and inter-process communication (IPC), because there is no inter-process communication or shared memory in a multi-threaded process model (all memory is shared by the same process). -## ntp +## chrony (ntp) -If your instance does not have public Internet access, make sure the ntp package is installed: +YugabyteDB relies on clock synchronization to guarantee consistency in distributed transactions. Chrony is the prefered NTP implementation for clock synchronization. + +To install chrony, run: ```sh -$ sudo yum install -y ntp +$ sudo yum install -y chrony ``` -As of CentOS 8, `ntp` is no longer available and has been replaced by `chrony`. To install, run: +### PTP + +**Supported Platforms:** AWS + +{{}} Precision Time Protocol (PTP) is a network protocol designed for highly accurate time synchronization across devices in a network. PTP provides microsecond-level accuracy. + +PTP relies on a PTP Hardware Clock (PHC), a dedicated physical clock device that enhances time synchronization accuracy. To check if your AWS instance supports PHC, see [AWS PTP Hardware Clock](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/configure-ec2-ntp.html#connect-to-the-ptp-hardware-clock). + +Configure PTP using the `configure_ptp.sh` script in the bin directory of your YugabyteDB home directory as follows: ```sh -$ sudo yum install -y chrony +sudo bash ./bin/configure_ptp.sh +``` + +### Clockbound Agent + +{{}} Although optional, configuring a clockbound agent improves clock accuracy by several orders of magnitude. +While PHC is beneficial, it is not required for configuring clockbound. However, Chrony is a necessary prerequisite. + +Configure a clockbound agent using the `configure_clockbound.sh` script in the bin directory of your YugabyteDB home directory as follows: + +```sh +sudo bash ./bin/configure_clockbound.sh ``` +After configuring a clockbound agent, you must configure the [YB-TServer](../start-tservers/) and [YB-Master](../start-masters/) servers with the `time_source=clockbound` flag. + +If the clockbound agent is configured with PTP, use a more aggressive clock error estimate such as `clockbound_clock_error_estimate_usec=100`. + +
+Verification + +Verify that clockbound is configured properly using + +```sh +systemctl status clockbound +``` + +A correctly configured clockbound service reports no errors. Example output with PTP enabled: + +```sh +● clockbound.service - ClockBound + Loaded: loaded (/usr/lib/systemd/system/clockbound.service; enabled; preset: disabled) + Active: active (running) since Wed 2024-10-16 23:49:38 UTC; 53s ago + Main PID: 92765 (clockbound) + Tasks: 3 (limit: 22143) + Memory: 4.1M + CPU: 18ms + CGroup: /system.slice/clockbound.service + └─92765 /usr/local/bin/clockbound --max-drift-rate 50 -r PHC0 -i eth0 + +Oct 16 23:49:38 ip-172-199-76-70.ec2.internal systemd[1]: Started ClockBound. +Oct 16 23:49:38 ip-172-199-76-70.ec2.internal clockbound[92765]: 2024-10-16T23:49:38.629593Z INFO main ThreadId(01) /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/c> +Oct 16 23:49:38 ip-172-199-76-70.ec2.internal clockbound[92765]: 2024-10-16T23:49:38.629874Z INFO ThreadId(02) /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/clock-> +Oct 16 23:49:38 ip-172-199-76-70.ec2.internal clockbound[92765]: 2024-10-16T23:49:38.630045Z INFO ThreadId(03) /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/clock-> +``` + +
+ ## ulimits In Linux, `ulimit` is used to limit and control the usage of system resources (threads, files, and network connections) on a per-process or per-user basis.