Skip to content

Commit

Permalink
[yugabyte#21963] YSQL: Configure clockbound agent
Browse files Browse the repository at this point in the history
  • Loading branch information
pao214 committed Oct 17, 2024
1 parent 3f7333a commit f76f199
Showing 1 changed file with 61 additions and 6 deletions.
67 changes: 61 additions & 6 deletions docs/content/preview/deploy/manual-deployment/system-config.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,28 +13,83 @@ type: docs

Perform the following configuration on each node in the cluster:

- ntp or chrony
- chrony (ntp)
- ulimits
- transparent hugepages

Keep in mind that, although YugabyteDB is PostgreSQL compatible and runs a postgres process, it is not a PostgreSQL distribution. The PostgreSQL it runs doesn't need the same OS and system resources that open source PostgreSQL requires. For this reason, the kernel configuration requirements are different.

In particular, the main YugabyteDB process, the YB-TServer, is multi-threaded. As a result, you don't need to modify settings for shared memory and inter-process communication (IPC), because there is no inter-process communication or shared memory in a multi-threaded process model (all memory is shared by the same process).

## ntp
## chrony (ntp)

If your instance does not have public Internet access, make sure the ntp package is installed:
YugabyteDB relies on clock synchronization to guarantee consistency in distributed transactions. Chrony is the prefered NTP implementation for clock synchronization.

To install chrony, run:

```sh
$ sudo yum install -y ntp
$ sudo yum install -y chrony
```

As of CentOS 8, `ntp` is no longer available and has been replaced by `chrony`. To install, run:
### PTP

**Supported Platforms:** AWS

{{<tags/feature/tp>}} Precision Time Protocol (PTP) is a network protocol designed for highly accurate time synchronization across devices in a network. PTP provides microsecond-level accuracy.

PTP relies on a PTP Hardware Clock (PHC), a dedicated physical clock device that enhances time synchronization accuracy. To check if your AWS instance supports PHC, see [AWS PTP Hardware Clock](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/configure-ec2-ntp.html#connect-to-the-ptp-hardware-clock).

Configure PTP using the `configure_ptp.sh` script in the bin directory of your YugabyteDB home directory as follows:

```sh
$ sudo yum install -y chrony
sudo bash ./bin/configure_ptp.sh
```

### Clockbound Agent

{{<tags/feature/tp>}} Although optional, configuring a clockbound agent improves clock accuracy by several orders of magnitude.
While PHC is beneficial, it is not required for configuring clockbound. However, Chrony is a necessary prerequisite.

Configure a clockbound agent using the `configure_clockbound.sh` script in the bin directory of your YugabyteDB home directory as follows:

```sh
sudo bash ./bin/configure_clockbound.sh
```

After configuring a clockbound agent, you must configure the [YB-TServer](../start-tservers/) and [YB-Master](../start-masters/) servers with the `time_source=clockbound` flag.

If the clockbound agent is configured with PTP, use a more aggressive clock error estimate such as `clockbound_clock_error_estimate_usec=100`.

<details>
<summary>Verification</summary>

Verify that clockbound is configured properly using

```sh
systemctl status clockbound
```

A correctly configured clockbound service reports no errors. Example output with PTP enabled:

```sh
● clockbound.service - ClockBound
Loaded: loaded (/usr/lib/systemd/system/clockbound.service; enabled; preset: disabled)
Active: active (running) since Wed 2024-10-16 23:49:38 UTC; 53s ago
Main PID: 92765 (clockbound)
Tasks: 3 (limit: 22143)
Memory: 4.1M
CPU: 18ms
CGroup: /system.slice/clockbound.service
└─92765 /usr/local/bin/clockbound --max-drift-rate 50 -r PHC0 -i eth0

Oct 16 23:49:38 ip-172-199-76-70.ec2.internal systemd[1]: Started ClockBound.
Oct 16 23:49:38 ip-172-199-76-70.ec2.internal clockbound[92765]: 2024-10-16T23:49:38.629593Z INFO main ThreadId(01) /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/c>
Oct 16 23:49:38 ip-172-199-76-70.ec2.internal clockbound[92765]: 2024-10-16T23:49:38.629874Z INFO ThreadId(02) /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/clock->
Oct 16 23:49:38 ip-172-199-76-70.ec2.internal clockbound[92765]: 2024-10-16T23:49:38.630045Z INFO ThreadId(03) /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/clock->
```

</details>

## ulimits

In Linux, `ulimit` is used to limit and control the usage of system resources (threads, files, and network connections) on a per-process or per-user basis.
Expand Down

0 comments on commit f76f199

Please sign in to comment.