Skip to content

Commit

Permalink
[#21963] YSQL: Support clockbound on other cloud providers
Browse files Browse the repository at this point in the history
Summary:
### Azure PHC Issue

Azure VMs have hardware clocks too. However, we haven't figured out how we can use them yet. Currently, the clockbound configuration script fatals with the following error.

```
PHC is not available on eth0
```

**Fix:** Configure PTP only when the script runs on an AWS machine.

### Missing policycoreutils package

Install policycoreutils-devel explicitly.

### Yugabyted changes

clockbound can now be used on any cloud provider. So, alter users with a warning when using Azure or GCP as well.
Jira: DB-10879

Test Plan:
Jenkins: compile only

Ran

```
sudo bash ./bin/configure_clockbound.sh
```

on AWS, Azure, and GCP

Reviewers: nikhil, sanketh

Reviewed By: sanketh

Differential Revision: https://phorge.dev.yugabyte.com/D39224
  • Loading branch information
pao214 committed Oct 25, 2024
1 parent 820d4f4 commit 689117b
Show file tree
Hide file tree
Showing 2 changed files with 46 additions and 25 deletions.
57 changes: 36 additions & 21 deletions bin/configure_clockbound.sh
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,9 @@ set -euo pipefail
VERBOSE=0
VALIDATE_ONLY=0
OS_RELEASE=""
CHRONY_CONF=""
CHRONY_USER=""
CLOUD_PROVIDER="unknown"

# Parse arguments
while [[ $# -gt 0 ]]; do
Expand Down Expand Up @@ -277,6 +279,7 @@ require {
allow bin_t chronyd_t:unix_dgram_socket sendto;
allow chronyd_t unconfined_service_t:unix_dgram_socket sendto;
EOF
dnf install policycoreutils-devel -y
checkmodule -M -m -o chrony_uds_access.mod chrony_uds_access.te
semodule_package -o chrony_uds_access.pp -m chrony_uds_access.mod
semodule -i chrony_uds_access.pp
Expand Down Expand Up @@ -309,6 +312,14 @@ install_clockbound() {
fi
}

# Function to detect cloud provider based on the chrony configuration
retrieve_cloud_provider() {
# Check if AWS PTP server is configured
if grep -q "server\s*169.254.169.123" "${CHRONY_CONF}"; then
CLOUD_PROVIDER="aws"
fi
}

configure_clockbound() {
if ! systemctl is-active --quiet clockbound; then
# Configure and start clockbound
Expand All @@ -325,29 +336,33 @@ configure_clockbound() {
fatal "Neither 'chrony' nor '_chrony' user exists. Exiting."
fi

# Pick ETH_DEVICE as the first non-loopback device.
for iface in /sys/class/net/*; do
iface=$(basename "$iface")
if [[ "${iface}" != "lo" ]]; then
ETH_DEVICE="${iface}"
break
fi
done

EXTRA_ARGS=""
if chronyc sources | grep "#.\s*PHC" > /dev/null 2>&1; then
# Check if PHC is available on ETH_DEVICE.
if ethtool -T "${ETH_DEVICE}" | grep -q "PTP Hardware Clock: none"; then
fatal "PHC is not available on ${ETH_DEVICE}."
fi

# Check whether a PHC source is selected.
if ! chronyc sources | grep "#\*\s*PHC" > /dev/null 2>&1; then
fatal "PHC source is not selected as the clock soruce."
# Check for PTP only on AWS instances.
retrieve_cloud_provider
if [[ "${CLOUD_PROVIDER}" == "aws" ]]; then
if chronyc sources | grep "#.\s*PHC" > /dev/null 2>&1; then
# Pick ETH_DEVICE as the first non-loopback device.
for iface in /sys/class/net/*; do
iface=$(basename "$iface")
if [[ "${iface}" != "lo" ]]; then
ETH_DEVICE="${iface}"
break
fi
done

# Check if PHC is available on ETH_DEVICE.
if ethtool -T "${ETH_DEVICE}" | grep -q "PTP Hardware Clock: none"; then
fatal "PHC is not available on ${ETH_DEVICE}."
fi

# Check whether a PHC source is selected.
if ! chronyc sources | grep "#\*\s*PHC" > /dev/null 2>&1; then
fatal "PHC source is not selected as the clock soruce."
fi

PHC_ID=$(chronyc sources | grep "#\*\s*PHC" | awk '{print $2}')
EXTRA_ARGS="-r ${PHC_ID} -i ${ETH_DEVICE}"
fi

PHC_ID=$(chronyc sources | grep "#\*\s*PHC" | awk '{print $2}')
EXTRA_ARGS="-r ${PHC_ID} -i ${ETH_DEVICE}"
fi

# Create the clockbound service file based on systemd version
Expand Down
14 changes: 10 additions & 4 deletions bin/yugabyted
Original file line number Diff line number Diff line change
Expand Up @@ -666,14 +666,20 @@ def get_cli_title():
cli_title += div_line
return cli_title

def has_aws_time_sync_service():
def using_time_sync_service():
# List of recognized IP addresses and sources
allow_list = ['169.254.169.123', 'metadata.google.internal', 'PHC',
'aws.com', 'google.com']

try:
# Run the chronyc sources command and capture the output
result = subprocess.run(['chronyc', 'sources'], capture_output=True, text=True, timeout=1)

# Check if 169.254.169.123 is in the output
if result.returncode == 0 and '169.254.169.123' in result.stdout:
return True
# Check if any allowed source is in the output
if result.returncode == 0:
for source in allow_list:
if source in result.stdout:
return True
except (subprocess.TimeoutExpired, FileNotFoundError):
return False

Expand Down

0 comments on commit 689117b

Please sign in to comment.