[YSQL] Provide a spanner-like behavior to wait out the clock skew at commit time.

[pao214]

Jira Link: DB-11226

Description

Context

We use hybrid timestamping mechanism to decide ordering between commits and reads. But this is problematic since clocks are skewed in a distributed database such as YB. This skew can be mitigated in two ways,

1. Use a spanner-like solution where you wait out the skew at commit time.
2. Use an uncertainty window during the read time to detect any commits that may have happened before us (but have a timestamp higher than that of the read).

YB uses approach 2, since approach 1 is too expensive when clock skews are high. The clock skews have gotten as large as 500ms in the past.

Motivation

However, times have changed, and clouds now provide services to get highly accurate clocks. For example, AWS provides announced microsecond-accurate clocks in their recent changes to AWS Time Sync Service.

In the presence of such accurate clocks, we wish to employ a spanner-like approach (approach 1). This has a couple of advantages

1. The waiting can be done in parallel to raft replication of the commit status. When clock skews are small, the raft replication takes longer, and we do not spend any additional time waiting.
2. We get external consistency for free.
3. No read restart errors at all.

Objective

1. Minimize clock skew

Explore AWS Time Sync Service. Use AWS Nitro system, configure PTP hardware clocks and use tight update intervals to see how good we can bound the clock skew. This AWS blog mentions that the clock error is typically bounded by 40us. Even with NTP (not PTP), we should get a 100us time. This is very much acceptable.

2. Check for inefficiencies in how we measure clock error bound.

I have also noticed that ntp_adjtime gives an error that is couple orders of magnitude higher than what chrony gives us. Use aws clock bound implementation as a reference.

3. Provide an option to wait out the clock skew at commit time.

This cannot be a runtime flag because of changes in hyrbid time values. Ideally, this is an AUTO FLAG.

Issue Type

kind/bug

Warning: Please confirm that this issue does not contain any sensitive information

• I confirm this issue does not contain any sensitive information.