Skip to content
This repository has been archived by the owner on Jan 19, 2022. It is now read-only.

Spanner added bounded staleness option and tests #1727

Merged
merged 8 commits into from
Jul 9, 2019
Merged
Show file tree
Hide file tree
Changes from 7 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 8 additions & 5 deletions docs/src/main/asciidoc/spanner.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -646,8 +646,8 @@ Main benefit of reads over queries is reading multiple rows of a certain pattern
===== Stale read

All reads and queries are *strong reads* by default.
A *strong read* is a read at a current timestamp and is guaranteed to see all data that has been committed up until the start of this read.
A *stale read* on the other hand is read at a timestamp in the past.
A *strong read* is a read at a current time and is guaranteed to see all data that has been committed up until the start of this read.
An *exact staleness read* is read at a timestamp in the past.
Cloud Spanner allows you to determine how current the data should be when you read data.
With `SpannerTemplate` you can specify the `Timestamp` by setting it on `SpannerQueryOptions` or `SpannerReadOptions` to the appropriate read or query methods:

Expand All @@ -656,7 +656,7 @@ Reads:
[source,java]
----
// a read with options:
SpannerReadOptions spannerReadOptions = new SpannerReadOptions().setTimestamp(Timestamp.now());
SpannerReadOptions spannerReadOptions = new SpannerReadOptions().setTimestamp(myTimestamp);
List<Trade> trades = this.spannerTemplate.readAll(Trade.class, spannerReadOptions);
----

Expand All @@ -665,10 +665,13 @@ Queries:
[source,java]
----
// a query with options:
SpannerQueryOptions spannerQueryOptions = new SpannerQueryOptions().setTimestamp(Timestamp.now());
SpannerQueryOptions spannerQueryOptions = new SpannerQueryOptions().setTimestamp(myTimestamp);
List<Trade> trades = this.spannerTemplate.query(Trade.class, Statement.of("SELECT * FROM trades"), spannerQueryOptions);
----

You can also read with https://cloud.google.com/spanner/docs/timestamp-bounds[*bounded staleness*] by setting `.setTimestampBound(TimestampBound.ofMinReadTimestamp(myTimestamp))` on the query and read options objects.
ChengyuanZhao marked this conversation as resolved.
Show resolved Hide resolved
Bounded staleness lets Cloud Spanner choose any point in time later than or equal to the given timestampBound, but it cannot be used inside transactions.


===== Read from a secondary index

Expand Down Expand Up @@ -864,7 +867,7 @@ The final returned value and type of the function is determined by the user.
You can use this object just as you would a regular `SpannerOperations` with
a few exceptions:

- Its read functionality cannot perform stale reads, because all reads happen at the single point in time of the transaction.
- Its read functionality cannot perform stale reads (other than the staleness set on the entire transaction), because all reads happen at the single point in time of the transaction.
- It cannot perform sub-transactions via `performReadWriteTransaction` or `performReadOnlyTransaction`
- It cannot perform any write operations.

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
/*
* Copyright 2017-2019 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package org.springframework.cloud.gcp.data.spanner.core;

import java.io.Serializable;
import java.lang.reflect.Array;
import java.util.ArrayList;
import java.util.List;
import java.util.Set;

import com.google.cloud.Timestamp;
import com.google.cloud.spanner.TimestampBound;

/**
* Abstract class of common Read and Query request settings.
*
* @author Chengyuan Zhao
*/
public class AbstractSpannerRequestOptions<A> implements Serializable {

protected transient List<A> requestOptions = new ArrayList<>();

protected Class<A> requestOptionType;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this actually work? This member is never assigned? Would it not be null?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's assigned in the constructors of the child classes. It's needed to create that array of ReadOption or QueryOption that the client lib requires.


private TimestampBound timestampBound;

private Set<String> includeProperties;

private boolean allowPartialRead;

public Set<String> getIncludeProperties() {
return this.includeProperties;
}

public AbstractSpannerRequestOptions setIncludeProperties(Set<String> includeProperties) {
this.includeProperties = includeProperties;
return this;
}

public TimestampBound getTimestampBound() {
return this.timestampBound;
}

/**
* Set if this query should be executed with bounded staleness.
* @param timestampBound the timestamp bound. Can be exact or bounded staleness.
* @return this options object.
*/
public AbstractSpannerRequestOptions setTimestampBound(TimestampBound timestampBound) {
this.timestampBound = timestampBound;
return this;
}

public Timestamp getTimestamp() {
return this.timestampBound.getMode() == TimestampBound.Mode.READ_TIMESTAMP
? this.timestampBound.getReadTimestamp()
: this.timestampBound.getMinReadTimestamp();
}

public AbstractSpannerRequestOptions setTimestamp(Timestamp timestamp) {
this.timestampBound = TimestampBound.ofReadTimestamp(timestamp);
return this;
}

public A[] getOptions() {
return this.requestOptions.toArray((A[]) Array.newInstance(this.requestOptionType, 0));
}

public boolean isAllowPartialRead() {
return this.allowPartialRead;
}

public AbstractSpannerRequestOptions setAllowPartialRead(boolean allowPartialRead) {
this.allowPartialRead = allowPartialRead;
return this;
}

}
Original file line number Diff line number Diff line change
Expand Up @@ -20,12 +20,12 @@
import java.util.function.Function;
import java.util.function.Supplier;

import com.google.cloud.Timestamp;
import com.google.cloud.spanner.DatabaseClient;
import com.google.cloud.spanner.Mutation;
import com.google.cloud.spanner.ReadContext;
import com.google.cloud.spanner.ReadOnlyTransaction;
import com.google.cloud.spanner.Statement;
import com.google.cloud.spanner.TimestampBound;

import org.springframework.cloud.gcp.data.spanner.core.admin.SpannerSchemaUtils;
import org.springframework.cloud.gcp.data.spanner.core.convert.SpannerEntityProcessor;
Expand Down Expand Up @@ -73,7 +73,7 @@ protected ReadContext getReadContext() {
}

@Override
protected ReadContext getReadContext(Timestamp timestamp) {
protected ReadContext getReadContext(TimestampBound timestampBound) {
throw new SpannerDataException(
"Getting stale snapshot read contexts is not supported"
+ " in read-only transaction templates.");
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,11 +20,11 @@
import java.util.function.Function;
import java.util.function.Supplier;

import com.google.cloud.Timestamp;
import com.google.cloud.spanner.DatabaseClient;
import com.google.cloud.spanner.Mutation;
import com.google.cloud.spanner.ReadContext;
import com.google.cloud.spanner.Statement;
import com.google.cloud.spanner.TimestampBound;
import com.google.cloud.spanner.TransactionContext;

import org.springframework.cloud.gcp.data.spanner.core.admin.SpannerSchemaUtils;
Expand Down Expand Up @@ -71,7 +71,7 @@ public long executeDmlStatement(Statement statement) {
}

@Override
protected ReadContext getReadContext(Timestamp timestamp) {
protected ReadContext getReadContext(TimestampBound timestampBound) {
throw new SpannerDataException(
"Getting stale snapshot read contexts is not supported"
+ " in read-write transaction templates.");
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,12 @@

package org.springframework.cloud.gcp.data.spanner.core;

import java.util.Set;

import com.google.cloud.Timestamp;
import com.google.cloud.spanner.Options;
import com.google.cloud.spanner.TimestampBound;

import org.springframework.data.domain.Sort;
import org.springframework.util.Assert;

Expand Down Expand Up @@ -63,9 +69,34 @@ public SpannerPageableQueryOptions setSort(Sort sort) {
return this;
}

@Override
public SpannerPageableQueryOptions addQueryOption(Options.QueryOption queryOption) {
super.addQueryOption(queryOption);
return this;
}

@Override
public SpannerPageableQueryOptions setIncludeProperties(Set<String> includeProperties) {
super.setIncludeProperties(includeProperties);
return this;
}

@Override
public SpannerPageableQueryOptions setTimestampBound(TimestampBound timestampBound) {
super.setTimestampBound(timestampBound);
return this;
}

@Override
public SpannerPageableQueryOptions setTimestamp(Timestamp timestamp) {
ChengyuanZhao marked this conversation as resolved.
Show resolved Hide resolved
super.setTimestamp(timestamp);
return this;
}

@Override
public SpannerPageableQueryOptions setAllowPartialRead(boolean allowPartialRead) {
super.setAllowPartialRead(allowPartialRead);
return this;
}

}
Original file line number Diff line number Diff line change
Expand Up @@ -16,13 +16,11 @@

package org.springframework.cloud.gcp.data.spanner.core;

import java.io.Serializable;
import java.util.ArrayList;
import java.util.List;
import java.util.Set;

import com.google.cloud.Timestamp;
import com.google.cloud.spanner.Options.QueryOption;
import com.google.cloud.spanner.TimestampBound;

import org.springframework.util.Assert;

Expand All @@ -35,59 +33,53 @@
*
* @since 1.1
*/
public class SpannerQueryOptions implements Serializable {

private transient List<QueryOption> queryOptions = new ArrayList<>();

private Timestamp timestamp;

private Set<String> includeProperties;

private boolean allowPartialRead;
public class SpannerQueryOptions extends AbstractSpannerRequestOptions<QueryOption> {

/**
* Constructor to create an instance. Use the extension-style add/set functions to add
* options and settings.
*/
public SpannerQueryOptions() {
this.requestOptionType = QueryOption.class;
}

public SpannerQueryOptions addQueryOption(QueryOption queryOption) {
Assert.notNull(queryOption, "Valid query option is required!");
this.queryOptions.add(queryOption);
this.requestOptions.add(queryOption);
return this;
}

public Set<String> getIncludeProperties() {
return this.includeProperties;
}

@Override
public SpannerQueryOptions setIncludeProperties(Set<String> includeProperties) {
this.includeProperties = includeProperties;
super.setIncludeProperties(includeProperties);
return this;
}

public Timestamp getTimestamp() {
ChengyuanZhao marked this conversation as resolved.
Show resolved Hide resolved
return this.timestamp;
@Override
public SpannerQueryOptions setTimestampBound(TimestampBound timestampBound) {
super.setTimestampBound(timestampBound);
return this;
}

@Override
public SpannerQueryOptions setTimestamp(Timestamp timestamp) {
Assert.notNull(timestamp, "A valid timestamp is required!");
this.timestamp = timestamp;
super.setTimestamp(timestamp);
return this;
}

public QueryOption[] getQueryOptions() {
return this.queryOptions.toArray(new QueryOption[this.queryOptions.size()]);
@Override
public SpannerQueryOptions setAllowPartialRead(boolean allowPartialRead) {
super.setAllowPartialRead(allowPartialRead);
return this;
}

public boolean isAllowPartialRead() {
return this.allowPartialRead;
/**
* Deprecated. Please use {@code getOptions}.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you use @deprecated like we do in other parts of the code base?

* @return the array of query request options.
*/
@Deprecated
public QueryOption[] getQueryOptions() {
return this.getOptions();
}

public SpannerQueryOptions setAllowPartialRead(
boolean allowPartialRead) {
this.allowPartialRead = allowPartialRead;
return this;
}
}
Loading