Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add support of customTime metadata #413

Merged
merged 7 commits into from
Aug 27, 2020
Merged

feat: add support of customTime metadata #413

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
7ed367e
feat: support customTime metadata
athakor Jun 30, 2020
50df38f
feat: add javaDoc
athakor Jun 30, 2020
9f0bbaa
feat: address few nits
athakor Aug 26, 2020
ed62141
feat: fix the api breaking change
athakor Aug 27, 2020
17ed9c5
Add exception to avoid breaking change
JesseLovelace Aug 27, 2020
fe2ffbf
Add exception to avoid breaking change
JesseLovelace Aug 27, 2020
7ee53f8
Merge branch 'storage-blob-customTime' of github.com:athakor/java-sto…
JesseLovelace Aug 27, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions google-cloud-storage/clirr-ignored-differences.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,4 +26,9 @@
<method>com.google.cloud.storage.BucketInfo$Builder setUpdateTime(java.lang.Long)</method>
<differenceType>7013</differenceType>
</difference>
<difference>
<className>com/google/cloud/storage/BlobInfo$Builder</className>
<method>com.google.cloud.storage.BlobInfo$Builder setCustomTime(java.lang.Long)</method>
<differenceType>7013</differenceType>
</difference>
</differences>
6 changes: 6 additions & 0 deletions google-cloud-storage/src/main/java/com/google/cloud/storage/Blob.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -434,6 +434,12 @@ Builder setCreateTime(Long createTime) {
return this;
}

@Override
public Builder setCustomTime(Long customTime) {
infoBuilder.setCustomTime(customTime);
return this;
}

@Override
Builder setIsDirectory(boolean isDirectory) {
infoBuilder.setIsDirectory(isDirectory);
Expand Down
37 changes: 37 additions & 0 deletions google-cloud-storage/src/main/java/com/google/cloud/storage/BlobInfo.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,7 @@ public StorageObject apply(BlobInfo blobInfo) {
private final String etag;
private final String md5;
private final String crc32c;
private final Long customTime;
private final String mediaLink;
private final Map<String, String> metadata;
private final Long metageneration;
Expand Down Expand Up @@ -260,6 +261,22 @@ public abstract static class Builder {
*/
public abstract Builder setCrc32c(String crc32c);

/**
* Sets the custom time for an object. Once set it can't be unset and only changed to a custom
* datetime in the future. If the custom_time must be unset, you must either perform a rewrite
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't use "custom_time" in this context, say "custom time" instead. Replace "If the custom_time must be unset," with "To unset the custom time,"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

* operation or upload the data again.
*
* <p>Example of setting the custom_time.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

custom time instead of custom_time

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

*
* <pre>{@code
* String bucketName = "my-unique-bucket";
* String blobName = "my-blob-name";
* long customTime = 1598423868301L;
* BlobInfo blob = BlobInfo.newBuilder(bucketName, blobName).setCustomTime(customTime).build();
* }</pre>
*/
public abstract Builder setCustomTime(Long customTime);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this abstract? If it's concrete it wouldn't be a breaking change.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's the development model we've used over the years. It was designed before my time and folks who worked on the library have since left the team.

We could update it to concrete method instead but we would start changing development flow. Maybe that's a good thing, but I'd prefer to prevent changing development flow until we have a new design that fits into that model.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The pot roast principle applies:

https://www.psychologytoday.com/us/blog/thinking-makes-it-so/201402/the-pot-roast-principle

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If it's implemented, it could be implemented to raise an exception of it not being implemented. Seems a little clunky, but would remove the need to create a breaking change.

Thanks for raising the simple questions @elharo, I haven't applied this train of thought to this problem, and I appreciate the feedback.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@athakor please implement this method instead of using abstract to prevent the breaking change. Instead raise an exception of unimplemented if called directly.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done


/**
* Sets the CRC32C checksum of blob's data as described in <a
* href="http://tools.ietf.org/html/rfc4960#appendix-B">RFC 4960, Appendix B;</a> from hex
Expand Down Expand Up @@ -325,6 +342,7 @@ static final class BuilderImpl extends Builder {
private String selfLink;
private String md5;
private String crc32c;
private Long customTime;
private String mediaLink;
private Map<String, String> metadata;
private Long metageneration;
Expand Down Expand Up @@ -360,6 +378,7 @@ static final class BuilderImpl extends Builder {
selfLink = blobInfo.selfLink;
md5 = blobInfo.md5;
crc32c = blobInfo.crc32c;
customTime = blobInfo.customTime;
mediaLink = blobInfo.mediaLink;
metadata = blobInfo.metadata;
metageneration = blobInfo.metageneration;
Expand Down Expand Up @@ -488,6 +507,12 @@ public Builder setCrc32c(String crc32c) {
return this;
}

@Override
public Builder setCustomTime(Long customTime) {
this.customTime = customTime;
return this;
}

@Override
public Builder setCrc32cFromHexString(String crc32cHexString) {
if (crc32cHexString == null) {
Expand Down Expand Up @@ -619,6 +644,7 @@ public BlobInfo build() {
selfLink = builder.selfLink;
md5 = builder.md5;
crc32c = builder.crc32c;
customTime = builder.customTime;
mediaLink = builder.mediaLink;
metadata = builder.metadata;
metageneration = builder.metageneration;
Expand Down Expand Up @@ -857,6 +883,11 @@ public Long getCreateTime() {
return createTime;
}

/** Returns the custom time specified by the user for an object. */
public Long getCustomTime() {
return customTime;
}

/**
* Returns {@code true} if the current blob represents a directory. This can only happen if the
* blob is returned by {@link Storage#list(String, Storage.BlobListOption...)} when the {@link
Expand Down Expand Up @@ -1002,6 +1033,9 @@ public ObjectAccessControl apply(Acl acl) {
if (createTime != null) {
storageObject.setTimeCreated(new DateTime(createTime));
}
if (customTime != null) {
storageObject.setCustomTime(new DateTime(customTime));
}
if (size != null) {
storageObject.setSize(BigInteger.valueOf(size));
}
Expand Down Expand Up @@ -1125,6 +1159,9 @@ static BlobInfo fromPb(StorageObject storageObject) {
if (storageObject.getTimeCreated() != null) {
builder.setCreateTime(storageObject.getTimeCreated().getValue());
}
if (storageObject.getCustomTime() != null) {
builder.setCustomTime(storageObject.getCustomTime().getValue());
}
if (storageObject.getSize() != null) {
builder.setSize(storageObject.getSize().longValue());
}
Expand Down
6 changes: 6 additions & 0 deletions google-cloud-storage/src/test/java/com/google/cloud/storage/BlobInfoTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,7 @@ public class BlobInfoTest {
private static final Long SIZE = 1024L;
private static final Long UPDATE_TIME = DELETE_TIME - 1L;
private static final Long CREATE_TIME = UPDATE_TIME - 1L;
private static final Long CUSTOM_TIME = CREATE_TIME - 1L;
private static final String ENCRYPTION_ALGORITHM = "AES256";
private static final String KEY_SHA256 = "keySha";
private static final CustomerEncryption CUSTOMER_ENCRYPTION =
Expand Down Expand Up @@ -101,6 +102,7 @@ public class BlobInfoTest {
.setSize(SIZE)
.setUpdateTime(UPDATE_TIME)
.setCreateTime(CREATE_TIME)
.setCustomTime(CUSTOM_TIME)
.setStorageClass(STORAGE_CLASS)
.setKmsKeyName(KMS_KEY_NAME)
.setEventBasedHold(EVENT_BASED_HOLD)
Expand Down Expand Up @@ -195,6 +197,7 @@ public void testBuilder() {
assertEquals(SIZE, BLOB_INFO.getSize());
assertEquals(UPDATE_TIME, BLOB_INFO.getUpdateTime());
assertEquals(CREATE_TIME, BLOB_INFO.getCreateTime());
assertEquals(CUSTOM_TIME, BLOB_INFO.getCustomTime());
assertEquals(STORAGE_CLASS, BLOB_INFO.getStorageClass());
assertEquals(KMS_KEY_NAME, BLOB_INFO.getKmsKeyName());
assertEquals(EVENT_BASED_HOLD, BLOB_INFO.getEventBasedHold());
Expand All @@ -214,6 +217,7 @@ public void testBuilder() {
assertNull(DIRECTORY_INFO.getCrc32c());
assertNull(DIRECTORY_INFO.getCrc32cToHexString());
assertNull(DIRECTORY_INFO.getCreateTime());
assertNull(DIRECTORY_INFO.getCustomTime());
assertNull(DIRECTORY_INFO.getDeleteTime());
assertNull(DIRECTORY_INFO.getEtag());
assertNull(DIRECTORY_INFO.getGeneration());
Expand Down Expand Up @@ -257,6 +261,7 @@ private void compareBlobs(BlobInfo expected, BlobInfo value) {
assertEquals(expected.getOwner(), value.getOwner());
assertEquals(expected.getSelfLink(), value.getSelfLink());
assertEquals(expected.getSize(), value.getSize());
assertEquals(expected.getCustomTime(), value.getCustomTime());
assertEquals(expected.getUpdateTime(), value.getUpdateTime());
assertEquals(expected.getStorageClass(), value.getStorageClass());
assertEquals(expected.getKmsKeyName(), value.getKmsKeyName());
Expand Down Expand Up @@ -299,6 +304,7 @@ public void testToPbAndFromPb() {
assertNull(blobInfo.getCrc32c());
assertNull(blobInfo.getCrc32cToHexString());
assertNull(blobInfo.getCreateTime());
assertNull(blobInfo.getCustomTime());
assertNull(blobInfo.getDeleteTime());
assertNull(blobInfo.getEtag());
assertNull(blobInfo.getGeneration());
Expand Down
5 changes: 5 additions & 0 deletions google-cloud-storage/src/test/java/com/google/cloud/storage/BlobTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -91,6 +91,7 @@ public class BlobTest {
private static final Long SIZE = 1024L;
private static final Long UPDATE_TIME = DELETE_TIME - 1L;
private static final Long CREATE_TIME = UPDATE_TIME - 1L;
private static final Long CUSTOM_TIME = CREATE_TIME - 1L;
private static final String ENCRYPTION_ALGORITHM = "AES256";
private static final String KEY_SHA256 = "keySha";
private static final BlobInfo.CustomerEncryption CUSTOMER_ENCRYPTION =
Expand Down Expand Up @@ -122,6 +123,7 @@ public class BlobTest {
.setSize(SIZE)
.setUpdateTime(UPDATE_TIME)
.setCreateTime(CREATE_TIME)
.setCustomTime(CUSTOM_TIME)
.setCustomerEncryption(CUSTOMER_ENCRYPTION)
.setKmsKeyName(KMS_KEY_NAME)
.setEventBasedHold(EVENT_BASED_HOLD)
Expand Down Expand Up @@ -510,6 +512,7 @@ public void testBuilder() {
.setContentLanguage(CONTENT_LANGUAGE)
.setCrc32c(CRC32)
.setCreateTime(CREATE_TIME)
.setCustomTime(CUSTOM_TIME)
.setCustomerEncryption(CUSTOMER_ENCRYPTION)
.setKmsKeyName(KMS_KEY_NAME)
.setEventBasedHold(EVENT_BASED_HOLD)
Expand Down Expand Up @@ -539,6 +542,7 @@ public void testBuilder() {
assertEquals(CRC32, blob.getCrc32c());
assertEquals(CRC32_HEX_STRING, blob.getCrc32cToHexString());
assertEquals(CREATE_TIME, blob.getCreateTime());
assertEquals(CUSTOM_TIME, blob.getCustomTime());
assertEquals(CUSTOMER_ENCRYPTION, blob.getCustomerEncryption());
assertEquals(KMS_KEY_NAME, blob.getKmsKeyName());
assertEquals(EVENT_BASED_HOLD, blob.getEventBasedHold());
Expand Down Expand Up @@ -589,6 +593,7 @@ public void testBuilder() {
assertNull(blob.getSelfLink());
assertEquals(0L, (long) blob.getSize());
assertNull(blob.getUpdateTime());
assertNull(blob.getCustomTime());
assertTrue(blob.isDirectory());
}

Expand Down
4 changes: 3 additions & 1 deletion google-cloud-storage/src/test/java/com/google/cloud/storage/it/ITStorageTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -519,9 +519,11 @@ public void testUpdateBucketDefaultKmsKeyName() throws ExecutionException, Inter
@Test
public void testCreateBlob() {
String blobName = "test-create-blob";
BlobInfo blob = BlobInfo.newBuilder(BUCKET, blobName).build();
BlobInfo blob =
BlobInfo.newBuilder(BUCKET, blobName).setCustomTime(System.currentTimeMillis()).build();
Blob remoteBlob = storage.create(blob, BLOB_BYTE_CONTENT);
assertNotNull(remoteBlob);
assertNotNull(remoteBlob.getCustomTime());
assertEquals(blob.getBucket(), remoteBlob.getBucket());
assertEquals(blob.getName(), remoteBlob.getName());
byte[] readBytes = storage.readAllBytes(BUCKET, blobName);
Expand Down