Code

streaming update#1

Code

Closed

jasonf20 wants to merge 47 commits intomainjasonf20/iceberg:mainfrom streaming-updatejasonf20/iceberg:streaming-updateCopy head branch name to clipboard

+7,444-2,342

Commits

Commits on Dec 5, 2023

• Spark: Don't allow branch_ usage with VERSION AS OF (apache#9219)

  nastra

  authored
  afe4aec

  Copy full SHA for afe4aec

• Flink: Document watermark generation feature (apache#9179)

  pvary

  authored
  8519224

  Copy full SHA for 8519224

• Build: Bump datamodel-code-generator from 0.24.2 to 0.25.0 (apache#9189)

  Show description for 68d491e
  

  Fokko

  and

  dependabot[bot]

  authored
  68d491e

  Copy full SHA for 68d491e

• Flink: backport PR apache#9216 for disabling classloader check (apache#9226)

  stevenzwu

  authored
  7b12a41

  Copy full SHA for 7b12a41

• Build: Bump actions/setup-java from 3 to 4 (apache#9200)

  Show description for 8b7a280
  dependabot[bot]

  authored
  8b7a280

  Copy full SHA for 8b7a280

• Core: Handle IAE in default error handler (apache#9225)

  nastra

  authored
  d80d7da

  Copy full SHA for d80d7da

• Core: Fix logic for determining set of committed files in BaseTransaction when there are no new snapshots (apache#9221)

  amogh-jahagirdar

  authored
  faa8b50

  Copy full SHA for faa8b50

• Style: Replace Arrays.asList with Collections.singletonList (apache#9213)

  yyy1000

  authored
  8e1900d

  Copy full SHA for 8e1900d

Commits on Dec 6, 2023

• Core: Add comment property to ViewProperties (apache#9181)

  amogh-jahagirdar

  authored
  367dc8b

  Copy full SHA for 367dc8b

• Spec: Clarify how column IDs are required (apache#9162)

  emkornfield

  authored
  a89fc46

  Copy full SHA for a89fc46

• Spark: Bump Spark minor versions for 3.3 and 3.4 (apache#9187)

  ajantha-bhat

  authored
  70ec4e5

  Copy full SHA for 70ec4e5

• Core: Introduce AssertViewUUID for REST catalog views (apache#8831)

  nastra

  authored
  d69ba05

  Copy full SHA for d69ba05

• Core: Fix equality in StructLikeMap (apache#9236)

  aokolnychyi

  authored
  e276753

  Copy full SHA for e276753

Commits on Dec 7, 2023

• Core: Add PartitionMap (apache#9194)

  aokolnychyi

  authored
  6a9d3c7

  Copy full SHA for 6a9d3c7

• Docs: Document reading in Spark using branch and tag identifiers (apache#9238)

  wypoon

  authored
  af9522a

  Copy full SHA for af9522a

• Nessie: Reimplement namespace operations (apache#8857)

  Show description for ea7665e
  adutra

  authored
  ea7665e

  Copy full SHA for ea7665e

• Docs: Update default format version to 2. (apache#9239)

  zhongyujiang

  authored
  d929590

  Copy full SHA for d929590

• Flink: Move flink/v1.17 to flink/v1.18

  

  rodmeneses

  authored and

  stevenzwu

  committed
  820fc3c

  Copy full SHA for 820fc3c

• Recover flink/1.17 files from history

  

  rodmeneses

  authored and

  stevenzwu

  committed
  b8ef64a

  Copy full SHA for b8ef64a

• Remove Flink 1.15

  

  rodmeneses

  authored and

  stevenzwu

  committed
  274390f

  Copy full SHA for 274390f

• Make Flink 1.18 to work

  

  rodmeneses

  authored and

  stevenzwu

  committed
  22b95dc

  Copy full SHA for 22b95dc

• Delta: Fix integration tests and Create DataFile by partition values instead of path (apache#8398)

  HonahX

  authored
  b79a8ff

  Copy full SHA for b79a8ff

• Spark 3.5: Support Specifying spec_id in RewriteManifestProcedure (apache#9242)

  puchengy

  authored
  263b530

  Copy full SHA for 263b530

Commits on Dec 8, 2023

• Spark 3.5: Rework DeleteFileIndexBenchmark (apache#9165)

  aokolnychyi

  authored
  feeaa8c

  Copy full SHA for feeaa8c

• Spark 3.2, 3.3, 3.4: Support specifying spec_id in RewriteManifestProcedure (apache#9243)(apache#9242)

  puchengy

  authored
  3f5f4d9

  Copy full SHA for 3f5f4d9

• Spark 3.5: Fix testReplacePartitionField for rewriting manifests (apache#9250)

  bknbkn

  authored
  504c134

  Copy full SHA for 504c134

• Core: Fix null partitions in PartitionSet (apache#9248)

  aokolnychyi

  authored
  62a23a3

  Copy full SHA for 62a23a3

• Flink: switch to use SortKey for data statistics (apache#9212)

  stevenzwu

  authored
  beb41b6

  Copy full SHA for beb41b6

Commits on Dec 9, 2023

• Flink: Fix IcebergSource tableloader lifecycle management in batch mode (apache#9173)

  mas-chen

  authored
  4d0b69b

  Copy full SHA for 4d0b69b

• Flink: backport PR apache#9212 to 1.16 for switching to SortKey for data statistics

  stevenzwu

  committed
  2152269

  Copy full SHA for 2152269

• Flink: backport PR apache#9212 to 1.18 for switching to SortKey for data statistics

  stevenzwu

  committed
  2c31acc

  Copy full SHA for 2c31acc

Commits on Dec 10, 2023

• Build: Bump actions/setup-python from 4 to 5 (apache#9266)

  Show description for 5e03d06
  dependabot[bot]

  authored
  5e03d06

  Copy full SHA for 5e03d06

• Build: Bump actions/labeler from 4 to 5 (apache#9264)

  Show description for 1b80537
  dependabot[bot]

  authored
  1b80537

  Copy full SHA for 1b80537

• Build: Bump actions/stale from 8.0.0 to 9.0.0 (apache#9265)

  Show description for ec92fa3
  dependabot[bot]

  authored
  ec92fa3

  Copy full SHA for ec92fa3

• Build: Bump mkdocs-material from 9.4.12 to 9.5.1 (apache#9256)

  Show description for 06894db
  dependabot[bot]

  authored
  06894db

  Copy full SHA for 06894db

• Build: Bump net.snowflake:snowflake-jdbc from 3.14.3 to 3.14.4 (apache#9257)

  Show description for d3deeec
  dependabot[bot]

  authored
  d3deeec

  Copy full SHA for d3deeec

• Build: Bump software.amazon.awssdk:bom from 2.21.29 to 2.21.42 (apache#9259)

  Show description for ce9186f
  dependabot[bot]

  authored
  ce9186f

  Copy full SHA for ce9186f

• Build: Bump com.google.cloud:libraries-bom from 26.27.0 to 26.28.0 (apache#9258)

  Show description for 0331aba
  dependabot[bot]

  authored
  0331aba

  Copy full SHA for 0331aba

• Core: Improve view/table detection when replacing a table/view (apache#9012)

  nastra

  authored
  7d06af3

  Copy full SHA for 7d06af3

• Core: Add REST catalog table session cache (apache#8920)

  Show description for 4090a88
  nastra

  authored
  4090a88

  Copy full SHA for 4090a88

Commits on Dec 11, 2023

• Build: Bump datamodel-code-generator from 0.24.2 to 0.25.1 (apache#9199)

  Show description for 61cf766
  dependabot[bot]

  authored
  61cf766

  Copy full SHA for 61cf766

• Revert "Build: Bump actions/labeler from 4 to 5 (apache#9264)" (apache#9271)

  Show description for 1b95305
  ajantha-bhat

  authored
  1b95305

  Copy full SHA for 1b95305

• Open-API: Refactor updates with discriminator (apache#9240)

  Show description for 4920189
  Fokko

  authored
  4920189

  Copy full SHA for 4920189

• MR: Migrate tests to JUnit5 (apache#9241)

  lschetanrao

  authored
  f21199d

  Copy full SHA for f21199d

• MergingSnapshotProducer: Change file holder to be generic

  Show description for 934a3f9
  jasonf20

  committed
  934a3f9

  Copy full SHA for 934a3f9

• MergingSnapshotProducer: Support adding data files at a specific sequence number

  jasonf20

  committed
  01762b1

  Copy full SHA for 01762b1

• Table Operations: Added streaming update operation

  Show description for 5357e35
  jasonf20

  committed
  5357e35

  Copy full SHA for 5357e35