[Umberella] Support Paimon in Authz

[bowenliang123]

Code of Conduct

• I agree to follow this project's Code of Conduct

Search before asking

• I have searched in the issues and found no similar issues.

Describe the proposal

Introduce Paimon support in Kyuubi Authz plugin.

Apache Paimon is an incubating Apache project of data lake platform for high-speed data ingestion, changelog tracking and efficient real-time analytics.

Task list

Paimon Spark 3 Reference: https://paimon.apache.org/docs/master/engines/spark3/

• DDL Create Tables:
  • Create Table: [AUTHZ] Support create table command for Paimon #5462
  • Create Partitioned Table: https://paimon.apache.org/docs/master/how-to/creating-tables/#partitioned-tables
  • Create Table As: https://paimon.apache.org/docs/master/how-to/creating-tables/#create-table-as
• DDL Alter Tables : https://paimon.apache.org/docs/master/how-to/altering-tables
  • Changing/Adding Table Properties
  • Rename Table Name
  • Removing Table Properties
  • Adding New Columns
  • Renaming Column Name
  • Dropping Columns
  • Changing Column Nullability
  • Changing Column Comment
  • Adding Column Position
  • Changing Column Position
  • Changing Column Type
  • Adding watermark
  • Dropping watermark
  • Changing watermark
• DML:
  • Insert Table
  • Query Table
    • Batch Time Travel: https://paimon.apache.org/docs/master/how-to/querying-tables/#batch-time-travel
  • Update Table
• Management procedures:
  • [TASK][EASY] Support checks for Paimon system procudures in Authz #5430
    • Rollback to snapshot: https://paimon.apache.org/docs/master/maintenance/manage-snapshots/#rollback-to-snapshot

Are you willing to submit PR?

• Yes. I would be willing to submit a PR with guidance from the Kyuubi community to improve.
• No. I cannot submit a PR at this time.

[davidyuan1223]

hello, i want try this issue, but i'm not sure how to improve it, this code is a createTableAsSql for paimon

  test("CreateTableAs") {
    withCleanTmpResources(Seq((s"$catalogV2.$namespace1.$table1", "table"),
      (s"$catalogV2.$namespace1.$table2", "table"))) {
      doAs(
        admin,
        sql(
          s"""
             |CREATE TABLE IF NOT EXISTS $catalogV2.$namespace1.$table1
             |(id int, name string, city string)
             |USING paimon
             |OPTIONS (
             |  primaryKey = 'id'
             |)
             |""".stripMargin
        )
      )
      interceptContains[AccessControlException] {
        doAs(
          someone,
          sql(
            s"""
               |CREATE TABLE IF NOT EXISTS $catalogV2.$namespace1.$table2
               |USING PAIMON
               |AS
               |SELECT * FROM $catalogV2.$namespace1.$table1
               |""".stripMargin
          )
        )
      }(s"does not have [select] privilege on [$table1/id]")
    }
  }

But when i try admin to execute the createTableAs Sql, the authz also will print exception, like user [admin] does not have [select] privilege on [$table1/id]

[bowenliang123]

hello, i want try this issue, but i'm not sure how to improve it, this code is a createTableAsSql for paimon

  test("CreateTableAs") {
    withCleanTmpResources(Seq((s"$catalogV2.$namespace1.$table1", "table"),
      (s"$catalogV2.$namespace1.$table2", "table"))) {
      doAs(
        admin,
        sql(
          s"""
             |CREATE TABLE IF NOT EXISTS $catalogV2.$namespace1.$table1
             |(id int, name string, city string)
             |USING paimon
             |OPTIONS (
             |  primaryKey = 'id'
             |)
             |""".stripMargin
        )
      )
      interceptContains[AccessControlException] {
        doAs(
          someone,
          sql(
            s"""
               |CREATE TABLE IF NOT EXISTS $catalogV2.$namespace1.$table2
               |USING PAIMON
               |AS
               |SELECT * FROM $catalogV2.$namespace1.$table1
               |""".stripMargin
          )
        )
      }(s"does not have [select] privilege on [$table1/id]")
    }
  }

But when i try admin to execute the createTableAs Sql, the authz also will print exception, like user [admin] does not have [select] privilege on [$table1/id]

Hi, I'm not reproducing the case you mentioned. And #5590 is raised for it, you may have a check on this.
If you still encounter this issue, please submit a dedicated issue with code and logs showing the full test cases for positive and negative.

[davidyuan1223]

@bowenliang123 Paimon with Spark looks like not support chaning column nullability and watermark currently, suggest remove this subtask, the command is support with flink