Skip to content

Commit

Permalink
Update mass_comment.py when we cut release branch (#26506)
Browse files Browse the repository at this point in the history 
* Update mass_comment when we cut release branch

* Update release guide
  • Loading branch information
@damccorm
damccorm authored May 3, 2023
1 parent 7a93e21 commit e868c8d
Show file tree
Hide file tree
Showing 5 changed files with 231 additions and 161 deletions.
43 changes: 42 additions & 1 deletion .github/workflows/cut_release_branch.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,12 @@ on:
description: Next release version
required: true
default: '2.XX.0'
JENKINS_USERNAME:
description: Username of the current Jenkins user. Used to update mass_comment.py with all Jenkins jobs.
required: true
JENKINS_TOKEN:
description: API Token for the current Jenkins user. Can be generated at https://ci-beam.apache.org/user/<user>/configure
required: true

jobs:
update_master:
Expand All @@ -46,6 +52,11 @@ jobs:
NEXT_RELEASE: ${{ github.event.inputs.NEXT_VERSION }}
SCRIPT_DIR: ./release/src/main/scripts
steps:
- name: Mask Jenkins token
run: |
echo "::add-mask::$JENKINS_TOKEN"
env:
JENKINS_TOKEN: ${{ github.event.inputs.JENKINS_TOKEN }}
- name: Validate Next Version
run: |
if [[ $NEXT_RELEASE =~ ([0-9]+\.[0-9]+\.[0-9]+) ]]; then
Expand All @@ -60,6 +71,36 @@ jobs:
run: |
git config user.name $GITHUB_ACTOR
git config user.email actions@"$RUNNER_NAME".local
- name: Install xmllint
run: sudo apt-get install -y libxml2-utils
- name: Update mass_comment.py with new Jenkins jobs
run: |
rm release/src/main/scripts/jenkins_jobs.txt
for obj in $(curl https://ci-beam.apache.org/api/json | jq '.jobs[]' -c); do
echo $obj
url=$(echo $obj | jq '.url' -r || echo "")
name=$(echo $obj | jq '.name' -r || echo "")
if [ "$url" != "" ] && [ "$name" != "" ]; then
curl --user $JENKINS_USERNAME:$JENKINS_TOKEN $url/config.xml > /tmp/config.xml
if [ "$(xmllint --xpath 'string(//disabled)' /tmp/config.xml)" = "true" ]; then
echo "SKIPPING $url - disabled"
else
phrase=$(xmllint --xpath 'string(//triggers/org.jenkinsci.plugins.ghprb.GhprbTrigger/triggerPhrase)' /tmp/config.xml)
if [ "$phrase" != "" ]; then
echo "Appending ${phrase},${name}"
echo "${phrase},${name}" >> /tmp/result
fi
fi
rm /tmp/config.xml
fi
done
cat /tmp/result | sort | uniq | grep -i -E 'precommit|postcommit|validates|vr|example|test|gradle build' | grep -v -i -E 'load|perf|website' >> release/src/main/scripts/jenkins_jobs.txt
env:
JENKINS_USERNAME: ${{ github.event.inputs.JENKINS_USERNAME }}
JENKINS_TOKEN: ${{ github.event.inputs.JENKINS_TOKEN }}
- name: Update master branch
run: |
bash "${SCRIPT_DIR}/set_version.sh" "${NEXT_VERSION_IN_BASE_BRANCH}"
Expand Down Expand Up @@ -152,4 +193,4 @@ jobs:
echo "NOTE: If there is no jenkins job started, please comment generated PR with: Run Gradle Publish"
echo "Things remained you need to do manually after build successful:"
echo "1. Close this generated PR in github website."
echo "2. Delete your remote branch ${BRANCH_NAME} form your beam repo in github website."
echo "2. Delete your remote branch ${BRANCH_NAME} form your beam repo in github website."
3 changes: 3 additions & 0 deletions build.gradle.kts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,9 @@ tasks.rat {
"website/www/site/static/js/keen-slider.min.js",
"website/www/site/assets/scss/_keen-slider.scss",

// Release automation files
"release/src/main/scripts/*.txt",

// Ignore ownership files
"ownership/**/*",
"**/OWNERS",
Expand Down
167 changes: 167 additions & 0 deletions release/src/main/scripts/jenkins_jobs.txt
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,167 @@
Run Beam Metrics deployment,beam_PostCommit_BeamMetrics_Publish_PR
Run Chicago Taxi on Dataflow,beam_PostCommit_Python_Chicago_Taxi_Dataflow_PR
Run Chicago Taxi on Flink,beam_PostCommit_Python_Chicago_Taxi_Flink_PR
Run CommunityMetrics PreCommit,beam_PreCommit_CommunityMetrics_Phrase
Run Dataflow Runner Nexmark Tests,beam_PostCommit_Java_Nexmark_Dataflow_PR
Run Dataflow Runner Tpcds Tests,beam_PostCommit_Java_Tpcds_Dataflow_PR
Run Dataflow Runner V2 Java 11 Nexmark Tests,beam_PostCommit_Java_Nexmark_DataflowV2_Java11_PR
Run Dataflow Runner V2 Java 17 Nexmark Tests,beam_PostCommit_Java_Nexmark_DataflowV2_Java17_PR
Run Dataflow Runner V2 Nexmark Tests,beam_PostCommit_Java_Nexmark_DataflowV2_PR
Run Dataflow Streaming ValidatesRunner,beam_PostCommit_Java_ValidatesRunner_Dataflow_Streaming_PR
Run Dataflow ValidatesRunner Java 11,beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11_PR
Run Dataflow ValidatesRunner Java 17,beam_PostCommit_Java_ValidatesRunner_Dataflow_Java17_PR
Run Dataflow ValidatesRunner,beam_PostCommit_Java_ValidatesRunner_Dataflow_PR
Run Direct Runner Nexmark Tests,beam_PostCommit_Java_Nexmark_Direct_PR
Run Direct ValidatesRunner Java 11,beam_PostCommit_Java_ValidatesRunner_Direct_Java11_PR
Run Direct ValidatesRunner Java 17,beam_PostCommit_Java_ValidatesRunner_Direct_Java17_PR
Run Direct ValidatesRunner in Java 11,beam_PostCommit_Java11_ValidatesRunner_Direct_PR
Run Direct ValidatesRunner,beam_PostCommit_Java_ValidatesRunner_Direct_PR
Run Flink Runner Nexmark Tests,beam_PostCommit_Java_Nexmark_Flink_PR
Run Flink Runner Tpcds Tests,beam_PostCommit_Java_Tpcds_Flink_PR
Run Flink ValidatesRunner Java 11,beam_PostCommit_Java_ValidatesRunner_Flink_Java11_PR
Run Flink ValidatesRunner,beam_PostCommit_Java_ValidatesRunner_Flink_PR
Run Go Flink ValidatesRunner,beam_PostCommit_Go_VR_Flink_PR
Run Go PostCommit,beam_PostCommit_Go_PR
Run Go PreCommit,beam_PreCommit_Go_Phrase
Run Go Samza ValidatesRunner,beam_PostCommit_Go_VR_Samza_PR
Run Go Spark ValidatesRunner,beam_PostCommit_Go_VR_Spark_PR
Run GoPortable PreCommit,beam_PreCommit_GoPortable_Phrase
Run Java 11 Examples on Dataflow Runner V2,beam_PostCommit_Java_Examples_Dataflow_V2_java11_PR
Run Java 17 Examples on Dataflow Runner V2,beam_PostCommit_Java_Examples_Dataflow_V2_java17_PR
Run Java Dataflow V2 ValidatesRunner Streaming,beam_PostCommit_Java_VR_Dataflow_V2_Streaming_PR
Run Java Dataflow V2 ValidatesRunner,beam_PostCommit_Java_VR_Dataflow_V2_PR
Run Java Examples on Dataflow Runner V2,beam_PostCommit_Java_Examples_Dataflow_V2_PR
Run Java Examples_Direct,beam_PostCommit_Java_Examples_Direct_PR
Run Java Examples_Flink,beam_PostCommit_Java_Examples_Flink_PR
Run Java Examples_Spark,beam_PostCommit_Java_Examples_Spark_PR
Run Java Flink PortableValidatesRunner Streaming,beam_PostCommit_Java_PVR_Flink_Streaming_PR
Run Java InfluxDbIO_IT,beam_PostCommit_Java_InfluxDbIO_IT_PR
Run Java Portability examples on Dataflow with Java 11,beam_PostCommit_Java11_Examples_Dataflow_Portability_PR
Run Java PostCommit,beam_PostCommit_Java_PR
Run Java PreCommit,beam_PreCommit_Java_Phrase
Run Java Samza PortableValidatesRunner,beam_PostCommit_Java_PVR_Samza_PR
Run Java Sickbay,beam_PostCommit_Java_Sickbay_PR
Run Java SingleStoreIO_IT,beam_PostCommit_Java_SingleStoreIO_IT_PR
Run Java Spark PortableValidatesRunner Batch,beam_PostCommit_Java_PVR_Spark_Batch_PR
Run Java Spark v3 PortableValidatesRunner Streaming,beam_PostCommit_Java_PVR_Spark3_Streaming_PR
Run Java examples on Dataflow Java 11,beam_PostCommit_Java_Examples_Dataflow_Java11_PR
Run Java examples on Dataflow Java 17,beam_PostCommit_Java_Examples_Dataflow_Java17_PR
Run Java examples on Dataflow with Java 11,beam_PostCommit_Java11_Examples_Dataflow_PR
Run Java_Amazon-Web-Services2_IO_Direct PreCommit,beam_PreCommit_Java_Amazon-Web-Services2_IO_Direct_Phrase
Run Java_Amazon-Web-Services_IO_Direct PreCommit,beam_PreCommit_Java_Amazon-Web-Services_IO_Direct_Phrase
Run Java_Amqp_IO_Direct PreCommit,beam_PreCommit_Java_Amqp_IO_Direct_Phrase
Run Java_Azure_IO_Direct PreCommit,beam_PreCommit_Java_Azure_IO_Direct_Phrase
Run Java_Cassandra_IO_Direct PreCommit,beam_PreCommit_Java_Cassandra_IO_Direct_Phrase
Run Java_Cdap_IO_Direct PreCommit,beam_PreCommit_Java_Cdap_IO_Direct_Phrase
Run Java_Clickhouse_IO_Direct PreCommit,beam_PreCommit_Java_Clickhouse_IO_Direct_Phrase
Run Java_Csv_IO_Direct PreCommit,beam_PreCommit_Java_Csv_IO_Direct_Phrase
Run Java_Debezium_IO_Direct PreCommit,beam_PreCommit_Java_Debezium_IO_Direct_Phrase
Run Java_Elasticsearch_IO_Direct PreCommit,beam_PreCommit_Java_ElasticSearch_IO_Direct_Phrase
Run Java_Examples_Dataflow PreCommit,beam_PreCommit_Java_Examples_Dataflow_Phrase
Run Java_Examples_Dataflow_Java11 PreCommit,beam_PreCommit_Java_Examples_Dataflow_Java11_Phrase
Run Java_Examples_Dataflow_Java17 PreCommit,beam_PreCommit_Java_Examples_Dataflow_Java17_Phrase
Run Java_File-schema-transform_IO_Direct PreCommit,beam_PreCommit_Java_File-schema-transform_IO_Direct_Phrase
Run Java_GCP_IO_Direct PreCommit,beam_PreCommit_Java_GCP_IO_Direct_Phrase
Run Java_Hbase_IO_Direct PreCommit,beam_PreCommit_Java_HBase_IO_Direct_Phrase
Run Java_Hcatalog_IO_Direct PreCommit,beam_PreCommit_Java_HCatalog_IO_Direct_Phrase
Run Java_IOs_Direct PreCommit,beam_PreCommit_Java_IOs_Direct_Phrase
Run Java_Influxdb_IO_Direct PreCommit,beam_PreCommit_Java_InfluxDb_IO_Direct_Phrase
Run Java_JDBC_IO_Direct PreCommit,beam_PreCommit_Java_JDBC_IO_Direct_Phrase
Run Java_Jms_IO_Direct PreCommit,beam_PreCommit_Java_Jms_IO_Direct_Phrase
Run Java_Kafka_IO_Direct PreCommit,beam_PreCommit_Java_Kafka_IO_Direct_Phrase
Run Java_Kinesis_IO_Direct PreCommit,beam_PreCommit_Java_Kinesis_IO_Direct_Phrase
Run Java_Kudu_IO_Direct PreCommit,beam_PreCommit_Java_Kudu_IO_Direct_Phrase
Run Java_MongoDb_IO_Direct PreCommit,beam_PreCommit_Java_MongoDb_IO_Direct_Phrase
Run Java_Mqtt_IO_Direct PreCommit,beam_PreCommit_Java_Mqtt_IO_Direct_Phrase
Run Java_Neo4j_IO_Direct PreCommit,beam_PreCommit_Java_Neo4j_IO_Direct_Phrase
Run Java_PVR_Flink_Batch PreCommit,beam_PreCommit_Java_PVR_Flink_Batch_Phrase
Run Java_PVR_Flink_Docker PreCommit,beam_PreCommit_Java_PVR_Flink_Docker_Phrase
Run Java_Parquet_IO_Direct PreCommit,beam_PreCommit_Java_Parquet_IO_Direct_Phrase
Run Java_Pulsar_IO_Direct PreCommit,beam_PreCommit_Java_Pulsar_IO_Direct_Phrase
Run Java_Rabbitmq_IO_Direct PreCommit,beam_PreCommit_Java_RabbitMq_IO_Direct_Phrase
Run Java_Redis_IO_Direct PreCommit,beam_PreCommit_Java_Redis_IO_Direct_Phrase
Run Java_Singlestore_IO_Direct PreCommit,beam_PreCommit_Java_SingleStore_IO_Direct_Phrase
Run Java_Snowflake_IO_Direct PreCommit,beam_PreCommit_Java_Snowflake_IO_Direct_Phrase
Run Java_Solr_IO_Direct PreCommit,beam_PreCommit_Java_Solr_IO_Direct_Phrase
Run Java_Spark3_Versions PreCommit,beam_PreCommit_Java_Spark3_Versions_Phrase
Run Java_Splunk_IO_Direct PreCommit,beam_PreCommit_Java_Splunk_IO_Direct_Phrase
Run Java_Thrift_IO_Direct PreCommit,beam_PreCommit_Java_Thrift_IO_Direct_Phrase
Run Java_Tika_IO_Direct PreCommit,beam_PreCommit_Java_Tika_IO_Direct_Phrase
Run Java_hadoop_IO_Direct PreCommit,beam_PreCommit_Java_Hadoop_IO_Direct_Phrase
Run Javadoc PostCommit,beam_PostCommit_Javadoc_PR
Run Jpms Dataflow Java 11 PostCommit,beam_PostCommit_Java_Jpms_Dataflow_Java11_PR
Run Jpms Dataflow Java 17 PostCommit,beam_PostCommit_Java_Jpms_Dataflow_Java17_PR
Run Jpms Direct Java 11 PostCommit,beam_PostCommit_Java_Jpms_Direct_Java11_PR
Run Jpms Direct Java 17 PostCommit,beam_PostCommit_Java_Jpms_Direct_Java17_PR
Run Jpms Flink Java 11 PostCommit,beam_PostCommit_Java_Jpms_Flink_Java11_PR
Run Jpms Spark Java 11 PostCommit,beam_PostCommit_Java_Jpms_Spark_Java11_PR
Run Kotlin_Examples PreCommit,beam_PreCommit_Kotlin_Examples_Phrase
Run PortableJar_Flink PostCommit,beam_PostCommit_PortableJar_Flink_PR
Run PortableJar_Spark PostCommit,beam_PostCommit_PortableJar_Spark_PR
Run Portable_Python PreCommit,beam_PreCommit_Portable_Python_Phrase
Run PostCommit_Java_Avro_Versions,beam_PostCommit_Java_Avro_Versions_PR
Run PostCommit_Java_Dataflow,beam_PostCommit_Java_DataflowV1_PR
Run PostCommit_Java_DataflowV2,beam_PostCommit_Java_DataflowV2_PR
Run PostCommit_Java_Hadoop_Versions,beam_PostCommit_Java_Hadoop_Versions_PR
Run Python 3.10 PostCommit Sickbay,beam_PostCommit_Sickbay_Python310_PR
Run Python 3.10 PostCommit,beam_PostCommit_Python310_PR
Run Python 3.11 PostCommit Sickbay,beam_PostCommit_Sickbay_Python311_PR
Run Python 3.11 PostCommit,beam_PostCommit_Python311_PR
Run Python 3.7 PostCommit Sickbay,beam_PostCommit_Sickbay_Python37_PR
Run Python 3.7 PostCommit,beam_PostCommit_Python37_PR
Run Python 3.8 PostCommit Sickbay,beam_PostCommit_Sickbay_Python38_PR
Run Python 3.8 PostCommit,beam_PostCommit_Python38_PR
Run Python 3.9 PostCommit Sickbay,beam_PostCommit_Sickbay_Python39_PR
Run Python 3.9 PostCommit,beam_PostCommit_Python39_PR
Run Python Dataflow V2 ValidatesRunner,beam_PostCommit_Py_VR_Dataflow_V2_PR
Run Python Dataflow ValidatesContainer,beam_PostCommit_Py_ValCont_PR
Run Python Dataflow ValidatesRunner,beam_PostCommit_Py_VR_Dataflow_PR
Run Python Direct Runner Nexmark Tests,beam_PostCommit_Python_Nexmark_Direct_PR
Run Python Examples_Dataflow,beam_PostCommit_Python_Examples_Dataflow_PR
Run Python Examples_Direct,beam_PostCommit_Python_Examples_Direct_PR
Run Python Examples_Flink,beam_PostCommit_Python_Examples_Flink_PR
Run Python Examples_Spark,beam_PostCommit_Python_Examples_Spark_PR
Run Python Flink ValidatesRunner,beam_PostCommit_Python_VR_Flink_PR
Run Python MongoDBIO_IT,beam_PostCommit_Python_MongoDBIO_IT_PR
Run Python PreCommit,beam_PreCommit_Python_Phrase
Run Python RC Dataflow ValidatesContainer,beam_PostCommit_Py_ValCont_with_RC_PR
Run Python Samza ValidatesRunner,beam_PostCommit_Python_VR_Samza_PR
Run Python Spark ValidatesRunner,beam_PostCommit_Python_VR_Spark_PR
Run PythonDocker PreCommit,beam_PreCommit_PythonDocker_Phrase
Run PythonDocs PreCommit,beam_PreCommit_PythonDocs_Phrase
Run PythonFormatter PreCommit,beam_PreCommit_PythonFormatter_Phrase
Run PythonLint PreCommit,beam_PreCommit_PythonLint_Phrase
Run Python_Coverage PreCommit,beam_PreCommit_Python_Coverage_Phrase
Run Python_Dataframes PreCommit,beam_PreCommit_Python_Dataframes_Phrase
Run Python_Examples PreCommit,beam_PreCommit_Python_Examples_Phrase
Run Python_Integration PreCommit,beam_PreCommit_Python_Integration_Phrase
Run Python_PVR_Flink PreCommit,beam_PreCommit_Python_PVR_Flink_Phrase
Run Python_Runners PreCommit,beam_PreCommit_Python_Runners_Phrase
Run Python_Transforms PreCommit,beam_PreCommit_Python_Transforms_Phrase
Run Python_Xlang_Gcp_Dataflow PostCommit,beam_PostCommit_Python_Xlang_Gcp_Dataflow_PR
Run Python_Xlang_Gcp_Direct PostCommit,beam_PostCommit_Python_Xlang_Gcp_Direct_PR
Run RAT PreCommit,beam_PreCommit_RAT_Phrase
Run Release Gradle Build,beam_Release_Gradle_Build
Run SQL PostCommit,beam_PostCommit_SQL_PR
Run SQL PreCommit,beam_PreCommit_SQL_Phrase
Run SQL_Java11 PreCommit,beam_PreCommit_SQL_Java11_Phrase
Run SQL_Java17 PreCommit,beam_PreCommit_SQL_Java17_Phrase
Run Samza ValidatesRunner,beam_PostCommit_Java_ValidatesRunner_Samza_PR
Run Spark Runner Nexmark Tests,beam_PostCommit_Java_Nexmark_Spark_PR
Run Spark Runner Tpcds Tests,beam_PostCommit_Java_Tpcds_Spark_PR
Run Spark StructuredStreaming ValidatesRunner,beam_PostCommit_Java_ValidatesRunner_SparkStructuredStreaming_PR
Run Spark ValidatesRunner Java 11,beam_PostCommit_Java_ValidatesRunner_Spark_Java11_PR
Run Spark ValidatesRunner,beam_PostCommit_Java_ValidatesRunner_Spark_PR
Run Spotless PreCommit,beam_PreCommit_Spotless_Phrase
Run Twister2 ValidatesRunner,beam_PostCommit_Java_ValidatesRunner_Twister2_PR
Run Typescript PreCommit,beam_PreCommit_Typescript_Phrase
Run ULR Loopback ValidatesRunner,beam_PostCommit_Java_ValidatesRunner_ULR_PR
Run Whitespace PreCommit,beam_PreCommit_Whitespace_Phrase
Run XVR_Direct PostCommit,beam_PostCommit_XVR_Direct_PR
Run XVR_Flink PostCommit,beam_PostCommit_XVR_Flink_PR
Run XVR_GoUsingJava_Dataflow PostCommit,beam_PostCommit_XVR_GoUsingJava_Dataflow_PR
Run XVR_JavaUsingPython_Dataflow PostCommit,beam_PostCommit_XVR_JavaUsingPython_Dataflow_PR
Run XVR_PythonUsingJavaSQL_Dataflow PostCommit,beam_PostCommit_XVR_PythonUsingJavaSQL_Dataflow_PR
Run XVR_PythonUsingJava_Dataflow PostCommit,beam_PostCommit_XVR_PythonUsingJava_Dataflow_PR
Run XVR_Samza PostCommit,beam_PostCommit_XVR_Samza_PR
Run XVR_Spark3 PostCommit,beam_PostCommit_XVR_Spark3_PR
Loading

0 comments on commit e868c8d

Please sign in to comment.