minor: update versions and paths in changelog scripts (#2429)

* update versions and paths in changelog scripts

* Update release process README

* fix crate publish order

* add ASL header for dot file

Author: andygrove

dev/release/README.md

@@ -103,9 +103,6 @@ If there is a ballista release, update versions in ballista Cargo.tomls, run
 ./dev/update_ballista_versions.py 0.5.0
 ```
 
-If there is a datafusion python binding release, update versions in
-`./python/Cargo.toml`.
-
 Lastly commit the version change:
 
 ```
@@ -114,11 +111,15 @@ git commit -a -m 'Update version'
 
 ### Update CHANGELOG.md
 
-Manully edit the base version tag argument in
-`dev/release/update_change_log-{ballista,datafusion,python}.sh`. Commits
+Manually edit the base version tag argument in
+`dev/release/update_change_log-{ballista,datafusion}.sh`. Commits
 between the base verstion tag and the latest upstream master will be used to
 populate the changelog content.
 
+You will need a GitHub Personal Access Token for the following steps. Follow 
+[these instructions](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token)
+to generate one if you do not already have one.
+
 ```bash
 # create the changelog
 CHANGELOG_GITHUB_TOKEN=<TOKEN> ./dev/release/update_change_log-all.sh
@@ -137,12 +138,12 @@ the changelog and update the PR accordingly.
 
 ## Prepare release candidate artifacts
 
-After the PR gets merged, you are ready to create releaes artifacts based off the
+After the PR gets merged, you are ready to create release artifacts based off the
 merged commit.
 
 (Note you need to be a committer to run these scripts as they upload to the apache svn distribution servers)
 
-### Pick an Release Candidate (RC) number
+### Pick a Release Candidate (RC) number
 
 Pick numbers in sequential order, with `0` for `rc0`, `1` for `rc1`, etc.
 
@@ -151,7 +152,7 @@ Pick numbers in sequential order, with `0` for `rc0`, `1` for `rc1`, etc.
 While the official release artifacts are signed tarballs and zip files, we also
 tag the commit it was created for convenience and code archaeology.
 
-Using a string such as `5.1.0` as the `<version>`, create and push the tag thusly:
+Using a string such as `5.1.0` as the `<version>`, create and push the tag by running these commands:
 
 ```shell
 git fetch apache
@@ -160,9 +161,6 @@ git tag <version>-<rc> apache/master
 git push apache <version>
 ```
 
-This should trigger the `Python Release Build` Github Action workflow for the
-pushed tag. You can monitor the pipline run status at https://github.com/apache/arrow-datafusion/actions/workflows/python_build.yml.
-
 ### Create, sign, and upload artifacts
 
 Run `create-tarball.sh` with the `<version>` tag and `<rc>` and you found in previous steps:
@@ -240,7 +238,7 @@ the `release-tarball.sh` script:
 ./dev/release/release-tarball.sh 5.1.0 0
 ```
 
-Congratulations! The release is now offical!
+Congratulations! The release is now official!
 
 ### Create release git tags
 
@@ -274,6 +272,14 @@ create an account and login to crates.io before asking to be added as an owner
 of the following crates:
 
 - [datafusion](https://crates.io/crates/datafusion)
+- [datafusion-cli](https://crates.io/crates/datafusion-cli)
+- [datafusion-common](https://crates.io/crates/datafusion-common)
+- [datafusion-data-access](https://crates.io/crates/datafusion-data-access)
+- [datafusion-expr](https://crates.io/crates/datafusion-expr)
+- [datafusion-jit](https://crates.io/crates/datafusion-jit)
+- [datafusion-physical-expr](https://crates.io/crates/datafusion-physical-expr)
+- [datafusion-proto](https://crates.io/crates/datafusion-proto)
+- [datafusion-row](https://crates.io/crates/datafusion-row)
 - [ballista](https://crates.io/crates/ballista)
 - [ballista-core](https://crates.io/crates/ballista-core)
 - [ballista-executor](https://crates.io/crates/ballista-executor)
@@ -282,15 +288,32 @@ of the following crates:
 Download and unpack the official release tarball
 
 Verify that the Cargo.toml in the tarball contains the correct version
-(e.g. `version = "5.1.0"`) and then publish the crate with the
-following commands
+(e.g. `version = "5.1.0"`) and then publish the crates with the
+following commands. Crates need to be published in the correct order as shown in this diagram.
+
+![](crate-deps.svg)
+
+_To update this diagram, manually edit the dependencies in [crate-deps.dot](crate-deps.dot) and then run:_
+
+``` bash
+dot -Tsvg dev/release/crate-deps.dot > dev/release/crate-deps.svg
+```
 
 ```shell
-(cd datafusion-common && cargo publish)
-(cd datafusion-expr && cargo publish)
+(cd data-access && cargo publish)
+(cd datafusion/common && cargo publish)
+(cd datafusion/expr && cargo publish)
+(cd datafusion/physical-expr && cargo publish)
+(cd datafusion/jit && cargo publish)
+(cd datafusion/row && cargo publish)
 (cd datafusion && cargo publish)
+(cd datafusion/proto && cargo publish)
+(cd datafusion-cli && cargo publish)
 ```
 
+Note that `datafusion-cli` will depend on the previously published version of Ballista (which depends on the 
+previously published version of DataFusion) until https://github.com/apache/arrow-datafusion/issues/2433 is resolved.
+
 If there is a ballista release, run
 
 ```shell
@@ -300,7 +323,6 @@ If there is a ballista release, run
 (cd ballista/rust/client && cargo publish)
 ```
 
-
 ### Publish datafusion-cli on Homebrew and crates.io
 
 For Homebrew, Send a simple PR to update tag and commit hash for the datafusion

dev/release/crate-deps.dot

@@ -0,0 +1,62 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+digraph G {
+
+    datafusion_common
+
+    datafusion_data_access
+
+	datafusion_expr -> datafusion_common
+
+	datafusion_physical_expr -> datafusion_common
+	datafusion_physical_expr -> datafusion_expr
+
+	datafusion_jit -> datafusion_common
+	datafusion_jit -> datafusion_expr
+
+    datafusion_row -> datafusion_common
+    datafusion_row -> datafusion_expr
+    datafusion_row -> datafusion_jit
+
+	datafusion -> datafusion_common
+	datafusion -> datafusion_data_access
+	datafusion -> datafusion_expr
+	datafusion -> datafusion_jit
+	datafusion -> datafusion_physical_expr
+	datafusion -> datafusion_row
+
+	datafusion_proto -> datafusion
+
+	ballista_core -> datafusion
+	ballista_core -> datafusion_proto
+
+	ballista_scheduler -> ballista_core
+	ballista_scheduler -> datafusion
+
+	ballista_executor -> ballista_core
+	ballista_executor -> datafusion
+
+    ballista -> datafusion
+    ballista -> ballista_core
+    ballista -> ballista_scheduler
+    ballista -> ballista_executor
+
+	datafusion_cli -> datafusion
+	datafusion_cli -> ballista
+
+}