Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Generating Reference Pages for Kubernetes Components and Tools: Update how to use the update-imported-docs tool #16777

Merged
merged 1 commit into from
Oct 14, 2019
Merged

Generating Reference Pages for Kubernetes Components and Tools: Update how to use the update-imported-docs tool #16777

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
106 changes: 65 additions & 41 deletions content/en/docs/contribute/generate-ref-docs/kubernetes-components.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,66 +17,48 @@ reference documentation for tools and components in the

* You need to have this software installed:

* [Python 2.7.16](https://www.python.org/downloads/)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hello @aimeeu . I looked through the changes found a few things to tidy up. I made note of some suggested edits.

Line 8: Possible edit of the overview:
This page shows how to use the update-imported-docs tool to generate documentation for the Kubernetes tools and components. The update-imported-docs Python script is located in the kubernetes/website/update-imported-docs/directory.

Line 20: Is it possible to use Python 3? I would assume so.
Line 22: Is it necessary to list the version of Kubernetes here? Golang 1.13 ...
Line 27: Is it true that you need to unset/not set the GOPATH. I would have to confirm this. If I recall correctly, the script first clones the named repo into the tmp dir, then sets the GOPATH, then runs the series of commands in the configuration file. If it is true, I would make this instruction a separate list item.
Line 31: fix the link, Work from a local clone. This link could be shortened in the corresponding file.
Line 37: possibly be more specific: Getting the kubernetes/website repository
Line 39: What happens if you already have a clone of your docs fork? I think we assume that the reader has a clone as they are looking at the docs and the update script?
Line 43: What is: $github.com/website.
Line 49: mkdir github.com? Where on the file system?
Line 53: Possibly add this link, from the Kubernetes source code repository
Line 61: Clean up the intro sentence as mentioned (remove: The tool consists of). Add a list item that indicates that the script takes an argument, a YAML configuration file. Add another list item about that the release version in the configuration should match the version of the current release.
Line 70: Possible edit: Updates kubectl command links in the generated file, kubectl.md.
Lines 76 - 101. I agree. The information is somewhat extraneous.
Line 112: You may want to add information about updating the release number in the configuration file.
Line 175: Fixing links: What does release.yml actually do? Is it only used to update the release notes?
Line 208: Possibly add a new header to build the gen'ed files:
If this page is like a playbook for generating ref docs, then before creating the pull request, the user should build the documentation locally and review the changes 😄

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @kbhawkey for your awesome feedback - very much appreciated!

Line 8: kinda funny - I went back and forth on using "Python script" vs "tool" and opted to keep the original wording

Line 20: I could not run the script with Python 3 and PyYAML 5.1.x (which is automatically installed using pip install pyyaml). That's why I started on PR #16681 To test with Python 2.7.16, I created a virtual environment and installed Python 2.7.16 and PyYAML 3.13 inside it - I figured those versions were close to what was available when the script was written.

Line 22: Yes, I believe it is. I ran into issues trying to run the script when I had Go 1.13 installed, and Kubernetes development does not support 1.13. Or do you mean to link to the list of the supported Go version for each Kubernetes version?

Line 27: recalling our Slack conversation, I think both you and I unset our GOPATH before running the script. I agree - more testing is needed since the script does set the GOPATH. My concern is that before I commented out my existing GOPATH in my .bashrc, the Go code executed by the Python script could not find the cloned repos, making me think that my GOPATH was being found before the GOPATH set by the script

Line 39: I assume the reader is experienced enough to know NOT to clone the website repo if it already exists locally. Maybe omit this bit altogether? Perhaps specifying a certain level of knowledge as a prerequisite... these instructions assume you know how to create a website pull request

Line 49: I think this section should be omitted - I tried to update because the kubernetes/website repo does not need to be cloned into an existing GOPATH; yet the doc refers to <website_root> further down so tried to give an example; the location of kubernetes/website is not relevant - I've run the script now with kubernetes/website cloned into different directories.

Line 175: I have no idea what the release.yml file does or why it's included - maybe @tenqm knows?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wondering if we have any error outputs when running the script against python3?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Regarding the release.yml file, it is used as an input to the script so that the release notes can be auto generated/imported from upstream rep.

./update-imported-docs release.yml

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had a moment to test the script with python3. I kept my GOPATH set (not including the dynamically gen'ed tmp location). With a few updates to the script, the script ran w/o error. I will create a PR with the changes to the script.
I did not set up a virtualenv.

Copy link
Contributor Author

@aimeeu aimeeu Oct 14, 2019 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kbhawkey I've had PR #16681 Update website/update-imported-docs/update-imported-docs script for Python 3, PyYAML 5.1.2 - open for a while now - can you check to see that the changes you made to the script are reflected in my version?

* [Git](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git)
* [Golang](https://golang.org/doc/install) version 1.9 or later
* [Golang](https://golang.org/doc/install) version 1.12 for Kubernetes 1.14+; Go 1.13 [is not supported](https://github.com/kubernetes/community/blob/master/contributors/devel/development.md#go)
* [PyYAML](https://pyyaml.org/) v3.13
* [make](https://www.gnu.org/software/make/)
* [gcc compiler/linker](https://gcc.gnu.org/)

* Your `$GOPATH` environment variable must be set.
* The Go binary must be in your path; **do not** set your `$GOPATH`. The `update-imported-docs` tool sets your GOPATH.

* You need to know how to create a pull request to a GitHub repository.
Typically, this involves creating a fork of the repository. For more
information, see [Creating a Documentation Pull Request](/docs/contribute/start/).
This involves creating your own fork of the repository. For more
information, see [Work from a local clone](/docs/contribute/intermediate/#work_from_a_local_clone).

{{% /capture %}}

{{% capture steps %}}

## Getting two repositories
## Getting the repository

If you don't already have the `kubernetes/website` repository, get it now:
Make sure your `website` fork is up-to-date with the `kubernetes/website` master and then clone your `website` fork.

```shell
mkdir $GOPATH/src
cd $GOPATH/src
go get github.com/kubernetes/website
mkdir github.com
cd github.com
git clone git@github.com:<your_github_username>/website.git
```

Determine the base directory of your clone of the
[kubernetes/website](https://github.com/kubernetes/website) repository.
For example, if you followed the preceding step to get the repository,
your base directory is `$GOPATH/src/github.com/kubernetes/website.`
The remaining steps refer to your base directory as `<web-base>`.
Determine the base directory of your clone. For example, if you followed the
preceding step to get the repository, your base directory is
`$github.com/website.` The remaining steps refer to your base directory as
`<web-base>`.

If you plan on making changes to the ref docs, and if you don't already have
the `kubernetes/kubernetes` repository, get it now:

```shell
mkdir $GOPATH/src
cd $GOPATH/src
go get github.com/kubernetes/kubernetes
```

Determine the base directory of your clone of the
[kubernetes/kubernetes](https://github.com/kubernetes/kubernetes) repository.
For example, if you followed the preceding step to get the repository,
your base directory is `$GOPATH/src/github.com/kubernetes/kubernetes.`
The remaining steps refer to your base directory as `<k8s-base>`.

The reference documentation for the Kubernetes components and tools is automatically
generated from the Kubernetes source code. If you want to change the reference documentation,
please follow [this guide](/docs/contribute/generate-ref-docs/contribute-upstream).

{{< note >}}
If you only need to generate, but not change, the reference docs, you don't need to
manually get the `kubernetes/kubernetes` repository. When you run the `update-imported-docs`
tool, it automatically clones the `kubernetes/kubernetes` repository.
{{< /note >}}
The reference documentation for the Kubernetes components and tools is generated
from the Kubernetes source code. The `update-imported-docs` tool automatically
clones the `kubernetes/kubernetes` repository. If you want to change the
reference documentation, please follow [this
guide](/docs/contribute/generate-ref-docs/contribute-upstream).

## Overview of update-imported-docs

The `update-imported-docs` tool is located in the `kubernetes/website/update-imported-docs/`
directory. The tool performs the following steps:
directory. The tool consists of a Python script that reads a YAML configuration file and performs the following steps:

1. Clones the related repositories specified in a configuration file. For the
purpose of generating reference docs, the repository that is cloned by
Expand All @@ -91,7 +73,43 @@ When the Markdown files are in your local clone of the `kubernetes/website`
repository, you can submit them in a [pull request](/docs/contribute/start/)
to `kubernetes/website`.

## Customizing the config file
## Configuration file format

Each config file may contain multiple repos that will be imported together.
When necessary, you can customize the configuration file by manually editing
it. You may create new config files for importing other groups of documents. Imported documents must follow these guidelines:

1. Adhere to the [Documentation Style Guide](/docs/contribute/style/style-guide/).

1. Have `title` defined in the front matter. For example:

```
---
title: Title Displayed in Table of Contents
---

Rest of the .md file...
```
1. Be listed in the `kubernetes/website/data/reference.yml` file

The following is an example of the YAML configuration file:

```yaml
repos:
- name: community
remote: https://github.com/kubernetes/community.git
branch: master
files:
- src: contributors/devel/README.md
dst: docs/imported/community/devel.md
- src: contributors/guide/README.md
dst: docs/imported/community/guide.md
```

Note: `generate-command` is an optional entry, which can be used to run a
given command or a short script to generate the docs from within a repo.
Comment on lines +76 to +110
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to include this or should it be removed since we tell people to use reference.yml?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm more inclined to leave it there.


## Customizing the reference.yml config file

Open `<web-base>/update-imported-docs/reference.yml` for editing.
Do not change the content for the `generate-command` entry unless you understand
Expand All @@ -108,7 +126,7 @@ repos:
cd $GOPATH
git clone https://github.com/kubernetes/kubernetes.git src/k8s.io/kubernetes
cd src/k8s.io/kubernetes
git checkout release-1.11
git checkout release-1.16
make generated_files
cp -L -R vendor $GOPATH/src
rm -r vendor
Expand Down Expand Up @@ -154,6 +172,12 @@ cd <web-base>/update-imported-docs
./update-imported-docs reference.yml
```

## Fixing Links

To fix relative links within your imported files, set the repo config's
`gen-absolute-links` property to `true`. You can find an example of this in
[`release.yml`](https://github.com/kubernetes/website/blob/master/update-imported-docs/release.yml).

## Adding and committing changes in kubernetes/website

List the files that were generated and copied to the `kubernetes/website`
Expand Down
67 changes: 2 additions & 65 deletions update-imported-docs/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,68 +1,5 @@
# Update imported docs

This script updates the docs files that are generated from other repos.
It accepts a YAML file name as its input which can be customized on a per-repo
basis.
This tool updates the reference docs generated from the Kubernetes repository.

## Requirements

Imported docs must follow these guidelines:

1. Adhere to the [Documentation Style Guide](/docs/home/contribute/style-guide/).

1. Have `title` defined in the front matter. For example:

```
---
title: Title Displayed in Table of Contents
---

Rest of the .md file...
```
1. Be listed somewhere in a file under the `data` subdirectory, for example,
the `data/imported.yml` file.

1. Make sure the `PyYAML` package is installed:

```
sudo apt-get install python-pip
pip install PyYAML
```

## Usage

From within this directory, run the following command:

```
+./update-imported-docs <CONFIG-FILE>
```

where `<CONFIG-FILE>` can be any YAML configuration file in this directory.

## Configuration file format

Each config file may contain multiple repos that will be imported together.
When necessary, you can customize the configuration file by manually editing
it. You may create new config files for importing other groups of documents.
The following is an example of the YAML configuration file:

```
repos:
- name: community
remote: https://github.com/kubernetes/community.git
branch: master
files:
- src: contributors/devel/README.md
dst: docs/imported/community/devel.md
- src: contributors/guide/README.md
dst: docs/imported/community/guide.md
```

Note: `generate-command` is an optional entry, which can be used to run a
given command or a short script to generate the docs from within a repo.

## Fixing Links

To fix relative links within your imported files, set the repo config's
`gen-absolute-links` property to `true`. You can find an example of this in
[`release.yml`](release.yml).
[Generating Reference Pages for Kubernetes Components and Tools](https://kubernetes.io/docs/contribute/generate-ref-docs/kubernetes-components/) contains detailed instructions for using this tool.