Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Installation and offline dist improvements #1043

Merged
merged 1 commit into from
Aug 3, 2020
Merged

Installation and offline dist improvements #1043

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions RELEASE.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ Please execute the following tasks after `make release` has finished:

### Preconditions

* This procedure requires a Linux workstation as the offline installation references platform specific artifacts. A Vagrant box will work fine.
* Ensure that the master branch is checked out and your working copy is clean (run `git status`).
* Ensure that integration tests can run to completion successfully on your machine (run `make it`).
* Ensure that the associated milestone on Github contains no open tickets (otherwise the release will fail).
Expand Down
32 changes: 9 additions & 23 deletions docs/install.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,9 +33,8 @@ Once ``pyenv`` is installed, install a compatible Python version::
# see https://github.com/pyenv/pyenv/blob/master/COMMANDS.md#pyenv-global for details
pyenv global 3.8.0

# Install pip3
curl -s https://bootstrap.pypa.io/get-pip.py -o get-pip.py
python3 get-pip.py --user
# Upgrade pip
python3 -m pip install --user --upgrade pip

git
~~~
Expand Down Expand Up @@ -113,32 +112,19 @@ To find the JDK, Rally expects the environment variable ``JAVA_HOME`` to be set
Installing Rally
----------------

Simply install Rally with pip: ``pip3 install esrally``

.. note::

Depending on your system setup you may need to prepend this command with ``sudo``.
1. Ensure ``~/.local/bin`` is in your ``$PATH``.
2. Install Rally: ``python3 -m pip install --user esrally``.
Copy link
Contributor Author

@dliappis dliappis Jul 30, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alternatively --user can be skipped and then esrally lives (in the form of a pyenv shim) in ~/.pyenv/shims/esrally. In this case the user would still need to either logout/login or exec $SHELL to make the shim active.

Since we are anyway upgrading pip (earlier) in a --user context, I think we should stick to that convention, which IMHO is a better practice, cleaner approach and ultimately a more discoverable location (~/.local/bin) for binaries to be stored. The requirement to have ~/.local/bin is also IMHO a best practice that users should follow.

All these instructions have been tested -- at least -- in CentOS 8 and Ubuntu 18.04. More testing very welcome.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, let's keep this as you've done it here.

Copy link
Contributor

@ebadyano ebadyano Jul 31, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for some reason python3 -m pip install --user esrally keeps installing 1.4.1 rally for me on Linux SUSE

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh i think that's because python version is 3.6 not 3.8


If you get errors during installation, it is probably due to the installation of ``psutil`` which we use to gather system metrics like CPU utilization. Ensure that you have installed the Python development package as documented in the prerequisites section above.

Non-sudo Install
----------------

If you don't want to use ``sudo`` when installing Rally, installation is still possible but a little more involved:

1. Specify the ``--user`` option when installing Rally (step 2 above), so the command to be issued is: ``python3 setup.py develop --user``.
2. Check the output of the install script or lookup the `Python documentation on the variable site.USER_BASE <https://docs.python.org/3/library/site.html#site.USER_BASE>`_ to find out where the script is located. On Linux, this is typically ``~/.local/bin``.

You can now either add ``~/.local/bin`` to your path or invoke Rally via ``~/.local/bin/esrally`` instead of just ``esrally``.

VirtualEnv Install
------------------

You can also use Virtualenv to install Rally into an isolated Python environment without sudo.

1. Set up a new virtualenv environment in a directory with ``virtualenv --python=python3 .``
1. Set up a new virtualenv environment in a directory with ``python3 -m venv .``
2. Activate the environment with ``source /path/to/virtualenv/dir/bin/activate``
3. Install Rally with ``pip install esrally``
3. Install Rally with ``python3 -m pip install esrally``

Whenever you want to use Rally, run the activation script (step 2 above) first. When you are done, simply execute ``deactivate`` in the shell to exit the virtual environment.

Expand All @@ -161,12 +147,12 @@ Offline Install
This documentation is for the version of Rally currently under development. We do not provide offline installation packages for development versions.
Were you looking for the `documentation of the latest stable version <//esrally.readthedocs.io/en/stable/>`_?

If you are in a corporate environment where your servers do not have any access to the Internet, you can use Rally's offline installation package. Follow these steps to install Rally:
If you are in a corporate environment using Linux servers that do not have any access to the Internet, you can use Rally's offline installation package. Follow these steps to install Rally:

1. Install all prerequisites as documented above.
2. Download the offline installation package for the `latest release <https://github.com/elastic/rally/releases/latest>`_ and copy it to the target machine(s).
3. Decompress the installation package with ``tar -xzf esrally-dist-*.tar.gz``.
4. Run the install script with ``sudo ./esrally-dist-*/install.sh``.
3. Decompress the installation package with ``tar -xzf esrally-dist-linux-*.tar.gz``.
4. Run the install script with ``sudo ./esrally-dist-linux-*/install.sh``.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The linux- portion of the file is not present on the release link above.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, should this be offline-install.sh ? Ahh it looks like we sed in teh values in the offline-install for the release version, and then its just install.sh.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The linux- portion of the file is not present on the release link above.

(If I understand the question correctly): Yeah it's not present now, but it will be after we release in the future due the changes here


Next Steps
----------
Expand Down
14 changes: 8 additions & 6 deletions release-checks.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,14 @@
set -eu
RELEASE_VERSION=$1

KERNEL_NAME=$(uname -s)
if [[ ${KERNEL_NAME} != *"Linux"* ]]
then
echo "Error: release needs to be run on a Linux workstation but you are running on ${KERNEL_NAME}."
echo "Switch to a Linux workstation and try again."
exit 1
fi

# test number of parameters
if [[ $# != 2 ]]
then
Expand All @@ -44,12 +52,6 @@ then
exit 1
fi

if [[ $(uname) == "Darwin" && -z "${GPG_TTY+set}" ]]
then
echo "Error: to allow git to create signed commits on Mac OS you need to set \"export GPG_TTY=\$(tty)\"."
exit 1
fi

ORIGIN_URL=$(git remote get-url --push origin)
if [[ ${ORIGIN_URL} != *"elastic/rally"* ]]
then
Expand Down
16 changes: 13 additions & 3 deletions scripts/offline-install.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,12 +31,22 @@ then
exit 1
fi

KERNEL_NAME=$(uname -s)
if [[ ${KERNEL_NAME} != *"Linux"* ]]
then
echo "Error: this script needs to be run on a Linux workstation but you are running on ${KERNEL_NAME}."
echo "Switch to a Linux workstation and try again."
exit 1
fi

readonly RALLY_VERSION=$1

readonly WD=$(pwd)
readonly RELATIVE_DOWNLOAD_DIR="esrally-dist-${RALLY_VERSION}"
readonly ABSOLUTE_DOWNLOAD_DIR="${WD}/${RELATIVE_DOWNLOAD_DIR}"
readonly ABSOLUTE_DOWNLOAD_BIN_DIR="${ABSOLUTE_DOWNLOAD_DIR}/bin"
readonly PYTHON_INSTALL_LINK="https://esrally.readthedocs.io/en/stable/install.html#python"
readonly PYTHON_ERROR_MSG="is required but not installed. Follow the instructions in ${PYTHON_INSTALL_LINK} and try again."

SOURCE="${BASH_SOURCE[0]}"
while [ -h "$SOURCE" ]; do # resolve $SOURCE until the file is no longer a symlink
Expand All @@ -47,7 +57,7 @@ done
SCRIPT_SRC_HOME="$( cd -P "$( dirname "$SOURCE" )" && pwd )"

function main {
local archive_name="esrally-dist-${RALLY_VERSION}.tar.gz"
local archive_name="esrally-dist-linux-${RALLY_VERSION}.tar.gz"
local install_script_file="install.sh"
local install_script="${ABSOLUTE_DOWNLOAD_DIR}/${install_script_file}"

Expand Down Expand Up @@ -84,8 +94,8 @@ SRC_HOME="\$( cd -P "\$( dirname "\$SOURCE" )" && pwd )"
echo "Installing Rally ${RALLY_VERSION}..."

# Check if mandatory prerequisites are installed
command -v python3 >/dev/null 2>&1 || { echo >&2 "Python3 is required but not installed."; exit 1; }
command -v pip3 >/dev/null 2>&1 || { echo >&2 "pip3 is required but not installed."; exit 1; }
command -v python3 >/dev/null 2>&1 || { echo >&2 "Python3 ${PYTHON_ERROR_MSG}"; exit 1; }
command -v pip3 >/dev/null 2>&1 || { echo >&2 "pip3 ${PYTHON_ERROR_MSG}"; exit 1; }

pip3 install esrally==${RALLY_VERSION} --no-index --find-links file://\${SRC_HOME}/bin
EOL
Expand Down