Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Geventhttpclientmergeconflicts #838

Merged
merged 31 commits into from
Sep 27, 2019
Merged
Show file tree
Hide file tree
Changes from 30 commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
9a0bc33
Started implementing a new Locust HTTP client that uses geventhttpcli…
heyman Dec 19, 2017
5de56c9
Added geventhttpclient to tests_require in setup.py. This makes the t…
heyman Dec 19, 2017
9fd6621
Added geventhttpclient to tox.ini
heyman Dec 19, 2017
0a8af7b
Python 2 fixes
heyman Dec 19, 2017
84bb988
Use timeit.default_timer() instead of time.time()
heyman Dec 19, 2017
255c76c
Added text property to Response objects returned by GeventHttpSession…
heyman Dec 19, 2017
cce8c66
Fixed wrong link target
heyman Dec 19, 2017
0351db8
Renamed GeventHttpLocust -> FasHttpLocust and GeventHttpSession -> Fa…
heyman Dec 19, 2017
f1e5761
Renamed files
heyman Dec 19, 2017
a33a4f5
Fixed import
heyman Dec 19, 2017
7fb8dd5
Renamed response class
heyman Dec 19, 2017
c5e0618
Added FastHttpSession.patch()
heyman Dec 19, 2017
5f464a1
Started writing docs for FastHttpLocust
heyman Dec 19, 2017
5116f30
Updated test that accidentally broke
heyman Dec 19, 2017
9ed5623
Minor RST syntax fix
heyman Dec 20, 2017
e8c87ac
Check that the given FastHttpLocust.host is valid and doesn't contain…
heyman Dec 20, 2017
0f0d047
Implemented support for using the catch_response argument when making…
heyman Dec 21, 2017
064f5e0
Implemented support for HTTP Basic Auth in FastHttp client.
heyman Dec 22, 2017
748ba8d
Typo
heyman Dec 22, 2017
a1a42f0
Fixed tests for manually failing an already failed request with catch…
heyman Dec 22, 2017
03d436e
Removed commented out code
heyman Dec 28, 2017
42bfffb
Clarified documentation about performance gains of FastHttpClient
heyman Dec 28, 2017
a8b8abd
Added HTTPConnectionClosed to "whitelisted" exceptions that should be…
heyman Dec 28, 2017
84b0579
Spelling typo
heyman Jan 24, 2018
f34dcac
Removed "Known limitations" (yay)
heyman Jan 24, 2018
f1dad45
Merge branch 'master' into geventhttpclient
heyman Jan 24, 2018
7055334
Added dependency on PyPI package geventhttpclient-wheels
heyman Jan 24, 2018
40aebc6
Removed geventhttpclient from tox dependencies
heyman Jan 24, 2018
a36adaf
Merge branch 'master' into geventhttpclientmergeconflicts
Jul 9, 2018
1171b9b
I was getting the assertionError 0.0 not greater than 0 -- I assumed …
Jul 9, 2018
72beb5b
Merge branch 'master' into geventhttpclientmergeconflicts
cgoldberg Apr 3, 2019
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
62 changes: 62 additions & 0 deletions docs/increase-performance.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
.. _increase-performance:

==============================================================
Increase Locust's performance with a faster HTTP client
==============================================================

Locust's default HTTP client uses `python-requests <http://www.python-requests.org/>`_.
The reason for this is that requests is a very well-maintained python package, that
provides a really nice API, that many python developers are familiar with. Therefore,
in many cases, we recommend that you use the default :py:class:`HttpLocust <locust.core.HttpLocust>`
which uses requests. However, if you're planning to run really large scale scale tests,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
which uses requests. However, if you're planning to run really large scale scale tests,
which uses requests. However, if you're planning to run really large scale tests,

Locust comes with an alternative HTTP client,
:py:class:`FastHttpLocust <locust.contrib.fasthttp.FastHttpLocust>` which
uses `geventhttpclient <https://github.com/gwik/geventhttpclient/>`_ instead of requests.
This client is significantly faster, and we've seen 5x-6x performance increases for making
HTTP-requests. This does not necessarily mean that the number of users one can simulate
per CPU core will automatically increase 5x-6x, since it also depends on what else
the load testing script does. However, if your locust scripts are spending most of their
CPU time in making HTTP-requests, you are likely to see signifant performance gains.


How to use FastHttpLocust
===========================

First, you need to install the geventhttplocust python package::

pip install geventhttpclient

Then you just subclass FastHttpLocust instead of HttpLocust::

from locust import TaskSet, task
from locust.contrib.fasthttp import FastHttpLocust

class MyTaskSet(TaskSet):
@task
def index(self):
response = self.client.get("/")

class MyLocust(FastHttpLocust):
task_set = MyTaskSet
min_wait = 1000
max_wait = 60000


.. note::

FastHttpLocust uses a whole other HTTP client implementation, with a different API, compared to
the default HttpLocust that uses python-requests. Therefore FastHttpLocust might not work as a d
rop-in replacement for HttpLocust, depending on how the HttpClient is used.


API
===

FastHttpSession class
=====================

.. autoclass:: locust.contrib.fasthttp.FastHttpSession
:members: __init__, request, get, post, delete, put, head, options, patch

.. autoclass:: locust.contrib.fasthttp.FastResponse
:members: content, text, headers
1 change: 1 addition & 0 deletions docs/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ Locust Documentation

running-locust-distributed
running-locust-without-web-ui
increase-performance

.. toctree ::
:maxdepth: 4
Expand Down
7 changes: 7 additions & 0 deletions docs/running-locust-distributed.rst
Original file line number Diff line number Diff line change
Expand Up @@ -85,3 +85,10 @@ Running Locust distributed without the web UI
=============================================

See :ref:`running-locust-distributed-without-web-ui`


Increase Locust's performance
=============================

If your planning to run large-scale load tests you might be interested to use the alternative
HTTP client that's shipped with Locust. You can read more about it here: :ref:`increase-performance`
6 changes: 4 additions & 2 deletions docs/writing-a-locustfile.rst
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@ classes. Say for example, web users are three times more likely than mobile user
The *host* attribute
--------------------

The host attribute is a URL prefix (i.e. "https://google.com") to the host that is to be loaded.
The host attribute is a URL prefix (i.e. "https://google.com") to the host that is to be loaded.
Usually, this is specified on the command line, using the :code:`--host` option, when locust is started.
If one declares a host attribute in the locust class, it will be used in the case when no :code:`--host`
is specified on the command line.
Expand Down Expand Up @@ -320,7 +320,7 @@ Since many setup and cleanup operations are dependent on each other, here is the
In general, the setup and teardown methods should be complementary.


Making HTTP requests
Making HTTP requests
=====================

So far, we've only covered the task scheduling part of a Locust user. In order to actually load test
Expand Down Expand Up @@ -403,6 +403,8 @@ Response object. The request will be reported as a failure in Locust's statistic
Response's *content* attribute will be set to None, and its *status_code* will be 0.


.. _catch-response:

Manually controlling if a request should be considered successful or a failure
------------------------------------------------------------------------------

Expand Down
Empty file added locust/contrib/__init__.py
Empty file.
Loading