Load testing with Python. Locust for testing and Bokeh for visualization
OUR Blog
Python developer
Artem Antsyferov
Python developer
PYTHON/DJANGO
Jun 09 2016

Load testing with Python. Locust for testing and Bokeh for visualization

Scalability testing is an important part of getting web service production ready. There is a lot of tools for load testing, like Gatling, Apache JMeter, The Grinder, Tsung and others. There is also one (and my favorite) written in Python and built on the Requests library: Locust.

As it is noticed on Locust website:

A fundamental feature of Locust is that you describe all your test in Python code. No need for clunky UIs or bloated XML, just plain code.

Locust installation

Locust requires Python 2.6+. It is not currently compatible with Python 3.x. Locust is available on PyPI and can be installed through pip or easy_install

pip install locustio or: easy_install locust

Example locustfile.py

Then create locustfile.py following the example from docs. To test Django project I had to add some headers for csrftoken support and ajax requests. Final locustfile.py could be something like the following:

# locustfile.py
from locust import HttpLocust, TaskSet, task


class UserBehavior(TaskSet):

    def on_start(self):
        self.login()

    def login(self):
        # GET login page to get csrftoken from it
        response = self.client.get('/accounts/login/')
        csrftoken = response.cookies['csrftoken']
        # POST to login page with csrftoken
        self.client.post('/accounts/login/',
                         {'username': 'username', 'password': 'P455w0rd'},
                         headers={'X-CSRFToken': csrftoken})

    @task(1)
    def index(self):
        self.client.get('/')

    @task(2)
    def heavy_url(self):
        self.client.get('/heavy_url/')

    @task(2)
    def another_heavy_ajax_url(self):
        # ajax GET
        self.client.get('/another_heavy_ajax_url/',
        headers={'X-Requested-With': 'XMLHttpRequest'})


class WebsiteUser(HttpLocust):
    task_set = UserBehavior

Start Locust

To run Locust with the above locust file, if it was named locustfile.py, we could run (in the same directory as locustfile.py):

locust --host=http://example.com

When Locust is started you should visit http://127.0.0.1:8089/ and there you'll find web-interface of our Locust instance. Then input Number of users to simulate (e.g. 300) and Hatch rate (users spawned/second) (e.g. 10) and press Start swarming. After that Locust will start "hatching" users and you will be able to see results in the table.

Visualization

So, the table is nice but we'd prefer to see results on a graph. There is an issue in which people ask to add graphical interface to Locust and there are several propositions how to display graphs for Locust data. I've decided to use Python interactive visualization library Bokeh.

It is easy to install Bokeh from PyPI using pip:

pip install booked

Here is an example of running Bokeh server.

We can get Locust data in JSON format visiting http://localhost:8089/stats/requests. Data there should be something like this:

    {
       "errors": [],
       "stats": [
           {
               "median_response_time": 350,
               "min_response_time": 311,
               "current_rps": 0.0,
               "name": "/",
               "num_failures": 0,
               "max_response_time": 806,
               "avg_content_length": 17611,
               "avg_response_time": 488.3333333333333,
               "method": "GET",
               "num_requests": 9
           },
           {
               "median_response_time": 350,
               "min_response_time": 311,
               "current_rps": 0.0,
               "name": "Total",
               "num_failures": 0,
               "max_response_time": 806,
               "avg_content_length": 17611,
               "avg_response_time": 488.3333333333333,
               "method": null,
               "num_requests": 9
           }
       ],
       "fail_ratio": 0.0,
       "slave_count": 2,
       "state": "stopped",
       "user_count": 0,
       "total_rps": 0.0
    }

To display this data on plots we'll create plotter.py file and put it to the directory in which our locustfile.py is:

    # plotter.py
    import requests
    import six

    from bokeh.client import push_session
    from bokeh.io import hplot
    from bokeh.plotting import figure, curdoc


    # here we'll keep configurations
    config = {'figures': [{'charts':
                               [{'color': 'black', 'legend': 'average response time', 'marker': 'diamond',
                                 'key': 'avg_response_time'},
                                {'color': 'blue', 'legend': 'median response time', 'marker': 'triangle',
                                 'key': 'median_response_time'},
                                {'color': 'green', 'legend': 'min response time', 'marker': 'inverted_triangle',
                                 'key': 'min_response_time'},
                                {'color': 'red', 'legend': 'max response time', 'marker': 'circle',
                                 'key': 'max_response_time'}],
                           'xlabel': 'Requests count',
                           'ylabel': 'Milliseconds',
                           'title': '{} response times'
                           },
                          {'charts': [{'color': 'green', 'legend': 'current rps', 'marker': 'circle',
                                       'key': 'current_rps'},
                                      {'color': 'red', 'legend': 'failures', 'marker': 'cross',
                                       'key': 'num_failures', 'skip_null': True}],
                           'xlabel': 'Requests count',
                           'ylabel': 'RPS/Failures count',
                           'title': '{} RPS/Failures'
                           }],
              'url': 'http://localhost:8089/stats/requests',  # locust json stats url
              'states': ['hatching', 'running'],  # locust states for which we'll plot the graphs
              'requests_key': 'num_requests'
              }

    data_sources = {}  # dict with data sources for our figures

    for state in config['states']:
        data_sources[state] = {}  # dict with data sources for figures for each state
        figures = []  # list of figures for each state
        for figure_data in config['figures']:
            # initialization of figure
            new_figure = figure(title=figure_data['title'].format(state.capitalize()))
            new_figure.xaxis.axis_label = figure_data['xlabel']
            new_figure.yaxis.axis_label = figure_data['ylabel']
            # adding charts to figure
            for chart in figure_data['charts']:
                # adding both markers and line for chart
                marker = getattr(new_figure, chart['marker'])
                scatter = marker(x=[0], y=[0], color=chart['color'], size=10, legend=chart['legend'])
                line = new_figure.line(x=[0], y=[0], color=chart['color'], line_width=1, legend=chart['legend'])
                # adding data source for markers and line
                data_sources[state][chart['key']] = scatter.data_source = line.data_source
            figures.append(new_figure)
        hplot(*figures)  # create a row with 2 figures for each state

    requests_key = config['requests_key']
    url = config['url']


    # Next line opens a new session with the Bokeh Server, initializing it with our current Document.
    # This local Document will be automatically kept in sync with the server.
    session = push_session(curdoc())


    # The next few lines define and add a periodic callback to be run every 1 second:
    def update():
        try:
            resp = requests.get(url)
        except requests.RequestException:
            return
        resp_data = resp.json()
        data = resp_data['stats'][-1]  # Getting "Total" data from locust
        if resp_data['state'] in config['states']:
            for key, data_source in six.iteritems(data_sources[resp_data['state']]):
                # adding data from locust to data_source of our graphs
                data_source.data['x'].append(data[requests_key])
                data_source.data['y'].append(data[key])
                # trigger data source changes
                data_source.trigger('data', data_source.data, data_source.data)

    curdoc().add_periodic_callback(update, 1000)

    session.show()  # open the document in a browser

    session.loop_until_closed()  # run forever

Running all together

So our Locust is running (if no start it with locust --host=http://example.com) and now we should start Bokeh server with bokeh serve and then run our plotter.py with python plotter.py. As our script calls show, a browser tab is automatically opened up to the correct URL to view the document.

If Locust is already running the test you'll see the results on graphs immediately. Else start a new test at http://localhost:8089/ and return to the Bokeh tab and watch the results of testing in real time.

That's it. You can find the whole code at github.

Feel free to clone it and run the example. Don't forget to use Python 2.6+ (Locust is not Python 3.x compatible for now):

  git clone https://github.com/steelkiwi/locust-bokeh-load-test.git
  cd locust-bokeh-load-test
  pip install -r requirements.txt
  locust --host=<place here link to your site>
  bokeh serve
  python plotter.py

You should have Bokeh tab opened in browser after running these commands. Now visit http://localhost:8089/ and start test there. Return to Bokeh tab and enjoy the graphs.

Useful links

  1. Requests: HTTP for Humans
  2. Locust - a modern load testing framework
  3. Locust Documentation - everything you need to know about Locust
  4. All about Bokeh
SIMILAR POSTS
Nov 29 2016
Create a project template with Vagrant, VirtualEnv and Ansible provisioner. And why is it necessary?
Nov 04 2016
Review of push notifications services and their functions. Read how to use push notifications effectively and make your app user-friendly in our article!
Sep 29 2016
This article will tell you why you need to keep Changelog. The history of the changelog management process on the example of python / django project using git instruments.