Working with TCP sockets in Python 2.x with select
OUR Blog
Python team lead
Alexander Stepanov
Python team lead
Mar 01 2016

Working with TCP sockets in Python 2.x with select

A network socket is an endpoint of an interprocess communication across a computer network. The Python Standard Library has a module called socket which provides a low-level internet networking interface. This interface is common across different programming languages since it uses OS-level system calls.

To create a socket, there is a function called socket. It accepts family, type, and proto arguments (see documentation for details). To create a TCP-socket, you should use socket.AF_INET or socket.AF_INET6 for family and socket.SOCK_STREAM for type:

import socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

It returns a socket object which has the following main methods:

  • bind()
  • listen()
  • accept()
  • connect()
  • send()
  • recv()

bind(), listen() and accept() are specific for server sockets. connect() is specific for client sockets. send() and recv() are common for both types. Here is an example of Echo server from documentation:

import socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind(('localhost', 50000))
conn, addr = s.accept()
while 1:
    data = conn.recv(1024)
    if not data:

Here we create a server socket, bind it to a localhost and 50000 port, and start listening for incoming connections. To accept an incoming connection we call accept() method which will block until a new client connects. When this happens, it creates a new socket and returns it together with the client's address. Then, in an infinite cycle, it reads data from the socket in batches of 1024 bytes using method recv() until it returns an empty string. After that, it sends all incoming data back using a convenient method sendall() which inside repeatedly calls recv(). And after that it simply closes the client's connection. This example can serve only one incoming connection because it does not call accept() in a cycle.

A client-side code looks simplier:

import socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(('localhost', 50000))
s.sendall('Hello, world')
data = s.recv(1024)
print 'Received', repr(data)

Here instead of bind() and listen() it calls only connect() and immediately sends data to the server. Then it receives 1024 bytes back, closes the socket, and prints the received data.

All socket methods are blocking. For example, when it reads from a socket or writes to it the program can't do anything else. One possible solution is to delegate working with clients to separate threads. However, creating threads and switching contexts between them is not really a cheap operation. To address this problem, there is a so-called asynchronous way of working with sockets. The main idea is to delegate maintaining the socket's state to an operating system and letting it notify the program when there is something to read from the socket or when it is ready for writing.

There are a bunch of interfaces for different operating systems:

  • poll, epoll (linux)
  • kqueue, kevent (BSD)
  • select (crossplatform)

They are all about the same so let’s create a server using select:

import select, socket, sys, Queue
server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.bind(('localhost', 50000))
inputs = [server]
outputs = []
message_queues = {}

while inputs:
    readable, writable, exceptional =
        inputs, outputs, inputs)
    for s in readable:
        if s is server:
            connection, client_address = s.accept()
            message_queues[connection] = Queue.Queue()
            data = s.recv(1024)
            if data:
                if s not in outputs:
                if s in outputs:
                del message_queues[s]

    for s in writable:
            next_msg = message_queues[s].get_nowait()
        except Queue.Empty:

    for s in exceptional:
        if s in outputs:
        del message_queues[s]

As you can see, there is much more code than in the blocking Echo server. That is primarily because we have to maintain a set of queues for different lists of sockets, i.e. writing, reading, and a separate list for erroneous sockets.

Creating server socket looks the same except for one line: server.setblocking(0). This is done to make the socket nonblocking. This server is more advanced since it can serve more than one client. The main point is in selecting sockets:

readable, writable, exceptional =
    inputs, outputs, inputs)

Here we call to ask the OS to check given sockets whether they are ready to write, read, or if there is some exception respectively. That is why it passes three lists of sockets to specify which socket is expected to be writable, readable, and which should be checked for errors. This call will block the program (unless a timeout argument is passed) until some of the passed sockets are ready. In this moment, the call will return three lists with sockets for specified operations.

Then it sequentially iterates over those lists and, if there are sockets in them, it performs corresponding operations. When there is the server socket in inputs, it means that a new client has arrived. Therefore, it calls accept(), adds a returned socket to inputs and adds a Queue for incoming messages which will be sent back. If there is another socket in inputs, then some messages have arrived and ready to be read so it reads them and places them into the corresponding queue.

For writable sockets, it gets pending messages (if any) and writes them to the socket. If there is any error in the socket, it removes the socket from the lists.

This is how sockets work at a lower level. However, in most cases, there is no need to implement the logic at such a low level. It is recommended to use some higher level abstractions such as Twisted, Tornado, or ZeroMQ, depending on the situation.

Sep 29 2016
This article will tell you why you need to keep Changelog. The history of the changelog management process on the example of python / django project using git instruments.
Aug 01 2016
Brief outline of the advantages of Python as a coding language and why we choose to work with Django, what we consider to be the best web framework for your project.
Jul 14 2016
This repo contains examples from article Emacs configuration for working with Python. It collects information for creating a usable Python programming environment in Emacs. Emacs already has out-of-the-box Python support via ‘python-mode’. The settings contained in this repo can be used to set up a more complete environment quickly.