Asynchronous Programming¶

Introduction¶

The asynchronous programming approach¶

Autobahn is written according to a programming paradigm called asynchronous programming (or event driven programming) and implemented using non-blocking execution - and both go hand in hand.

A very good technical introduction to these concepts can be found in this chapter of an “Introduction to Asynchronous Programming and Twisted”.

Here are two more presentations that introduce event-driven programming in Python

Another highly recommended reading is The Reactive Manifesto which describes guiding principles, motivations and connects the dots

Non-blocking means the ability to make continuous progress in order for the application to be responsive at all times, even under failure and burst scenarios. For this all resources needed for a response—for example CPU, memory and network—must not be monopolized. As such it can enable both lower latency, higher throughput and better scalability.

—The Reactive Manifesto

The fact that Autobahn is implemented using asynchronous programming and non-blocking execution shouldn’t come as a surprise, since both Twisted and asyncio - the foundations upon which Autobahn runs - are asynchronous network programming frameworks.

On the other hand, the principles of asynchronous programming are independent of Twisted and asyncio. For example, other frameworks that fall into the same category are:

Tip

While getting accustomed to the asynchronous way of thinking takes some time and effort, the knowledge and experience acquired can be translated more or less directly to other frameworks in the asynchronous category.

Other forms of Concurrency¶

Asynchronous programming is not the only approach to concurrency. Other styles of concurrency include

Obviously, we cannot go into much detail with all of above. But here are some pointers for further reading if you want to compare and contrast asynchronous programming with other approaches.

With the Actor model a system is composed of a set of actors which are independently running, executing sequentially and communicate strictly by message passing. There is no shared state at all. This approach is used in systems like

Software Transactional Memory (STM) applies the concept of Optimistic Concurrency Control from the persistent database world to (transient) program memory. Instead of lettings programs directly modify memory, all operations are first logged (inside a transaction), and then applied atomically - but only if no conflicting transaction has committed in the meantime. Hence, it’s “optimistic” in that it assumes to be able to commit “normally”, but needs to handle the failing at commit time.

Green Threads is using light-weight, run-time level threads and thread scheduling instead of OS threads. Other than that, systems are implemented similar: green threads still block, and still do share state. Python has multiple efforts in this category:

Twisted or asyncio?¶

Since Autobahn runs on both Twisted and asyncio, which networking framework should you use?

Even more so, as the core of Twisted and asyncio is very similar and relies on the same concepts:

Twisted	asyncio	Description
Deferred	Future	abstraction of a value which isn’t available yet
Reactor	Event Loop	waits for and dispatches events
Transport	Transport	abstraction of a communication channel (stream or datagram)
Protocol	Protocol	this is where actual networking protocols are implemented
Protocol Factory	Protocol Factory	responsible for creating protocol instances

In fact, I’d say the biggest difference between Twisted and asyncio is Deferred vs Future. Although similar on surface, their semantics are different. Deferred supports the concept of chainable callbacks (which can mutate the return values), and separate error-backs (which can cancel errors). Future has just a callback, that always gets a single argument: the Future.

Also, asyncio is opinionated towards co-routines. This means idiomatic user code for asyncio is expected to use co-routines, and not plain Futures (which are considered too low-level for application code).

But anyway, with asyncio being part of the language standard library (since Python 3.4), wouldn’t you just always use asyncio? At least if you don’t have a need to support already existing Twisted based code.

The truth is that while the core of Twisted and asyncio are very similar, Twisted has a much broader scope: Twisted is “batteries included” for network programming.

So you get tons of actual network protocols already out-of-the-box - in production quality implementations!

asyncio does not include any actual application layer network protocols like HTTP. If you need those, you’ll have to look for asyncio implementations outside the standard library. For example, here is a HTTP server and client library for asyncio.

Over time, an ecosystem of protocols will likely emerge around asyncio also. But right now, Twisted has a big advantage here.

If you want to read more on this, Glyph (Twisted original author) has a nice blog post here.

Resources¶

Below we are listing a couple of resources on the Web for Twisted and asyncio that you may find useful.

Twisted Resources¶

We cannot give an introduction to asynchronous programming with Twisted here. And there is no need to, since there is lots of great stuff on the Web. In particular we’d like to recommend the following resources.

If you have limited time and nevertheless want to have an in-depth view of Twisted, Jessica McKellar has a great presentation recording with Architecting an event-driven networking engine: Twisted Python. That’s 45 minutes. Highly recommended.

If you really want to get it, Dave Peticolas has written an awesome Introduction to Asynchronous Programming and Twisted. This is a detailed, hands-on tutorial with lots of code examples that will take some time to work through - but you actually learn how to program with Twisted.

Then of course there is

and lots and lots of awesome Twisted talks on PyVideo.

Asyncio Resources¶

asyncio is very new (August 2014). So the amount of material on the Web is still limited. Here are some resources you may find useful:

However, we quickly introduce core asynchronous programming primitives provided by Twisted and asyncio:

Asynchronous Programming Primitives¶

In this section, we have a quick look at some of the asynchronous programming primitive provided by Twisted and asyncio to show similarities and differences.

Twisted Deferreds and inlineCallbacks¶

Documentation pointers:

Programming with Twisted Deferreds involves attaching callbacks to Deferreds which get called when the Deferred finally either resolves successfully or fails with an error

d = some_function() # returns a Twisted Deferred ..

def on_success(res):
   print("result: {}".format(res))

def on_error(err):
   print("error: {}".format(err))

d.addCallbacks(on_success, on_error)

Using Deferreds offers the greatest flexibility since you are able to pass around Deferreds freely and can run code concurrently.

However, using plain Deferreds comes at a price: code in this style looks very different from synchronous/blocking code and the code can become hard to follow.

Now, Twisted inlineCallbacks let you write code in a sequential looking manner that nevertheless executes asynchronously and non-blocking under the hood.

So converting above snipped to inlineCallbacks the code will look like

try:
   res = yield some_function()
   print("result: {}".format(res))
except Exception as err:
   print("error: {}".format(err))

As you can see, this code looks very similar to regular synchronous/blocking Python code. The only difference (on surface) is the use of yield when calling a function that runs asynchronously. Otherwise, you process success result values and exceptions exactly as with regular code.

Note

We’ll only show basic usage here - for a more basic and complete introduction, please have a look at this chapter from this tutorial.

Example

The following demonstrates basic usage of inlineCallbacks in a complete example you can run.

First, consider this program using Deferreds. We simulate calling a slow function by sleeping (without blocking) inside the function slow_square

from twisted.internet import reactor
from twisted.internet.defer import Deferred

def slow_square(x):
   d = Deferred()

   def resolve():
      d.callback(x * x)

   reactor.callLater(1, resolve)
   return d

def test():
   d = slow_square(3)

   def on_success(res):
      print(res)
      reactor.stop()

   d.addCallback(on_success)

test()
reactor.run()

This is just regular Twisted code - nothing exciting here:

We create a Deferred to be returned by our slow_square function (line 5)
We create a function resolve (a closure) in which we resolve the previously created Deferred with the result (lines 7-8)
Then we ask the Twisted reactor to call resolve after 1 second (line 10)
And we return the previously created Deferred to the caller (line 11)

What you can see even with this trivial example already is that the code looks quite differently from synchronous/blocking code. It needs some practice until such code becomes natural to read.

Now, when converted to inlineCallbacks, the code becomes:

from twisted.internet import reactor
from twisted.internet.defer import inlineCallbacks, returnValue
from autobahn.twisted.util import sleep

@inlineCallbacks
def slow_square(x):
   yield sleep(1)
   returnValue(x * x)

@inlineCallbacks
def test():
   res = yield slow_square(3)
   print(res)
   reactor.stop()

test()
reactor.run()

Have a look at the highlighted lines - here is what we do:

Decorating our squaring function with inlineCallbacks (line 5). Doing so marks the function as a coroutine which allows us to use this sequential looking coding style.
Inside the function, we simulate the slow execution by sleeping for a second (line 7). However, we are sleeping in a non-blocking way (autobahn.twisted.util.sleep). The yield will put the coroutine aside until the sleep returns.
To return values from Twisted coroutines, we need to use returnValue (line 8).

Note

The reason returnValue is necessary goes deep into implementation details of Twisted and Python. In short: co-routines in Python 2 with Twisted are simulated using exceptions. Only Python 3.3+ has gotten native support for co-routines using the new yield from statement, Python 3.5+ use await statement and it is the new recommended method.

In above, we are using a little helper autobahn.twisted.util.sleep for sleeping “inline”. The helper is really trivial:

from twisted.internet import reactor
from twisted.internet.defer import Deferred

def sleep(delay):
   d = Deferred()
   reactor.callLater(delay, d.callback, None)
   return d

The rest of the program is just for driving our test function and running a Twisted reactor.

Asyncio Futures and Coroutines¶

Asyncio Futures like Twisted Deferreds encapsulate the result of a future computation. At the time of creation, the result is (usually) not yet available, and will only be available eventually.

On the other hand, asyncio futures are quite different from Twisted Deferreds. One difference is that they have no built-in machinery for chaining.

Asyncio Coroutines are (on a certain level) quite similar to Twisted inline callbacks. Here is the code corresponding to our example above:

Example

The following demonstrates basic usage of asyncio.coroutine in a complete example you can run.

First, consider this program using plain asyncio.Future. We simulate calling a slow function by sleeping (without blocking) inside the function slow_square

import asyncio

def slow_square(x):
   f = asyncio.Future()

   def resolve():
      f.set_result(x * x)

   loop = asyncio.get_event_loop()
   loop.call_later(1, resolve)

   return f

def test():
   f = slow_square(3)

   def done(f):
      res = f.result()
      print(res)

   f.add_done_callback(done)

   return f

loop = asyncio.get_event_loop()
loop.run_until_complete(test())
loop.close()

Using asyncio in this way is probably quite unusual. This is because asyncio is opinionated towards using coroutines from the beginning. Anyway, here is what above code does:

We create a Future to be returned by our slow_square function (line 4)
We create a function resolve (a closure) in which we resolve the previously created Future with the result (lines 6-7)
Then we ask the asyncio event loop to call resolve after 1 second (line 10)
And we return the previously created Future to the caller (line 12)

What you can see even with this trivial example already is that the code looks quite differently from synchronous/blocking code. It needs some practice until such code becomes natural to read.

Now, when converted to asyncio.coroutine, the code becomes:

import asyncio

async def slow_square(x):
   await asyncio.sleep(1)
   return x * x


async def test():
   res = await slow_square(3)
   print(res)

loop = asyncio.get_event_loop()
loop.run_until_complete(test())

The main differences (on surface) are:

The declaration of the function with async keyword (line 3) in asyncio versus the decorator @defer.inlineCallbacks with Twisted
The use of defer.returnValue in Twisted for returning values whereas in asyncio, you can use plain returns (line 6)
The use of await in asyncio, versus yield in Twisted (line 5)
The auxiliary code to get the event loop started and stopped

Most of the examples that follow will show code for both Twisted and asyncio, unless the conversion is trivial.