The Futures of Python

Futures defer execution of code. They can allow code to wait for external resources like file reads, network calls, and database access, and then automatically continue without stopping every other operation. They allow the definition (i.e. coding) of entire sequences of work in one place rather than having one function hand-off to another (presumably coordinated) function

Python’s syntax for Futures (and coroutines) is simple and, for me, surprisingly confusing at first. I have concluded that Futures in Python boil down to three (3) rules:

  1. Functions prefixed with async return a Future1 when called even if the function itself does not return a result. The code inside the function does not execute until something resolves the Future

  2. The ways to resolve a Future are

    • await <Future>
    • asyncio.get_event_loop().run_until_complete(<Future>) or ayncio.run(<Future>)2
  3. Use await inside async functions only

That’s really all the mechanics

Example #

import asyncio

async def ensure_grapes() -> None:
    pass

async def do_grapes_exist() -> bool:
    get_grapes =  ensure_grapes()  # even though the function -> None
    print("one second")
    await get_grapes

    return True

async def does_future_me_like_grapes() -> bool:
    grapes_ok = await do_grapes_exist()

    return grapes_ok

i_am_a_future = does_future_me_like_grapes()
assert(asyncio.iscoroutinefunction(i_am_a_future))

i_will_like_grapes : bool =  asyncio.run(i_am_a_future)

Unit Tests #

To test async functions, one can descend one’s test case class from unittest.IsolatedAsycioTestCase (instead of TestCase) and make the testing functions async def + await i_am_an_async_function(). Alternatively, one can decorate an async test function with @pytest.mark.asyncio

Simultainous Processing #

Note that asynchronous does not mean parallel or threaded; it only means the code will run later (when resolved). One might get pseudo-parallel processing if several Futures resolve and await at the same time (e.g. on IO); Python has several libraries for these cases

Closures #

Futures are not closures: they store call-syntax only. They are more like a function pointer than a closure. They do not make a copy of the function or its environment at the moment of their creation (the call to the async function). When resolved, the Future calls whatever object the function-name points to at that moment, and that function will use any globals or instance field values it finds at that time

Troubleshooting #

Symptom Explanation
TypeError: cannot unpack a non-iterable coroutine object Calling an async function without an await
The result is a Future, and nothing else knows how to deal with a Future
RuntimeWarning: coroutine ‘async’ was never awaited Calling an async function without an await
Even if code does not try and use the result of an async function, Python knows that the resulting Future never got resolved; something called the function and the code never executed
TypeError: object <type> can’t be used in ‘await’ expression Passing something that is not and does not return a Future to anawait
SyntaxError: ‘await’ outside async function See #3 above
RuntimeError: cannot reuse already awaited coroutine Futures are a pointer to a function call, not a function. Once that call resolves, it’s done

  1. yes, it is really a coroutine, but, as noted in the documentation, it uses this term inconsistently 

  2. which can have awkward side-effects if more than one strand of code uses this 

 
4
Kudos
 
4
Kudos

Now read this

Technical Debt Is Cholera, not Quicksand

Technical debt (TD) – the compromises we make to get stuff out the door – is more often acknowledged and occasionally quantified. We categorize the inefficiency and difficulty working with it—the “interest” on the debt—as an unfortunate... Continue →