The Futures of Python
Futures defer execution of code. They can allow code to wait for external resources like file reads, network calls, and database access, and then automatically continue without stopping every other operation. They allow the definition (i.e. coding) of entire sequences of work in one place rather than having one function hand-off to another (presumably coordinated) function
Python’s syntax for Futures (and coroutines) is simple and, for me, surprisingly confusing at first. I have concluded that Futures in Python boil down to three (3) rules:
Functions prefixed with
async
return a Future1 when called even if the function itself does not return a result. The code inside the function does not execute until something resolves the FutureThe ways to resolve a Future are
await <Future>
asyncio.get_event_loop().run_until_complete(<Future>)
orayncio.run(<Future>)
2
Use
await
insideasync
functions only
That’s really all the mechanics
Example #
import asyncio
async def ensure_grapes() -> None:
pass
async def do_grapes_exist() -> bool:
get_grapes = ensure_grapes() # even though the function -> None
print("one second")
await get_grapes
return True
async def does_future_me_like_grapes() -> bool:
grapes_ok = await do_grapes_exist()
return grapes_ok
i_am_a_future = does_future_me_like_grapes()
assert(asyncio.iscoroutinefunction(i_am_a_future))
i_will_like_grapes : bool = asyncio.run(i_am_a_future)
Unit Tests #
To test async functions, one can descend one’s test case class from unittest.IsolatedAsycioTestCase
(instead of TestCase
) and make the testing functions async def
+ await i_am_an_async_function()
. Alternatively, one can decorate an async
test function with @pytest.mark.asyncio
Simultainous Processing #
Note that asynchronous does not mean parallel or threaded; it only means the code will run later (when resolved). One might get pseudo-parallel processing if several Futures resolve and await at the same time (e.g. on IO); Python has several libraries for these cases
Closures #
Futures are not closures: they store call-syntax only. They are more like a function pointer than a closure. They do not make a copy of the function or its environment at the moment of their creation (the call to the async
function). When resolved, the Future calls whatever object the function-name points to at that moment, and that function will use any globals or instance field values it finds at that time
Troubleshooting #
Symptom | Explanation |
---|---|
TypeError: cannot unpack a non-iterable coroutine object | Calling an async function without an await The result is a Future, and nothing else knows how to deal with a Future |
RuntimeWarning: coroutine ‘async’ was never awaited | Calling an async function without an await Even if code does not try and use the result of an async function, Python knows that the resulting Future never got resolved; something called the function and the code never executed |
TypeError: object <type> can’t be used in ‘await’ expression | Passing something that is not and does not return a Future to anawait |
SyntaxError: ‘await’ outside async function | See #3 above |
RuntimeError: cannot reuse already awaited coroutine | Futures are a pointer to a function call, not a function. Once that call resolves, it’s done |