Skip to main content

async/await introspection

Project description

Await, What?

Tell you what waits for what in an async/await program.

TL;DR

Say you have this code:

async def job():
    await foo()


async def foo():
    await bar()


async def bar():
    await baz()


async def baz():
    await leaf()


async def leaf():
    await asyncio.sleep(1)  # imagine you don't know this


async def work():
    await asyncio.gather(..., job())

Now that code is stuck and and you want to know why.

Python built-in

Stack for <Task pending coro=<job() > wait_for=<Future pending cb=[<TaskWakeupMethWrapper >()]> cb=[]> (most recent call last):
  File "test/test_stack.py", line 34, in job
    await foo()

This library

Stack for <Task pending coro=<job() > wait_for=<Future pending cb=[<TaskWakeupMethWrapper >()]> cb=[]> (most recent call last):
  File "test/test_stack.py", line 34, in job
    await foo()
  File "test/test_stack.py", line 38, in foo
    await bar()
  File "test/test_stack.py", line 42, in bar
    await baz()
  File "test/test_stack.py", line 46, in baz
    await leaf()
  File "test/test_stack.py", line 50, in leaf
    await asyncio.sleep(1)
  File "/…/asyncio/tasks.py", line 568, in sleep
    return await future
  File "<Sentinel>", line 0, in <_asyncio.FutureIter object at 0x7fb6981690d8>: 

Dependency Graph

References

https://mail.python.org/archives/list/async-sig@python.org/thread/6E2LRVLKYSMGEAZ7OYOYR3PMZUUYSS3K/

Hi group,

I'm recently debugging a long-running asyncio program that appears to get stuck about once a week.

The tools I've discovered so far are:

  • high level: asyncio.all_tasks() + asyncio.Task.get_stack()
  • low level: loop._selector._fd_to_key

What's missing is the middle level, i.e. stack-like linkage of what is waiting for what. For a practical example, consider:

async def leaf(): await somesocket.recv()
async def baz(): await leaf()
async def bar(): await baz()
async def foo(): await bar()
async def job(): await foo()
async def work(): await asyncio.gather(..., job())
async def main(): asyncio.run(work())

The task stack will contain:

  • main and body of work with line number
  • job task with line number pointing to foo

The file descriptor mapping, socket fd, loop._recv() and a Future.

What's missing are connections foo->bar->baz->leaf. That is, I can't tell which task is waiting for what terminal Future.

Is this problem solved in some way that I'm not aware of? Is there a library or external tool for this already?

Perhaps, if I could get a list of all pending coroutines, I could figure out what's wrong.

If no such API exists, I'm thinking of the following:

async def foo():
    await bar()

In [37]: dis.dis(foo)
  1           0 LOAD_GLOBAL              0 (bar)
              2 CALL_FUNCTION            0
              4 GET_AWAITABLE
              6 LOAD_CONST               0 (None)
              8 YIELD_FROM
             10 POP_TOP
             12 LOAD_CONST               0 (None)
             14 RETURN_VALUE

Starting from a pending task, I'd get it's coroutine and:

Get the coroutine frame, and if current instruction is YIELD_FROM, then the reference to the awaitable should be on the top of the stack. If that reference points to a pending coroutine, I'd add that to the "forward trace" and repeat.

At some point I'd reach an awaitable that's not a pending coroutine, which may be: another Task (I already got those), a low-level Future (can be looked up in event loop), an Event (tough luck, shoulda logged all Event's on creation) or a dozen other corner cases.

What do y'all think of this approach?

Thanks, D.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for awaitwhat, version 19.1a3
Filename, size File type Python version Upload date Hashes
Filename, size awaitwhat-19.1a3-cp37-cp37m-macosx_10_9_x86_64.whl (8.9 kB) File type Wheel Python version cp37 Upload date Hashes View hashes
Filename, size awaitwhat-19.1a3.tar.gz (7.0 kB) File type Source Python version None Upload date Hashes View hashes

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN SignalFx SignalFx Supporter DigiCert DigiCert EV certificate StatusPage StatusPage Status page