async/await introspection
Project description
Await, What?
Tell you what waits for what in an async/await
program.
Alpine
You'll need apk add build-base openssl-dev libffi-dev
Sprint Setup
Comms: https://gitter.im/awaitwhat/community
- Python 3.9, Python 3.8 (preferred) or Python 3.7
- Your platform dev tools (compiler, etc).
- Ensure that
python
is 3.9 or 3.8 or 3.7 - Install
poetry
- Install
graphviz
- Clone this repository
- Look at tests
- Look at issues
> python --version
Python 3.9.0b4 #🧡
Python 3.8.4 #👌
> dot -V
dot - graphviz version 2.40.1
> curl -sSL https://raw.githubusercontent.com/sdispater/poetry/master/get-poetry.py | python
# add ~/.poetry/bin to your PATH
> git clone git@github.com:dimaqq/awaitwhat.git
> cd awaitwhat
~/awaitwhat (dev|✔) > poetry shell # creates a venv and drops you in it
(awaitwhat-x-py3.9) ~/awaitwhat (dev|✔) > poetry install # installs projects dependencies in a venv
(awaitwhat-x-py3.9) ~/awaitwhat (dev|✔) > poetry build # builds a C extension in this project
(awaitwhat-x-py3.9) ~/awaitwhat (dev|✔) > env PYTHONPATH=. python examples/test_shield.py | tee graph.dot
(awaitwhat-x-py3.9) ~/awaitwhat (dev|✔) > dot -Tsvg graph.dot -o graph.svg
(awaitwhat-x-py3.9) ~/awaitwhat (dev|✔) > open graph.svg # or load it in a browser
TL;DR
Say you have this code:
async def job():
await foo()
async def foo():
await bar()
async def bar():
await baz()
async def baz():
await leaf()
async def leaf():
await asyncio.sleep(1) # imagine you don't know this
async def work():
await asyncio.gather(..., job())
Now that code is stuck and and you want to know why.
Python built-in
Stack for <Task pending coro=<job() …> wait_for=<Future pending cb=[<TaskWakeupMethWrapper …>()]> cb=[…]> (most recent call last):
File "test/test_stack.py", line 34, in job
await foo()
This library
Stack for <Task pending coro=<job() …> wait_for=<Future pending cb=[<TaskWakeupMethWrapper …>()]> cb=[…]> (most recent call last):
File "test/test_stack.py", line 34, in job
await foo()
File "test/test_stack.py", line 38, in foo
await bar()
File "test/test_stack.py", line 42, in bar
await baz()
File "test/test_stack.py", line 46, in baz
await leaf()
File "test/test_stack.py", line 50, in leaf
await asyncio.sleep(1)
File "/…/asyncio/tasks.py", line 568, in sleep
return await future
File "<Sentinel>", line 0, in <_asyncio.FutureIter object at 0x7fb6981690d8>: …
Dependency Graph
References
https://mail.python.org/archives/list/async-sig@python.org/thread/6E2LRVLKYSMGEAZ7OYOYR3PMZUUYSS3K/
Hi group,
I'm recently debugging a long-running asyncio program that appears to get stuck about once a week.
The tools I've discovered so far are:
- high level:
asyncio.all_tasks()
+asyncio.Task.get_stack()
- low level:
loop._selector._fd_to_key
What's missing is the middle level, i.e. stack-like linkage of what is waiting for what. For a practical example, consider:
async def leaf(): await somesocket.recv() async def baz(): await leaf() async def bar(): await baz() async def foo(): await bar() async def job(): await foo() async def work(): await asyncio.gather(..., job()) async def main(): asyncio.run(work())The task stack will contain:
- main and body of work with line number
- job task with line number pointing to foo
The file descriptor mapping, socket fd,
loop._recv()
and aFuture
.What's missing are connections
foo->bar->baz->leaf
. That is, I can't tell which task is waiting for what terminalFuture
.Is this problem solved in some way that I'm not aware of? Is there a library or external tool for this already?
Perhaps, if I could get a list of all pending coroutines, I could figure out what's wrong.
If no such API exists, I'm thinking of the following:
async def foo(): await bar() In [37]: dis.dis(foo) 1 0 LOAD_GLOBAL 0 (bar) 2 CALL_FUNCTION 0 4 GET_AWAITABLE 6 LOAD_CONST 0 (None) 8 YIELD_FROM 10 POP_TOP 12 LOAD_CONST 0 (None) 14 RETURN_VALUEStarting from a pending task, I'd get it's coroutine and:
Get the coroutine frame, and if current instruction is
YIELD_FROM
, then the reference to the awaitable should be on the top of the stack. If that reference points to a pending coroutine, I'd add that to the "forward trace" and repeat.At some point I'd reach an awaitable that's not a pending coroutine, which may be: another
Task
(I already got those), a low-levelFuture
(can be looked up in event loop), anEvent
(tough luck, shoulda logged allEvent
's on creation) or a dozen other corner cases.What do y'all think of this approach?
Thanks, D.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distributions
Hashes for awaitwhat-20.3-cp39-cp39-manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 5171a4f6ef8993b2a8c47061b6bc169a00d1ab59e20dbcba398438254240eebe |
|
MD5 | aa0cccf1a6831e6d23a1349ec6b3b23f |
|
BLAKE2b-256 | 2473e1d2ad98dd9ac2e004cefe81561672ae1b33487543e39cf8a28beb5a9f0f |
Hashes for awaitwhat-20.3-cp39-cp39-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9aae435a06872ee06854394a6836d85c2ef65aa7ec57591b55034178446d4467 |
|
MD5 | 620116e44005da5acbb1b2b3cc6a4d78 |
|
BLAKE2b-256 | cbe6e70532dcf7aadd3fefd387fb8a3774bd74916ce0c797362440092417a474 |
Hashes for awaitwhat-20.3-cp39-cp39-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | adee6697092b03aa27c651adf8c0d581d2e71abdf707eb390ab8bfb86d0e2316 |
|
MD5 | 33aee087465efeafa485bb24d228c1fb |
|
BLAKE2b-256 | df3854ade0f1b720f59604ef53549cc8e6bd0a5ae735d77298b40fe9d2aae71e |
Hashes for awaitwhat-20.3-cp38-cp38-manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 00b836c9ce06da7a682416bba89c9a01e592557972d25be99f91aeab1395e4cc |
|
MD5 | 0f0cb8c8a65179de4859535bd722e745 |
|
BLAKE2b-256 | a506dcda78cec47e8d92a80b7eedd88ff42011742923a621efbdd08cd218b749 |
Hashes for awaitwhat-20.3-cp38-cp38-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 2e62bd91997bb63bfb20974397dd7e6ddda1a9430bc8ecf5969485061d78b0e0 |
|
MD5 | c44b7f16b67f454e73a48576c1895bb2 |
|
BLAKE2b-256 | 2b1582a0a3ee4068fdc66e97191821f283c6fab47771ececd3fd1097ed9a0f4b |