dunebuggy 1.0.8

A lightweight (unofficial) Python SDK for Dune.com

Dunebuggy

A lightweight (unofficial) Python SDK for Dune.com

Installation • Getting started • Roadmap • Notes

Installation

pip install dunebuggy

Getting started

Retrieving a public query

To retrieve a query, all we'll need is the query_id for the public query we're interested in. In the below example we can take a look at the popular "Custom NFT Floor Tracker" query by @smaroo (The query_id below can be found in the URL).

from dunebuggy import Dune

dune = Dune()
query = dune.fetch_query(83579)

query here is a DuneQuery object, we can get the pandas DataFrame for the query output bf calling df on the object

print(query.df.head())

	Floor (Approx)	Time Interval
0	0.122649	2021-06-01T00:00:00+00:00
1	0.130000	2021-06-02T00:00:00+00:00
2	0.193455	2021-06-03T00:00:00+00:00
3	0.189000	2021-06-04T00:00:00+00:00
4	0.189930	2021-06-05T00:00:00+00:00

We can also take a look at some basic information about the returned query with query.info

print(query.info)

{'name': 'Custom NFT Floor Tracker',
 'author': '@smaroo',
 'length': 264,
 'query_id': 83579,
 'result_id': UUID('e5aef8a0-1453-44d1-a27b-f576ea2b3ba2'),
 'job_id': UUID('ec680fa9-217f-44c5-b223-56730cd07473'),
 'columns': ['Time Interval', 'Floor (Approx)']}

Some queries in Dune are "parameterized", meaning that the author exposes certain variables for the user to enter custom values. The example query (83579) happens to be parameterized, we can verify this by inspecting query.parameters

print(query.parameters)

[QueryParameter(key='Enter NFT Contract Address', type='text', value='xc3f733ca98e0dad0386979eb96fb1722a1a05e69', enumOptions=None),
 QueryParameter(key='Floor Time Interval', type='enum', value='Day', enumOptions=['Day', 'Hour']),
 QueryParameter(key='Start Date', type='datetime', value='2021-06-01 00:00:00', enumOptions=None)]

If you'd like to run this query with your own custom parameters, all we'll need to do is take the parameters from from the initial query, change the values to what we want, and re-fetch the query.

Below we are replacing the old NFT contract address param with a new one (the contract address for BAYC)

params = query.parameters

# Replacing with contract address for BAYC
params[0].value = 'xBC4CA0EdA7647A8aB7C2061c2E118A18a936f13D'
custom_query = dune.fetch_query(83579, parameters=params)

Note: You can also create a fresh set of parameters by importing QueryParameter from dunebuggy.models.query and adding the values to the new object.

from dunebuggy.models.query import QueryParameter

param_to_change = QueryParameter(
  key='Enter NFT Contract Address',
  value='xBC4CA0EdA7647A8aB7C2061c2E118A18a936f13D',
)
params[0] = param_to_change
custom_query = dune.fetch_query(83579, parameters=params)

print(custom_query.info)

{'name': 'Custom NFT Floor Tracker',
 'author': '@smaroo',
 'length': 265,
 'query_id': 83579,
 'result_id': UUID('42a3c13d-5fbd-42bd-86c0-acc9adcdc803'),
 'job_id': UUID('9051ebe7-862f-46d0-9999-b4645659ca56'),
 'columns': ['Time Interval', 'Floor (Approx)']}

Note that the result_id and job_id here are different, this is because we ran the query with our changed params

print(custom_query.parameters)

[QueryParameter(key='Enter NFT Contract Address', type='text', value='xBC4CA0EdA7647A8aB7C2061c2E118A18a936f13D', enumOptions=None),
QueryParameter(key='Floor Time Interval', type='enum', value='Day', enumOptions=['Day', 'Hour']),
QueryParameter(key='Start Date', type='datetime', value='2021-06-01 00:00:00', enumOptions=None)]

print(custom_query.df.head())

	Floor (Approx)	Time Interval
0	0.8000	2021-06-01T00:00:00+00:00
1	0.8518	2021-06-02T00:00:00+00:00
2	0.8260	2021-06-03T00:00:00+00:00
3	0.7400	2021-06-04T00:00:00+00:00
4	0.8499	2021-06-05T00:00:00+00:00

Creating a new query

dunebuggy also allows you to create a new using an existing Dune.com account.To login just need to pass in your username/password into the Dune object.

You can verify your login by inspecting your Dune user_id

import os

username = os.environ.get('DUNE_USERNAME')
password = os.environ.get('DUNE_PASSWORD')

dune = Dune(username=username, password=password)
# print(dune.user_id)

To create a query now, all we need to do is pass in a name, query_string and dataset_id

We can construct the SQL query by using a raw sql string

query_string = "select * from ethereum.transactions\nLIMIT 100\n"

Or we could use a fancy ORM-style library like pypika

from pypika import Database, Query

ethereum = Database('ethereum')
q = Query.from_(ethereum.transactions).select('*').limit(100)
query_string = q.get_sql(quote_char=None)
print(query_string)

'SELECT * FROM ethereum.transactions LIMIT 100'

Dune requires us to specify a dataset_id for each of their supported blockchain datasets upon query creation. The currently supported datasets are the following:

Blockchain Dataset	Id
ETHEREUM	4
XDAI	6
POLYGON	7
OPTIMISM_1	8
OPTIMISM_2	10
BINANCE	9
SOLANA	1

We can access these integer codes via the DatasetId enum

from dunebuggy.models.constants import DatasetId
created_query = dune.create_query("My Query's Name", query_string, DatasetId.ETHEREUM)

Our created query can be accessed like any other, you can also log into your Dune account as see it there as well!

print(created_query.df.head())

	access_list	block_hash	block_number	block_time	data	from	gas_limit	gas_price	gas_used	hash	index	max_fee_per_gas	max_priority_fee_per_gas	nonce	priority_fee_per_gas	success	to	type	value
0	None	\x887c665b0c52ccace092d817e984e2e828ef59079295...	47287	2015-08-07T08:50:01+00:00	None	\xdb312d1d6a2ccc64dd94a3892928bac82b4e8c15	21000	100000000000	21000	\xd3e6a2fc34066d20bb83020b1ee95b9dc7919fd242bd...	0	None	None	0	None	None	\x34bb6978c5a1ad68777ad388c6787df53903430c	None	1000000000000000000
1	None	\x4869e218b0a8f5784f16193ac66cbf35c4510ace0c9b...	48698	2015-08-07T15:29:53+00:00	None	\x48040276e9c17ddbe5c8d2976245dcd0235efa43	90000	57550496008	21000	\x8ba39f908731171fe96ee4e700e71d170ef8e651fac7...	0	None	None	0	None	None	\xd8d0549637b65d58e7fb6cbdd11530b399d1ddac	None	100000000000000000000
2	None	\xab9491b62b16bd928b281a83db82483584c22aeebc0d...	49051	2015-08-07T17:03:48+00:00	None	\x8686578c4f7c75246f548299d6ffdac3b67b5cd1	90000	57178423039	21000	\x57f8ba638903d6335e211eb470159587c73316788880...	0	None	None	0	None	None	\x87abffa6b80f712c852a9558120ba6611f0b5e46	None	45150000000000000000
3	None	\x1f9adc2190701ca3085b28252e4f1f467d980f763dad...	49174	2015-08-07T17:41:03+00:00	None	\x18e4ce47483b53040adbab35172c01ef64506e0c	90000	58589751415	21000	\xb8280da44f8d35011c3f431f7d1a82213477a4e742de...	2	None	None	0	None	None	\xfb26ae2d3621829472555fbd11bb2a324b7a5c57	None	10000000000000000000
4	None	\xf1f392fd197a149afe9f8843d7ba759d1a9f79d1ef62...	49938	2015-08-07T21:06:21+00:00	None	\xc6bf5b6558f2ee21f2e43d9ff9b5408a0cb89413	90000	71214529679	21000	\x538e1664c12c55287c98dc5dd248f60c642cbbbd7a18...	0	None	None	4	None	None	\x33a3f479f6c3e7f91128348490d1f7e8d2a0fab5	None	5000000000000000000

Saving to CSV

To save a query to a CSV, we can take advantage of the to_csv method on our df

created_query.df.to_csv('my_test_data.csv')

Roadmap

• Cleanup punding TODO comments
• Add support for embedding Dune graphs/ plotting w/ Dune style colors
• Add tests (lol)
• Add support for query updating
• Investigate whether dashboard support makes sense?
• Investigate whether there is a max row limit for data returned, if so, query in batches?
• Better formatting for certain returned columns (links etc..)
• Add Documentation (Sphinx or something else)

Notes

This project was inspired by the itzemstar's duneanalytics repo

README image is from the IAN Symbols dataset