Skip to main content

Extract transactions from the Trongrid API.

Project description

Usage

No configuration is needed to use trongrid_extractoor either from the command line or from your own python code.

Command Line

The extract_tron_events script will extract events from Trongrid to either CSV (for Transfer events) or JSON (for all other event types). Run extract_tron_events --help to see the command line options. --since and --until arguments should be specified in ISO8601 time format.

Examples:

# By address
extract_tron_events --until 2023-06-26T16:07:39+00:00 -t TR7NHqjeKQxGTCi8q8ZY4pL8otSzgjLj6t

# For symbol (only works for some preconfigured tokens)
extract_tron_events --since 2023-06-26T16:07:39+00:00 -t USDT

# All events (not just 'Transfer')
extract_tron_events -t wstUSDT --e all

# Resume an extraction you already started
extract_tron_events --resume-csv events_HT_TDyvndWuvX5xTBwHPYJi7J3Yq8pq8yh62h.csv

Other Helpful Command Line Tools

Convert between timestamp seconds and ISO8601 format strings:

$ epoch_ms_to_datetime 1616507145000
=> 2021-03-23T13:45:45+00:00

$ datetime_to_epoch_ms 2021-03-23T13:45:45
=> 1616507145000000

Convert between address formats:

# From hex format (starting with "0x" (eth) or "41" (tron)):
$ hex_address_to_tron 41102af1de57f7389468e22a72c529d78f2d4a5fde
=> TBShFz6ZKyEySS2vgd4s2yDsCTkQxtfqvy
hex_address_to_tron 0x102af1de57f7389468e22a72c529d78f2d4a5fde
=> TBShFz6ZKyEySS2vgd4s2yDsCTkQxtfqvy

# From base58 ("Tron format)
tron_address_to_hex TBShFz6ZKyEySS2vgd4s2yDsCTkQxtfqvy
=> 0x102af1de57f7389468e22a72c529d78f2d4a5fde

Get Tronscan / Trongrid account info:

$ tron_address_info TBShFz6ZKyEySS2vgd4s2yDsCTkQxtfqvy
╭────────────────────────────────────────────────────────────────╮
│ [NO_TYPE]: TBShFz6ZKyEySS2vgd4s2yDsCTkQxtfqvy (p2pb2bexchange) │
╰────────────────────────────────────────────────────────────────╯

{
   'account_name': 'p2pb2bexchange',
   'address': 'TBShFz6ZKyEySS2vgd4s2yDsCTkQxtfqvy',
   'balance': 3720438,
   'create_time': 1576393722000,
   'latest_opration_time': 1693071735000,
   'latest_withdraw_time': 1662297549000,
   'latest_consume_time': 1621455285000,
   'latest_consume_free_time': 1693071735000,
   'net_window_size': 28800000,
   'net_window_optimized': True,
   'account_resource': {
      'latest_consume_time_for_energy': 1685804475000,
      'energy_window_size': 28800000,
      'energy_window_optimized': True
   }
}

Show contract info (functions, events, arguments, etc.)

$ tron_contract_info TR7NHqjeKQxGTCi8q8ZY4pL8otSzgjLj6t
╭──────────────────────────────────────────────────╮
│ TR7NHqjeKQxGTCi8q8ZY4pL8otSzgjLj6t (TetherToken) │
╰──────────────────────────────────────────────────╯
{
   'bytecode': '[REDACTED_FOR_BREVITY]',
   'consume_user_resource_percent': 30,
   'name': 'TetherToken',
   'origin_address': 'THPvaUhoh2Qn2y9THCZML3H815hhFhn5YC',
   'abi': {
      'entrys': [
         {
            'outputs': [
               {
                  'type': 'string'
               }
            ],
            'constant': True,
            'name': 'name',
            'stateMutability': 'View',
            'type': 'Function'
         },
         <...snip...>
         {
            'name': 'Pause',
            'type': 'Event'
         },
         {
            'name': 'Unpause',
            'type': 'Event'
         },
         {
            'inputs': [
               {
                  'indexed': True,
                  'name': 'previousOwner',
                  'type': 'address'
               },
               {
                  'indexed': True,
                  'name': 'newOwner',
                  'type': 'address'
               }
            ],
            'name': 'OwnershipTransferred',
            'type': 'Event'
         },
         <...snip...>
      ]
   },
   'origin_energy_limit': 10000000,
   'contract_address': 'TR7NHqjeKQxGTCi8q8ZY4pL8otSzgjLj6t',
   'code_hash': '99bb60e56b4cd2642c6847e372b18b6e0f9514229e3086d3a042d60a4c7b78a9'
}

As Python Package

Event Extraction

contract_events() hits the contracts/[CONTRACT_ADDRESS]/events endpoint and can pull all transfers for a given contract by filtering for event_name=Transfer. Other endpoints like contracts/[CONTRACT_ADDRESS]/transactions don't seem to really work. Arguments for contract_events() can be found here.

from trongrid_extractoor.api import Api

api = Api()

# For when there's a reasonably small number of events you can use get_all_contract_events()
mints = api.get_all_contract_events(self, 'Mint')

# If there's a whole lot of events it's good to page through them
for events in api.contract_events('TR7NHqjeKQxGTCi8q8ZY4pL8otSzgjLj6t', since='2022-05-05', until='2022-08-31'):
    for event in events:
        do_something_with(event)

# Get contract:
usdt = api.get_contract('TR7NHqjeKQxGTCi8q8ZY4pL8otSzgjLj6t')

# Get account info:
tether_tronscan_account = api.get_account('TR7NHqjeKQxGTCi8q8ZY4pL8otSzgjLj6t')

Resources

Contributing

This project was developed with poetry and as such is probably easier to work with using poetry to manage dependencies in the dev environment. Install with:

poetry install --with=dev

Running Tests

pytest

Publishing to PyPi

Configuration:

  1. poetry config repositories.chain_argos_pypi https://upload.pypi.org/legacy/
  2. poetry config pypi-token.chain_argos_pypi [API_TOKEN]

Publishing:

  1. Update pyproject.toml version number
  2. Update CHANGELOG.md
  3. poetry publish --build --repository chain_argos_pypi

TODO

  1. Walk forward not backward
  2. Weird that this yielded dupes on the first page:
    {request_params.py:29} INFO - Request URL: https://api.trongrid.io/v1/contracts/TR7NHqjeKQxGTCi8q8ZY4pL8otSzgjLj6t/events
    Params requesting 'Transfer' events from 2021-04-01T04:00:01+00:00 to 2021-04-01T05:00:00+00:00 (no extra params).
    {response.py:48} INFO - New query requesting data from 2021-04-01T04:00:01+00:00 to 2021-04-01T05:00:00+00:00.
    {progress_tracker.py:67} INFO -   Removed 11 duplicate transactions...
    
  3. USDT looks incomplete here as 9pm was the last time:
    WARNING - 0 txns found. We seem to be stuck at 2020-07-09T21:04:24+00:00.
    [2023-06-29, 06:34:36 UTC] {logging_mixin.py:137} INFO -                     WARNING    Last request params:                   api.py:127
                              {'only_confirmed': 'true', 'limit': 200,
                              'min_timestamp': 1594252801000.0,
                              'max_timestamp': 1594328664000.0,
                              'event_name': 'Transfer'}
    
  4. USDD around this time should be double checked:
     INFO       Returning 1000 transactions from _rescue_extraction(), modified params in place.                                    api.py:191
     INFO     Writing 1000 rows to 'events_USDD_written_2023-06-28T04.22.00.csv'...                                           csv_helper.py:17
     [06/28/23 10:22:34] INFO       Removed 200 duplicate transactions...                                                                   progress_tracker.py:47
     WARNING  0 txns found. We seem to be stuck at 2023-01-26T03:18:54+00:00.                                                       api.py:103
     WARNING    Last request params: {'only_confirmed': 'true', 'limit': 200, 'min_timestamp': 1483228800000.0, 'max_timestamp':    api.py:104
             1674703134000.0, 'event_name': 'Transfer'}
    

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

trongrid_extractoor-4.10.2.tar.gz (44.3 kB view hashes)

Uploaded Source

Built Distribution

trongrid_extractoor-4.10.2-py3-none-any.whl (56.5 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page