Skip to main content

Library for interfacing with a Substrate node

Project description

Python Substrate Interface

Build Status Latest Version Supported Python versions License

Python Substrate Interface Library

Description

This library specializes in interfacing with a Substrate node, providing additional convenience methods to deal with SCALE encoding/decoding (the default output and input format of the Substrate JSONRPC), metadata parsing, type registry management and versioning of types.

Table of Contents

Documentation

https://polkascan.github.io/py-substrate-interface/

Installation

pip install substrate-interface

Initialization

The following examples show how to initialize for supported chains:

Autodiscover mode

substrate = SubstrateInterface(
    url="wss://rpc.polkadot.io"
)

When only an url is provided, it tries to determine certain properties like ss58_format and type_registry_preset automatically by calling the RPC method system_properties.

At the moment this will work for Polkadot, Kusama, Kulupu and Westend nodes, for other chains the ss58_format (default 42) and type_registry (defaults to latest vanilla Substrate types) should be set manually.

Manually set required properties

Polkadot

substrate = SubstrateInterface(
    url="wss://rpc.polkadot.io",
    ss58_format=0,
    type_registry_preset='polkadot'
)

Kusama

substrate = SubstrateInterface(
    url="wss://kusama-rpc.polkadot.io/",
    ss58_format=2,
    type_registry_preset='kusama'
)

Rococo

substrate = SubstrateInterface(
    url="wss://rococo-rpc.polkadot.io",
    ss58_format=42,
    type_registry_preset='rococo'
)

Westend

substrate = SubstrateInterface(
    url="wss://westend-rpc.polkadot.io",
    ss58_format=42,
    type_registry_preset='westend'
)

Substrate Node Template

Compatible with https://github.com/substrate-developer-hub/substrate-node-template

substrate = SubstrateInterface(
    url="ws://127.0.0.1:9944",
    ss58_format=42,
    type_registry_preset='substrate-node-template'
)

If custom types are introduced in the Substrate chain, the following example will add compatibility by creating a custom type registry JSON file and including this during initialization:

{
  "runtime_id": 2,
  "types": {
    "MyCustomInt": "u32",
    "MyStruct": {
      "type": "struct",
      "type_mapping": [
         ["account", "AccountId"],
         ["message", "Vec<u8>"]
      ]
    }
  },
  "versioning": [
  ]
}
custom_type_registry = load_type_registry_file("my-custom-types.json")

substrate = SubstrateInterface(
    url="ws://127.0.0.1:9944",
    ss58_format=42,
    type_registry_preset='substrate-node-template',
    type_registry=custom_type_registry
)

Features

Retrieve extrinsics for a certain block (Method 1: access serialized value)

# Set block_hash to None for chaintip
block_hash = "0x51d15792ff3c5ee9c6b24ddccd95b377d5cccc759b8e76e5de9250cf58225087"

# Retrieve extrinsics in block
result = substrate.get_block(block_hash=block_hash)

for extrinsic in result['extrinsics']:

    if 'address' in extrinsic.value:
        signed_by_address = extrinsic.value['address']
    else:
        signed_by_address = None

    print('\nPallet: {}\nCall: {}\nSigned by: {}'.format(
        extrinsic.value["call"]["call_module"],
        extrinsic.value["call"]["call_function"],
        signed_by_address
    ))

    # Loop through call params
    for param in extrinsic.value["call"]['call_args']:

        if param['type'] == 'Balance':
            param['value'] = '{} {}'.format(param['value'] / 10 ** substrate.token_decimals, substrate.token_symbol)

        print("Param '{}': {}".format(param['name'], param['value']))

Retrieve extrinsics for a certain block (Method 2: access nested objects)

# Set block_hash to None for chaintip
block_hash = "0x51d15792ff3c5ee9c6b24ddccd95b377d5cccc759b8e76e5de9250cf58225087"

# Retrieve extrinsics in block
result = substrate.get_block(block_hash=block_hash)

for extrinsic in result['extrinsics']:

    if 'address' in extrinsic:
        signed_by_address = extrinsic['address'].value
    else:
        signed_by_address = None

    print('\nPallet: {}\nCall: {}\nSigned by: {}'.format(
        extrinsic["call"]["call_module"].name,
        extrinsic["call"]["call_function"].name,
        signed_by_address
    ))

    # Loop through call params
    for param in extrinsic["call"]['call_args']:

        if param['type'] == 'Balance':
            param['value'] = '{} {}'.format(param['value'] / 10 ** substrate.token_decimals, substrate.token_symbol)

        print("Param '{}': {}".format(param['name'], param['value']))

Subscribe to new block headers

def subscription_handler(obj, update_nr, subscription_id):

    print(f"New block #{obj['header']['number']} produced by {obj['author']}")

    if update_nr > 10:
        return {'message': 'Subscription will cancel when a value is returned', 'updates_processed': update_nr}


result = substrate.subscribe_block_headers(subscription_handler, include_author=True)

Storage queries

The modules and storage functions are provided in the metadata (see substrate.get_metadata_storage_functions()), parameters will be automatically converted to SCALE-bytes (also including decoding of SS58 addresses).

Example:

result = substrate.query(
    module='System',
    storage_function='Account',
    params=['F4xQKRUagnSGjFqafyhajLs94e7Vvzvr8ebwYJceKpr8R7T']
)

print(result.value['nonce']) #  7695
print(result.value['data']['free']) # 635278638077956496

Or get the account info at a specific block hash:

account_info = substrate.query(
    module='System',
    storage_function='Account',
    params=['F4xQKRUagnSGjFqafyhajLs94e7Vvzvr8ebwYJceKpr8R7T'],
    block_hash='0x176e064454388fd78941a0bace38db424e71db9d5d5ed0272ead7003a02234fa'
)

print(account_info['nonce'].value) #  7673
print(account_info['data']['free'].value) # 637747267365404068

Using ScaleType objects

The result of the previous storage query example is a ScaleType object, more specific a Struct.

The nested object structure of this account_info object is as follows:

account_info = <AccountInfo(value={'nonce': <U32(value=5)>, 'consumers': <U32(value=0)>, 'providers': <U32(value=1)>, 'sufficients': <U32(value=0)>, 'data': <AccountData(value={'free': 1152921503981846391, 'reserved': 0, 'misc_frozen': 0, 'fee_frozen': 0})>})>

Every ScaleType have the following characteristics:

Shorthand lookup of nested types

Inside the AccountInfo struct there are several U32 objects that represents for example a nonce or the amount of provider, also another struct object AccountData which contains more nested types.

To access these nested structures you can access those formally using:

account_info.value_object['data'].value_object['free']

As a convenient shorthand you can also use:

account_info['data']['free']

ScaleType objects can also be automatically converted to an iterable, so if the object is for example the others in the result Struct of Staking.eraStakers can be iterated via:

for other_info in era_stakers['others']:
    print(other_info['who'], other_info['value'])

Serializable

Each ScaleType holds a complete serialized version of itself in the account_info.serialize() property, so it can easily stored or used to create JSON strings.

So the whole result of account_info.serialize() will be a dict containing the following:

{
    "nonce": 5,
    "consumers": 0,
    "providers": 1,
    "sufficients": 0,
    "data": {
        "free": 1152921503981846391,
        "reserved": 0,
        "misc_frozen": 0,
        "fee_frozen": 0
    }
}

Comparing values with ScaleType objects

[TODO]

Storage subscriptions

When a callable is passed as kwarg subscription_handler, there will be a subscription created for given storage query. Updates will be pushed to the callable and will block execution until a final value is returned. This value will be returned as a result of the query and finally automatically unsubscribed from further updates.

def subscription_handler(account_info_obj, update_nr, subscription_id):

    if update_nr == 0:
        print('Initial account data:', account_info_obj.value)

    if update_nr > 0:
        # Do something with the update
        print('Account data changed:', account_info_obj.value)

    # The execution will block until an arbitrary value is returned, which will be the result of the `query`
    if update_nr > 5:
        return account_info_obj


result = substrate.query("System", "Account", ["5GNJqTPyNqANBkUVMN1LPPrxXnFouWXoe2wNSmmEoLctxiZY"],
                         subscription_handler=subscription_handler)

print(result)

Query a mapped storage function

Mapped storage functions can be iterated over all key/value pairs, for these type of storage functions query_map can be used.

The result is a QueryMapResult object, which is an iterator:

# Retrieve the first 199 System.Account entries
result = substrate.query_map('System', 'Account', max_results=199)

for account, account_info in result:
    print(f"Free balance of account '{account.value}': {account_info.value['data']['free']}")

These results are transparantly retrieved in batches capped by the page_size kwarg, currently the maximum page_size restricted by the RPC node is 1000

# Retrieve all System.Account entries in batches of 200 (automatically appended by `QueryMapResult` iterator)
result = substrate.query_map('System', 'Account', page_size=200, max_results=400)

for account, account_info in result:
    print(f"Free balance of account '{account.value}': {account_info.value['data']['free']}")

Querying a DoubleMap storage function:

era_stakers = substrate.query_map(
    module='Staking',
    storage_function='ErasStakers',
    params=[2100]
)

Create and send signed extrinsics

The following code snippet illustrates how to create a call, wrap it in a signed extrinsic and send it to the network:

from substrateinterface import SubstrateInterface, Keypair
from substrateinterface.exceptions import SubstrateRequestException

substrate = SubstrateInterface(
    url="ws://127.0.0.1:9944",
    ss58_format=42,
    type_registry_preset='kusama'
)

keypair = Keypair.create_from_mnemonic('episode together nose spoon dose oil faculty zoo ankle evoke admit walnut')

call = substrate.compose_call(
    call_module='Balances',
    call_function='transfer',
    call_params={
        'dest': '5E9oDs9PjpsBbxXxRE9uMaZZhnBAV38n2ouLB28oecBDdeQo',
        'value': 1 * 10**12
    }
)

extrinsic = substrate.create_signed_extrinsic(call=call, keypair=keypair)

try:
    receipt = substrate.submit_extrinsic(extrinsic, wait_for_inclusion=True)
    print("Extrinsic '{}' sent and included in block '{}'".format(receipt.extrinsic_hash, receipt.block_hash))

except SubstrateRequestException as e:
    print("Failed to send: {}".format(e))

The wait_for_inclusion keyword argument used in the example above will block giving the result until it gets confirmation from the node that the extrinsic is succesfully included in a block. The wait_for_finalization keyword will wait until extrinsic is finalized. Note this feature is only available for websocket connections.

Examining the ExtrinsicReceipt object

The substrate.submit_extrinsic example above returns an ExtrinsicReceipt object, which contains information about the on-chain execution of the extrinsic. Because the block_hash is necessary to retrieve the triggered events from storage, most information is only available when wait_for_inclusion=True or wait_for_finalization=True is used when submitting an extrinsic.

Examples:

receipt = substrate.submit_extrinsic(extrinsic, wait_for_inclusion=True)
print(receipt.is_success) # False
print(receipt.weight) # 216625000
print(receipt.total_fee_amount) # 2749998966
print(receipt.error_message['name']) # 'LiquidityRestrictions'

ExtrinsicReceipt objects can also be created for all existing extrinsics on-chain:

receipt = ExtrinsicReceipt(
    substrate=substrate,
    extrinsic_hash="0x56fea3010910bd8c0c97253ffe308dc13d1613b7e952e7e2028257d2b83c027a",
    block_hash="0x04fb003f8bc999eeb284aa8e74f2c6f63cf5bd5c00d0d0da4cd4d253a643e4c9"
)

print(receipt.is_success) # False
print(receipt.extrinsic.call_module.name) # 'Identity'
print(receipt.extrinsic.call.name) # 'remove_sub'
print(receipt.weight) # 359262000
print(receipt.total_fee_amount) # 2483332406
print(receipt.error_message['docs']) # [' Sender is not a sub-account.']

for event in receipt.triggered_events:
    print(f'* {event.value}')

ink! contract interfacing

Deploy a contract

Tested on canvas-node with the Flipper contract from the tutorial_:

substrate = SubstrateInterface(
    url="ws://127.0.0.1:9944",
    type_registry_preset='canvas'
)

keypair = Keypair.create_from_uri('//Alice')

# Deploy contract
code = ContractCode.create_from_contract_files(
    metadata_file=os.path.join(os.path.dirname(__file__), 'assets', 'flipper.json'),
    wasm_file=os.path.join(os.path.dirname(__file__), 'assets', 'flipper.wasm'),
    substrate=substrate
)

contract = code.deploy(
    keypair=keypair,
    endowment=10 ** 15,
    gas_limit=1000000000000,
    constructor="new",
    args={'init_value': True},
    upload_code=True
)

print(f'✅ Deployed @ {contract.contract_address}')

Work with an existing instance:

# Create contract instance from deterministic address
contract = ContractInstance.create_from_address(
    contract_address=contract_address,
    metadata_file=os.path.join(os.path.dirname(__file__), 'assets', 'flipper.json'),
    substrate=substrate
)

Read data from a contract:

result = contract.read(keypair, 'get')
print('Current value of "get":', result.contract_result_data)

Execute a contract call

 # Do a gas estimation of the message
gas_predit_result = contract.read(keypair, 'flip')

print('Result of dry-run: ', gas_predit_result.contract_result_data)
print('Gas estimate: ', gas_predit_result.gas_consumed)

# Do the actual transfer
print('Executing contract call...')
contract_receipt = contract.exec(keypair, 'flip', args={

}, gas_limit=gas_predit_result.gas_consumed)

print(f'Events triggered in contract: {contract_receipt.contract_events}')

See complete code example for more details

Create mortal extrinsics

By default, immortal extrinsics are created, which means they have an indefinite lifetime for being included in a block. However, it is recommended to use specify an expiry window, so you know after a certain amount of time if the extrinsic is not included in a block, it will be invalidated.

extrinsic = substrate.create_signed_extrinsic(call=call, keypair=keypair, era={'period': 64})

The period specifies the number of blocks the extrinsic is valid counted from current head.

Keypair creation and signing

mnemonic = Keypair.generate_mnemonic()
keypair = Keypair.create_from_mnemonic(mnemonic)
signature = keypair.sign("Test123")
if keypair.verify("Test123", signature):
    print('Verified')

By default, a keypair is using SR25519 cryptography, alternatively ED25519 can be explictly specified:

keypair = Keypair.create_from_mnemonic(mnemonic, crypto_type=KeypairType.ED25519)

Creating keypairs with soft and hard key derivation paths

mnemonic = Keypair.generate_mnemonic()
keypair = Keypair.create_from_uri(mnemonic + '//hard/soft')

By omitting the mnemonic the default development mnemonic is used:

keypair = Keypair.create_from_uri('//Alice')

Getting estimate of network fees for extrinsic in advance

keypair = Keypair(ss58_address="EaG2CRhJWPb7qmdcJvy3LiWdh26Jreu9Dx6R1rXxPmYXoDk")

call = substrate.compose_call(
    call_module='Balances',
    call_function='transfer',
    call_params={
        'dest': 'EaG2CRhJWPb7qmdcJvy3LiWdh26Jreu9Dx6R1rXxPmYXoDk',
        'value': 2 * 10 ** 3
    }
)
payment_info = substrate.get_payment_info(call=call, keypair=keypair)
# {'class': 'normal', 'partialFee': 2499999066, 'weight': 216625000}

Offline signing of extrinsics

This example generates a signature payload which can be signed on another (offline) machine and later on sent to the network with the generated signature.

  • Generate signature payload on online machine:
substrate = SubstrateInterface(
    url="ws://127.0.0.1:9944",
    ss58_format=42,
    type_registry_preset='substrate-node-template',
)

call = substrate.compose_call(
    call_module='Balances',
    call_function='transfer',
    call_params={
        'dest': '5GrwvaEF5zXb26Fz9rcQpDWS57CtERHpNehXCPcNoHGKutQY',
        'value': 2 * 10**8
    }
)

era = {'period': 64, 'current': 22719}
nonce = 0

signature_payload = substrate.generate_signature_payload(call=call, era=era, nonce=nonce)
  • Then on another (offline) machine generate the signature with given signature_payload:
keypair = Keypair.create_from_mnemonic("nature exchange gasp toy result bacon coin broccoli rule oyster believe lyrics")
signature = keypair.sign(signature_payload)
  • Finally on the online machine send the extrinsic with generated signature:
keypair = Keypair(ss58_address="5EChUec3ZQhUvY1g52ZbfBVkqjUY9Kcr6mcEvQMbmd38shQL")

extrinsic = substrate.create_signed_extrinsic(
    call=call,
    keypair=keypair,
    era=era,
    nonce=nonce,
    signature=signature
)

result = substrate.submit_extrinsic(
    extrinsic=extrinsic
)

print(result.extrinsic_hash)

Accessing runtime constants

All runtime constants are provided in the metadata (see substrate.get_metadata_constants()), to access these as a decoded ScaleType you can use the function substrate.get_constant():

constant = substrate.get_constant("Balances", "ExistentialDeposit")

print(constant.value) # 10000000000

Cleanup and context manager

At the end of the lifecycle of a SubstrateInterface instance, calling the close() method will do all the necessary cleanup, like closing the websocket connection.

When using the context manager this will be done automatically:

with SubstrateInterface(url="wss://rpc.polkadot.io") as substrate:
    events = substrate.query("System", "Events")

# connection is now closed

Keeping type registry presets up to date

When on-chain runtime upgrades occur, types used in call- or storage functions can be added or modified. Therefor it is important to keep the type registry presets up to date, otherwise this can lead to decoding errors like RemainingScaleBytesNotEmptyException.

At the moment the type registry presets for Polkadot, Kusama, Rococo and Westend are being actively maintained for this library, and a check and update procedure can be triggered with:

substrate.reload_type_registry()

This will also activate the updated preset for the current instance.

It is also possible to always use the remote type registry preset from Github with the use_remote_preset kwarg when instantiating:

substrate = SubstrateInterface(
    url="wss://rpc.polkadot.io",
    ss58_format=0,
    type_registry_preset='polkadot',
    use_remote_preset=True
)

To check for updates after instantiating the substrate object, using substrate.reload_type_registry() will download the most recent type registry preset from Github and apply changes to current object.

Contact and Support

For questions, please reach out to us on our matrix chat group: Polkascan Technical.

License

https://github.com/polkascan/py-substrate-interface/blob/master/LICENSE

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

substrate-interface-1.0.3.tar.gz (166.2 kB view details)

Uploaded Source

Built Distribution

substrate_interface-1.0.3-py3-none-any.whl (172.1 kB view details)

Uploaded Python 3

File details

Details for the file substrate-interface-1.0.3.tar.gz.

File metadata

  • Download URL: substrate-interface-1.0.3.tar.gz
  • Upload date:
  • Size: 166.2 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.2 importlib_metadata/4.8.1 pkginfo/1.7.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.8.12

File hashes

Hashes for substrate-interface-1.0.3.tar.gz
Algorithm Hash digest
SHA256 0d42e8ac2c2b9d41b5a7f33c8ca7af0bfc10c4290881766f41ba145338273a5f
MD5 9dde802169e62ef8fbb8442ee93fbf71
BLAKE2b-256 9a81a1a6cbdc1de9555d4b7f981312b6b94724a2484d66ae9d0a259dbb196db8

See more details on using hashes here.

File details

Details for the file substrate_interface-1.0.3-py3-none-any.whl.

File metadata

  • Download URL: substrate_interface-1.0.3-py3-none-any.whl
  • Upload date:
  • Size: 172.1 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.2 importlib_metadata/4.8.1 pkginfo/1.7.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.8.12

File hashes

Hashes for substrate_interface-1.0.3-py3-none-any.whl
Algorithm Hash digest
SHA256 7707ae6e3da32f37f32682767cb7ff5afbaa33bf9f82be608cd648c0bf377b85
MD5 7e4ec995985e7616a07f12fe3ed4a20d
BLAKE2b-256 7e68a1e38ac0064dd8071dae758c25bcb54fe297b2fd01057cc26194cfa59615

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page