neo-fairy-client 3.7.5.18

Test & debug your Neo3 smart contracts

Non-official test suite inherited from https://github.com/Hecate2/neo-ruler/ , in Python.

Features comply with https://github.com/Hecate2/neo-rpc-server-with-session/ .

A crude JavaScript version is available at https://github.com/Hecate2/neo-fairy-gate/blob/master/src/libs/NeoFairyClient.jsx .

Tutorial for testing

pip install neo-fairy-client or py -m build && cd dist && pip install neo_fairy_client***.whl. The only dependency is requests.

Python >= 3.8 required! Some steps in this tutorial is to help you understand the details about how Fairy works. In actual combat, you can read the source codes of FairyClient and enjoy many automatic conveniences that Fairy offers.

Extremely fast but close-to-base version:

Visit test_nftloan.py as a sample of usage. The tested contract can be found at https://github.com/Hecate2/NFTLoan . Contract AnyUpdateShortSafe is an old-fashioned contract for testing, deployed on testnet T4 (which has been deprecated; we now use testnet T5), with source codes at https://github.com/Hecate2/AnyUpdate/ . You can skip using AnyUpdate by calling the RPC method virtualdeploy.

Step 1: Run a neo-cli with Fairy plugin!

Head to https://github.com/Hecate2/neo-fairy-test/ to prepare it. You do not really have to wait for the blocks to be completely synchronized. The plugin is an HTTP server that will help you interact with Neo.

Step 2: Using your client, prepare your server snapshot

Place a json file of neo wallet (assumed to be testnet.json with password 1) beside neo-cli.exe, and call your Fairy server with the following Python codes: (Complete codes available at https://github.com/Hecate2/neo-fairy-client/blob/master/tutorial.py )

from neo_fairy_client.rpc import FairyClient
from neo_fairy_client.utils import Hash160Str
target_url = 'http://127.0.0.1:16868'
wallet_address = 'Nb2CHYY5wTh2ac58mTue5S3wpG6bQv5hSY'
wallet_scripthash = Hash160Str.from_address(wallet_address)
wallet_path = 'testnet.json'
wallet_password = '1'
client = FairyClient(fairy_session='Hello world! Your first contact with Fairy!',
                     wallet_address_or_scripthash=wallet_address,
                     auto_preparation=True)

Here auto_preparation=True tries to delete the old snapshot on the Fairy server named Hello world! Your first contact with Fairy!, and creates a new snapshot of the same name based on the current Neo system snapshot, then opens the wallet on Fairy server and automatically sets you NEO and GAS balance both to 100 (*10^8).

If you are planning to run a public Fairy server, you need to open the Fairy wallet so that users do not have to open it through RPC. I am also planning to remove wallet objects in Fairy service.

Step 2.1: Mint billions of NEO and transfer them!

(Of course these are just fairy NEO in the memory of your imaginiation)

from neo_fairy_client.utils import NeoAddress
client.set_neo_balance(1_000_000_000)
print(f"Your NEO balance: {client.invokefunction_of_any_contract(NeoAddress, 'balanceOf', [wallet_scripthash])}")
client.invokefunction_of_any_contract(NeoAddress, 'transfer', [wallet_scripthash, Hash160Str.zero(), 1_000_000_000, None])
print(f"NEO balance of zero address: {client.invokefunction_of_any_contract(NeoAddress, 'balanceOf', [Hash160Str.zero()])}")

Hello world! Your first contact with Fairy!::balanceOf[0xb1983fa2479a0c8e2beae032d2df564b5451b7a5] relay=None [{'account': '0xb1983fa2479a0c8e2beae032d2df564b5451b7a5', 'scopes': 'CalledByEntry', 'allowedcontracts': [], 'allowedgroups': [], 'rules': []}]
Your NEO balance: 1000000000
Hello world! Your first contact with Fairy!::transfer[0xb1983fa2479a0c8e2beae032d2df564b5451b7a5, 0x0000000000000000000000000000000000000000, 1000000000, None] relay=None [{'account': '0xb1983fa2479a0c8e2beae032d2df564b5451b7a5', 'scopes': 'CalledByEntry', 'allowedcontracts': [], 'allowedgroups': [], 'rules': []}]
Hello world! Your first contact with Fairy!::balanceOf[0x0000000000000000000000000000000000000000] relay=None [{'account': '0xb1983fa2479a0c8e2beae032d2df564b5451b7a5', 'scopes': 'CalledByEntry', 'allowedcontracts': [], 'allowedgroups': [], 'rules': []}]
NEO balance of zero address: 1000000000

Step 2.2: I just want to interact with the real mainnet and testnet...

DO NOT set the fairy_session string for you FairyClient, or set it to None. Fairy will play real transactions without fairy session. Set function_default_relay=True in FairyClient or relay=True in a single invokefunction to automatically relay the transaction.

**BE CAREFUL: ** By default, Fairy does interact with the real blockchain and relay transactions. Do not use a wallet with real assets when you just want a test!

Sometimes you may want to actually relay something after fairy tests. In such cases, set confirm_relay_to_blockchain=True in FairyClient to prevent automatic relaying as the final safety belt.

Step 3: Deploy your contract virtually

Get the tested contracts in my example through these repos:

https://github.com/Hecate2/AnyUpdate

https://github.com/Hecate2/NFTLoan/

and place them properly.

nef_file, manifest = client.get_nef_and_manifest_from_path('../NFTLoan/NFTLoan/bin/sc/NFTFlashLoan.nef')
test_nopht_d_hash = client.virutal_deploy_from_path('../NFTLoan/NophtD/bin/sc/TestNophtD.nef')
anyupdate_short_safe_hash = client.virutal_deploy_from_path('../AnyUpdate/AnyUpdateShortSafe.nef')

client.virutal_deploy_from_path deploys the .nef and .manifest.json to the snapshot of your Fairy session. The snapshot is similar to a fork of the current blockchain, named by your session string. You can now access to the deployed contract through your snapshot, but not through the actual blockchain.

In our case, we deployed AnyUpdate to be updated to any other contract, and test_nopht_d as a divisible NFT to be operated. Though you can continue deploying NFTLoan by yourself, we are now going to call AnyUpdate to perform all the actions the same as NFTLoan.

Step 4: Call your contracts!

By design, NFTLoan initializes its token ID to be 1. However, this is not performed by AnyUpdate. Therefore, we first ask AnyUpdate to prepare the storage environment:

client.invokefunction('putStorage', params=[0x02, 1])

Here we did not explicitly indicate the address of the called contract. This is because client.virutal_deploy_from_path has set client.contract_scripthash to be the address of the just deployed contract in the previous step.

Also notice that we should always put some string in client.fairy_session. If fairy_session is set to None, the client will (by my design) directly interact with the real blockchain, and write real transactions.

Step 5: The storage written by Fairy is always valid in the same snapshot

import json
manifest_dict = json.loads(manifest)
manifest_dict['name'] = 'AnyUpdateShortSafe'
manifest = json.dumps(manifest_dict, separators=(',', ':'))
print(
    client.invokefunction('anyUpdate', params=
    [nef_file, manifest, 'registerRental',
     [wallet_scripthash, test_nopht_d_hash, 68, 1, 5, 7, True]
    ]
    )
)

Extremely complex, huh? Not really.

In the first 4 lines we are changing the contract name of NFTLoan in its manifest. This is because the contract cannot change its name in an update. Then we are just calling AnyUpdate to update itself becoming our NFTLoan, and execute the method registerRental.

And happily we will see a lot of red alerts ending with:

[0x5c1068339fae89eb1a743909d0213e1d99dc5dc9] AnyUpdateShortSafe: Transfer failed

Whiskey Tango Foxtrot? Well, by reading the codes, we can assume that we have forgotten to add proper witnesses (in other words, signatures) to our call (we are not going to explain how to make the assumption for now). But how to add signatures?

Step 6: Adding signatures to your call

Signatures are important elements in Neo blockchain to check whether the operation is really allowed by stakeholders. In smart contracts, you should always check the witness of the token holder before transferring his/her tokens to someone else.

import json
manifest_dict = json.loads(manifest)
manifest_dict['name'] = 'AnyUpdateShortSafe'
manifest = json.dumps(manifest_dict, separators=(',', ':'))
from neo_fairy_client.utils import Signer, WitnessScope
signer = Signer(wallet_scripthash, scopes=WitnessScope.Global)  # watch this!
print(
    client.invokefunction('anyUpdate', params=
    [nef_file, manifest, 'registerRental',
     [wallet_scripthash, test_nopht_d_hash, 68, 1, 5, 7, True]
    ], 
    signers=signer)  # watch this!
)

For testing purposes, you can just use WitnessScope.Global to allow any contract the transfer the assets of wallet_scripthash freely. A good news is that Fairy does not actually check if your signatures are really signed by the wallet owner. You can use any scripthash in Signer (does not have to be the scripthash of the wallet), and Fairy will always recognize it to be a valid signature.

If everything goes well, your Fairy client should print:

Hello world! Your first contact with Fairy!::putStorage[2, 1] relay=True [{'account': '0xb1983fa2479a0c8e2beae032d2df564b5451b7a5', 'scopes': 'CalledByEntry', 'allowedcontracts': [], 'allowedgroups': [], 'rules': []}]
Hello world! Your first contact with Fairy!::anyUpdate[b'NEF3Neo.Compiler.CSharp 3.1.0\x00\x00\x00\x00...
68

Step 7: Cloning snapshots

By cloning snapshots, you are "forking the blockchain" from your old snapshot again. The written transactions in the old snapshot will be remembered in the new snapshot.

client.copy_snapshot('Hello world! Your first contact with Fairy!', 'Cloned snapshot')
client.fairy_session = 'Cloned snapshot'  # selecting the new snapshot

Now just select a snapshot continue to invoke more methods! Everything happening in the cloned snapshot will affect neither the real blockchain nor the old snapshot.

Last step: Understanding the errors and fixing the bugs

We are not going to continue with the cloned snapshots, but explain the red error information given by Fairy. Head to tutorial.py and comment out the line mentioned in Step 4:

# client.invokefunction('putStorage', params=[0x02, 1])

And run the whole tutorial. You'll see confusing errors like this:

Hello world! Your first contact with Fairy!::anyUpdate[b'NEF3Neo.Compiler.CSharp 3.1.0\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00...
{"jsonrpc":"2.0","method":"invokefunctionwithsession","params":["Hello world! Your first contact with Fairy!",true,"0x5c1068339fae89eb1a743909d0213e1d99dc5dc9","anyUpdate",[{"type":"ByteArray","value":"TkVG...
{'jsonrpc': '2.0', 'id': 1, 'result': {'script': 'ERcVEQBEDBSRizQfl9u4WOivUwoue4ci+vAEJwwUpbdRVEtW39Iy4OorjgyaR6I/mLEXwAwOcmVnaXN...
Traceback (most recent call last):
  File "C:/Users/RhantolkYtriHistoria/NEO/neo-test-client/tutorial.py", line 26, in <module>
    client.invokefunction('anyUpdate', params=
  File "C:\Users\RhantolkYtriHistoria\NEO\neo-test-client\neo_fairy_client\rpc\fairy_client.py", line 478, in invokefunction
    return self.invokefunction_of_any_contract(self.contract_scripthash, operation, params,
  File "C:\Users\RhantolkYtriHistoria\NEO\neo-test-client\neo_fairy_client\rpc\fairy_client.py", line 465, in invokefunction_of_any_contract
    result = self.meta_rpc_method(
  File "C:\Users\RhantolkYtriHistoria\NEO\neo-test-client\neo_fairy_client\rpc\fairy_client.py", line 224, in meta_rpc_method
    raise ValueError(result_result['traceback'])
ValueError:    at Neo.VM.ExecutionEngine.ExecuteInstruction(Instruction instruction) in C:\Users\RhantolkYtriHistoria\NEO\neo-vm\src\Neo.VM\ExecutionEngine.cs:line 1143
   at Neo.VM.ExecutionEngine.ExecuteNext() in C:\Users\RhantolkYtriHistoria\NEO\neo-vm\src\Neo.VM\ExecutionEngine.cs:line 1454
Invalid type for SIZE: Any
CallingScriptHash=0x5c1068339fae89eb1a743909d0213e1d99dc5dc9[AnyUpdateShortSafe]
CurrentScriptHash=0x5c1068339fae89eb1a743909d0213e1d99dc5dc9[AnyUpdateShortSafe]
EntryScriptHash=0xb493e9d3b67262ba1f35dfc85dbfe6464c83c092
   at Neo.VM.ExecutionEngine.ExecuteInstruction(Instruction instruction) in C:\Users\RhantolkYtriHistoria\NEO\neo-vm\src\Neo.VM\ExecutionEngine.cs:line 1143
   at Neo.VM.ExecutionEngine.ExecuteNext() in C:\Users\RhantolkYtriHistoria\NEO\neo-vm\src\Neo.VM\ExecutionEngine.cs:line 1454
InstructionPointer=3532, OpCode SIZE, Script Length=8518
InstructionPointer=3814, OpCode DUP, Script Length=8518
InstructionPointer=4574, OpCode STLOC3, Script Length=8518
InstructionPointer=502, OpCode STLOC2, Script Length=3194
InstructionPointer=21384, OpCode , Script Length=21384

Now pay attention to the InstructionPointer stacks at last, especially the first line of the InstructionPointers. By reading NFTFlashLoan.nef.txt (available in NFTLoan repository releases) created by Dumpnef, you'll get the following information near InstructionPointer=3532:

# Code NFTLoan.cs line 141: "ExecutionEngine.Assert(id.Length < 0xFD, "Too long id");"
3518 PUSHDATA1 54-6F-6F-20-6C-6F-6E-67-20-69-64 # as text: "Too long id"
3531 LDLOC2
3532 SIZE
3533 PUSHINT16 FD-00 # 253
3536 LT
3537 CALL_L 07-FB-FF-FF # pos: 2264, offset: -1273

And you can immediately locate the problem, finding that id is actually null and the operation id.Length is invalid.

Tutorial for debugging

Still feeling difficult to locate bugs in testing? Just debug the contract with step-in, step-out and step-over. Prepare your debugging storage environment automatically with your test codes, set breakpoints on either source code lines or InstructionPointers of assembly codes, and watch all the values of variables based on their names!

Step 0: Set debug info

Well... Thanks to the auto_set_debug_info=True option in client.virutal_deploy_from_path, Fairy has automatically registered the debug info of AnyUpdate and TestNophtD for you when you deployed them. But Fairy does not recognize the source codes of NFTLoan because we did not actually deploy it. Now we are going to set debug info manually.

with open('../NFTLoan/NFTLoan/bin/sc/NFTFlashLoan.nefdbgnfo', 'rb') as f:
    nefdbgnfo = f.read()
with open('../NFTLoan/NFTLoan/bin/sc/NFTFlashLoan.nef.txt', 'r') as f:
    dumpnef = f.read()
client.virutal_deploy_from_path('../NFTLoan/NFTLoan/bin/sc/NFTFlashLoan.nef', auto_set_debug_info=False)  # client.contract_scripthash is set
client.set_debug_info(nefdbgnfo, dumpnef)  # the debug info is by default registered for client.contract_scripthash

Step 1: Call a method in debug mode

Your debugging runtime storage environment is always inherited from a test session. It is recommended to build the debugging environment automatically with testing codes.

print(breakpoint := client.debug_function_with_session(  # do not invokefunction in debugging!
    'registerRental',
    params=[wallet_scripthash, test_nopht_d_hash, 68, 1, 5, 7, True],
    signers=None,  # Watch this! I wrote this on purpose
))

With signers=None your Fairy client uses CalledByEntry signature, which is actually insufficient in our case. You should get the following to be printed

Cloned snapshot::debugfunction registerRental
RpcBreakpoint VMState.FAULT ExecutionEngine.cs line 33 instructionPointer 2281: Assert(false);

which directly leads you to the source code! Now you can easily figure out the signature problems.

If no fault occurs, you should get VMState.HALT in breakpoint.

Note that all debugging executions write nothing to the snapshot!

Step 2: Set breakpoints; step-in, step-out, step-over, and watch variable values

Head to test_debug.py in this repo to learn these operations!