ethspecify 0.3.9

A utility for processing Ethereum specification tags.

ethspecify

A tool for referencing the Ethereum specifications in clients. This will:

• Help developers keep track of specification changes.
• Help auditors find important functions in client implementations.

Table of contents

• Getting Started
  • Install
  • Initialize
• Inline References
• Centralized References
  • Configure
  • Update
  • Map
  • Check
  • Exceptions
• Style Options
  • hash
  • full
  • link
  • diff

Getting Started

Install

pipx install ethspecify

Initialize

Create a .ethspecify.yml config file:

$ ethspecify init v1.6.0-beta.0
Successfully created .ethspecify.yml

To also generate a specrefs/ directory with a YAML file for each specification category, use the --specrefs flag:

$ ethspecify init v1.6.0-beta.0 --specrefs
Initializing specrefs directory: v1.6.0-beta.0
Successfully created .ethspecify.yml and specrefs/ directory

[!TIP] nightly is also a valid version, which tracks the latest development specifications.

Inline References

Add <spec> tags anywhere in the codebase to reference specification items. For example:

<spec fn="is_active_validator" fork="phase0" />

Then run ethspecify to populate the tag body with the corresponding specification content:

<spec fn="is_active_validator" fork="phase0" hash="5765e850">
def is_active_validator(validator: Validator, epoch: Epoch) -> bool:
    """
    Check if ``validator`` is active.
    """
    return validator.activation_epoch <= epoch < validator.exit_epoch
</spec>

[!NOTE] The indentation of the <spec> tag is preserved when populating the body. This means spec tags inside comments will have their content indented to match:

// <spec fn="is_active_validator" fork="phase0" hash="5765e850">
// def is_active_validator(validator: Validator, epoch: Epoch) -> bool:
//     """
//     Check if ``validator`` is active.
//     """
//     return validator.activation_epoch <= epoch < validator.exit_epoch
// </spec>

Centralized References

Specification references (specrefs) are YAML files that map specification items (constants, functions, containers, etc.) to their corresponding source file locations.

Configure

The following options can be set in the specrefs section of .ethspecify.yml:

Option	Default	Description
search_root	.	Root directory for resolving source file paths
auto_standardize_names	false	Rename entries to item#fork format
auto_add_missing_entries	false	Add missing specification items with empty sources
require_exceptions_have_fork	false	Require exceptions to use item#fork format

Update

To update to a newer specification version, change the version field in .ethspecify.yml and run:

$ ethspecify

This updates the specification content in the YAML files to match the new version.

Map

Edit the YAML files to add sources for where each specification item is implemented.

If it is the entire file:

- name: BlobParameters
  sources:
    - file: ethereum/spec/src/main/java/tech/pegasys/teku/spec/logic/versions/fulu/helpers/BlobParameters.java
  spec: |
    <spec dataclass="BlobParameters" fork="fulu" hash="a4575aa8">
    class BlobParameters:
        epoch: Epoch
        max_blobs_per_block: uint64
    </spec>

If it is multiple entire files:

- name: BlobsBundleDeneb
  sources:
    - file: ethereum/spec/src/main/java/tech/pegasys/teku/spec/datastructures/execution/BlobsBundle.java
    - file: ethereum/spec/src/main/java/tech/pegasys/teku/spec/datastructures/builder/BlobsBundleSchema.java
    - file: ethereum/spec/src/main/java/tech/pegasys/teku/spec/datastructures/builder/versions/deneb/BlobsBundleDeneb.java
    - file: ethereum/spec/src/main/java/tech/pegasys/teku/spec/datastructures/builder/versions/deneb/BlobsBundleSchemaDeneb.java
  spec: |
    <spec dataclass="BlobsBundle" fork="deneb" hash="8d6e7be6">
    class BlobsBundle(object):
        commitments: List[KZGCommitment, MAX_BLOB_COMMITMENTS_PER_BLOCK]
        proofs: List[KZGProof, MAX_BLOB_COMMITMENTS_PER_BLOCK]
        blobs: List[Blob, MAX_BLOB_COMMITMENTS_PER_BLOCK]
    </spec>

If it is a specific part of a file:

- name: EFFECTIVE_BALANCE_INCREMENT
  sources:
    - file: ethereum/spec/src/main/resources/tech/pegasys/teku/spec/config/presets/mainnet/phase0.yaml
      search: "EFFECTIVE_BALANCE_INCREMENT:"
  spec: |
    <spec preset_var="EFFECTIVE_BALANCE_INCREMENT" fork="phase0" hash="23dfe52c">
    EFFECTIVE_BALANCE_INCREMENT: Gwei = 1000000000
    </spec>

Regex is also supported in searches:

- name: ATTESTATION_DUE_BPS
  sources:
    - file: ethereum/spec/src/main/resources/tech/pegasys/teku/spec/config/configs/mainnet.yaml
      search: "^ATTESTATION_DUE_BPS:"
      regex: true
  spec: |
    <spec config_var="ATTESTATION_DUE_BPS" fork="phase0" hash="929dd1c9">
    ATTESTATION_DUE_BPS: uint64 = 3333
    </spec>

Check

Run the check command in CI to verify all specification items are properly mapped:

$ ethspecify check
MISSING: constants.BLS_MODULUS#deneb

Exceptions

Some specification items may not have a corresponding implementation. Add them to the exceptions list in .ethspecify.yml:

specrefs:
  exceptions:
    containers:
      # Not defined, unnecessary
      - Eth1Block

    functions:
      # No light client support
      - is_valid_light_client_header
      - process_light_client_update

Style Options

This attribute can be used to change how the specification content is shown.

hash

This style adds a hash of the specification content to the <spec> tag, without showing the content.

<spec fn="apply_deposit" fork="electra" hash="c723ce7b" />

[!NOTE] The hash is the first 8 characters of the specification content's SHA256 digest.

full

This style displays the whole content of this specification item, including comments.

<spec fn="is_fully_withdrawable_validator" fork="deneb" style="full">
def is_fully_withdrawable_validator(validator: Validator, balance: Gwei, epoch: Epoch) -> bool:
    """
    Check if ``validator`` is fully withdrawable.
    """
    return (
        has_eth1_withdrawal_credential(validator)
        and validator.withdrawable_epoch <= epoch
        and balance > 0
    )
</spec>

link

This style displays an ethspec.tools link to the specification item.

<spec fn="apply_pending_deposit" fork="electra" style="link" hash="83ee9126">
https://ethspec.tools/#specs/v1.7.0-alpha.1/functions-apply_pending_deposit-electra
</spec>

diff

This style displays a diff with the previous fork's version of the specification.

<spec ssz_object="BeaconState" fork="electra" style="diff">
--- deneb
+++ electra
@@ -27,3 +27,12 @@
     next_withdrawal_index: WithdrawalIndex
     next_withdrawal_validator_index: ValidatorIndex
     historical_summaries: List[HistoricalSummary, HISTORICAL_ROOTS_LIMIT]
+    deposit_requests_start_index: uint64
+    deposit_balance_to_consume: Gwei
+    exit_balance_to_consume: Gwei
+    earliest_exit_epoch: Epoch
+    consolidation_balance_to_consume: Gwei
+    earliest_consolidation_epoch: Epoch
+    pending_deposits: List[PendingDeposit, PENDING_DEPOSITS_LIMIT]
+    pending_partial_withdrawals: List[PendingPartialWithdrawal, PENDING_PARTIAL_WITHDRAWALS_LIMIT]
+    pending_consolidations: List[PendingConsolidation, PENDING_CONSOLIDATIONS_LIMIT]
</spec>

[!NOTE] Comments are stripped from the specifications when the diff style is used. This is because they complicate the diff; the "[Modified in Fork]" comments are not valuable here.

This can be used with any specification item, like functions too:

<spec fn="is_eligible_for_activation_queue" fork="electra" style="diff">
--- phase0
+++ electra
@@ -4,5 +4,5 @@
     """
     return (
         validator.activation_eligibility_epoch == FAR_FUTURE_EPOCH
-        and validator.effective_balance == MAX_EFFECTIVE_BALANCE
+        and validator.effective_balance >= MIN_ACTIVATION_BALANCE
     )
</spec>