PyBook 10: Bitcoin & Solana

What this notebook proves: "Bitcoin: xpub derivation, gap-limit scan, UTXO handling. Done when: one xpub produces the full derived balance set."

What this notebook proves: "Solana: SPL, Token-2022 (including ScaledUiAmount). Done when: SPL balances work offline against cassettes."

Two chains, two lessons the EVM never teaches:

Bitcoin has no accounts. A wallet is a tree of addresses; the balance is a scan. And the rule is absolute: "derive locally with BIP32, never send an extended key off-box."
Solana can lie to raw / 10**decimals. Token-2022's ScaledUiAmount extension puts a multiplier on the mint, so the node's displayed amount is not the raw amount rescaled. Both numbers are true; both are carried.

Everything replays committed cassettes, no network, no keys.

from pathlib import Path


def repo_root() -> Path:
    """The checkout this book lives in, wherever it is executed from."""
    here = Path.cwd().resolve()
    for candidate in (here, *here.parents):
        if (candidate / "tests" / "cassettes").is_dir():
            return candidate
    raise RuntimeError("run this book from inside a checkout of auradefi")


CASSETTES = repo_root() / "tests" / "cassettes"
print("cassettes:", CASSETTES.relative_to(repo_root()))

cassettes: tests/cassettes

The xpub never leaves the box

sources/bitcoin/xpub.py is a from-scratch BIP32 public-derivation implementation (validated against the published BIP32 test vectors) with no httpx import and no reference to the network module at all: a test asserts that mechanically, on the AST, because "we promise not to send it" is not a security property.

The host binds the key into a closure and hands the scanner a residual derive(chain, start, count) callable. The scanner receives addresses and never sees the extended key.

import functools

from auradefi.sources.bitcoin.xpub import derive_addresses, parse_xpub

# BIP32 test vector 1 master public key.
XPUB = (
    "xpub661MyMwAqRbcFtXgS5sYJABqqG9YLmC4Q1Rdap9gSE8NqtwybGhePY2gZ29ESFjqJoC"
    "u1Rupje8YtGqsefD265TMg7usUDFdp6W1EGMcet8"
)
parsed = parse_xpub(XPUB)
assert parsed.depth == 0 and len(parsed.chain_code) == 32 and len(parsed.pubkey) == 33
print("depth", parsed.depth, "| pubkey", parsed.pubkey.hex()[:20] + "…")

external = derive_addresses(XPUB, "p2wpkh", 0, 0, 3)   # chain 0 = receive
change = derive_addresses(XPUB, "p2wpkh", 1, 0, 1)     # chain 1 = change
assert external[0] == "bc1qp5wfcq48h6d63wyy9qz0awtpfqwwv4sma86mhz"
assert change[0] == "bc1q7zwtzcqsm3k43ha0ac7nl8cz0hqrhckywf6sew"
assert all(address.startswith("bc1q") for address in external + change)

derive = functools.partial(derive_addresses, XPUB, "p2wpkh")  # the key is now BOUND
for index, address in enumerate(external):
    print(f"m/0/{index}  {address}")

depth 0 | pubkey 0339a36013301597daef…
m/0/0  bc1qp5wfcq48h6d63wyy9qz0awtpfqwwv4sma86mhz
m/0/1  bc1qrfxr69jqnhwufxgkqgcdep9prq4j4vuw2wyg0v
m/0/2  bc1qhvd6suvqzjcu9pxjhrwhtrlj85ny3n2mqql5w4

The gap-limit scan: where does a wallet end?

BIP44's answer: keep deriving until 20 consecutive unused addresses appear. The stop rule is a fact about the request count, so the cassette pins it: exactly 44 recorded lookups.

chain 0: indices 0, 1, 2 are used, so the unused run starts at 3 and reaches 20 at index 22 → 23 requests;
chain 1: index 0 is used, the run reaches 20 at index 20 → 21.

Ask for one address more and the next URL is unrecorded, so CassetteMissError fires instead of a silently wrong balance. That is the point of recording the wire rather than mocking the function.

import httpx

from auradefi.sources.bitcoin.esplora import Esplora, scan
from auradefi.testing.cassettes import load

seen = []


def recording_client(cassette):
    def handler(request):
        seen.append(str(request.url))
        return cassette.handle(request)

    return httpx.Client(transport=httpx.MockTransport(handler))


result = scan(Esplora(recording_client(load(CASSETTES / "phase6_xpub.json"))), derive, gap=20)

assert len(seen) == 44
assert seen[:23] == ["https://blockstream.info/api/address/" + a
                     for a in derive_addresses(XPUB, "p2wpkh", 0, 0, 23)]
# Asserted against the recorded traffic rather than promised:
# no request carries a key, and the string "xpub" is nowhere in the file.
assert all(url.rsplit("/", 1)[-1].startswith("bc1") for url in seen)
assert "xpub" not in (CASSETTES / "phase6_xpub.json").read_text(encoding="utf-8")
print(len(seen), "address lookups, every one a bc1 address; the xpub appears in none of them")

44 address lookups, every one a bc1 address; the xpub appears in none of them

UTXO arithmetic: confirmed only, mempool ignored

A balance is funded_txo_sum − spent_txo_sum over chain stats. The mempool is deliberately excluded, so an unconfirmed transaction cannot move a reported balance. A used but swept address still appears, with a zero balance and its transaction count, because "this address is part of your wallet" is information a wallet needs even when it holds nothing.

from auradefi.money.quantity import Quantity

assert [(row.chain, row.index, row.balance_sats, row.tx_count) for row in result.addresses] == [
    (0, 0, 100_000_000, 3),
    (0, 1, 25_000, 1),     # the cassette also holds 7,777 sats in the mempool: ignored
    (0, 2, 0, 4),          # used but swept: still reported
    (1, 0, 999_000_000, 12),
]
assert result.total_sats == 1_099_025_000
assert result.total == Quantity(1_099_025_000, 8)
assert str(result.total) == "10.99025"
assert result.caip19 == "bip122:000000000019d6689c085ae165831e93/slip44:0"

for row in result.addresses:
    print(f"m/{row.chain}/{row.index}  {row.address}  {row.balance_sats:>11} sats  ({row.tx_count} tx)")
print(f"{'TOTAL':>10}  {'':38} {result.total_sats:>11} sats = {result.total} BTC")

m/0/0  bc1qp5wfcq48h6d63wyy9qz0awtpfqwwv4sma86mhz    100000000 sats  (3 tx)
m/0/1  bc1qrfxr69jqnhwufxgkqgcdep9prq4j4vuw2wyg0v        25000 sats  (1 tx)
m/0/2  bc1qhvd6suvqzjcu9pxjhrwhtrlj85ny3n2mqql5w4            0 sats  (4 tx)
m/1/0  bc1q7zwtzcqsm3k43ha0ac7nl8cz0hqrhckywf6sew    999000000 sats  (12 tx)
     TOTAL                                          1099025000 sats = 10.99025 BTC

from auradefi.errors import CassetteMissError

# gap=21 needs index 23 on the external chain, which was never recorded.
try:
    scan(Esplora(load(CASSETTES / "phase6_xpub.json").client()), derive, gap=21)
except CassetteMissError:
    print("gap=21 walks off the end of the recording: the 44-request count IS the stop rule")

gap=21 walks off the end of the recording: the 44-request count IS the stop rule

Solana: where `raw / 10**decimals` stops being true

The warning, made concrete. Three things happen in one balance call:

native SOL: lamports at 9 decimals;
SPL tokens: one getTokenAccountsByOwner per program; an owner can hold several accounts of the same mint, so they are summed by mint;
Token-2022 lives under a different program id, so one call cannot return both sets. The balance path is pinned at two calls, TOKEN_PROGRAM then TOKEN_2022_PROGRAM.

import json

from auradefi.sources.solana.rpc import TOKEN_2022_PROGRAM, TOKEN_PROGRAM, SolanaBalances, SolanaRpc

ADDRESS = "9wFFyRfZBsuAha4YcuxcXLKwMxJR43S7fPfQLXMFxbAF"
posts = []
cassette = load(CASSETTES / "solana_balances.json")


def solana_client():
    def handler(request):
        posts.append(json.loads(request.content))
        return cassette.handle(request)

    return httpx.Client(transport=httpx.MockTransport(handler))


rpc = SolanaRpc(solana_client())
balances = SolanaBalances(rpc).balances(ADDRESS)

assert [post["method"] for post in posts] == [
    "getBalance", "getTokenAccountsByOwner", "getTokenAccountsByOwner",
]
assert [post["params"][1]["programId"] for post in posts[1:3]] == [TOKEN_PROGRAM, TOKEN_2022_PROGRAM]
for balance in balances:
    print(f"{str(balance.quantity):>12}  ui={balance.ui_amount_string:<6} scaled={balance.scaled_ui}  {balance.caip19}")

         3.5  ui=3.5    scaled=False  solana:5eykt4UsFv8P8NJdTREpY1vzqKqZKvdp/slip44:501
        1000  ui=1000   scaled=False  solana:5eykt4UsFv8P8NJdTREpY1vzqKqZKvdp/token:EPjFWdd5AufqSSqeM2qN1xzybapC8G4wEGGkZwyTDt1v
           1  ui=2      scaled=True  solana:5eykt4UsFv8P8NJdTREpY1vzqKqZKvdp/token:ScaLedUiAmountMint22222222222222222222222222

The identity break

The Token-2022 account holds raw 1,000,000,000 at 9 decimals, exactly 1 by the usual identity, while the node reports uiAmountString of "2", because the mint carries a ScaledUiAmount multiplier of 2.

Both are kept: the exact Quantity for arithmetic, the node's string for display, and scaled_ui=True to say out loud that they diverge. An implementation that recomputed the display from the raw value would be wrong by 2x and would look completely normal doing it.

native, usdc, t22 = balances

assert str(native.quantity) == "3.5" and native.mint is None            # 3.5 SOL
assert usdc.quantity == Quantity(1_000_000_000, 6)                      # 250M + 750M summed
assert usdc.ui_amount_string == "1000" and usdc.scaled_ui is False

assert str(t22.quantity) == "1"          # raw / 10**decimals
assert t22.ui_amount_string == "2"       # what the mint says it displays as
assert t22.scaled_ui is True             # ...and the divergence is FLAGGED
assert t22.caip19.endswith("/token:ScaLedUiAmountMint22222222222222222222222222")  # base58 case preserved

print(f"Token-2022: raw/10^d = {t22.quantity}, node says {t22.ui_amount_string}, scaled_ui={t22.scaled_ui}")
print("two token accounts of one mint summed:", usdc.quantity, "USDC")

Token-2022: raw/10^d = 1, node says 2, scaled_ui=True
two token accounts of one mint summed: 1000 USDC

Paging and failure

Signature history pages until a page comes back short; the cassette pins the exact stop (2 POSTs for a limit of 2). And every RPC failure shape, JSON-RPC error member, HTTP 429, a non-JSON body, an envelope with no result, surfaces as one SourceError, never as None or a partial list.

from auradefi.errors import SourceError, ValidationError

signatures = rpc.get_signatures(ADDRESS, limit=2)
assert [signature.signature for signature in signatures] == ["SigNewest1", "SigErr2", "SigLast3"]
assert signatures[1].failed is True          # err -> failed
assert signatures[2].block_time is None      # a null blockTime stays null
assert posts[3]["params"][1] == {"limit": 2}
assert posts[4]["params"][1] == {"limit": 2, "before": "SigErr2"}
assert len(posts) == 5
print("signatures:", [(s.signature, s.slot, s.failed) for s in signatures])

# A malformed address is refused BEFORE any HTTP.
attempted = []
guard = SolanaRpc(httpx.Client(transport=httpx.MockTransport(
    lambda request: attempted.append(request) or httpx.Response(200))))
try:
    SolanaBalances(guard).balances("not-base58-0OIl")
except ValidationError as exc:
    print("refused pre-flight:", exc)
assert attempted == []

errors = SolanaRpc(load(CASSETTES / "solana_rpc_errors.json").client())
try:
    errors.get_balance(ADDRESS)
except SourceError as exc:
    print("rpc error surfaces as SourceError:", exc)

signatures: [('SigNewest1', 250000200, False), ('SigErr2', 250000100, True), ('SigLast3', 250000000, False)]
refused pre-flight: Solana address must be 32..44 base58 chars: 'not-base58-0OIl'
rpc error surfaces as SourceError: solana rpc getBalance error: code=-32602 message='Invalid params: unable to parse pubkey'

Honest edges

Bitcoin

Address kinds: p2wpkh (bech32) derivation is what ships.
One backend: Esplora (blockstream.info shape). No Electrum, no full-node RPC.
Public derivation only, no hardened paths, no private keys, ever.
Balances come from address chain_stats; UTXO listing exists but there is no coin selection or spending path.

Solana

Balances and signature history. Transaction decoding for Solana is not implemented: the rich parts[] model in 07_transactions is EVM-only today.
Prices for SPL mints are not wired into the oracle.

Absent entirely: Cosmos, and every EVM chain beyond the ones the registry seeds.

Next: 11_accounting. What all this cost you.