PyBook 11: Accounting

What this notebook proves: "cost basis, lot tracking, FIFO/LIFO/HIFO/ACB, realised & unrealised PnL, arbitrary-date PnL. Done when: arbitrary-date PnL works on a large fixture."

The incumbent limitation this exists to beat:

"PnL is pre-computed at standard marks … other values are supported only if fewer than 3,000 transactions sit between your timestamp and the nearest mark: otherwise the request errors out."

auradefi does not pre-compute marks. It replays lots, so any instant is answerable, and it does so with exact rational arithmetic. Fraction internally, rounding only at the Fraction → Money boundary, and flagged when it happens.

accounting/ is completely pure: no I/O, no HTTP, no clock. An event's time comes from the transaction that produced it, never from now(). That is precisely what makes arbitrary-date PnL replayable.

from decimal import Decimal

from auradefi.accounting.lots import AcquisitionEvent, DisposalEvent
from auradefi.accounting.pnl import METHODS, pnl_at, process
from auradefi.accounting.report import report
from auradefi.money.fiat import Money
from auradefi.money.quantity import Quantity

ETH = "eip155:1/slip44:60"
DAY = 86_400_000
T0 = 1_700_000_000_000


def usd(amount):
    return Money(Decimal(amount), "USD")


def units(count):
    return Quantity(count, 0)


# Three buys at different prices, then one sale of a single unit.
EVENTS = (
    AcquisitionEvent(T0 + 0 * DAY, ETH, units(1), usd("10"), "txn_buy_1"),
    AcquisitionEvent(T0 + 1 * DAY, ETH, units(1), usd("30"), "txn_buy_2"),
    AcquisitionEvent(T0 + 2 * DAY, ETH, units(1), usd("26"), "txn_buy_3"),
    DisposalEvent(T0 + 3 * DAY, ETH, units(1), usd("40"), "txn_sell_1"),
)
assert sorted(METHODS) == ["acb", "fifo", "hifo", "lifo"]
print("methods:", sorted(METHODS))
print("bought at 10, 30, 26: then sold ONE unit for 40")

methods: ['acb', 'fifo', 'hifo', 'lifo']
bought at 10, 30, 26: then sold ONE unit for 40

One trade history, four legal answers

Which unit did you sell? The tax code, not the blockchain, decides:

method	picks	jurisdictional home
FIFO	the oldest lot	the default nearly everywhere
LIFO	the newest lot	permitted in some US filings
HIFO	the highest-cost lot	minimises the gain
ACB	the pooled average cost	mandatory in Canada

Same events, same proceeds, four different realised gains. A tool that hard-codes one of these is wrong for most of its users: silently.

MARKS = {ETH: usd("50")}
AT = T0 + 3 * DAY

results = {method: report(process(EVENTS, method), AT, MARKS) for method in ("fifo", "lifo", "hifo", "acb")}

assert results["fifo"].realized == usd("30")   # 40 - 10 (oldest)
assert results["lifo"].realized == usd("14")   # 40 - 26 (newest)
assert results["hifo"].realized == usd("10")   # 40 - 30 (dearest)
assert results["acb"].realized == usd("18")    # 40 - 22 (pool average of 10, 30, 26)

print(f"{'method':<6} {'realized':>10} {'unrealized':>12}  cost basis of the unit sold")
for method, result in results.items():
    basis = usd("40") - result.realized
    print(f"{method:<6} {str(result.realized):>10} {str(result.unrealized):>12}  {basis}")

method   realized   unrealized  cost basis of the unit sold
fifo       30 USD       44 USD  10 USD
lifo       14 USD       60 USD  26 USD
hifo       10 USD       64 USD  30 USD
acb        18 USD       56 USD  22 USD

Why the answers differ, lot by lot

The open lots after the sale show the mechanism. FIFO consumed the 10-dollar lot, HIFO the 30-dollar one, and ACB left the lots untouched. The pool is a costing overlay and the lots remain ground truth for lot reporting.

Every lot is projected in Plaid's tax_lots shape: an institution_lot_id, an original_purchase_datetime in ms epoch, a Decimal quantity, and position_type. That is what makes crypto merge with brokerage data downstream instead of living in its own schema.

for method in ("fifo", "hifo", "acb"):
    lots = results[method].open_lots
    print(f"{method:<5} open lots: " + " ".join(
        f"{lot.quantity} unit @ {lot.cost_basis}" for lot in lots))

lot = results["fifo"].open_lots[0]
assert lot.institution_lot_id.startswith("lot_") and len(lot.institution_lot_id) == 20
assert lot.position_type == "LONG"
assert isinstance(lot.original_purchase_datetime, int)  # ms epoch
assert isinstance(lot.quantity, Decimal)
assert lot.current_value == usd("50")                   # 1 unit at the mark
print("\nPlaid tax_lot shape:", lot)

fifo  open lots: 1 unit @ 30 USD 1 unit @ 26 USD
hifo  open lots: 1 unit @ 10 USD 1 unit @ 26 USD
acb   open lots: 1 unit @ 30 USD 1 unit @ 26 USD

Plaid tax_lot shape: TaxLot(institution_lot_id='lot_78401758fa9d8a38', original_purchase_datetime=1700086400000, quantity=Decimal('1'), purchase_price=Money(amount=Decimal('30'), currency='USD'), cost_basis=Money(amount=Decimal('30'), currency='USD'), current_value=Money(amount=Decimal('50'), currency='USD'), position_type='LONG', flags=())

Arbitrary-date PnL: no marks, no windows, no error

pnl_at(events, method, at_ms, marks) filters the stream to the cutoff and replays it. Ask at any instant: the day before the sale, one millisecond before it, a year later. Nothing is pre-computed, so nothing has to be near a mark.

The process() path keeps an incremental state instead, for hosts that report continuously, and the two paths must not be able to disagree, which is asserted below.

before_sale = pnl_at(EVENTS, "fifo", T0 + 3 * DAY - 1, MARKS)
assert before_sale.realized == usd("0")            # exact zero, not "no data"
assert before_sale.missing_realized_count == 0
assert len(before_sale.open_lots) == 3
assert before_sale.unrealized == usd("84")         # 3 x 50 - (10 + 30 + 26)

after_sale = pnl_at(EVENTS, "fifo", AT, MARKS)
assert after_sale.realized == usd("30")
assert after_sale == results["fifo"]               # replay == incremental, exactly

a_year_later = pnl_at(EVENTS, "fifo", AT + 365 * DAY, MARKS)
assert a_year_later.realized == after_sale.realized
assert a_year_later.as_of_ms == AT + 365 * DAY     # the answer is stamped, not the data

for label, result in (("1 ms before the sale", before_sale), ("at the sale", after_sale),
                      ("a year later", a_year_later)):
    print(f"{label:<22} realized={str(result.realized):>8}  unrealized={str(result.unrealized):>8}  open lots={len(result.open_lots)}")

1 ms before the sale   realized=   0 USD  unrealized=  84 USD  open lots=3
at the sale            realized=  30 USD  unrealized=  44 USD  open lots=2
a year later           realized=  30 USD  unrealized=  44 USD  open lots=2

Missing data propagates as `None`, never as zero

A disposal with no known proceeds, or a lot with no known cost, does not quietly become a zero-dollar gain. realized is None for that disposal, the report counts it in missing_realized_count, and unrealized collapses to None for the whole report if any held asset lacks a mark or a complete basis (docs/internal/DECISIONS.md, "None-propagation (PnL)"). A partial sum that looks like a complete answer is the failure mode being designed out.

unpriced = EVENTS + (
    AcquisitionEvent(T0 + 4 * DAY, ETH, units(1), None, "txn_airdrop"),   # basis unknown
    DisposalEvent(T0 + 5 * DAY, ETH, units(1), None, "txn_sent_out"),     # proceeds unknown
)
gappy = pnl_at(unpriced, "fifo", T0 + 6 * DAY, MARKS)

assert gappy.missing_realized_count == 1     # the proceeds-less disposal is COUNTED
assert gappy.realized == usd("30")           # ...and excluded from the sum, not zeroed
assert gappy.unrealized is None              # one basis-less lot poisons the total, loudly
print("realized:", gappy.realized, "| disposals with unknown realised amount:", gappy.missing_realized_count)
print("unrealized:", gappy.unrealized, "(one lot has no cost basis)")

# A mark in the wrong currency is refused rather than silently mixed.
from auradefi.errors import CurrencyMismatchError

try:
    report(process(EVENTS, "fifo"), AT, {ETH: Money(Decimal("50"), "EUR")})
except CurrencyMismatchError as exc:
    print("refused:", exc)

realized: 30 USD | disposals with unknown realised amount: 1
unrealized: None (one lot has no cost basis)
refused: mark for 'eip155:1/slip44:60' is 'EUR', report is 'USD'

Self-transfers are not income

Moving your own coins between your own wallets is not a taxable event, but on-chain it looks exactly like a send followed by a receive. derive_events therefore drops Direction.SELF entries, anything listed in internal_transfer_ids, and every reorg-removed transaction. Without that switch, every self-transfer reads as income and every tax report is wrong.

from auradefi.accounting.lots import derive_events
from auradefi.ledger.models import Direction, Entry, LedgerTransaction

def txn(txn_id, direction, raw, removed=False):
    return LedgerTransaction(
        id=txn_id, chain_id="eip155:1", tx_hash="0x" + txn_id[-2:] * 32,
        account_id="acct_1", block_number=100, initiated_at=T0, confirmed_at=T0,
        entries=(Entry(asset_id=ETH, quantity=Quantity(raw, 18), direction=direction),),
        removed=removed,
    )


rows = [
    txn("txn_aa", Direction.IN, 10**18),
    txn("txn_bb", Direction.SELF, 10**18),      # wallet-to-wallet: never an event
    txn("txn_cc", Direction.OUT, 10**18),       # a real disposal...
    txn("txn_dd", Direction.OUT, 10**18),       # ...and one the host KNOWS is internal
    txn("txn_ee", Direction.IN, 10**18, removed=True),   # orphaned by a reorg
]
events = derive_events(rows, internal_transfer_ids=frozenset({"txn_dd"}))

assert [event.source_tx_id for event in events] == ["txn_aa", "txn_cc"]
assert isinstance(events[0], AcquisitionEvent) and isinstance(events[1], DisposalEvent)
print("5 ledger rows ->", len(events), "taxable events:", [e.source_tx_id for e in events])

5 ledger rows -> 2 taxable events: ['txn_aa', 'txn_cc']

Scale: 50,000 events, three arbitrary cutoffs

The gate runs a generated 50,000-event stream through FIFO and LIFO and pins the totals at three arbitrary instants, including two chosen where the two methods disagree, which is what proves the method is genuinely plugged in at an arbitrary date rather than one hard-coded algorithm wearing four names. The abbreviated version below reproduces that disagreement.

# 2,000 pairs of (buy 2 @ alternating cost, sell 1 @ 15) on one asset.
CHEAP, DEAR, SALE = usd("20"), usd("24"), usd("15")
stream = []
for index in range(2_000):
    slot = T0 + index * 60_000
    cost = CHEAP if index % 2 == 0 else DEAR
    stream.append(AcquisitionEvent(slot, ETH, units(2), cost, f"txn_b{index:05d}"))
    stream.append(DisposalEvent(slot + 30_000, ETH, units(1), SALE, f"txn_s{index:05d}"))

cutoff = T0 + 1_999 * 60_000 + 30_000
fifo = pnl_at(stream, "fifo", cutoff, MARKS)
lifo = pnl_at(stream, "lifo", cutoff, MARKS)

assert fifo.realized == usd("8000") and lifo.realized == usd("8000")
# FIFO drains its oldest lot every second pair, so half its lots close;
# LIFO always sells the unit it just bought, so nothing ever closes.
assert len(fifo.open_lots) == 1_000
assert len(lifo.open_lots) == 2_000
assert fifo.per_asset[ETH].quantity_held == units(2_000)
print(f"4,000 events -> realized {fifo.realized}; open lots: fifo {len(fifo.open_lots)}, lifo {len(lifo.open_lots)}")

# Mid-stream, the two methods have realised DIFFERENT totals, which is
# what proves the cutoff and the method are both really in play.
mid = T0 + 1_001 * 60_000 + 30_000
mid_fifo = pnl_at(stream, "fifo", mid, MARKS).realized
mid_lifo = pnl_at(stream, "lifo", mid, MARKS).realized
assert mid_fifo.amount - mid_lifo.amount == Decimal("2")
print(f"at an arbitrary mid-stream instant: fifo {mid_fifo} vs lifo {mid_lifo}")

4,000 events -> realized 8000 USD; open lots: fifo 1000, lifo 2000
at an arbitrary mid-stream instant: fifo 4010 USD vs lifo 4008 USD

A documented caveat you should know about

Under ACB, report.unrealized subtracts the pool's cost, while each TaxLot.cost_basis reports that lot's own remaining basis. The two will not agree, and that is deliberate: the pool is what ACB actually costs with, and the lots remain ground truth for lot-level reporting (docs/internal/DECISIONS.md, "ACB pooling").

The report says which one it used. basis_source is "pool" under ACB and "lots" everywhere else, and unrealized_basis and open_lots_basis expose both figures, so the gap is a number you can read rather than a discrepancy you have to reverse-engineer and mistake for a bug.

acb = results["acb"]

# Read BOTH bases off the report; no hand arithmetic, so this cell cannot
# drift from the engine the way a restated constant would.
assert acb.basis_source == "pool"
assert acb.unrealized_basis == usd("44")   # the pool: 66 bought - 22 consumed
assert acb.open_lots_basis == usd("56")    # the lots: 30 + 26, untouched
assert acb.unrealized == usd("56")         # 2 units x 50 - 44, i.e. the POOL

# Every lot method costs from the lots, so there the two figures agree.
for method in ("fifo", "lifo", "hifo"):
    assert results[method].basis_source == "lots"
    assert results[method].unrealized_basis == results[method].open_lots_basis

print(f"acb  basis_source={acb.basis_source}  pool={acb.unrealized_basis}  "
      f"lots={acb.open_lots_basis}  -> unrealized={acb.unrealized}")
print("fifo basis_source=" + results["fifo"].basis_source
      + f"  both={results['fifo'].unrealized_basis}")

acb  basis_source=pool  pool=44 USD  lots=56 USD  -> unrealized=56 USD
fifo basis_source=lots  both=56 USD

Honest edges

Marks are the caller's. There is no historical price service, so unrealized is only as good as the marks you pass; without one for a held asset the answer is None, never a guess.
Internal transfers are declared, not detected. You pass internal_transfer_ids; nothing infers them from address ownership.
No wash-sale rules, no jurisdiction-specific superficial-loss handling, no fee capitalisation into basis, and no form generation. This is a lot engine, not a tax filing.
position_type is always LONG; short positions surface through the synthetic-holdings projection (08_positions), not here.

Next: 12_http_api: the same library, over HTTP.