What a paper trade is
A paper trade is a pretend trade. The bot decides, records the price it would have paid and the price it would have sold at, and counts the result as if the trade had happened. No order is sent, so nothing tests whether the price was real. That makes the price the one thing that has to be right.
What happened
Our crypto paper bot trades Kalshi's 15-minute Bitcoin markets, which ask whether Bitcoin's average over the final minute will finish above a strike price. Each second it compares its own estimate of fair value, worked out from Bitcoin's live price, with the market's bid and ask, and decides whether to buy.
Until Oct 29, 1:48 AM ET, the bid and ask it used came from Kalshi's market list — the call that lists markets — and not from the order book. We read the list every second, but the prices in it changed only about every 33 seconds. Over ten days of recordings (Oct 18–27) the gap between changes was a median of 33 seconds, and on every one of those days the 10th to 90th percentile was 31 to 36 seconds. The recorded bid and ask changed about 1.6 times a minute; Bitcoin's price changed 25 to 37 times a minute. So the bot's estimate of fair value was never more than a second old, while the price it was compared with was, on average, about 17 seconds old.
A comment in our own code said the list's prices and the order book were “the same book a few milliseconds apart.” Nobody had measured it.
How we found it
Not by looking at the strategy's results. We checked the data the results were built on. In a first pass over the recordings, the quoted price almost never moved in the five seconds after a simulated fill, which is odd for a market that trades all day. So we measured how often the recorded price changed at all, and set it beside how often Bitcoin's did. The answer was the same on every day: about every 33 seconds.
A result can look fine and still be built on a price nobody could have traded at. The only way to know is to check where the price came from and how old it was.
What changed
- The crypto bot reads the order book every second and acts only on a fresh quote. A quote has to come from the book, have been read within the last 3 seconds (the default limit), have both a bid and an ask to buy on, and not be crossed. A quote that fails any of that — a book that could not be read, one older than 3 seconds, one-sided or crossed, or a price taken from the market list instead — is recorded with the reason (“stale quote”, “one-sided book”, “no book quote” or “crossed book”) and never used to open a trade.
- Older results are labelled and set apart. Every paper result priced before Oct 29, 1:48 AM ET says “priced on a delayed snapshot — not tradable”, and none of them is in the record that judges the strategy. They are kept, not deleted.
- A data-health line every day. How many readings were on a fresh two-sided book, how old the quotes were, how many book reads failed, and whether any stretch went unrecorded. A day that falls short is marked as a data fault, and the rule is to judge nothing from a day with a fault.
- The sports bot too. Its price ticks are now read from the order book, and a tick that could only be read from the market list is recorded but never starts an entry; the next poll's book read decides. When we checked the earlier ticks, the list's quote matched the book's best prices on 26,661 of 39,332 ticks (67.8%, Oct 29 – Nov 1) — about a third of the quotes the bot saw were not what the book showed. Older sports results are marked “unverified prices (a third of quotes didn't match the book)”.
What this does and does not show
It shows that a paper result is only as good as its prices, and that ours were not good enough until we checked. It does not show whether the strategy works. Results priced on a delayed snapshot cannot say, in either direction, and the record on the order book starts at that line, so it begins short; wherever it appears it shows how many trades and days it covers. This page gives no profit, win rate or other result for either bot, on purpose.
The lesson for any record
Before you read a track record, a backtest or a paper-trading result, ask where the prices came from, how old they were, and whether anyone could have traded at them. It is the question underneath the six in How to read an AI stock-picks track record, and it applies to ours first.
The daily stock picks are not affected by any of this. They are graded from stored daily closing prices, 5 and 21 trading days after the pick day, against the S&P 500, and the track record checks its own inputs every night and shows how many alarms are open.