Learning MegaGem, from self-play to price discovery

The 30-gem deck contains six gems of each color, and the terminal Value Display is the 15 gems dealt into players’ hands. Considering your 5 gems and the 2 revealed publicly, 23 gems remain unseen and the opponents’ 10 cards are drawn from them. Let $N_R$ and $N_Y$ be the display counts for Red and Yellow respectively at the end of the game, and $V$ the corresponding value dictated by the value chart A, which pays 4 points per displayed gem up to 5. Then,

Winning the Blue–Green auction changes your collection to Red–Red–Blue–Blue–Green, completing two of the missions for 25 points total. Factoring in the blue and green gems from the lot and their values on the display, we know that winning this lot is worth at least $25+2\cdot4+2\cdot4=41$. Because neither Alice nor Bob has more than 12 coins and you are first in tiebreak order, you can win the auction for 12 and earn at least $41-12=29$ in positive EV. Their lack of liquidity lets you take the auction much more cheaply.

Because you are last in tiebreak order, matching Alice’s bid is a losing action: to clear a predicted bid of $b$, you must bid $b+1$. Alice can bid at most 12 and Bob at most 6, so 13 is the minimum bid to guarantee the lot. However, this increase must still be weighed against the gem’s value and the option value of keeping that coin for the remaining auctions because it may make the lot no longer worth bidding on.

Both cards enter the value display regardless, so revealing Red or Purple does not change final gem values, but it changes what other players are willing to pay before then. Revealing Purple moves the current quote from 8 to 12. Bob already owns Yellow gems, so another Purple would also complete the 5-point Pendant mission. This reveal raises his visible reservation value for a future Purple from roughly 13 to 17 which can be useful if the goal is to induce Bob to spend his six coins, but it also makes the Purple harder to buy or deny, especially when you have only two coins and act last in ties.

Revealing Red has the opposite advantage. Moving Red’s quote from 4 to 8 has a null effect on public perception because there are no more red gems to be auctioned off (all 6 are either in a collection or will be in the value display). It also keeps Purple quoted at 8 while your hidden Purple guarantees that its terminal supply is higher than the table can see. This usually makes Purple easier to snipe later (for example, to block Bob’s pendant). Revealing Red does advertise more of your lead (making you a bigger threat to other players who want to block you), but its direct price impact is confined to a market that has already closed.

The 20 borrowed coins are matched by a 20-point repayment at scoring, so the principal itself creates no terminal value. Winning the loan for 10 changes your available cash to $C_{\mathrm{after}}=2+20-10=12$, but terminally you are 10 coins poorer. For example, if you win and have 14 coins, you can guarantee winning the last two auctions by first paying 7 (beating Bob in the tiebreak) and 7 again (winning moves you last in the tiebreak while Bob’s liquidity remains 6).

Of the five cards still hidden, two are Purple, which puts a Purple gem’s expected value at $0.1(12)+0.6(16)+0.3(20)=16.8$.

With 2 coins, under rational play by your opponents, you would never be able to win the last two auctions. But for the reasons explained above, the loan gives you an out to win these last two, and because the EV suggests a positive trade, it is worth winning the loan auction.

Before the final lot, terminal-value accounting puts you at 77, Alice at 75, and Bob at 63. The Purple gem is worth 16 on Chart A and it would also complete Bob’s 5-point Pendant mission, but Bob has only 1 coin. Playing it out, 5 is the smallest bid to cover both Bob and Alice, allowing 11 in free EV.

Chess, Go, and heads-up poker are standard examples. Due to 0-sum, a value function can describe the expected outcome from a position for either player, with the sign reversed for the opponent. Think of the evaluation board on online chess.

You must assume the opponent chooses whatever hurts your position the most. The natural response is to choose the strategy with the best worst case, the maximin. The opponent does the converse, minimizing the maximum value you can obtain, the minimax. Von Neumann’s minimax theorem says these values are equal. If neither player can improve by deviating alone, the joint policy is in Nash equilibrium. In a 2p0s game, its equilibrium strategies are also minimax.

Ordinary policy-gradient self-play can still cycle (e.g. rock-paper-scissors), so practitioners often average policies or train with fictitious self-play, policies designed for exploitation, and more diverse policies. These mechanisms can stop the fresh policy from forgetting old weaknesses.

General-sum games remove the requirement that the players’ payoffs add to a constant in every outcome.

1. In mixed-motive games, interests are neither perfectly opposed nor perfectly aligned. The prisoner’s dilemma is the classic example: both players prefer cooperating to mutual defection, but each is incentivized to deviate. Settlers of Catan has the same flavor through beneficial trades, temporary coalitions, and coordinated attempts to block players.
2. In fully cooperative games, every player receives the same payoff. Self-play can still learn arbitrary private conventions that fail when a new partner has learned a different convention. Hanabi is the standard example: two policies can each be effective with copies of themselves yet coordinate poorly with one another.

If you’re interested in the cooperative side of multi-agent RL, Arturo wrote a great hands-on account of training LLM agents to play Hanabi, including the communication conventions that emerge under self-play.

The additional seat changes what “best play” means. Suppose Alice and Bob are close to first while Carol is too far behind to catch either. Carol may still decide the winner by bidding up one leader or letting the other buy cheaply, even when both choices leave her own score almost unchanged. A policy-gradient update based on Carol’s reward has little basis for preferring one.

Even an ordinary bid depends on who is sitting across the table. One coin is enough to beat a player who always bids 0; it is useless against an opponent likely to bid higher. There is no opponent-independent optimal bid.

We can still ask whether a player would gain by deviating from a specified set of policies, but that is a local question. Here, “improvement” means improvement against a particular opponent distribution, equilibrium selection, or training population. A self-play reward describes that population rather than providing a universal ranking of policies.

Comparing policies on the same deal removes most of the noise from hand luck, but subtler problems remain.

During play, a seat sees its private hand and the public history, not the full deal. That view represents many hidden worlds that are still possible, with each bid or reveal changing their relative plausibility. A poker four-bet works the same way: the cards remain hidden, but stronger hands become more likely. MegaGem is therefore a partially observable Markov decision process (POMDP). A board snapshot alone is not enough to value a position; we also need the player’s beliefs about what remains hidden.

Sealed bids add another unknown. To evaluate a candidate bid $b$, we need $P(\text{win}\mid b)$, the chance that it clears both opponents. Raising the bid improves that chance but leaves less surplus if it wins.

Reveals have an unusual role in this game. The initial deal fixes the terminal gem values because every private gem eventually enters the public Value Display. Choosing which gem to show does not alter that final display, but it does alter what the table believes before the remaining auctions.

The auction result carries information too. If both opponents decline to pay your price for a lot with shared value, you may have overestimated it—the auction version of the winner’s curse. That signal is weaker when private missions matter, since the same lot can reasonably be worth different amounts to different seats.

We choose the teachers on two criteria:

• skill: The initial screen, run before the final benchmark ranking, compares Gemini 3 Pro, Gemini 3 Flash, and Claude Opus 4.5. A full-field score can conflate strategic skill with exploiting weaker opponents, so we also run a 24-game-per-model probe with three copies of each model. Clone-game scores also reflect how a policy changes total surplus, so we use this result only for screening.
• length and cost: Gemini 3 Flash is about fourteen times cheaper per game than Claude Opus. A blend supplies enough long rationales to diversify the data without making every game Opus-priced.

For production collection, we keep Gemini 3 Flash and replace the screened Claude Opus 4.5 with its newer Claude Opus 4.6 successor.

Keeping only the winner’s traces would waste data and select the luckiest copy of an otherwise identical policy. It would also remove the lower liquidity and slower tempo states that Qwen3-4B-Instruct needs to handle. So instead, we consider the top two models’ traces from each game while removing unparseable or illegal turns (only 10 training turns fail those validity gates). The final split contains:

Checkpoint selection uses a fixed 6-model panel on the 10 validation deals. Qwen faces two copies of each opponent and occupies all three seats, for 180 panel games, plus 10 self-play games.

The self-play mean is only a sanity check, since all three copies co-evolve, but it does show that the model hasn’t degenerated.

All configurations share the same DAPO-style clipped objective; the labels below describe what changed in the training pool or schedule. Snapshots are lagged copies of the learner, and SFT anchor is the frozen step-0 policy. Both snapshots-plus-Flash rows replicate the same basic pool. In the frontier-API pool, the opponents are Gemini 3.1 Pro, Claude Opus 4.6, and Claude Sonnet 4.5; the 400-step run doubles the usual training horizon; seat rotation moves the learner through all three seats; and the final mixed pool combines the SFT anchor, lagged snapshots, and a decaying scripted-heuristic share (abbreviated in the figure as “heterogeneous”).

The heuristic contrasts use 60 matched seed groups with eight samples each, except the 24-game seat-rotation diagnostic. Some of the gains against the script get large (up to +16 points) while nothing moves against frozen SFT: handed a fixed, exploitable opponent, the optimizer happily learns a narrow exploit.

For player $i$, playing against $j$ and $k$, define competitive margin as $m_i=s_i-\frac{s_j+s_k}{2}$. The terminal reward is

We calibrate the scale 19.6 so the median absolute margin maps to 0.5 via $\tanh\left(\frac{10.75}{19.6}\right) \approx 0.5$, which compresses extreme wins and losses while keeping their order. A shaping channel weighted at 0.01 adds intermediate credit from reconstructed public-score changes between rounds, plus a terminal correction that aligns the channel’s cumulative accounting with the final margin. Illegal actions receive $-0.5$.

Let $r_u$ denote the terminal, legality, or shaping reward assigned at turn $u$. For each trainable turn $q$, define the return $G_q=\sum_{u\ge q}r_u$ and subtract an exponential-moving-average baseline $b_q$ indexed by game phase and seat role. Let $\mathcal G$ contain all trainable turns from the eight sibling games. Then, we define the advantage here as

Only turns from the trainable seat receive gradients.

We sample at temperature 1.0 and top-$p=0.95$, then update a rank-32 LoRA with learning rate $10^{-5}$ and KL coefficient 0.01.

The SFT-anchor mixed-pool run completes 100 optimizer steps with 192 games per rollout generation. KL, clipping, reward ordering, and parse rates all look healthy, so the flat anchor is unlikely to be a broken run.

A separate problem is assigning a game’s outcome to the individual decisions inside it.

trajectory-level credit

GRPO is easy to wire up here because it avoids a separate critic, but the statistics are a poor fit. A game lasts roughly 15–25 rounds of interleaved bids, reveals, loans, and investments, with most reward arriving at the end. So an early bid and a late reveal can inherit nearly the same verdict even though either may have changed every state that follows, and same-deal comparisons remove hand luck, not this within-trajectory credit problem. Denser rewards, process supervision, or much larger groups might change this picture, but testing them was out of scope for these runs.

state-value probe

For each fixed board $b$, let $\bar Y_b$ and $s_b^2$ be the sample mean and within-board variance of its rollout returns. Because the observed variance of board means includes finite-sample noise, we estimate the true between-board variance as

with within-board variance $\hat\sigma^2_{\text{trajectory}}=\mathbb E_b[s_b^2]$. In the probe, 99.8% of the estimated variance lies among trajectories from the same board, while only 0.2% lies between corrected board means. And the within-board term isn’t a clean action signal either, since it mixes action effects with continuation noise. A state-value critic must learn the much smaller between-board component from labels dominated by within-board variation.

Under relative-margin self-play, all three copies benefit from the same favorable deal, which suppresses most common board advantage. The fixed-heuristic probe allows repeated outcomes to recover more of the position’s value.

State value $V(I)$ looks too noisy under these relative-margin labels. Action value $Q(I,a)$ is the more plausible target because it compares bids within the same state.

rollout-$Q$ distillation with piKL

The first plausible fix is a piKL-style target, inspired by Meta’s work on Diplomacy, a seven-player game outside the 2p0s setting. It builds a search distribution that favors bids with higher action value $Q(I,a)$ while paying a KL penalty for moving too far from the SFT policy. Here, $Q(I,a)$ means the expected outcome after taking bid $a$ from information state $I$ and playing the game to completion.

We estimate $Q$ by simulation. The acting player cannot see the opponents’ hands or the undealt deck order, so we sample several worlds $w$: complete assignments of the hidden information that remain consistent with the player’s hand and the public history. In each world, we force candidate bid $a$, let the SFT continuation policy $\pi$ finish the game in the acting seat while policy $\omega$ controls the opponents, and read off the terminal score margin:

Here, $\xi$ is the continuation of the game generated by $\pi$ and $\omega$. We score every candidate on the same sampled worlds to reduce variance. The estimate $Q$ still depends on $\omega$ because changing the continuation opponent can reorder bids even when $(I,a)$ stays fixed.

piKL combines these action values with sampled action frequencies $\hat\tau$ from the SFT policy, upweighting high-$Q$ bids while remaining KLed to the SFT policy:

The temperature $\lambda_{\mathrm{KL}}$ controls the strength of the anchor. As $\lambda_{\mathrm{KL}}\to\infty$, the $Q$ tilt vanishes and the distribution returns to the SFT frequencies $\hat\tau$; as $\lambda_{\mathrm{KL}}\to0$, it concentrates on the sampled candidate with the largest $Q(I,a)$. The $\lambda_{\mathrm{KL}}=0$ arm in the table implements that limiting argmax directly.

We label decisions with this $Q$, build piKL targets at several $\lambda_{\mathrm{KL}}$ values, and distill them with LoRA SFT. Each candidate then occupies every seat against two frozen SFT copies over the same 120 games, 40 per seat.

Mean-margin arm intervals use the normal approximation; paired intervals use a $t$ interval over matched games.

At 40 bid nodes with up to six candidates each, changing only the continuation opponent from self-play to base Qwen reduces median action-rank correlation with the self-play ordering from 1.0 to 0.43, while top-action agreement falls from 100% to 27.5%. The sign of each candidate’s advantage over the reference bid still agrees 72.5% of the time. Base Qwen is admittedly a weak alternative opponent, but a drop that size is enough to show that fine $Q$ rankings depend on the continuation policy.

A second five-arm sweep replaces the self-play continuation with Gemini 3 Flash. Relative to its $\lambda=\infty$ control, the $\lambda=1$ and argmax contrasts are +6.10 [0.38, 11.82] and +5.81 [0.70, 10.93]. Take both intervals with some salt: the sweep ran once, with no correction for multiple comparisons.

On a separate offline check over 43 bid nodes, scored by $Q$ labels rather than live games, aggregating $Q$ over five fixed continuation types also never beats the single-self-play-$Q$ baseline.

action-critic pilot

GRPO’s group return cannot identify which bid changes the outcome. PPO could instead update from an action-specific advantage supplied by a learned critic. Before paying for an actor update, we test whether such a critic can rank candidate bids reliably enough to be useful.

critic target

The actor produces a bid using $I$ before the opponents’ bids are known. Once the auction resolves, those bids become public. During training, a critic can use this newly revealed information even though the actor never receives it at inference (centralized training with decentralized execution). We can fix the observed opponent bids $\mathbf b_{-i}^{\mathrm{obs}}$ to determine what happens when only our bid changes.

Most raw $Q$ variation reflects whether the state itself is favorable. To isolate differences among bids, we center each target over the retained candidate set $C(I)$:

The centering weights candidates equally, whereas a PPO update would require an advantage weighted by the actor policy.

rollout labels

At each treasure-bid state reached by the actor, we sample $K=16$ bids and compress them to 6 diverse candidates. The selection retains the played bid while approximately preserving both the policy’s sampled mass and the numerical range of bids. We score every candidate on the same sampled worlds and random seeds, and hold the opponents’ observed bids fixed. After the forced bid, a deterministic frozen SFT policy completes the game, so the labels are only exact for that frozen continuation; an updated PPO actor would induce a different distribution.

world-sampling budget

For 32 games, we evaluate each candidate with $M=64$ hidden-world samples to get 379 bid states and ~1.5k candidate rows. Because smaller estimates reuse prefixes of the same 64 worlds, we can compare 4, 8, 16, or 32 worlds without further rollouts. “Pair-sign agreement” measures whether actions separated by a $Q$ gap of at least 0.05 and 95% paired sign confidence retain their ordering while regret measures how much reference $Q$ is lost by selecting the lower-sample-budget argmax.

Since the later training set retains confidently separated pairs, we adopt 32 worlds at half the rollout cost.

structured-model probe

Across all 379 calibration states, the mean spread from worst to best retained action is 0.2149 in $\tanh$-margin utility, and the estimated best action improves 0.0931 over the SFT-weighted candidate mean. That second number is an in-sample rollout-oracle bound, not realizable policy uplift. The average gap between the top two is only 0.0599, and the SFT policy’s played action is already best 36.1% of the time.

Most candidate-$Q$ variation is irrelevant to choosing a bid: 84.8% lies between states, while only 15.2% separates actions within a state. (unlike the earlier 99.8% figure, which measured whole-trajectory variance across replicas of an initial deal, this one compares candidates within a state.) PPO can use only the 15.2% component because a state-wide offset raises or lowers every candidate together.

Across 4 split seeds, 4-fold cross-validation fits small structured models to multi-action states from the $M=64$ reference labels. In the table, “uplift” is offline $Q$ improvement over the SFT candidate mixture.

Each non-control row is a gradient-boosted tree ensemble fitted to the candidate-$Q$ labels. The centralized model receives the observed opponent bids, while the actor-visible model does not. Two negative controls either shuffle the targets or drop the candidate bid, leaving a model that can predict only common state value. “Filtered pair accuracy” scores only the confidently separated action pairs defined above.

4B critic results

We reuse three confidence gates from calibration: gap $\ge0.05$, 95% sign confidence, and at most 6 pairs per state. The SFT-initialized critic’s Bradley–Terry loss ranks the better bid above the worse one, while a Huber regression term trains the numeric centered value at weight 0.25.

Even with the confidence gates fixed in advance, every neural run misses the 55% argmax agreement we had set as a success bar and trails the 40% linear-bid baseline.

The next meaningful scale-up uses 128 games and roughly 1,485 decision states, at an estimated 328 H200-hours with $\pm40\%$ uncertainty; reaching 1,024 games and then evaluating on the sequestered 128-game final test approaches 3,000 H200-hours.

PPO could in principle improve beyond a fixed expert, but it must first compress expensive rollouts into small critic differences and then refresh them as the actor’s state distribution moves.

Given those costs and the calibration results above, we stop after the critic diagnostics, without building a production label corpus or running an actor update.

One final search variant keeps the terminal rollouts but replaces the opponents’ current bids with deterministic predictions from a fitted market model. The best arm changes paired margin by only +0.58 [−5.6, +6.8], which leaves the question unresolved.

A separate diagnostic asks whether a much better point predictor would solve the problem. A gradient-boosted tree trained on ~3.2k logged Gemini 3 Flash bids reaches an out-of-fold mean absolute error (MAE) of 1.10 coins with essentially zero bias (all variance). However, it still predicts the wrong side of the actual win/loss threshold on 26.1% of auctions. 55% of auctions resolve within one coin, where Flash’s own sampling noise straddles the boundary even when the conditional mean is accurate.

We check ordinary chosen-action calibration on the games used for the myopic selector evaluation. Across ~1.8k seat-0 treasure bids against 2x Gemini 3 Flash opponents, mean predicted win probability is 40.3% and empirical win frequency is 36.2%. The model is close at the extremes but less calibrated in several middle-probability bins.

For each seat-0 decision, the final display reveals the treasure lot’s realized value $V$. Let $p$ be the highest recorded opponent bid. The ideal clearing-price surplus is $S^*=\max(0,V-p)$. For a counterfactual bid $b$, we resolve the auction against the opponent bids, giving realized surplus $S(b)=(V-b) \cdot \mathbf 1[b\text{ wins}]$. Then just define decision regret as

Zero regret can be unattainable when the budget cannot beat a profitable clearing price, or when $V>p$ but losing the tiebreak makes the cheapest winning bid $p+1$ instead of $p$.

We introduce a control variate to reduce auction-resolution noise: a decision should not receive full credit because an opponent happens to sample below its usual price, or full blame because the opponent samples above it.

For each treasure decision $d$, let $y_d=1$ if the bid wins and $y_d=0$ otherwise, let $p_d$ be the model’s predicted win probability, and let $V_d-b_d$ be the lot’s estimated surplus. Define the decision-level resolution surprise as $x_d=(y_d-p_d)(V_d-b_d)$, which converts auction luck into score units: a surprising outcome matters more when the lot’s surplus is larger. For arm $a$, game $g$, and seat $k$, sum the surprises over that seat’s decisions:

For the adjustment to be unbiased, the model must satisfy the stronger, value-weighted calibration condition

where $I_d$ is the information available at the decision. Ordinary calibration of $p_d$ alone isn’t sufficient because the surplus $V_d-b_d$ varies across decisions. For each matched seed, define the seatwise luck contrast $Z_{g,k}=X_{\mathrm{on},g,k}-X_{\mathrm{off},g,k}$ and the paired margin difference $D_g=\text{margin}_{\mathrm{on},g}-\text{margin}_{\mathrm{off},g}$. We regress

The intercept $\tau$ is the adjusted treatment effect. In a paired pilot, this regression explains 42% of the paired variance, a 1.72× reduction. No direct validation of the stronger value-weighted calibration condition is available, so later gates report the raw paired effect as the primary evidence and the adjusted estimate as model-assisted corroboration.

Adjustment removes about 3 points of adverse resolution luck, yielding −1.88 [−6.57, +2.81], and the adjusted history-repair arm is outright harmful at −5.16 [−9.90, −0.41]. Even with luck accounted for, the myopic selector still loses.

ExIt alternates between a policy that visits states and an expert that improves the action taken in those states. If the current policy $\pi$ induces a state distribution $I\sim d_\pi$, the expert supplies

and the resulting dataset $\mathcal D_E$ trains a new policy by imitation:

The new policy can generate another state distribution, which the expert can label again, giving the loop $\pi_k\rightarrow d_{\pi_k}\rightarrow\mathcal D_E\rightarrow\pi_{k+1}$. Unlike ordinary SFT from a fixed teacher corpus, the supervision is concentrated on states the current policy actually reaches and on actions the expert would change.

1. deviations: replace the SFT bid with the analytic expert’s target and expose the quantities that determine it: $\hat V$, opponent price means, $P(\text{win})$ for both the SFT bid and the selected bid, budget, and paced surplus
2. pass-throughs: retain the raw SFT response when the selector agrees or finds less than one coin of EV lift, so the adapter does not relearn every sound decision
3. loan and investment anchors: retain the SFT policy’s original financing responses, so an improvement to treasure bidding does not erase the liquidity discipline that makes it work

Pass-through and financing anchors come from all three seats. Only seat 0 contains selector deviations, so all-seat anchors reduce the risk that the adapter learns a player_id = 0 shortcut.

In one game, round 9, seat 0 has 22 coins and a public collection of three Green and one Yellow gems. The current auction offers two more Green gems, enough to complete a mission, and the ordinary SFT policy samples a bid of 22.

After the calibration correction, the frozen expert values the lot at 30.59 coins, including the 10-point mission bonus it would complete, and predicts opponent mean bids of 10 and 11. A bid of 22 wins with certainty, but once each coin’s option value is priced in, its paced utility turns negative, and a lower bid preserves most of the win probability at a far better price:

The final 12 percentage points of win probability cost too much, so the expert shades the bid to 13. The training target reads:

This treasure is worth about 31 to me net of bias. 22 would overpay once each coin’s option value is priced in (budget 22); shading to 13 keeps the win probability at ~88% and earns more surplus per coin. Expected surplus favors 13.

{"bid": 13}

That prose comes from a template filled with four logged quantities: $\hat V$, both bids, win probability, and remaining budget. At this stage, every deviation target shares the same synthetic voice.

Since the log preserves neither the SFT policy’s free-form rationale nor a student response, what is shown above is the supervision target itself.

The denominator is positive by construction. A value of 0 means no utility gain over the recorded SFT bid; 1 means equal utility to the expert, not necessarily the same bid; values above 1 are possible. The “SFT reference” below is a fresh stochastic response to the same prompts, so it need not reproduce the recorded $b_{\mathrm{SFT}}$ and can score above zero.

Across separate experiments, adjusted point estimates leave a 2.99-point gap; the direct within-run selector increment is +3.95 [−0.83, 8.74]. Applying the two measured distillations’ 73–78% retention range to that gap suggests only ~2.2 points from another round, but this estimate is just an extrapolation.

Here, the model plays seat 0 against GPT-5.5 and Gemini 3.1 Pro. By round 8, it has 14 coins and a collection of Yellow and Purple. A Blue gem would complete the 5-point Blue–Yellow mission, and revealing the Blue in its private hand would move Blue from 4 to 8 points per gem. One opponent has only 6 coins, while the model has tiebreak priority over the other.

The model writes:

Winning this Blue gem would immediately complete the “specific_2_by” mission (Blue + Yellow) for 5 coins and put me one step closer to the “specific_3_bgp” mission (Blue + Green + Purple) for 10 coins.

A bid of 6 ensures I beat Player 1’s maximum possible bid. If Player 2 bids 6, I win on tiebreak. If Player 2 bids higher, they are spending a significant portion of their capital early. Given the mission rewards and the value of the gem, 6 is a strong but efficient bid that preserves my coins for future rounds.

{"bid": 6}

The bids come in 6, 6, and 5: the model wins the tie, claims the mission, and reveals Blue. Ignoring any later increase in Blue’s price, the immediately visible surplus is already at least

The bid of 6 captures at least 7 points of immediate surplus while preserving 8 coins. The model eventually wins 93–89–59.

The same game produces a cleaner test of state tracking in rounds 13 and 14. Both opponents have exhausted their cash. In round 13, the model is first in tiebreak order, so it recognizes that a zero bid wins the Blue gem:

Player 1 has 0 coins, Player 2 has 0 coins. Neither can bid above 0!

I’m first in tiebreak order, so if we all bid 0, I win.

This is a free win at 0 coins! Since both opponents have 0 coins, they can only bid 0. I have tiebreak priority, so bidding 0 guarantees I win the Blue gem for free.

{"bid": 0}

All three players bid 0, and the model wins. Winning moves it to the back of the tiebreak order. On the next auction, a two-gem Yellow–Blue lot, it updates rather than repeating the previous action:

Player 2 > Player 1 > Me. So if everyone bids 0, Player 2 wins. I need to bid at least 1 to guarantee winning.

Bidding 1 is clearly optimal—it guarantees the win while costing almost nothing.

{"bid": 1}

The other players again bid 0, so the model buys both gems for 1, and these two purchases contribute to the same 93-point win above.

In another game, the model plays seat 1 against Gemini 3.1 Pro and GPT-5.5. In round 13 it has 14 coins, while GPT-5.5 has 7 and acts earlier in ties. Purple would complete the 5-point Red–Purple mission, so the model chooses the exact guaranteed price:

Player 2 has 7 coins and is ahead of me in the tiebreak order. To guarantee a win against Player 2, I must bid 8.

Spending 8 now for a gem that is worth at least 12–16 coins (plus the 5-coin mission) is a net positive, and I still retain 6 coins for future rounds.

{"bid": 8}

The bids are 0, 8, and 7, so the model wins Purple and the mission. It now has only 6 coins. The next card is a 10-coin loan, and its rationale connects that financing decision to the remaining mission:

Winning this loan would provide immediate liquidity (10 - bid) to compete for the remaining 4 auctionable gems, which is crucial because I need a Yellow gem to complete the “specific_3_rgy” mission (10 coins) and a second Purple for the “same_2_purple” mission (5 coins). However, bidding too high on a loan is counterproductive as it increases my end-game debt without adding net value.

{"bid": 5}

The loan bids land at 2, 5, and 4, and winning raises the model’s available cash to $6+(10-5)=11$. On the next auction, another Purple, GPT-5.5 still has 7 coins and the better tiebreak position, so the covering price is again $7+1=8$:

Player 2 has 7 coins max. They might bid up to 7. I should bid 8 to guarantee winning over Player 2’s maximum.

Given the expected surplus-maximizing bid is 8, and this is worth ~17+ to me, bidding 8 is clearly optimal.

{"bid": 8}

Again the bids are 0, 8, and 7, and the model wins its second Purple and the 5-point mission. It finishes first, 72–66–49, with

The first Purple purchase leaves the model short of cash. The loan repairs that shortage and funds a second exact covering bid for the mission.