<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:media="http://search.yahoo.com/mrss/">
  <channel>
    <title>Machine Learning · Jared Lynskey</title>
    <link>https://jared.lynskey.co.nz/en/tags/machine-learning/</link>
    <description>Short stories of value on software engineering and engineering leadership — performance reviews, hiring and scaling teams, CI/CD, DevOps monitoring and agentic AI. Written from Seoul.</description>
    <generator>Hugo</generator>
    <language>en</language>
    <copyright>© 2026 Jared Lynskey</copyright>
    <lastBuildDate>Wed, 28 Oct 2026 00:00:00 +1300</lastBuildDate>
    <atom:link href="https://jared.lynskey.co.nz/en/tags/machine-learning/index.xml" rel="self" type="application/rss+xml" />
    <item>
      <title>What Determines a Painting&#39;s Value? Building an Algorithm to Predict Art Auction Prices</title>
      <link>https://jared.lynskey.co.nz/en/posts/2026/2026-04-01-art-evaluation-algorithm/</link>
      <guid isPermaLink="true">https://jared.lynskey.co.nz/en/posts/2026/2026-04-01-art-evaluation-algorithm/</guid>
      <pubDate>Wed, 01 Apr 2026 00:00:00 +0000</pubDate>
      <dc:creator>Jared Lynskey</dc:creator>
      <category>Machine Learning</category>
      <category>XGBoost</category>
      <category>Art Market</category>
      <category>Python</category>
      <category>Data Science</category>
      <description>What drives art auction prices? Reproducing a 2024 XGBoost study of 34,200 sales shows the market prices the artist, not the artwork. The features, the results, and the art valuation platform I built around them.</description>
      <content:encoded><![CDATA[<p>What determines a painting&rsquo;s value at auction? Mostly the artist, not the painting. In a 2024 study of 34,200 contemporary art auction sales, a model that only looked at the artworks explained about 5% of the variation in price. One built on the artist&rsquo;s career, exhibition history and past sales explained about 73%, and adding the auction house&rsquo;s pre-sale estimate took it to 92%. Here&rsquo;s how I reproduced that, and what it means for valuing art with an algorithm.</p>
<p>I spent the last few weeks on two related projects. The first was reproducing a 2024 <em>Scientific Reports</em> paper by Lee et al., &ldquo;Social signals predict contemporary art prices better than visual features, particularly in emerging markets&rdquo;. It trains an XGBoost model on 34,200 auction records from 590 living artists. The second was building an art platform with a knowledge graph of artists, owners, and auction records. It&rsquo;s basically the data plumbing you&rsquo;d need around a model like that one if you wanted to do anything useful with it.</p>
<p>Both projects kept circling back to the same question: can you actually predict what an artwork sells for? The answer is yes, but not from looking at the artwork. The features that matter are almost entirely social.</p>

<h2 class="relative group">Why Art Is Hard to Value
    <div id="why-art-is-hard-to-value" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#why-art-is-hard-to-value" aria-label="Anchor">#</a>
    </span>
    
</h2>
<p>Before writing any code, I spent a couple of days reading about how appraisers and auction houses price work. The list of factors is long and most of them don&rsquo;t quantify cleanly. Provenance — who&rsquo;s owned it, where it&rsquo;s been shown, which catalogues reference it. Attribution: a piece confirmed by the artist&rsquo;s foundation is worth orders of magnitude more than one that&rsquo;s only &ldquo;attributed to&rdquo; them. The artist&rsquo;s reputation, built from solo shows, biennial inclusions, museum acquisitions, and critical reception. Then condition and restoration history, rarity within the artist&rsquo;s body of work, and medium — oil on canvas tends to outperform works on paper, which outperform prints. A piece held by a major museum carries a stamp the secondary market trusts. And timing matters: five-year-old comparables are stale.</p>
<p>Auction-house specialists also talk about &ldquo;emotional value&rdquo; — collector desire, bidding-war dynamics, personal resonance. It can push hammer prices well past any rational estimate. You can&rsquo;t model that directly, but you can pick up its shape indirectly through auction-house estimates.</p>
<p>The dominant valuation method in practice is comparable sales analysis: find recent auction results for similar works by the same artist and adjust for differences. An XGBoost model is doing the same thing, just at scale and with more features than any human appraiser would track by hand.</p>

<h2 class="relative group">The Paper&rsquo;s Central Claim
    <div id="the-papers-central-claim" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#the-papers-central-claim" aria-label="Anchor">#</a>
    </span>
    
</h2>
<p>Lee et al. make a counterintuitive argument: the visual content of an artwork explains very little of its price. Their visual-only model, using color, composition, edge density, and ResNet18 embeddings, gets an R² of about 0.055. That&rsquo;s barely better than guessing the mean.</p>
<p>A model using only artist-level &ldquo;social&rdquo; features (career stage, exhibition history, past auction prices, ranking on ArtFacts) hits R² ≈ 0.73. Add the auction house&rsquo;s pre-sale estimate as a feature and it climbs to 0.92.</p>
<p>In other words, the artwork barely matters. What matters is who made it and what the market has already said about them.</p>
<p>The paper&rsquo;s secondary claim is that this gap widens further in emerging markets, by which they mean anything outside the established USA, UK, France, and Germany core. In those markets, social signals matter more and expert estimates are less accurate, leaving more room for algorithmic prediction to add value.</p>

<h2 class="relative group">Reproducing the Paper
    <div id="reproducing-the-paper" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#reproducing-the-paper" aria-label="Anchor">#</a>
    </span>
    
</h2>
<p>The brief was straightforward: reproduce the paper as a single-developer MVP. No production deployment, no live data pipeline. Faithful reimplementation, prototype-quality code, and a written report on findings.</p>

<h3 class="relative group">The Data
    <div id="the-data" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#the-data" aria-label="Anchor">#</a>
    </span>
    
</h3>
<p>The paper provides cleaned CSVs as supplementary material:</p>
<table>
	<thead>
			<tr>
					<th>File</th>
					<th>Rows</th>
					<th>What it is</th>
			</tr>
	</thead>
	<tbody>
			<tr>
					<td><code>Df_mloutfull.csv</code></td>
					<td>86,221</td>
					<td>Full raw dataset, 500+ columns</td>
			</tr>
			<tr>
					<td><code>df_for_ml_improved_up_to_2012.csv</code></td>
					<td>34,200</td>
					<td>Main cleaned dataset, 1996–2012</td>
			</tr>
			<tr>
					<td><code>df_for_ml_improved_old_market.csv</code></td>
					<td>29,853</td>
					<td>Established markets only</td>
			</tr>
			<tr>
					<td><code>df_for_ml_improved_new_market.csv</code></td>
					<td>4,346</td>
					<td>Emerging markets only</td>
			</tr>
			<tr>
					<td><code>transactions.csv</code></td>
					<td>114,283</td>
					<td>Raw auction records with image links</td>
			</tr>
	</tbody>
</table>
<p>The whole thing is about 5 GB. It fits comfortably in RAM, and XGBoost training on a dataset this size takes under five minutes on a laptop CPU. No GPU needed for the core model. Gradient-boosted trees on tabular data at this scale are nearly free to train, which is easy to forget if you&rsquo;ve spent the last few years in neural-net land.</p>

<h3 class="relative group">Features
    <div id="features" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#features" aria-label="Anchor">#</a>
    </span>
    
</h3>
<p>The model uses 38 features split across three categories.</p>
<p><strong>About the artist (30 features):</strong></p>
<ul>
<li>Demographics: age, gender, education level, top-school flag.</li>
<li>Career: ArtFacts ranking, solo shows, group shows, biennial inclusions, awards.</li>
<li>Collections: counts of private and public collections.</li>
<li>Price history: mean, median, max, and min from the artist&rsquo;s last 5 and last 10 sales.</li>
<li>Size-adjusted price history (price per square inch).</li>
<li>Geography: where the artist works and lives, encoded as country flags.</li>
<li>Match flags: does the artwork&rsquo;s genre match the artist&rsquo;s typical genre? Does the sale country match?</li>
</ul>
<p><strong>About the market (8 features):</strong></p>
<ul>
<li>Auction house tier (1–4).</li>
<li>Country and region price levels for that year (min, mean, median, max).</li>
</ul>
<p><strong>About the artwork itself (non-visual):</strong></p>
<ul>
<li>Width, height, area in square inches.</li>
<li>Medium category (painting, print, photo, sculpture, other).</li>
</ul>
<p>Notice what&rsquo;s absent: anything about the artwork&rsquo;s actual visual content. That comes in only as a separate ablation study.</p>

<h3 class="relative group">The Train/Test Split
    <div id="the-traintest-split" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#the-traintest-split" aria-label="Anchor">#</a>
    </span>
    
</h3>
<p>Time-based, not random. Everything before May 2011 goes into training (~80%, ~27,360 records); everything after goes into the test set (~20%, ~6,840 records). Random splits would leak future information through the artist&rsquo;s price-history features. If you scatter one artist&rsquo;s records randomly across train and test, the test rows have already &ldquo;seen&rdquo; each other&rsquo;s prices through the rolling windows, and your R² gets artificially inflated.</p>
<p>It&rsquo;s the kind of thing that&rsquo;s obvious once you see it but easy to miss on the first build.</p>

<h3 class="relative group">The Model
    <div id="the-model" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#the-model" aria-label="Anchor">#</a>
    </span>
    
</h3>
<p>XGBoost regressor, predicting <code>log10(price_usd)</code>. Hyperparameter search over <code>max_depth</code> and <code>learning_rate</code> against a held-out validation slice. Two model variants:</p>
<ol>
<li><strong>Without expert estimates.</strong> Target R² ≈ 0.73.</li>
<li><strong>With expert estimates.</strong> Target R² ≈ 0.92.</li>
</ol>
<p>The &ldquo;with estimates&rdquo; variant uses the auction house&rsquo;s pre-sale low and high estimates as additional features. Those two estimates alone, used by themselves, get R² ≈ 0.90. Adding the social features on top picks up another 0.02. That&rsquo;s small in absolute terms, but it&rsquo;s the part of the model doing real predictive work beyond what the experts already know.</p>

<h3 class="relative group">What I Aimed For
    <div id="what-i-aimed-for" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#what-i-aimed-for" aria-label="Anchor">#</a>
    </span>
    
</h3>
<p>I wasn&rsquo;t trying to beat the paper. The goal was to land within ±0.03 of their R² values across all conditions. For the headline numbers (~0.73 metadata-only, ~0.92 with estimates), reproducibility is good. XGBoost is deterministic with a fixed seed, and the features are well-specified. I expected drift in two places: the visual-feature ablation, where small differences in PCA dimensions or image preprocessing can move things, and the emerging-market split, where there are only 4,346 records and the noise floor is higher.</p>

<h2 class="relative group">Why Visual Features Barely Help
    <div id="why-visual-features-barely-help" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#why-visual-features-barely-help" aria-label="Anchor">#</a>
    </span>
    
</h2>
<p>This was the most interesting result for me. The paper&rsquo;s visual pipeline extracts 8,971 numbers per image:</p>
<ul>
<li><strong>GIST descriptor</strong> (960 dims): overall spatial layout.</li>
<li><strong>Histogram of Oriented Gradients</strong> (2,915 dims): edge structure.</li>
<li><strong>Color histogram</strong> (4,096 dims): palette breakdown.</li>
<li><strong>ResNet18 features</strong> (1,000 dims): high-level pretrained embedding.</li>
<li><strong>Colorfulness</strong> (1 dim): vividness scalar.</li>
<li><strong>Complexity</strong> (1 dim): edge-density scalar.</li>
</ul>
<p>Compress all that with PCA and feed it to XGBoost, and you get R² ≈ 0.055. Color alone gets 0.056. Edge structure gets 0.029. The neural-net embedding gets 0.009, basically zero.</p>
<p>The clean explanation is that auction prices are set at the level of the artist, not the individual work. Two paintings by the same artist sell for similar amounts regardless of what&rsquo;s on the canvas, because what&rsquo;s being priced is the signature.</p>
<p>You can see why this would be true if you&rsquo;ve ever read auction catalogue notes. They spend almost all their words on provenance, exhibition history, and comparable sales. They barely describe the work itself.</p>
<p>If you take that 5% number seriously, the aesthetic content of a work is almost decoupled from its market value. Whatever the market rewards, it isn&rsquo;t what&rsquo;s on the canvas. A model that understood art at the pixel level would predict prices worse than one that only knew the artist&rsquo;s CV.</p>

<h3 class="relative group">Phase 3 Was Conditional
    <div id="phase-3-was-conditional" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#phase-3-was-conditional" aria-label="Anchor">#</a>
    </span>
    
</h3>
<p>The paper&rsquo;s image-link column points to URLs that were valid in 2012. By 2026, most are dead. The plan I wrote up flagged this risk on day 8: pull a hundred random URLs and see how many resolve. If more than half were gone, skip the visual phase entirely and document why.</p>
<p>Visual features only explain about 5% of price variance anyway. Spending two weeks fighting bit rot to confirm a number that small isn&rsquo;t a good trade. Better to write up the dead-link finding as part of the report and move on.</p>

<h2 class="relative group">Established vs. Emerging Markets
    <div id="established-vs-emerging-markets" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#established-vs-emerging-markets" aria-label="Anchor">#</a>
    </span>
    
</h2>
<p>The paper trains separate models for established markets (USA, UK, France, Germany) versus emerging markets (19 other countries). The headline finding:</p>
<table>
	<thead>
			<tr>
					<th>Feature set</th>
					<th>Established R²</th>
					<th>Emerging R²</th>
			</tr>
	</thead>
	<tbody>
			<tr>
					<td>Visual only</td>
					<td>0.053</td>
					<td>0.056</td>
			</tr>
			<tr>
					<td>Metadata (social signals)</td>
					<td>0.667</td>
					<td>0.750</td>
			</tr>
			<tr>
					<td>Metadata + estimates</td>
					<td>0.916</td>
					<td>0.859</td>
			</tr>
	</tbody>
</table>
<p>Two things stand out. First, social signals matter more in emerging markets, not less. I&rsquo;d assumed established markets, with their richer data, would show better social-feature performance, but it&rsquo;s the opposite.</p>
<p>Second, expert estimates are less accurate in emerging markets — that&rsquo;s the 0.916 vs 0.859 gap once estimates are added in. In an established market, an algorithmic prediction has to compete with a specialist at a major auction house who has decades of comparable-sales data behind them. In an emerging market, that specialist has thinner data and the algorithm has more room to add value.</p>
<p>If you ever wanted to commercialise a model like this, emerging markets are where it would earn its keep.</p>

<h2 class="relative group">Limitations Worth Stating
    <div id="limitations-worth-stating" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#limitations-worth-stating" aria-label="Anchor">#</a>
    </span>
    
</h2>
<p>This is a paper reproduction, not a production system. It has no live data pipeline — the model trains on 1996–2012 auction data and never updates. There&rsquo;s no deployment; the outputs are notebooks and a report, not a prediction API. There&rsquo;s no drift detection, so once trained, the model has no idea when the market has changed. And the dataset is frozen: crypto art, NFTs, and post-COVID market shifts simply don&rsquo;t exist in it. If you wanted to predict prices at a Korean auction house, you&rsquo;d want auction-specific features and possibly a separate model for the Korean market — nothing here specialises that far.</p>
<p>The model also reflects historical patterns. If the contemporary market changed structurally around 2015 (and there are arguments it did, with art fairs becoming a major primary-market venue and a new generation of collectors entering), then the 1996–2012 patterns may not generalise.</p>

<h2 class="relative group">From Paper to Platform
    <div id="from-paper-to-platform" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#from-paper-to-platform" aria-label="Anchor">#</a>
    </span>
    
</h2>
<p>Reading the paper changed how I thought about the second project. The platform was originally framed as a crowdfunding marketplace for art exhibitions. Artists secure funding for upcoming shows, and investors get exposure to emerging artists earlier than the auction market typically allows.</p>
<p>The product surface is deliberately simple. Artists post upcoming shows that need funding (venue, dates, expected works). Investors back specific shows in tiered amounts ($100, $500, $1k, $5k) and share in the upside when the work sells. AI agents handle most of the artist-investor chat so that neither side has to be online for the conversation to keep moving. That&rsquo;s particularly useful when a Korean gallery is talking to a U.S. collector across a 14-hour gap.</p>
<p>Once it sinks in that the artist is the asset and not the artwork, the data model has to change to match. What you need to capture is who has owned each work, where it has been shown, what comparable works have sold for, and who else owns work by the same artist. Ownership chains, institutional recognition, auction comps, collector networks — that&rsquo;s what a knowledge graph is for.</p>
<p>The data model is small on purpose: artists, artworks, auction sales, ownership periods, and a provenance log for the data itself, so every record can be traced back to where it was scraped from. That last part sounds like bookkeeping, but with scraped data it&rsquo;s what lets you trust the graph at all.</p>
<p>The mobile app renders all of this as a force-directed graph. A time-range slider lets you scrub through years and watch the network evolve: which collectors entered when, which artists got picked up by which museums, which works changed hands during the 2008 dip.</p>
<p>Each kind of entity gets its own shape and colour. Artists are diamonds, artworks are rounded rectangles, auction houses are hexagons, and the different kinds of owner (collectors, dealers, museums, estates) are circles in different colours.</p>
<p>The shapes aren&rsquo;t decorative. When you zoom out to a few hundred nodes the silhouette tells you what type of entity you&rsquo;re looking at without waiting for labels to render. A diamond surrounded by circles is an artist with a collector network. A hexagon connected to many diamonds is an auction house that handles a roster of artists. You can read the structure of a market at a glance.</p>
<p>A second tab shows the same data on a timeline: ownership periods as horizontal bars, sales as point events. Same underlying graph, different lens. Useful when you want to see the sequence of who held what, rather than the network shape at a single moment.</p>
<p>The platform isn&rsquo;t itself a price-prediction tool. It&rsquo;s the data layer underneath one. If you wanted to feed something like the Lee et al. model with live, current data instead of frozen 2012 CSVs, this is the structure you&rsquo;d want.</p>

<h2 class="relative group">Crowdsourced Valuation as a Game
    <div id="crowdsourced-valuation-as-a-game" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#crowdsourced-valuation-as-a-game" aria-label="Anchor">#</a>
    </span>
    
</h2>
<p>The piece I haven&rsquo;t built yet, and the part I&rsquo;m most interested in, is turning art valuation into a game.</p>
<p>The Lee et al. paper takes auction-house pre-sale estimates as a feature and gets a big R² lift from them. Those estimates are valuable because they&rsquo;re the aggregated judgment of trained specialists who have seen thousands of comparable works and have skin in the game. But auction-house specialists are a narrow, expensive resource. There are a few hundred of them globally, they work on specific consignments, and they don&rsquo;t touch the long tail of artists nobody is consigning yet.</p>
<p>The question I keep coming back to is whether you can synthesise a comparable signal from a much wider, cheaper pool of judgments.</p>
<p>The rough design:</p>
<ul>
<li>Show a user an artwork: image, dimensions, year, artist name, brief CV.</li>
<li>Ask them what it would sell for at auction.</li>
<li>Let them pick a price bracket, or guess a hammer figure directly.</li>
<li>Reveal the actual sale price after they commit.</li>
<li>Score them with a Brier-score-style accuracy rating over time.</li>
<li>Surface a leaderboard, daily streaks, and badges for hot streaks.</li>
<li>Weight each user&rsquo;s future predictions by their historical accuracy.</li>
</ul>
<p>It&rsquo;s a prediction market for art prices in the form of a guessing game. The loop is the same as a geography guessing game or online chess puzzle ratings: repeatable, scorable, with feedback you can build skill against. People who guess well rise on the leaderboard, and their future guesses count for more in the aggregated signal.</p>
<p>The model side is where this gets useful. A weighted aggregation of those guesses becomes a feature you can feed into XGBoost alongside the social signals. If the crowd&rsquo;s accuracy-weighted median guess correlates well with hammer prices, you have something close to a synthetic auction-house estimate for works that don&rsquo;t have a real one. That&rsquo;s exactly the gap on the long tail of artists where the Lee et al. model needs the most help.</p>
<p>There&rsquo;s also a cold-start angle. The model trains well on artists with rich price histories and badly on artists with three sales to their name. Three sales plus a thousand crowd guesses is at least something to work with. Crowdsourced valuation is a way of generating training signal for the cases the underlying data doesn&rsquo;t cover.</p>
<p>Gamification matters here because the only way to make the crowd reliable is to make participating fun enough that people do it a lot. Single-question forms get you noisy data from people who clicked through once. A daily streak with a leaderboard and a calibration badge gets you the same person making 500 carefully considered predictions over a year. Volume plus self-selection (good predictors stay, bad predictors get bored) is what gives you a signal rather than noise.</p>
<p>There are design questions I&rsquo;d need to answer before building any of it. How much do you show the user — just image and CV, or comparable sales too? More context means better-informed guesses, but past a point the user is mostly regurgitating what you&rsquo;ve already shown them. How much price feedback do you give? Reveal the exact hammer, a range, or just whether they landed within ±20%? Too granular and you train people to anchor on specific numbers; too vague and they can&rsquo;t calibrate. Then there&rsquo;s leaderboard gaming: players will gravitate toward artists they already know, and a Banksy guess is mostly trivia rather than skill. Probably the fix is forced random sampling, the way language-learning apps pick lessons, plus a separate ranking for &ldquo;blind&rdquo; rounds where the artist&rsquo;s name is hidden. And hindsight bias is unavoidable — once a user knows a Basquiat sold for $110M, they can&rsquo;t un-know it. The supply of fresh, unseen artworks is what keeps the loop honest, and that supply is finite per week.</p>
<p>The gamification framing also pulls the platform&rsquo;s two halves together. The crowdfunding side rewards investors when shows perform well financially. The guessing-game side rewards anyone, paying investor or not, for being good at calling prices. Both are versions of the same idea: let the crowd&rsquo;s collective judgment turn into a tradeable signal. One trades capital, the other trades attention. Players who develop a real edge could eventually graduate to the investor side, bringing a verifiable track record of price-calling accuracy with them. That&rsquo;s a better filter than &ldquo;knew the right gallerist.&rdquo;</p>

<h2 class="relative group">What I Took Away
    <div id="what-i-took-away" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#what-i-took-away" aria-label="Anchor">#</a>
    </span>
    
</h2>
<p>Pre-sale estimates are very good. Auction houses get a lot of grief for opaque pricing, but their estimates capture something close to 90% of price variance on their own. Whatever specialists are doing, they&rsquo;re doing it well. And yet the artwork itself barely matters statistically — which is uncomfortable if you care about art for aesthetic reasons. The market isn&rsquo;t pricing what you see. It&rsquo;s pricing what other people have already paid for adjacent work.</p>
<p>On the modelling side, two habits earned their keep. Time-based splits are the only honest way to evaluate price models; random splits give you flattering R² values that don&rsquo;t survive contact with production. And gradient-boosted trees are still the right tool here — XGBoost beats every neural baseline in the paper, and for tabular data at this scale, with this much engineered feature structure, there&rsquo;s no reason to reach for a transformer.</p>
<p>The commercial observations: the interesting opportunity is in emerging markets, where expert pricing is thinnest and algorithmic prediction has the biggest gap to fill. And the crowd is an underused data source. The single biggest feature in the Lee et al. model was an auction house&rsquo;s pre-sale estimate; if a calibrated crowd can produce a comparable signal for the long tail of artists, that&rsquo;s a real opening, both as a model input and as a product in its own right.</p>
<p>The two projects ended up more complementary than I planned. The paper reproduction taught me what predicts prices. The platform work taught me what data structure you&rsquo;d need to put that knowledge to use. The gamification idea ties the two together: the knowledge graph supplies the artworks and historical sales, the guessing game generates a crowd-derived estimate feature, and the model consumes that feature alongside the social signals to produce predictions for artists who would otherwise be invisible to it.</p>
<p>None of this is finished. The paper reproduction is a working prototype with a written report, the platform is a demo with seeded data, and the crowdsourced game only exists as the design notes you just read. But the connecting idea is consistent enough that I think the right shape is reasonably clear: price the artist rather than the artwork, aggregate the crowd rather than relying only on experts, and treat valuation as something the market produces collectively rather than something specialists hand down.</p>
<p>That&rsquo;s the version I&rsquo;d ship next, given the time to build it.</p>
]]></content:encoded>
      <media:content url="https://jared.lynskey.co.nz/en/posts/2026/2026-04-01-art-evaluation-algorithm/featured.jpg" medium="image" type="image/jpeg" />
    </item>
    <item>
      <title>Content Ranking Algorithms: Signals Beyond Views and Likes</title>
      <link>https://jared.lynskey.co.nz/en/posts/2023/2023-10-06-reinenforcement-learning/</link>
      <guid isPermaLink="true">https://jared.lynskey.co.nz/en/posts/2023/2023-10-06-reinenforcement-learning/</guid>
      <pubDate>Fri, 06 Oct 2023 00:30:03 +0000</pubDate>
      <dc:creator>Jared Lynskey</dc:creator>
      <category>content ranking</category>
      <category>metrics</category>
      <category>SEO</category>
      <category>machine learning</category>
      <category>user engagement</category>
      <category>content strategy</category>
      <description>What a content ranking algorithm does, why ranking on views and likes alone makes the popular more popular, and the quieter signals worth blending into one score.</description>
      <content:encoded><![CDATA[<p>A content ranking algorithm decides the order a feed, a search page or a &ldquo;top posts&rdquo; list shows things in. It gives every item a score built from several signals (how people engage with it, what the content itself is like, how fresh and relevant it is) and sorts by that score. Rank on views and likes alone, though, and you get a rich-get-richer loop: whatever sits at the top gets seen, so it collects more views and likes, so it stays at the top.</p>
<p>That&rsquo;s the obvious flaw in views and upvotes: they reward whatever&rsquo;s already popular. Fresh content starves at the bottom of the feed, and the platform slowly turns into a rerun channel. Ranking a feed well is one of the quietly hard problems where I work, so I&rsquo;ve spent a fair bit of time thinking about which other signals are worth trusting.</p>

<h3 class="relative group">Watch what people do, not what they tap
    <div id="watch-what-people-do-not-what-they-tap" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#watch-what-people-do-not-what-they-tap" aria-label="Anchor">#</a>
    </span>
    
</h3>
<p>Explicit feedback is scarce and biased; behaviour is plentiful. Time on page and scroll depth tell you whether someone actually read the thing or bailed after the first paragraph. Revisits are even better — coming back to a piece is a stronger endorsement than any upvote. And a fast exit is a useful negative signal: the content promised something the reader didn&rsquo;t get.</p>

<h3 class="relative group">Signals baked into the content itself
    <div id="signals-baked-into-the-content-itself" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#signals-baked-into-the-content-itself" aria-label="Anchor">#</a>
    </span>
    
</h3>
<p>Good content doesn&rsquo;t always collect early likes, but it usually shows its quality in other ways. Novelty is one — does the piece cover ground the platform hasn&rsquo;t seen? Structure is another; clean formatting, decent grammar and clarity correlate with quality more than we&rsquo;d like to admit. An author&rsquo;s track record is a fair prior too, judged on how their past work held readers rather than on raw like counts. Early comments help as well: three genuinely positive comments beat thirty reflexive upvotes, and a bit of semantic analysis can flag content that fills a gap the platform has been missing.</p>

<h3 class="relative group">Borrow signals from outside
    <div id="borrow-signals-from-outside" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#borrow-signals-from-outside" aria-label="Anchor">#</a>
    </span>
    
</h3>
<p>The wider internet tells you what&rsquo;s resonating right now. Content that lines up with trending search keywords has built-in relevance, and quick publishing on a rising topic deserves a boost. What&rsquo;s buzzing on Twitter or Reddit is often a preview of what your own users will want next — with a little prediction you can promote matching fresh content before the wave instead of after it. Your own search box is underrated too: if users keep searching for a topic, or a piece pulls steady traffic in from search engines, that&rsquo;s relevance you didn&rsquo;t have to guess at.</p>

<h3 class="relative group">Blending the signals into one score
    <div id="blending-the-signals-into-one-score" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#blending-the-signals-into-one-score" aria-label="Anchor">#</a>
    </span>
    
</h3>
<p>Here&rsquo;s the idea in a few lines of Python. It&rsquo;s an illustration, not a production system. Each signal is normalised to a 0 to 1 range so that one with big raw numbers can&rsquo;t drown the rest, a weighted sum turns them into a single score, and a time decay gives a new post a fair shot against last month&rsquo;s hit.</p>
<div class="highlight-wrapper"><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="n">WEIGHTS</span> <span class="o">=</span> <span class="p">{</span><span class="s2">&#34;read_depth&#34;</span><span class="p">:</span> <span class="mf">0.3</span><span class="p">,</span> <span class="s2">&#34;revisits&#34;</span><span class="p">:</span> <span class="mf">0.25</span><span class="p">,</span> <span class="s2">&#34;quality&#34;</span><span class="p">:</span> <span class="mf">0.25</span><span class="p">,</span> <span class="s2">&#34;early_comments&#34;</span><span class="p">:</span> <span class="mf">0.2</span><span class="p">}</span>
</span></span><span class="line"><span class="cl"><span class="n">HALF_LIFE_HOURS</span> <span class="o">=</span> <span class="mi">48</span>  <span class="c1"># a post loses half its score every two days</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="k">def</span> <span class="nf">normalise</span><span class="p">(</span><span class="n">values</span><span class="p">):</span>
</span></span><span class="line"><span class="cl">    <span class="n">lo</span><span class="p">,</span> <span class="n">hi</span> <span class="o">=</span> <span class="nb">min</span><span class="p">(</span><span class="n">values</span><span class="p">),</span> <span class="nb">max</span><span class="p">(</span><span class="n">values</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">    <span class="k">return</span> <span class="p">[(</span><span class="n">v</span> <span class="o">-</span> <span class="n">lo</span><span class="p">)</span> <span class="o">/</span> <span class="p">(</span><span class="n">hi</span> <span class="o">-</span> <span class="n">lo</span><span class="p">)</span> <span class="k">if</span> <span class="n">hi</span> <span class="o">&gt;</span> <span class="n">lo</span> <span class="k">else</span> <span class="mf">0.0</span> <span class="k">for</span> <span class="n">v</span> <span class="ow">in</span> <span class="n">values</span><span class="p">]</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="k">def</span> <span class="nf">rank</span><span class="p">(</span><span class="n">items</span><span class="p">):</span>
</span></span><span class="line"><span class="cl">    <span class="s2">&#34;&#34;&#34;items: dicts holding raw signal values plus &#34;age_hours&#34;.&#34;&#34;&#34;</span>
</span></span><span class="line"><span class="cl">    <span class="n">norm</span> <span class="o">=</span> <span class="p">{</span><span class="n">name</span><span class="p">:</span> <span class="n">normalise</span><span class="p">([</span><span class="n">item</span><span class="p">[</span><span class="n">name</span><span class="p">]</span> <span class="k">for</span> <span class="n">item</span> <span class="ow">in</span> <span class="n">items</span><span class="p">])</span> <span class="k">for</span> <span class="n">name</span> <span class="ow">in</span> <span class="n">WEIGHTS</span><span class="p">}</span>
</span></span><span class="line"><span class="cl">    <span class="n">scores</span> <span class="o">=</span> <span class="p">[]</span>
</span></span><span class="line"><span class="cl">    <span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">item</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">items</span><span class="p">):</span>
</span></span><span class="line"><span class="cl">        <span class="n">base</span> <span class="o">=</span> <span class="nb">sum</span><span class="p">(</span><span class="n">weight</span> <span class="o">*</span> <span class="n">norm</span><span class="p">[</span><span class="n">name</span><span class="p">][</span><span class="n">i</span><span class="p">]</span> <span class="k">for</span> <span class="n">name</span><span class="p">,</span> <span class="n">weight</span> <span class="ow">in</span> <span class="n">WEIGHTS</span><span class="o">.</span><span class="n">items</span><span class="p">())</span>
</span></span><span class="line"><span class="cl">        <span class="n">scores</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">base</span> <span class="o">*</span> <span class="mf">0.5</span> <span class="o">**</span> <span class="p">(</span><span class="n">item</span><span class="p">[</span><span class="s2">&#34;age_hours&#34;</span><span class="p">]</span> <span class="o">/</span> <span class="n">HALF_LIFE_HOURS</span><span class="p">))</span>
</span></span><span class="line"><span class="cl">    <span class="n">order</span> <span class="o">=</span> <span class="nb">sorted</span><span class="p">(</span><span class="nb">range</span><span class="p">(</span><span class="nb">len</span><span class="p">(</span><span class="n">items</span><span class="p">)),</span> <span class="n">key</span><span class="o">=</span><span class="k">lambda</span> <span class="n">i</span><span class="p">:</span> <span class="n">scores</span><span class="p">[</span><span class="n">i</span><span class="p">],</span> <span class="n">reverse</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">    <span class="k">return</span> <span class="p">[</span><span class="n">items</span><span class="p">[</span><span class="n">i</span><span class="p">]</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="n">order</span><span class="p">]</span></span></span></code></pre></div></div>
<p>The weights are guesses until the tests below say otherwise. Views and likes can still go in; they just become one input among several instead of the whole score. The half-life decides how quickly old posts sink, which is a topic of its own: I went into it in <a href="https://jared.lynskey.co.nz/en/posts/2023/2023-09-27-simulating-views/" >how feed ranking algorithms decide when to bury old posts</a>. Real systems usually learn the weights with a model rather than picking them by hand, but the shape is the same: many signals in, one number out, sort.</p>

<h3 class="relative group">Then test it honestly
    <div id="then-test-it-honestly" class="anchor"></div>
    
    <span
        class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none">
        <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#then-test-it-honestly" aria-label="Anchor">#</a>
    </span>
    
</h3>
<p>However clever the signals, A/B test them — show new content to a slice of users and compare its engagement against the incumbents. And watch how things spread: shares, mentions and embeds, on and off the platform, say more about value than a view counter ever will.</p>
<p>No single metric survives contact with reality. The rankings that actually work blend behaviour, intrinsic quality, timeliness and honest experiments, and no one signal gets the final word.</p>
]]></content:encoded>
      <media:content url="https://jared.lynskey.co.nz/en/posts/2023/2023-10-06-reinenforcement-learning/featured.jpeg" medium="image" type="image/jpeg" />
    </item>
  </channel>
</rss>
