Production audit · 60-day reconstruction
The data is rich.
The plumbing distorts it.
The deeper read found enough behavioral evidence to personalize well—but several implementation details inflate freshness, duplicate purchases, exclude high-intent users and let repeated rows masquerade as broader taste.
Fix signal truth before tuning weights or training a larger model.
PriorityFailureEvidenceRequired correction
P0Purchase double-count + false recency
Almost every transaction-buy wallet row duplicates a cart purchase. Backfilled records use ingestion time, including 96.6k events older than 180 days.
Implemented locally: wallet attendance contributes native wallet facts only; completed cart purchase owns the buy and occurrence time drives recency.
P0Active-user selector omits strong lanes
The embedding population query includes views, likes, alerts and wallet attendance—but not direct purchase, bid, listing, sale or search sources.
Build eligibility from the same canonical signal registry used by embedding construction.
P1Three rows ≠ three taste facts
32.1% of embeddings sit exactly at the old three-row minimum; 7,479 embedded active users had evidence from one event.
Implemented locally: canonicalize repeated event/kind rows, require three facts across two events, and delete stale vectors/cache entries.
P1Cart reserve lane is effectively dead
Current state yields 0 recent user-event pairs; history contains 12,862 never-completed buyer-intent pairs.
Implemented locally: reconstruct reserve intent from cart + cart_history and exclude every cart that ever completed.
P1Search identity bridge
All 547,475 existing search rows are anonymous because web and mobile dropped auth on the logging request.
Fixed in both clients: signed-in search logs now carry the same bearer token as search; historical anonymous rows remain aggregate-only.
P1Event co-attendance coverage
Only 15 of 4,203 event vectors have any co-attend input. Artist-only vectors previously implied a richer blend.
Implemented locally: store applied components and exact performer/co-attendance coverage on every vector.
P2Future wallet demand is concentrated
20,973 future user-event holdings can be suppressed; the top five resolved performer clusters hold 31.8% of 20,261 holdings.
Implemented locally: 75% affinity, p90-capped demand, held-event exclusion and at most two cards per primary performer.
Who gets a vector
Coverage by active-user cohort
Corrected 60-day facts exclude duplicate transaction-buy attendance. Seller = listing or sale; intent = bid, alert or cart.
Commercial association
Pairs that also purchased
Same-user, same-event overlap—not causal lift. Bid is a near-conversion action and should not be mistaken for broad taste.
History leverage
Repeat view depth is recoverable
40.1% of 1.07M view pairs repeat. Current-state sourcing keeps only the last view and throws away frequency and acceleration.
644k1 view
130k2 views
136k3–5
72.5k6–10
92.2k11+
Rankable supply
Upcoming event-vector coverage
Even a valid user vector cannot score events without an event vector. Content, popularity and editorial fallbacks remain mandatory.
System diagnosis
Enough signal.
Fix truth, then learn.
CrowdVolt already has purchases, wallet history, carts, asks, bids, likes, views, editorial taste and a dense DICE co-holding graph. First correct timestamps, deduplication and eligibility; then add exposure logging and learn how the cleaned signals should rank.
Build hybrid retrieval now; do not train on duplicated or falsely fresh facts.
Target system
One loop, six explicit stages
Every stage can improve independently. Candidate generators optimize recall; the ranker optimizes relevance; re-ranking protects product and editorial intent.
- upcoming inventory
- performer + venue metadata
- availability / market
- exposure + rank
- action + context
- identity confidence
- user ↔ event vectors
- DICE co-holding graph
- editorial buckets
- metro popularity
- taste affinity
- commercial intent
- novelty + distance
- availability
- diversity
- scene/editorial mix
- freshness
- frequency caps
- holdouts
- lift by signal
- calibration
- drift monitoring
Catalog coverage
Active buckets vs upcoming events
Covered means present in current auto or persistent curated buckets—not personalized or evaluated.
What the data says
Three findings that change the build
Wallet data is a candidate engine
Use co-holding to retrieve scenes and event neighborhoods. Downweight free/guest-like observations and separate possession from scan attendance.
medium causal certainty
Search identity starts forward-only
547,475 historical rows remain anonymous. New signed-in web/mobile rows carry auth after the client bridge ships.
ready
Editorial should constrain, not disappear
Nine curated scenes encode brand taste. Use them as a lane and a re-ranking policy, then let the graph find residual audiences.
80 events
Signal inventory
Behavioral evidence, with history intact
Bars are log-scaled so low-volume, high-intent events remain visible. “Current” means live table state; “History” means temporal history rows.
History preserves state transitions, not independent people. Counts are evidence volume, not training labels. The ranker should deduplicate into user × event × action facts first.
Exposure
What the user could have seen
PostHog has 3.07M event views in 90 days, but recommendation surface, rank and model version are not consistently attached. Without exposure, no offline label can distinguish “ignored” from “never shown.”
Identity
Which signals personalize
Purchases, wallets, carts, bids, asks, alerts and likes join to users. Search joins only for new signed-in rows after the web/mobile auth bridge; anonymous history stays market demand.
Outcome
What success means
Optimize a ladder: view → detail → intent → purchase. Train separate heads or calibrated probabilities; do not make a low-frequency purchase label erase discovery value.
DICE wallet topology
The audience graph is already a map
1,873 events with at least 40 wallets; 8,396 Jaccard edges. Possession is a proxy for affinity, not proof of attendance.
Scale
15,241wallets across 27,564 unique events
Retrieval guardrail
2 maxcards per primary performer after held-event suppression
Caveat
15.7%historical pairs look guest/free-like; filter or downweight
Human taste × discovered structure
Editorial protects the graph's blind spots
Auto buckets recover 60% of curated events, but the ≥40-wallet graph maps only 18 of 97 placements. Music First, Out East and Scene Favorites are precisely where human taste contributes most.
Current production
Persistent curated collections
Operating rule
Keep the editorial voice, remove the bottleneck
Give curated collections a protected quota, use audience/behavioral candidates to fill coverage gaps, and surface “why this entered” to editors. Membership locks remain authoritative.
Recommended mix is a constraint to test, not a fixed percentage.
Week-versioned automation
Auto bucket evolution
A stable 52-bucket ceiling with one partial/paused week. Volume stability says the generator runs; it does not establish recommendation quality.
Production embedding health
The vectors are useful—but narrower than their names
User vectors are recency-weighted engagement summaries. Event vectors are performer means in practice: only 15 of 4,203 production rows have any co-attendance contribution.
source: user/engagement-weighted-recency/v1
median 5; long tail remains sparse
projection/artist-mean+optional-coattend/v2
0.36% of 4,203 event vectors
Memory horizon
Current user taste window
Signal window
60 days
Half-life
14 days
Durable history
underused
Keep two memories: fast intent (days/weeks) and durable taste (wallet/purchase/attendance over years). Blend at rank time based on session context.
Metadata readiness
Content features are uneven
Content is sufficient as a candidate lane, not as the sole model. Performer genre/track coverage is the main cold-start gap.
Source mix
Catalog joins already create leverage
External snapshots contribute different features; normalize provenance instead of flattening them into one opaque event record.
Sequenced build
Correct the facts before asking them to learn
The first three steps are plumbing, but production analysis shows they are not optional: the current facts contain duplicated purchases, false freshness and population-selection gaps.
Patch signal truth in production
Deduplicate transaction attendance against cart purchases, restore actual occurrence timestamps, union active users from the canonical signal registry and gate embeddings on distinct event support.
label corruption
Ship a recommendation exposure contract
Log surface, request id, user/anonymous id, event id, rank, lane, score components, model version, bucket/editorial source and timestamp. Join subsequent actions to that exposure.
counterfactual labels
Materialize the interaction fact
Collapse current + history + PostHog into one identity-confidence-aware user × event table. Separate exposure, attention, intent, conversion and negative/ambiguous evidence.
training + audits
Stand up four candidate lanes
Behavioral vector similarity, DICE co-holding neighborhoods, editorial/LLM buckets and contextual popularity. Measure recall per lane before blending.
coverage + diversity
Train or calibrate the ranker
Start interpretable: logistic/GBDT or calibrated weighted model with recency and context features. Learn weights from downstream lift conditional on exposure; do not vote percentages by intuition.
relevance lift
Add policy re-ranking and editor tooling
Apply availability, metro, diversity, freshness, frequency caps, protected editorial quota and membership locks. Show editors the origin and reason for every candidate.
brand-safe scale
Operate with holdouts and drift checks
Track view→detail, intent and purchase lift, catalog coverage, novelty, concentration, calibration and source drift. Keep a stable control and per-lane ablations.
continuous learning
Do now
Correct facts + exposure
Stop corrupting recency and identity first; then make ranking outcomes measurable.
Do next
Hybrid retrieval
Turn existing wallets, embeddings and editorial structures into complementary lanes.
Do not do
One giant embedding
It will hide identity, exposure, recency and policy problems behind a similarity score.