Bugs to Revenue: Ad Slot Experiments at SideQuest

Context

SideQuest runs a self-serve ad platform that lets app developers pay for placement across the site. I owned the product side (UX, the placement decisions, and performance monitoring) while our Content team managed direct developer relationships.

Problem

Two separate bug reports came in around the same time, both about inflated impression counts. Developers were seeing high impressions but low CTR and wanted to know why. The instinct from engineering was to treat both as technical bugs and fix the counting logic. I thought the more interesting question was whether the UX itself was the problem.

Hero Carousel Slot

The homepage hero carousel was our best-performing ad placement, but CTR had plateaued and some developers were frustrated by impression counts that didn't match their intuition. Working with our data scientist, I pulled the numbers and we started asking why impressions were climbing without a corresponding lift in clicks.

The answer turned out to be the autoscroll. Every time the carousel looped back to the first slide, it logged another impression, so a user sitting on the homepage would rack up impressions on that first slot without ever seeing a new ad. Technically it was working as implemented. But it was inflating counts in a way that made performance look worse than it was for developers, and it raised a real question: was the autoscroll actually serving users, or was it just inherited behavior from the old site design?

I brought the hypothesis to the team. Our design lead liked the autoscroll from a UX perspective and didn't want to remove it without a reason. So I dug into the engagement data: it turned out users on this carousel were already clicking through slides manually at a higher rate than typical carousels. The autoscroll wasn't driving discovery, users were doing that themselves. That gave us the justification to stop it, which also resolved the impression inflation cleanly. Fixing the count logic alone would have hidden the symptom without addressing the underlying UX question.

Search Ad Slot

The search ad slot had been effectively dead for as long as anyone could remember: CTR was hovering between 0.00% and 0.05%. The bug report here was similar: impressions were getting counted multiple times when users typed incrementally into the search bar. Each keystroke re-fetched results and pulled a new ad impression. Engineering had a fix ready: just don't re-fetch the ad on query refinement.

Before approving that fix, I wanted to understand why the ad was in the quick search panel in the first place. It felt like an awkward placement; the panel is a transient UI, not somewhere that users linger. I went to the Wayback Machine to look at older versions of the site, then confirmed with the design lead who'd worked on the last major redesign: it had always been there, and nobody had explicitly decided it should be. It was inherited.

This opened more questions. If the placement was wrong, fixing the impression bug was just polishing a bad decision. The CTR would be accurate, but still too low to be a compelling purchase. I collaborated with design, marketing, and engineering to test a hypothesis: move the ad to the full-screen search results page, where users actually spend time and have clearer selection signals. That placement also made the double-counting bug irrelevant, since results are fetched differently there.

In both cases I monitored telemetry after the changes to confirm there were no meaningful drop-offs in user behavior. We didn't want to trade ad performance for engagement. For the search slot specifically, the fullscreen results page was already seeing consistent daily traffic in the 20K range before the change, which validated the core hypothesis: users were already going there, the ad just wasn't.

Outcome

The hero carousel change lifted CTR by 41–80% depending on the advertiser, based on a case study marketing ran with the first cohort of affected apps. Developer sentiment improved measurably. They'd been frustrated by metrics that didn't make sense, and now the numbers reflected reality. Marketing used the results as an evergreen doc for developer sales conversations, and our CTR compared favorably against Facebook and similar ad benchmarks.

The search slot went from essentially zero (0.00–0.05% CTR) to 0.5–1.0%, putting it on par with our other placements. Developers who'd written it off responded positively when we shared the results, both to the numbers and to the reasoning behind the change.

What I'd Do Differently

On the hero carousel, we made a judgment call to stop the autoscroll based on existing engagement data, but we didn't run a formal A/B test. We were confident enough in the signal to move, but a controlled experiment would have given us cleaner attribution and a stronger story for developers. It also would have let us test variations, like adjusting the scroll timing rather than removing it entirely. That's the right move if we revisit it.

On the search slot, I'd push to question inherited UX decisions earlier and more systematically. The placement had been wrong for years and nobody had stopped to ask why it existed. The bug report was the forcing function that finally surfaced it, but that's not a great way to find these things. A periodic audit of "why does this work this way" is worth building into the process, not waiting for a complaint to prompt it.