エピソード

  • How Tiny Button Effects Kill Your A-B Test Results

    How Tiny Button Effects Kill Your A-B Test Results
    2026/06/07
    Lucas and Luna dig into the 'button effect' — the subtle, often invisible ways that changing one element in an A/B test (like a button's size, color, or wording) can ripple through user behavior and distort results. They use a real example from a SaaS company that tested a green versus red CTA button: the red button won by 12%, but when they dug deeper, they found it was actually driving fewer high-value signups. They explain the concept of 'interaction effects' where a button change affects other metrics like time-on-page and scroll depth, and how to isolate those effects with multivariate testing and guardrail metrics. The episode offers a practical framework: always measure secondary metrics, run five-minute user tests before launching a full A/B test, and use a 'button effect audit' checklist to avoid misleading conclusions. #ABTesting #ConversionRateOptimization #ButtonEffect #UserExperience #SaaS #CTAButton #MultivariateTesting #GuardrailMetrics #InteractionEffects #LandingPage #CROStrategy #DigitalMarketing #UserBehavior #FexingoBusiness #BusinessPodcast #MarketingPodcast #DataDrivenMarketing #ConversionOptimization Keep every episode free: buymeacoffee.com/fexingo
    続きを読む 一部表示
    6 分
  • How Your A-B Test Results Disappear After Ship

    How Your A-B Test Results Disappear After Ship
    2026/06/06
    In this episode of CRO with Fexingo, Lucas and Luna dig into the phenomenon of vanishing lift: why an A-B test that shows a clear winner in the experiment often fails to move the needle after the winning variant goes live. Drawing on a case study from Etsy, where a button redesign showed a 5 percent relative lift in the lab but zero impact in production, the hosts explore three root causes: novelty bias, audience mismatch, and the Hawthorne effect. They walk through specific fixes, including holdout groups, staggered rollouts, and pre-registering secondary metrics. If you've ever shipped a winner only to see flat conversion rates, this episode explains why and what to do about it. #A-BTesting #CRO #Marketing #Etsy #ConversionRateOptimization #NoveltyBias #HawthorneEffect #AudienceMismatch #HoldoutGroups #StaggeredRollout #VanishingLift #ExperimentDesign #DataDriven #StatisticalValidity #FexingoBusiness #BusinessPodcast #MarketingPodcast #CROStrategy Keep every episode free: buymeacoffee.com/fexingo
    続きを読む 一部表示
    9 分
  • Why Your A-B Test Results Disappear After You Ship

    Why Your A-B Test Results Disappear After You Ship
    2026/06/06
    Episode 34 of Conversion Rate Optimization with Fexingo tackles a painful, rarely discussed failure mode: the winner that vanishes the moment you deploy it. Lucas and Luna dig into the 'novelty effect' and the 'Hawthorne effect' — two behavioural phenomena that inflate A-B test results during the experiment but evaporate once the change becomes permanent. They walk through a real case study from an e-commerce company that saw a 15 percent lift for a redesigned checkout flow during the test, only to see conversion revert to baseline within two weeks of launch. The hosts break down the diagnostic steps CRO teams can take: running longer tests to measure decay rates, building a 'persistence metric' into your testing framework, and differentiating between treatments that work because they're genuinely better versus treatments that work because users are paying attention to something new. If you've ever shipped a winning variant that quietly stopped winning, this episode explains why — and how to stop wasting resources on illusions of improvement. #NoveltyEffect #HawthorneEffect #ABTesting #CRO #ConversionRateOptimization #StatisticalValidity #BehavioralEconomics #ExperimentDesign #PersistenceMetric #DecayRate #Ecommerce #CheckoutOptimization #LandingPageTest #DataReliability #MarketingScience #FexingoBusiness #BusinessPodcast #Business Keep every episode free: buymeacoffee.com/fexingo
    続きを読む 一部表示
    12 分
  • How a Single Font Change Lifted Conversion by 12 Percent

    How a Single Font Change Lifted Conversion by 12 Percent
    2026/06/05
    In this episode of Conversion Rate Optimization with Fexingo, Lucas and Luna explore a case study that often gets overlooked: how a small e-commerce brand improved its checkout conversion rate by 12% simply by changing its font. They break down the psychology behind typeface readability, the specific A/B test design (including sample size and duration), and why the brand's original font—a stylish but hard-to-read serif—was costing them sales. The discussion also covers how the test was structured to avoid common pitfalls like peeking and insufficient sample sizes, and why the results held up over a full two-week run. Finally, they touch on how font choice interacts with other design elements like button color and spacing, offering practical takeaways for any marketer running CRO tests. Listeners will learn one concrete tip they can apply to their own landing pages or checkout flows today. #FontAesthetic #FontChoiceCRO #ReadabilityExperiment #TypefacePsychology #CheckoutConversion #ABTesting #MarketingPodcast #CROStrategy #LandingPageOptimization #UserExperience #ConversionRate #SmallEcommerceBrand #SerifVsSansSerif #BusinessPodcast #FexingoBusiness #PodcastShow #OptimizationTips #DataDrivenMarketing Keep every episode free: buymeacoffee.com/fexingo
    続きを読む 一部表示
    6 分
  • The One Metric That Makes A-B Tests Lie

    The One Metric That Makes A-B Tests Lie
    2026/06/05
    Lucas and Luna dig into one of the most misunderstood pitfalls in A-B testing: how your chosen success metric can make a perfectly run experiment give you the wrong answer. Using Booking.com's infamous failed test of a 'Book Now' button — where conversion rate went up but revenue per visitor went down — they explain the difference between a metric that's easy to measure and one that actually matters. They walk through why 'click-through rate' can be a vanity metric, how Amazon prioritizes 'units per session' over 'purchase rate', and why the right north-star metric forces you to define what 'better' really means before you start. No jargon, just a concrete framework you can apply to your next test. #ABTesting #VanityMetrics #ConversionRate #SuccessMetric #Bookingcom #RevenuePerVisitor #NorthStarMetric #MarketingExperiments #DataDriven #LandingPageOptimization #ClickThroughRate #CROStrategy #BusinessPodcast #MarketingPodcast #FexingoBusiness #ConversionRateOptimization #Experimentation #AmazonStrategy Keep every episode free: buymeacoffee.com/fexingo
    続きを読む 一部表示
    9 分
  • The One Question That Makes Every A-B Test Smarter

    The One Question That Makes Every A-B Test Smarter
    2026/06/04
    Most A-B tests ask 'which variant wins?' but the smartest CRO teams ask a different question first: 'what is the smallest change that could produce the largest effect?' In this episode, Lucas and Luna break down the 'minimum viable change' framework—why testing big redesigns is usually a waste of time, how Amazon's one-click button emerged from an M.V.C. mindset, and why Booking.com's famously relentless testing culture hinges on incrementality. They also dig into a 2025 study from Unbounce showing that pages with a single focused change outperformed full redesigns 73 percent of the time. If your M.V.P. tells you what to build, your M.V.C. tells you what to test. No fluff, just a sharper way to think about every experiment you run. #CRO #ABTesting #MinimumViableChange #ConversionRateOptimization #Marketing #LandingPages #Experimentation #Bookingcom #Amazon #Unbounce #IncrementalGains #DataDrivenMarketing #BusinessGrowth #DigitalMarketing #Optimization #FexingoBusiness #BusinessPodcast #PodcastEpisode Keep every episode free: buymeacoffee.com/fexingo
    続きを読む 一部表示
    8 分
  • Why Your A-B Test Changes Don't Move the Needle

    Why Your A-B Test Changes Don't Move the Needle
    2026/06/04
    Episode 30 of Conversion Rate Optimization with Fexingo. Lucas and Luna tackle one of the most frustrating problems in CRO: you run a textbook A-B test, get a statistically significant winner, implement it, and nothing happens to overall revenue. They break down the 'dilution effect' — a concept from multi-armed bandit theory — using Amazon's real-world experience with search result page tests. You'll learn why small changes in low-traffic areas rarely move aggregate metrics, how to identify tests that can actually move the needle, and why Amazon rerouted its entire testing philosophy around this one insight. No theory without application: they walk through a specific 2025 example from a mid-market e-commerce site that wasted six months on a test that was never going to matter. #ConversionRateOptimization #ABTesting #DilutionEffect #Amazon #CROStrategy #MultiArmedBandit #StatisticalSignificance #Ecommerce #Marketing #FexingoBusiness #BusinessPodcast #DataDriven #Experimentation #Productivity #WebAnalytics #UserExperience #LandingPage #TestDesign Keep every episode free: buymeacoffee.com/fexingo
    続きを読む 一部表示
    12 分
  • The Sample Size Fallacy That Ruins Your A-B Tests

    The Sample Size Fallacy That Ruins Your A-B Tests
    2026/06/03
    Most marketers think bigger sample sizes always mean better A-B test results. In this episode, Lucas and Luna break down the sample size fallacy — why too large a sample can detect statistically significant but practically meaningless effects, and how Booking.com discovered this lesson the hard way when a test with 2 million users showed a 0.1 percent lift that vanished on rollout. They walk through the math behind minimum detectable effect, explain why a test with 500,000 users can be worse than one with 5,000, and share a simple framework for choosing sample size based on business impact rather than statistical purity. Plus: how Netflix avoids this trap by pre-registering effect sizes before every test. If you run A-B tests, this episode will save you from wasting months chasing irrelevant wins. #A-BTesting #SampleSizeFallacy #CRO #ConversionRateOptimization #Marketing #Bookingcom #Netflix #MinimumDetectableEffect #StatisticalSignificance #DataScience #BusinessStrategy #Experimentation #Analytics #MarketingPodcast #FexingoBusiness #BusinessPodcast #CROStrategy #ABTestMistakes Keep every episode free: buymeacoffee.com/fexingo
    続きを読む 一部表示
    10 分