Episodes

  • IntelliJAMS EP 060: The Frequently Unanswered Questions

    IntelliJAMS EP 060: The Frequently Unanswered Questions
    Jun 8 2026

    Your product page probably does a decent job explaining what you sell. But is it answering the questions customers are asking themselves and never typing into a chat widget? Adrian Stewart, co-founder of Scale Messaging, has a framework for finding those gaps, and it can change the way you approach A/B testing your messaging.

    In this episode of IntelliJAMS, Alex and Adrian dig into the "Frequently Unasked Questions" framework: four categories of questions shoppers silently ask themselves while browsing your site. They cover where most Shopify brands fall short on messaging, why reducing friction is easier than building motivation (but both matter), when urgency tactics actually work versus when they erode trust, and how to build hypotheses around messaging that you can test and learn from.

    Timestamps:
    0:00 - Intro and the Frequently Unasked Questions framework
    0:44 - Why "unasked" questions matter more than FAQs
    1:57 - The four categories: understanding, motivation, difference, trust
    4:26 - Motivation vs. friction and how they drive behavior
    6:35 - Why motivation is harder to build than friction is to reduce
    7:40 - Urgency as a bonus category (and when it gets hacky)
    9:56 - Fake scarcity vs. real scarcity: the windscreen wiper example
    11:22 - Difference: competing within your range, against competitors, and against inertia
    14:09 - How to figure out which unasked questions to prioritize
    17:13 - Message, expression, and placement: the three layers of testing
    19:02 - Why one test usually leads to five more questions
    20:15 - Where to start: trust is the fastest win, difference is the biggest opportunity
    24:07 - Recap and where to find Adrian

    Topics covered:

    The Frequently Unasked Questions framework (understanding, motivation, difference, trust)

    Why "difference" is the most overlooked messaging gap on product pages

    The motivation-to-friction ratio and how it affects conversion

    When urgency and scarcity tactics help vs. hurt your brand

    How traffic source (paid social vs. search) should shape your messaging strategy

    Building messaging hypotheses you can A/B test

    Message vs. expression vs. placement as three layers of experimentation

    Why trust is the fastest win for most e-commerce brands

    Ready to start testing your messaging? Join GEM Academy for free courses and a community of brands sharing what works: https://www.skool.com/intelligems-academy-1535

    Connect with Adrian / Scale Messaging:

    Website: https://scalemessaging.com
    LinkedIn: https://www.linkedin.com/in/adrianjstewart/

    Connect with Intelligems:

    Website: https://intelligems.io
    Blog: https://intelligems.io/blog
    LinkedIn: https://www.linkedin.com/company/intelligems
    Show More Show Less
    26 mins
  • IntelliJAMS EP 059: Why Post-Purchase Is the Money Moment for Shopify Brands

    IntelliJAMS EP 059: Why Post-Purchase Is the Money Moment for Shopify Brands
    Jun 1 2026

    Post-purchase upsells sit at the exact moment a customer has already committed — credit card swiped, conversion done. That means there's zero risk of hurting conversion and pure upside potential for your AOV.

    In this episode, we break down why post-purchase might be the highest-leverage Shopify A/B testing opportunity most brands haven't explored yet.
    In this episode of IntelliJAMS, Adam and Alex explore the economics of post-purchase upsells — from a CBD brand that grew AOV 20% overnight to the consumption psychology that makes "buy more now" actually better for long-term LTV.

    Timestamps:
    0:00 - Intro
    0:22 - Why post-purchase upsells are worth your attention
    1:04 - "Be close to the money" — does post-purchase qualify?
    1:52 - What brands are doing with post-purchase today
    2:11 - The CBD gummies case study: 20% AOV lift overnight
    3:32 - Why there's zero downward pressure on conversion
    5:05 - The consumption psychology surprise: more supply = faster consumption
    6:09 - Scarcity vs. abundance mindset and second-order effects
    8:04 - Three post-purchase upsell strategies that work
    9:35 - Matching upsell strategy to your margin profile
    10:26 - Using post-purchase for inventory clearance and sell-through
    11:02 - How Intelligems handles post-purchase testing and measurement
    12:28 - Why post-purchase tests can run in parallel with everything else

    Topics covered:

    The economics behind post-purchase upsells (zero incremental CAC, no conversion risk)

    A real-world CBD brand case study: 50% off second pack, 40% take rate, 20% AOV increase

    Three post-purchase strategies: same product discount, complementary products, and clearance/inventory sell-through

    Why consumption scales with supply — and what that means for repurchase rates

    How to match your upsell strategy to your margin profile

    Running post-purchase tests in parallel without interaction effects

    How Intelligems measures incrementality, revenue, and profit on post-purchase offers

    Want to start testing post-purchase upsells (or anything else)? Join GEM Academy for free courses and a community of brands sharing what works: https://www.skool.com/intelligems-academy-1535

    Connect with Intelligems:

    Website: https://intelligems.io
    Blog: https://intelligems.io/blog
    LinkedIn: https://www.linkedin.com/company/intelligems
    Show More Show Less
    14 mins
  • IntelliJAMS EP 058: Outputs vs. Outcomes: A Better Way to Think About A/B Testing

    IntelliJAMS EP 058: Outputs vs. Outcomes: A Better Way to Think About A/B Testing
    May 27 2026

    Running 10 tests a month sounds productive, but if none of them are tied to a real business question, you're just taking swings for the sake of swinging. In this episode Adam and Alex dig into why "how many tests should I run?" is often the wrong question, and what to ask instead when you're building an experimentation program on Shopify.

    In this episode of IntelliJAMS, Adam and Alex explore the difference between test output and test outcomes, how to run concurrent tests without confounding your results, and why elevating the conversation from "number of tests" to "strategic priorities" changes everything.

    Timestamps:
    0:00 - Intro
    0:27 - The question everyone's asking right now
    0:52 - Why "how many tests" is the wrong question
    2:23 - Outputs vs. outcomes: reframing productivity
    3:10 - Test the plan, don't plan the tests
    3:51 - You can't predict how many tests it'll take
    4:54 - Running concurrent tests without confounding results
    5:53 - Scoping metrics to the right part of the funnel
    6:48 - Every test needs a hypothesis
    7:30 - One test often leads to five new questions
    8:15 - Real example: ending a checkout upsell test early
    9:11 - How to push back on "I need 10 tests this month"
    11:49 - Elevating the conversation from output to strategy
    13:28 - Final thoughts: see the forest for the trees

    Topics covered:

    Why counting tests is an output metric, not an outcomes metric

    How to reframe testing around strategic business goals

    Running concurrent A/B tests on Shopify without confounding data

    Segmenting tests by funnel stage and visitor type

    How one test can lead to five new questions

    Pushing back when stakeholders demand a test quota

    Real-world example: killing a checkout upsell test 24 hours in

    The "test the plan, don't plan the tests" framework

    Want to sharpen your experimentation skills? Join GEM Academy for free courses and a community of brands sharing what works: https://www.skool.com/intelligems-academy-1535

    Website: https://intelligems.io
    Blog: https://intelligems.io/blog
    LinkedIn: https://www.linkedin.com/company/intelligems
    Show More Show Less
    14 mins
  • IntelliJAMS EP 057: Calling out the anxieties of A/B testing

    IntelliJAMS EP 057: Calling out the anxieties of A/B testing
    May 27 2026
    A/B testing can feel high-stakes, especially in e-commerce. You launch a test, check it an hour later, and either think you've broken your store or discovered a goldmine. Neither is true yet. This episode is a therapy session for anyone who's felt the anxiety of running A/B tests on their Shopify store.In this episode of IntelliJAMS, Alex McEachern and experimentation expert Ally Petretti walk through the most common anxieties of A/B testing and how to manage them with better process instead of more stress.Should you check your A/B test results every day?Checking your test daily is like weighing yourself multiple times a day on a diet. The number isn't wrong, but it's not ready to be meaningful yet. Check to make sure the test is collecting data and firing correctly, but don't read into the directional results until you've hit your planned runtime and traffic thresholds.What is statistical significance and when should you end an A/B test?Statistical significance is often treated as a finish line, but reaching stat sig on day one doesn't mean you have a winner. Stat sig only looks at the math. You also need time (to account for day-of-week behavior changes, promotions, and the novelty effect) and volume (small sample sizes are fragile and can skew results dramatically). A test typically needs to run for at least one to two full weeks to account for these variables.What is the novelty effect in A/B testing?The novelty effect happens when returning visitors see something different on your site and react to the change itself rather than the actual experience. In price testing, this can look like a customer treating a different price as a discount and converting out of urgency. The novelty effect fades after a week or two as your sample includes more new visitors and repeat visitors adjust.What is metric shopping in CRO?Metric shopping is when you pick a winning metric after a test is already running, instead of sticking to the success metric defined in your hypothesis. For example, if your hypothesis targeted conversion rate but AOV happened to spike, declaring a win based on AOV is metric shopping. It's what Ally calls "the silent killer of CRO." The right move is to note the unexpected metric change, form a new hypothesis around it, and run a separate test.How should you communicate A/B test results?Don't just drop a list of metrics. That's left open to interpretation. Data storytelling means framing results around your original hypothesis, explaining what customers did and why, and tailoring the narrative for your audience. A CFO needs a different story than a CMO. Use tools like the Intelligems Slack bot or MCP integration to help frame results for different stakeholders.How do you handle test ideas that aren't backed by data?Ideas from teammates or Twitter threads aren't bad, but they need a hypothesis before they become tests. A strong hypothesis includes a data-backed problem, a proposed solution (the test idea), and a success metric. If an idea doesn't have that, either formulate the hypothesis yourself from existing data or add it to the backlog until you can.How do you coordinate A/B testing across teams?CRO teams and paid media teams often test independently, which creates noise in each other's data. The key is sharing hypotheses (not just test plans), coordinating testing calendars, and recognizing that every team is working toward the same growth goal. Regular cross-team syncs where you discuss challenges, upcoming campaigns, and testing angles can turn conflict into collaboration.Join GEM Academy for free courses and a community of brands sharing what works: https://www.skool.com/intelligems-aca...Connect with Ally Petretti:https://www.linkedin.com/in/ally-petretti-kuhn-47a27014/ Connect with Intelligems:Website: https://intelligems.ioBlog: https://intelligems.io/blog----0:00 - Intro: Why we need a CRO therapist1:05 - The #1 stressor in testing: urgency and timing1:44 - "I'm a genius" vs. "I'm killing the business" (early test reactions)2:33 - Peeking at tests: checking vs. reacting4:51 - Pre-test analysis: setting guardrails before you launch5:30 - Statistical significance isn't the finish line6:13 - The novelty effect and why it matters for price testing7:47 - Using the time series chart to read test stability8:39 - When to call a test early (the right way)10:12 - No test losers: every result is a learning10:59 - Hypothesis discipline and metric shopping12:18 - What metric shopping is and why it's the silent killer of CRO13:25 - Communicating test results: data storytelling15:57 - The three E's framework: Explore, Experiment, Extend17:25 - Understanding your customers through testing18:20 - "Feelings over data": handling test ideas without a hypothesis21:19 - Cross-team testing: getting paid media and CRO on the same page25:42 - Wrap-up and where to connect with Ally
    Show More Show Less
    27 mins
  • IntelliJAMS EP 056: Instead of planning your tests, test your plan

    IntelliJAMS EP 056: Instead of planning your tests, test your plan
    May 11 2026

    Conversion rate went up, but did profit? Most testing programs optimize for one metric in isolation, and that's exactly where they go wrong. In this episode, Amanda Siegel (Director of E-commerce at Scale Media) breaks down why she stopped calling what she does "CRO" five years ago and started treating testing as a tool to validate business strategy, not just chase website wins.

    In this episode, Alex and Amanda explore what happens when you balance conversion rate, AOV, and subscription adoption in every test instead of picking one, why the old UX tricks (spin-to-wins, slashed prices, CTA color swaps) are losing their edge, and how to get buy-in for short-term metric dips that lead to long-term growth.

    Timestamps:
    0:00 - Intro and why Amanda calls herself "a little crazy"
    1:20 - The limits of traditional CRO and flat testing
    3:00 - What's changed in the last 10 years of e-commerce UX
    4:17 - Consumers are catching on to optimization tricks
    6:10 - Why reducing discounts is actually improving conversion
    7:36 - Shifting from product sales to business strategy (bundles, subscriptions)
    9:02 - No silver bullets: what works for your business vs. someone else's
    10:20 - Conversion rate is one-dimensional and easy to game
    11:43 - The growth experiment mindset: testing for business growth
    13:58 - Intentional trade-offs and why short-term losses can be worth it
    15:10 - The two most important questions in commerce
    16:19 - Getting buy-in when a metric temporarily looks bad
    19:15 - "Testing is a tool, not a job title"
    21:14 - Diminishing returns of small, isolated tests
    22:58 - "Don't plan your tests. Test your plan."
    24:03 - Amanda's advice: be bold, back-plan from ideal outcomes
    25:39 - Why testing has to be cyclical, not one-and-done
    27:06 - Where to connect with Amanda

    Topics covered:

    Why conversion rate optimization (CRO) is too narrow for modern e-commerce

    Balancing conversion rate, AOV, and subscription adoption in a single test
    How consumer behavior has evolved and why old UX tricks are losing effectiveness

    The counterintuitive finding that reducing discounts can improve conversion
    Using A/B testing and experimentation to validate business strategy, not just website changes

    Getting executive buy-in for short-term metric trade-offs

    The growth experiment mindset for Shopify brands

    Why testing needs to be cyclical and revisited over time

    ---

    Want to think bigger about your testing program? Join GEM Academy for free courses and a community of e-commerce operators sharing what actually works: https://www.skool.com/intelligems-academy-1535

    Connect with Amanda Siegel: https://www.linkedin.com/in/amandadsiegel/

    Connect with Intelligems:

    Website: https://intelligems.io
    Blog: https://intelligems.io/blog
    LinkedIn: https://www.linkedin.com/company/intelligems
    Show More Show Less
    28 mins