Was this email forwarded to you? Get Behavior Analytics​

---

Imagine having breakfast with the guy who inspired “Crawl Walk Run” and other super-smart people, talking about books, philosophy, and what AI means to humanity.

Imagine having dinner with three French men talking about products, sports, and what matters (family, friends, and living).

Imagine drinking what Forbes calls “The Best Whiskey in the World”,.

Awesome!

That happened at the "Experimentation Island 2025" conference, which Ton Wesseling and Kelly Wortham hosted on St. Simon Island.

Today,

Let's explore the nuances of testing in the digital and physical worlds.

🤔What’s *Digital* Experimentation?

Digital Experimentation refers to improving online website conversions and software product engagements.

Specifically,

Digital/online experimentation benefits from rigorous A/B testing, behavioral science insights, and data-driven decisions to optimize user experiences.

For example,

Scale and Speed: Online experiments can be deployed quickly and reach a large audience. For example, Aleksander Fabijan said that Microsoft conducts over 100,000 A/B tests annually.

Precise Measurement: Digital environments enable precise tracking of user behavior. For example, click-through rates, conversion rates, and time spent on the page can be accurately measured.

Controlled Environment: A/B testing ensures that users are randomly assigned to different variations, minimizing bias and ensuring observed differences are due to the tested changes.

Cost-Effectiveness: Online experimentations are relatively cost-effective because the replication cost of digital content is close to zero. The technical complexity and costs align with scale requirements.

Behavioral Science: Designers can create effective engagements by understanding how contextual factors influence decisions.

For example, Bookings.com increased conversations by inducing scarcity and fear of missing out (1 room left at this cost) and social proof (7 people booked rooms in this hotel in the last hour).

The controlled testing environments, cost and scale challenges, and statistical rules break down in physical stores.

And yet,

Physical stores offer the unique advantage of directly observing and interacting with people in real time during the shopping journey in their local environments.

😀*In-Store Experimentation* with Behavior Analytics

Behavior Analytics combines quantitative, qualitative, and zero-party research. It means “you cannot get closer to your customers than in a physical store.”

For example,

Real-World Context: Physical retail experiments provide insights into consumer behaviors in real-world shopping environments.

For example, you can track behaviors before payment to identify why 73% of store visitors did not purchase.

Sensory Experience: Physical retail allows experimentation with sensory elements, such as lighting, music, and scent. For example, remember that Christmas music during the holidays.

A less obvious example is how Costco deploys "less is more" by limiting items to one brand and one private lable per category.

Direct Customer Interactions: The store’s staff can interact directly with customers, gathering feedback and observing their reactions.

For example, tracking shopping behaviors in apparel stores gives you insights into trends such as design, color, and fabrics.

Community Engagement: Stores can excel in serving as community hubs. While this is a prominent feature of stores in Latin America and rural areas, you can still foster engagement with local events and other community-focused initiatives.

Brand Building: Well-designed in-store experiments can enhance brand perception and create memorable customer experiences, fostering loyalty and advocacy.

For example, Nike has seven categories of stores that target different audiences while maintaining the brand.

Basically,

If you ditch the *data-first* and focus on *people-first* methodologies, you will be a phenomenal Store Optimizer. 🤣🤣🤣

🤩Experimentation for *Store Optimizers*

Start with the problem you want to solve.

I start with the building blocks of the Win-Win-Win Alignment.

The formula is:

THE [BUSINESS USER] WANTS TO ACHIEVE AN [OUTCOME] BY MANAGING A [BEHAVIOR] WITH [TECHNOLOGY SOLUTION]

For example,

✍️ The category manager wants to increase shelf sales by 5% by tracking switching behaviors with computer vision technology.

✍️The workforce manager wants to improve the customer experience score by scheduling with a 90% or higher service success ratio.

✍️The brand manager wants to sell 100 perfume bottles for impulse buyers with a 30% discount promotions on Valentine’s Day.

✍️The store designer wants to increase sales per square foot by 1.5% by evaluating the effectiveness of the customer’s field of view.

✍️The customer service manager wants to reduce customer service outliers to less than 1% by using an escalation policy that notifies the store managers in real time.

As Paula Sappington from Hilton Hotels emphasized, online and offline experiments can enhance customer understanding and drive better results.

As Nike says, “Just Do It”.

Cheers,

​