Welcome to our podcast, where we delve into cutting-edge advancements in personalization! Today, we're highlighting MCM: A Multi-task Pre-trained Customer Model for Personalization, developed by Amazon LLC.

This innovative BERT-based model, with 10 million parameters, revolutionises how e-commerce platforms deeply understand customer preferences and shopping intents. Its novelty stems from significantly improving the state-of-the-art BERT4Rec framework by handling heterogeneous customer signals and implementing multi-task training. Key innovations include a random prefix augmentation method that avoids leaking future information and a task-aware attentional readout module that generates highly specific representations for different items and tasks.

MCM’s applications are extensive, empowering diverse personalization projects by providing accurate preference scores for recommendations, customer embeddings for transfer learning, and a pre-trained model for fine-tuning. It excels in next action prediction tasks, outperforming original BERT4Rec by 17%. While generally powerful, for highly specific behaviours like those driven by incentives, fine-tuning MCM with task-specific data can yield even greater improvements, driving over 60% uplift in conversion rates for incentive-based recommendations compared to baselines.

Discover how MCM is shaping the future of personalised e-commerce experiences!

Find the full paper here: https://assets.amazon.science/d7/a5/d17698634b70925612c07f07a0fa/mcm-a-multi-task-pre-trained-customer-model-for-personalization.pdf

Welcome to our podcast! Today, we delve into groundbreaking research from Google DeepMind and Johns Hopkins University titled "On the Theoretical Limitations of Embedding-Based Retrieval". This paper uncovers a fundamental flaw in the widely used single-vector embedding paradigm: the number of unique top-k document combinations an embedding model can represent is inherently limited by its dimension.

Despite the common belief that better training or larger models can overcome these issues, the researchers demonstrate these theoretical limits in surprisingly simple, realistic settings. They introduce LIMIT, a novel dataset that exposes how even state-of-the-art embedding models severely struggle with straightforward tasks, scoring less than 20 recall@100 in some cases, due to these theoretical underpinnings. This suggests that existing academic benchmarks might be inadvertently hiding these limitations by testing only a minute fraction of possible query-relevance combinations.

This work calls for a re-evaluation of how we approach information retrieval. While single-vector embeddings are powerful, their capacity for handling diverse, instruction-following queries with complex relevance definitions is fundamentally capped. The paper suggests exploring alternative architectures like cross-encoders, multi-vector models, or sparse models to address these limitations. Tune in to understand why pushing the boundaries of current embedding models requires a shift beyond the single-vector paradigm.

Find the full paper at: https://arxiv.org/pdf/2508.21038

Welcome to our podcast, where we delve into cutting-edge AI in e-commerce! Today, we're exploring ItemSage, Pinterest's innovative product embedding system for shopping recommendations. Developed by engineers at Pinterest, ItemSage revolutionises how users discover products across Home, Closeup, and Search surfaces.

A key novelty is its transformer-based architecture, combining both text and image modalities to create rich product representations, significantly outperforming single-modality approaches. ItemSage also leverages multi-task learning to optimise for diverse engagement objectives, including purchases and add-to-cart actions, making the recommendation funnel more efficient, particularly for sparse labels. This unified embedding system, compatible with existing PinSage and SearchSage embeddings, drastically reduces infrastructure and maintenance costs by three times across different recommendation verticals.

While ItemSage has delivered substantial gains—up to +7% Gross Merchandise Value per user and +11% click volume in online A/B experiments—future work aims to enhance text feature modeling with pre-trained Transformers. Join us to understand this powerful system transforming shopping at Pinterest!

Paper link: https://arxiv.org/pdf/2205.11728