Webinars
How The White Company Optimises Online Performance with Data and A/B Testing
182 views
View transcript
Good afternoon, everyone. Welcome to today's session. We are thrilled to be joined by Hannah from The White Company and Ella and Graham at Fred Hopper. Before we kick things off, I just want to draw your attention to the Q&A box at the bottom of the screen. If you have any questions at all during the session, please pop them in here and we'll get to them at the end. This session is being recorded and you'll receive a copy of this tomorrow. And without further ado, I'm going to hand you over to Ella and the team for introductions. Thanks, Maddy. So, yes, I'm Ella. I head up our customer success team for our Fred Hopper and XO customers. And yeah, I'll pass over to Graham now to introduce himself. Yeah, I'm Graham. I'm a senior technical consultant here at Crown Peak specialising in Fred Hopper. Work with many of our customers. And yeah, I will now hand over to Hannah. Hi, I'm Hannah. I'm the online trading manager at The White Company and I lead on our Fred Hopper tool, which manages our search and merch for The White Company. Thank you both. So I guess, yeah, over to you, Hannah. Could you start to kick things off by sharing how Fred Hopper integrates into your merchandising strategy? Yeah, so Fred Hopper is central to The White Company's online merchandising and browse strategy. We use it to optimise how products are displayed to customers on our PLPs. So whether that's we're prioritising our bestsellers, or we're supporting a new season campaign launch, or we're in promotional sale activity. But it also gives us as merchandisers control. For those that don't know, The White Company, especially as we're coming into peak, we're a key gifting destination. So if you think of like our gifting PLPs, we tend to rank these by bestsellers. But we have a very famous fragrance called Winter at The White Company. And when we rank by bestsellers, something like Winter dominates our gifting PLPs. So what we can also do in Fred Hopper is we can dynamically pull in different departments, showcasing that variety of products that we have at The White Company and what we can offer to customers. It also allows us to manipulate existing product attributes to enhance our front end experience without needing back end changes. And now it also provides flexibility for us to test and implement changes that improve the user experience and conversion. Thank you, Hannah. Thank you, Hannah. That's a really comprehensive overview. And it's great to kind of see how Fred Hopper as a tool kind of empowers your merchandising approach at The White Company. I guess it would be good to kind of get an overview of how kind of our partnership began and how your team became more comfortable with the tool. Yeah, so the collaboration kind of coincided with our contract renewal and aligned perfectly with our digital roadmap planning. A big part of our partnership. A big part of our partnership was about the desire to implement A-B testing. We knew Fred Hopper had that capability, but we were on an old version of Fred Hopper. So there was lots of steps that had to be taken to get us there. And we had some great discussions with both yourself and Graham as to how we could prioritise that. We were also a fairly new team into Fred Hopper, just where there'd been team turnover. So we also had allocated training hours in the contract. And I think I knew we needed to take the team on a bit of a journey and start from the basics and work our way up. So the Fred Hopper's consultant, consultancy services allowed that hands-on training in a low pressure environment, meaning we were just in a bit of a safe space to be able to ask questions and learn. And having that expert support for brainstorming and problem solving, has been invaluable. I think, for example, we were on a team training day with Graham, and he was able to make instant changes in the back end based on our feedback and just making our merchandisers jobs easier. Those sessions provided reassurance and confidence to experiment with features and optimizations that were new to the team. Also having those regular sessions like Merch Labs also encouraged that shared learning both within the team and across other users. Absolutely. And I think those collaborative sessions that we had at the start of the partnership really sort of strengthened our partnership together. Obviously, as Hannah said, they were kind of an in-person workshop with Graham and myself where we got to meet all the key users of the Fred Hopper tool and was really, really beneficial. And I guess, yeah, kind of reflecting on the beginning, how did your team ensure kind of alignment when you first started using the tool? Yeah, so when we first started using it, our team made alignment a priority through open discussion and shared problem solving. We regularly compared our approaches and results, which helped us benchmark against each other and learn faster. I think what I always find really interesting is, I mean, often you guys ask us like, out of 10, how do you rate how confident you are in the tool and things like that? And if you asked me internally at the white company, I'd be like, yeah, I'm around an eight or a nine, I'm pretty confident with what I'm doing. But then when we go to Merch Labs and there's other Fred Hopper customers there, I kind of always downplay my number a little bit. And I'll be a bit more like, oh, six or seven. Because I just think when you speak to other Fred Hopper users, we're all using the tool in different ways. And we can all learn things from each other, which I think is really exciting. I think there was a customer success day where I think it was Doc Martin were talking about how they had an out of 10, that's a lot of different ways. And I think it was really exciting. And I think it was really exciting. And I think it's encouraging that proactive thinking rather than waiting for leadership or technical teams to drive changes. We've built a culture of shared ownership and advocacy around how we use Fred Hopper. So we're also reviewing our rankings, what's working. We have different ranking strategies in different departments. So we just need to make sure we're happy with what's being pulled through to the live site. Whilst also, you know, as we come into peak, we're having to be reactive to trade, what's on the trading calendar, what activities have we got coming up where we need to maybe make those minor amendments. As I mentioned, we're also now reviewing insights, any live tests we've got going, often reviewing product attributes that feed into our facets. So whether we're reviewing or reordering our facets, live where, or we're actively manipulating internal attributes to improve our customer facing presentation. And we're also regularly checking the back end to ensure configurations reflect our merchandising goals. I think it's really interesting, you know, you're talking before about learning from other customers. We also learn from our customers. And, you know, our idea of what Fred Hopper best practice implementations are will evolve over time. And I think this is one of the really key things about this relationship that we have with the white company and we have with our other customers is that, you know, our learnings can influence our best practice and then we can advise you what best practice is. So, you know, a customer who perhaps hasn't been as engaged with us may have kind of missed they may be on an older version of best practice if you like. And it's really good that we have these regular conversations, you know, in the form of quarterly reviews or like the training session we had face-to-face in London. You know, that just really helps you to keep up to date with what's like, you know, considered latest best practice for Fred Hopper and for Fred Hopper users. Just going back before to what you were talking about, you know, your product attributes and rankings. What influences your decisions when you're looking at what you're going to adjust? Yeah, so whether it be product attributes or rankings, we are now taking very much a data-driven approach. We use the Fred Hopper insights tool. We've got A-B testing. So we're looking at the performance data from previous campaigns to understand what has worked where and where there's room for improvement. I think what's really important within the white company is we do have a diversity of products. So what works in one department isn't the same across everywhere else. We have things from cashmere jumpers to a dinner plate to a baby sleeping bag. So how we strategize across those departments is very different depending on our business needs and our goals. A lot of it comes down to optimizing the customer journey. What products do we need to appear for? First, how are the facets organized within that department? How are the facets organized within that department? And what elements do we want to highlight to make that experience for the customer as smooth and as engaging as possible? We collaborate really closely with Ella and Graham and Fred Hopper team. And those kind of conversations often spark new ideas for optimizations. They help us follow best practice and help us reduce the risk of errors. I mean, at a recent merch lab, Graham gave a great, great framework around dynamic pinning. So it's how we can kind of use those and learn from the guys at Fred Hopper. I think ultimately it's not, it's about not just letting the tool run in the background. It's really about understanding which changes will have the biggest impact on the customer experience and overall performance. For sure. And, you know, in that session you just mentioned there, we talked, I remember about sometimes a really small, simple, change can actually have a really big impact. You know, you don't need to throw everything out and build something new from scratch. In fact, we wouldn't generally advise that, you know, just a simple change to a ranking cocktail or something like that can have a really big impact. And, you know, yeah, the example you gave of the, you know, the diversity of the products you sell, that's really crucial to us when we're working with you on developing new ideas, because obviously we know Fred Hopper really well and we've got a lot of work. We've got access to all the documents and we've got access to the developers who built the thing, you know, but what we don't have is your understanding for your own products. And that's why the collaboration really works because you know what you're selling, you know what your customers like, you know how your customers behave. And that all feeds in together when we're working on new ideas. And I think, you know, it's just a really good collaborative relationship we have. I completely agree with kind of everything you both said. Obviously, as you both touched on, obviously leveraging the data that's in the insights tool to obviously make informed decisions is really, really key. And we've obviously seen that since working together. I guess kind of building on that, Hannah, when did Amy testing become part of the white company's strategy? And why was it so important to you and the team? Yeah, I think we always knew like we were doing great things in the tool, but we almost had like no way to prove it. So we knew this would give us the ability to make data led decisions for site merchandising and compare and campaigns. And really just prove the value of the tool. Like, I'm a big supporter of Fred Hopper, but to be able to kind of give, give the business like the data behind what we're doing, just proves its value. And it's allowed the team to be able to do. And it's allowed the team to validate some of our assumptions rather than relying solely on our intuition. It's critical for understanding which optimizations actually improve the customer experience. And it supports trading decisions during key periods like peak and our seasonal campaigns. Definitely. Really fantastic. So, yeah, I think that's a big part of the AB testing has transformed your approach and can obviously provide concrete evidence into kind of the white company's team efforts. How did you first set up your first tests and what sort of resources were necessary? Yeah, so again, we still had training hours within our contract. So we knew we definitely wanted to set some aside as part of the AB testing, but we kind of this time kept it to a little bit of a smaller team to almost become like, subject matter experts in a smaller team. So we were able to be subject matter experts in AB testing and be that kind of voice of Fred Hopper AB testing. So the consultancy days provided by the Fred Hopper team included planning and set up guidance. Sorry, we, we knew the business really wanted to focus on rankings and wanted results on what rankings work where. Because as I mentioned, because as I mentioned, because as I mentioned, the different departments have different priorities. So we set up our hypothesis and got tests up and running. Whilst we still consulted with Graham, there's also Andreas within Fred Hopper, who really helped us. We defined goals for each test, selected product variants and identified our success metrics. I think that time investment upfront was balanced by the long term insights that helped us. Yeah, for sure. I mean, you know, a lot of a lot of a lot of the AB testing sort of training that we do with customers isn't so much about how to physically configure a test, right? Because they're actually quite straightforward to configure, I'm sure you'd agree. They, you know, you build them much like any other rule in Fred Hopper, just with a couple of extra parameters. The really sort of interesting stuff with with AB tests is focusing on concepts like, you know, concepts like, you know, planning the test, developing a hypothesis, understanding what it is you're trying to actually achieve, what you're trying to prove, and then learning how to read the results as well and make sure that, you know, the results you're seeing aren't a false positive or a false negative. So a lot of the training focuses on these kind of things just to help merchandisers to fully get the most out of not just creating a tool, but creating a test, but creating a really good test and applying it in the right way. Right place at the right time, and really learning from from the results you get from the test. Thanks, Graham. And then I like, obviously, there's going to be a lot of people on this webinar today that are kind of either in the process of implementing insights and AB testing, or it's on their roadmap. So I think it would be really helpful, Hannah, if you could supply any sort of surprises or key learnings from the initial setup phase. Yeah, yeah, yeah. Firstly, just want to echo what Graham said, the actual setting up of tests is so easy. Like it's, it's the interface that you're used to in Fred Hopper. It's so easy to do. Like, yeah, can't fault that. We definitely learned that every company's data structure and processes are unique. So that one, one approach, one size doesn't fit all. So there was definitely a collaboration piece across like the merchandising and technical teams to make sure that set up. And so that's the technical teams to make sure that set up is smooth. And Ellen Graham were always kind of there to jump on calls with our dev team when it came to the AB testing and insights set up process, which was so helpful, and just made it easier, fast setup. For us as merchandisers, I would emphasize that the initial testing is more about understanding the tool and the data. So that's the initial testing is more about understanding the data rather than like the immediate results. Just because a test has reached that stat sig doesn't mean we turn it off. We might amend it. We might amend the traffic split so that we can keep gathering data. Because again, as, as Graham kind of mentioned, like if you've got an email going out, you're, you're driving high volumes of traffic, which can influence the test. It's all about how to iterate. on the test and kind of learning that as we go. Yeah, absolutely. You know, you mentioned about the implementation of AB testing. So from a merchandiser's perspective, it's relatively simple. You get some new stuff to play within the UI. There is some technical work that we need our customers to get involved with. But we fully support that as well. You know, we're not just here to support the merchandisers. We're here to support the implementation teams, make sure that everything's working behind the scenes to sort of facilitate the tests that the merchandisers want to do. And, you know, we'll help with, with every aspect of that, you know, testing, making sure that everything's flowing through correctly, making sure the, the data we're capturing through the insights platform is, is being registered correctly. So the AB test data is all being gathered correctly as well. So, you know, we're here for the whole process, not just to help the merchandisers do their bit, but we're also here to help the tech teams with the implementation on that side of things as well. Thanks, Graham. And I guess, obviously, Graham kind of touched on the collaborative efforts between, obviously, us and the technical team, such as the merchandising teams. And it's been really great to kind of see those efforts between both the Fred Hopper team and the white company. One for you, Hannah, I guess now that the white company have implemented these strategies and these AB tests, have you seen any early results from your testing and changes? Yeah, absolutely. We've seen improved click through and engagement rate on our optimized categories. I think a focus recently for us has been about rankings versus a manual pinning or dynamic pinning. Again, it's different across different departments. Within clothing, we've, we've kind of focused on manual pinning versus our ranking where you can really VM a PLP and, and make it look really, enticing for the way that's going to be a bit more enticing for the customer to click through versus maybe something like a home accessories PLP where we would usually rank by units sold. It sometimes surfaces things like candle lids and plates, etc. Whereas our focus is around driving AOV. We've used a dynamic pinning to surface those different product types, which has been really effective. And that has then been rolled out into a permanent change, a permanent dynamic pinning on those PLPs. I think it's just that those small back end adjustments have translated to visible front end improvements and demonstrating the value of Fred Hopper in measurable terms for the leadership team at the white company. That's great to hear. I guess, obviously, it's great to kind of hear that you guys are seeing an improvement in kind of metrics. That's obviously what we love to hear. But more for the users, the day to day users, of Fred Hopper, how are the team kind of feeling about using the tool day to day? It's actually been so nice to see them become like more knowledgeable and empowered to drive changes themselves. I think before, we just they just kind of knew what we knew in Fred Hopper and just kind of carried on doing what we're doing. Whereas now, they're kind of asking questions, can we do this? Is this possible? And most of the time it is like, that's what I love about Fred Hopper. Fred Hopper, you can, you know, you can edit things, you can manipulate things. So actually, it's all about us just getting involved in the tool and trying. And if we do ever reach any blockers, that's where we kind of either raise a ticket, ask Graham, those kind of things. I think our internal advocacy is strengthened, as the team can show tangible improvements and ongoing support from the Fred Hopper team reinforces their confidence in decision making. It's so easy to hear, obviously, about the team's confidence growing within the tool. It's what we love to hear at Fred Hopper. And I guess, obviously, the things that we've touched on at the start of this webinar about kind of working together and the resources available, obviously, we want to enable our customers in the tool and make sure they're getting the most out of it. And I mean, have there been any improvements that have surprised you in how they've impacted performance or team confidence? I think it is just that those minor tweaks can have a significant impact on the UX and campaign visibility. And those early wins helped secure buy in from leadership and improved the internal morale around using the tool. So it just reinforces the investing in small smart changes is really worthwhile. Definitely. And I know when we first started working on this together, a major focus for both of our teams, obviously, was getting everything implemented and ready ahead of peak trading. How are you and the team using insights from A-B testing to plan for peak trading periods? Yeah, so we're prioritizing optimizations for our high traffic categories and key campaigns, where we've had early promotional activity on the site. And we've had early promotional activity on the site last week. That's helping us inform our decisions ahead of Black Friday. I think we don't want to risk testing around Black Friday. So to be able to have some of those decisions made earlier, it just really gives us confidence in our merchandising plan. We've also been doing a big piece of work around search and insights is now great. It gives us some really valuable search data, whether it's your worst performance, following searches, or we're now going to be able to work around the market, or we're now seeing our average number of results versus the maximum. So we can really critique that on a weekly basis going forward. We're applying Fred Hopper rules dynamically to respond to promotions and marketing activity, ensuring our resources focus on the most impactful changes. That sounds like a really great strategy in the way of dealing with things. I think being proactive, obviously, within the Fred Hopper tool will give you an advantage. during peak trading. Are there any specific strategies or optimizations you're particularly excited to implement during peak? I think it's, yeah, applying those lessons from previous tests to make faster, smarter decisions in real time. And just knowing that we're set up for success. We'll be highlighting the most relevant products to maximize engagement and conversion. And this dynamic approach will allow us to respond quickly to trading trends. And yeah, as I mentioned, Graham gave a really great framework for that dynamic pinning that we're rolling out in peak. Yeah, I'm really, as your customer success manager, I'm really looking forward to seeing how these strategies play out, obviously, in our kind of post-peak debrief. What do you hope to see in terms of trading performance this season? I'd probably say like better engagement and conversion across those optimized categories. Essentially, we're all working towards a successful peak and driving demand as many retailers are. But just being confident that those previous tests and small improvements will translate into meaningful results. And expecting our peak trading performance to benefit directly from insights and A-B testing. And as you say, being able to like wrap it up and share that with the business is great. Definitely. And I know that obviously within kind of technology roadmaps, there's always different priorities and it can sometimes delay implementation. So I guess a question for you, Hannah, is how did you secure senior leadership buy-in on your work with us and the Fred Hopper team? Yeah, I think it's definitely around educating leadership about the daily value and impact of the tool. And highlighting those changes that sometimes maybe go under the radar or go unnoticed. I'm a big advocate of Fred Hopper and I want to share what value it brings to the business. So it's definitely about shouting about your successes. I still know that I need to get better about that doing that and kind of PR-ing Fred Hopper a little bit within the business. But I think giving examples of optimization wins is really key and demonstrating the progress and the progress and the result builds credibility and trust internally. Definitely. And that's a really great strategy. I think keeping leadership informed is really essential for ongoing support. And obviously with the data-backed results that you're getting now from insights and A-B testing, it only makes that easier. What advice would you give to someone in a similar role trying to champion a new tool internally? I would definitely say, like, back yourself and trust your expertise. I think when you're confident in the tool that you're talking about, you talk about it with passion and that really shows. I would also say, like, stay curious, like, keep learning, keep asking questions. It's only going to help you with the tool more. Leverage that partner support for validation and guidance. Like, I can't, sing Graham and Ellis praises enough. Like, it's been amazing working with these guys. And yeah, just sharing those small wins to build momentum and advocacy. But also you can share the tests that don't work. Like, you don't need to shy away from that. You still learn from them. So just kind of using those clear examples shows the tangible benefits of your approach. Yeah, they're both excellent points. I think it's so important to celebrate the wins, no matter how big or small. I also think a big part of using any form of tech is obviously having confidence within it. So how do you make sure the team at the White Company stays self-assured and empowered using the tool? Yeah, I think it's around, like, continuous training and open collaboration with the team. I think it was really important for me being the lead on Fred Hopper at the White Company that I wasn't a single point of failure. So I love helping and teaching the team how to use the tool. So more than happy for them to come to me with questions. But it's about showing them how they can take action, set up tests. And that's just really nice to see. And it just builds their confidence. And I think it celebrates their successes as well. So just encouraging that experimentation with guidance from the Fred Hopper team to reduce risk. Definitely. It's all about kind of fostering a culture of learning and growth. And I think even in the past couple of months, we've seen kind of a lot more engagement from kind of your team, Hannah, which is great to see a lot more kind of curiosity about the tool and what it can do. Is there anything you've learned about this journey that others might not expect? It is all around, like, the small thoughtful changes have the biggest impact. I think that strong partner support combined with internal advocacy is essential. The continuous learning, benchmarking, peer discussion helps maintain high standards. And your confidence comes from understanding the tool deeply, but knowing that there's expert support available. Amazing. Thank you so much, Hannah. That's come to the end of our kind of Q&As of this webinar. So yeah, thank you for sharing kind of your insights and experience with us today. It's obviously amazing to see how the White Company has kind of embraced Fred Hopper and the positive impact that Fred Hopper has had on your team and overall performance. I guess handing over to Maddie now, do we have any questions? We do. We do. We've had a few questions come in. So do keep them coming. The first question I hopefully will be able to help with quickly. Someone has asked, when is the next Merch Lab in London? Our next Merch Lab in London will be in April. And that's because we have our annual customer connect date at the end of January. So we are just in the final processes of finalizing those details and your customer success manager will have more information. on that in the coming weeks. So let's have a look at the questions. Okay. So the first one, were there any unexpected insights that came out of your early tests? Yeah, definitely. So I would say this is very much based around the hierarchy that we have set up at the White Company. But I did an early test on our baby and children department, which was just a ranking test. I think it was just a ranking test. I think it was newness versus bestsellers, for example. But because I had set it at quite a high department level, it essentially reached significance in like a day. So obviously, Fred Hopper, Graham has taught me not to probably trust that in first instance. So it was then like, how do you iterate on that test? Do we look at baby clothing, baby sleep, children's pajamas? And how do we kind of set up tests within those areas rather than those areas rather than being such a high level? How do you kind of focus the test in? Perfect. And next one, why do you think it's important for merchandising teams to have hands on control, rather than relying solely on automation? Yeah, I would say definitely like merchandisers know the detail. We know what the business priorities are. We know when stock is about to land. We, I think you're in the background. We, I think you still need that element of influence and product detail rather than just allowing it to run in the background. I think it's really key. The amount of work that goes into whether it be with the marketing teams, with buyers and merchandisers, that we still have that kind of touch on how we want to influence some of these things sometimes. And again, that might be that you set up a test and it, and it actually doesn't work. That's fine because then at least you've got results. But I think it's important that we as merchandisers still can influence in what we think we should be doing. Brilliant. And I'm just kind of combining a couple of questions we've had into one here. But has the focus on A-B testing influenced your broader digital strategy? I think it's more about bringing the trade team, the merchandisers aligned with the wider business. Like we have an optimisation team that is running tests across the site on various different elements, whether it be in basket, on PDP. So now that we can run A-B tests on PLPs or search result pages, it kind of brings us as part of the conversation. It kind of brings us as part of the conversation and yeah, how we can influence what we're doing for a campaign and what we're testing. Brilliant. Well, that is it for now for questions. If anyone does have any more questions, please reach out to your customer success manager and we can happily help with those. But in the meantime, thank you so much to Hannah, Ella and Graham for joining us today. It's been a fantastic session and we truly appreciate your time, especially considering it's very close to peak. So it's truly appreciated and thank you to everyone that has joined.