SMS Review Solicitation

Increased kiosk review volume on Google by 120% by making it easier for customers to leave reviews.

Kiosk Responsive web app
KeyMe kiosk showing a screen asking the customer to leave a review and mobile device asking the customer to tell KeyMe about their experience. The kiosk's screen is the screen shown after an order session ends. The mobile device displays a webpage with a button labeled 'Get help now' as well as five emojis labeled 'Poor', 'Bad', 'Okay', 'Good', and 'Great'.
After the kiosk finishes dispensing the customer's key, the customer receives a receipt of their order via SMS. The SMS includes a link to a page where they can leave a review or get help with their order.
New experience map for receiving a receipt and review solicitation. Step 1: Order a key dispensed from the kiosk. Step 2: Enter mobile number into kiosk for receipt. Step 3: Receive SMS receipt and feedback link. Step 4: Open feedback link in web browser. Step 5a: Get help with any issues via contact form. Step 5b: Leave review on Google about order.
In the old experience, customers received receipts via email, and the review solicitation was a separate email from the receipt. Sending the receipt and review solicitation in the same message via SMS increased the visibility of the review solicitation and made it easier to get help with orders if needed.
Four different versions of the feedback link landing page that we tested. Each has a review link for the kiosk and a link to the contact form for help with orders. The links take the form of emoji buttons with text labels or rectangular buttons with text labels.
We tested four different designs concurrently for the feedback link landing page. Each design was linked to a different subset of kiosks.

Overview

Note: Some content cannot be displayed due to the NDA.

One of the main motivators for copying a key at a KeyMe kiosk is social proof - aka ratings and reviews. The more ratings a given kiosk has, the more credible it seems. However, while every kiosk has customers, not all of them leave reviews on their experiences. Therefore, kiosks with fewer reviews seem less trustworthy, and new customers are reluctant to try them out. We needed to encourage more reviews to draw more customers.

At the time, we solicited reviews via email. After a customer's order was fulfilled, the customer received an email asking about their experience. They were then directed to the Google Business Profile (GBP) page for the kiosk and asked to leave a review. This method was ineffective because:

  • Not every customer reads every email.
  • The customer's email client may have thought the review solicitation was a marketing email and moved it to spam.
  • The experience might not have been fresh in the customer's mind by the time they saw the solicitation.

Our solution was a new feature - SMS receipts. Instead of receiving the traditional email receipt and review solicitation email, customers receive an SMS (text message) containing both a receipt and review solicitation. The SMS asks the customer to let us know about their experience and includes a link to a "landing page" on our website. That page encourages the customer to leave a review and/or contact our CX team to resolve any issues they may have had.

We hypothesized that switching to SMS receipts and review solicitations would be more successful because:

  • Customers are more likely to see the SMS than an email.
  • The SMS encouraged a review while the experience was still fresh in the customer's memory.
  • The solicitation was in the same view as the receipt details, so it was obvious which order the solicitation applied to.
  • The SMS made it easier to get help (if needed) and then leave a review since both actions were on the same page.

Due to the complexity of order status notifications for mailed keys (vs. keys dispensed right from the kiosk), we limited our SMS receipts feature to dispense-only orders - which also happened to be our most common type of order. Customers that placed orders including at least one mailed key still received the traditional email receipt and review solicitation. In a future iteration, the feature could be expanded to cover mailed orders too.

Don't have time to read the process? Jump to Results for more final designs!

Process

Objectives

User journeys before and after the transition to SMS receipts. Prior to SMS receipts, customers received email receipts and review solicitations for all orders, so they were less likely to leave a review. SMS receipts made it easier to leave reviews and get help with orders.
  1. Increase review volume for our kiosk listings on Google - especially those with few ratings - by making it easier for a customer to leave a review. We ideally want positive ratings (3 to 5 stars).

Research: Need to easily get help with different types of orders and leave reviews

User Research

We couldn't interview actual customers at the time due to security concerns and financial constraints. Instead, we asked others from the organization to provide feedback on current and proposed designs from a customer's perspective. We also chatted with the CX and analytics teams for more insights.

Because KeyMe kiosks offered both dispensed keys (dispensed immediately from the kiosk) and mailed keys (sent from our fulfillment center after a few days), we also had to consider multiple user journeys and task flows.

A profile of a typical kiosk customer named Silvia. She is a first-time homeowner that wants to copy her key so she can lease rooms out. She struggles to find options that are both affordable and convenient. Silvia also doesn't like phone calls - she prefers asynchronous methods like email or contact forms.
The average kiosk customer is female, in their mid-20s to mid-40s, and going through a stressful time in their lives (like buying a house). Customers' most common reasons for contacting our CX team are to request refunds on orders and get order status updates, so it's important that we make both easy to do.
The old user journey for dispensed-only orders. Silvia orders keys dispensed at the kiosk. When she tries them out at home, one of them doesn't work. She struggles to contact customer support because she has to dig through her emails, but she manages to contact them and get a refund. She also doesn't notice the review solicitation email. The old user journey for mailed orders. Silvia orders a mix of dispensed and mailed keys. She tries the mailed keys when they arrive, and one of them doesn't work. She struggles to contact customer support because she has to dig through her emails, but she manages to contact them and get a refund. She doesn't notice the review solicitation email though.
The process of ordering a key is simple. But because all post-purchase communications are sent via email, the customer might not see them, especially for orders that don't require tracking (i.e. keys dispensed right away). Email addresses are also tedious to enter into the kiosk.
The old task flow for dispensed-only orders, part 1 of 2. The customer pays for their order, enters and confirms their email address for the receipt, waits for the keys to cut, and then their order session ends. The old task flow for dispensed-only orders, part 2 of 2. The customer receives the receipt via email, then receives a review solicitation email. The old task flow for mailed orders, part 1 of 2. The customer enters their shipping info (name, email, and address), verifies their address, pays for their order, waits for keys to cut (if any dispensed keys are part of the order), and then their order session ends. The old task flow for mailed orders, part 2 of 2. The customer receives the receipt via email. When their order has shipped, they receive an email notification. Lastly, after they have received their order, they get a review solicitation email too.
The flows for dispensed and mailed keys are similar. The main difference is when the customer enters their email. For dispensed-only orders, customers enter it after paying for their order. For orders with at least 1 mailed key, they enter it on the address entry page prior to checkout.
Design Research

Inspired by Jackson Hewitt's success story, our leadership wanted to try soliciting reviews via text. Therefore, we studied design precedents and heuristics for:

  • Large displays like desktops, kiosks, or posters that text to your phone or give you QR codes to scan
  • Receipts and order status notifications sent via SMS
  • Review solicitation methods
Precedents for cross-platform communication (between mobile devices and large displays like desktops, posters and kiosks). Examples include scanning QR codes, texting to a number to get more info, and entering a phone number for an app to send a text to. Precedents for transactional notifications. Examples include order status, refund status, order confirmation, and receipts. Precedents for feedback solicitation. Examples for review formats include freeform text, choosing emojis or star values, and numerical scales. Examples for solicitation formats include text messages, emails, and in-app pop-ups with or without external links.
Examples of cross-platform interactions (large displays to mobile), receipts and order status notifications sent via SMS, and review solicitation methods.
Best practices for transactional notifications and soliciting customer reviews or feedback - provided by Nielsen Norman Group and Podium.
Design heuristics from Nielsen Norman Group (NN/g) and Podium.

Ideation: Numerous issues with converting all emails to SMS

Our leadership wanted to replace email receipts and review solicitations entirely with SMS. But just from our task flow analyses, we realized that the scope would be big because:

  1. We would have to update multiple areas in the kiosk experience to accommodate mobile numbers.
  2. We have to replicate our emails' functionality in SMS.
    • In other words, customers would need the ability to get receipts, review solicitations, order status updates, and other communications via SMS (or links within the SMS). If we used SMS only, we had limited formatting options and length. If we used a combination of SMS and links, we have to design and develop the pages those links go to.

To convey this complexity, we wireframed flows for both dispensed-only and mailed orders, including additional pages we would need to develop. We also depicted alternative flows where users could input both email addresses and mobile numbers or choose between the two (but not be restricted to SMS only). Accepting both email and mobile would be more conventional and what users would expect, while allowing them to choose would support user agency for a better experience, though both options would also have a large scope.

Additionally, we noted that most e-commerce sites offer both email and mobile order status updates, so we at least needed email for our mailed orders. Receipts for mailed orders are typically sent to customers' email addresses instead of texted to them. We recommended to our leadership that we consider keeping emails in the mailed order flow at least.

Idea for new dispensed-only order task flow. If we want to replace the email receipt and solicitation with SMS, require the customer to enter both email and mobile, or choose between email and mobile, we need to update the email input page, email confirmation page, and session end page on the kiosk. We would also need to design an SMS receipt and a review solicitation webpage. Idea for new mailed order task flow. If we want to replace the email receipt and solicitation with SMS, require the customer to enter both email and mobile, or choose between email and mobile, we need to update the address entry page and session end page on the kiosk. We would also need to design an SMS receipt, SMS order shipped notification, SMS review solicitation, and a webpage for them to link to (for tracking order status, getting help, and leaving a review).
Proposed new flows and areas that would need to change if switching from email to SMS entirely, accepting both email and SMS, or allowing the customer to choose between email and SMS.
Ideas for a 'Get Your Receipt' page on the kiosk that accept email, mobile number, both email and mobile number, or allow you to choose between email or mobile number. Ideas for the kiosk page where the customer confirms the email, mobile number, or both email and mobile number they entered for their receipt. Ideas for an address entry page on the kiosk that accept email, mobile number, both email and mobile number, or allow you to choose between email or mobile number. Ideas for an 'End Session' page on the kiosk that display the email, mobile number, or both the email and mobile number entered for the receipt.
Kiosk pages in the flow to update to accept/display only a mobile number, both email and mobile number, or either email or mobile number.
Ideas for converting the email receipt for dispensed-only orders to SMS and including a review solicitation. The customer gets a text with a thank you message, link to leave feedback, and order details (amount paid, order ID, and kiosk number). The link leads to a page with links to the contact form and a page to leave a review on Google. Ideas for converting the email receipt for mailed orders to SMS. The customer gets a text with a thank you message, link to track their order and get help, and order details (amount paid, order ID, and kiosk number). The link leads to a page with the order details, a link to the detailed order tracking page, and a link to the contact form. Ideas for converting the 'order shipped' email to SMS. The customer gets a text with a link to track their order and get help. The link leads to a page with the order details, a link to the detailed order tracking page, and a link to the contact form. Ideas for a review solicitation text sent after the customer receives their mailed order. The customer gets a text with a thank you message, link to leave feedback, and order details (amount paid, order ID, and kiosk number). The link leads to a page with links to the contact form and a page to leave a review on Google.
New SMS flows that could replace each existing post-purchase email.

Finally, we designed a few concepts for the review solicitation methods and layouts to test with our team.

Prototyping: Compromising and limiting the test to dispensed-only orders

We presented our ideation and recommendations to our leadership. After they understood the expansive scope, they decided to limit the SMS review solicitation test to dispensed-only orders. Customers ordering mailed items would still receive receipt and review solicitation emails.

Our leadership also preferred that the solicitation be part of the SMS receipt instead of a message on the kiosk's End Session page (which appears after an order session has finished). According to historical data, we've had low interaction with QR codes elsewhere in the kiosk, and we would also have to generate a unique QR code for each kiosk. Data also suggests that customers don't pay attention to the End Session page, but our leadership thought it couldn't hurt to include a solicitation message there in addition to the one in the receipt.

For the solicitation page design itself, they preferred designs based on the ones we were currently using in our review solicitation email tests.

The flow we decided to move forward with. We decided to only test SMS receipts on dispensed-only orders. In the flow, the customer enters their mobile number for the receipt, the kiosk cuts the keys, and the order session ends. They then get an SMS receipt and review solicitation, which takes them to a page where they can get help or leave a review. The four review solicitation page designs we decided to test. They test different designs for the links to get help and links to leave a review.
Flow for dispensed-only orders with SMS receipt and review solicitation plus review solicitation page designs to test. The latter is made of components (emojis, "Review us on Google" button, etc.) that we had been testing in email campaigns.

With that, we created a prototype for the dispense-only flow with SMS receipts and solicitation, then tested them with our team as if they were customers. They said the flow was straightforward and made sense.

Our CX team felt that the review solicitation designs with emojis were more inviting and enticing. They helped us reduce and refine copy to be more neutral, which can help with angry customers.

Development: No major changes or blockers

Development was straightforward since - after reducing the scope - the new flow was similar to the original flow. We didn't have to make any major changes along the way. After some testing and bug fixes, we deployed the feature.

Results: 120% increase in kiosk review volume

We first released the feature to a subset of kiosks. Within two weeks, review volume increased by 200% to 300%. We later released to all kiosks, and after another two weeks, review volume increased by 121%. However, the average rating also dropped 28%, possibly because we made it easier to leave any review - including bad ones.

Although a drop in rating wasn't ideal, we considered this feature successful because it increased review volume - our main goal and the most important SEO metric. Then, we started testing different landing page designs to see which one leads to the best balance between higher volume and higher ratings. We found that using emojis encourages reviews, and making it easier to get help with issues encourages more positive reviews.

The new user journey for dispensed-only orders. Silvia orders keys dispensed at the kiosk. When she tries them out at home, one of them doesn't work. She is able to quickly get help because the link in the SMS receipt is easier to find. She gets a refund from customer support a day later, and she goes back to that link to leave a review.
With the new SMS receipts and review solicitations for dispensed-only orders, customers have an easier time leaving a review and getting help. They also receive the message when the experience is fresh (right after their keys dispense), so the message feels more relevant, and they are more likely to leave a review.
The new task flow for dispensed-only orders, part 1 of 2. After the customer places their order, the kiosk asks them to enter their mobile number for their receipt. The customer enters and confirms the number, then the kiosk cuts their key(s). Finally, the kiosk indicates that their order session has ended and asks for them to review their experience on Google.
The new task flow for dispensed-only orders, part 2 of 2. The customer receives a receipt via SMS. The SMS receipt contains the order details and encourages the customer to leave feedback via a link. The link goes to a landing page with buttons where they can leave a review on Google and/or get help with their order via a contact form.
Two different landing page designs tested. Design A only displays five emojis labeled 'Poor', 'Bad', 'Okay', 'Good', and 'Great'. Design B displays a green 'Review us on Google' button and a red 'Get help now' button. Another two different landing page designs tested. Design C displays a yellow 'Get help now' button at the top and the 5 emojis at the bottom. Design D displays the 5 emojis at the top and a red 'Get help now' button at the bottom.
We tested four designs for the landing page. Each differed from the others in copy, placement of buttons, and colors of buttons.

Reflection: Survey past customers about leaving reviews

Since this project was simple, not much went wrong. We could have surveyed past customers anonymously to understand why they did or didn't leave a review. Then, we could have used those insights to further encourage reviews in our designs.

Next Steps

  1. Explore ways for the user to choose between SMS or email (or choose both) for receipts and other communications.
  2. Explore ways to encourage more reviews via email.
  3. Conduct a survey to understand why users don't leave reviews.
  4. Explore additional landing page designs to test (might have to observe paid users for insights).

Links

Landing Page A - 5 emojis Landing Page B - 1 review button, then 1 "get help" button Landing Page C - 1 "get help" button, then 5 emojis Landing Page D - 5 emojis, then 1 "get help" button Return to Gallery