In many ways, mobile can be evaluated similarly to any other marketing initiative. Generally, every marketing campaign is evaluated in terms of efficiency (getting value for invested money) and effectiveness (achieving the objective set for the campaign). In this sense, mobile is no different. We must determine how well the program did when factoring in cost and measuring against objectives – the two evaluated inclusively and exclusively.
However, the evaluation of a mobile campaign is no easy task. It must involve a variety of quantitative and qualitative measurements that assess the program’s efficiency and effectiveness. Mobile is moving away from the early adopter stage to the innovation stage and this shift brings added pressure to demonstrate tangible ROI. These measurements include, but are not limited to:
Quantitative Analysis
• Cost per – Any of the below measures, but forth most, cost per a unique user, per an interaction or in terms of harder ROI cost per acquisition.
• Delivery Measures – Number of messages sent, number delivered, number failed delivery, attrition rates.
• User Messages – Number of unique users, number of interactions, number of repeat users, frequency of interaction, time of users’ interaction, from where users interact, number of replies, number of stop messages, number of inquiries.
• Click Through Rates – When URL links are included.
• Purchase Tracking – Number of people to redeem a coupon or make a purchase.
The metrics used should be determined by the campaign objective and its primary mechanics. For example, if it were a text to win campaign, the number of unique users and total entrants would be the primary value. For a program where it asks users to answer a series of questions through SMS, the same metrics would apply, but equally important would be how much interaction each user had with the brand – i.e. the average user answered three questions.
One of the key benefits of mobile marketing is the ability to track the campaign results immediately and across a wide range of available data. These numbers are a science. They are exact and can be easily reported.
Qualitative
Conversely, a qualitative review is not a science. It is based on an evaluation of the following:
• User Interaction – The amount of time the user interacts.
• User Engagement – The depth of user interaction – how engaged is the user?
• Brand Connection – Link between the brand, campaign execution, and end goal.
• Marketing Mix Integration – How effectively is mobile integrated into the mix?
• Surveys and Awareness tracking – Test and control groups are used to test brand saliency as a result of incorporating mobile.
Factors Influencing Success
After the above analysis, it’s important to review the below factors. These factors will greatly influence the result of the program, not necessarily for better or worse, but expectations must be set based on them. For example, in terms of value proposition, you’re more likely to get more entrants if there’s a car prize versus a t-shirt prize, or if the brand is Bud Light Lime or Steelback.
• Value proposition – win a car or a t-shirt, receive great alerts, entertainment value, etc.
• The brand – premium brands vs. value brands.
• What is required of the user – text a keyword versus answering multiple questions.
• Where is the media with the call-to-action – in an area of downtime or in an active, hustling environment.
• Quality of the execution – street teams, messaging, creative, etc.
• Emphasis of the Mobile call-to-action.
Deciphering the Results
Despite having access to a vast array of accurate and immediate metrics, the application of those metrics are inexact and it’s difficult to benchmark against other campaigns. In traditional media we can easily track results – GRPs, impressions, recall, brand awareness, etc. - however, mobile marketing is not at this point, nor will it be in the future, as the salient value is user interaction and what it can accomplish, something that can not be easily tracked. If anything, mobile marketing is a tool to track the effectiveness of traditional marketing techniques as mobile marketing often complements a campaign, not is the campaign.
Here’s what I mean: Take for example a billboard in downtown Toronto. I am a marketer for a car company and I want to advertise on this billboard for two weeks. My objective is to sell more cars. The billboard goes up, I’m told the weight is 90 GRPs with 15k of daily traffic. Without complex and costly statistical analysis, the billboard’s impact is unquantifiable, and even then nearly impossible to extrapolate.
Take the same situation, but add in a mobile component. As a marketer, we know to sell cars we must push consumers to a dealership, and ultimately try the car. Therefore, messaging will push users to text testdrive to 12345 to schedule a test drive. On average, 10 people text in a day, two of which schedule a test drive. Therefore, we know the conversion from billboard to dealership (not including any residual benefit of long-term marketing initiatives on branding). At first glance, by just comparing the numbers, this would seem like a poor performance, however in reality this is quite successful; however, this can only be determined by evaluating qualitatively as well. Beyond metrics, some of these qualitative are:
• Immediate interaction - Consumers do not need to call a number to engage the brand, they can do it right there on their phone.
• Qualified users – Only those consumers that are interested in the brand will text in. It’s like a two-stage program: Nearly all passer-bys will see the billboard, but to take it a step further, those interested that actually respond to the billboard can engage the brand.
• Not a benefit, but it is very important to keep in mind this is a high value, expensive brand, advertised in a location with no downtime, to an untargeted audience.
• Opportunity for continued dialogue – garnering an opt-in from a consumer in any form is a big win, doing so via mobile is a HUGE win. Assuming your brand has a communication plan in place you know have a direct feed with your consumers anytime and anywhere. Start thinking about EXCLUSIVITY, RELEVANCE and TIMELINESS to your messaging strategy.
The general idea is that in mobile marketing, the qualitative analysis must be evaluated in tandem with the quantitative results, while considering the factors; only together can they provide a complete picture of the program’s success.
