Yesterday we delivered part three of our four-part series of posts directed at the brand advertising community: Branded Software Experiences – A Cost-Effective and Critical Component of your Brand Advertising Mix. Today we’re delivering our fourth and final part.

The first step in understanding the benefits you’re receiving from your new branded software experience is to understand the actual cost of the effort. Your time sunk as well as the cost of the vendors you used to build the experience obviously need to be calculated as a one-time cost. And as we mentioned in the previous post, the costs of operating the experience on an ongoing basis need to be factored in. But there’s a final component you need to factor into your budgeting – distribution.

In the days of packaged software, distribution meant putting boxed software on retail shelves or sending catalogs through the mail. On the Internet, it means driving traffic to your site, and there’s a whole different methodology to measuring success and return on investment. Normally online services advertise once they’ve built an audience that is big enough to be attractive to advertisers. But once their site has built an audience, they maximize your revenue by offering scalable advertising inventory to multiple advertisers, which makes you just another company competing for eyeballs. The benefit of the branded software experience, versus a typical online service, is that in the best case, you are the only sponsor and thus you accrue all the value to your own brand. Of course, the cost of getting in on the ground floor is that you have to help drive traffic to the site. This is not a difficult challenge, but it should be part of your budgeting. And if the experience is done well, the money you set aside for distribution should shrink over time as the site becomes more popular and builds a self-sustaining audience. If the experience isn’t done well, people won’t spend time on the site, return to the site, or recommend it to their friends, which is why it’s important to take to heart the advice we’ve given you in the other posts.

If you’re a large brand, you likely already have a significant audience on your product’s website, at least a portion of which you could redirect to your branded experience, and that is a good place to be in. Above and beyond that, there are many traditional avenues for getting the word out and acquiring users for your new service (both online and offline advertising being the main methods). But as mentioned above, if the site resonates with users both from a utilitarian and emotional point of view, then it’s possible (and even likely) that your distribution budget is merely a catalyst to get the word out to early adopters and influentials. The rest will follow. And of course, building viral and social capabilities into the experience that you can’t find on your typical content site can increase the pace at which usage of the service spreads.

And in fact, that’s really the first measure of whether you’ve succeeded or not – at some point you should be able to drastically reduce or eliminate your budget for advertising the site itself. If traffic to the site is completely dependent on your continued advertising, then the site has failed to engage the audience. No number of viral features will help a site catch on if nobody wants to show it to their friends in the first place. And beyond just eliminating your advertising, if your site is significantly compelling, it should not only attract traffic, but grow its audience over time. Organic distribution is your ultimate goal. If your service is not (to a large extent) selling itself, then all you’ve made is a fancy billboard. (Yes, I’ll acknowledge that it is possible in this very noisy world that you’ll need to invest in driving distribution on an ongoing basis, but in that situation your ability to retain your users for multiple visits had better be very good. More on that later.)

Growing traffic on the Internet is like growing an audience for any product. News items and promotional pushes spike the audience for a time, but you only see the real impact after the first blush of excitement has faded and you find out how much of that audience you’ve retained. To the extent that you’re able to constantly grow the baseline (even if you fade a bit from the high of each spike in awareness) then you’re on the right track. Then the question becomes how comfortable are you with the pace of growth, and if you’re impatient, how much are you willing to spend to increase it. (And just one note on “spikes,” related to a previous post. If you didn’t think scalability from day one was important before, you will the day that a news item comes out about your site, causing people to flock to it, only to see that the site is down. These people will never ever return, no matter how good your site eventually becomes.)

There is of course another level of detail that’s critical in understanding the mechanics of what your users are doing once they arrive at the site (and thus, how effective your site is). Some key metrics:

• How many unique visitors are you getting per day?
• How many page views are you getting per day?
• How many of your visitors each day are return visitors vs. new visitors?
• How many visitors come once and never come again?
• How many people get to the site and then leave without doing anything there?
• What is the average length of time someone spends on the site? (What does this distribution curve look like? – are 50% leaving immediately, but the other 50% are spending 10 minutes on average at the site?)
• If there’s a user contribution component (posting stories or comments about your product, for example), what percentage of users are contributing?
• When users stay on the site, where do they spend their time?
• When users leave the site, where do they leave from?
• What is the usage pattern in terms of times of the day and days of the week?
• What is the geographical distribution of people using the experience?
• If you have viral features, how many of your users are inviting others to the experience? How many are they inviting? How many accept?

These are all relatively standard metrics for anyone operating an Internet site/experience. But you’d be surprised at how many people do not take these numbers as seriously as they should—and often even when advertising professionals know to review these numbers it’s not clear they have a deep sense of what the results mean. I have some numbers from existing branded software experiences that I could offer as a baseline, but honestly given the immaturity of this space I’m not sure the numbers would make for meaningful comparisons. Until we have audited and consistently measured results across a broad range of branded software experiences, I recommend these tactics for evaluating each number.

• If your goal is to engage your users on an ongoing basis the single most important statistic is how many of your visitors come back vs. how many show up once never to return. The data on how many people you get each day is simply a function of how effective your pr/advertising is. But knowing how many people see the site and make it a part of their life is showing you whether spending all that money getting traffic was worth it. What’s the point of driving people to a site they abandon after one visit (or even two for that matter)?
• Once you’ve established that you are getting people to the site and they are coming back more than once, it’s time to tune your experience to make their time with your software as engaging and positive as possible. The key metrics helping us answer this question are a) how much time your users spend on the site, what they do while they’re using it, and where (and why) they leave. You should have a sense of how much time a normal person might want to spend in your experience if you’ve built it properly. Then you need to gauge whether the average time spent is below or above your expectations.

The mechanisms for gathering these metrics should be built into the software from the start, so make sure that the people building the software have an instrumentation plan. Looking at these key metrics, updating your experience, watching the results, and then updating again based on your impact is the essence of how you turn your “great start” into a long-lasting, and incredibly cost-effective, brand messaging monster. Shipping and promoting the experience is not the end of the process, it is the beginning. You must plan for going through this iterative loop multiple times. And you must be disciplined as you do it. Small changes (moving a button a few pixels to the left, making a link bigger or smaller) can have incredible positive or negative effects on the numbers above. You have to make small changes, and watch their effects over time (days not hours) to get a real sense. You are running clinical trials on your branded software experience: Think like a clinician. Companies such as Amazon who ship sites highly tuned to users habits often run controlled, comparative experiments to see what people like the best. You can do things like offering two versions of your site at the same time and randomly directing users to one or the other so you can compare the results of your iterative changes to your baseline. (This is called “multiple flights.”) Patience, and willingness to think outside the box (some positive changes can be very counter-intuitive) are the essential ingredients to success here.

And finally, there are a host of traditional tools at your disposal that can be migrated from traditional brand advertising mediums to the software experience: e.g., sampling your audience to understand demographics, surveys measuring brand awareness and attribution, and customer satisfaction studies. These tools all translate into the world of software and interactivity quite well.

In terms of understanding how cost-effective your branded software experience is versus your other initiatives, it’s critical to define a consistent way to measure across all mediums. Software will likely be only one component of your strategy, as your audience hasn’t entirely (or even mostly) migrated to the online world. Calculating the reach and effectiveness you have per dollar spent in each medium, and then comparing the two will also start to give you an idea of whether you’re getting acceptable results. Our speculation is that the world of software can be so exponentially effective when done properly, that once you see the results of a branded software experience you’ll start counting the days until you can move the bulk of your efforts to the digital world.

Of course, we are in fairly unexplored territory. Our job now is not only to build great branded software experiences but to measure their effectiveness religiously and consistently so that we can together start to understand the dimensions of this new world. So please look at this guidance as a starting point. We may find out that some of these assumptions are incorrect or just off-base. That’s a fine result as long as we do our best to understand the reality, and adapt our techniques to take advantage of our learnings. Over the next weeks, months, and years, we will endeavor to do exactly that.