I would like to speak about a mistake I actually see client after consumer making. (I work at a tech consultancy. All of us have a lot of clients. Not all of them get this mistake!… Although many do. ) That mistake is to over analyze analytics data, without the strategy; to assume that all that has to be done is to gather all the data as possible, and then this data will magically become knowledge, and knowledge will mystically become wisdom.
I understand the temptation. Suppose if you’re building a new website, or a new software. Certainly you want to know as much as you can, in as much detail as possible, about how precisely users make use of it. Of course you want activity trackers, heat maps, route analysis; of course you want every twitch and false start from every user to be logged for any eternity. Data is the new oil, the new gold. Of course you need to collect as much of it as you possibly can, to be parsed and enhanced and mined later on. Right?
… Well, yes. Yet. I place it to you that data due to its own sake is incomprehensible; that you should really know what questions you want might of it, what standards you want to assess, what targets you want to aim for, before you start collecting it. There is an opportunity cost to analytics data: time placed in defining and collecting its about time not put into focusing and refining your product. Of course, if you determine what questions you want to ask of your data first, rather than declaring “collect it all and let Future Me type it out” — My spouse and i think you’ll generally realize that this will advise your product design in an extremely helpful way.
There is a fantasy, of course, a fantasy that grows with nigh every case study at every MBA school, that somewhere within the stats data your site or iphone app or service records, in some obscure line or column, you will find the secret to your ultimate success. The famous Facebook “aha second. ” Or the famous… well, actually, that Facebook . com example is repeated advertisement nauseum because it’s generally the merely one people can think of. But, more generally, the parable that your analytics data will make you understand how to hockey-stick your users.
99% of the time that is not how functions. 99% of the time you get screwed by selection bias. You get no data at all from the users who never come to you, because that they can’t be bothered, because they not necessarily interested enough, because they never heard of you. You get almost no data from the users who immediately bounce. The data you do get, the so-called “rich” data, are from your involved, interested users — but making marginal improvements on their behalf won’t help you. You would like to increase the experience for you for which you have no or little data. Explain to me again how your analytics can help you there?
I’m not expressing data is valueless. I am just not saying analytics are completely unimportant. But My spouse and i is saying that before you obsess about them — and believe me personally, with quite a few of the clients I’ve had, “obsess” is the right phrase — ask what questions you will ask of your analytics data, and what value you expect to receive. Don’t envision its value is automated, and just needs to be mined, when all too often it is fool’s gold at best. Don’t acquire data for its own sake, accumulate it to resolve specific questions — and really know what those questions are very well before you launch.