{"id":1457,"date":"2016-02-04T11:45:00","date_gmt":"2016-02-04T16:45:00","guid":{"rendered":"http:\/\/www.acaweb.ca\/en\/?p=1457"},"modified":"2017-11-03T13:38:18","modified_gmt":"2017-11-03T17:38:18","slug":"experimental-design-measuring-media-effectiveness","status":"publish","type":"post","link":"https:\/\/acaweb.ca\/en\/2016\/experimental-design-measuring-media-effectiveness\/","title":{"rendered":"Experimental Design: Measuring Media Effectiveness"},"content":{"rendered":"
\"Grant
Grant le Riche<\/figcaption><\/figure>\n

Advertisers are obsessed with measurement \u2013 and for good reason: you can\u2019t improve what you can\u2019t count.<\/p>\n

For a long time, media measurement was relatively simple: how many people did my ad reach, and how many ads did those people see? This became the basis of the gross rating point (GRP), which became the currency on which most TV ads were bought and sold. Newspapers had their circulation count and outdoor ads used traffic figures and population. Life was good.<\/p>\n

Then digital came along and upended everything. All of a sudden there were things like click rates, conversion rates, and their subsequent cost-per that put everything on a per-unit basis. More recently, metrics like viewability, audience on-target percentage and even upper-funnel metrics like awareness and favourability are quantifiable. We can measure both economic and campaign performance more accurately and more quickly today than at any point in history.<\/p>\n

But despite all the progress, we still have a tougher time than we should at answering the most important question for marketers: did my advertising drive sales? Perhaps the most common way advertisers try and answer this question is by calculating CPAs, or cost-per-acquisition. This term is commonplace in direct-response marketing and is the common success metric used for campaigns designed to get consumers to take action (like clicking \u201cbuy now\u201d).<\/p>\n

But even in the digital age, CPA-based attribution models don\u2019t take all the necessary variables into account in order to accurately understand how marketing spend impacts the bottom line. There are three common mistakes \u2013 or biases \u2013 that occur naturally within CPA-based models.<\/p>\n

    \n
  1. In-market bias.<\/strong> Did your ad drive a sale? Or did you measure someone who was already going to buy? How do you know for sure it was your ad that incited action? Correlation does not equal causation.<\/a> CPA-based models don\u2019t have a way to tell if someone already intended to purchase. This is an important distinction, because certain audiences like brand loyalists or someone who recently visited a webpage will naturally cost less to reach.\n

    In-market bias is compounded by the fact that people who are already going to buy a certain product act in ways that are easier to measure \u2013 visiting a webpage, signing up for emails or searching for a specific product. This is why search does so well from a cost perspective.<\/li>\n

  2. Low-rate bias.<\/strong> Most marketers have probably heard of \u201ccookie bombing.\u201d This tactic refers to blasting out as many digital ads to as many sites and as many people as possible at the lowest possible cost. The more ads served, the more people are cookied \u2013 and when you measure success using a formula where the total cost is divided by the total number of people (cookies), the larger your denominator, the lower your final number.\n

    Look at the below scenario:<\/p>\n

    \"Low-rate<\/p>\n

    Although the first scenario enjoys a lower cost-per-buyer, the second scenario drives the results that most marketers actually care about: two new<\/em> buyers.<\/li>\n

  3. Digital signal bias. While digital channels will represent about 36% of Canadian advertising spend in 2016, according to eMarketer<\/a>, most sales and transactions still happen offline. However, most advertisers only include online ad spend in their attribution models because it\u2019s fairly straightforward to correlate online ad spend to online sales. While it can seem complex, advertisers need to start thinking about ways they can factor both their online and<\/em> offline sales figures in the \u201cacquisition\u201d portion of their attribution measurements.<\/li>\n<\/ol>\n

    Admittedly, these biases can be daunting for even the most sophisticated marketer. But there\u2019s light at the end of the tunnel. By employing an academic principle called \u201cexperimental design\u201d – which argues that for a measurement to be valid, marketers need to take all<\/strong> the variables into account before<\/strong> the experiment begins \u2013 marketers can start to get a better picture of their return-on-ad-spend (ROAS).<\/p>\n

    So what should marketers account for before a campaign starts? All the people who were going to buy anyway. These people are your control group \u2013 and by having a control group, marketers can stop guessing at whether or not their ad actually drove a sale and measure actual<\/em> lift as opposed to just relative<\/em> lift. In other words, instead of measuring how many total sales occurred, marketers measure how many additional<\/em> sales occurred.<\/p>\n

    There are a few ways to establish these control groups, but the best way is through using placebos: ads that aren\u2019t related to the actual brand that shouldn\u2019t affect how an individual feels about that brand. Advertisers can also try combinations of different creatives; the same creative across different geographic markets; or different ad formats in the same market \u2013 but keep in mind that will only give you part of the story.<\/p>\n

    A lot of marketers have legacy systems and partners in place that will make experimental design tough to implement. A lot of marketers don\u2019t have budget to test different solutions. But the marketers that do have the willingness to look for a better way to measure success will surely find more of it than they can count.<\/p>\n","protected":false},"excerpt":{"rendered":"

    Advertisers are obsessed with measurement \u2013 and for good reason: you can\u2019t improve what you can\u2019t count. For a long time, media measurement was relatively simple: how many people did my ad reach, and how many ads did those people see? This became the basis of the gross rating point (GRP), which became the currency […]<\/p>\n","protected":false},"author":59,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"content-type":"","footnotes":""},"categories":[432,127],"tags":[369,371],"acf":[],"yoast_head":"\nExperimental Design: Measuring Media Effectiveness | Association of Canadian Advertisers<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/acaweb.ca\/en\/2016\/experimental-design-measuring-media-effectiveness\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Experimental Design: Measuring Media Effectiveness | Association of Canadian Advertisers\" \/>\n<meta property=\"og:description\" content=\"Advertisers are obsessed with measurement \u2013 and for good reason: you can\u2019t improve what you can\u2019t count. For a long time, media measurement was relatively simple: how many people did my ad reach, and how many ads did those people see? This became the basis of the gross rating point (GRP), which became the currency […]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/acaweb.ca\/en\/2016\/experimental-design-measuring-media-effectiveness\/\" \/>\n<meta property=\"og:site_name\" content=\"Association of Canadian Advertisers\" \/>\n<meta property=\"article:published_time\" content=\"2016-02-04T16:45:00+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2017-11-03T17:38:18+00:00\" \/>\n<meta property=\"og:image\" content=\"http:\/\/www.acaweb.ca\/en\/wp-content\/uploads\/sites\/2\/2015\/06\/grant-le-riche.jpg\" \/>\n<meta name=\"author\" content=\"grantlericheen\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@ACA_tweets\" \/>\n<meta name=\"twitter:site\" content=\"@ACA_tweets\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"grantlericheen\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"4 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/acaweb.ca\/en\/2016\/experimental-design-measuring-media-effectiveness\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/acaweb.ca\/en\/2016\/experimental-design-measuring-media-effectiveness\/\"},\"author\":{\"name\":\"grantlericheen\",\"@id\":\"https:\/\/acaweb.ca\/en\/#\/schema\/person\/8a343fda770bcac65ac6992b77b7cee7\"},\"headline\":\"Experimental Design: Measuring Media Effectiveness\",\"datePublished\":\"2016-02-04T16:45:00+00:00\",\"dateModified\":\"2017-11-03T17:38:18+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/acaweb.ca\/en\/2016\/experimental-design-measuring-media-effectiveness\/\"},\"wordCount\":855,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/acaweb.ca\/en\/#organization\"},\"image\":{\"@id\":\"https:\/\/acaweb.ca\/en\/2016\/experimental-design-measuring-media-effectiveness\/#primaryimage\"},\"thumbnailUrl\":\"http:\/\/www.acaweb.ca\/en\/wp-content\/uploads\/sites\/2\/2015\/06\/grant-le-riche.jpg\",\"keywords\":[\"Grant le Riche\",\"TubeMogul\"],\"articleSection\":[\"Benchmarking\",\"Digital Marketing\"],\"inLanguage\":\"en-CA\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/acaweb.ca\/en\/2016\/experimental-design-measuring-media-effectiveness\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/acaweb.ca\/en\/2016\/experimental-design-measuring-media-effectiveness\/\",\"url\":\"https:\/\/acaweb.ca\/en\/2016\/experimental-design-measuring-media-effectiveness\/\",\"name\":\"Experimental Design: Measuring Media Effectiveness | Association of Canadian Advertisers\",\"isPartOf\":{\"@id\":\"https:\/\/acaweb.ca\/en\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/acaweb.ca\/en\/2016\/experimental-design-measuring-media-effectiveness\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/acaweb.ca\/en\/2016\/experimental-design-measuring-media-effectiveness\/#primaryimage\"},\"thumbnailUrl\":\"http:\/\/www.acaweb.ca\/en\/wp-content\/uploads\/sites\/2\/2015\/06\/grant-le-riche.jpg\",\"datePublished\":\"2016-02-04T16:45:00+00:00\",\"dateModified\":\"2017-11-03T17:38:18+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/acaweb.ca\/en\/2016\/experimental-design-measuring-media-effectiveness\/#breadcrumb\"},\"inLanguage\":\"en-CA\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/acaweb.ca\/en\/2016\/experimental-design-measuring-media-effectiveness\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-CA\",\"@id\":\"https:\/\/acaweb.ca\/en\/2016\/experimental-design-measuring-media-effectiveness\/#primaryimage\",\"url\":\"http:\/\/www.acaweb.ca\/en\/wp-content\/uploads\/sites\/2\/2015\/06\/grant-le-riche.jpg\",\"contentUrl\":\"http:\/\/www.acaweb.ca\/en\/wp-content\/uploads\/sites\/2\/2015\/06\/grant-le-riche.jpg\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/acaweb.ca\/en\/2016\/experimental-design-measuring-media-effectiveness\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/acaweb.ca\/en\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Experimental Design: Measuring Media Effectiveness\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/acaweb.ca\/en\/#website\",\"url\":\"https:\/\/acaweb.ca\/en\/\",\"name\":\"Association of Canadian Advertisers\",\"description\":\"Just another Association of Canadian Advertisers site\",\"publisher\":{\"@id\":\"https:\/\/acaweb.ca\/en\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/acaweb.ca\/en\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-CA\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/acaweb.ca\/en\/#organization\",\"name\":\"Association of Canadian Advertisers\",\"url\":\"https:\/\/acaweb.ca\/en\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-CA\",\"@id\":\"https:\/\/acaweb.ca\/en\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/acaweb.ca\/en\/wp-content\/uploads\/sites\/2\/2017\/10\/aca-logo.png\",\"contentUrl\":\"https:\/\/acaweb.ca\/en\/wp-content\/uploads\/sites\/2\/2017\/10\/aca-logo.png\",\"width\":682,\"height\":278,\"caption\":\"Association of Canadian Advertisers\"},\"image\":{\"@id\":\"https:\/\/acaweb.ca\/en\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/x.com\/ACA_tweets\",\"https:\/\/ca.linkedin.com\/company\/association-of-canadian-advertisers\"]},{\"@type\":\"Person\",\"@id\":\"https:\/\/acaweb.ca\/en\/#\/schema\/person\/8a343fda770bcac65ac6992b77b7cee7\",\"name\":\"grantlericheen\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-CA\",\"@id\":\"https:\/\/acaweb.ca\/en\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/fccb9cec1a91c78688ea4ceb15c32dd3?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/fccb9cec1a91c78688ea4ceb15c32dd3?s=96&d=mm&r=g\",\"caption\":\"grantlericheen\"},\"description\":\"Managing Director, Canada, TubeMogul\",\"url\":\"https:\/\/acaweb.ca\/en\/author\/grantlericheen\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Experimental Design: Measuring Media Effectiveness | Association of Canadian Advertisers","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/acaweb.ca\/en\/2016\/experimental-design-measuring-media-effectiveness\/","og_locale":"en_US","og_type":"article","og_title":"Experimental Design: Measuring Media Effectiveness | Association of Canadian Advertisers","og_description":"Advertisers are obsessed with measurement \u2013 and for good reason: you can\u2019t improve what you can\u2019t count. For a long time, media measurement was relatively simple: how many people did my ad reach, and how many ads did those people see? This became the basis of the gross rating point (GRP), which became the currency […]","og_url":"https:\/\/acaweb.ca\/en\/2016\/experimental-design-measuring-media-effectiveness\/","og_site_name":"Association of Canadian Advertisers","article_published_time":"2016-02-04T16:45:00+00:00","article_modified_time":"2017-11-03T17:38:18+00:00","og_image":[{"url":"http:\/\/www.acaweb.ca\/en\/wp-content\/uploads\/sites\/2\/2015\/06\/grant-le-riche.jpg"}],"author":"grantlericheen","twitter_card":"summary_large_image","twitter_creator":"@ACA_tweets","twitter_site":"@ACA_tweets","twitter_misc":{"Written by":"grantlericheen","Est. reading time":"4 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/acaweb.ca\/en\/2016\/experimental-design-measuring-media-effectiveness\/#article","isPartOf":{"@id":"https:\/\/acaweb.ca\/en\/2016\/experimental-design-measuring-media-effectiveness\/"},"author":{"name":"grantlericheen","@id":"https:\/\/acaweb.ca\/en\/#\/schema\/person\/8a343fda770bcac65ac6992b77b7cee7"},"headline":"Experimental Design: Measuring Media Effectiveness","datePublished":"2016-02-04T16:45:00+00:00","dateModified":"2017-11-03T17:38:18+00:00","mainEntityOfPage":{"@id":"https:\/\/acaweb.ca\/en\/2016\/experimental-design-measuring-media-effectiveness\/"},"wordCount":855,"commentCount":0,"publisher":{"@id":"https:\/\/acaweb.ca\/en\/#organization"},"image":{"@id":"https:\/\/acaweb.ca\/en\/2016\/experimental-design-measuring-media-effectiveness\/#primaryimage"},"thumbnailUrl":"http:\/\/www.acaweb.ca\/en\/wp-content\/uploads\/sites\/2\/2015\/06\/grant-le-riche.jpg","keywords":["Grant le Riche","TubeMogul"],"articleSection":["Benchmarking","Digital Marketing"],"inLanguage":"en-CA","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/acaweb.ca\/en\/2016\/experimental-design-measuring-media-effectiveness\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/acaweb.ca\/en\/2016\/experimental-design-measuring-media-effectiveness\/","url":"https:\/\/acaweb.ca\/en\/2016\/experimental-design-measuring-media-effectiveness\/","name":"Experimental Design: Measuring Media Effectiveness | Association of Canadian Advertisers","isPartOf":{"@id":"https:\/\/acaweb.ca\/en\/#website"},"primaryImageOfPage":{"@id":"https:\/\/acaweb.ca\/en\/2016\/experimental-design-measuring-media-effectiveness\/#primaryimage"},"image":{"@id":"https:\/\/acaweb.ca\/en\/2016\/experimental-design-measuring-media-effectiveness\/#primaryimage"},"thumbnailUrl":"http:\/\/www.acaweb.ca\/en\/wp-content\/uploads\/sites\/2\/2015\/06\/grant-le-riche.jpg","datePublished":"2016-02-04T16:45:00+00:00","dateModified":"2017-11-03T17:38:18+00:00","breadcrumb":{"@id":"https:\/\/acaweb.ca\/en\/2016\/experimental-design-measuring-media-effectiveness\/#breadcrumb"},"inLanguage":"en-CA","potentialAction":[{"@type":"ReadAction","target":["https:\/\/acaweb.ca\/en\/2016\/experimental-design-measuring-media-effectiveness\/"]}]},{"@type":"ImageObject","inLanguage":"en-CA","@id":"https:\/\/acaweb.ca\/en\/2016\/experimental-design-measuring-media-effectiveness\/#primaryimage","url":"http:\/\/www.acaweb.ca\/en\/wp-content\/uploads\/sites\/2\/2015\/06\/grant-le-riche.jpg","contentUrl":"http:\/\/www.acaweb.ca\/en\/wp-content\/uploads\/sites\/2\/2015\/06\/grant-le-riche.jpg"},{"@type":"BreadcrumbList","@id":"https:\/\/acaweb.ca\/en\/2016\/experimental-design-measuring-media-effectiveness\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/acaweb.ca\/en\/"},{"@type":"ListItem","position":2,"name":"Experimental Design: Measuring Media Effectiveness"}]},{"@type":"WebSite","@id":"https:\/\/acaweb.ca\/en\/#website","url":"https:\/\/acaweb.ca\/en\/","name":"Association of Canadian Advertisers","description":"Just another Association of Canadian Advertisers site","publisher":{"@id":"https:\/\/acaweb.ca\/en\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/acaweb.ca\/en\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-CA"},{"@type":"Organization","@id":"https:\/\/acaweb.ca\/en\/#organization","name":"Association of Canadian Advertisers","url":"https:\/\/acaweb.ca\/en\/","logo":{"@type":"ImageObject","inLanguage":"en-CA","@id":"https:\/\/acaweb.ca\/en\/#\/schema\/logo\/image\/","url":"https:\/\/acaweb.ca\/en\/wp-content\/uploads\/sites\/2\/2017\/10\/aca-logo.png","contentUrl":"https:\/\/acaweb.ca\/en\/wp-content\/uploads\/sites\/2\/2017\/10\/aca-logo.png","width":682,"height":278,"caption":"Association of Canadian Advertisers"},"image":{"@id":"https:\/\/acaweb.ca\/en\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/x.com\/ACA_tweets","https:\/\/ca.linkedin.com\/company\/association-of-canadian-advertisers"]},{"@type":"Person","@id":"https:\/\/acaweb.ca\/en\/#\/schema\/person\/8a343fda770bcac65ac6992b77b7cee7","name":"grantlericheen","image":{"@type":"ImageObject","inLanguage":"en-CA","@id":"https:\/\/acaweb.ca\/en\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/fccb9cec1a91c78688ea4ceb15c32dd3?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/fccb9cec1a91c78688ea4ceb15c32dd3?s=96&d=mm&r=g","caption":"grantlericheen"},"description":"Managing Director, Canada, TubeMogul","url":"https:\/\/acaweb.ca\/en\/author\/grantlericheen\/"}]}},"_links":{"self":[{"href":"https:\/\/acaweb.ca\/en\/wp-json\/wp\/v2\/posts\/1457"}],"collection":[{"href":"https:\/\/acaweb.ca\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/acaweb.ca\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/acaweb.ca\/en\/wp-json\/wp\/v2\/users\/59"}],"replies":[{"embeddable":true,"href":"https:\/\/acaweb.ca\/en\/wp-json\/wp\/v2\/comments?post=1457"}],"version-history":[{"count":0,"href":"https:\/\/acaweb.ca\/en\/wp-json\/wp\/v2\/posts\/1457\/revisions"}],"wp:attachment":[{"href":"https:\/\/acaweb.ca\/en\/wp-json\/wp\/v2\/media?parent=1457"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/acaweb.ca\/en\/wp-json\/wp\/v2\/categories?post=1457"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/acaweb.ca\/en\/wp-json\/wp\/v2\/tags?post=1457"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}