There is always that nagging doubt—just how accurate are those Nielsen ratings? The Council for Research Excellence (CRE), a think-tank of senior-level media and advertising professionals, intends to find out. CRE will conduct a three-market study of various audience-measurement methods. CRE’s objective is to improve diary-based TV-audience measurement and in turn improve ratings quality.
The three television markets to be included in the “SQ:L” (for “Sample Quality: Local”) Study are Dallas-Fort Worth, the fifth-largest TV market and a Nielsen local people-meter (LPM) market; Albuquerque-Santa Fe, the 45th-largest TV market and a standard Nielsen-meter market; and Paducah, Kentucky-Cape Girardeau, Missouri-Harrisburg, Illinois -- Nielsen market # 81 and a diary market.
The markets were selected due to varying measurement methodology; varying market size; and divergent characteristics, such as geographical coverage, ethnic make-up, number of over-the-air households and penetration levels of electronic devices. During the standard May 2012 diary measurement, a separate diary sample was selected for Dallas, while identified “non-TV homes” in all three markets will receive a modified diary.
Sampled homes in all markets will be mailed a short questionnaire seeking answers on media equipment ownership and general viewing patterns. Homes identified as not having a television set will be contacted and asked if they have any source of viewing television programming; those saying “yes” will be sent a modified diary in which they will be asked to record what they view and the device used for viewing.
The study, involving the CRE’s Sample Quality, Set-top Box, Local Measurement and Media-related Universe Estimates Committees, marks the CRE’s second study of audience measurement in Dallas-Fort Worth.
Data analysis from a 2009 Universe Estimates Committee study of the market revealed significant differences from the Nielsen sample in ownership of HD sets and DVD players, and an unexpectedly high percentage of non-TV households. That same year, Nielsen garnered considerable bad press when local affiliate stations dropped the services. As Broadcasting & Cable reported, New York-based WKBW Buffalo dropped Nielsen in favor of Media Audit. Sunbeam Television in Miami leveled a lawsuit, claiming Nielsen's Local People Meters "produced defective, wildly inaccurate ratings data which-literally overnight-created havoc in [Miami]." Nielsen responded that it was not unusual for stations to cycle in and out of its services.
“Some of our prior research has helped us realize we have many more questions that need to be answered in order to improve diary sampling,” said Ceril Shagrin, executive vice president of Univision Communications, who serves as chair of the CRE as well as its Sample Quality Committee. “We need to learn whether expanded media-related equipment ownership can be obtained from diary samples, whether return-path data can improve diary measurement, and how much ‘TV program’ viewing is now done – and on what devices -- in what are currently defined as ‘non-TV homes.’
“By making comparisons of diary-based measurement to meter-based measurement and set-top-box information – and conducting specific follow-up studies with non-responding as well as non-TV homes – this effort should provide new insights into responders and non-responders of the address-based sampling diary service,” Shagrin added.
This newest study also marks the second major effort for the CRE’s Sample Quality Committee, formerly known as the Non-Response Bias Committee. Its 2009 study, “Measuring the Unmeasured Viewer,” was the most comprehensive of its kind to determine the impact of non-response on ratings -- revealing more about unmeasured viewers than any prior effort.
Data collected from the new study, conducted with the assistance of Research Triangle Incorporated, will be compared with Nielsen LPM, metered and diary market data in an effort to determine, among other details, the impact on response bias of weighting, adjusting return path or set-meter data, geographic and demographic variables, and ownership of a traditional TV set as well as a land-line phone or cellphone only. Findings from the study are expected to be made available by first quarter 2013.
To date, the CRE has completed several major studies, including the Video Consumer Mapping Study, conducted in 2008, involving in-person, computer-assisted observation of media consumption; a Set-Top Box Study, examining the state of set-top box-based audience research; the landmark Non-Response Bias Study, exploring the impact and correlates of non-response to Nielsen surveys; a study of Media-related Universe Estimates; an initial phase of a Study of User Experience on multiple video screens and formats; and a study of Digital publishers’ handling of user data.