Music testing and on-demand streaming are two of the most widely used methods of measuring the success of music in the radio industry, used to gain insight into listener music preferences.
Music testing has been in use for decades, and it is the process of testing a new song or album by playing it for a group of individuals who represent the target audience.
On-demand streaming music research, on the other hand, involves collecting streaming data from streaming platforms such as Spotify, Apple Music or Amazon music.
And because on-demand music streaming allows a consumer to listen to music whenever and wherever they want, the information is more relevant to actual consumption.
Each research method measures different behaviors and each offers radio programmers variances on preference.
Traditional music testing generally rates songs by consumer response to short 7-10 second “hooks” or usually the most familiar portion of a song such as its chorus. The theory is that upon hearing this familiar segment of a song, the consumer reacts in a positive or negative way and scores the song based on this subjective reaction. Our analysis has shown that the “rating” achieved from this method reflects attitude about the “hook” and does not represent how a listener consumes music in real world circumstances.
On-demand music streaming research is pure data which represents true consumption of songs.
One significant difference in reliability of the streaming data as a research tool is that on-demand music streaming data reflects songs actively chosen and consumed by the user. Traditional music testing uses pre-selected lists of songs radio programmers are interested in learning about.
Both methods have their advantages and disadvantages, and as such, the results of each method may vary. This report aims to explain the reasons why traditional music testing results may differ from on-demand streaming data.
The sample size The sample size used in traditional music testing is usually smaller than charts composed of the number of people who stream music on-demand. Music testing may involve testing a new song on a group of people who represent the target audience or utilize a small sample of listeners who are directed to an on-line music test website where songs are rated and typically involve 100 or more songs. In contrast, on-demand streaming data includes millions of listeners who may have different tastes and preferences. As such, the sample size of traditional music testing may not accurately represent an overall targeted population's preferences.
The testing environment - Traditional Music testing is usually conducted in a controlled environment, such as a testing facility or hotel meeting room, where the participants are given specific instructions and asked to rate the music based on different criteria. On the other hand, on-demand streaming data is collected in real-life situations in which listeners are free to choose the songs they want to stream. This freedom may result in listeners choosing songs that they would not have chosen in a music testing environment, leading to different results.
The time frame - Traditional Music testing is usually conducted weekly if online or a couple of times a year if conducted in a hotel ballroom. Another approach tests songs before a song or album is released to the public. On-demand streaming data is collected continually. As such, traditional music testing may not accurately predict how a new song or album will perform in the market. On-demand streaming data provides real-time information on how the music is consumed in the market, and as such, it is a more accurate representation of the public's preferences.
Demographics Traditional music testing may not accurately represent the diversity of a target audience. Music testing may involve a specific group of people who represent the target audience, but this group may not be diverse enough to represent the different age groups, genders, races, and cultures that make up the overall population. On the other hand, on-demand streaming data includes millions of listeners with diverse backgrounds, providing a more accurate representation of the population's preferences.
Technology - Traditional music testing can be conducted in a variety of ways from old school pencil and paper questionnaires to website interactive sessions or high tech hand-held devices that consumers use to track their impressions while listening to music. On-demand streaming is more straightforward in which a listener visits favorite streaming platform - either desktop or mobile - and streams music on their own schedules. This ease-of-use aspect removes bias associated with consumers proactively traveling to a hotel or website to provide opinions about a song’s personal appeal.
In addition to the factors that may cause traditional music testing results to differ from on-demand streaming data, some songs may perform better with traditional music testing than they do with on-demand streaming data. Here are some reasons why:
Familiarity bias Traditional music testing may favor songs that are familiar to the test audience. In a controlled environment, test participants may have a bias towards songs that they have heard before, leading to higher ratings for those songs. However, on-demand streaming data provides a more accurate representation of how a song performs in the real world, where listeners are free to choose songs.
Niche appeal Some songs may have a niche appeal that may not be reflected in traditional music testing. For example, a song may appeal to a specific subculture or group of people who may not be represented in the traditional music testing audience. In contrast, on-demand streaming data allows for a wider range of listeners, including those with niche tastes, to access and enjoy the song.
Timelessness Traditional music testing may favor songs that are trendy or currently popular, while on-demand streaming data allows for the discovery and continued enjoyment of songs over time. A song that may not have performed well in traditional music testing may gain popularity over time and become a hit through on-demand streaming.
Emotional connection Traditional music testing may not accurately capture the emotional connection that listeners have with a song. A song that may not have performed well in traditional music testing may still have a strong emotional connection with a particular group of listeners, leading to continued streaming and success on on-demand platforms.
Click on image to enlarge
This example of a popular song from 2022, “Numb Little Bug” by Em Beihold, reveals how this song’s appeal based on actual streaming consumption reduced over time, while airplay exposure on radio increased due to traditional music testing results.
While traditional music testing and on-demand streaming data are both useful methods of measuring a song's success, there may be instances where a song may perform better in one method over the other. Understanding the reasons why a song may perform better in traditional music testing can help music industry professionals make informed decisions about how to promote and distribute the song to a wider audience.
Conclusion:
Traditional music testing results may differ from on-demand streaming data due to several factors, including the sample size, familiarity bias, the testing environment, the time frame, and demographics. On-demand streaming research sourced to local radio station listeners reflect actual consumption, not just opinion.
Both methods have their advantages and disadvantages, and as such, it is essential to have an understanding of what each method measures in order to get a comprehensive understanding of song success. By understanding the factors that contribute to the differences in results, music industry and radio professionals can make informed decisions about music appeal and song passion and whether audiences will respond positively.
For a deeper dive into Bridge Ratings’ on-demand streaming music research service, StreamStats, contact Dave Van Dyke at dvd@bridgeratings.com or 323.696.0967.
Let’s discuss your music research needs with a short conversation and how our StreamStats music streaming research can help align your playlist to your audience’s actual preferences. Read more here. A StreamStats Overview