Traditional Music Testing vs. On-demand Research

Music testing and on-demand streaming are two of the most widely used methods of measuring the success of music in the radio industry, used to gain insight into listener music preferences.

Music testing has been in use for decades, and it is the process of testing a new song or album by playing it for a group of individuals who represent the target audience.

On-demand streaming music research, on the other hand, involves collecting streaming data from streaming platforms such as Spotify, Apple Music or Amazon music.

And because on-demand music streaming allows a consumer to listen to music whenever and wherever they want, the information is more relevant to actual consumption.

Each research method measures different behaviors and each offers radio programmers variances on preference.

Traditional music testing generally rates songs by consumer response to short 7-10 second “hooks” or usually the most familiar portion of a song such as its chorus. The theory is that upon hearing this familiar segment of a song, the consumer reacts in a positive or negative way and scores the song based on this subjective reaction. Our analysis has shown that the “rating” achieved from this method reflects attitude about the “hook” and does not represent how a listener consumes music in real world circumstances.

On-demand music streaming research is pure data which represents true consumption of songs.

One significant difference in reliability of the streaming data as a research tool is that on-demand music streaming data reflects songs actively chosen and consumed by the user. Traditional music testing uses pre-selected lists of songs radio programmers are interested in learning about.

Both methods have their advantages and disadvantages, and as such, the results of each method may vary. This report aims to explain the reasons why traditional music testing results may differ from on-demand streaming data.

  1. The sample size The sample size used in traditional music testing is usually smaller than charts composed of the number of people who stream music on-demand. Music testing may involve testing a new song on a group of people who represent the target audience or utilize a small sample of listeners who are directed to an on-line music test website where songs are rated and typically involve 100 or more songs. In contrast, on-demand streaming data includes millions of listeners who may have different tastes and preferences. As such, the sample size of traditional music testing may not accurately represent an overall targeted population's preferences.

  2. The testing environment - Traditional Music testing is usually conducted in a controlled environment, such as a testing facility or hotel meeting room, where the participants are given specific instructions and asked to rate the music based on different criteria. On the other hand, on-demand streaming data is collected in real-life situations in which listeners are free to choose the songs they want to stream. This freedom may result in listeners choosing songs that they would not have chosen in a music testing environment, leading to different results.

  3. The time frame - Traditional Music testing is usually conducted weekly if online or a couple of times a year if conducted in a hotel ballroom. Another approach tests songs before a song or album is released to the public. On-demand streaming data is collected continually. As such, traditional music testing may not accurately predict how a new song or album will perform in the market. On-demand streaming data provides real-time information on how the music is consumed in the market, and as such, it is a more accurate representation of the public's preferences.

  4. Demographics Traditional music testing may not accurately represent the diversity of a target audience. Music testing may involve a specific group of people who represent the target audience, but this group may not be diverse enough to represent the different age groups, genders, races, and cultures that make up the overall population. On the other hand, on-demand streaming data includes millions of listeners with diverse backgrounds, providing a more accurate representation of the population's preferences.

  5. Technology - Traditional music testing can be conducted in a variety of ways from old school pencil and paper questionnaires to website interactive sessions or high tech hand-held devices that consumers use to track their impressions while listening to music. On-demand streaming is more straightforward in which a listener visits favorite streaming platform - either desktop or mobile - and streams music on their own schedules. This ease-of-use aspect removes bias associated with consumers proactively traveling to a hotel or website to provide opinions about a song’s personal appeal.

In addition to the factors that may cause traditional music testing results to differ from on-demand streaming data, some songs may perform better with traditional music testing than they do with on-demand streaming data. Here are some reasons why:

  1. Familiarity bias Traditional music testing may favor songs that are familiar to the test audience. In a controlled environment, test participants may have a bias towards songs that they have heard before, leading to higher ratings for those songs. However, on-demand streaming data provides a more accurate representation of how a song performs in the real world, where listeners are free to choose songs.

  2. Niche appeal Some songs may have a niche appeal that may not be reflected in traditional music testing. For example, a song may appeal to a specific subculture or group of people who may not be represented in the traditional music testing audience. In contrast, on-demand streaming data allows for a wider range of listeners, including those with niche tastes, to access and enjoy the song.

  3. Timelessness Traditional music testing may favor songs that are trendy or currently popular, while on-demand streaming data allows for the discovery and continued enjoyment of songs over time. A song that may not have performed well in traditional music testing may gain popularity over time and become a hit through on-demand streaming.

  4. Emotional connection Traditional music testing may not accurately capture the emotional connection that listeners have with a song. A song that may not have performed well in traditional music testing may still have a strong emotional connection with a particular group of listeners, leading to continued streaming and success on on-demand platforms.

Click on image to enlarge

This example of a popular song from 2022, “Numb Little Bug” by Em Beihold, reveals how this song’s appeal based on actual streaming consumption reduced over time, while airplay exposure on radio increased due to traditional music testing results.

While traditional music testing and on-demand streaming data are both useful methods of measuring a song's success, there may be instances where a song may perform better in one method over the other. Understanding the reasons why a song may perform better in traditional music testing can help music industry professionals make informed decisions about how to promote and distribute the song to a wider audience.

Conclusion:

Traditional music testing results may differ from on-demand streaming data due to several factors, including the sample size, familiarity bias, the testing environment, the time frame, and demographics. On-demand streaming research sourced to local radio station listeners reflect actual consumption, not just opinion.

Both methods have their advantages and disadvantages, and as such, it is essential to have an understanding of what each method measures in order to get a comprehensive understanding of song success. By understanding the factors that contribute to the differences in results, music industry and radio professionals can make informed decisions about music appeal and song passion and whether audiences will respond positively.

For a deeper dive into Bridge Ratings’ on-demand streaming music research service, StreamStats, contact Dave Van Dyke at dvd@bridgeratings.com or 323.696.0967.

Let’s discuss your music research needs with a short conversation and how our StreamStats music streaming research can help align your playlist to your audience’s actual preferences. Read more here. A StreamStats Overview

Creating Radio Variety?

Part of our business here at Bridge Ratings over the last seven years has focused on music streaming research to assist radio programmers in properly aligning their on-air playlists with the songs their audience is listening to when not listening to the radio.

Our client stations have found this information very useful and helpful in increasing time-spent listening and daily listening occasions.

We certainly have found programmers in this business unwilling to utilize this valuable resource of our on-demand music streaming data for a number of reasons. This is what we’ve learned:

Some radio programmers play songs with little or no streaming consumption for a variety of reasons.

Why do music consumers prefer streaming services over radio?

First, we’re told that some radio stations have a goal to provide their listeners with a diverse range of music, and not just what is popular or heavily streamed at the moment. This helps them cater to a wider audience and attract listeners who enjoy a variety of music styles.

Second, some radio stations may play lesser-known or under-the-radar songs as a way to discover and promote new artists and music. This can be seen as a way of supporting the music industry and helping new artists gain exposure and recognition.

Third, radio stations may also play older songs that may not be as popular in terms of streaming, but still have a dedicated fanbase or hold sentimental value to listeners.

Finally, radio stations may choose to play songs with low streaming numbers as a way to differentiate themselves from other radio stations or streaming services, which tend to prioritize popular music. By playing a more eclectic mix of music, radio stations can offer a unique listening experience and build a loyal audience.

Interesting insights, yes, but does this approach yield higher ratings which can be monetized which is the real reason radio stations operate in the first place.

  1. Diversity: According to a Nielsen study, radio listeners in the United States are exposed to a diverse range of music, with an average of 12 different genres played in a typical week. This shows that radio stations prioritize playing a diverse mix of music, even if it may not be heavily streamed.

  2. Promotion of new artists: A report from the Radio Advertising Bureau found that radio remains the number one source for discovering new music, with 63% of people saying they discovered new music through radio. This demonstrates the role that radio plays in promoting new artists and music, even if it may not have high streaming numbers yet.

  3. Nostalgia: A survey by the website Vinyl Me, Please found that 84% of respondents said they listen to music on the radio because it brings back memories and emotions from their past. This highlights the importance of playing older, nostalgic songs on the radio, even if they may not have high streaming numbers.

  4. Differentiation: According to a report from Edison Research, 45% of radio listeners in the United States say they listen to the radio because they like the variety of music played. This highlights the importance of radio stations offering a unique mix of music that sets them apart from other media sources, and the willingness of listeners to tune in for this reason.

In the land of traditional radio, most markets are saturated with radio stations, many of which have comparatively few listeners. But these stations serve a purpose as highlighted above, to offer variety, choice and a different way of providing musical entertainment. These approaches rarely create ratings gold. There are several commercial stations that do well financially with a variety approach, but most do not.

In our work with station ownership and programmers, we have learned that most radio listeners use traditional radio for familiar music and tune-in primarily in the car when instant gratification is required. These same listeners who use radio several times a week mostly discover new songs and unfamiliar older songs through streaming services and get their fix for this type of listening experience due to the ease and simplicity of streaming services. Streaming and radio listening are joined at the hip and those radio listeners who also stream music, utilize both platforms in a symbiotic way using radio to find songs they wish to add to their playlists and using on-demand streaming to discover songs they may want to hear on their favorite station.

Stop Programming By Consensus

If you've been following Bridge Ratings' research pieces about on-demand music streaming and its value to radio programmers, the following is the latest insight we've gained from discussing this important technology with programmers in markets large and small.

Back in the day when there were no radio station monitoring services like BDSradio (from Nielsen) or Mediabase, radio programmers had to rely on their own gut, local research and listener input to determine the best songs to play.

Sometimes, programmer ingenuity provided insight.

If a programmer wanted to hear a respected station in another city, they'd ask the GM for some travel money, get on a plane and spend a few days in that other market manually monitoring the station, logging all songs, promos and clocks.

Returning to their home market, the Program Director would lovingly analyze their notes and determine the application - if any - to their local situation.

With the coming of technology these types of market trips are generally no longer necessary, what with monitoring services and on-line streaming.

Isn't technology great?

In this case, I think not.

Published station and radio format charts are now available to programmers, many of whom depend on these charts to determine song selection and rotations. The published charts do have their value to some program directors.

These format charts are an aggregation of dozens - even hundreds - of stations in different markets. Now that music research has been eliminated from many radio station budgets, the phenomenon of "Consensus Programming" has disrupted radio's ability to properly expose music to its listeners.

Programming by consensus means that programmers all across our great land look to the published charts for their formats and adjust their music categories based on the aggregate.

The resulting playlists may be 100% appropriate for some market situations.

Or more likely - those lists are a general view of radio airplay across fifty states.

The result is hundreds - maybe thousands - of radio stations are playing song lists that are very similar.

And this is where the wheels come off.

For over two years, Bridge Ratings has been providing on-demand streaming music research to our clients and we have learned at least a couple of important concepts:

1. Programming by consensus results in stations adding songs too late and getting off songs too early in more cases than not.

2. The lifecycle of hit songs - whether current or old - is much longer than we've ever thought.

Here are two examples:

A) The current multi-format smash "I Took A Pill In Ibiza" by Mike Posner has just recently appeared in published charts in the top ten most-played songs on Top 40 radio. It's still trending up. Our streaming research showed that true consumption of that song was in the Top 5 eight weeks ago!

What does this mean? It means that the published charts showed "Ibiza" gradually climbing the charts from outside the top 50 to it's current Top 10 status. Radio's listeners were streaming this song multiple times a week long before radio caught on!

B) Country music star Chris Stapleton's song "Fire Away"  blasted into the top 20 most on-demand streamed songs right after Chris' ACM award windfall on April 3. Yet the song was not even ranked in the top 50 most-played songs by the aggregate of America's Country music stations.  Based on personal guidelines, a Country music programmer seeing this may consider that it's too early to play that song and will wait to see if it rises high enough to warrant adding to their playlist.

Meanwhile, Country music fans were streaming the heck out of that song.

If not enough Country stations add "Fire Away", it could very well stall outside the top 40 and never get a rightful place on American broadcast radio.

In our analysis, Bridge Ratings found that in 55% of the cases studied, the aggregate music charts are not representative of true music consumption has observed in week after week of on-demand streaming data.

As digital data becomes more available through streaming data providers and platforms like Shazam, programmers are, indeed, better equipped to see how music fans are consuming.

Click Here to enlarge.

Yet, the most accurate method we have found to determine song popularity, longevity and viability, is on-demand streaming data.

The chart to the right compares a recent Pop song's on-demand streaming lifecyle with that of consensus/published charts over the course of 15 weeks.

Upon release, on-demand streaming for this song vaulted into the top five almost immediately. It's popularity grew as more fans became aware through word-of-mouth, broadcast radio and other streaming services.

It sustained this lofty position for the full fifteen weeks.

By comparison, upon its release, radio added the song and it was first ranked #78 on published radio airplay charts. As the chart shows, it took six to seven weeks before the aggregate of radio had pushed the song into the top 10 where it slowly faded after programmers must've considered the song overplayed or burned out.

As this song's progress on the published chart slowed, programmers got off the record or slowed its rotation.  Meanwhile, demand for the song remained extremely hot through on-demand streaming platforms from YouTube to Spotify to Amazon Prime.

We have found that on-demand streaming where consumers choose what they want to listen to, is more closely aligned with the behavior of radio listening than any other type of music research.

So, if you're a radio programmer reading this...do your listeners a favor and stop programming by consensus.