Los Angeles Rapped

An Interactive Archive

Nate Coffin, Taha Mohamad, Miles Bernhard

Introduction

As first year students at Claremont McKenna College, located 30 miles east of Los Angeles, we (Nate, Taha, Miles) soon found that our immediate circle of friends came from all walks of life while subconsciously possessing one commonality: a love for rap music. It soon became self-apparent that hip-hop culture had manifested itself within us in more ways than we were actively aware, from the shoes we wore, to the vernacular we understood, to the like-minded worldviews we shared. Hip-hop culture goes way beyond a popular genre of music, it is an energy. An energy that in its purest form excites social interaction, inspires ambition, and drives change.

The Gould Center for Humanistic Studies, a research institute at CMC, provided us the opportunity to explore our passion for hip-hop culture through the Imagining Los Angeles humanities lab. This project aims to develop an understanding of the cultures development over time, and analyze the ways in which it has impacted the city of Los Angeles and the world in its entirety, both through qualitative and quantitative assessment.

A special thank you to Professor Moffett for his guidance in developing this project.

TIMELINE

While the genre started in New York, hip-hop's rise to popularity can be largely attributed to West Coast and LA specifically. Hip-hop in Los Angeles spawned from the poverty-stricken streets of South Central LA in the 80s, in the trenches of a war on drugs and the peak of street gang activity. Over the past several decades, it has evolved into the epicenter of a multi-billionaire dollar industry that has influenced culture all over the world- extending beyond music to fashion, art, film, language, and social interaction as a whole. At its worst, hip-hop culture is seen to promote themes of violence, drug use, misogyny, and criminal activity. However, just as prevalently, the movement has undoubtedly been a catalyst for social activism, civil rights, and upward mobility. This project aims to track the genesis of hip-hop in Los Angeles and tell an untold story, as an archive of the prominent contributors to this movement and as a commentary on the effects it has had on society.

1970's

A Genre is Born

1980's

Electro-Hip/Hop Fusion Era

Gangsta Rap

N.W.A. - "The World's Most Dangerous Group"

1990's

Evolving Past N.W.A.

Los Angeles Riots

A defining moment for early '90s LA

Death Row Rises

The establishment of classics

Hip Hop & Art

Estevan and Mr. Cartoon

The Aftermath of Gangster Rap

The establishment of classics

Kendrick Lamar

L.A. Hip-Hop's Prophet from Compton

YG & Nipsey Hussle

Modern Day Gangsta Rappers

SoundCloud Era

The Rise(and fall) of Emo Rappers

Alternative Sound

Odd Future and Beyond

The Future

Data Sets

The Goal

We attempted to compile a dataset that contained every rap song ever made, with as much information as possible for each song. With this data, we hoped to quantify trends in rap music, identify key characteristics that differentiate the eras of rap, and analyze the changing impact of geographic origin on rap songs.

The Process

Artists

We first pursued a comprehensive list of rappers. We considered finding as many ranked lists of rappers as we could, then compiling them all together. However, we felt this may lead to a recency bias, and could neglect the “mediocre rapper”, who is in no way notable. Instead, we started with a list from Wikipedia, which was the largest we could find. We then began matching these artists to their Spotify profiles using Spotify’s remarkably powerful API and the python library Spotipy. Of the 1423 artists, we were able to match 1240 of them. From there, we pared the list down using the data Spotify provides on popularity and genre. We removed artists whose genres suggested a non-American following, as well as those with very low popularity scores or and very low follower counts, as we wanted to focus on popular, American rap culture. The result was 819 artists on which we would focus, for which we had their names, Spotify ID’s, associated genre’s, followers, and popularity score.

Discographies

We used Spotify’s API to get the track name, release date, album, duration, and Spotify ID of each of the artist’s songs. We were careful to avoid clean versions and duplicates, for example when an artist releases a song as a single and then includes it on an album. We dropped songs with durations less than 60 seconds, as we found that they were mistakes of some kind, skits, or otherwise insignificant.

Song Data

We then began compiling as much data as we could for each song. First, we used Spotify’s API to collect a variety of “features” for each song: acousticness, danceability, energy, instrumentalness, liveness, loudness, speechiness, valence, and tempo. More about these metrics can be read here. Any songs that did not have this data were dropped from the data set. We then focused on collecting lyrics. For this, we looked to Genius, a site that provides crowd sourced lyrics that are well maintained and widely considered to be reputable. We used the Genius API and accessed it through John Miller’s fantastic python library, LyricsGenius. We dropped songs where lyrics were not available, and songs where the lyrics we found were less than 20 words long, as this points to either an error in the lyric data or an insignificant song. Finally, we collected data on the Billboard success of the song using a data set provided by Sean Miller that compiles the Billboard Weekly Hot 100 chart from 1958 through the end of 2019. We cross referenced our songs with this data set to assign each song a Hot 100 score. Each week spent at number 1 earned 100 points, number 2 earned 99 points, all the way down to number 100, which earned 1 point. We summed these points to determine each songs score. We dropped all the songs from 2020, seeing as we didn’t have Billboard history for this time period. The result was a dataset of including 819 artists, 10167 albums, and 58775 songs, with release dates ranging from the beginning of 1980 through the end of 2019. For each song, we had the artist, Spotify followers, Spotify popularity, artist genres, album name, album release date, song name, lyrics, Hot 100 score, acousticness, danceability, energy, instrumentalness, liveness, loudness, speechiness, valence, and tempo. This was stored in a plain text CSV that was over a third of a gigabyte.

A Note on Data Quality

Throughout the process of collecting this data, we balanced two ideals in mind: “quantity over quality” and “garbage in, garbage out”. While we made sure to avoid erroneous data as much as possible, and we ignored songs where we didn’t have all the data we wanted, we quickly learned that our data was not going to be perfect. There were errors in lyrics on Genius, mismatched songs and lyrics, and songs that would be missed, for example. However, we determined that if we did our best to prevent these problems, that they would “average out” over time. By assembling as much data as possible, we felt that the legitimacy of the overall trends could be established despite a few errors here and there.

Computing More Metrics

We then went about computing a few additional columns for easier analysis going forwards. First, we added a column called ‘words’, which consisted of each word in the lyrics, all lowercase, with all punctuation and line breaks removed. We then used the natural language processing library nltk to create a list of unique word stems for each song. We were then able to compute total word count by counting the number of words and unique word count by counting the number of unique stems. This further allowed us to computer words per minute, unique words per minute, and lexical diversity (a ratio of unique words to total words). Furthermore, we computed a metric called profanity using a Python library called profanity-check. This column represents the ratio of words were flagged as profane to words that weren’t.

Analysis

We determined two primary avenues for analyzing the data. The first of which is plotting the value of one of our metrics over time for a given subset of the songs. For example, we could plot the average Hot 100 score per year of all rap songs, or rap songs by Kanye West, or rap songs of a certain genre, ect. The second direction for data analysis was through the lyrics in particular. We could assemble word lists and plot their frequency in various subsets of songs over time. For example, we could create a list of words related to drugs, and plot drug references over time. We also created a list of every genre that Spotify provided for the artists, and sorted them into east coast, west coast, midwest, and the south to allow us to create different graphs for each region.