Generating Song Lyrics

For the second task of generating our own lyrics of a song given a genre, we used a TensorFlow Char-RNN, a character level language model using multilayer Recurrent Neural Network. Using the scraped lyrics data, we created one large text file (~600 kb) for each genre that contained a sample of the lyrics from that genre, about 300 songs from each genre. The five genres we chose to run for the project were: country, indie, hip hop, pop, and rock. We then trained a char-rnn on each of the those genres, and once trained, created a sample text of about 1000 words.

Each training parameter for all the genres were the same. We used a batch size of 100, 10 epochs to train, and didn’t use a dropout. After training, we then used the models we created to generate a sample set of lyrics for each of the genres. For generating each sample, we set the output length to be 1000 and tried various temperatures. The temperatures that were closer to 1 tended to be less coherent in both vocabulary and grammar, whereas the temperatures that were closer to 0 tended to be coherent but contain a lot of repetition. We found that the most accurate temperature overall, balancing coherency and repetition, was 0.5.

Each of the models we created generated the same general results; we had validation perplexities from around 4.5 - 6, and test perplexities which were slightly lower by a margin of 0.5. In general, this meant that our model had higher accuracy on the test data, which meant that our model in general did well on the test sets. The only exception to that was the model we generated for rock; for that model the best validation perplexity was 4.76, while the test perplexity wasa 5.35. This discrepancy could have likely occurred because the random sample of data we used for training did not contain as much variety as the other genres, and therefore the vocabulary distribution was lower. As a result, the accuracy on the training set was lower, leading to a higher perplexity, and leading to a lower quality sample that the other genres.