Text to song synchronization using deep neural network

Abstract

The most crucial aspect of the learning process is memory. There is no knowledge without memory. However, most people lack memory because they memorize things through rote memorization. So in this paper, we implemented a deep neural network (DNN) model that converts plain text into a song, which is done by synchronization of a Musical Instrument Digital Surface (MIDI) file or the music with the text. MIDI files are symbolic representations of the music score. The melodic line of the composition is normally on one of them, while the background music is on the others. Analyzing the audio to identify voice portions where a human voice is present, as well as non-voice segments in proximity to the voice segments, is part of a method for syncing text with audio. The text segments of segmented text related to the audio can be recognized and synchronized to the speech segments.