In this activity, we are going to sing!
Okay, not really, but we are going to make music! Yes, we will make music with the use of image processing.Maybe you’re thinking, “Is that even possible?”. It is possible with the help of morphological operations (I told you it’s powerful 🐙).
For this activity, I chose Twinkle Twinkle Little Star. I know all us are familiar with this song. It’s one of the songs/nursery rhymes that I really liked when I was a child and now that I’m living far away from the province, this song reminds me of how the night sky looks like from our place. Shown in Figure 1 is the musical sheet of the said song.
First, I have segmented the image in such a way that the head of each note will be isolated from the staff. This was done with the use of morphological operations. Shown in Fig. 2 and 3 are the code used for the first stanza and the results obtained, respectively. The following morphological operations are used to remove the vertical lines and horizontal lines. Followed by the refining the shape of the heads of each note.
After that, the position of each blob was determined to assign the frequency of each note. To do this, I have calculated the centroid (or the position of the center of the blob) for each blob. The centroid is just equal to the sum of all row pixels divided by the total number of pixels. I have also obtained the area of each blob to be able to identify which blobs represent the quarter notes and half notes. This is important since the value of each note determines the duration of each note. The code for finding the centroid and the area as well as the note assignments is shown in Fig. 4.
As stated above, I have used the area of each blob to determine the period of each note. If the area is less than a certain threshold, I have set it to be a half note. Otherwise, the blob will be a quarter note as shown in Fig. 5.
I have used the same procedure for each stanza of the song and combined them together. Tada! Here is the result of the note playing using image processing : twinkle. I was really happy when I finally got the right tune. I encounter some glitches at the start (the sound playing did not have the right tune) but I have resolved the error by correcting the range of centroid positions for each note.
Over all, I would like to rate myself 10/10 for this activity, because I was able to get the right tune of the song. I would like to thank Kit Guial for sharing me the link for note reading tutorials. 🐧
Little girl 