​Smart Indic Lyrics
Interactive music player that helps
language learners improve vocabulary
UX Lead & Web Developer, Team of 2
Self Initiated Project, 16 weeks
​
SKILLS
UX Research, Interaction Design,
Full Stack Development (AngularJS & MongoDb), User Study
​
HONORS
Poster at India HCI 2016
Lightning talk in HCI track at
Grace Hopper Computing India 2016
​
OUTCOME
1.5x effective in vocabulary learning as compared to dual lyrics in user study
Language learners often listen to music in the language they are learning. When a learner doesn't understand a word, they pause the song, carefully listen to the word and look up the meaning in a dictionary - a time consuming process that reduces enjoyability.
​
Alternatively, learners refer to dual lyrics. These provide translation of phrases, not words and lay emphasis on comprehension rather than vocabulary, thus not improving vocabulary learning drastically.
Existing system - Dual Lyrics
​How can we help intermediate-level language learners improve their vocabulary while maintaining an enjoyable song listening experience?
Smart Indic Lyrics is an interactive web-based Indian language music player oriented towards effective vocabulary learning while maintaining an enjoyable experience. The lyrics are annotated with word-level lookups that display dictionary definition, contextual sentence usage and pictorial illustrations. Further, a revisiting mechanism allows learners to save word annotations for long term learning.
​
The below prototype has music, please put on earphones!
Process​
Music helps in language learning - A literature review ​
I started off this project by perusing multiple research papers to understand how music helps in language learning.
Interviewing intermediate level learners & mapping their current user journey
​
To understand if learners actually listen to music in the language being learnt and their experiences with vocabulary building, I spoke to 6 Hindi language learners. These were intermediate learners who had a basic grasp of comprehension of foreign language and continuously learning new words. Some common patterns emerged, and I was able to outline a user journey using interview data.
Understanding what competitors have to offer
​
Next, my teammate and I did a competitive analysis to see what current tools offer. Findings were consistent with interviews in that most applications don't support vocabulary building with music. A key advantage we identified in apps were that combining music/video and lyrics/subtitles at the same place could make language learning convenient and time-effective experience.
Designing the lyrics for quick vocabulary building
​
With all the wealth of information from research, I started making key design decisions to address pain points & needs of my users. I started researching the different ways in which foreign-native language are represented. I made 3 key design decisions that would help my users in vocabulary building.
Animation of the lyric annotations on hover
Designing an interactive music player with everything in one place for time effectiveness
​
The next major decision I made was to incorporate the lyrics in a music player that has songs, lyrics and vocabulary learning in the same place. This makes the learning experience more time-effective, convenient and maintains enjoyability of the music experience without having to shift between mediums.
Highlighting currently played word to focus learner's attention
​
In order to focus learner's attention on currently played word, the lyrics are highlighted in sync with the music. This helps in quickly recognizing an unfamiliar word without having to pause the song or moving back-forth in the song to carefully listen to the word - a common pain point users mentioned in interviews. In addition, this also helps learner enjoy the song without having to constantly look at the screen and look at it only when they want to learn a word.
Please increase the volume in the below video to experience the music syncing with lyrics!
Designing for long term learning
​
Most users (5/6) mentioned that they are not meticulous in collating newly learnt words and expressed a desire to do so. Smart Indic Lyrics allows learners to star words that are added to Learnt Words list. Users can remove words from the learnt words and download it on their devices to revisit it at any point of time.
Coding prototype of Smart Indic Lyrics
​
With the aim of building a real world application for language learners and also evaluating our design through a user study, my teammate and I coded Smart Indic Lyrics using HTML, CSS, AngularJS and MongoDb. We first explored online APIs to obtain dictionary definitions for words, however they didn't support all languages and all words. So we obtained a starter online bilingual dictionary, and manually created JSONS for word level translations, sentences and parts of speech. We implemented the system for Hindi & Tamil songs.
We believe in open source & collaboration. Feel free to contribute to code!
Tale of a User Study - Validating the design & comparing it to Dual Lyrics
​
My teammate and I conducted a study with 10 participants, 5 males and 5 females. The participants were all students aged between 11-12 and studying Hindi. They self-reported that they weren’t from a Hindi speaking background. Our tool was evaluated by comparing the number of newly learnt words learnt while listening to songs using Smart Indic Lyrics to that in the dual Hindi-English lyrics. Participants listened to two 4-minute Hindi songs, one on each tool. Half the participants listened to the first song using Smart Indic Lyrics followed by dual lyrics and the other half followed the reverse order. After hearing both songs, a vocabulary test was conducted and a feedback questionnaire was answered.
Vocabulary Quiz - Which tool wins the most number of newly learnt words?
​
Once participants completed listening to songs on both tools, a vocabulary quiz consisting of 20 questions (10 on each song) of 2 types were asked: 1) Definition when given the word 2) Definition given the word in the context of a sentence. Additional 10 questions were asked to test the definition of the word given its picture. This was asked only for words that appeared using Smart Indic Lyrics. Participants could opt to listen to the pronunciation of the word while taking the quiz. We additionally found out whether the participant already knew the meaning of the word before listening to the song or it was newly learnt. This helped in quantifying the number of learnt words correctly knew the meaning of the word before listening to the song or was it newly learnt.
Feedback Questionnaire - Which tool is easier and more enjoyable?
​
Participants gave feedback of their listening and learning experience with each tool on a 5-point Likert scale.
Vocabulary Quiz Results
While comparing Smart Indic Lyrics (without pics) and Dual Lyrics, users correctly defined 1.5 times as many new words, more questions were correctly answered (t = 2.4097, df = 9, p < 0.0393) and more new words were correctly defined (t = 2.6234, df=9, p= 0.0277) when using Smart Indic Lyrics.
Feedback Questionnaire Results
Users considered it easier to use Smart Indic Lyrics. Listeners enjoyed using both tools almost equally & rated their overall understanding of songs as similar in both cases.
Coding feature tracking during user study
To monitor user behavior, we coded feature tracking such as hover over word, starring of word and so on to further analyze tool usage, successes and scope of improvement for our users.
Reflecting - The wonderful word of HCI
After multiple failed attempts, I learnt how to conduct interviews, sketch my ideas on paper, iterate my design based on user feedback and conduct user study. As it was hard to emulate an experience related to music on paper, I used hi fidelity prototyping for conducting user testing. Instead, I could have done medium-fi testing first and gathered feedback.
Overall, this independent project was a tremendous learning experience for me. While presenting this project at two major conferences, I met wonderful people in the field of HCI and learnt that HCI was much more than imagined - it had data visualization, social computing, designing for tangible interfaces and so on. This was when I decided to pursue a career in this field, and applied for a Master's program to formally learn it.