nav bg

Alexa Classical Study Music

Description

Classical Study Music is a voice application I built which gives your Alexa-enabled device the ability to instantly play Bach in the background. Ideally, allowing you to concentrate on your latest project, study for an upcoming assessment, or simply provide a gentle ambiance filling the room.

“Alexa, play classical study music.”
Through 2018, Classical Study Music has produced over 200,000 total use sessions and 24,000 unique users— averaging 550 users per week! 🎉

Process

I began digging into the Alexa development ecosystem by building 2 small applications through my Software Engineering curriculum at Bloc. I referenced Amazon's Alexa Skills Kit as a starting point in building 2 beginner apps:

  • A trivia game skill
  • A fact-of-the-day type skill

From there, I started to become more comfortable with developing for voice and sought to create my own Alexa Skill. My overall steps where:

  • Design a Skill
  • Develop a Skill
  • Enhance the Skill with analytics and user testing
  • Go live and promote my skill on the app store

Technology

  • The App's core language is Node.js
  • Utilizes a distributed, serverless content delivery stack
  • Powered by Amazon Web Services: AWS Lambda, S3 and DynamoDb
  • A completely invisible Vocal Frontend Interface
  • The source audio library is Open Goldberg Variations

Challenges

Voice UI Design

There's a big difference when creating a UI for voice compared to conventional UI design. You have to consider how actions, prompts, and responses are pronounced.

For example, writing the year 1995 in your code as part of the user prompt of a trivia question will be spoken by the engine as the number "1,995". Therefore, your code has to carefully consider language as you compose your output, in this case, to read "nineteen ninety-five" instead. Thorough testing was critical in this regard.

App Store Verification

The team at Amazon has gotten a lot better with the approval process over time. My first iteration took the longest for approval as Amazon kicks the tires on a brand new app and inspects your code. Their team was keen to start a dialogue which provided edit suggestions. This process made sure the app's settings and invocation phrases complied with the platform.

Determining Intents

Another challenge was determining the unique Intents for this app. Each command is mapped to a particular function execution. Each execution also depends on its context e.g. "is a song currently playing"? The code snippet below provides a glimpse into that deterministic implementation.

Dev Details

A code sample from the Node.js implementation hosted in AWS Lambda

'use strict';

var Alexa = require('alexa-sdk');
var audioData = require('./audioAssets');

var stateHandlers = {
  startModeIntentHandlers: Alexa.CreateStateHandler(constants.states.START_MODE, {
    /*
     *  All Intent Handlers for state : START_MODE
     */
    LaunchRequest: function() {
      // Initialize Attributes
      this.attributes['playOrder'] = Array.apply(null, { length: audioData.length }).map(
        Number.call,
        Number
      );
      this.attributes['index'] = 0;
      this.attributes['offsetInMilliseconds'] = 0;
      this.attributes['loop'] = true;
      this.attributes['shuffle'] = true;
      this.attributes['playbackIndexChanged'] = true;

      //  Change state to START_MODE
      this.handler.state = constants.states.START_MODE;

      var message = 'Welcome to Classical Study music. You can say, play the audio, to begin.';
      var reprompt = 'You can say, play the audio, to begin.';

      this.response.speak(message).listen(reprompt);
      this.emit(':responseReady');
    },
    PlayAudio: function() {
      if (!this.attributes['playOrder']) {
        // Initialize Attributes if undefined.
        this.attributes['playOrder'] = Array.apply(null, { length: audioData.length }).map(
          Number.call,
          Number
        );
        this.attributes['index'] = 0;
        this.attributes['offsetInMilliseconds'] = 0;
        this.attributes['loop'] = true;
        this.attributes['shuffle'] = false;
        this.attributes['playbackIndexChanged'] = true;
        //  Change state to START_MODE
        this.handler.state = constants.states.START_MODE;
      }
      controller.play.call(this);
    },
    'AMAZON.HelpIntent': function() {
      var message = 'Welcome to Classical Study music. You can say, play the audio, to begin.';
      this.response.speak(message).listen(message);
      this.emit(':responseReady');
    },
    'AMAZON.StopIntent': function() {
      var message = 'Good bye.';
      this.response.speak(message);
      this.emit(':responseReady');
    },
    'AMAZON.CancelIntent': function() {
      var message = 'Good bye.';
      this.response.speak(message);
      this.emit(':responseReady');
    },
    SessionEndedRequest: function() {
      // No session ended logic
    },
    Unhandled: function() {
      var message = 'Sorry, I could not understand. Please say, play the audio, to begin.';
      this.response.speak(message).listen(message);
      this.emit(':responseReady');
    },
  }),
};

module.exports = stateHandlers;

var controller = (function() {
  return {
    play: function() {
      /*
       *  Using the function to begin playing audio when:
       *    Play Audio intent invoked.
       *    Resuming audio when stopped/paused.
       *    Next/Previous commands issued.
       */
      this.handler.state = constants.states.PLAY_MODE;

      if (this.attributes['playbackFinished']) {
        // Reset to top of the playlist when reached end.
        this.attributes['index'] = 0;
        this.attributes['offsetInMilliseconds'] = 0;
        this.attributes['playbackIndexChanged'] = true;
        this.attributes['playbackFinished'] = false;
      }

      var token = String(this.attributes['playOrder'][this.attributes['index']]);
      var playBehavior = 'REPLACE_ALL';
      var audio = audioData[this.attributes['playOrder'][this.attributes['index']]];
      var offsetInMilliseconds = this.attributes['offsetInMilliseconds'];
      // Since play behavior is REPLACE_ALL, enqueuedToken attribute need to be set to null.
      this.attributes['enqueuedToken'] = null;

      if (canThrowCard.call(this)) {
        var cardTitle = 'Playing ' + audio.title;
        var cardContent = 'Playing ' + audio.title;
        this.response.cardRenderer(cardTitle, cardContent, null);
      }

      this.response.audioPlayerPlay(playBehavior, audio.url, token, null, offsetInMilliseconds);
      this.emit(':responseReady');
    },
    stop: function() {
      this.response.audioPlayerStop();
      this.emit(':responseReady');
    },
    playNext: function() {
      /*
       *  Called when AMAZON.NextIntent or PlaybackController.NextCommandIssued is invoked.
       *  Index is computed using token stored when AudioPlayer.PlaybackStopped command is received.
       *  If reached at the end of the playlist, choose behavior based on "loop" flag.
       */
      var index = this.attributes['index'];
      index += 1;
      // Check for last audio file.
      if (index === audioData.length) {
        if (this.attributes['loop']) {
          index = 0;
        } else {
          // Reached at the end. Thus reset state to start mode and stop playing.
          this.handler.state = constants.states.START_MODE;

          var message = 'You have reached the end of the playlist.';
          this.response.speak(message).audioPlayerStop();
          return this.emit(':responseReady');
        }
      }
      // Set values to attributes.
      this.attributes['index'] = index;
      this.attributes['offsetInMilliseconds'] = 0;
      this.attributes['playbackIndexChanged'] = true;

      controller.play.call(this);
    },
    playPrevious: function() {
      /*
       *  Called when AMAZON.PreviousIntent or PlaybackController.PreviousCommandIssued is invoked.
       *  Index is computed using token stored when AudioPlayer.PlaybackStopped command is received.
       *  If reached at the end of the playlist, choose behavior based on "loop" flag.
       */
      var index = this.attributes['index'];
      index -= 1;
      // Check for last audio file.
      if (index === -1) {
        if (this.attributes['loop']) {
          index = audioData.length - 1;
        } else {
          // Reached at the end. Thus reset state to start mode and stop playing.
          this.handler.state = constants.states.START_MODE;

          var message = 'You have reached the start of the playlist.';
          this.response.speak(message).audioPlayerStop();
          return this.emit(':responseReady');
        }
      }
      // Set values to attributes.
      this.attributes['index'] = index;
      this.attributes['offsetInMilliseconds'] = 0;
      this.attributes['playbackIndexChanged'] = true;

      controller.play.call(this);
    },
    loopOn: function() {
      // Turn on loop play.
      this.attributes['loop'] = true;
      var message = 'Loop turned on.';
      this.response.speak(message);
      this.emit(':responseReady');
    },
    loopOff: function() {
      // Turn off looping
      this.attributes['loop'] = false;
      var message = 'Loop turned off.';
      this.response.speak(message);
      this.emit(':responseReady');
    },
    shuffleOn: function() {
      // Turn on shuffle play.
      this.attributes['shuffle'] = true;
      shuffleOrder(newOrder => {
        // Play order have been shuffled. Re-initializing indices and playing first song in shuffled order.
        this.attributes['playOrder'] = newOrder;
        this.attributes['index'] = 0;
        this.attributes['offsetInMilliseconds'] = 0;
        this.attributes['playbackIndexChanged'] = true;
        controller.play.call(this);
      });
    },
    shuffleOff: function() {
      // Turn off shuffle play.
      if (this.attributes['shuffle']) {
        this.attributes['shuffle'] = false;
        // Although changing index, no change in audio file being played as the change is to account for reordering playOrder
        this.attributes['index'] = this.attributes['playOrder'][this.attributes['index']];
        this.attributes['playOrder'] = Array.apply(null, { length: audioData.length }).map(
          Number.call,
          Number
        );
      }
      controller.play.call(this);
    },
    startOver: function() {
      // Start over the current audio file.
      this.attributes['offsetInMilliseconds'] = 0;
      controller.play.call(this);
    },
    reset: function() {
      // Reset to top of the playlist.
      this.attributes['index'] = 0;
      this.attributes['offsetInMilliseconds'] = 0;
      this.attributes['playbackIndexChanged'] = true;
      controller.play.call(this);
    },
  };
})();

Demo

  1. Login to alexa.amazon.com with your Amazon account
  2. Search and add the Classical Study Music skill via Amazon's app store
  3. Use echosim.io or your Alexa-enabled device to try it live!

An Amazon Alexa application