AI and Music

Professor Roger B. Dannenberg

  • Co-creator of Audacity! Future in AI and Music

  • Humans are driven by an innate desire to achieve automation, this includes the field of music

  • Three main areas of exploration

    • create sound
      • Generate new type of sound by incorporating computer to make new instruments
        • Example
          • Utilize robotics to preform instruments
          • Create electronic instruments such as electric piano and guitar
    • create music
      • Computer-generated music score
    • re-create and extend music performance practice
      • By inputting a huge database of repertoires by a certain composer and using a machine learning model to have the computer “learn” the style of the composer and to create music in the style of deceased artists

  • Combination of performance model and instrument model to create performances from music scores.

    • The performance model allows for the computer to analyze performances by musicians and to incorporate the expressions in their playing

  • Computer-generated, real-time accompaniment

    • Provide piano accompaniment following detailed scores
    • The timing of the accompaniment is flexible and follows along with the speed of the performer according to the human input. The machine has also been taught to be able to continue and follow the rhythm of the player even when he or she makes a mistake.

  • Human-computer music performance

  • Expressive collaborative performance

    • Record timing, dynamics, etc. of duet piano performances, note them on the score, and feed this dataset into a machine learning model

  • Problem with style

    • Style is actually very predictable, machine is very good at learning and predicting by providing examples and dependent variable

  • Related college major

    • CMU: CS, music, EE major
      • School of music has major in electronic in music

  • Successful music generation ML model will be successful if it can incorporate knowledge of music structure and music theory

  • Computer might be able to create its own genre of music!

    • Once it has been fed with enough musical knowledge, computer could figure out its genre

  • Music production

    • Today can mostly be done entirely using software

  • Musical instrument

    • Electric piano, synthesizer, drum machine, linnstrument, sonic spring

  • Music distribution

    • compression, storage, networks
    • Eg. Apple iPod, Apple iTunes

  • Music recommendation

    • Search, recommendation, music fingerprinting
    • Eg. Google Music China, Pandora music recommendation

  • Computer accompaniment

    • Almost 40 years
    • When performance and accompaniment is off sync, how to fix the timing musically

  • Style classification

  • Score alignment

  • Onset detection

  • Sound synthesis

  • Makemusic, smartmusic

  • Fill in needed instruments for performances

  • Very big market for musicians and performances

  • wikifonia