Sound check. Reset clock before starting.
A spiritual sequel of a demo/talk I gave at Brooklyn.js back in 2018 about making hip hop with JavaScript.
An overview of algorithmic composition with some ML approaches covered at the end.
A spiritual sequel of a demo/talk I gave at Brooklyn.js back in 2018 about making hip hop with JavaScript.
An overview of algorithmic composition with some ML approaches covered at the end.
A collection of data viz, tech demos & experiments using WebAudio API and various JavaScript/Python tools.
I'm omar delarosa. 👋
I'm a Senior Engineer at Spotify. 🎶
I play music in my spare time. 🎸 🎹
I'm omar delarosa. 👋
I'm a Senior Engineer at Spotify. 🎶
I play music in my spare time. 🎸 🎹
I studied (almost) a minor's-worth of music theory courses. 👶🎼
I'm omar delarosa. 👋
I'm a Senior Engineer at Spotify. 🎶
I play music in my spare time. 🎸 🎹
I studied (almost) a minor's-worth of music theory courses. 👶🎼
I studied a master's-worth of computer science courses. 🤖🎓
I'm omar delarosa. 👋
I'm a Senior Engineer at Spotify. 🎶
I play music in my spare time. 🎸 🎹
I studied (almost) a minor's-worth of music theory courses. 👶🎼
I studied a master's-worth of computer science courses. 🤖🎓
Maybe you know how to code 💻
... but you don't play any instruments
Maybe you know how to code 💻
... but you don't play any instruments
... and you want to make music 🎸
Maybe you know how to code 💻
... but you don't play any instruments
... and you want to make music 🎸
This offers some ideas 🤔
Basics 👶
Basics 👶
Music Theory 🎶
Music Graphs 🕸
Basics 👶
Music Theory 🎶
Music Graphs 🕸
Markov Chains ⛓
Basics 👶
Music Theory 🎶
Music Graphs 🕸
Markov Chains ⛓
Basics 👶
Music Theory 🎶
Music Graphs 🕸
Markov Chains ⛓
Advanced 👴
Basics 👶
Music Theory 🎶
Music Graphs 🕸
Markov Chains ⛓
Advanced 👴
Vectorizing Music ♾
Machine Learning and Music 🤖
Frequences like 440hz
are named notes such as A4
(more info)
Exact tunings can vary by instrument and styles
Frequences like 440hz
are named notes such as A4
(more info)
Exact tunings can vary by instrument and styles
Frequences like 440hz
are named notes such as A4
(more info)
Exact tunings can vary by instrument and styles
Relationships between sound waves
Often occurs when multiple waves oscillate at whole number ratios with each other
Definitions:
Relationships between sound waves
Often occurs when multiple waves oscillate at whole number ratios with each other
Definitions:
duration - how long a tone lasts
beat - a single unit of rhythm
measure - a regularly spaced group of beats
Durations are all described as fractions of a measure
1/4 Note
Durations are all described as fractions of a measure
1/4 Note
1/8 Note
Durations are all described as fractions of a measure
1/4 Note
1/8 Note
1/16 Note
Not all are multiples of 2.
Some interesting things happen when you mix up durations where the denominator of the fraction is a multiple of 3.
Not all are multiples of 2.
Some interesting things happen when you mix up durations where the denominator of the fraction is a multiple of 3.
Chopin's Etudes, Opus 10, No.1 as a pianoroll plot
Music sequences can be thought of as directed walks along the edges of a graph of the state-space in which each discrete* state is a node.
Each state (Sn) can be a single note or a beat or any music element in a sequence of musical events.
*Assume each state in music is discrete. Further discussions of microtonality and certain continuous musical spaces are out of scope in this section.
The entire action space of a sequence of music can thus be described using a finite state machine.
The most straightforward way to produce algorithmic compositions.
"a stochastic model describing a sequence of possible events in which the probability of each event depends only on the state attained in the previous event"
Andrey Markov, namesake of Markov chains and more.
Avoid talking over synths
Avoid talking over synths
Markov chain transition matrices can be learned from a corpus text (or a MIDI file)
Data can be represented and stored easily as structured data formats such as JSON
Markov chain transition matrices can be learned from a corpus text (or a MIDI file)
Data can be represented and stored easily as structured data formats such as JSON
Can extend to more complex, multi-state transition matrices, but this is outside the scope of this prez.
Centered around 1/16th note ticks
Can be difficult to "escape the grid" with durations < 1/16.
Also tough to use subdivisions that are multiples of 3.
// 16-element arrays can represent rhythm patterns, but are tough to read.const kicks = [1,0,1,0, 0,0,1,0, 0,0,1,0, 0,0,1,0];const snares = [0,0,0,0, 1,0,0,0, 0,0,0,0, 1,0,0,0];const hats = [1,1,1,1, 1,1,1,1, 1,1,1,1, 1,1,1,1];const cowbell = [0,0,0,0, 0,0,0,0, 0,0,0,0, 1,0,1,0];
// Easier to read, CPU-trivial preprocessingconst kicks = fmt('1010 0010 0010 0010');const snares = fmt('0000 1000 0000 1000');const hats = fmt('1111 1111 1111 1111');const cowbell = fmt('0000 0000 0000 01010');
// A list of patternsconst kick_patterns = [ fmt("1010 0010 0010 0010"), fmt("1001 0001 0101 0010"), fmt("1000 0101 0100 0010"), fmt("1000 0010 0000 0100"),];
const kick_patterns = [ fmt("1010 0010 0010 0010"), fmt("1001 0001 0101 0010"), fmt("1000 0101 0100 0010"), fmt("1000 0010 0000 0100"),];const snare_patterns = [ fmt("0000 1000 0000 1000"), fmt("0010 1000 0000 1010"), fmt("0000 1000 0010 1000"),];
const kicks_sequence = [ ..._.sample(kick_patterns), ..._.sample(kick_patterns), ..._.sample(kick_patterns), ..._.sample(kick_patterns),];const snare_sequence = [ ..._.sample(snare_patterns), ..._.sample(snare_patterns), ..._.sample(snare_patterns), ..._.sample(snare_patterns),];playParallel(kicks_sequence, snare_sequence);
class MarkovChain { constructor(obj = {}, states = [], initialState = 0) { this.graph = { ...obj }; this.states = [...states]; this.currentState = initialState; } set() { const newState = this.sample(this.graph[this.currentState]); this.currentState = newState; } next() { this.set(); return this.states[this.currentState]; } sample(list) { return list[Math.floor(list.length * Math.random())]; }}
const NOTES = ["C", "D", "E", "F", "G", "A", "B"];const G = { // Repeated notes represent higher probabilities 0: [1, 1, 0, 3, 4, 5, 6], // 0 -> 1 is 2/7, the rest 1/7 1: [0, 0, 2, 3], // 1 -> 0 is 1/2 the others 1/4 2: [1, 3, 4], 3: [4], // 3 -> 4 means state 4 always follows 3 or 1/1 probability 4: [5], 5: [5, 4, 1, 0], 6: [2, 2, 2, 3, 3],};const mc = new MarkovChain(G, NOTES);
const CHORDS = [ "C maj", "D min", "E min", "F maj", "G maj", "A min", "B dim"];// Favors I <-> IV, V -> I cadencesconst G = { 0: [3, 3, 3, 5], 1: [2, 5], 2: [3], 3: [4, 4, 4, 1, 1], 4: [0, 0, 0, 5], 5: [1, 6], 6: [4],};const mc = new MarkovChain(G, CHORDS);
// Tuples of duration & number of beatsconst HATS = [ [16, 4], [12, 3], [24, 6], [32, 4], [48, 6], [64, 8],];// Favors steady 1/16 notes -- common in hip hopconst G = { 0: [0, 0, 0, 0, 0, 0, 1, 2, 3, 4], // 0 -> 0 has 3/5 odds 1: [0, 0, 0, 3], 2: [0, 0, 0, 3], 3: [2, 5], 4: [2, 3, 4, 1], 5: [3, 2, 4, 2, 2],};
Music is a graph.
Music sequences are walks along the edges of those graphs.
Music is a graph.
Music sequences are walks along the edges of those graphs.
Markov chains are cool.
Chopin's Etudes, Opus 10, No.1 as a pianoroll plot
Let s = a given state, t = a point in time and i = a note in some scale.
⬇
2-D Matrices Representing Pitch (Columns) and Time (Rows)
When scaled up to an 8-D hypercube this can easily represent any diatonic scale such as C Major or B minor.
And arpeggiator sequence of nearest neighbor "chord nodes" on an 8-D hypercube:
Generate row vectors using CA update rules:
Stack them to form music state matrices:
Sample notes from the state matrices to make melodies or chords:
Using a data set of MIDI files, transform notes into chord-name strings and then vectors.
(see word2vec for more information on how this works.)
This results in a vector space made of each chord in your dataset
(Each point in this space can be thought of as a node in a graph)
Each chord has some number K nearest neighbors in this space.
Prompt: Make a chill track in the style of lofi hip hop
Beyond Markov Chains there are many other ways to compose algorithmically.
You can use cellular automata
Beyond Markov Chains there are many other ways to compose algorithmically.
You can use cellular automata
Hypercubes
Beyond Markov Chains there are many other ways to compose algorithmically.
You can use cellular automata
Hypercubes
NLP tools
Beyond Markov Chains there are many other ways to compose algorithmically.
You can use cellular automata
Hypercubes
NLP tools
LLMs and beyond
Slides:
Other Links:
Keyboard shortcuts
↑, ←, Pg Up, k | Go to previous slide |
↓, →, Pg Dn, Space, j | Go to next slide |
Home | Go to first slide |
End | Go to last slide |
Number + Return | Go to specific slide |
b / m / f | Toggle blackout / mirrored / fullscreen mode |
c | Clone slideshow |
p | Toggle presenter mode |
t | Restart the presentation timer |
?, h | Toggle this help |
Esc | Back to slideshow |