Audio Software Development
Raag Sequencer (In Progress)
A sequencer / melody generator module for VCV Rack, the modular synthesis environment. It allows the user to build a musical scale as a directed graph and then generate melodies by traversing the graph.
Coming soon
Web-based Interactive Musical Systems
Groove Machine
GrooveMachine aims to foster interest in computer science among children through the music-driven exploration of a computational system. It is a physical table interface which acts as a step-sequencer. A software version of Groove Machine was created as an interactive website which was developed using Pixi.js and Tone.js
​
I presented a paper on Groove Machine, titled "From the museum to the browser: Translating a music-driven exhibit from physical space to a web app" at the Web Audio Conference 2019 held in Trondheim, Norway.
Ball Dance
Ball Dance is an interactive musical system. It is an artwork that allows users to explore sounds that are generated as a result of chaos in the system which is constrained by rules that govern the behaviour of the floating balls. The website is developed using p5.js and matter.js. Sound Design is done using Ableton and Max.
AV Loops
AV Loops is an interactive and collaborative webapp which lets users explore musical ideas together to create an immersive audiovisual experience. Users can create sequences using nodes, add musical notes to them using the keyboard keys and select sounds from instrument presets. Each sound is mapped to a short animation. Multiple users can use the app at the same time, each adding sounds to their own node to create an audiovisual experience together.
Machine Learning and Deep Learning
Generating Room Impulse Responses using Recurrent Neural Networks
Convolutional reverb is a reverb generation method that simulates acoustics of a room by convolving the an input audio signal with an impulse response recorded at that room. For this, we require recorded impulse responses which might not always be easily available. In this project, we try to generate the room impulse responses using a LSTM. We survey techniques like truncated backpropogation through time, teacher forcing and auxiliary loss.
Slides describing the setup and experiments:
Audio for Vibraphone C6 note:
Original
Generated
Modelling Equalization Effects Using Neural Networks
Audio equalization is an audio effect which involves filtering the audio signal to modify the frequencies present in it. This project proposes an end to end learning approach to reproduce such filtering effects using a minimal fully connected network. The proposed network is able to model the transfer functions for lowpass, highpass, lowshelf, highshelf, peaking and notch filters. Since time domain data is used, such an approach can be extended to real time systems.
Plot for second order lowpass filter (cutoff: 750Hz)
Audio for second order lowpass filter (cutoff: 750Hz):
Input
Target
Prediction
Equation Recognition
This project aims to understand and compute basic mathematical expressions. It is an android app that works as a calculator which takes handwritten input. The character recognition was done using K-means clustering. We also explored another approach that used convolutional neural networks to do the same.
Game Development
SpaceBoy
SpaceBoy is a 2D game developed using the Unity Game Engine. The protagonist, an astronaut, is stranded on an unknown planet and has to save himself from the wrath of the aliens. You can kill the aliens by using the lazer beam gun. If an alien touches you, your health reduces. You can fly around by using the jetpack. You can also teleport by creating portals using your portal gun.
Interactive Music using Max/MSP
Generative Music
This project / composition converts text to music by mapping its ASCII values to pitches. The composition consists of two sections of 30 seconds where the first section directly sonifies the text as is and the second section generates new text using Markov Chains and then sonifies it.
Sound Synthesis
This Max program utilizes Ableton's Live instruments and creates a synthesizer with modifiable parameters. It provides the user with five interpretable parameters that can be used to modify the sound. The parameter are mappings are defined such that a high ceiling of sounds can be achieved.
Learnable Sound Mappings using Machine Learning
This is an extension of the Sound Synthesis project. An built interface using TouchOSC is used to interface with the five synthesis parameters. An artificial neural network is used to generate mappings from the input interface to the parameter configuration that produces sounds. The interface also allows to train the network through the app.
Sequence Controller using Sensor Inputs
This project allows you to select samples to be played in loop and build a sequence. TouchOSC is used to build a mobile interface that allows to activate and deactivate pre-recorded samples. The accelerometer of the phone is used to control center the frequency of bandpass filters that are applied to each type of sound.
Misc
MP3 Metadata Editor
‘MP3 Metadata Editor’ is python program to edit the metadata of an mp3 file. ID3 tags is a standard used by the MP3 audio file format for its metadata. It allows information such as the title, artist, album, track number, and other information about the file to be stored in the file itself in the form of frames. The program allows you to edit this information by locating and the appropriate frame, updating its data field and accordingly updating the frame size and flags.