FACTORSYNTH 2 by JJ BURRED

Factorsynth is a Max For Live device created by J.J. Burred that uses a data analysis algorithm called matrix factorization to decompose any audio clip into a set of temporal and spectral elements. By rearranging and modifying these components you can do powerful transformations to your clips, such as removing notes or motifs, creating new ones, randomizing melodies or timbres, changing rhythmic patterns, remixing loops in real time, applying effects selectively only to certain elements of the sound, creating complex sound textures…

After 2 years of the initial release comes Factorsynth 2, the first major update. Following many user suggestions and requests, version 2 is an even more versatile yet easier to use device, with a simplified workflow and numerous new features. It is now possible to individually pan the components, allowing to do things such as upmixing a mono clip to stereo. Another powerful new feature is the quantized shifting of the components, which allows changing the rhythmic structure of riffs and drum loops. A second, alternative decomposition algorithm is available, as well as a more detailed control of the playback region.

A restricted Demo version of Factorsynth is available here – FACTORSYNTH 2 DEMO

How does Factorsynth work?
Factorsynth is based on a modified version of an algorithm called Non-Negative Matrix Factorization (NMF). Simply put, NMF can automatically extract interesting patterns from data. It has been used in fields such as computer vision and movie recommendations. I had to heavily adapt and tweak it in order to meet the real-time needs of music production.

Can Factorsynth remove a full voice/instrument from a mix?
That’s unlikely, unless your voice or instrument plays only a few sustained notes, with no effects and no vibrato. Factorsynth can extract interesting sound events, such as individual notes, attack noises, impulses or rhythmical structures (watch the demo video to get an idea), but it’s not aimed at separating full instruments. That’s the job of source separation, which is a harder thing to do! On the other hand, you can often nicely separate drum sets and individual drum instruments (kick drums, hi-hats, snares…). For more details about this, check out this blog post.

Can Factorsynth’s parameters be controlled by Live’s automation envelopes or MIDI mappings?
Most of them can (factorization buttons, matrix operation buttons, solo buttons, number of components, all levels and pan sliders, shifting buttons, transport controls and quantization). Also, the first 8×8 buttons on the switchboards are MIDI-assignable. The ones that cannot be controlled by Live are: analysis parameters, clear panel, export panel, “snap” and “all” buttons on the output waveform display, reset and threshold buttons and individual buttons on the switchboards corresponding to component numbers higher than 8.

Can Factorsynth be used with standalone Max?
Yes, starting from version 1.5 it is possible to use them with the standalone Max application, without Ableton Live. There is a bit of patching needed to set it up. For details, you can take a look at the “Usage from Max” section in the manuals.