Something a bit off-topic, but still close to OM workflow.

In the book „Fundamentals of Composition with Opusmodus“ (1st book) the authors introduce the GAT algorithm.

GAT stands for Generate-And-Test.

Developing code myself in my day job as Data-Scientist, the testing approach resonates very much in me in general.

The majority of posts in this community forum deal with the Generate-part.

In contrast to Data-Science, the Test-part in music to my understanding is very subjective and depends on intentions. So thats OK, But:

My personal expereince is that once I hear a probalistically generated piece I loose the ability over time to test in a consistent way.

I wish there would be a way that my test is structured, stored and ready for comparision, say a day later.

Also I wish that this test may be integrated in code for a fast rinse-reapeat cycle.

Still I ask myself whether there‘s a disciplined semi-quantitative approach, without giving up the creative flow an acceptance for personal preferences etc..

Any Best-Practice or other experience from the musicians and composers?

[Edit: No worries - I am not implicitely targeting with my question toward reinforcement learning.]

Idea:

• I have a bit crazy and still very vague idea for the Generate-And-Test-Loop to get data-driven support for parameter optimization after auditory test assessment which is capable to learn the effects accross multiple parameters and thus helps the composer to explore the option space somwhat guided and more informed.

• As alternative the composer just gets new parameters to test next in a Design-of-Experiment like approach

• Its crazy as it might stretch the imagination of artists too much. After all maybe all the esthetic preperences are not easily captured, especially in a undirected composition session.

• Anyways here the idea. I believe if theres one system which can implement it, its Opusmodus.

Outline:

• A piece has 5 musical parameters which are chosen with some randomness but with constraints.

• With every rendering and audition the 5 parameters are stored in a table. The OM system provides functions to do this bookkeeping.

• After audition the composer rates the newest instantiation on a normalized scale, say 0..10.

• After rating, the system performs a Machine Learning Model (example a linear regression) and returns the effects (how does rating change for one unit parameter change) coefficients for each parameter.

• The coefficients in a simple linear model indicate which parameters have the strongest effect on rating.

• It not bound to linear regression, the are for more flexible modeling techniques and also those have ways to interpret a model and the effects.

• Recommended: Maybe the composer even does not look under the hood of the model and instead just gets a recommendation how to explore the parameter space with next trial in light of data, this approach maybe be implented as Bayesian optimization. https://en.wikipedia.org/wiki/Bayesian_optimization

Limitations:

• Most likely the linear regression approach is too simple for the auditory objective function. This may be adressed by extending (still in linear regression technique) the model by allowing interactions between predictor parameters. Also extending the link funktion to generalized linear model may help a bit.

• The more realistic way would be to employ more advanced Machine-Learning algos, which normally can better learn complex reations, including non-linear effects.

• I have no idea whether common-lisp has good ML packages (I work professionally as Data-Scientist with R). Eventually interop with c libraries become necessary, like GitHub - rmcantin/bayesopt: BayesOpt: A toolbox for bayesian optimization, experimental design and stochastic bandits.

• Probably the number of auditions might become pretty large to end up with good estimates for effetcs. The are techniques like the Bayesian frameworkinstead of classic frequentist stats which help to express uncertainty in estimates in a very elegant way.

Benefits:

• The composer recieves an estimate which parameter setting has strongest influence on the rating. The Generate-And-Test cycle maybe supported numerically.

• The calculated effects are capable to decompose the interaction between multivariate parameters.

• Parameters having no or low influence can be removed, in favour of other parameters, keeping things more simple and better to reason about.

• The Bayesian optimization approach is sample efficient, and invites for experimentation.

• It also helps to stop the parameter tweaking at some point where diminishing returns are identified, hence for a given budget of attention and time one may get to a point which is close to optimimal.

• In addition reaching this point of diminishing returns may invite to extend parameters and therefore restart to explore on ground already identified.

• Integration such mechanism might further underline the claim of OM as Compositional-Continuum.

Question:

Any thoughts on this crazy idea?

Resources:

• Incomplete list

Dear @Cliff

I find the idea genuinely interesting ... on first consideration a few issues come to mind. But maybe I’m missing something or misunderstanding certain aspects, so feel free to correct me or put things into perspective.

Defining a piece through a small number of parameters feels too reductive to me, because musical dimensions are rarely independent. Things like form, harmony, rhythm, accentuation, etc. are tightly interconnected and can’t really be evaluated in isolation. So the parameters would need to be connected across different levels and hierarchically organized.

A simple example would be harmony and rhythm: the way consonance and dissonance are treated is often directly linked to rhythmic structure and accent patterns, and this relationship is highly dependent on style or genre. What counts as “good” or “bad” harmonic tension only makes sense within a very specific stylistic context.

This also means that any kind of rating system would have to operate within very clearly defined stylistic boundaries, both on a larger level (genre, style, compositional language) and on a smaller, more local level.

On a more technical note, numerical ratings don’t really provide direction? If an algorithm generates a randomized twelve-tone row and I rate the result as “1” instead of “10”, the system has no idea why it failed or in which direction it should change — different intervals, an all-interval row, Webern-style cells, or something else entirely. Without explicit stylistic and aesthetic goals, such ratings don’t give the system much to work with?

In roughly these areas, there has been research and various approaches for decades — see pioneers like David Cope or projects such as DeepBach, Bachbot, E.M.I.

What I find artistically interesting about OPUSMODUS (or bach/maxmsp or ...) much more so, at least up to now, than AI or machine learning—is that it allows me to intervene very precisely in a structure or a specific aspect of a piece. I can simulate processes (sometimes randomized) and then begin with concrete, directionally guided (!) modifications, giving me specific control. This applies both on a global scale and at a very local level. In essence, it’s also a continuous generate-and-test process, but with a wider (and more specific) range of parameters—both objective and subjective—that I can consider within artistic, historical, and genre-specific contexts?

Greetings

André

Dear @AM :

Thanks for your thoughts and response.

I am not claiming the viability of the idea, hence the phrase crazy idea.

As Data-Scientist I am pretty limited with my professional experience to the technical insights and may assist to explain a few things addressing some of your concerns.

In addition allow me to also add some personal comments, .

I take your post and reply in bold, italics.

• Defining a piece through a small number of parameters feels too reductive to me, because musical dimensions are rarely independent Agreed, but other fields where Bayesian Optimization is used its similar. Things like form, harmony, rhythm, accentuation, etc. are tightly interconnected and can’t really be evaluated in isolation. So the parameters would need to be connected across different levels and hierarchically organized. Modern Machine Learning methods like Bayesian Optimization are definately capable to handle this high dimensional space covering dependecies und non-linearities. Especially Bayesian optimization is very good in this, well established in many fields of modern daily life, design of experiments for drug discovery etc. And one thing for me is very important, the creativity and flexibility and vast amount of early design decisions is still something which I want to decide (and enjoy as real creativity), but once I have a basic framework of composition, then playing with parameters gets at least in my short period of experience more mechanical and in this phase I see room for algorithmic support.

• A simple example would be harmony and rhythm: the way consonance and dissonance are treated is often directly linked to rhythmic structure and accent patterns, and this relationship is highly dependent on style or genre. What counts as “good” or “bad” harmonic tension only makes sense within a very specific stylistic context. Agreed, I am only proposing a one-to-one individual human-in-the-loop mechanism for experimental support limited to one piece and one composer, it will not generalize across composers and genres.

• This also means that any kind of rating system would have to operate within very clearly defined stylistic boundaries, both on a larger level (genre, style, compositional language) and on a smaller, more local level. Agreed, see above. Every composition will start from scratch not fuzz on AI but rather solid experimental support.

• On a more technical note, numerical ratings don’t really provide direction? Number will be necessary to make this work, at the end its applied math. If an algorithm generates a randomized twelve-tone row and I rate the result as “1” instead of “10”, the system has no idea why it failed or in which direction it should change — different intervals, an all-interval row, Webern-style cells, or something else entirely. Without explicit stylistic and aesthetic goals, such ratings don’t give the system much to work with? I do not propose a mechanism which start with a goal. I propose a mechanism to support experimentation, which in other fields like drug research, color formulation, or engineering is established and always keeps information gain by experimental iteration in the very focus. On “has no idea”,these algorithms do not really understand, they just exploit correlations. But in this they are extremely good. Please understand that with Bayesian Optimization the algo keeps a record about uncertainty which in the beginning is large and it than makes recommendations on most efficient next try. After a couple of experiments the algo will become nothing more than an assistant telling the composer what to do next to explore option space and maximize chances to increase the rating. The composer may manually decide to choose for next experiment other parameters. Still the mechanism will also learn from that (provided OM does the bokkkeeping and subsequent calculations).Consisteny in rating may be crutial critical point which refers to a question i raised in my initial post asking for best-practice.

Extremely interesting topic! Unfortunately, I’m currently in travel but will revisit it when I return to Vienna.

This kind of constraint also reminded me of constraint programming. We have a very good tool for constraint programming: the Cluster Engine from Orjan Sandred, which was ported to Opusmodus by @torstenanders. I use it occasionally, and it’s a very good tool as well!

Thx @Stephane Boussuge : As far as I could reflect, I think the GAT-datadriven mechanism I propose needs following functional components:

1. An bookkept inventory of tunable parameters (which unique id), with domain constraints.

   Randomization of parameters with automatic guidance by previous iteration of Bayesian optimization or manual.

2. Rendering and audition of new score.

3. Bookkept rating by composer (given tunable parameter id and their latest settings).

4. Bayesian optimization based on latest rating and applied parameters.

   Back to step 1 with fresh parameters for further exploration of esthetic preference multifold.

Cluster engine might be an implementation detail of step 1. But I think it’s not necessary.