Schön, F., & Hans, T. (2026). Rotary Informational Embeddings for Symbolic Music Generation. In L. Bouzar-Benlabiod & C. Leung (Eds.), Proceedings of the 39th Canadian Conference on Artificial Intelligence (pp. 1181–1185). PMLR.
E192-03 - Forschungsbereich Knowledge Based Systems
-
Published in:
Proceedings of the 39th Canadian Conference on Artificial Intelligence
-
Volume:
318
-
Date (published):
2026
-
Event name:
The 39th Canadian Conference on Artificial Intelligence
-
Event date:
25-May-2026 - 29-May-2026
-
Event place:
Burnaby, British Columbia, Canada
-
Number of Pages:
5
-
Publisher:
PMLR
-
Keywords:
Attention; Symbolic Music Generation; RoPE
en
Abstract:
In this paper, we present preliminary results on rotary informational embeddings (RotIE), an extension of rotary positional embeddings (RoPE) for Transformer-based symbolic music generation. With RotIE, we adapt the rotary mechanism to encode arbitrary integer-valued information such as pitch, absolute time, or intra-bar positions directly into the attention computation, allowing the model to depend on relative differences in musical attributes rather than on sequential position only. We focus on one representative per-head strategy and evaluate it on the Lakh MIDI and POP909 datasets. The presented results show improved perplexity over a regular Transformer, the Music Transformer, and a RoPE baseline, particularly on longer unseen sequences.
