Creating a new instrument
The following describes the process of creating a new instrument.
- Create assets for the instrument, including 3D model and texture.
- Export these assets in a way that the code can manipulate.
- Create a Kotlin class for the instrument.
- Write logic to drive animation procedurally from MIDI data.
Creating assets
First, create assets for the instrument. This includes the 3D model and texture.
3D Model
It might be necessary to export the 3D model in pieces for separate animation of individual components in the code. For example, the keys of the piano are separate from the body of the piano, so that the code animates they keys independently of the body.
Make sure each part exports with a transform of (0, 0, 0) and rotation of (0, 0, 0) so that the initial
transformations don't offset transformations applied in code.
Free software such as Blender can be used to create the 3D model.
Export the 3D model as an Autodesk Filmbox—FBX—file.
Texture
Export the texture as a PNG file. The texture should be a single image that has all the textures for the instrument. It should ideally be a power of two in both dimensions—for example, 1024×1024, or 2048×2048.
Free software such as GIMP can be used to create the texture.
Loading assets
Place assets in the src/main/resources/Assets directory.
Creating a Kotlin class
Depending on the instrument, a relevant class likely already exists—use this class as the superclass.
If not, create a new class that extends Instrument.
The following is the hierarchy of abstract classes for instruments. This promotes code re-usability.
classDiagram
direction RL
Instrument <|-- SustainedInstrument
Instrument <|-- DecayedInstrument
Instrument <|-- ToggledInstrument
SustainedInstrument <|-- MonophonicInstrument
SustainedInstrument <|-- FrettedInstrument
SustainedInstrument <|-- WrappedOctaveSustained
ToggledInstrument <|-- KeyedInstrument
MonophonicInstrument <|-- HandedInstrument
DecayedInstrument <|-- TwelveDrumOctave
DecayedInstrument <|-- OneDrumOctave
DecayedInstrument <|-- PercussionInstrument
PercussionInstrument <|-- NonDrumSetPercussion
class Instrument {
The superclass of all instruments, it handles
common logic:
- Location of instrument on stage
- Offset for stacking of instruments
- Visibility calculations
All instruments depend on the MidiNoteOnEvent.
}
class SustainedInstrument {
Instruments that depend on the
time of each note's MidiNoteOffEvent.
}
class DecayedInstrument {
Instruments that depend only on
the time of each note's MidiNoteOnEvent.
}
class ToggledInstrument {
Instruments that depend on both the
MidiNoteOnEvent and MidiNoteOffEvent,
but the instrument does not need to know
the time of the off event when the note starts.
}
class KeyedInstrument {
ToggledInstruments that use piano keys.
}
class MonophonicInstrument {
SustainedInstruments that can only
play one note at a time. Extension of the
Clone class provides common logic.
}
class HandedInstrument {
MonophonicInstruments that use hands
for animation.
}
class FrettedInstrument {
SustainedInstruments that display strings
and yellow fingers for animation.
}
class WrappedOctaveSustained {
SustainedInstruments that modulate note
values by 12, then apply their animation
to respective twelfths.
}
class TwelveDrumOctave {
DecayedInstruments that have twelve
drums, where each drum receives events
for notes modulated by 12.
}
class OneDrumOctave {
DecayedInstruments that have one drum,
and the drum is hit in twelve different
locations to represent the note modulated
by 12.
}
class PercussionInstrument {
DecayedInstruments whose events come
from a percussion channel.
}
class NonDrumSetPercussion {
PercussionInstruments that are not
attached to the drum set.
}
This diagram shows properties and methods in those classes.
classDiagram
direction LR
class DecayedInstrument {
+ calcVisibility(Double, Boolean) Boolean
+ tick(Double, Float) Unit
List~MidiNoteOnEvent~ hits
List~MidiNoteOnEvent~ hitsV
MidiNoteOnEvent? lastHit
}
class FrettedInstrument {
- animateString(Int, Int, Float, Float) Unit
+ toString() String
- fretToDistance(Int, Float) Float
- currentFret(Int) Int
+ tick(Double, Float) Unit
Spatial[][] lowerStrings
FrettedInstrumentPositioning positioning
VibratingStringAnimator[] animators
Spatial[] upperStrings
Spatial[] noteFingers
Map~NotePeriod_FretboardPosition~ notePeriodFretboardPosition
Float stringHeight
FrettingEngine frettingEngine
}
class HandedInstrument {
# moveForMultiChannel(Float) Unit
}
class Instrument {
+ calcVisibility(Double, Boolean) Boolean
+ tick(Double, Float) Unit
# similar() List~Instrument~
# updateInstrumentIndex(Float) Float
+ checkInstrumentIndex() Double
+ toString() String
# similarVisible() List~Instrument~
# debugProperty(String, String) String
# debugProperty(String, Float) String
# moveForMultiChannel(Float) Unit
Double visibility
Boolean visible
Node highestLevel
Node instrumentNode
Node offsetNode
Midis2jam2 context
}
class KeyedInstrument {
+ noteEnded(MidiNoteOffEvent) Unit
+ tick(Double, Float) Unit
+ noteStarted(MidiNoteOnEvent) Unit
+ keyCount() Int
+ toString() String
# keyByMidiNote(Int) Key?
Int rangeLow
Int rangeHigh
Key[] keys
EventCollector~MidiNoteEvent~ eventCollector
}
class MonophonicInstrument {
+ tick(Double, Float) Unit
+ toString() String
+ handlePitchBend(Double, Float) Unit
List~Clone~ clones
ClonePitchBendConfiguration pitchBendConfiguration
FingeringManager manager
PitchBendModulationController pitchBendModulationController
Node groupOfPolyphony
}
class NonDrumSetPercussion {
+ calcVisibility(Double, Boolean) Boolean
}
class OneDrumOctave {
+ tick(Double, Float) Unit
Node recoilNode
Striker[] strikers
}
class PercussionInstrument {
# moveForMultiChannel(Float) Unit
List~MidiNoteOnEvent~ hits
Node recoilNode
}
class SustainedInstrument {
+ tick(Double, Float) Unit
# calculateCurrentNotePeriods(Double) Unit
+ calcVisibility(Double, Boolean) Boolean
NotePeriodCollector notePeriodCollector
Set~NotePeriod~ currentNotePeriods
List~NotePeriod~ notePeriods
}
class ToggledInstrument {
+ noteStarted(MidiNoteOnEvent) Unit
+ noteEnded(MidiNoteOffEvent) Unit
+ toString() String
+ tick(Double, Float) Unit
+ calcVisibility(Double, Boolean) Boolean
EventCollector~MidiNoteEvent~ eventCollector
}
class TwelveDrumOctave {
+ tick(Double, Float) Unit
Node[] percussionNodes
Node[] offsetNodes
TwelfthOfOctaveDecayed[] twelfths
}
class WrappedOctaveSustained {
+ toString() String
+ tick(Double, Float) Unit
TwelfthOfOctave[] twelfths
}
DecayedInstrument --> Instrument
FrettedInstrument --> SustainedInstrument
HandedInstrument --> MonophonicInstrument
KeyedInstrument --> ToggledInstrument
MonophonicInstrument --> SustainedInstrument
NonDrumSetPercussion --> PercussionInstrument
OneDrumOctave --> DecayedInstrument
PercussionInstrument --> DecayedInstrument
SustainedInstrument --> Instrument
ToggledInstrument --> Instrument
TwelveDrumOctave --> DecayedInstrument
WrappedOctaveSustained --> SustainedInstrument
Procedural animation
The best way to create a new instrument is to reference an existing one. However, there are some components that are common to many instruments:
BellStretcher– Animates the bell of an instrument by stretching it based on the elapsed time of the NotePeriod.EventCollector– On each frame, it collects MIDI events that have elapsed since the last frame.FingeringManager– Handles the lookup of fingerings for an instrument.NotePeriodCollector– On each frame, it collects NotePeriods whose start time has elapsed.PercussionInstrument.recoilDrum(...)– Animates a drum recoiling.PitchBendModulationController– Handles the calculation of pitch bend and modulation effects.Striker– Provides common logic for objects that animate with a striking motion, such as drum sticks or mallets.VibratingStringAnimator– Animates vibrating strings, as seen on the guitar, violin, and others.
The jMonkeyEngine documentation is a good place to start for learning how to animate objects in jMonkeyEngine.
Registering the instrument
Register the instrument in the buildInstrument function, located in the InstrumentAssignment class.
Use existing instruments as a reference.