Editing Creating cave music

Randomly generating cave music in Pikmin 2 needs to have two files: a .bms (sequence) file, and a .cnd (conductor) file.

The BMS file type is a sequenced music format used in many games. For the cave music in Pikmin 2 however, it is specially-formatted. It mainly contains many short snippets of musical information to be randomized.

The CND files are the actual "songs"; they take specific groups of snippets from a BMS file, and endlessly puzzle-piece them together to form a coherent piece of music.

This guide will function as a walkthrough for creating your own randomly sequencing cave song. All of the editing will be done in hexadecimal.

''Requirements:''
* A DAW to create your own short MIDIs in, like FL Studio or Logic Pro. A DAW is also great for sketching out your song ideas/song's structure beforehand.
* A MIDI to BMS converter, like [[Pikmin 2 custom music|flaaffy]]. You will be generating a lot of tiny BMS files, to turn them into "gestures".
* A basic understanding of hexadecimal, and a hex editor. An editor that can dynamically resize the window is ideal (HxD on Windows, or Hex Fiend on MacOS are fantastic examples).

Basic understanding of how [https://youtube.com/watch?v=QPFWzSlg-78 cave music works] and the [[Creating Interactive BMS Files|standard interactive music]] is recommended before starting.<br>Additionally, studying the existing cave songs on your own and how those exploit the conductor system in interesting ways can greatly help you come up with ideas for your own custom music.

== Overview of file functions ==
Here are some infographics that you can reference at any time. These show all the functions of a CND file, and all the contents of a BMS file.
<gallery mode="packed-hover" widths=250px heights=250px>
File:CND_functions.png|Conductor file functions.
File:BMS_functions.png|How information is stored in a BMS file.
</gallery>

== Setting up your BMS file ==
Before you begin, set your byte grouping to "3", so every 3 bytes is visually separated by a space. This will make it easier to see stuff.<br><br>
Starting with a completely blank BMS file, begin by pasting the following:
<source lang="powershell">
AC1000 00AC11 016ECC 0210CC 0311AC 100000 AC1101 8FCC04 10CC05 11E703 00A000 03A401 00A300 01C801 00003C B1C180 010000 7FA501 01C800 000025 FE0078 C40000 0058C6 00E70E 00DF07 00006A E72E00 CF03C8 000000 4EE70E 00DF07 00006A E72E00 CF03C8 000000 60E71E 00E3E7 0000E7 0F00DF 070000 7BC600 E71F00 E3C400 00006E C40000 0155E7 0600CF 03E708 00CF03 CB0306 E70900 A00003 A70002 C80100 0096A7 0001C8 010000 8EA700 03C401 0000BE C40000 00C2C8 000000 96DA00 C600E7 0A00E7 0C00A0 0003A7 0000C8 010000 F0A700 01C801 000104 A70002 C80100 00FAA7 0003C8 010001 0EC800 000109 C40000 0118C8 000001 09C400 000118 C80000 00C5C4 000001 42E70B 00C600 C40000 0142C8 000000 C5CB06 10CB07 11CB08 04A005 04A93C 058000 A70500 C40100 014EA9 3C047F FFCF04 E70D00 C40100 00BEC6 00CB08 04A93C 047FFF CF04C6 00B1C1 400028 C600DB 04EF61 6161C6 00DB00 A40801 A40900 A40A01 EF3421 34C600
</source>
Every cave music BMS begins with this same information, and will go mostly untouched.
<br>
<br>

== Gesture set list ==
Exactly right after <code>34C600</code> will be a list of every gesture set's location in the BMS. Each entry is the gesture set's hexadecimal location (offset), and each entry in the sequence begins with "<code>00</code>". <code>FFFFFF</code> indicates the end of this list.<br><br>
Here is an example of how this sequence should be formatted:<br>
<code>0001B0 0001BF 0001CE 0001D4 0001DA 0001E9 0001F8 0001FE 000204 00020A FFFFFF</code><br>
Gesture set "0" will be at offset <code>0x1B0</code>, gesture set "1" will be at offset <code>0x1BF</code>, gesture set "2" will be at <code>0x1CE</code>, and so on. There are a total of 10 gesture sets listed in the above sequence.
<br><br>
Until we create our gesture sets, let's use <code>00AAAA</code> as placeholders so we can easily find and get back to this section later on. You can have as many gesture sets as you like; we'll create 16 entries for this tutorial. You may insert this into your BMS:<br>
<code>00AAAA 00AAAA 00AAAA 00AAAA 00AAAA 00AAAA 00AAAA 00AAAA 00AAAA 00AAAA 00AAAA 00AAAA 00AAAA 00AAAA 00AAAA 00AAAA FFFFFF</code><br>
<br>
Our BMS should now currently look like this:<br>
<source lang="powershell">
AC1000 00AC11 016ECC 0210CC 0311AC 100000 AC1101 8FCC04 10CC05 11E703 00A000 03A401 00A300 01C801 00003C B1C180 010000 7FA501 01C800 000025 FE0078 C40000 0058C6 00E70E 00DF07 00006A E72E00 CF03C8 000000 4EE70E 00DF07 00006A E72E00 CF03C8 000000 60E71E 00E3E7 0000E7 0F00DF 070000 7BC600 E71F00 E3C400 00006E C40000 0155E7 0600CF 03E708 00CF03 CB0306 E70900 A00003 A70002 C80100 0096A7 0001C8 010000 8EA700 03C401 0000BE C40000 00C2C8 000000 96DA00 C600E7 0A00E7 0C00A0 0003A7 0000C8 010000 F0A700 01C801 000104 A70002 C80100 00FAA7 0003C8 010001 0EC800 000109 C40000 0118C8 000001 09C400 000118 C80000 00C5C4 000001 42E70B 00C600 C40000 0142C8 000000 C5CB06 10CB07 11CB08 04A005 04A93C 058000 A70500 C40100 014EA9 3C047F FFCF04 E70D00 C40100 00BEC6 00CB08 04A93C 047FFF CF04C6 00B1C1 400028 C600DB 04EF61 6161C6 00DB00 A40801 A40900 A40A01 EF3421 34C600 00AAAA 00AAAA 00AAAA 00AAAA 00AAAA 00AAAA 00AAAA 00AAAA 00AAAA 00AAAA 00AAAA 00AAAA 00AAAA 00AAAA 00AAAA 00AAAA FFFFFF 000000 000000 000000 000000 000000 000000 000000 000000 000000 000000 000000 000000 000000 000000 000000 000000 000000 000000 000000 000000 000000
</source>
You can also add an entire row of padding/zeros after this. You can do this for each section, to better visually separate them.
<br>

== Timing ruleset list ==
This is another sequence of offsets. These just point to the location of a "[[#Inserting_timetables|timetable]]", which will be located at the end of our BMS file. <code>FFFFFF</code> indicates the end of this list.<br><br>
Here is an example of how this sequence should be formatted:<br>
<code>0005AB 0005AB 0005AF 0005C1 0005AB 0005AB 0005C1 0005C1 0005C1 0005C1 FFFFFF</code><br>
Timing ruleset "0" will use the timetable at <code>0x5AB</code>, timing ruleset "1" will also use the timetable at <code>0x5AB</code>, timing ruleset "2" will use the timetable at <code>0x5AF</code>, and so on. There are a total of 10 timing rulesets listed in the above sequence.
<br><br>
Let's use <code>00BBBB</code> as placeholders again, and create 16 of them. Insert this anywhere into your BMS:<br>
<code>00BBBB 00BBBB 00BBBB 00BBBB 00BBBB 00BBBB 00BBBB 00BBBB 00BBBB 00BBBB 00BBBB 00BBBB 00BBBB 00BBBB 00BBBB 00BBBB FFFFFF</code><br>
<br>

== Gesture set contents ==
A ''gesture set'' is a group of many gestures.<br>
Here, we list each gesture set's contents (which will be... the locations of the gestures!). Each gesture set begins before with <code>00</code> and ends with <code>FFFFFF</code>.<br><br>
Here is an example of how the gesture set list should be formatted:<br>
<code>000210 00022A 000258 000281 FFFFFF 0002B4 0002D9 000303 000331 FFFFFF 000355 FFFFFF 000361 FFFFFF 0003A8 0003C2 0003F0 000419 FFFFFF 00044C 000471 00049B 0004C9 FFFFFF 0004ED FFFFFF 000543 FFFFFF 000580 FFFFFF 00059E FFFFFF</code><br>
There are 10 gesture sets listed above. The first gesture set has 4 gestures, located at <code>0x210</code>, <code>0x22A</code>, <code>0x258</code>, and <code>0x281</code>. You can have any amount of gestures in a gesture set; even 1 is fine.
<br><br>
You can paste the following placeholder information for now:<br>
<code>00CCCC 00CCCC 00CCCC FFFFFF 00CCCC 00CCCC 00CCCC FFFFFF 00CCCC 00CCCC 00CCCC FFFFFF 00CCCC FFFFFF</code><br>
It might be a good idea to leave a decent amount of zeros after this list; this one will expand a lot as gestures and gesture sets are added to it.
<br><br>

== How is your BMS looking? ==
If you have been copy-pasting everything correctly, your BMS will look something like this:
<gallery mode="packed-hover" widths=300px heights=300px>
File:BMS_file_progress_1.png|Orange is "gesture set list", yellow is "timing ruleset list", and blue is "gesture set contents"
</gallery>
Once again, you can add as much padding between each section as you want, it will work fine.

== Creating gestures ==
First, you should sketch out your song idea in your DAW, and then cut your song into tiny blocks. These blocks will be your song's individual gestures. You should also consider dragging these blocks around in various arrangements to make sure they sound good in any order.<br><br>
Once you are ready, have them nicely organized and ready to be exported into individual MIDI files:
<gallery mode="packed-hover" widths=400px heights=400px>
File:Logic_Pro_gestures.png|A Logic Pro project with a bunch of short blocks of music. Each block will be a "gesture".
</gallery>
For some reason, imported BMS files play 4 times as slow in cave songs. So if you import these gestures as is, they will play 4 times as slow.<br>
Before you export your gestures, you need to modify your gestures to run ''4 times as fast''. How you will go about this will depend on your DAW; some will allow you to stretch entire regions of notes easily.<br><br>
Logic Pro can easily speed up MIDI regions with MIDI Transform. You want to use the "Double Speed" transformation twice on each block.
<gallery mode="packed-hover" widths=400px heights=400px>
File:Logic_pro_double_speed.gif|Shorten the length of the MIDI by 4x, but don't change your project's tempo.
</gallery>

== Inserting gestures ==
Gestures use same the note/music information as standard BMS songs.<br>

You should have converted all your MIDIs into BMS files. Open up one of your converted BMS files, and look for the first note/pitch value. If you used [[Pikmin 2 custom music|flaaffy]], this will most likely be somewhere after <code>0x1B</code>, <code>0x1C</code>, or <code>0x1D</code>. The note's pitch value will often be followed by a <code>01</code>, then the velocity value. Copy this note, all the way to the end of the sequence, which should be <code>FF</code>.
<gallery mode="packed-hover" widths=600px>
File:Copy_pasting_BMS_info.png|The value "2D" here is the hex value of a certain pitch, and the very first note of the sequence. Copy this, all the way to "FF".
</gallery>
To add your gesture, start it with <code>C40000015D</code>, then paste in what you copied. Each gesture should end with that <code>FF</code>. Add as many gestures as you please!
<gallery mode="packed-hover" widths=400px heights=400px>
File:BMS_file_progress_2.png|The purple lines indicate where a gesture starts.
</gallery>

== Inserting timetables ==
So, how do we choose how often gestures will play? Will a gesture play every beat? Every 1/2 a beat? You will need to add a timetable for that!
A timetable is a sequence that dictates when gestures should play. A timetable ends with <code>FFFF</code>. You can make a timetable as complex as you like.<br> Some examples of timetables you can use are:<br>
* <code>0780FFFF</code> Play a gesture every 16 beats.<br>
* <code>03C0FFFF</code> Play a gesture every 8 beats.<br>
* <code>01E0FFFF</code> Play a gesture every 4 beats.<br>
* <code>03C003C003C001E001E001E001E003C0FFFF</code> This sequence will play a gesture every 8 beats (3x), play a gesture every 4 beats (4x), then play a gesture for 8 beats (1x), in that order.<br>
* <code>00F0FFFF</code> Play a gesture every 1/2 note (every 2 beats).<br>
* <code>0078FFFF</code> Play a gesture every 1/4 note (every beat).<br>
* <code>003CFFFF</code> Play a gesture every 1/8th note.<br>
* <code>001EFFFF</code> Play a gesture every 1/16th note.<br>
* <code>003C0078001EFFFF</code> This sequence will play a gesture for an 1/8th note, play a gesture for a 1/4 note, then play a gesture for a 1/16th note, in that order.<br>
Considering that <code>001E</code> (30 in decimal) is a 1/16th note, you can create any specific timing you want. Just add another 30 to the value for each additional sixteenth note in length.<br>
For example, Soil 1's enemy mix piano plays very rapidly; its timetable uses a value of 90, which looks like <code>005AFFFF</code>. The gestures last for three sixteenth notes.<br>
You can insert your timetables anywhere at the end of your BMS.
<gallery mode="packed-hover" widths=250px heights=250px>
File:BMS_file_progress_3.png|Each timetable has a pink box around it.
</gallery>

== Filling in info ==
Once you have your gestures and timetables inserted, go back and fill in those <code>00AAAA</code>, <code>00BBBB</code>, <code>00CCCC</code> placeholders with the locations of your gesture sets, timetables, and gestures you want to use. Remember that gesture sets begin with the <code>00</code>.

Also in the very beginning of your BMS, make sure the value at <code>0x6</code> has the location of where your gesture set list begins (this one will probably be fine), and the value at <code>0x14</code> has the location of where your timing ruleset list begins.
<gallery mode="packed-hover" widths=250px heights=250px>
File:BMS_file_progress_4.png|Put the location of where the "timing ruleset list" begins here.
</gallery>

== Filling in the conductor file ==
<gallery mode="packed-hover" widths=500px heights=500px>
File:CND_functions.png|Conductor file functions.
</gallery>
In your song's associated conductor file, try to stretch out the window so that the instrument names are all in the same column if you can, like in the image above.<br><br>
Each line represents one track. You give each track a instrument (program), and what gesture set and timing ruleset to use.<br>
The CND infographic above does a good job explaining all the functions, but they'll be explained again down here.<br><br>

In the top left corner are three important values:
* The first value is the swing time. <code>1E</code> is 1/16th swing, while <code>3C</code> is 1/8th swing. If your song is below 100bpm, you might want to use <code>1E</code>. If its faster, try <code>3C</code> instead.
* The second value is the tempo, in hex.
* The third value is how many tracks there will be, in hex.

Each column explained:
* A: Initial delay. How many beats until this track starts running. In the vanilla game, you can notice how some song's melodies do not immediately begin playing upon the start of level. This is useful for that; ease the listener into the music.
* B/Q: If set to 1, all gestures will be delayed by an 1/8th note..
* C: Text string. You can make this say whatever you like. Useful for keeping track of what you want each track to be, like "Lead", "Trumpet", "MelodyA"/"MelodyB", "Bass", or "Drums"
* D: Track copy. Select a previous track # to copy the gestures from. If the track its copying plays "gesture 3" from its gesture set, this track will also play its "gesture 3". Often used for treasure mix instruments. You can have them use different gesture sets if you like.
* E: Echo strength.
* F: If set to 1, the values in this track's timetable will always be played in order, in a loop. If set to 0, it will randomly select values from its timetable, thus, allowing you to randomize when gestures play.
* G: Bank number.
* H: Program number. For a list of all programs in the default game and the wScenes that support them, view [[Pikmin 2 custom_music/available instruments|this page]].
* I: Gesture set, select the set # you want to use from the BMS. 
* J: Timing ruleset, select the set # you want to use from the BMS.
* K: How many gestures this track will play, before playing "silent gestures". Linked to below. These function as a sort of on/off cycle.
* L: How many "silent gestures" this track will play, before playing normal gestures. Linked to above. These function as a sort of on/off cycle.
* M: If notes are being transposed down (see below), this is set to FF. But this column doesn't actually do anything, so you don't need to edit this.
* N: Transpose all notes played by this track. To transpose up, use values 1-127. To transpose down, use the sign 2 compliment (values 128-255).
* O: Volume.
* P: Stereo panning.
<br>
Some ideas/considerations:
* You can use the on/off cycle feature (column K and L) to make your melodic instrument "take a break" every once in a while. You can have it play 4 gestures, but then be silent for 2.
* If your timing ruleset is strict (as in, the timings for when gestures occur is always consistent), something fun to try is to have the melody be a back-and-forth between two different instruments. Set the initial delay for the second instrument to be longer than the first one, and then cleverly use the on/off cycle to have the instruments take turns sounding a gesture. This is done for the melody in the Bulblax Kingdom and Frontier Cavern for example.
* If you are using the "track copy" feature (column D), you can have the primary instrument play a melody, and have this instrument always play chords that perfectly match that melody.

== Mapping the tracks ==
With all that done, don't forget to edit the song's [[Trackmap parameters|trackMap]]! Edit your song in the trackMap files in either <code>/user/Totaka</code> or <code>/user/Wakai</code>, and set how many instruments will be in each mix. It goes in order from: Main Mix, Task (event), Treasure (otakara), Enemy near (kehai), Combat mix, and Spiderwort mix (ground). Although you cannot find Spiderworts underground, you will have to sacrifice at least one track for that mix. You cannot set any of the mixes to have 0 instruments in them.

== Bonus: Filenames for brand new CND files ==
If you are adding a brand new CND file instead of replacing an existing one, the prefix will determine which BMS is loaded, and which wScene. Try not to add special characters to filenames, aside from underscores.<br>

In order to load the correct BMS file, the CND must start with the name of the BMS file.
* "caveconc.cnd", "caveconc_458435.cnd", "caveconc_Waffles.cnd", and "caveconc_Waffles_Song_2.cnd" will all load caveconc.bms.<br>
* "caverelax.cnd", "caverelax_Gmajor.cnd", and "caverelax_mamuta_dinner.cnd" will all load caverelax.bms.<br>
* "new_09_4.cnd", "new_09_4000.cnd", and "new_09_portal_PotaTOS.cnd" will load new_09.bms.<br>

For cavesoil, caveconc, and cavemetal only, the game also checks for an additional "_00", "_05", and "_10" right after the prefix, for the purpose of loading one of three different wScenes. You must include it if you want the correct wScene to load. For example, caveconc_05 loads wScene 31, so if you want your custom song to also load wScene 31, acceptable names include "caveconc_05_WAFFLES.cnd", "caveconc_05_52_46356.cnd", or "caveconc_05_super_boss.cnd".<br>

After adding your new CND file, be sure to add it to its respective trackMap file.

== Bonus: Adding new content to a vanilla BMS file ==
Adding new hex data in the middle of a file will shift all the offsets that come after it, causing your BMS to jump to the wrong locations, and likely cause a crash if vanilla songs are loaded.<br>
In order to add new content in addition to what's already in the BMS:
* Find and copy the gesture set list and the timing ruleset list, and paste them at the end of the file. (Separate them with lots of zeros so you have lots more space to add content to these lists)
* Change <code>0x6</code> and <code>0x14</code> to point to your new gesture set list and timing ruleset list. These two values tell the game where the lists are.
The game should now use these brand new lists, instead of the original instances that occur earlier in the file. Since they still contain all the locations of the vanilla gesture sets/timetables from above in the same order, the existing CND songs won't be affected and will continue to function, but now you can expand upon these lists with brand new entries now that you've made the space for them. All new gestures, gesture sets and timetables can be added further down the file.
<gallery mode="packed-hover" widths=400px heights=400px>
File:Editing_vanilla_BMS_file.png|The new gesture set list and timing ruleset list we want to use will be at the bottom.
</gallery>

== Bonus: Adding gesture cutoffs in timetables ==
An intentional compositional choice in some songs is to cutoff gestures while they're in the middle of playing. The most common way this is done is to quickly trigger another gesture; if the last gesture has not finished playing, it will stop playing that gesture and begin playing the new one. It's also possible to add "empty" gestures to gesture sets (like <code>C40000015DFF</code>, a gesture which ends immediately and plays nothing), so you can randomly interrupt gestures in your song with silence, or have there be a chance to play nothing.<br><br>
However, if you wish to interrupt gestures in your song at specified times without the randomness, there is a special function in timetables for doing this. In the vanilla game, this is used for the trumpet in [https://www.youtube.com/watch?v=qOhmS1YLpF8 Fire]. The trumpet plays two interrupted gestures, then plays a third gesture in full. Below is the timetable for this trumpet:<br> 
<code>0078 8168 81E0 00F0 80F0 81E0 0780 FFFF</code><br>
The trumpet will:
* <code>0078</code> play a gesture for 1 beat
* <code>8168</code> cutoff after this 1 beat
* <code>81E0</code> extend the silence for another 4 beats
* <code>00F0</code> play a gesture for 2 beats
* <code>80F0</code> cutoff after these 2 beats
* <code>81E0</code> extend the silence for another 4 beats
* <code>0780</code> play a gesture for 16 beats.<br><br>
In order to put a "cutoff" in your timetable:
* Start with how long you want your "playing" gesture to play for, before it gets cutoff. I want the the first gesture to last for one beat (four 1/16th notes), so I'll put <code>0078</code>. Remember that every "30" in decimal is equivalent to a sixteenth note.
* Immediately next in the sequence will be a value that indicates when the cutoff begins. Calculate the difference between "81E0" and the previous value you had just inserted. 81E0 - 0078 = 8168 (in decimal, 33248 - 120 = 33128). I will put <code>8168</code> next in my sequence.
My sequence is now <code>0078 8168 FFFF</code>; this will play a gesture normally for 1 beat, but then immediately cut it off after that 1 beat.<br>
Putting an additional <code>81E0</code> in the sequence right after will extend the silence for another 4 beats if you so desire. You can keep putting <code>81E0</code> right after several times in a row if you want silence to last extra long until the next gesture plays.

[[Category:Pikmin 2]]