Line Momentum is a minimalist, fast-paced rhythm game with a dynamic soundtrack that evolves along the game. In this post, I will detail how I used Wwise with Unity in order to set this system into motion. It's my first time using Wwise in a project, and I learned quite a lot!

Naturally, I invite you to play the game if you want to hear that soundtrack in action!

The music in Line Momentum is made of four loops of 1 bar. Each bar represents one level, and is made of several layers of instruments. In a level, we will activate these layers one by one. The more times in a row the player validates a loop, the more instruments are playing. At the end of the level, all layers play together, and we can transition to the next bar!

Looping on only one bar can seem repetitive, and it is! The music genre of the Line Momentum soundtrack is minimalism, a style consisting of repeating short musical patterns. It's very efficient to convey moods. If you're curious, I recommend Music for 18 Musicians by Steve Reich, an incredible song that was a heavy inspiration for Line Momentum!

The music was composed in Waveform, a DAW with a pretty good UI, and which is, surprisingly, also available on Linux

Each track of each segment was exported in a separate file (most DAW offer a handful option for this). Note that the length of those exports was not 1 bar, but 2. See the blue markers on top of the screenshot? That's the exported segment. But why exporting 2 bars, one of them being completely silent, when we need to loop on only one? This is because we want to include the reverb of the instrument in the loops, and if we cut those right at the end of the bar, it won't sound great. We'll see later how to configure Wwise to loop only on the first bar.

But first, we need to configure the structure of the song! Let's open Wwise and setup the song's parts.

The first thing we create is a Switch Container called Music. This will transition from one level to another according to a State. It's also here that we setup the BPM and time signature of the song, since it is the upmost parent. We have to setup the transitions so that it only changes at the start of the next bar ("Start of Playlist"), and then manually associate each level with the different values of the newly created State called "Level". A State is like an enum, it's a variable that can have a value from a set of possibilities, configured in the Game Syncs in Wwise. The State "Level" indicates which is the current level, and will be updated from within the game.

Below Music are the Playlist Containers, each one representing a level. Playlist Containers in WWise allow to arrange several segments of a music (horizontal layering). However, in our case, we only need to loop on one single bar. So their configuration is rather simple: loop infinitely on the only child.

Next comes the interesting part: the Music Segment. A Music Segment is composed of several Music Tracks, each one playing on top of the other. This is where we will do our vertical layering! But first, we need to specify the start and the end of the segment, which will be our loop. By default, Wwise will use the longest track as length, we don't want that, we only need the first bar. We thus adjust the End Cue of the segment (the red line) to put it at the end of the first bar. Since we configured the BPM and the time signature, Wwise displays its exact position and even snap the cursor to the grid!

_Notice that beats numbering starts with "1". If you're a programmer, that's sure to confuse you!

This will make Wwise play the first bar on loop. But it will still play the part of the audio file that comes after (the reverb of the instruments), overlapping it with the loop that comes after! This makes each loop sounds natural. That feature alone is already a good reason to use Wwise for your music system. I created other game with dynamic soundtrack before, but had to come up with my own convoluted solutions to avoid the issue of "loop clicking", none of them being as powerful and easy to implement.

We need to do the layering now! Come back to the arborescence screenshot above, and notice how the tracks inside the level do not have the same symbol. This is because some are regular Tracks, and other are Switch Tracks. A Switch Track uses the value of a Switch to determine which file to play. A Switch is, well, kind of like a State, an enum. It's a set of values. What's the difference between the two of them, then? There are several, but the most important in my opinion is that a State is global to the entire game, while a Switch can be associated to a game object. Remember that Wwise is also useful for SFX, so it's not uncommon for a game to have several entities playing their own sound.

Another notable difference however, is that a Switch can be synchronized with an RTPC. An RTPC in Wwise is a numeric value that can be updated from the game (it has nothing to do with the RTCP protocol, I was very confused the first time I heard a sound designer mention that word). It can be used to change parameters in the sound relatively to a value in the game, such as making changes in the volume or reverb of a music track according to the distance between the player and an object, or their health. In Line Momentum, I use a RTPC called Progress to indicates how many "points" the player has in a level. It goes from 1 to 10: each time the player succeeds a loop, it gains 1, and when they fail, it goes back to 1 (again, I decided to start from 1 just to be consistent with Wwise numbering, but it could as well be 0). Then I created several Switches, one for each level, to configure which layer should be played according to the player progress.

With this system, I can configure each level differently to have their own layers! And the game won't have to do anything: just update the Progress value, Wwise will do the rest!

The downside of this approach is that for each Switch track, I have to register every Switch value where they have to play, and not just the layer from which they're activated. Maybe I could have done this differently using Progress on the volume of the track? But it would be more laborious to automatize, especially since the transition needs to be done only at the start of the bar and not immediately (just like in the Switch Container). Here it's a bit more verbose, but in the end it's faster and more easy to copy-paste the tracks and register a switch value.

_Piano plays for layers 4, 5 and 6 only. Also, even if there's several tracks, Wwise keeps only one copy of the original audio file, so no memory issue here.

The last thing missing now is an Event that will play the Switch Container Music... And we're done! We have a complete music system, that is fully interactive. On top of that, it's very easy to implement new sections. Just add a new State value for the level, then a new Segment in Music, import all tracks, create a Switch and sync it to progress, then assign the Switch values to the tracks. The harder part is composing the layers themselves, and then deciding in which order to introduce them.

And the beautiful part? We haven't written a single line of code! Now, this argument doesn't usually speak to me. I'm a developer, I enjoy coding! However, I have written musical systems such as this one using code in the past. And they can be very hard to debug. Because music is timed, you can't see your changes immediately, you have to play the track from the beginning, wait for the right event, be very attentive at the log to know what happened when... To set a position in a music track, you have to know its length, calculate the BPM, set the value in your code and cross finger when you launch the game. But in Wwise, not only the UI (as overwhelming as it is) makes those operations easier and faster (you just need to read the track, not listen to it), you also have powerful tools to debug! You can play any segment separately, and try transitions the moment you implemented them. You even have a Soundcaster, a configurable board that you can use to fully test a scenario. Here I can simulate events in the game and check how the music responds!

Note: Another useful debug tool is that, while working in Unity, you can connect Wwise to profile the sounds states and events. But you can also make some change in live, notably the playback speed! In a rhythm game it can be difficult to debug when everything goes so fast. But if you play the music at 0.1 speed, now you can see in slow motion what's happening.

Once this is exported as a Soundbank, it is quite easy to use in Unity. Wwise entities (Events, State, RTPCs...) must be set as public properties in a game object script. So in the script MusicManager, I have those lines:

(c#) public AK.Wwise.Event StartEvent; public AK.Wwise.RTPC Progress;

In the editor, I can now select the Event to play the music, and the RTPC Progress:

Now from the code, starting the music or updating the Progress can be done in one line:

(c#) // Don't worry about all those parameters, I'll explain them another time StartEvent.Post(gameObject, (uint)AkCallbackType.AK_MusicSyncAll | (uint)AkCallbackType.AK_EnableGetMusicPlayPosition, OnMusicEvent); // [...] // Like Switches, RTPC can have local values, but we only need a global one Progress.SetGlobalValue(value);

As for the level selection? Well, in a script terribly named GameManager, I have this serializable class that allows me to configure the levels. The most important thing to notice is that each one is associated with a State, corresponding to the State "Level" we created in Wwise.

```(c#) public class LevelParams { public LevelId id; public Color color; public Color barColor; public AK.Wwise.State state; public List beats; }

public List levels; ```

_Note that we don't register the State "Level", but a value of this State!

Then, when we prepare to change the level, we just have to do:

level.state.SetValue(); Progress.SetGlobalValue(1); Yes, this is how you change the value of a State with Wwise. level.state here is not the enum, but a value of this enum. SetValue() makes this value the current one of the global State.

I say "prepare", because this is one important thing, that is relevant for every rhythm game or dynamic soundtrack, no matter what tool you use: always schedule!. You never execute an audio change the moment it needs to be done, because otherwise it might be already too late. Remember, in Wwise, we configured every transitions so that they happen at the start of the loop (or at the end of the previous one, those are the same). Thus, we don't need to wait before updating progress or the level. As soon as the player did the correct input, we change those values so that on the next loop, the transition will be made! This also means that if another thing happens before (such as the player entering a wrong input), we have to cancel those actions to schedule another transition.

And there you go, a complete dynamic soundtrack! As you can see, even in Unity it's very easy to implement a new level (which I might do in a post-jam update).

However, how does the rhythm game part works? How do we read info from the music to know how to position the cursor, if the player pressed the button on time, or if they waited for too long? I'll detail this in a next devlog, focused on the rhythm game mechanics in Unity. I hope you enjoyed reading this one!

This is the second part of a devlog about Line Momentum, a rhythm game made for the Ludum Dare 51. The first part was about how the dynamic music system was implemented using Wwise. In this one, we'll see how to use scripts to keep track of the position in the music, and validate players input according to the rhythm. Of course, you're more than welcome to play the game.

So far, we've been able to send events to Wwise in order to play the music, and make it evolve with the game. But now we need the other way around: we need information coming from Wwise in order to be alerted when a beat or a new bar happen. And at each frame, we need to know with precision where we are in the track.

This is initialized at the line where we send the event to start the music. Let's write it step by step. The most basic usage is just to send the event (named StartEvent) to Wwise.

All scripts in this article are from several Game Objects. For the sake of simplicity, I won't describe their names, structure or relations. Just consider that all variables and functions declared are available to any scripts.

(c#) StartEvent.Post(gameObject); An event is always linked to a Game Object, hence why we gives the instance as the first argument.

Now we need a callback for each "music sync" events: beats, bar, etc. This is done by specifying two more arguments to the function: the type of events we want to be triggered for, and the callback function to call. For the first one, while we could be called only for bars, it's better to specify all kind of events related to music. We'll see later how to read the event's type. The next argument is a new function that we'll call OnMusicEvent, which we'll describe below.

(c#) StartEvent.Post(gameObject, (uint)AkCallbackType.AK_MusicSyncAll, OnMusicEvent);

Wait, why is the CallbackType parsed as an uint? Well, good question actually. The Post method only accepts uint, and the constant provided by Wwise happens to be a int, therefore it must be converted. Why isn't it directly the right type? I don't know, but I'm sure there must be a logical reason. For now, remember that it must be implemented that way.

Now, how does our OnMusicEvent look like?

(c#) void OnMusicEvent(object in_cookie, AkCallbackType in_type, AkCallbackInfo in_info) {

• The first argument, in_cookie, are additional data sent with the original event. We don't use it at all here.
• The second one in_type is the type of the event. It's with it that we'll know if the event triggered is a beat, a bar, or something else.
• Finally in_info provides useful data about the music track itself, such as the position in seconds, or the length of a bar.

In Line Momentum, we only react to the bar events. But know that there are many more that can be useful! For example, you could have animations triggered on the beats. It's even possible to have a time unit smaller than the beat, called "grid" in Wwise, which can be configured to be aligned with quarter notes, eight notes, or others fragments.

The best pattern to sync actions from your game with music events is to use UnityEvents. A Unity Event will have several object's methods subscribed to them, and will call all of them when it is triggered. This is super useful for code organization, because it means that your objects are not dependent between each other, and can easily be plugged and unplugged to events. Personally, I even like to use a Scriptable Object as a Store with UnityEvents that any Script can subscribe to. I bet you can also scale even better with plugins such as UniRx.

But for the sake of this article, let's keep it simple, and just declare our events locally. In the music event callback, we will check the type of the event, then invoke the corresponding Unity Event.

```(c#) public UnityEvent onBar = new UnityEvent(); public UnityEvent onBeat = new UnityEvent();

void OnMusicEvent(object incookie, AkCallbackType intype, AkCallbackInfo ininfo) { if (ininfo is AkMusicSyncCallbackInfo) { if (intype is AkCallbackType.AKMusicSyncBar) { onBar.Invoke(); } else if (intype is AkCallbackType.AKMusicSyncBeat) { onBeat.Invoke(); } } } ```

Now any function can be subscribed to beat or bar events! Either from the editor GUI, or through code (using onBar.AddListener).

That's not enough though! Reacting on beats is one thing, but we still need to know at every frame what is the position in the bar! There are several use cases for it, but our first one will be to update the position of the visual indicator.

Sure, we could use an animation or a tween for this, and make sure that it's approximately the same time as the bar. But there's a non neglectable risk that it falls out of sync eventually. Depending of the nature of your game, you might need to handle longer durations than a single bar, and there are cases where you want to be precise. You're never immune to a small lag or a music interruption. Thus your timing must always comes from the music data. Never use timers or scripted animations, unless it's not for a critical purpose (such as a short visual animation). For Line Momentum, the indicator is here to help the player get the right timing, so it must be precise!

_(Plus, doing this way means that you can pause the music and still keep everything in sync with no worries!)

Let's get back to the post of the StartMusic event! We need to track more than beats and bar. We want to be able to read directly the time position at any time. For this, we can pass the AK_EnableGetMusicPlayPosition constant as a second argument. Since we already pass AK_MusicSyncAll, we have to use a Bitwise OR (|) between the two. This option will allow us to track the music position from the player. However, we need a player ID for this. We must thus retrieve it from the Post result.

```(c#) private uint PlayerID;

// [...] When starting the music: PlayerID = StartEvent.Post(gameObject, (uint)AkCallbackType.AKMusicSyncAll | (uint)AkCallbackType.AKEnableGetMusicPlayPosition, OnMusicEvent); ```

Now, in the FixedUpdate, we can read the position in the bar by creating a SegmentInfo! For the reference, it is the same type we receive as in_info from our callback. Only this time, it's available at any time in the music.

(c#) private void FixedUpdate() { AkSegmentInfo segmentInfo = new AkSegmentInfo(); AkSoundEngine.GetPlayingSegmentInfo(PlayerID, segmentInfo, true); print(segmentInfo.iCurrentPosition); // --> position in ms }

The i in iCurrentPosition stands for int. There are other measurements in SegmentInfo that are prefixed with f, for float. Thus it indicates that it is not in milliseconds, but in seconds! This is important, because you'll want to choose one single unit when comparing values. And unfortunately, some values are only available in milliseconds, others only in seconds. So remember to convert them when needed!

Okay we have the time position in seconds now, but it's not super useful as it is. If we want to position the dot on the line, we need a value going from 0 to 1, so that we we'll never have to worry about the size of the line in pixels. We need to know the duration of a bar. Now that's something we could actually calculate by hand, using the BPM and the time signature. But what if those change during the development of the game? Your composer can have a change of mind after all. Or you could even have several tracks, with different tempo! It's better to just read them from Wwise data. While we're at it, we will also register the duration of a beat, because this data will be very useful.

We can do that in our Callback function! It will be triggered at the very start of the song for the first bar. Note that we only need to write those values once.

(c#) void OnMusicEvent(object in_cookie, AkCallbackType in_type, AkCallbackInfo in_info) { if (in_info is AkMusicSyncCallbackInfo) { AkMusicSyncCallbackInfo musicInfo = (AkMusicSyncCallbackInfo )in_info; if (in_type is AkCallbackType.AK_MusicSyncBar) { onBar.Invoke(); if (setDurations) { barDuration = musicInfo.segmentInfo_fBarDuration; beatDuration = musicInfo.segmentInfo_fGridDuration; // In Line Momentum, I used quarter notes as a "beat" setDuration = false } } // [...] } }

In FixedUpdate, we can now register the position in the bar for the dot indicator to use:

(c#) float fCurrentPosition = (float)segmentInfo.iCurrentPosition / 1000f barPosition = fCurrentPosition / barDuration

Alright, now let's tackle the big section: player interaction. The system in Line Momentum is quite simple: there are 10 beats (quarter notes) in a bar, the player must click on a selection of them. For example, on the first level, the player must click on beats number 2, 6 and 8 (beats going from 0 to 9). On level 2 it's 1, 3, 4, 6, and 8. And so on for other levels. Let's say those values are in an array called levelBeats. If the player correctly clicks on all the beats, they succeeds, if not, they fail.

Now I won't describe the whole logic of the game, because there's just too much. Instead, I'll focus on the rhythm logic that can be used for any kind of rhythm game: how to read input from the player, and also how to detect when they miss.

The first thing we need to do is to prepare the next beat that must be hit. Remember: always schedule! So, when the bar starts, we must setup the position (in seconds) of the next beat in the bar. This is calculated with the duration of a beat that we initialized earlier.

```(c#) private uint nextBeatIndex; private float nextBeatPosition;

void onBarStart() { updateNextBeatPosition(0) }

void updateNextBeatPosition(uint index) { nextBeatIndex = index; nextBeatPosition = levelBeats[index] * beatDuration; } ```

This is not the only way to write this value. Another approach is to use relative times, especially useful if you deal with long segments, where time can create delay between the theoretical beat position and their actual value. Basically it consist of reading the current position on a beat event callback, then setup the position with nextBeatPosition = currentPosition + (nbBeatsRemaining * beatDuration).

We now know exactly when the player should hit for the next beat. So when they will trigger an input, we will compare the current time to that value, which will determine if they hit correctly. However, it's absolutely impossible for players to be that precise! We need a margin error. It's a small time duration before and after the beat in which we consider the timing as still valid. It means that the time the player has to react will be twice the margin. With that in mind, let's write a function to compare if the current time is approximately the desired one:

```(c#) public float Precision = 0.08f; // Gives a 0.16 time of reaction

void isCloseTo(float time) { return Math.Abs(time - currentTime) < Precision } ```

But wait, there's a catch! Suppose that the next beat to validate is at position 0 (the very first beat). Since the bar is looping, what will happen if the player hits just before the end? That should be validated, because we are close enough to the start of the next bar. But our current algorithm will return false, because the time between 0 and the very end of the bar is, well, way higher than 0.08! I'd like to present you a one-line formula to solve this problem. If you have an idea, please share it. But as of today, we have to handle this special case.

```(c#) public bool IsCloseTo(float time) { if (Math.Abs(time - currentTime) < Precision) { return true; }

// Time is at 0
if (time < Precision) {
    return Math.Abs(barDuration - currentTime) < Precision;
}
// Time is at the end of the bar
else if ((barDuration - time) < Precision) {
   return currentTime < Precision;
}
return false;

} ```

Alright! Using this, we can now register players' input:

(c#) if (Input.GetMouseButtonDown(0) { if (IsCloseTo(nextBeatPosition)) { onValidateBeat(); // Do whatever you need to do when the player scores updateNextBeatPosition(nextBeatIndex + 1); } else if (!IsCloseTo(0f)) // Fail only if we're not close to the end of the bar { onFail(); // Register a failure } }

We're now able to detect when the player succeeds! But not when they fail. There are two ways for a player to miss a beat:

• They clicked too soon. This is what we have already implemented: when they are not close to a targeted beat, we register the failure.
• They clicked too late, or didn't click at all. This is what we need to implement now.

To register that a beat have been missed, we need a new function similar to IsCloseTo, to know if a time is considered as passed (that is, after the error margin).

(c#) public bool IsPassed(float time) { return currentTime > time && (currentTime - time) > Precision; }

Simple enough. But... Remember how we had issues with the start and end of the loop? Well, it's the same here. But it's mostly because of Wwise this time. As counter intuitive as it seems, currentTimer can sometime be above barDuration! But it's supposed to loop on the bar though, right? Yes, but timing in music is not an exact science. But the worse is: this can happen after the new bar event! So when we reach a new loop, after we have initialized our first target beat, there are few frames where the current time is not reset at 0, but instead slightly above the loop length. This completely breaks our calculation (because we consider being at the start of a new loop, and currentTime is still at the end). So, for this special case, the best is just to ignore the curentTime if it's near the end. Because it means that we're at the beginning, and thus no time could possibly be passed!

(c#) public bool IsPassed(float time) { return Math.Abs(barDuration - currentTime) > Precision && currentTime > time && (currentTime - time) > Precision; }

We are now able to detect when a player miss a beat! We just need to do the check in a FixedUpdate:

(c#) void FixedUpdate() { if (IsPassed(nextBeatPosition)) { onMiss(); updateNextBeatPosition(nextBeatIndex + 1); } } }

And there it is, you now have the basics to build a rhythm game! You know how to subscribe to music events, how to read the exact position in the track, how to validate a player input, and how to detect when a beat has been missed.

Naturally, there is more to it. I spared you the part of the logic that are inherent to the game's rules. There will always be exceptions, special cases to check, values to update at only precise moments, and many flags to know in which state we are. The code shared in this article are only the very basic bricks for building a rhythm game.

One last advice I could give is to use flags and Unity Events to schedule actions on timed events only. In line Momentum, there are a lot of visual updates that only happen at the start of the bar. So instead of calling an update function immediately, I usually schedule it so that it happens on the next bar.

```(c#) private bool doStuffOnNextBar = false;

void OnStart() { MusicManager.onBar.AddListener(BarTrigger) }

void OnThingHappen() { doStuffOnNextBar = true; }

void BarTrigger() { if (doStuffOnNextBar) { doStuffOnNextBar = false; DoStuff(); } } ```

Bonus exercise: you can probably use IEnumerator to yield instructions until the next bar or beat! I didn't have to do that in Line Momentum, but I had some logics like this in Godot games. Trust me, it offers really clean code!

If you're interested in the logic behind of Line Momentum, it is now available in open-source! You can check out its code, or open it with Unity or Wwise and see how it is structured.

I hope this guide was helpful to you. Those information, while available in various places in the Wwise documentation, are certainly the ones I wish I had when working for this game! Making rhythm games is tricky, but not too difficult if you have the right tools, and know how to proceed.

Here comes a more polished version of Line Momentum! This update aims to be more accessible to every players, and smoothen some aspects for a more pleasant experience. If you had trouble completing the game (or even the first level), this version should be way more enjoyable.

• Less punitive difficulty: Losing no longer makes you start from the beginning
• More available controls schemes: arrow keys and gamepads
• Adaptive resolution: the game plays now in your native resolution, and you can even resize its window
• Accessibility mode allows you to keep the visual help
• Smoother transitions between levels
• Reduced the input lag on Linux (and hopefully other systems)
• Fixed some bugs (one on level 4 notably)

I hope you will have fun playing this version of the game! You can find it on itch.io.

As for those who desire more challenge (or who liked the high stakes of the original), don't worry, I have a little something coming for you. The next update will be focused on new content, with a very special unlockable mode for those who aim for the perfect. Be patient, it will come probably in December, or early next year.