This is the second part of a devlog about Line Momentum, a rhythm game made for the Ludum Dare 51. The first part was about how the dynamic music system was implemented using Wwise. In this one, we'll see how to use scripts to keep track of the position in the music, and validate players input according to the rhythm. Of course, you're more than welcome to play the game.

So far, we've been able to send events to Wwise in order to play the music, and make it evolve with the game. But now we need the other way around: we need information coming from Wwise in order to be alerted when a beat or a new bar happen. And at each frame, we need to know with precision where we are in the track.

This is initialized at the line where we send the event to start the music. Let's write it step by step. The most basic usage is just to send the event (named StartEvent) to Wwise.

All scripts in this article are from several Game Objects. For the sake of simplicity, I won't describe their names, structure or relations. Just consider that all variables and functions declared are available to any scripts.

(c#) StartEvent.Post(gameObject); An event is always linked to a Game Object, hence why we gives the instance as the first argument.

Now we need a callback for each "music sync" events: beats, bar, etc. This is done by specifying two more arguments to the function: the type of events we want to be triggered for, and the callback function to call. For the first one, while we could be called only for bars, it's better to specify all kind of events related to music. We'll see later how to read the event's type. The next argument is a new function that we'll call OnMusicEvent, which we'll describe below.

(c#) StartEvent.Post(gameObject, (uint)AkCallbackType.AK_MusicSyncAll, OnMusicEvent);

Wait, why is the CallbackType parsed as an uint? Well, good question actually. The Post method only accepts uint, and the constant provided by Wwise happens to be a int, therefore it must be converted. Why isn't it directly the right type? I don't know, but I'm sure there must be a logical reason. For now, remember that it must be implemented that way.

Now, how does our OnMusicEvent look like?

(c#) void OnMusicEvent(object in_cookie, AkCallbackType in_type, AkCallbackInfo in_info) {

• The first argument, in_cookie, are additional data sent with the original event. We don't use it at all here.
• The second one in_type is the type of the event. It's with it that we'll know if the event triggered is a beat, a bar, or something else.
• Finally in_info provides useful data about the music track itself, such as the position in seconds, or the length of a bar.

In Line Momentum, we only react to the bar events. But know that there are many more that can be useful! For example, you could have animations triggered on the beats. It's even possible to have a time unit smaller than the beat, called "grid" in Wwise, which can be configured to be aligned with quarter notes, eight notes, or others fragments.

The best pattern to sync actions from your game with music events is to use UnityEvents. A Unity Event will have several object's methods subscribed to them, and will call all of them when it is triggered. This is super useful for code organization, because it means that your objects are not dependent between each other, and can easily be plugged and unplugged to events. Personally, I even like to use a Scriptable Object as a Store with UnityEvents that any Script can subscribe to. I bet you can also scale even better with plugins such as UniRx.

But for the sake of this article, let's keep it simple, and just declare our events locally. In the music event callback, we will check the type of the event, then invoke the corresponding Unity Event.

```(c#) public UnityEvent onBar = new UnityEvent(); public UnityEvent onBeat = new UnityEvent();

void OnMusicEvent(object incookie, AkCallbackType intype, AkCallbackInfo ininfo) { if (ininfo is AkMusicSyncCallbackInfo) { if (intype is AkCallbackType.AKMusicSyncBar) { onBar.Invoke(); } else if (intype is AkCallbackType.AKMusicSyncBeat) { onBeat.Invoke(); } } } ```

Now any function can be subscribed to beat or bar events! Either from the editor GUI, or through code (using onBar.AddListener).

That's not enough though! Reacting on beats is one thing, but we still need to know at every frame what is the position in the bar! There are several use cases for it, but our first one will be to update the position of the visual indicator.

Sure, we could use an animation or a tween for this, and make sure that it's approximately the same time as the bar. But there's a non neglectable risk that it falls out of sync eventually. Depending of the nature of your game, you might need to handle longer durations than a single bar, and there are cases where you want to be precise. You're never immune to a small lag or a music interruption. Thus your timing must always comes from the music data. Never use timers or scripted animations, unless it's not for a critical purpose (such as a short visual animation). For Line Momentum, the indicator is here to help the player get the right timing, so it must be precise!

_(Plus, doing this way means that you can pause the music and still keep everything in sync with no worries!)

Let's get back to the post of the StartMusic event! We need to track more than beats and bar. We want to be able to read directly the time position at any time. For this, we can pass the AK_EnableGetMusicPlayPosition constant as a second argument. Since we already pass AK_MusicSyncAll, we have to use a Bitwise OR (|) between the two. This option will allow us to track the music position from the player. However, we need a player ID for this. We must thus retrieve it from the Post result.

```(c#) private uint PlayerID;

// [...] When starting the music: PlayerID = StartEvent.Post(gameObject, (uint)AkCallbackType.AKMusicSyncAll | (uint)AkCallbackType.AKEnableGetMusicPlayPosition, OnMusicEvent); ```

Now, in the FixedUpdate, we can read the position in the bar by creating a SegmentInfo! For the reference, it is the same type we receive as in_info from our callback. Only this time, it's available at any time in the music.

(c#) private void FixedUpdate() { AkSegmentInfo segmentInfo = new AkSegmentInfo(); AkSoundEngine.GetPlayingSegmentInfo(PlayerID, segmentInfo, true); print(segmentInfo.iCurrentPosition); // --> position in ms }

The i in iCurrentPosition stands for int. There are other measurements in SegmentInfo that are prefixed with f, for float. Thus it indicates that it is not in milliseconds, but in seconds! This is important, because you'll want to choose one single unit when comparing values. And unfortunately, some values are only available in milliseconds, others only in seconds. So remember to convert them when needed!

Okay we have the time position in seconds now, but it's not super useful as it is. If we want to position the dot on the line, we need a value going from 0 to 1, so that we we'll never have to worry about the size of the line in pixels. We need to know the duration of a bar. Now that's something we could actually calculate by hand, using the BPM and the time signature. But what if those change during the development of the game? Your composer can have a change of mind after all. Or you could even have several tracks, with different tempo! It's better to just read them from Wwise data. While we're at it, we will also register the duration of a beat, because this data will be very useful.

We can do that in our Callback function! It will be triggered at the very start of the song for the first bar. Note that we only need to write those values once.

(c#) void OnMusicEvent(object in_cookie, AkCallbackType in_type, AkCallbackInfo in_info) { if (in_info is AkMusicSyncCallbackInfo) { AkMusicSyncCallbackInfo musicInfo = (AkMusicSyncCallbackInfo )in_info; if (in_type is AkCallbackType.AK_MusicSyncBar) { onBar.Invoke(); if (setDurations) { barDuration = musicInfo.segmentInfo_fBarDuration; beatDuration = musicInfo.segmentInfo_fGridDuration; // In Line Momentum, I used quarter notes as a "beat" setDuration = false } } // [...] } }

In FixedUpdate, we can now register the position in the bar for the dot indicator to use:

(c#) float fCurrentPosition = (float)segmentInfo.iCurrentPosition / 1000f barPosition = fCurrentPosition / barDuration

Alright, now let's tackle the big section: player interaction. The system in Line Momentum is quite simple: there are 10 beats (quarter notes) in a bar, the player must click on a selection of them. For example, on the first level, the player must click on beats number 2, 6 and 8 (beats going from 0 to 9). On level 2 it's 1, 3, 4, 6, and 8. And so on for other levels. Let's say those values are in an array called levelBeats. If the player correctly clicks on all the beats, they succeeds, if not, they fail.

Now I won't describe the whole logic of the game, because there's just too much. Instead, I'll focus on the rhythm logic that can be used for any kind of rhythm game: how to read input from the player, and also how to detect when they miss.

The first thing we need to do is to prepare the next beat that must be hit. Remember: always schedule! So, when the bar starts, we must setup the position (in seconds) of the next beat in the bar. This is calculated with the duration of a beat that we initialized earlier.

```(c#) private uint nextBeatIndex; private float nextBeatPosition;

void onBarStart() { updateNextBeatPosition(0) }

void updateNextBeatPosition(uint index) { nextBeatIndex = index; nextBeatPosition = levelBeats[index] * beatDuration; } ```

This is not the only way to write this value. Another approach is to use relative times, especially useful if you deal with long segments, where time can create delay between the theoretical beat position and their actual value. Basically it consist of reading the current position on a beat event callback, then setup the position with nextBeatPosition = currentPosition + (nbBeatsRemaining * beatDuration).

We now know exactly when the player should hit for the next beat. So when they will trigger an input, we will compare the current time to that value, which will determine if they hit correctly. However, it's absolutely impossible for players to be that precise! We need a margin error. It's a small time duration before and after the beat in which we consider the timing as still valid. It means that the time the player has to react will be twice the margin. With that in mind, let's write a function to compare if the current time is approximately the desired one:

```(c#) public float Precision = 0.08f; // Gives a 0.16 time of reaction

void isCloseTo(float time) { return Math.Abs(time - currentTime) < Precision } ```

But wait, there's a catch! Suppose that the next beat to validate is at position 0 (the very first beat). Since the bar is looping, what will happen if the player hits just before the end? That should be validated, because we are close enough to the start of the next bar. But our current algorithm will return false, because the time between 0 and the very end of the bar is, well, way higher than 0.08! I'd like to present you a one-line formula to solve this problem. If you have an idea, please share it. But as of today, we have to handle this special case.

```(c#) public bool IsCloseTo(float time) { if (Math.Abs(time - currentTime) < Precision) { return true; }

// Time is at 0
if (time < Precision) {
    return Math.Abs(barDuration - currentTime) < Precision;
}
// Time is at the end of the bar
else if ((barDuration - time) < Precision) {
   return currentTime < Precision;
}
return false;

} ```

Alright! Using this, we can now register players' input:

(c#) if (Input.GetMouseButtonDown(0) { if (IsCloseTo(nextBeatPosition)) { onValidateBeat(); // Do whatever you need to do when the player scores updateNextBeatPosition(nextBeatIndex + 1); } else if (!IsCloseTo(0f)) // Fail only if we're not close to the end of the bar { onFail(); // Register a failure } }

We're now able to detect when the player succeeds! But not when they fail. There are two ways for a player to miss a beat:

• They clicked too soon. This is what we have already implemented: when they are not close to a targeted beat, we register the failure.
• They clicked too late, or didn't click at all. This is what we need to implement now.

To register that a beat have been missed, we need a new function similar to IsCloseTo, to know if a time is considered as passed (that is, after the error margin).

(c#) public bool IsPassed(float time) { return currentTime > time && (currentTime - time) > Precision; }

Simple enough. But... Remember how we had issues with the start and end of the loop? Well, it's the same here. But it's mostly because of Wwise this time. As counter intuitive as it seems, currentTimer can sometime be above barDuration! But it's supposed to loop on the bar though, right? Yes, but timing in music is not an exact science. But the worse is: this can happen after the new bar event! So when we reach a new loop, after we have initialized our first target beat, there are few frames where the current time is not reset at 0, but instead slightly above the loop length. This completely breaks our calculation (because we consider being at the start of a new loop, and currentTime is still at the end). So, for this special case, the best is just to ignore the curentTime if it's near the end. Because it means that we're at the beginning, and thus no time could possibly be passed!

(c#) public bool IsPassed(float time) { return Math.Abs(barDuration - currentTime) > Precision && currentTime > time && (currentTime - time) > Precision; }

We are now able to detect when a player miss a beat! We just need to do the check in a FixedUpdate:

(c#) void FixedUpdate() { if (IsPassed(nextBeatPosition)) { onMiss(); updateNextBeatPosition(nextBeatIndex + 1); } } }

And there it is, you now have the basics to build a rhythm game! You know how to subscribe to music events, how to read the exact position in the track, how to validate a player input, and how to detect when a beat has been missed.

Naturally, there is more to it. I spared you the part of the logic that are inherent to the game's rules. There will always be exceptions, special cases to check, values to update at only precise moments, and many flags to know in which state we are. The code shared in this article are only the very basic bricks for building a rhythm game.

One last advice I could give is to use flags and Unity Events to schedule actions on timed events only. In line Momentum, there are a lot of visual updates that only happen at the start of the bar. So instead of calling an update function immediately, I usually schedule it so that it happens on the next bar.

```(c#) private bool doStuffOnNextBar = false;

void OnStart() { MusicManager.onBar.AddListener(BarTrigger) }

void OnThingHappen() { doStuffOnNextBar = true; }

void BarTrigger() { if (doStuffOnNextBar) { doStuffOnNextBar = false; DoStuff(); } } ```

Bonus exercise: you can probably use IEnumerator to yield instructions until the next bar or beat! I didn't have to do that in Line Momentum, but I had some logics like this in Godot games. Trust me, it offers really clean code!

If you're interested in the logic behind of Line Momentum, it is now available in open-source! You can check out its code, or open it with Unity or Wwise and see how it is structured.

I hope this guide was helpful to you. Those information, while available in various places in the Wwise documentation, are certainly the ones I wish I had when working for this game! Making rhythm games is tricky, but not too difficult if you have the right tools, and know how to proceed.