{"author_link":"\/users\/245333","author_name":"LDJam user 245333","author_uid":"245333","comments":[],"epoch":1665839407,"event":"LD51","format":"md","ldjam_node_id":310272,"likes":5,"metadata":{"p_key":"83951","p_author":"LDJam user 245333","p_authorkey":"1245333","p_urlkey":"294691","p_title":"How to build a rhythm game with Unity and Wwise","p_cat":"LDJam ","p_event":"LD51","p_time":"1665839407","p_likes":"5","p_comments":"0","p_status":"WAYBACK","us_key":"1245333","us_name":"LDJam user 245333","us_username":"245333","event_start":"1664496000","event_key":"114","event_name":"Ludum Dare 51"},"node":{"id":310272,"parent":307048,"superparent":296586,"author":245333,"type":"post","subtype":"","subsubtype":"","published":"2022-10-15T13:10:07Z","created":"2022-10-15T09:51:52Z","modified":"2022-10-15T14:37:20Z","version":967806,"slug":"how-to-build-a-rhythm-game-with-unity-and-wwise","name":"How to build a rhythm game with Unity and Wwise","body":"This is the second part of a devlog about Line Momentum, a rhythm game made for the Ludum Dare 51. The [first part](https:\/\/ldjam.com\/events\/ludum-dare\/51\/line-momentum\/how-the-music-system-works-in-line-momentum) was about how the dynamic music system was implemented using Wwise. In this one, we'll see how to use scripts to keep track of the position in the music, and validate players input according to the rhythm. Of course, you're more than welcome to [play the game](https:\/\/ldjam.com\/events\/ludum-dare\/51\/line-momentum).\n\n![screenshot_2.gif](\/\/\/raw\/55e\/b3\/z\/53269.gif)\n\nSo far, we've been able to send events to Wwise in order to play the music, and make it evolve with the game. But now we need the other way around: we need information coming _from_ Wwise in order to be alerted when a beat or a new bar happen. And at each frame, we need to know with precision where we are in the track.\n\nThis is initialized at the line where we send the event to start the music. Let's write it step by step. The most basic usage is just to send the event (named `StartEvent`) to Wwise.\n\n*All scripts in this article are from several Game Objects. For the sake of simplicity, I won't describe their names, structure or relations. Just consider that all variables and functions declared are available to any scripts.*\n\n```(c#)\nStartEvent.Post(gameObject);\n```\n_An event is always linked to a Game Object, hence why we gives the instance as the first argument._\n\nNow we need a callback for each \"music sync\" events: beats, bar, etc. This is done by specifying two more arguments to the function: the type of events we want to be triggered for, and the callback function to call. For the first one, while we could be called only for bars, it's better to specify *all* kind of events related to music. We'll see later how to read the event's type. The next argument is a new function that we'll call `OnMusicEvent`, which we'll describe below.\n\n```(c#)\nStartEvent.Post(gameObject, (uint)AkCallbackType.AK_MusicSyncAll, OnMusicEvent);\n```\n\nWait, why is the CallbackType parsed as an `uint`? Well, good question actually. The `Post` method only accepts _uint_, and the constant provided by Wwise happens to be a _int_, therefore it must be converted. Why isn't it directly the right type? I don't know, but I'm sure there must be a logical reason. For now, remember that it must be implemented that way.\n\nNow, how does our `OnMusicEvent` look like?\n\n```(c#)\nvoid OnMusicEvent(object in_cookie, AkCallbackType in_type, AkCallbackInfo in_info) {\n```\n\n* The first argument, **in_cookie**, are additional data sent with the original event. We don't use it at all here.\n* The second one **in_type** is the type of the event. It's with it that we'll know if the event triggered is a beat, a bar, or something else.\n* Finally **in_info** provides useful data about the music track itself, such as the position in seconds, or the length of a bar.\n\nIn Line Momentum, we only react to the bar events. But know that there are many more that can be useful! For example, you could have animations triggered on the beats. It's even possible to have a time unit smaller than the beat, called \"grid\" in Wwise, which can be configured to be aligned with quarter notes, eight notes, or others fragments.\n\nThe best pattern to sync actions from your game with music events is to use **UnityEvents**. A Unity Event will have several object's methods subscribed to them, and will call all of them when it is triggered. This is super useful for code organization, because it means that your objects are not dependent between each other, and can easily be plugged and unplugged to events. Personally, I even like to use a *Scriptable Object* as a Store with UnityEvents that any Script can subscribe to. I bet you can also scale even better with plugins such as *UniRx*.\n\nBut for the sake of this article, let's keep it simple, and just declare our events locally. In the music event callback, we will check the type of the event, then invoke the corresponding Unity Event.\n\n```(c#)\npublic UnityEvent onBar = new UnityEvent();\npublic UnityEvent onBeat = new UnityEvent();\n\nvoid OnMusicEvent(object in_cookie, AkCallbackType in_type, AkCallbackInfo in_info) {\n    if (in_info is AkMusicSyncCallbackInfo)\n    {\n        if (in_type is AkCallbackType.AK_MusicSyncBar)\n        {\n            onBar.Invoke();\n        }\n        else if (in_type is AkCallbackType.AK_MusicSyncBeat)\n        {\n            onBeat.Invoke();\n        }\n    }\n}\n```\n\nNow any function can be subscribed to beat or bar events! Either from the editor GUI, or through code (using  `onBar.AddListener`).\n\nThat's not enough though! Reacting on beats is one thing, but we still need to know at every frame what is the position in the bar! There are several use cases for it, but our first one will be to update the position of the visual indicator.\n\n![dot-movement.gif](\/\/\/raw\/55e\/b3\/z\/5327e.gif)\n\nSure, we could use an animation or a tween for this, and make sure that it's approximately the same time as the bar. But there's a non neglectable risk that it falls out of sync eventually. Depending of the nature of your game, you might need to handle longer durations than a single bar, and there are cases where you want to be precise. You're never immune to a small lag or a music interruption. Thus **your timing must always comes from the music data**. Never use timers or scripted animations, unless it's not for a critical purpose (such as a short visual animation). For Line Momentum, the indicator is here to help the player get the right timing, so *it must be precise*!\n\n_(Plus, doing this way means that you can pause the music and still keep everything in sync with no worries!)\n\nLet's get back to the post of the `StartMusic` event! We need to track more than beats and bar. We want to be able to read directly the time position at any time. For this, we can pass the `AK_EnableGetMusicPlayPosition` constant as a second argument. Since we already pass `AK_MusicSyncAll`, we have to use a Bitwise OR (`|`) between the two. This option will allow us to track the music position from the player. However, we need a player ID for this. We must thus retrieve it from the *Post* result.\n\n```(c#)\nprivate uint PlayerID;\n\n\/\/ [...] When starting the music:\nPlayerID = StartEvent.Post(gameObject, (uint)AkCallbackType.AK_MusicSyncAll | (uint)AkCallbackType.AK_EnableGetMusicPlayPosition, OnMusicEvent);\n```\n\nNow, in the `FixedUpdate`, we can read the position in the bar by creating a *SegmentInfo*! For the reference, it is the same type we receive as `in_info` from our callback. Only this time, it's available at any time in the music.\n\n```(c#)\nprivate void FixedUpdate()\n{\n    AkSegmentInfo segmentInfo = new AkSegmentInfo();\n    AkSoundEngine.GetPlayingSegmentInfo(PlayerID, segmentInfo, true);\n    print(segmentInfo.iCurrentPosition); \/\/ --> position in ms\n}\n```\n\nThe _i_ in `iCurrentPosition` stands for _int_. There are other measurements in SegmentInfo that are prefixed with _f_, for _float_. Thus it indicates that it is not in milliseconds, but in seconds! This is important, because you'll want to choose one single unit when comparing values. And unfortunately, some values are only available in milliseconds, others only in seconds. So remember to convert them when needed!\n\nOkay we have the time position in seconds now, but it's not super useful as it is. If we want to position the dot on the line, we need a value going from 0 to 1, so that we we'll never have to worry about the size of the line in pixels. We need to know the **duration of a bar**. Now that's something we could actually calculate by hand, using the BPM and the time signature. But what if those change during the development of the game? Your composer can have a change of mind after all. Or you could even have several tracks, with different tempo! It's better to just read them from Wwise data. While we're at it, we will also register the **duration of a beat**, because this data will be very useful.\n\nWe can do that in our Callback function! It will be triggered at the very start of the song for the first bar. Note that we only need to write those values once.\n\n```(c#)\nvoid OnMusicEvent(object in_cookie, AkCallbackType in_type, AkCallbackInfo in_info) {\n    if (in_info is AkMusicSyncCallbackInfo)\n    {\n        AkMusicSyncCallbackInfo musicInfo = (AkMusicSyncCallbackInfo )in_info;\n        if (in_type is AkCallbackType.AK_MusicSyncBar)\n        {\n            onBar.Invoke();\n            if (setDurations)\n            {\n                barDuration = musicInfo.segmentInfo_fBarDuration;\n                beatDuration = musicInfo.segmentInfo_fGridDuration; \/\/ In Line Momentum, I used quarter notes as a \"beat\"\n                setDuration = false\n            }\n        }\n        \/\/ [...]\n    }\n}\n```\n\nIn `FixedUpdate`, we can now register the position in the bar for the dot indicator to use:\n\n```(c#)\nfloat fCurrentPosition = (float)segmentInfo.iCurrentPosition \/ 1000f \nbarPosition = fCurrentPosition \/ barDuration\n```\n\nAlright, now let's tackle the big section: **player interaction**. The system in Line Momentum is quite simple: there are 10 beats (quarter notes) in a bar, the player must click on a selection of them. For example, on the first level, the player must click on beats number 2, 6 and 8 (beats going from 0 to 9). On level 2 it's 1, 3, 4, 6, and 8. And so on for other levels. Let's say those values are in an array called `levelBeats`. If the player correctly clicks on all the beats, they succeeds, if not, they fail.\n\nNow I won't describe the whole logic of the game, because there's just too much. Instead, I'll focus on the rhythm logic that can be used for any kind of rhythm game: **how to read input from the player**, and also **how to detect when they miss**.\n\nThe first thing we need to do is to prepare the next beat that must be hit. Remember: always schedule! So, when the bar starts, we must setup **the position (in seconds) of the next beat** in the bar. This is calculated with the duration of a beat that we initialized earlier.\n\n```(c#)\nprivate uint nextBeatIndex;\nprivate float nextBeatPosition;\n\nvoid onBarStart() {\n    updateNextBeatPosition(0)\n}\n\nvoid updateNextBeatPosition(uint index) {\n    nextBeatIndex = index;\n    nextBeatPosition = levelBeats[index] * beatDuration;\n}\n```\n\n*This is not the only way to write this value. Another approach is to use relative times, especially useful if you deal with long segments, where time can create delay between the theoretical beat position and their actual value. Basically it consist of reading the current position on a beat event callback, then setup the position with `nextBeatPosition = currentPosition + (nbBeatsRemaining * beatDuration)`.*\n\nWe now know exactly when the player _should_ hit for the next beat. So when they will trigger an input, we will compare the current time to that value, which will determine if they hit correctly. However, it's absolutely impossible for players to be that precise! We need a **margin error**. It's a small time duration before and after the beat in which we consider the timing as still valid. It means that the time the player has to react will be twice the margin. With that in mind, let's write a function to compare if the current time is approximately the desired one:\n\n```(c#)\npublic float Precision = 0.08f; \/\/ Gives a 0.16 time of reaction\n\nvoid isCloseTo(float time) {\n    return Math.Abs(time - currentTime) < Precision\n}\n```\n\nBut wait, there's a catch! Suppose that the next beat to validate is at position _0_ (the very first beat). Since the bar is looping, what will happen if the player hits just before the end? That _should_ be validated, because we are close  enough to the start of the next bar. But our current algorithm will return _false_, because the time between 0 and the very end of the bar is, well, way higher than 0.08! I'd like to present you a one-line formula to solve this problem. If you have an idea, please share it. But as of today, we have to handle this special case.\n\n```(c#)\npublic bool IsCloseTo(float time)\n{\n    if (Math.Abs(time - currentTime) < Precision) {\n        return true;\n    }\n\n    \/\/ Time is at 0\n    if (time < Precision) {\n        return Math.Abs(barDuration - currentTime) < Precision;\n    }\n    \/\/ Time is at the end of the bar\n    else if ((barDuration - time) < Precision) {\n       return currentTime < Precision;\n    }\n    return false;\n}\n```\n\nAlright! Using this, we can now register players' input:\n\n```(c#)\nif (Input.GetMouseButtonDown(0) {\n   if (IsCloseTo(nextBeatPosition))\n    {\n       onValidateBeat(); \/\/ Do whatever you need to do when the player scores\n       updateNextBeatPosition(nextBeatIndex + 1);\n    }\n    else if (!IsCloseTo(0f)) \/\/ Fail only if we're not close to the end of the bar\n    {\n        onFail(); \/\/ Register a failure\n    }\n}\n```\n\nWe're now able to detect when the player succeeds! But not when they fail. There are two ways for a player to miss a beat:\n\n- **They clicked too soon**. This is what we have already implemented: when they are not close to a targeted beat, we register the failure.\n- **They clicked too late, or didn't click at all**. This is what we need to implement now.\n\nTo register that a beat have been _missed_, we need a new function similar to `IsCloseTo`, to know if a time is considered as _passed_ (that is, after the error margin).\n\n```(c#)\npublic bool IsPassed(float time) {\n    return currentTime > time && (currentTime - time) > Precision;\n}\n```\n\nSimple enough. But... Remember how we had issues with the start and end of the loop? Well, it's the same here. But it's mostly because of Wwise this time. As counter intuitive as it seems, `currentTimer` can sometime be above `barDuration`! But it's supposed to loop on the bar though, right? Yes, but timing in music is not an exact science. But the worse is: _this can happen after the new bar event_! So when we reach a new loop, _after_ we have initialized our first target beat, there are few frames where the current time is not reset at 0, but instead slightly above the loop length. This completely breaks our calculation (because we consider being at the start of a new loop, and `currentTime` is still at the end). So, for this special case, the best is just to ignore the curentTime if it's near the end. Because it means that we're at the beginning, and thus no time could possibly be passed!\n\n```(c#)\npublic bool IsPassed(float time) {\n    return Math.Abs(barDuration - currentTime) > Precision && currentTime > time && (currentTime - time) > Precision;\n}\n```\n\nWe are now able to detect when a player miss a beat! We just need to do the check in a `FixedUpdate`:\n\n```(c#)\nvoid FixedUpdate()\n{\n    if (IsPassed(nextBeatPosition)) {\n        onMiss();\n        updateNextBeatPosition(nextBeatIndex + 1);\n    }\n}\n}\n```\n\nAnd there it is, you now have the basics to build a rhythm game! You know how to **subscribe to music events**, how to **read the exact position in the track**, how to **validate a player input**, and how to **detect when a beat has been missed**.\n\nNaturally, there is more to it. I spared you the part of the logic that are inherent to the game's rules. There will always be exceptions, special cases to check, values to update at only precise moments, and many flags to know in which state we are. The code shared in this article are only the very basic bricks for building a rhythm game.\n\nOne last advice I could give is to use flags and Unity Events to schedule actions on timed events only. In line Momentum, there are a lot of visual updates that only happen at the start of the bar. So instead of calling an update function immediately, I usually schedule it so that it happens on the next bar.\n\n```(c#)\nprivate bool doStuffOnNextBar = false;\n\nvoid OnStart() {\n   MusicManager.onBar.AddListener(BarTrigger)\n}\n\nvoid OnThingHappen() {\n   doStuffOnNextBar = true;\n}\n\nvoid BarTrigger() {\n   if (doStuffOnNextBar) {\n      doStuffOnNextBar = false;\n      DoStuff();\n   }\n}\n```\n\n_Bonus exercise: you can probably use IEnumerator to yield instructions until the next bar or beat! I didn't have to do that in Line Momentum, but I had some logics like this in Godot games. Trust me, it offers really clean code!_\n\nIf you're interested in the logic behind of _Line Momentum_, it is now [available in open-source](https:\/\/github.com\/ClementRivaille\/line-momentum)! You can check out its code, or open it with Unity or Wwise and see how it is structured.\n\nI hope this guide was helpful to you. Those information, while available in various places in the Wwise documentation, are certainly the ones I wish I had when working for this game! Making rhythm games is tricky, but not too difficult if you have the right tools, and know how to proceed.","scope":"public","node-timestamp":"2022-10-15T14:10:00Z","meta":[],"path":"\/events\/ludum-dare\/51\/line-momentum\/how-to-build-a-rhythm-game-with-unity-and-wwise","parents":[1,5,9,296586,307048],"files":[],"files-timestamp":0,"love":5,"love-timestamp":"2022-10-15T14:37:04Z","comments":1,"comments-timestamp":"2022-10-15T14:37:20Z","_wayback":{"timestamp":"20221015144735","url":"https:\/\/api.ldjam.com\/vx\/node2\/get\/310293+310295+310272+310277+310266+310265+310258+310259+310253+310249?author&parent&superparent"}},"node_metadata":{"n_key":"310272","n_urlkey":"294691","n_parent":"307048","n_path":"\/events\/ludum-dare\/51\/line-momentum\/how-to-build-a-rhythm-game-with-unity-and-wwise","n_slug":"how-to-build-a-rhythm-game-with-","n_type":"post","n_subtype":"","n_subsubtype":"","n_author":"245333","n_created":"1665827512","n_modified":"1665844640","n_version":"967806","n_status":"WAYBACK"},"source_url":"https:\/\/ldjam.com\/events\/ludum-dare\/51\/line-momentum\/how-to-build-a-rhythm-game-with-unity-and-wwise","text":"This is the second part of a devlog about Line Momentum, a rhythm game made for the Ludum Dare 51. The [first part](https:\/\/ldjam.com\/events\/ludum-dare\/51\/line-momentum\/how-the-music-system-works-in-line-momentum) was about how the dynamic music system was implemented using Wwise. In this one, we'll see how to use scripts to keep track of the position in the music, and validate players input according to the rhythm. Of course, you're more than welcome to [play the game](https:\/\/ldjam.com\/events\/ludum-dare\/51\/line-momentum).\n\n![screenshot_2.gif](\/\/\/raw\/55e\/b3\/z\/53269.gif)\n\nSo far, we've been able to send events to Wwise in order to play the music, and make it evolve with the game. But now we need the other way around: we need information coming _from_ Wwise in order to be alerted when a beat or a new bar happen. And at each frame, we need to know with precision where we are in the track.\n\nThis is initialized at the line where we send the event to start the music. Let's write it step by step. The most basic usage is just to send the event (named `StartEvent`) to Wwise.\n\n*All scripts in this article are from several Game Objects. For the sake of simplicity, I won't describe their names, structure or relations. Just consider that all variables and functions declared are available to any scripts.*\n\n```(c#)\nStartEvent.Post(gameObject);\n```\n_An event is always linked to a Game Object, hence why we gives the instance as the first argument._\n\nNow we need a callback for each \"music sync\" events: beats, bar, etc. This is done by specifying two more arguments to the function: the type of events we want to be triggered for, and the callback function to call. For the first one, while we could be called only for bars, it's better to specify *all* kind of events related to music. We'll see later how to read the event's type. The next argument is a new function that we'll call `OnMusicEvent`, which we'll describe below.\n\n```(c#)\nStartEvent.Post(gameObject, (uint)AkCallbackType.AK_MusicSyncAll, OnMusicEvent);\n```\n\nWait, why is the CallbackType parsed as an `uint`? Well, good question actually. The `Post` method only accepts _uint_, and the constant provided by Wwise happens to be a _int_, therefore it must be converted. Why isn't it directly the right type? I don't know, but I'm sure there must be a logical reason. For now, remember that it must be implemented that way.\n\nNow, how does our `OnMusicEvent` look like?\n\n```(c#)\nvoid OnMusicEvent(object in_cookie, AkCallbackType in_type, AkCallbackInfo in_info) {\n```\n\n* The first argument, **in_cookie**, are additional data sent with the original event. We don't use it at all here.\n* The second one **in_type** is the type of the event. It's with it that we'll know if the event triggered is a beat, a bar, or something else.\n* Finally **in_info** provides useful data about the music track itself, such as the position in seconds, or the length of a bar.\n\nIn Line Momentum, we only react to the bar events. But know that there are many more that can be useful! For example, you could have animations triggered on the beats. It's even possible to have a time unit smaller than the beat, called \"grid\" in Wwise, which can be configured to be aligned with quarter notes, eight notes, or others fragments.\n\nThe best pattern to sync actions from your game with music events is to use **UnityEvents**. A Unity Event will have several object's methods subscribed to them, and will call all of them when it is triggered. This is super useful for code organization, because it means that your objects are not dependent between each other, and can easily be plugged and unplugged to events. Personally, I even like to use a *Scriptable Object* as a Store with UnityEvents that any Script can subscribe to. I bet you can also scale even better with plugins such as *UniRx*.\n\nBut for the sake of this article, let's keep it simple, and just declare our events locally. In the music event callback, we will check the type of the event, then invoke the corresponding Unity Event.\n\n```(c#)\npublic UnityEvent onBar = new UnityEvent();\npublic UnityEvent onBeat = new UnityEvent();\n\nvoid OnMusicEvent(object in_cookie, AkCallbackType in_type, AkCallbackInfo in_info) {\n    if (in_info is AkMusicSyncCallbackInfo)\n    {\n        if (in_type is AkCallbackType.AK_MusicSyncBar)\n        {\n            onBar.Invoke();\n        }\n        else if (in_type is AkCallbackType.AK_MusicSyncBeat)\n        {\n            onBeat.Invoke();\n        }\n    }\n}\n```\n\nNow any function can be subscribed to beat or bar events! Either from the editor GUI, or through code (using  `onBar.AddListener`).\n\nThat's not enough though! Reacting on beats is one thing, but we still need to know at every frame what is the position in the bar! There are several use cases for it, but our first one will be to update the position of the visual indicator.\n\n![dot-movement.gif](\/\/\/raw\/55e\/b3\/z\/5327e.gif)\n\nSure, we could use an animation or a tween for this, and make sure that it's approximately the same time as the bar. But there's a non neglectable risk that it falls out of sync eventually. Depending of the nature of your game, you might need to handle longer durations than a single bar, and there are cases where you want to be precise. You're never immune to a small lag or a music interruption. Thus **your timing must always comes from the music data**. Never use timers or scripted animations, unless it's not for a critical purpose (such as a short visual animation). For Line Momentum, the indicator is here to help the player get the right timing, so *it must be precise*!\n\n_(Plus, doing this way means that you can pause the music and still keep everything in sync with no worries!)\n\nLet's get back to the post of the `StartMusic` event! We need to track more than beats and bar. We want to be able to read directly the time position at any time. For this, we can pass the `AK_EnableGetMusicPlayPosition` constant as a second argument. Since we already pass `AK_MusicSyncAll`, we have to use a Bitwise OR (`|`) between the two. This option will allow us to track the music position from the player. However, we need a player ID for this. We must thus retrieve it from the *Post* result.\n\n```(c#)\nprivate uint PlayerID;\n\n\/\/ [...] When starting the music:\nPlayerID = StartEvent.Post(gameObject, (uint)AkCallbackType.AK_MusicSyncAll | (uint)AkCallbackType.AK_EnableGetMusicPlayPosition, OnMusicEvent);\n```\n\nNow, in the `FixedUpdate`, we can read the position in the bar by creating a *SegmentInfo*! For the reference, it is the same type we receive as `in_info` from our callback. Only this time, it's available at any time in the music.\n\n```(c#)\nprivate void FixedUpdate()\n{\n    AkSegmentInfo segmentInfo = new AkSegmentInfo();\n    AkSoundEngine.GetPlayingSegmentInfo(PlayerID, segmentInfo, true);\n    print(segmentInfo.iCurrentPosition); \/\/ --> position in ms\n}\n```\n\nThe _i_ in `iCurrentPosition` stands for _int_. There are other measurements in SegmentInfo that are prefixed with _f_, for _float_. Thus it indicates that it is not in milliseconds, but in seconds! This is important, because you'll want to choose one single unit when comparing values. And unfortunately, some values are only available in milliseconds, others only in seconds. So remember to convert them when needed!\n\nOkay we have the time position in seconds now, but it's not super useful as it is. If we want to position the dot on the line, we need a value going from 0 to 1, so that we we'll never have to worry about the size of the line in pixels. We need to know the **duration of a bar**. Now that's something we could actually calculate by hand, using the BPM and the time signature. But what if those change during the development of the game? Your composer can have a change of mind after all. Or you could even have several tracks, with different tempo! It's better to just read them from Wwise data. While we're at it, we will also register the **duration of a beat**, because this data will be very useful.\n\nWe can do that in our Callback function! It will be triggered at the very start of the song for the first bar. Note that we only need to write those values once.\n\n```(c#)\nvoid OnMusicEvent(object in_cookie, AkCallbackType in_type, AkCallbackInfo in_info) {\n    if (in_info is AkMusicSyncCallbackInfo)\n    {\n        AkMusicSyncCallbackInfo musicInfo = (AkMusicSyncCallbackInfo )in_info;\n        if (in_type is AkCallbackType.AK_MusicSyncBar)\n        {\n            onBar.Invoke();\n            if (setDurations)\n            {\n                barDuration = musicInfo.segmentInfo_fBarDuration;\n                beatDuration = musicInfo.segmentInfo_fGridDuration; \/\/ In Line Momentum, I used quarter notes as a \"beat\"\n                setDuration = false\n            }\n        }\n        \/\/ [...]\n    }\n}\n```\n\nIn `FixedUpdate`, we can now register the position in the bar for the dot indicator to use:\n\n```(c#)\nfloat fCurrentPosition = (float)segmentInfo.iCurrentPosition \/ 1000f \nbarPosition = fCurrentPosition \/ barDuration\n```\n\nAlright, now let's tackle the big section: **player interaction**. The system in Line Momentum is quite simple: there are 10 beats (quarter notes) in a bar, the player must click on a selection of them. For example, on the first level, the player must click on beats number 2, 6 and 8 (beats going from 0 to 9). On level 2 it's 1, 3, 4, 6, and 8. And so on for other levels. Let's say those values are in an array called `levelBeats`. If the player correctly clicks on all the beats, they succeeds, if not, they fail.\n\nNow I won't describe the whole logic of the game, because there's just too much. Instead, I'll focus on the rhythm logic that can be used for any kind of rhythm game: **how to read input from the player**, and also **how to detect when they miss**.\n\nThe first thing we need to do is to prepare the next beat that must be hit. Remember: always schedule! So, when the bar starts, we must setup **the position (in seconds) of the next beat** in the bar. This is calculated with the duration of a beat that we initialized earlier.\n\n```(c#)\nprivate uint nextBeatIndex;\nprivate float nextBeatPosition;\n\nvoid onBarStart() {\n    updateNextBeatPosition(0)\n}\n\nvoid updateNextBeatPosition(uint index) {\n    nextBeatIndex = index;\n    nextBeatPosition = levelBeats[index] * beatDuration;\n}\n```\n\n*This is not the only way to write this value. Another approach is to use relative times, especially useful if you deal with long segments, where time can create delay between the theoretical beat position and their actual value. Basically it consist of reading the current position on a beat event callback, then setup the position with `nextBeatPosition = currentPosition + (nbBeatsRemaining * beatDuration)`.*\n\nWe now know exactly when the player _should_ hit for the next beat. So when they will trigger an input, we will compare the current time to that value, which will determine if they hit correctly. However, it's absolutely impossible for players to be that precise! We need a **margin error**. It's a small time duration before and after the beat in which we consider the timing as still valid. It means that the time the player has to react will be twice the margin. With that in mind, let's write a function to compare if the current time is approximately the desired one:\n\n```(c#)\npublic float Precision = 0.08f; \/\/ Gives a 0.16 time of reaction\n\nvoid isCloseTo(float time) {\n    return Math.Abs(time - currentTime) < Precision\n}\n```\n\nBut wait, there's a catch! Suppose that the next beat to validate is at position _0_ (the very first beat). Since the bar is looping, what will happen if the player hits just before the end? That _should_ be validated, because we are close  enough to the start of the next bar. But our current algorithm will return _false_, because the time between 0 and the very end of the bar is, well, way higher than 0.08! I'd like to present you a one-line formula to solve this problem. If you have an idea, please share it. But as of today, we have to handle this special case.\n\n```(c#)\npublic bool IsCloseTo(float time)\n{\n    if (Math.Abs(time - currentTime) < Precision) {\n        return true;\n    }\n\n    \/\/ Time is at 0\n    if (time < Precision) {\n        return Math.Abs(barDuration - currentTime) < Precision;\n    }\n    \/\/ Time is at the end of the bar\n    else if ((barDuration - time) < Precision) {\n       return currentTime < Precision;\n    }\n    return false;\n}\n```\n\nAlright! Using this, we can now register players' input:\n\n```(c#)\nif (Input.GetMouseButtonDown(0) {\n   if (IsCloseTo(nextBeatPosition))\n    {\n       onValidateBeat(); \/\/ Do whatever you need to do when the player scores\n       updateNextBeatPosition(nextBeatIndex + 1);\n    }\n    else if (!IsCloseTo(0f)) \/\/ Fail only if we're not close to the end of the bar\n    {\n        onFail(); \/\/ Register a failure\n    }\n}\n```\n\nWe're now able to detect when the player succeeds! But not when they fail. There are two ways for a player to miss a beat:\n\n- **They clicked too soon**. This is what we have already implemented: when they are not close to a targeted beat, we register the failure.\n- **They clicked too late, or didn't click at all**. This is what we need to implement now.\n\nTo register that a beat have been _missed_, we need a new function similar to `IsCloseTo`, to know if a time is considered as _passed_ (that is, after the error margin).\n\n```(c#)\npublic bool IsPassed(float time) {\n    return currentTime > time && (currentTime - time) > Precision;\n}\n```\n\nSimple enough. But... Remember how we had issues with the start and end of the loop? Well, it's the same here. But it's mostly because of Wwise this time. As counter intuitive as it seems, `currentTimer` can sometime be above `barDuration`! But it's supposed to loop on the bar though, right? Yes, but timing in music is not an exact science. But the worse is: _this can happen after the new bar event_! So when we reach a new loop, _after_ we have initialized our first target beat, there are few frames where the current time is not reset at 0, but instead slightly above the loop length. This completely breaks our calculation (because we consider being at the start of a new loop, and `currentTime` is still at the end). So, for this special case, the best is just to ignore the curentTime if it's near the end. Because it means that we're at the beginning, and thus no time could possibly be passed!\n\n```(c#)\npublic bool IsPassed(float time) {\n    return Math.Abs(barDuration - currentTime) > Precision && currentTime > time && (currentTime - time) > Precision;\n}\n```\n\nWe are now able to detect when a player miss a beat! We just need to do the check in a `FixedUpdate`:\n\n```(c#)\nvoid FixedUpdate()\n{\n    if (IsPassed(nextBeatPosition)) {\n        onMiss();\n        updateNextBeatPosition(nextBeatIndex + 1);\n    }\n}\n}\n```\n\nAnd there it is, you now have the basics to build a rhythm game! You know how to **subscribe to music events**, how to **read the exact position in the track**, how to **validate a player input**, and how to **detect when a beat has been missed**.\n\nNaturally, there is more to it. I spared you the part of the logic that are inherent to the game's rules. There will always be exceptions, special cases to check, values to update at only precise moments, and many flags to know in which state we are. The code shared in this article are only the very basic bricks for building a rhythm game.\n\nOne last advice I could give is to use flags and Unity Events to schedule actions on timed events only. In line Momentum, there are a lot of visual updates that only happen at the start of the bar. So instead of calling an update function immediately, I usually schedule it so that it happens on the next bar.\n\n```(c#)\nprivate bool doStuffOnNextBar = false;\n\nvoid OnStart() {\n   MusicManager.onBar.AddListener(BarTrigger)\n}\n\nvoid OnThingHappen() {\n   doStuffOnNextBar = true;\n}\n\nvoid BarTrigger() {\n   if (doStuffOnNextBar) {\n      doStuffOnNextBar = false;\n      DoStuff();\n   }\n}\n```\n\n_Bonus exercise: you can probably use IEnumerator to yield instructions until the next bar or beat! I didn't have to do that in Line Momentum, but I had some logics like this in Godot games. Trust me, it offers really clean code!_\n\nIf you're interested in the logic behind of _Line Momentum_, it is now [available in open-source](https:\/\/github.com\/ClementRivaille\/line-momentum)! You can check out its code, or open it with Unity or Wwise and see how it is structured.\n\nI hope this guide was helpful to you. Those information, while available in various places in the Wwise documentation, are certainly the ones I wish I had when working for this game! Making rhythm games is tricky, but not too difficult if you have the right tools, and know how to proceed.","title":"How to build a rhythm game with Unity and Wwise","wayback_source":{"timestamp":"20221015144735","url":"https:\/\/api.ldjam.com\/vx\/node2\/get\/310293+310295+310272+310277+310266+310265+310258+310259+310253+310249?author&parent&superparent"}}