Continuing my Ludum Dare 42 project

So this past Ludum dare I produced probably my favourite game yet. I received fairly good comments from friends, family, and reviewers and I am really happy with the outcome! Lately, I have been thinking of expanding it, turning it from a 72-hour project into a much longer-term project. I don't know where to start, so asking people who have continued their own games in the past - what would be a better course of action? Do I restart from scratch, maybe borrowing some art assets and glancing at the code, or do I continue with what I have got? The way I left it I didn't think I would be coming back to it so it's VERY messy and most of the code I would have to change anyway, but if I start from scratch it adds a, probably large, chunk of time into the development.

If you are interested, you can get the game here: https://micdoes.itch.io/dwn