Neural Network Ratatouille by polegar

[raw]
made by polegar for LD 41 (COMPO)

Incompatible Genres ? Hmmm is it combatible to have a rat in a kitchen or to control your game with your webcam ?

For each ludum dare in which I participate, I try to learn something new. This time it's https://js.tensorflow.org. Because it's incredible to be able to do deep learning programming in JS !

I've used this tutorial as a starting point : https://js.tensorflow.org/tutorials/webcam-transfer-learning.html and then develop the game with pure JS and HTML5.

Surprinsingly, I've had more trouble with my rusty JS than with tensorflow ... I probably shoud have use a lib like Phaser Js instead of writing everything on my own.

/!\ I just added a post comp version with some minor bux fixes (you can now restart by pressing R and you can no longer launch the game without having trained your neural network), I suggest you to play this version for a better experience /!\

HERE'S HOW YOU PLAY THE GAME (with your webcam) :

1- For each control you need to provide some pictures of you holding an object(a fruit or a vegetable for example :)) in a specific direction. For IDLE I suggest you to hide the object your holding. (hint : you can hold clic to capture a lot of picture fast)

2- Then you have to clic on "Train ratatouille" and wait a little for the number to go down.

3- After that clic on "Let's cook" and be ready to play !

4- Show the object on the direction you want to move the character. You need to catch the cheese and avoid the mouse trap !

Ratings

Overall 241th 3.557⭐ 46🧑‍⚖️
Fun 357th 3.25⭐ 44🧑‍⚖️
Innovation 3th 4.57⭐ 45🧑‍⚖️
Theme 358th 3.571⭐ 44🧑‍⚖️
Graphics 563th 2.69⭐ 44🧑‍⚖️
Humor 248th 2.952⭐ 44🧑‍⚖️
Mood 466th 2.805⭐ 43🧑‍⚖️
Given 53🗳️ 40🗨️

Feedback

nander
24. Apr 2018 · 12:59 UTC
It actually works!! The game itself isn't that special, but the technique actually works, nice!
Faloon
24. Apr 2018 · 13:32 UTC
Yes, true. Astonishing.
Ananace
24. Apr 2018 · 15:24 UTC
Well, that actually worked much better than I expected it to, though the gameplay is quite simplistic.

Can't say I've played many games with webcam controls.
legavroche
24. Apr 2018 · 15:31 UTC
This webcam control game is fun and innovative! The technicality of this game is astonishing.
deep neural network programming(for machine learning?) + webcam + tensor flow and building the game in Javascript is a plus plus plus!
microwerx
24. Apr 2018 · 15:36 UTC
That was pretty cool. I thought it was innovative to use your webcam for training a neural net.
incobalt
24. Apr 2018 · 22:52 UTC
It took me far too long to realize I had to click the train button even though it was written right here >.< This was cool and very innovative! I do wish that it was attached to a more drawn-out game, though.
aplite
24. Apr 2018 · 22:56 UTC
What a unique mechanic! Really impressive and it works well given the limited time constraint. The movement was hard to get precise so avoiding the traps was difficult at times.

To help players get used to the mechanic you could start them off in a room with just a piece of cheese, and then move them into the room with traps and cheese. I killed myself a couple of times at the very start and had to redo the setup process each time.
🎤 polegar
25. Apr 2018 · 09:24 UTC
Thank you everyone for the kind comments :) . @incobalt @aplite I just posted a post comp version with some fixes for the issues you encountered (add a restart with R, and the button "let's cook" is no longer visible if you don't train the model)
Orteil
25. Apr 2018 · 11:13 UTC
oh dang, that actually works great ! good stuff
GastricSparrow
25. Apr 2018 · 13:18 UTC
Wow, as a machine learning novice I love what you've done here. I've been looking into how to incorporate machine learning into games, and here you are making this on a whim! Great innovation happening, love it.
jfraser314
25. Apr 2018 · 20:19 UTC
Difficult but hilarious. Had my class playing this for a while, here is one of my students trying (made sure to use one that wasn't too embarrassing). Thanks for the great idea (though it made me dizzy)

![Nerual Network.gif](///raw/d93/c/z/14043.gif)
🎤 polegar
25. Apr 2018 · 21:37 UTC
@jfraser314 Hahaha thank you for trying it in class !
sndr
25. Apr 2018 · 23:30 UTC
Whoa, this is impressive! Unexpected entry with a lot of creativity, both on the technical side and the design one. Controls worked surprisingly smooth (I was expecting it to be a bit more fiddly), it creates a really cool challenge, and the theme is most certainly super cute. :) Loved this! Really good job!
candlesan
26. Apr 2018 · 04:49 UTC
Well that was really different! How fun. Thanks for giving this piece of tech a try and I also applaud your desire to learn something new every jam - I think that's a great idea!!!
DigNZ
26. Apr 2018 · 07:39 UTC
What an amazing idea! It didn't quite work for me but was close enough to see that it should be possible. I probably should've chosen more different images for the actions. Impressive and innovative technical effort and a simple little game.
Weston Walker
26. Apr 2018 · 15:26 UTC
Extremely innovative idea here. Props to you for breaking out of the mold. Everything seemed to work as it should. Great job!
evandowning
26. Apr 2018 · 15:32 UTC
Neat!
Team Luminous
26. Apr 2018 · 17:24 UTC
What an exciting idea! It worked — tho holding the object both left and right ended up being hard. :D I'd love a way to remove samples, too, as I added bad ones by accident.

-Rao
John Darrington
27. Apr 2018 · 02:14 UTC
Holy sh**. This was amazing - and one of my favorite ideas here this year. I was impressed with the hard work in finding a good mechanic.
sebbernery
27. Apr 2018 · 11:25 UTC
Awesome, that's a very good usage of NN in JS. And it works perfectly (for me) and gives very fun game moments (I played with my head).
mrwillowb
27. Apr 2018 · 15:31 UTC
Not bad
designernap
27. Apr 2018 · 16:30 UTC
Lovely bit of tech and game here! wonderful work!
morrilet
29. Apr 2018 · 21:52 UTC
The tech behind this was amazing, but the game itself was lacking. Stick this on a more interesting game or a series of minigames and it could be pretty cool. Full points for innovation, though, as this is a really interesting concept. Great work!
bloodyaugust
30. Apr 2018 · 07:22 UTC
Tried to play this on stream, but alas, cannot use OBS and have the webcam free at the same time. :frowning2: I'll be back to play off-stream and give further feedback.
🎤 polegar
30. Apr 2018 · 11:20 UTC
@bloodyaugust oh that's a pity it could have been fun on stream, it's good to know though.
Paul Avallone
30. Apr 2018 · 22:49 UTC
This is really cool. I would love to see the game expanded to operating in a kitchen like in the movie. I am impressed how well the controls worked. Great job.
JackLeHamster
01. May 2018 · 15:14 UTC
Le concept est pas mal. Faudrait pousser l'idée pour avoir un gameplay plus fun, mais sinon l'idée est originale.

Voilà ma video sur le jeu (parmis d'autres):

https://www.youtube.com/watch?v=_cze6Dpc6JQ
🎤 polegar
02. May 2018 · 12:04 UTC
@JackLeHamster Hehe je viens de voir la vidéo c'est cool que tu y ai joué :). Et oui effectivement le jeu est plutôt une expérience technologique qu'autre chose. C'était dur en 48h d'arriver à faire un gameplay sympa tout en apprenant à utiliser tensorflow JS pour détecter les mouvements par webcam.
Par contre le jeu semble assez lent sur la vidéo c'est étonnant, tu as quoi comme CPU (ca doit être lié au fait que tu streamais en meme temps) ?
JackLeHamster
02. May 2018 · 14:42 UTC
Ouais, tout est un peu plus lent que j'enregistre. C'était une bonne idée. Je pense peut être tu peux utiliser ta technologie pour faire un jeu de course, tu te mets à la place du cockpit, et suivant que tu te penches à droite ou à gauche, ça bouge la voiture dans cette direction. Et tu voix ta reflection dans le parbrise ou queq chose comme ça.
🎤 polegar
16. May 2018 · 10:19 UTC
@sndr @DigNZ @nander @Faloon @aplite @Orteil @Ananace @JackLeHamster @PaulAvallone @bloodyaugust @morrilet @designernap @sebbernery @JohnDarrington @TeamLuminous @evandowning @WestonWalker @candlesan @jfraser314 @GastricSparrow

Third place in innovation that's amazing ! Thank you everyone for playing, commenting and rating my game. I can't wait for the next LD !
GastricSparrow
19. May 2018 · 04:57 UTC
@polegar Congrats! I barely got beat out by you in innovation, shouldn't have rated you a 5... just kidding, you deserve it!