I dream of pixel art: AI game by loveapplegames

[raw]
made by loveapplegames for Ludum Dare 53 (JAM)

Sorry guys, my entry has nothing to do with the theme. This LD, I had the insane idea of creating an AI and putting it in a browser game. And I had an insane weekend, being a total noob to Python, PyTorch, and deep learning.

My entry includes a image-to-image generative AI, is made with the Pytorch library, which enables deep learning using a PC's GPU. It is a 5-layer convolution network trained as an autoencoder (originally as a GAN, but this did not work). The weight data is converted to the ONNX format for loading in the browser. It uses WebGL to accelate execution. The AI feeds back into itself, changing into different shapes. This is also called "dreaming".

The original entry was trained on a set of 1800 8x8 RGBA images. The resulting shapes vagely resemble pixel art, which is the best I could do in the limited time. This AI basically sucks. It adds noise so it keeps changing, but sometimes converges into a static pattern, just reload or copy back to try again.

Click on the animating image to make a copy. The copy will be shown below the animation. Click on a copy to copy it back to the AI. Have fun creating "pixel art"!

POSTCOMPO UPDATE 1: POTION GENERATOR

After a week of hard work, I managed to improve the AI a lot. I found there was a bug in the data exporter, also I cleaned up the dataset and optimized the network. Once it worked well, I fed it a new dataset of similar images (potion sprites), that allows the network to not just recall the data but also generalize from it. At this point, the network converges reliably to valid images, and can be used as an in-game sprite generator. The network has been shrunk from 32Mb to about 350Kb. Check the postcompo demo to see the result!

potions-demo-screenshot-1.png

POSTCOMPO UPDATE 2: ANIMATED CHARACTER GENERATOR

My next step was to go from faithful reproduction to actual creative generation. For this, I created a dataset of 376 8x8 characters taken from freely available sources. Because pixel art is prone to overfitting, I removed oddballs to arrive at 166 humanoid characters, and shrunk the latent size to 3. The result is a quite useful animated game character generator. You can download a sprite sheet with just 2 clicks.

Screenshot 2023-05-18 at 11-28-31 Screenshot.png

Ratings

Given 1🗳️ 0🗨️

Feedback

jward
02. May 2023 · 00:51 UTC
Interesting, the page is throwing for me (in Linux) with error: Error: invalid wire type 4 at offset 3

I'll have to come back and try it in Windows.
cheesepencil
02. May 2023 · 02:20 UTC
Maybe I'm not doing it right but I don't see a moving image to click on :-(

FireFox + Windows 11
🎤 loveapplegames
02. May 2023 · 07:29 UTC
Sorry guys! There is a problem with the files. My hosting service is malfunctioning so I can't fix it now. I have created a support ticket. Please be patient...
🎤 loveapplegames
02. May 2023 · 08:56 UTC
Fixed the problem! It should work now.
Austin Sierra
10. May 2023 · 15:58 UTC
Cool math but not really a game. I love AI projects though. Great work, and have a great ludum dare!
milq
18. May 2023 · 11:04 UTC
I can't vote for you but it's not a game, it's a tool. But if I could you would be the winner, what a tool! As a PhD in artificial visio, I congratulate you. Also, my games are 8 bit pixel art (like mine from this LD53, Frogivio) and 16 bits and I have to admit that the result is fantastic. What surprises me is that there is always a black border, can it be that there are results without that black border always? I say this to add more variety. The tool would be great for creating items too. And if you mix it with a prompt it would be spectacular. But come on, I can only sincerely congratulate you on the work, it's not easy. What's more, none of the professional AIs get pixel art animations that I can remember, and above with several animations to choose! I insist, my most sincere congratulations!