Back again with the 2nd bi-weekly report.
These weeks were spent fixing the server, optimizing, and continuing to make things better. Try browsing around. The website should feel faster than it ever has. I’ve only just started. 
Related: Digest Week 1. Changing the Hashtag.
The Future of Ludum Dare Part 1, Part 2, and Part 3. Suggestion Thread.
Server RAM Upgrade, lower monthly cost
Of course I didn’t install it myself, but it did take a couple days back and forth with our host to get it sorted out. We now have 12 GB of RAM, which used to be 8 GB. And at the same time, shaved $35 off our monthly cost for our dedicated server (now about $189 /mo, was $224).
I agree that we should be spending less, but remember we have 40,000 users and 7 years of data on WordPress, a piece of software designed for maybe a dozen users max. I’m amazed it works at all. 😀
Big Software Updates, Fixed Legacy Plugins
Apache 2.2, PHP 5.5 (was 5.2), and MySQL 5.6 (was 5.1), we’re now running the latest stable version of each. Some folks may have noticed a bit of downtime last weekend. Turns out some of our WordPress plugins weren’t compatible with PHP 5.5. If you’re reading this, then it’s fair to assume I fixed (or disabled) them.
Nitty gritty: PHP introduced something called “Strict Standards” some versions ago. WordPress enables this by default, and in early PHPs wasn’t an issue, but it is now. Something that happens over 7 years of running a WordPress blog, you pick up several plugins that are eventually abandoned by the authors.
More grit: I don’t want to do anything drastic like switch to Nginx or MariaDB yet. I totally want to do it, but what’s most important right now is that everything is still working. The PHP 5.5 and Apache upgrade was enough incompatibility for now.
Zend OPcache Installed (PHP)
One of the nicer optimizations, OPcache is bytecode cache now included with PHP 5.5+. Before, PHP scripts would have to compile every time before they ran. Now they only need to compile once. Ahh, so much less wasteful.
APCu User Data Cache Installed (i.e. Shared PHP Memory)
This is something I’ve wanted for a long time (and sorry, I’m gonna nerd out). By design, PHP scripts are isolated from each other. Typically, if you want to share data, you write it to a database (MySQL, etc). Talking to the database can be slow though. Indexing aside, communication with the database is done over a TCP connection, even if the database is on the same machine!! There’s also software like Memcached that lets you store shared data, but you also talk to it via TCP.
As a gamedev, one that’s done his fair share of low level assembly and graphics coding, I am not a fan of this slow way of thinking (variables over TCP). It makes total sense for scaling up to multiple servers, but is so wasteful on a one machine setup. Heck, it’s wasteful on 2 machine setups too!
And that’s where APCu comes in. APC used to be a popular OPcode and User Data Cache for PHP, but now that OPcache is bundled with PHP, it’s less important… EXCEPT FOR THAT USER DATA CACHE! APCu is the now standalone User Data part of APC. And as it should be, APCu allocates and stores your shared data right on the Heap, and is directly accessible with just a PHP function call. No time wasted with encoding and sending request/response packets, the data is right there.
That said it’s not perfect. I’ve filed a bug regarding a poor uptime problem I’m having (minutes to 2 hours before a reset). It’s still ridiculously fast though. And if we can get this uptime bug resolved, it’ll be perfect.
Page Query Optimizations
When I started looking in to it, ludumdare.com/compo was wasting a lot of time. Some bigger SQL queries, but more importantly a lot of queries: about 43 total (potentially 47, but WordPress does cache queries internally).
There were redundant queries. The website theme was based on a very old version of the default theme that shipped with WordPress (7 years ago). That stupid theme made 3 wasteful queries in the header. That code is soooo gone now.
Some of the bigger queries were my fault. The Steam Widget, I do an ‘order by rand()’ in my query, which has shockingly terrible performance. Also there’s something wrong with the top-donors widget (lacking useful indexes maybe?). Either way, I made them store a cached copy of their outputs in the APCu storage. The cached data expires after a minute or five, so technically they’re not fixed, but running a query once is far better than every time.
Also the black-header-bar at the top is now gone. It’s convenient, but the absolute worst query of all has that to blame. Some stupid comment validation check, which might be fine for sites without 70,000 comments. I’ve since created a replacement (see below).
And then we get to the meat. Remember when I said WordPress wasn’t designed for communities as large as ours? Funny thing though, they almost got it right. The way it generates post lists and threaded comments, it uses a query to retrieve all posts and comments… but then if you want to know anything about the person that posted the comment, it’s doing 2 requests per user! That’s for all posts, and all comments! The part they got right is that there actually is an undocumented function available to pre-cache a list of users by ID in a single request. So if you crawl the lists of posts and comments, collect a list of IDs, feed it to that function, then it will cache all the user data for all the users quickly in just 2 queries. That way better than 28 queries. 😀
So on average, many pages require 14 (or fewer) queries to complete, but the root ludumdare.com/compo page is still about 17 queries. Less than half of where we started.
There are several pages I still need to clean up, but under 15 queries is what I’m aiming for. After that, it’s going to require outright changes to WordPress to optimize more, and even then I think we’re better off replacing it outright. We’re still going to need WordPress for a while though.
File Permissions Bug
I think this one’s fixed, but do let me know if you see any more permission issues cropping up. Some of the security stuff I set up with Apache made things more strict.
ludumdare.com redirects to /compo/ now
Just a minor tweak. Rather than fix the hub website, since all the work is going in to the compo website anyway (the goal of making it easier for everyone), why not just go direct?
Again, moving everything in /compo/ to root will break the internet. A lot of links will just stop working. Yes I could do some Apache rewrite rule wackyness, but I’d rather keep all the “old website” stuff in /compo/. That way, when we do migrate over, I can better track URLs that need to be send elsewhere on the site.
New Hastag #LDJAM, more supported
Like this post says, I’ve gone ahead and changed the default hashtag to #LDJAM. All official tweets will now use #LDJAM, and #LDJAM will be the tag we recommend.
That said, the Twitter Widget actually supports multiple hashtags: #LDJAM, #LDCompo, #LD48, #LD72, #LDtv, #LudumDare, #LDGame, and #MiniLD. It also supports some silly ones: #LDJelly, #LDKittens, #LDPotato, and #towlr.
In the future I’d like to do more with Twitter and social media support. I’m glad I could do at least this one little improvement.
Page Header Cleanup
I’ve wanted to do this for a VERY long time.
The colors still need work, and I’m sure I’ll be forever tweaking it, but I hope you agree even in its current rough state it’s an improvement. Let’s compare:
Tell me, do you miss the old header? (I might, a little… but just a little)
Event Boxes
These replace a lot of the information scattered across the header. Gone are the ugly red rows of 63 days, 13 hours, 24 minutes, and 18 seconds, replaced with a nice compact digital’ish countdown clock contained within an informative box.
And just to be crazy, I’ve included an actual start-time guide in Ludum Dare 31 box. This also meant I *cough* had to learn a thing or two about UTC time. You lazy fake-‘merican Mike. 😉
The clock code I rewrote from scratch, so please let me know if you see anything weird.
The Login Box
When I disabled the WordPress Login bar (due to it being HORRIBLY designed, see above), that meant we needed a replacement RIGHT AWAY!
So, I took the opportunity to kill a flock of birds with an awesome rock (wuht). Welcome to the future, where the website can detect if you’re logged in or not! Instead of the “Sign in/Create Account” link constantly found in the header bar, welcome to the future: Context Sensitivity.
And then when you’re logged in…
Yes, it’s the stupidest thing, but it just goes to show how little time I had to work on Ludum Dare before this crazy project of mine. The little things get neglected… for YEARS.
Clicking on your name takes you to your profile page, and +New Post does something (what could it be *shrug*). Finally, clicking your avatar takes you to gravatar.com, something we used to link way down in the sidebar (where nobody saw it). Hopefully this means we’ll see more avatars and less blank faces.
The rest of the header
I’m trying out the navigation bubbles. I’m not entirely happy with the color scheme yet, but at the same time I don’t want them to overpower things.
The “What is Ludum Dare” (about page) and “Rules / Guide” buttons, I want them to be obvious, but I’m still not happy with them. Not to mention the content they link to still needs work.
Posts and Activity
I’m still working on this, but I think it’s a good start.
Post minimizing is a feature I did add earlier this year. One of the more frustrating parts of the site is the clutter that builds-up in this part of the site. But when there’s important things to say, it has to go somewhere (here or in the header… annoyingly). Anyways, minimized posts look a little nicer now.
A visual thing: I’m no longer using pure white on regular posts backgrounds. Special posts have always had their off colors, but now regular posts have a slight grey color to them. And had I not pointed it out, many people wouldn’t have noticed. That’s the magic of color. And now that we’re off white, white things in posts will stand out better. That, and I can do the subtle shine seen along the top of all posts now.
Also gradients. I like gradients. 😀
The #LD48 website… Oh, sorry. The #LDJAM website is looking a look curvier.
— Alex Taylor (@tayl1r) October 27, 2014
Oh yes, I’ve been rounding everything too. 😀
One tiny feature I snuck in today: Post ages. Long story short, I didn’t want to have to do the math in my head for “when did I post the Hashtags post?”. I shouldn’t have to do math. Let the computers do the math! And now I know it was 5 days ago. 😉
The Event Navigation Bar
Finally, the last thing, also the first thing in the picture above, the new Event Navigation Bar. It’s a bit rough right now (just to support the October Challenge), but I’m going to be doing a lot of work here very soon.
Coming soon: This little widget will soon be the hub for quickly getting information about your submissions. How many comments you’ve received, how many ratings, how many more before you actually place. In general, the whole idea of Coolness really needs a major overhaul. We’ll see how much I can do.
Coming soon: For users that aren’t logged in, the bar will soon unroll showing either a random selection of games from the current event, or a random selection of recent winners after events. One of the biggest complaints about the website is how terrible it is for finding games. So, once I’m a bit further along, random visitors will now immediately see games.
Funding
Well, I never did find the time to finish the automated stuff, so I manually filled in the numbers. We raised almost $1800 in the days before October started, and are close to $1000 for November. Like I said, I’m going to keep working at it for the rest of the year whether we raise enough money or not, but it’s looking like we’re going to have to consider sponsors.
Ludum Dare 31 Kickoff
Well, it’s almost time to start the big countdown to Ludum Dare 31! This Friday marks the 5-week mark. I’ve opened Theme Suggestions early (secretly), but Friday will be the official countdown kickoff day. If you have a Ludum Deal for us, contact me. The Deals go live Friday.
Art Jam?
If everything goes according to plan, this will also kick-off on Friday. We’ll see. Part of the *new* 5 week countdown extravaganza.
TODOs
Now that my “uh oh’s” are out of the way (mostly, the big server upgrades), I can give a clearer picture of what comes next.
- About Page – Needs Work. I’ve roughed it out, but needs work.
- Rules Page – Needs work too. I’ve had folks offer to help work on the page, but it doesn’t help that I might be changing a lot this upcoming LD. Effectively the same, but I’m hoping to introduce “opt outs” very soon. This is to better support pre-made assets, either by you or from 3rd parties.
- For the sake of explanation, I’m also planning to flip the order of events around, as the Jam is simpler and easier to understand. Come for the Jam, but if you’re feeling hard-core, stay for the Compo.
- LDtv Widget – The Twitch widget is fine, but you all want the new LDtv widget am-I-right? Well, I do anyway. As you can see on the LDtv page, we have support for Hitbox and Youtube Live as well, but currently you have click over to the LDtv page to see them. Not to mention, slots for just 4 live streams isn’t enough. I’m hoping to find time to clean up what we have, replacing the 4 streams with a nice side pop-out of more streams, more info, more TV-ness.
- New Submission System – This is the big one for me. I’m really really reeeeeeally hoping to get this going ASAP. It’s currently the thing holding back the Art Jam, since I want to use the Art Jam to test it out. Some things I’m looking to support: Multiple submissions (to make it more versatile, for things like the Art Jam and Tools page), opt-outs, user ratings (in addition to judging ratings), and eventually: Linkage. Eventually I’d like there to be a master database of development tools, added the same way as games (but to the Tools page), and I want it to be easy to say “my game was made using X, Y and Z”, which links back to those tools. Once-and-for-all, we’ll finally know how many games are made in Unity, and be able to generate all kind of useful data. Heck, we’ll know for certain what the most popular tools for creating assets are, and could even start tracking the growth of new tools. Again, I use tools as my example, but this flexible submission system is actually the root of my website redesign plan. 😉
- Search Box – No, this isn’t the magical search you’ve all been asking for. Rather, when I removed the login bar, I also removed our only way of searching the website for posts. Oops! I need to re-add this.
- Event Navigation Bar – I’m repeating myself, but I’d like to make a lot more information about your entries available to you immediately after logging in. It’s not quite the long desired “notification system”, but it’ll hopefully hold us over until the rewrite.
- Comment Styles – Speaking of rounding, I’ve neglected something big: the comments. Really there are a number of parts of the site that need some effort to make them more consistent with the new “look” I’m making. I can’t do much with the WordPress-Admin/Posting interfaces (not really), but I can make the rest of the site pleasant.
I hope to tackle all of the above in some capacity in the coming weeks. Oh, and running Ludum Dare 31. That’s kind-of a thing I should do too.
If you made it this far, congratulations! You win at listening to the ramblings of a mad-man.
And with that I’ll say adieu. Until next time.
