Hue Shifting with Transforms

Recently I had the opportunity to fiddle with some image adjustment algorithms to show an approximate preview for a texture on a mesh which among others included a hue shift.

Of course shifting the hue is no big deal, there’s already of plethora of online resources that demonstrate how to change a RGB color to HSL and back which makes hue shifting as easy as incrementing a value, which is very nifty of course, however let’s assume you already have a setup that computes a matrix to adjust brightness, contrast and saturation which is sent to a shader, this is where things get interesting.

Now the obvious solution would have been to modify the shader, pass the hue value do the adjustment and be done with it, however that broke backwards compatibility, so this was not an option. Luckily, HSL isn’t just a formula, once you try to visualize RGB as a coordinate system instead of pixel values, you start to realize that a rotation on the (1,1,1) axis would essentially create a hue shift, HSL is only a convention to bring the Z (or blue in our case) axis to the (1,1,1) direction so you can make the rotation.

So let’s consider that Create*Rotation would make a rotation matrix on the specified axis with the specified angle and that MultiplyMatrix multiplies two matrices leaving the result in the last argument, then the following sequence will create a matrix adjustment for a hue shift:

CreateXRotation(xrot, PI / 4)
CreateYRotation(yrot, PI / 6)
CreateZRotation(hueShift, hue)
CreateYRotation(invyRot, -PI / 6)
CreateXRotation(invxRot, -PI / 4)
MultiplyMatrix(xrot, yrot, tohsl)
MultiplyMatrix(tohsl, hueShift, shiftedMat)
MultiplyMatrix(invyRot, invxRot, torgb)
MultiplyMatrix(shiftedMat, torgb, adjustment)

Which gives us an adjustment matrix that we can premultiply into the existing matrix, problem solved and we keep backwards compatibility.

One thing to be noted is that hue has to be in radians, the most intuitive course of action is to let users specify it in degrees then turn it into radians before creating the matrix, but that is just a minor detail.

And there you have it, an easy to use transform to implement hue shifting when working with HSL values is not an option

Troll Game Jam: After the rush

Before I start getting into the nitty gritty details, I’d like to warn you that this post will be focused on some very technical, albeit awesome, details regarding cross platform development.

Now that you have been warned, let’s start where we left off, Snake Fight is a shiny neon shooter that adds the extra mechanic from the game of snake to make it more challenging. Gameplay wise there isn’t much to discuss, there’s one of you and a lot of baddies that shoot lasers and make colourful explosions.

Behind the curtains however, the first interesting thing is the way the game adapts to the current platform it’s being played on. My unofficial goal was to make an experiment to see how hard it is to develop something that can be played on anything (well almost anything, I don’t think there’s any chance of the Hurd being supported in the near future). The first problem that needed to be solved was resolution. Resolution actually spawns two problems:

• Getting a consistent positioning for every possible resolution.
• Stopping graphics from getting all smudgy from resizing

My approach was to stop thinking of the screen as a big rectangle comprised of a variable number of pixels, but rather as a rectangle that can span from 0.0 to 1.0. Basically instead of using absolute values, I used a percentage to describe the size and movement speed of the playable characters. The disadvantage here is that as the aspect ratios vary quite a lot, you will inevitably get unwanted stretching. My solution in this case was to make the percentage dependent only on the smallest size the screen currently has.

First problem tackled, there was just the issue of resizing assets that got fuzzy if they were stretched too much. I solved this one by generating all my assets at runtime from basic geometrical shapes. Luckily enough, the components I decided to use to build the game have an API that solved this for me, so I didn’t have to create wrapper over the drawing API to make squares appear on screen.  This also has the advantage that if there’s ever a need to move towards actually drawn images, I could just use the my shapes as target rectangles to blit the images. Of course, I’m still hoping that the nice shader bling will be enough to distract players from the lack of proper hand drawn graphics, but if not it feels good to know there’s an exit plan somewhere in there.

That being said, even though the game feels like it’s nearing completion, there’s always a little extra something that needs to be done before it feels good enough to be released to the public, there’s still some core graphics and input issues that need to be ironed out first and then there’s also thee need for some sort of graphical representation of your current lives and active effects, but that’s a discussion for another time.

For now, here’s a teaser:

Troll Game Jam: The Result

As it turns out, I was wrong in assuming that I was the only one actively making a game for game jam mentioned in the previous post, I’m still going to name it troll game jam though because even with my doubts aside, there was a lot of trolling involved.

In other news, the result of the game jam was a small game that for now I called Snake Fight which is based on an older idea of mine of a snake + space invaders mashup, you can see a little teaser over here.

The game is playable on anything in theory, in practice it’s tested on linux, windows and with the cost of some reduced bling even android.

As always any form of criticism is very much appreciated.

Troll Game Jam

I tend to really like game jams because I always learn a lot of things during the 24-ish hours of non stop development, so when a friend told me he is organizing a small LD like competition, my coder senses went all tingly.

I’ve long been an advocate of diversity, so just for the fun of it I wanted to see if I can do a game based on Qt that could run on anything (anything as in every platform that supports Qt), so this time around I fired up QtCreator and started making the game.

Interestingly enough, it took around 6 hours to implement a standard game of snake and another 6 for the rest of the gameplay,  and I say that because everything is made in C++. When I started game development I transitioned to interpreted and JITed languages because pushing pixels in C++ was a very long process, even with existing frameworks, maybe it’s the development experience I’ve gained over the years, but that no longer seems to be true, so it’s definitely something that I’ll explore in the future.

Back to the game though, the idea was to create a game so buggy that it becomes a challenge to play it. I think I’ve managed to do that to some degree, however if you don’t want to take my word for it, you can download it here

The classic game of snake with lasers

I’m calling it a troll game jam because the unofficial focus of the jam was trolling the other opponents, and since at least for now I’m the only one with a playable prototype, I think I might have lost that round, but on the other hand something pretty fun came out of it.

Android support soon to follow.

7DRTS: The Game

It’s been a while since I’ve announced that I’ll be participating in 7DRTS and I’ve been hard at work thinking of blocky graphics and gradients to fill the screen under a shabby pretense for the general amusement, and this is what I ended up with.

The showdown begins

Jokes aside this game isn’t necessarily about war itself, but rather about consequences, the focus of the game being to be able to balance your resources properly. You will be able to win the war quickly if you go all out, but on the other hand the remaining survivors won’t be able to find enough resources to survive.

The game itself will feature in the beginning just 2 opponents inspired from Zapas’ childhood creations prism and cube (Yes, your own characters are competing against you!). And four main mechanisms:

• The bomb: Atomic bomb, decimates the competing army, but also uses up a lot of resources.
• Poison: Stop resource regeneration for the competing army, but also uses up most of your resources
• Spur: Increases infantry production rate
• Conversion: Essentially a slider that decides how many resources are converted into infantry

Your resources will gradually drop by a rate dependent on the total population of your country and when they reach 0 they cannot be replenished anymore, so it’s essentially game over.

The goal of the game is to survive for as long as you can and to make it  a little bit more interesting, you cannot do anything to control the resource fluctuation while you are at peace.

That being said, let the competition begin. I’m gonna go grab a cup of coffee.

The Challenge of 7DRTS

The middle of the summer is one of the greatest moments of the year. Apart from being an arbitrary point in time, for some it’s a marker that pinpoints the exact moment when getting a vacation seems like a good idea and for me it’s the exact point in time when the nice people from ludum dare give a go to competitions.

This time it’s about making an RTS game in 7 days, or 7DRTS for short and the scope as @sorceress explains in the official 7DRTS announcement is to make a RTS (real time strategy), TBS(turned based strategy) or just some jumping pixels requiring minimal interaction, doesn’t really matter, the important thing is to have fun.

This competition however is special because with this occasion Zapa and I decided that it is time to revive the tradition of break design challenge in a rapid exchange of tweets

So @zapakitul, we meet again.

Overall I think it will be a pretty fun thing to do as we’re restricting our actual game dev to 2 days and therefore expect the competition itself will probably be pretty tight, but mostly because I will be doing something that involves networking and differential equation models for game balancing. I’m still figuring out the details, but I will be explaining all implementation details, so you might want to come back once in a while during this challenge.

Also, everything will be hosted on github, so you can follow the repository from now if you want to, I plan to support at least mac os, linux and windows, but iOS and Android might also be a possibility. I don’t plan to upload builds there, I will be putting automatic compilation scripts, but if you just want to play the game without the hassle of compilation, leave a comment with a way to contact you and I’ll try to send a compiled binary.

And last, but not least I wish everyone participating good luck.

Direct Show and COM

There comes a time in every programmers’ life when they take a webcam and have the idea to programatically save the stuff it’s filming for hilarious purposes. Usually such purposes are best suited by some (semi)interpreted language that offers you a very high level API which abstracts away the low level details of begging the OS for a buffer with some pixel values in it, but sometimes you don’t have the luxury of assuming that whoever is going to use your stuff has the latest and greatest in terms of raw computing power so you have to go down a few levels until you reach the inhospitable land of platform dependent APIs.

To be more specific I’m talking about DirectShow here of course, other platforms will probably soon follow as I go through them, but for now let’s focus on this. I’m pretty sure that at some point during the development of windows, programmers sent angry complain letters about how they write too little code to make things work and how their bosses always think they do nothing because of that. And that is how COM was born.

The idea around COM isn’t all that bad though, with C++ not having an ABI and all, this is the next best thing, you query an interface you know that the calling convention is stdcall, strings are always wide char unicode, you have the refcount and a separate COM heap to stop worrying about incompatibilities between runtimes if somehow you ended up deleting an object someone else gave you, all in all a very good framework in theory. Practice however teaches us that in this case abstraction lead to more abstraction and it’s pretty easy to lose yourself in the layers.

So back to our goal, the first thing you need to know about DirectShow is that it has a display graph, the graph is made out of tiny black boxes called filters. Each filter has one or more input pins and/or one or more output pins that you can connect to each other. So the first thing you need to do is get a filter that will connect to a device and output the video. That’s pretty easy, you just call BindToObject and you’re mostly done.

The next step is the tedious one, first you have to implement a IBaseFilter, keep a FILTER_INFO structure in your class and fill it in JoinFilterGraph with the arguments you receive, the rest of it should be straightforward. Unfortunately for the filter to actually work you have to implement everything here, but most of the functions are 1-2 lines of code.

After you’re done with the filter, you have to start with the pins. I used a pin for audio and a pin for video because I don’t really like to mix functionality, it’s your choice how you go about it, but in any case, to be able to save video/audio data, you have to implement the IPin and IMemInputPin, now since neither of them were declared with virtual inheritance the easiest way is to use aggregation and have a pointer to each other in the implementation (IPin has a pointer to IMemInputPin and IMemInputPin to IPin). I used a normal pointer as a weak reference for the IPin in the IMemInputPin and a CComPtr in the IPin so that I don’t have to worry about the ref count.

The good part is that for the purpose of capturing stream data, you don’t actually need to seriously implement everything in the interface, you can forget about the flush methods, just return S_OK or something, EndOfStream, Connect and NewSegment are insignificant as well, QueryAccept should be implemented to accept everything, you’re far better off controlling the data type by querying the camera. For the IMemInputPin, tou can forget about the allocators. In the ReceiveCanBlock you need to return false, and the receive part is where you do your actual saving. The only thing you need to watch out for is that the IMediaSample you receive sometimes returns S_FALSE when you ask about the media type, it’s their way of saying that you need to use the last known media type, which is most likely the one you received when you first connected the pin.

Fortunately after all this is over and you realised you just spent two days of your life implementing interfaces that you aren’t sure will work, the graph system we learned about earlier will figure out by itself what comes where when you call the RenderStream function, you just need to call AddFilter for the filters we implemented earlier. If you’re lucky and you considered everything it will just work and you should see some rgb data saved to disk (I recommend the ppm format for rgb stills and y4m for yuv movies). If not then the first thing you need to do is see if every component is being initialized, if the ReceiveConnection is called in IPin, if the Receive function is called in IMemInputPin and so forth.

There’s also the problem of having to make the enumerating interfaces for the filter pins and the pin media types. If your goal is to only capture video / audio you can make a filter for audio capture and a filter for video capture and have both of them contain a single input pin that will save the streams and you can have the enumeration class just return a pointer to that pin which will very much simplify the implementation. For the media type enumeration, return a AM_MEDIA_TYPE with a null pbformat and a GUID_NULL for every guid in there, this way you tell DirectShow that you don’t really care about the format of the pin that connects to yours, so you avoid some other problems later on, if you need to choose a format you can use the IAMStreamConfig interface on the filter you get from the bind to select a configuration that suits your needs.

Oh and if you need to allocate or free memory always use CoMemTaskAlloc and CoMemTaskFree.

Conclusions? It will take you around 1000+ lines of code and 2 days to move some YUV and WAV data from memory into a file.

Watch out for the APIs

There’s an old saying in the computer industry that says you should be lenient with what you receive and accurate with what you give, mainly it means that when parsing data you receive, it’s not uncommon to receive bad input, so it would be nice of you to either tell the program or person that generated the error that they did something wrong, try to correct the data or use a default.

Back in the real world though, writing software has taught me that this is just saying that gets ignored to several degrees by almost every API. My guess is that once the spec is done, no one actually bothers to actually whatever comes out. This post serves as a list for the problems people seem to ignore when they  make an API for C++.

Let’s start with a little bit of math, for those not very interested in 3D concepts, a Quaternion is a 4 element vector that you can turn into a rotation matrix. It basically represents the rotation around an axis, so when you see a Quaternion class with a constructor that looks like the following:

Quaternion(double, Vector3&);

your natural reaction would be to assume that it initializes the new quaternion so that it represents the rotation specified by the double argument around the axis represented by the Vector class, right ? Well as it turns out, it more often than not it just copies the values into its’ members. How on earth is that more useful than something that it was created for is beyond me, but that’s probably because I’ve actually used quaternions.

Moving on to some geometry, when you’re working with images you’re bound to run into something that requires coordinates. Now there are a lot of ways to say where an image begins and where it ends, but fortunately only two of them are used in practice. Most of you have probably seen the one where coordinates start in the upper left corner and go toward the bottom right as they increase, so for instance if you have a resolution of 1280 x 800, then the bottom right corner is (1280, 800). The second one is the mathematical one where images start in the bottom left corner and go towards the upper right, however this one is seldomly present in libraries, so if you insist that the Y axis should be up because that’s how things were in highschool, then at least leave a comment to let the rest of the world know that the top should be bigger than the bottom of the image in order for something to work.

And last, but not least, never ever use STL in an API, ever. Containers are awesome for programming in general, I agree, but when you want to send data from one dll to another you have to keep in mind that the people making the other library are sometimes using other tools than your own, and containers are almost never compatible between toolsets, something always crashes sooner or later unless everyone uses the same compiler and linker, in theory that’s not much of a problem, in practice however you’re often forced by the environment you’re developing for to use certain tools if you want to be able to debug your application. So in the end it all comes down to either using some really awkward hacks to be able to accommodate the library or dropping that library altogether. This is why nice, clean APIs are usually made in C, well that and no name mangling to take into account when importing functions by hand.

So yeah, making an API isn’t as easy as one would originally assume, however if you have some patience and are lenient with what you expect from it sometimes it’s not that bad, after all everyone makes mistakes, they’re ok so long as you learn to recognise them early.

On the state of cross platform development

I’ve been a bit lately and while I can’t complain about anything in particular, my time dedicated to example coding for the blog hasn’t been exactly stellar.I do however try to keep a schedule of a new post every now and then and since my activities have brought me to cross-platform development research, I’d like to share a few thoughts about it.

The first thing that everyone thinks about nowadays when you say cross platform is HTML5 and it’s true that there’s no other alternative that can come close to the plethora of platforms that you can reach with that technology simply because getting a consumer electronic device out nowadays without including a browser in it is somewhat of a niche market. It’s also true that you can get some pretty nifty stuff out of it , except the obvious canvas, CSS transitions and shaders come to mind and the army of developers working round the clock to make browsers lightning fast also helps you as a developer since they will basically do all the optimization work.

All of this is of course amazing and to be honest most ideas an indie developer could create in a decent amount of time could probably give a decent experience, maybe with 1-2 optimizations that you can learn from the awesome guys at netflix, but this is something that most people already know and is well documented on other sites that have better graphics than mine, so I’d much rather talk about the next best thing.

Another alternative worth taking into account when trying to build a product that runs on almost everything is Qt. It’s probably not for the less experienced indies as the preferred language for development is C++ (and you will have to use C++ if you want your stuff to be cross platform) but on the other hand it’s still a UI toolkit that actually works and compiles with 1-2 clicks on everything that’s worth bringing into discussion. The good thing is that it has the cleanest API you can ask for and native capabilities are of course a plus (device discovery over LAN comes to mind), so there is still a gap left by HTML5 that Qt can fill, however the biggest drawback is that it’s not exactly backed by the same same amount of developers working on webkit, so the ports created for mobile devices are not exactly matching the same level of functionality you can get from the framework on desktops.

It’s slowly getting there though and it’s gotten to a point where it might be worth it to try and see if you as a company can make one Qt app and adapt it for all the platforms you intend to support (did I mention you can style controls with CSS?) since at this point making a mobile build is as easy as adding a new target to the project.

In other news I just made my tile editor run on tablets so I can make levels on my tablet and play them on my PC. The other way around is too mainstream.

Happy 2013

I have a feeling this year’s going to be the best yet.
Game on!