Kinect Connect

So I haven't posted for a while, but that doesn't mean I haven't been doing anything.  Quite to the contrary, David and I have been pushing ahead full-throttle with the Microsoft Kinect.  For this coming quarter I am beginning my gaming capstone development project, which will take 20 weeks with two other team members, I am working hard on Zombie Yoga with David to get pose detection working, and we are going to jump into Game Jam at the end of this month to develop something awesome.

So yeah, a lot of games, a lot of work, and not much time to do it all and my homework.  Oh dear.

F*#k

Anyway, where shall I begin?  Where did I last leave off, anyway? Dang.

Well let's start with the Kinect, then.  Kinect in it's current state, or rather using the current SDK that Microsoft has released (The Kinect SDK beta) returns raw, unfiltered, and mostly really, really, really, really wonky data.

Like having a conversation with the Kinect

Now, to be fair, the Kinect is an awesome and amazing tool, and given what they have to work with, Microsoft has done a spectacular job.  I could sit here and grumble about the Kinect having issues, which is what a lot of people might prefer to do (the let's-rag-on-what's-popular-to-rag-on mentality) but the truth is what they have done with the input the sensor gets is phenomenal.
It's just not perfect like many developers would prefer, because that takes all of the serious effort out of our hands.  No, we have to do some of the tweaking and fiddling ourselves.


So what David and I have decided to do is kind-of-sort-of what the Kinect already does, brings together an infrared depth image and pair it with some bone data.  The Kinect software does the hard part and using the information it gets puts together a "skeleton" structure that represents a player's body.  The only problem is if my limbs go off-screen, or begin to cross, the Kinect just loses it's marbles and spends the duration of the limb-crossing by comically running around and bumping into tables while trying to collect the said stray marbles.

Now, for Zombie Yoga we want the Kinect to register poses, and the thing about Yoga is it's not all nice and planar, some of the yoga positions people can get themselves into are downright nuts.

Like cramming a monster into my homework assignment

Solution?  Take what the Kinect gives us depth-wise, split that into individual chunks of depth-based objects that know where they are in relative 3D space, and then check those against the bones that are whacky.  If where we think a bone should be there is an object of appropriate size at about the right depth we can tell the program "hey, this crazy bone-joint that is freaking out, it should go here)



That's the gist of it, anyway.  The actual implementation is going a little bit slower.  We are just about done with the code that checks out the depth image and comes up with a bunch of depth objects from said image.

Again, the Kinect and the SDK most of the heavy lifting, it scans the environment, picks out up to 7 players and sends back pretty darn accurate depth data pertaining to how far each player is from the camera (and not just the player, but every single part of the player, it can tell the difference between your nose and your forehead, which if you have a bigger nose like me isn't too impressive, but trust me)

Snakes in a Notebook.

Anyway, this part might be boring, but just how you go about extracting depth information from the depth image isn't as simple as just looking and saying "oh, look, there's my hand!"
It's easy for us, because we have eyes, but computers, like justice and awkward dates, are blind.



The trouble was being able to break apart one large image into a bunch of smaller pieces of data, without spending a whole lot of time on the image.  Basically, when a computer displays or processes an image it starts from the top left and goes through the entire image row by row.  Pixel by pixel.  That's how it has to draw.  While there are lots of "fancy" computer concepts such as multi-threading and other sweet processor behaviors, essentially a computer operates linearly.  Line by line, instruction by instruction.  One at a time.  It just does it awesomely fast.

Anyway, the problem is when you only get one pixel of information at a time, how do you, quickly and efficiently, begin to group pixels that share the same properties together when you only have information pertaining to the current pixel?

Through the power of dragons?

Well I'll skip all the ways my logic failed and give you the good version: As I run across "depth pixels" of a similar depth I keep track of the first pixel I saw that matched my depth description, and as we go across horizontally (how pixels are read, one by one, row by row) I keep track of the latest pixel that met my requirements.  Then, when I find a pixel that doesn't match what I was looking at, I kick that last line to an object that maintains a grouping of what I called "depth objects" and the line glues itself onto a depth object that the line is directly underneath, or it creates a new depth object that will be looked at during later passes.

So why is this good?  It sounds stupid, right, adding things on line by line?  And keep in mind, these can be short lines too.  When I go through a row of pixels I will probably find several different lines of different layer depths in the depth image.  Why just not look to the depth pixels immediately surrounding the one I'm currently looking at, and letting it create a depth object by expanding outwards and checking outlying pixels?

Several reasons: It doesn't maintain any sort of order, my depth-checking of an individual pixel just beings expanding out to the surrounding pixels, and how do I know if I've found enough? That problem is likely to also be recursive, which if you are checking nodes for pathfinding maybe isn't so bad, but in my case I'm checking a 320x240 depth image, which if you do the math is 76800 pixels.  That's a lot of ground to cover, and even when I do find the edges of this certain depth I want to check, how do I find the next place to start checking?  I may very well just be jumping around to random places on the image checking all the pixels in no particular order with recursion, running up how much stack space I have left.  And this is happening at about 24 frames per second, on average.  Sure, computers could handle it, but on top of running a game?  A 3D game at that? You might as well set your computer on fire as is.

So by just collecting information of lines and then sending the lines to sorting into their appropriate depth object we manage to build up an entire grouping of depth objects whose size and location match the different depth's seen on your computer screen when you boot up the Kinect Skeletal veiwer.

Wait, you can see it already? So why did we go to all that trouble?  Because the computer is going to have to look at that image every time the Kinect sends out an DepthReady read event (whenever a new depth image is ready) and it keeps track of only which depth pixels are how deep, not their mass or the average location of the mass, that's what the Kinect bone tracker is doing.. to an extent.  It's still gives crazy and weird data sometimes, which is why we went to all the trouble to do this thing.  And plus, it has given us some pretty crazy game ideas that you will see up here for game jam and my capstone project, so stay tuned!

Indeed.

-Kevin

Doodle crazy.

I haven't done any programing today... just doodled the heck out of myself.
Enjoy... or not, it's up to you, and your taste for my "style"

There's only 8, some more serious that others

In there tongue, he is Pierre, Dragonpainter!

Victory, messy, painful victory, but victory nonetheless

Just, like... chill, bro.

-Kev

I love the smell of napalm in the morning.

(Writing on the go, no doodles... yet!)
(Edit, Got a doodle in, scroll down for it, or, you know, read it and then see the picture)

Success, however you choose to measure it, is a fickle beast.  It certainly is no friend of mine.  Most often when I achieve success it isn't enough, and that is where I find myself again today.  After hours upon hours of struggling and using my most abundant resource, stubbornness, almost to exhaustion I finally found a solution to the issue.

The success is in the that my camera system is now fully operational, it works smooth as silk and there are no graphical glitches or processing slow down.  Good news for David, and the system as a whole.  It is now more flexible and powerful than ever.  So why am I not pleased with success?

It doesn't feel mine.  The methods I had built before were of my own doing, my own math and created by me and for me.  The problem was that my old ways were inefficient and messy.  And the fact of the matter is the new system I implemented, which is based off of previously existing systems in XNA (well, not just in XNA, in most graphics programs as a whole), is far better than what I had before, despite my affinity for my own hard work and code.

Work harder, not smarter, a maxim I am still learning to this day.  And it's an unfortunate truth that programmers, like anyone else who creates, get very attached to their creations and can be loathe to abandon all of their hard work even if a better solution is available.

As is, I ended up using an XNA class called RenderTexture2D, which is a separate buffer to draw too, and then used that entire buffer as a single texture and drew that texture according to the dimensions of the camera.  Buffer work is not something I've had a whole lot of experience with, but the solution I came to using the separate buffers feels almost too easy for the incessant slaving away that I've put into the camera system in the past few days.  And what I came up with today was almost too easy, and it took no more than an hour to research and implement.

I'm not sure if finding this solution makes me a good developer because I found the solution, or a bad developer because it took me so long to find this solution.

I guess that remains to be seen.

-Kevin

Endless looping

One of the worst errors you can manage to drum up in programming is that of the endless loop.  It's not the worst in it's difficulty to track or fix (usually) but in that it is a common pitfall that the "get-some" programmers like myself who charge into projects with guts and not a much else fall into.  I'm not saying other programmers who are more careful or intent don't find these nasty errors awaiting for them, but they happen, quite often.  A slip of the fingers while typing or a momentary lapse in logic is all that it takes for your program to decide that it's going to sit and keep doing the same thing over and over and over again because it said so.  Or rather, because I said so.

It breaks your program, without fail, and depending on the location of this error or what your loop was trying to preform, it eats up a heck of a lot of memory on you processor and slows everything else down.

The solutions to these endless loops should be evident.  You forgot to add a good ending condition to your loop, such as saying

do {
//stuff
} (while true);

That, while a very simple case, is usually what happens.  But then you get the times where the solution is not clear, and you spend hours searching through what looks like perfectly good code for a tiny error that renders all of your hard work obsolete.

Especially in the case of linked lists (which is a list of objects that know where the previous object and the next object are in memory, and nothing else) if you don't manage your memory correctly then you can lose your place in the loop, and it will never complete, sitting idle, lost in the maze of your computer's heap memory, unable to find the next link and move on.

That's a bit dramatic, but if you couldn't tell I'm attempting to relate this endless loop to something more than programming.

Endless loops don't just occur within the realm of the nerds, geeks or computationally-proficient-yet-socially-ineffective.  They occur in real life, too.  And they are every bit as bad, if not worse, as the kind that I run into while jumping feet-first into programming hell. (a.k.a. my creative process)

It would seem I'm prone to all kinds of ceaseless loops.  Making the same mistakes, acting in the same manner and preferring to stay in what I know as comfortable instead of stepping out of the hovel I've made for myself in life and doing what I haven't dared to do, and speaking when I have made myself err on the side of caution and kept my words and feelings to myself.

This is what happens when I try and program late into the night, logic leads way to philosophy and pragmatism melts into introversion.  At least let me say I envy the man who can claim to live without regrets, for he must be able to sleep much better than I, or he's a filthy liar but at least he exudes confidence, a point for him.

I guess while I'm here I should also update as to what I've done programming-wise, since this is the purpose of the blog, after all.

Part of the reason why I'm in the melancholy and reflective endless-loop mood is because today I spent far too much time trying to debug some of my old camera code, which I did not comment well and took me forever to add and extra feature.  It was something I had always planned on adding, camera rotation, but I had never gotten around to until David mentioned he would like to use such a feature in his up-and-coming project.  So I got it done.  There's still some stuff to iron out, but that's for tomorrow.  That and trying to finally get my stupid Kinect working with my computer.

Oh, technology.  I just couldn't have been an athlete, could I?
But then, I wouldn't be Kevin, I guess.

-Kev

Re-do!

Taking more programming classes and learning from, you know, experts, professionals and people who have been doing this for many more years than I have is a good thing, but it has some nasty side effects.

E.G. growing sleazy mustaches

I'm starting to care about things such as efficiency and clarity, and other under-the-hood stuff that I never used to give two thoughts about.

So now that I'm home and I've finally had time to get back to some of my own projects, I'm abhorred by how sloppy my code has been and my driving desire is to make what I have work better.

The result?  Two days of coding delays while I try to merge my new fancy collision system with my old code.

BUILDING PUNCH

What I had been doing wasn't exactly terribly inefficient, per se, it took what needed to be compared and then did the comparisons, but it did so in no organized manner.  Random sections of code in different places all did collision checks by accessing high-level objects in a rather sloppy mess of referencing.  There was no centralized means of checking for these collisions.  Solution?

Pizza.

Well I ended up changing everything around.  I took the collision checking and abstracted it up one level.  What does that mean?  Well I have a super-class called Sprite Object that does most of the heavy lifting of my game objects.  But for the specific game objects I must derive them from the base sprite object, which is like saying a lion is in the cat family, but a member of the cat family is not necessarily a lion.  So instead of doing all of my collision checking in "lion" for all the rest of my objects, I do all of the checking in the "cat family"

That puts all of the collision checking in one place which keeps the checking all very, very simple.  Before, if I wanted to check collisions I was creating new methods in each derived class to do checking against all of the other, different derived classes from the main sprite object.  For a universal necessity I was writing different pieces of code across the spectrum, which makes a bit of sense because an object has to react differently to different collisions.

So I gave the sprite object class a special identifier that marks the object with a certain collision value.  And each object also gets a bitflag value defining which other objects it can collide with.  Once a collision is detected, each object that has a valid collision receives some collision information that stacks up with each overlapping collision.  Then the object, in it's own time, will go through that stack of collision information and decide what to do with each different collision.

This means that, yes, I am still writing code in each derived class for what happens when a collision is registered, but the actually checking for the collision still happens at the surface level without going too deep into the program.

Elephant-charge the cannon-tower!!

This post may be sloppy and hard to follow, but that is my own fault for trying to watch a Christmas movie with family and write at the same time.  I get too focused on one or the other.  But it's vacation, and vacation time must be enjoyed for all it's worth.

I now sit at at the precipice of a very deep and daunting canyon.  It's like the grand canyon, only larger, deeper, and I do not know what's at the bottom.  On the other side lies both what I most desire and fear: completion, finishing, the end, wholeness.

And a face full of FIEAH! (SSB, anyone?)

This may seem like a strange thing to be afraid of, but if you know me well enough you know that I'm prone to incompletion. (Which apparently is not a word recognized by Firefox's dictionary)  I am far from a perfect person, but one of my many faults is the failure to follow through.  I can't begin to list all of the half-finished things in my life.  A million projects, both solo and with friends that I had/still have great faith in and admiration and love for.  Books, games, animations, and even some music, though of all the gifts I've received in my personal life musical ability as I have come to find is most certainly not one of them.  Even simple things such as chores and relationships I will leave undone or put in effort only to eventually retract it.  Especially on the relationship aspect I feel particularly bad, I know a plethora of amazing people on this here chunk of floating space-rock and I feel I have not given enough to them as they deserve.

...but nobody came here to read about that. (..Or did you? How can I know, this is almost like writing a letter to somebody with no pens, paper, and no access to Office Depot?)  So what I wanted to get at was now I have a completely functioning piece of interactive media.  Everything works as I have designed it, now what is left is the small things, adding a Heads Up Display, a menu screen, a means of keeping track of score, adding in controller support, fixing small bugs and polishing the game to make it look better.  I'm the nerdy equivalent of the guy at the gym who lifts a whole bunch of barbells and has massive arms but still has a keg for a stomach because he doesn't want to change his diet or run.  I can jump into a project and do the heavy lifting and feel great when I get the core things finished, but when it comes to the small things they pile up and I feel as if I won't be able to complete all the small tasks, and I don't feel the same rewards as getting a whole system up and running as I do when I add in a HUD or menu screen.  My drive begins to huff and puff, I ran the sprint but making games (and almost everything in life) is won by marathoners, not sprinters.  Anyone can run fast for short distances, it's being able to go the long stretch that counts.  I get interested in other things,  I think of other projects, ideas, or I simply get bored.  There's a good quote (and of course I'm going to paraphrase and not remember who the quote is originally from...) that says (approximately) to never have reasonable goals, as long as you strive to be better than yourself, you will improve.

This, though, I wrote this, I can remember that much.

That must be taken with a grain of salt.  Keep in mind it's easy to distract, discourage, or defeat yourself.  Most of my battles aren't against the world, but are against my own "dark side" which sounds a lot more awesome than the lazy, insecure and instant-gratification-seeking side of myself.

Yeah, I'm done with all the soul searching, my bad.




-Kev

Think I already used this one; not sure, too many doodles cluttering up Blog Space

Back to the swing of things

Srsly.

Evening, world wide web!

I love vacation.  I think vacation is the best thing ever.  It's even better than deep dish pizza.  And if you know me, you know I love that stuff.

Pic Unrelated.

But as much as I love sleeping in and stuffing my face (Aka vacation and being home) I have to be productive and stick with the rhythm of programming life.  So what have I done?  Nothing special or remarkable, just have kindof/sortof kept the ball rolling.

I created a very basic first build for my Zombie Yoga team, and I've received my very first ever Kinect (Bought refurbished off of amazon b/c I'm lazy and cheap like that) which means I get to start working with David on putting the Kinect to good use in our game!

We are building the game in the Unity game engine, which is a cross-platform development engine (literally, it will create games for mac, PC [perhaps ubuntu, but I'm speaking out of my butt right now] iOS, android and, if you pay big bucks, Xbox arcade, which is what we are currently banking on)

speshul tree for speshul kevin

Unity is, admittedly, really to use in a very very scary way.  For programmers one of the most important things they beat into our stereotypically-socially-awkward and stereotypically-glasses-wearing faces (I don't actually have glasses... yet, no comment on the social aspect of my life) is that all of your variables should be private unless you have a very good reason to make them public.

For those of you who have no idea what I mean by public or private, in code when you declare an instance of an object, or set up the foundation for an object in code, you have the choice to make your variables either accessible to the entire program (public) or to just that object and other objects of the same type (private).

If you make your variables, or the stuff that holds all the information in your object like health, or id number, or rank, etc. public then anyone anywhere who uses your code can suddenly get in and mess with your variables in ways you didn't intend.  If that was still too programmer-y for you, then the simple way of saying it is making a variable public is a very bad thing to do, and can break your code if other people decide to mess with stuff.

However, in Unity where everything (or almost everything) is based on drag-and-drop functionality and someway, somehow, everything has access to everything else, making your variables public is one of the only ways you can access those variables in the normal Unity editor.  The core game logic, such as the asset loading, using, rendering, and the game loops are all taken away from the control of the programmer (at least from what I can tell at this point) so where I would usually be able to pass variables to new objects upon creation (avoiding having to use public variables) in Unity when I drag and drop something into the scene the global set up is taken away from me, the instance pre-exists any commands that I'm used to being able to use during a game, so to rectify the situation I make variables public so that I can tweak their settings in the Unity Editor.

It gives me goosebumps thinking about how everything in Unity can just access everything else.  It's creepy for me as a programmer.  Then again I'm not programming wizard, others may see it differently, but let me use the following metaphor to describe how comfortable I am with Unity: I see unity as a very large and powerful rifle that descended from heaven to my lap during the zombie apocalypse that has one little flaw in it: every so often it fires a bullet that is secretly a tiny boomerang that comes back straight at you.

Aside from the fun I've been having with a full-fledged game editor, I've gotten back to work a little bit on my own stuff. David and I will be re-hauling what small fragment of a 2D engine I have now, but I'm determined to finish off at least one game with what I have (which I now see as an ugly mess of code, but that's what any programmer would see when he revisits a project he didn't properly comment/document)

And in that regard I've made a little bit of progress.

I've got a bunch of different buildings (see blue planet) and spaceship types up, now I just need to tweak, smooth, and slap a HUD onto this baby and see if I can get it on Xbox Live Arcade.  This is my pet project for the winter break, aside from Zombie Yoga.

I have all the necessary skill and knowledge to finish this game, it's just finding the time to put into it, I believe.
It's nothing big, but it'd be one small step for Kevin, one giant leap for... Kevin.  First game to be completed in, what, 5-6 years?!
Hope I didn't just jinx myself.

Cuz' all white whales look alike, right Ahab?  It's allllll about skin color with that dude....

Flowers, anyone?

- Kev

So uh... yeah, quarter's over!

I'm essentially writing to kill time.  I'll be home in a little over 16 hours, flight out of here is 10:30 and I land at 1:00-ish.

So, um, where to start this?  The beginning?  Well, not the entire beginning, I pretty much stopped posting around September... so October?

Wait, no... story time?

Taking four programming classes at one time was probably not the wisest choice ever made.  Taking two of those classes online was also not incredibly intelligent, but for better or worse, I have pulled through, piling other stuff on my plate as well.
But this isn't a blog about me, it's about the games I strive to create.  So what have I gotten done?

Does this answer you question? No?  Hm...

Well, pretty much nothing.  I'm now working on a game project lead by one of the professors here at DePaul, the title of the project is Zombie Yoga, I'll be taking on the role of a programmer and we'll be using the Unity engine as our main framework, so that's really awesome, but it takes away from the time I have for my own project.

Regardless, as the final wound to a close I was able to finally meet up with my equally-or-insanely-more-busy friend David and we have big plans to go back to the base "2D engine" that I started on over the summer and to add oodles and oodles of functionality and features.

We discussed putting in a template zombie plant class.

One of the concepts that we toyed with (read: David thought up) was creating a very dynamic class manager.

How many of you are programmers?  ...right, since my audience is probably zero, that number is probably zero, but on the off chance any of my friends are actually reading this (thanks so much!) you may or may not be familiar with the programming and the funny terms I'm throwing around.

I'll do my best to break this down as simply as possible.  And as somebody said, if you can't explain it simply, you don't understand it well enough.  Hope I understand this stuffz then.
Anyway, in terms of programming dynamic refers to anything that is created at runtime.  That is to say, when the program is building up these "dynamic" objects don't have to exist, they can be created on the fly.  The structure of these objects must exist before hand (in most cases), and their functionality, i.e. what they do needs to be defined, but when the program starts I as the programmer only have the reference to what this thing can do, and then if it's dynamic I can create as many of these objects as I want and tweak their values individually to make them do different things. 

To recap: I can make as many as my computer can handle and mess with them based off of one framework.

That handles the dynamic thingy I was talking about, now what was this about a class manager?

fer' real.

Well the way a program works is that you can easily create as many dynamic objects as you darn well please, as long as your computer has enough space (I'm simplifying the whole thing a bit, bear with me though)

However, when I create these new and dynamic objects, I have a problem.  Since my code is creating these things, I as the programmer don't have direct access to them, per se.  If I create a new object, it puts the memory for this object on the heap, but if I don't keep a reference to this object, or declare a variable to hold onto all of it's information, then, in languages like C++ I absolutely lose the object.  Poof, it's gone, it's still on the heap, taking up space, chillin, probably having a drink or two, or three depending, but there is zero means of me being able to access it again (I CAN go back through the heap and look for it, but that's a pain, not available  in all languages and unreliable at best)

POINT BEING that to keep track of all of these objects I need a container for all of them, most likely a list or deque, but that's also not important.  What's important is each type of object needs to have it's own container, or that's how it works best in games.  Certain objects can be grouped in their own containers, but it's a good idea to have an object manager, which is a class that holds the container of all of a certain type of object, such as tanks.  So I have a tank manager, it knows where all of my tanks are and updates them, but a tank manager can also do special things to all of the tanks, or change their behavior based on certain circumstances, such as let them all know "HEY! THERE'S A BAD GUY OVER HERE!"

Fire the slings and arrows of misfortune!  ...And the flaming harpoon of indifference.

Back to David... why is a dynamic class manager important?  Because Class managers, such as the tank manager, are going to look darn similar to my manager for bullets, or my manager for players in a multiplayer game.  They have the same structure, have the same container but different contents, and will pretty much run the same, but they will have different group behaviors that are specific to tanks or bullets.
Which means that I would have to write almost the exact same manager for each different object I wanted to create, differing only in the group implementation.  Which, as any programmer will tell you, is a pain and is generally really really really really really not cool, bro.

So David's solution was to create a template-ed game manager (A template class means, simply enough, it can take any kind of class if you want, usually it's used to put that class into a container)
That can take any kind of game object we wanted, tanks, bullets, players, chickens, self-aware bacteria, etc.
The template thing is awesome: essentially we say, hey, compiler, we want a manager class that can take any kind of class and manage it! The problem is that the template has the same code for each different kind of class that we put into the template.  So we can have a bullet manager, tank manager, and chicken manager that are literally written for us by the compiler based on our original template.(Though I don't think they are called templates in C#, whatever same concept)
So, we lose our ability to have special group behaviors.

Excuse the language

Well nuts, we can either write a bunch of really similar but kinda different manager classes, or write one manager class, but lose the ability to have those small differences that could make a big impact gameplay wise.
This is where David saves the day.

In our template-ed managers, we create a shell object, a game object that doesn't "technically" exist, and that isn't updated or drawn or etc.  Each of our game objects now contains a special function that actually contains the special behavior that we want our game object manager to have, and each game object we create (not dynamically, but in code, such as the base code for a tank, and the base code for a bullet, and the base code for a chicken, so on and so forth...) can have different versions of this function, which means the managers, even though they all run the same way on the surface (in the template) when they call the unique game object's special function, they get a different group behavior than a game object manager of a different type.  So tanks can be alerted to an enemy, but chickens can be told to run away by their respective managers, even though we only have one template manager class.

I have a feeling I didn't do this concept justice, and probably explained it very poorly, but if you skipped everything else and just got to this, or read all the way through and didn't understand any of it, I'll break it down very, very simply:
David is a genius.
I'm back to working on my own stuff, starting with this new template manager.
I'm so ready to go back home.

-Kevin

p.s. Get your doodle fix.

Plants versus... humans?

This isn't supposed to make sense, fyi