Random header image... Refresh for more!

Category — Crazy Weekend Project 2: Wesley Crusher Removal

LCARS: Little Crusher Automated Removal System

Here’s the video:

[mediaplayer src=’https://mathpirate.net/hold/LCARS1.wmv’ ]

https://mathpirate.net/hold/LCARS1.wmv

This is a scene from the Next Generation episode Journey’s End.  The first run is simply the result with Wesley Crusher blacked out.  The second part displays the faces that are recognized.  There are occasional blips (Data gets blacked out in several frames, Wesley’s ear gets “recognized” a few times), but overall, not too shabby for something thrown together in a few days with no virtually no tuning of the system being done.  The reason he’s not blacked out at the end is likely due to the decision to expand the minimum face size to 50×50.  If the face size were smaller, it likely would have detected and blocked that part, as well.

November 29, 2009   No Comments

Inpainting

RemovalInProgress

With this, I tried a technique called “Inpainting”, where you give it a mask of the damaged area, and the software will remove the damage.

November 29, 2009   No Comments

Circular Logic

I tried running it on a different episode and the results were…  A bit different.

STTNGFail2

STTNGFail1

I guess the high success rate was due to finding the training images again.  Oh well, at least it did that right.

And this wasn’t a complete loss, JeanLucPicard was identified correctly most of the time!  And some of the other characters were properly identified some of the time.

I also took a look at the distances for each match.  You see, the match is actually a 2500 dimensional distance calculation between two images.  I was hoping to use this distance to weed out false matches, unfortunately, there wasn’t really that much of a separation between correct and incorrect matches.

At any rate, all that is just tweaking the system.

November 29, 2009   No Comments

Its Continuing Mission…

With that, I’ve actually reached the point I was hoping to reach for this project.  I set out to write something that would detect and identify faces from a video stream and that’s what I’ve done.  However, there’s obviously another step that needs to be done, which is the automatic removal of Wesley Crusher.  It knows who he is, so it’s halfway there.

Of course, on the technical side, there are a few things left to do, like figure out why I’m getting Access Violations randomly when calling EigenDecomposite() or trying to make it not be so horribly slow or figuring out if there’s some way I can return nobody when it’s not confident enough about its identification.

And, of course, I also have to run it against a different episode in order to make sure that the recognition isn’t so accurate because it’s simply recognizing the original faces that it trained on…

But that’s what tomorrow is for.

November 28, 2009   No Comments

Visualization Activated

Images in action:

STTNGFaces1

Target Acquired

STTNGFaces2

Multiple hits.  Note that it doesn’t see Geordi…  I think the face detector relies heavily on the eyes, so his VISOR is confusing it.

STTNGFaces3

It can even detect redshirts!

November 28, 2009   No Comments

It Works!

IT WORKS!

HOLY CRAP IT ACTUALLY WORKS!

I trained it on a set of about half of the images, because it takes a long time to train.  Then I ran it on the video and it was actually returning the right person most of the time.  I’m actually shocked that it’s working so well, especially seeing that this is the first time I’ve had the recognition part actually compiling and running.  No real tweaking necessary.  The problems I’ve been having have all been with calling the functions and simply using the library, it’s not that the software was doing what I told it to do and it all came out wrong.

Looking at this trace, you can even get a sense of the cuts in the scene:

Recognized: JeanLucPicard
Recognized: DeannaTroi
Recognized: JeanLucPicard
Recognized: DeannaTroi
Recognized: DeannaTroi
Recognized: JeanLucPicard
Recognized: DeannaTroi
Recognized: Anthwarta
Recognized: Anthwarta
Recognized: Anthwarta
Recognized: Anthwarta
Recognized: Anthwarta
Recognized: Anthwarta
Recognized: Anthwarta
Recognized: DeannaTroi
Recognized: DeannaTroi
Recognized: DeannaTroi
Recognized: JeanLucPicard
Recognized: JeanLucPicard
Recognized: DeannaTroi
Recognized: JeanLucPicard
Recognized: DeannaTroi
Recognized: DeannaTroi
Recognized: JeanLucPicard
Recognized: DeannaTroi
Recognized: DeannaTroi
Recognized: DeannaTroi
Recognized: DeannaTroi
Recognized: JeanLucPicard
Recognized: DeannaTroi
Recognized: JeanLucPicard
Recognized: Anthwarta
Recognized: Anthwarta
Recognized: Anthwarta
Recognized: Anthwarta

Of course, right now, it’s only working on the positive cases.  When it knows someone, it knows them.  When it doesn’t know someone, it randomly guesses about who it is.

Now, something interesting about that…  I haven’t trained the system on Wesley yet.  He’s down in the Ws and the images I trained with were alphabetical.  However, in the scenes with him, he’s fairly regularly identified as “JeanLucPicard”.

Gotta wonder if the Doctor and the Captain have something they need to talk to him about…

Now I just have to label the faces on the screen, since screenshots are better than random text.

November 28, 2009   No Comments

Tip of the Day

If your facial recognition system is only capable of recognizing a single face, you will have 100% accuracy on that face when that face is in the scene.

Looks like it’s time to try processing the full set of images…

November 28, 2009   No Comments

I Need A Universal Translator

I was expecting the facial recognition system to give me odd results when I first ran it.

I was not expecting the results to be quite this odd:

UniversalTranslator

November 28, 2009   No Comments

Convenient != Efficient

OpenCV has some convienent data storage functions for all of its crazy datatypes.  I’m thankful that I don’t have to worry about reading and writing CvMat, IplImage, and so on.

However, the file size leaves something to be desired.  It writes everything as text in XML.  As a result, a float increases in size by 400%.  AWESOME!  And that’s not even counting all of the XML overhead.

I’m thinking I might have to tweak this at some point…

November 28, 2009   No Comments

Not what I was going for…

KeyValuePairOfInt32String

So many things are wrong with this that I’m not even going to mention any of them.

November 28, 2009   No Comments