Motion Capture

The weekend went well. I talked to my sound designer, Clayton De Wet about recording a performance. He said films usually record the audio first and sync it up during the performance. When I get to the point of having a performer come to play the piano, I will have him or her go to Clayton’s studio first to record and then we will sync it as it is played in the motion capture studio.

As for the skeleton my modeler told me he would be finished this coming week. He’s a bit late on the deadline as I was expecting it this weekend.

Also last Friday I met with Matt Ward from Image Movers Digital. I spoke with him a lot about motion capture and what they’re looking for on a demo reel. He told me the default characters or a basic character in MotionBuilder would work fine. When I told him I was bringing things into Maya he asked why I even took that step. What they look for in demo reels are the moves not the rendering and painted weights. He told me not to waste my time. With that advice I’ve decided to bring the piano and possibly the character into MotionBuilder so that I can render there instead of in Maya.

He also told me that the hand capture was a great idea. It is one of the few things not perfected in the industry and if I could do it with a 12 camera setup that it would look very good for getting a job. He also said that facial or animal motion capture would be a good thing to try.

As for my work I managed to get a working piano playing actor. The movements do not translate very well to the Mia character and I may go ahead and just render with the actor. I’ll be working on animating the keys on the piano with the actor and then possibly moving on to the character. The problem is I need the character from my modeler before I can begin fixing the hands in Maya. The animation comes from an FBX and is merged into the scene with a character already in a TPose. I need the character’s exact finger length to animate the piano in Maya. For now I will animate it in MotionBuilder and get everything working there.

I’ve managed to bring in the piano to MotionBuilder from Maya. I also have playblasts of my coneman with the applied animation from the Mia skeleton.

I’m currently looking into getting a trial of the PeelSolver Plug-in for Maya. PeelSolver allows the user to by-pass motion builder and attach the data to a skeleton right in Maya. The issues I am having in MotionBuilder are with attaching the fingers to the actor. I’m hoping that this plug-in would allow the data to be better read by the final skeleton.

Today’s capture session went well. Being at the computer was much easier than directing someone at the computer and being in the suit. I was able to get several decent captures and I have been working with them since we finished up earlier today.

John helping out

After a long weekend of trying to get my piano captures to work, I finally decided to work on a different capture I had done for fun.  I decided to do this capture so that I could see if I could even get a sucessful five finger capture seeing as the piano captures aren’t going well.  I had made a cone man character for my Motion Capture Technology class during Spring of 2008 and I applied to him a five finger capture of me miming.

I’m starting to come to the conclusion that SCAD’s studio is just not set up well enough for such a central thing as a piano performance.

I spoke to a modeler, Kyle Blair, who is going to be modeling a mime for me and also a “Mickey” type skeleton to use for other captures.  I will continue to see if the piano data will work but am thinking that at this point I’m better off getting multiple five finger captures so that I have more for my demo reel than one piano playing performance that took two quarters to do.

Mickey Skeleton

The capture session on Thursday went well.  Though Vicon is having some issues with crashing, we pulled through and I have several “successful” captures.

T-Pose

Playing

After going through the post processing steps everything appeared to be working well - that is up until MotionBuilder.  It appears that either the skeletal rig I built in Vicon is not proportionate or that I’m so out of proportion myself that I’m having issues lining everything up.  The fingers are schrunching as you can see in the following screenshot.  For Wendesday I hope to troubleshoot and get the problem resolved.

Hand Crunching in MotionBuilder

Playing

I have some progress on the five finger skeleton I’m building in ViconiQ. The skeleton is a modification of the three fingered skeleton. Vicon’s modeling aspect is not particularly user friendly so this took some time. It appears I’ll have to start with the three finger now on the right side as well. I have found no way to mirror it.

Left Side Complete, 5 Finger Skeleton

Left Side Has 5 Fingers

Bones & Markers

Markers

Studio II began yesterday with professor Joe Pasquale. Today I’m working in the studio finding a ’sweet spot’ for the hand motion capture animation. In other words I need to find a spot in which all the markers on my hands can be seen by the cameras with minimal flickering.

To test for an ideal spot I first held out my hands and walked around the boundary box slowly to see which area could see all my fingers as I moved. Then I made a second round where I wiggled my fingers like I was playing a piano.

Hands

Gloves

It appears that the best corner is by cameras #4 and #5. At this corner I’m getting readings from cameras 3, 4, 5, 6, 7, 8, 10, 11 and 12. That’s 9/12 cameras.

Spot by #4 and #5

The second potential spot is directly between cameras #2 and #3. At this location there are actually 11/12 cameras seeing parts of my hands but the number of cams that can see the entire hand is lower. The cameras that can see the hands straight on are critical.

Spot by #2 and #3

A third spot would be directly in front of camera #8. At this spot there are 9/12 cameras that can see my hands.

Spot by #8

I have a capture session scheduled for Thursday at 5pm. I will try all three spots at this time. I also will be playing Fur Elise and Mary Had a Little Lamb for test songs and potentially to replace the song Mad World that I had originally planned to use. I also have a DVX100 camera checked out for this session so that if the capture is successful then I will be able to do a breakdown.

As a secondary idea for when this project is finished, I would like to try more full body captures including the three finger gloves. I’ve contacted an animation major I know to try to get in contact with others that would be interested in sharing their modeled biped characters for me to add motion capture to. If I did this then they would be able to showcase their model while I can have a previously modeled character to animate.

Mocap Studio

Mocap Studio

This week I’ve been re-rigging the Alien. She has a modified Mia skeleton out of MotionBuilder. I worked with finishing off the dance motion capture and am close to a final product. Below is the image of the skinned alien.

Rigged Alien

This is the dancing alien. She is rigged with the modified Mia skeleton and had a 3 fingered capture attached to her that was captured in ViconiQ.

This weekend I worked with the data I recorded the previous weekend.  To show I have a successful MotionBuilder render of a freestyle dance.

I spend the majority of my free time in the studio this weekend. Sunday’s capturing session was relatively successful. On Sunday I captured myself playing the piano with a 5 finger capture using the gloves in the previous post. However I do not have a 5 fingered skeleton and the 3 fingered skeleton was getting markers confused in Vicon during post-processing. As a result I decided to first get some working 3 finger captures. Below is the range of motion that I did added to the character Mia.

Test render through MotionBuilder. Default character Mia and the actor with the labeled markers from a 3 fingered skeleton from ViconiQ.

I also worked on animating the keys.