CSCI 520, Spring 2012, Assignment 2: Motion Capture Interpolation

This is an old assignment. Do not use.

Due Wed Mar 7, 2012, by 11:59pm



Overview

In this assignment, you will implement three interpolation schemes to interpolate human motion data obtained from an optical mocap system. The human model (skeleton) is represented using a hierarchy, composed of a root node and several children nodes, corresponding to the various joints. The optical mocap system generates two data files: ASF file, which encodes the skeleton kinematics (lengths of bones, degrees of freedom, etc.) and AMC file, which stores the motion (translation, joint angles) in time (at 120 frames per second). The AMC file contains the translation of the root node (xyz coordinates), rotation of the root node (in Euler angles), and the joint angles (for the various joints, in Euler angles). We provide starter code that can parse ASF and AMC files and store the data into a datastructure. The starter code also includes an OpenGL motion capture player that renders the skeleton (parsed from the ASF file), and plays back the motion from an AMC file. You will be given several ASF files and AMC files corresponding to different motion capture sequences (walking, dancing, running, martial arts). For each motion sequence, you will then form a subsequence, by dropping certain frames (described below). You will use this subsequence to create an interpolated sequence of the original length, and analyze and plot the difference to the original motion.


Source: Wikipedia

Requirements

The AMC files use Euler angles to represent joint rotations. The starter code can load these Euler angles into a datastructure in memory. Your program should take the motion from the input AMC file, and then form the keyframes for interpolation. The keyframes are obtained from the original motion by dropping N consecutive frames, where N is a given input parameter. For example, suppose the input motion has 30 frames, labeled 0,1,...,29. Suppose N=6. Then, the keyframes are frames 0,7,14,21,28. Your program must use interpolation to compute new frames 1,2,...,6, 8,9,...,13, 15,16,...,20, 22,23,...,27. Set frame 29 to be identical to the input frame 29. Thus, your program produces a motion sequence of the same length as the original. Depending on the value N, and the choice of the particular interpolation technique, the interpolated motions will more or less closely resemble the input motion. For example, it is expected that quaternion interpolation produces motions that more closely resemble the input than Euler interpolation. You will analyze the different interpolation techniques in your report (more below).

The starter code can perform linear interpolation in Euler space (see interpolator.cpp), which also serves as an example for how to access motion capture joint data and create/set new motion sequences. Your task is to do the following:


Submission

Please submit the following:

Please submit the assignment using the USC Blackboard. In order to do so, create a single zip file containing your entire submission. Submit this file to the blackboard using the "Attach File" field. After submission, please verify that your zip file has been successfully uploaded. You may submit as many times as you like. If you submit the assignment multiple times, we will grade your LAST submission only. Your submission time is the time of your LAST submission; if this time is after the deadline, late policy will apply to it.

Note: The instructors will read your code. Quality of your code and comments might affect your grade. This is an individual assignment. Your code might be compared to code of other students using a code comparison tool.


Starter code and data

You can download the starter code and data here.

Code setup instructions

====================================================================================
How to build fltk in Windows, Linux, MAC OS X
====================================================================================
Please follow the corresponding readme files in the folder: fltk-1.3.0
Note: On Linux, you may need to install the "autoconf" tool.

====================================================================================
How to build starter code in Linux, MAC OS X (Assuming fltk has been compiled)
====================================================================================
1) Enter the %homeworkFolder%/mocapPlayer
2) make

====================================================================================
How to build starter code using Visual Studio 2010 (Assuming fltk has been compiled)
====================================================================================
1) Open the project file in homework folder: IDE-starter/VS2010/mocapPlayer.sln
2) Choose Debug/Release mode
3) Compile project: mocapPlayer
4) Compile project: interpolate

====================================================================================
How to compile starter code in other version of Visual Studio
====================================================================================
1) Launch Visual Studio.
2) Select: File->New->Project->Win32 Console Application
3) Change "Location" to the homework folder (the one that contains 3 sub-folders: fltk-1.3.0, IDE-starter, mocapPlayer-starter). Pick up a name for your homework project (e.g. homework2). Click OK.
4) In next window, click "Application Settings",choose: Console application, DESELECT "Precompiled Header", and select "Empty project"
5) You can see a new project (e.g. homework2) under the "Solution". Change the project name if you want (e.g. "motionPlayer"). Add the following files to the project:
displaySkeleton.h/.cpp
interface.h/.cpp
mocapPlayer.h/.cpp
motion.h/.cpp
performanceCounter.h/.cpp
pic.h/.cpp
posture.h/.cpp
skeleton.h/.cpp
transform.h/.cpp
vector.h/.cpp
types.h
6) In VS panel, select menu: Tools->Options
   Setup the "Include Directories", So VS could find fltk-1.3.0
   Setup the "Library Directories", So VS could find fltk-1.3.0\lib
7) Right Click the "Solution", Add->New Project Pick up a name for "interpolation" project (e.g. interpolation). Then repeat step 4)
8) Add the following files to this new project:
interpolator.h/.cpp
interpolate.cpp
motion.h/.cpp
performanceCounter.h/.cpp
pic.h/.cpp
posture.h/.cpp
quaternion.h/.cpp
skeleton.h/.cpp
transform.h/.cpp
vector.h/.cpp
types.h

All set.

Source code structure and description

The code consists of several classes. We only provide a brief description here. For more detailed description of each class, see the corresponding .h files. When the "Load Skeleton" button is pressed, the selected ASF file is parsed, and skeleton loaded is into memory and saved into an instance of the skeleton class. When the "Load Motion" button is pressed, the AMC file is read and saved into an instance of the motion class.


Description of motion capture ASF and AMC file formats

The starter code reads and displays the data from the optical motion capture system (ASF and AMC). The world coordinate system is Y up.

Skeleton file

The skeleton file (ASF) describes how the bones in the skeleton are connected, and their degrees of freedom (see Figure 1 for a representative skeleton). All bones are described in the ":bonedata" section of this file. Each bone has the following fields:

begin
id bone_id                  //Unique id for each bone
name bone_name        //Unique name for each bone
direction dX dY dZ    //Vector describing direction of the bone in world coordinate system
length 7.01722           //Length of the bone
axis 0 0 20 XYZ         //Rotation of local coordinate system for
                                   //this bone relative to the world coordinate
                                   //system. In AMC file the rotation angles
                                    //for this bone for each time frame will be
                                   //defined relative to this local coordinate
                                    //system (see Figure 2)
dof rx ry rz                //Degrees of freedom for this bone.
limits (-160.0 20.0)
            (-70.0 70.0)
            (-60.0 70.0)
end

The parent/child relationship among all bones in skeleton hierarchy is defined in the ":hierarchy " section of the ASF file. Each line list the parent first, then its children.
For example:
"root lhipjoint rhipjoint lowerback"
Root is a parent and lhipjoint, rhipjoint, lowerback are root's children.

The following picture shows the skeleton hierarchy as defined in a representative ASF file (length of the bones is not up to scale).

Figure 1. The skeleton hierarchy, in particular name and id for each bone as defined in a representative ASF file.


Figure 2. The names and local coordinate system for some of the bones.

For a longer introduction on the mocap file format, you can look at http://www.cs.wisc.edu/graphics/Courses/cs-838-1999/Jeff/ASF-AMC.html.

Motion file

The motion file (AMC) gives each bone's rotation angles (in Euler angles) relative to the bone's reference local coordinate system. The bone's reference local coordinate system is defined in the skeleton (ASF) file. One entry is given for each frame, at a frame rate of 120 fps.

The ":FORCE-ALL-JOINTS-BE-3DOF" keyword in the header (if present), is not a standard AMC keyword. We added this keyword because the result of quaternion interpolation may produce three-dimensional Euler angles, even when original Euler angles were only one or two-dimensional. You do not need to worry about this keyword. The "interpolate" driver (starter code) will create it automatically as needed, and the "mocapPlayer" will correctly load any motion files that use this keyword.


Motion capture player instructions

The "mocapPlayer" player can play back any motion capture sequence (AMC file), on any skeleton (ASF file). A simple camera control is provided. You can drag the left mouse button to move the camera, middle mouse button to zoom in/out, and the right mouse button to rotate the camera.

Below is the picture of GUI for the program (by Yili Zhao and Jernej Barbic; first version by Steve Lin, Alla Safonova, Kiran Bhat).

Displaying the animation

You first need to load the skeleton (ASF) file. Then load the motion (AMC) file. You can now view the motion using the player buttons. You can save the current frame as an image by pressing the Screenshot button. To record a sequence of images (to make a video), first press record (R) button, then play the animation. All the frames played while the record button is enabled will be saved to disk as images.

Displaying two or more motions simultaneously

You can also display two motions on top of each other, say, to visualize the difference between the input motion and the interpolated motion. In order to do so, first load the first ASF file and AMC file, then load the second ASF file (may be same as first, but must be still loaded), and the second AMC file. You will see the two motion on top of each other.

Adjusting playback speed

By default, the player plays the motion at normal speed, that is, 120 frames per second. In order to slow down the motion, enter the corresponding value into the "Speed" box. For example, to make the playback 5x slower, enter 0.2 into the "Speed" box.

Displaying the frame of a joint

You can display the frame of a joint by entering its index (positive integer) into the "Show Joint" textbox. The default value is "-1" (no joint shown).

Interpolation

Use the provided "interpolate" driver application to perform the interpolation. This driver loads a skeleton and motion file, performs interpolation, and outputs the resulting AMC motion file. You can then load this AMC file into the mocap player for visualization. You can use the "Reload" button to reload the AMC motion (useful, for example, when tweaking your interpolator class). See the command line options for how to execute various interpolation techniques. For example, to perform Bezier Euler interpolation, using N=10, use:

./interpolate 131-dance.asf 131_04-dance.amc b e 10 131_04-dance-be-N10.amc


Tips


Extra credit

Here are some extra credit ideas:

Also, you might be awarded credit for any other creative or interesting solution to an issue in your implementation, as long as it is well-documented in your readme file. So, if you can think of any interesting feature, feel free to implement it.

Please note that the amount of extra credit awarded will not exceed 20% of this assignment's total value.


by Jernej Barbic and Yili Zhao, February 2012; credits also to the CMU graphics lab