For this entire unit, I have to
focus on developing my skills/techniques in either an area I wish to improve on
or develop a skill/technique in an area that I’m unfamiliar with. I chose to use Maya, After
Effects and Adobe Premiere Pro as these are the three software programs that I
have been using for several years. Because I was unfortunately relieved from a
group project under certain circumstances, I decided to focus on my own project
for the time being, until I had a chance to give my own contributions to
certain group projects. For this project, originally I decided to mainly focus on
modelling, rigging, texturing, lighting and dynamic effects, with occasional
animating, but after a talk with Dan, and despite the fact that I could perform all the basics of each asset anyway, I decided to focus on what would be considered the main problems for this project. In this case, these would be motion tracking, film directorial techniques and cinematography (camera shots). Quite a plateful of stuff I know, but the more roles I could focus
and work on, I figure it would be more beneficial for me to secure a specific
job role in any animation company I could apply for.
Also for this project, I decided
to add a basic storyline to my film as well as combining all my
skills/techniques together. After showing several ideas to Mike Smith (one of my
tutors), I settled on the idea of a little girl wandering through the bombed streets
of London during the Second World War as she remembers all the happy memories
of her family before the war began. I chose to develop this idea as I’ve always
been interested in this kind of history since I was young and I thought it
would be a perfect idea for using certain techniques. Although I originally intended
to produce this project using traditional hand drawn animation which is my
favourite form of animation, after having another talk with Mike, I’m
considering producing this idea in live action. Now to do that, I would have to
perform research on certain areas of live action filming in order to achieve
the kind of results I want.
So, the first step to all this is
research and development, so I will be collecting information by finding
images, viewing several tutorials and reading books on the major subjects of motion tracking, film directing and cinematography in
order to gain a great understanding of the basics of each subject.
As my final project involves combining
live action with CGI, I have to commit research on the subject of motion tracking/
match moving and figure out the methods/techniques that would be the most
effective in order to achieve the desired effect. I managed to conduct most of
my research on this subject by discovering several videos and sites on the
internet that show many specific tutorials on how to perform match moving and
motion tracking. I learnt that Autodesk MatchMover and Adobe After Effects
could both be useful in different ways of achieving motion tracking, although
MatchMover is in some ways more complex than After Effects.
Research
The links below are to a series of tutorials from which I acquired information and several techniques on how to perform match moving in MatchMover and After Effects.
I attempted to test out Matchmover first to see if I can operate the basic settings after viewing several of the tutorials I have listed in my research.
To start off, I attempted to use
a sequence of footage that I had filmed for this test. I produced an image
sequence of the clip in After Effects first, saved the image sequence into a
folder and then imported it as a continuous clip into MatchMover.
Selecting an image in order to create a sequence beginning with said image.
Then I used the option “Automatic
Tracking” so that several markers will be automatically applied to several
points throughout the clip so I would use those markers as reference in order
to synchronise with the camera movements later on.
I then used the Automatic
Tracking Settings menu to alter the density and sensitivity modes, therefore
altering the amount of trackers applied to each image of the clip and how they
will all stick to a certain point continuously.
After all the trackers were rendered and converted into cone markers, I used the navigational tools at the bottom right to view the virtual camera scene in order to see how the scene worked out. And also seeing how the virtual camera would follow a path, similar to how the live action camera filmed the clip.
Attempting to use the Solve for Camera mode to find out what it does, I still have yet to know exactly how it works.
For a while I couldn’t figure out
how to get the exporting to work but then I realized that the export would only
work by creating a new file from scratch but by saving it to an already existing
Maya file. Tried it on a .mb file that I created to test it on, it still didn’t
work and I got the same error but then I noticed it said “Maya(*.ma)” so I went
back into Maya and saved the same scene as a .ma file and when I exported to
that file, it finally worked.
Opening up the file back in Maya
and I was surprised that it actually worked. Of course the markers were all
laid out in the scene in the same space as it was rendered back in MatchMover
so I had a little play with the navigational tools once again to see the layout
more clearly. The image plane that was fixed on the camera was not displaying
the image sequence of the clip at first so I had to select the plane and browse for the
file in order to set it back up so I could view it in the perspective panel.
Of course, I viewed the scene in the camera panel to see how it played out and I found that due to certain keyframe errors, the camera jumped dramatically every few seconds in one part of the clip. All the markers stayed in the same place but the virtual camera needed fixing in certain areas so I would have to attempt to solve the problem by using another technique in MatchMover to get a better result. But I saved a rough clip of the sequence using Playblast and attempted to produce a few movie clips to keep a reference of the test.
This test didn't turn out quite so well as I thought so I'll have to make a few adjustments in order to get the next test working better than this one. Of course, I still have yet to figure out how to work the rest of MatchMover’s techniques so I’ll be able to use it more with Maya later on in order to properly synchronize CGI environments with live action camera movements.
After Effects Motion Tracking
With this test, I decided to use
After Effects for a simpler tracking technique. After importing a footage clip of a field (where I take my dog for walks), I started off by using the tracker controls menu in the bottom right of the screen. I didn't have to stabilize the camera movement so I just chose "Track Motion" and began searching the clip for certain points where the markers could stay focused on. I may have used the markers in several areas of the clip before I settled on using both position and rotation mode, had to use more than one marker. Then I analyzed the clip so that the markers would stay on the certain points as the camera moved, creating a keyframe for each single frame of the clip.
Attempted to use a perspective parallel mode with the markers but it did not have the desired effect.
Once the clip had been analyzed and the markers had been set with keyframes, I attempted to transfer those keyframes over to another layer. I did this by creating a null object, editing the target of the null object and transferring the key frames to that object. As I was applying the keyframes to the null object, I set the option to "X and Y" so that it would affect the object horizontally and vertically.
Then to try it out, I imported an image of a CGI building that I had produced in Maya at an earlier time and set it up where I wanted it to be. Then I used the Keylight mode, which is an advanced chroma key technique, and altered the settings to remove all green colour as well as alter the screen matte settings so as not to have any part of the building looking transparent.
Then using the parent link mode, I linked the the image of the building to the null object so that it moved with it, staying in the same place as the camera moved instead of remaining stationary. Then to make it blend in more with the footage, I altered the colour, tone and added a photo filter layer on top.
For an extra, I attempted to use the same image twice in the same clip, using a simple masking method so that it looked as if it was behind a chair and parented that image to the same null object.
This here is the original footage with no changes made at all.
As you can see in this clip, now with the CGI fake looking building
added in, I have managed to successfully achieve some effect with the motion
tracker mode in After Effects but if you look closely, you can see how the
building changes position slightly as it goes out of sync with the footage
which is due to a defect in a few keyframes on the tracker’s part.
This clip is almost the same with
the motion tracking defect but with another building set in the distance with
some masking attempt.
So these tests have had a few defects already but I will be sure to overcome those problems by trying other methods in different ways to achieve a better result.
Since I was going to involve a live action character in the final version of the project, I had to brush up on some techniques in cinematography as I wanted to use sufficient filmic language in the film.
References and Research
Below is a series of links to online sites that I have found that are relevant to this specific subject, including tips, techniques and advice on using cinematography such as camera shots.
Second Test This here is the second test of match-moving and placing CGI objects in a live action environment. Once again I used a video camera to record a hand-held shot of moving through a kitchen and into a back garden. The purpose of this of course was to further expand on the match-moving technique and being able to put CGI objects in a live action environment.
Above is the original footage clip that I used for this test.
Once again, I imported the image sequence of the shot into MatchMover, set up automatic tracking for this particular shot and then exported the results to an exisitng Maya file that I created for the scene. No problems at all with this procedure, this part was successful.
I opened up the file in Maya and began to insert primitive shapes to test out the scene. Arranged a few cubes and cylinders to certain positions, making sure to check the view of the camera panel so that the objects can be seen. Then I rendered the whole scene and composited them together using After Effects.
First test of the match-moving procedure. As you might notice, the two cylinders near the top of the camera view are supposed to be within the kitchen but instead they remain ahead of us, making their existence in the environment unrealistic.
I attempted to fix that error by rearranging the cylinder objects in the Maya scene, moving them closer to the camera and within the area of markers shown in the image above.
Second test, which shows the changed positioning of the cylinders. As you can see, the camera moves towards and past them so they are blended in much better and it now seems as if they were actually in the kitchen.
Decided to expand on this scene by using more objects in the scene, setting them up within the main capacity of markers which of course is the layout of the kitchen, if you know what I mean. Also for more effect, I key-framed several of the cube objects to slowly rotate throughout the timeline.
Camera perspective of the scene, showing the cubes positioned throughout so that the camera can move past them without colliding. Rendered the scene out again and composited the sequence with the original footage in After Effects.
Final version, giving the scene more of a three dimensional feel.
Third Test
This test here proved to be one
of the most difficult match-moving tests I’ve done for this particular project.
The footage clip I used for this, which of course is a quick shot in my back
garden, had camera movements that were too hard for the automatic tracking mode
in MatchMover to comprehend, trackers couldn’t focus on a certain point for too
long due to quick movement and the result of camera blur.
Original footage clip of the shot. No audio.
I attempted
to adjust the density and sensitivity modes in the automatic tracking settings
and even slowed the footage down 400%, by time stretching the image sequence in
After Effects, to try and make the trackers focus more effectively on the points
but to no avail as again the trackers couldn’t remain in one point of the clip
for long enough.
As you could see in the image above, not all trackers could be processed for the camera due to lack of information in certain frames, again due to the quick camera movement.
As mentioned before, I attempted to adjust the sensitivity and density of the tracker settings in order to achieve better results, unfortunately this was proved unsuccessful.
After trying all these methods, I
realised I couldn’t rely on MatchMover for this particular shot and so I finally
resorted to using the tracker controls in Adobe After Effects to use for the
shot. By setting up three trackers in the image sequence of the clip, each one
focusing on a certain point such as the tip of a roof of a house or a white
football lying on the ground, I managed to achieve a simple effect. Of course
even the tracker controls had a few problems with focusing on a point for too
long due to the camera’s quick movement so I had to painstakingly and manually
move the trackers in each key-frame to the same point of the object it should
be focusing on.
Of course, I had to move the
tracker to the same position/point in each frame as in focusing on the football
in the image above. It still got messed up slightly as I played the sequence as I noticed that the tracker suddenly jumped to another point in one frame and then to a different position in the next frame so I fixed that by deleting certain key points that were affecting the motion path of the tracker when it should be following the desired object.
After using three trackers
focused on certain objects/points in the clip, I created three null objects in
order to convert each key-frame of the trackers to said null objects so then I could assign
the appropriate movie clip to each one. For this clip, I decided to use a glass
ball that I made in an earlier After Effects project and an action essentials
clip of a line of fire to blend in with the fence.
Final version with added effects assigned to null objects.
All these match-moving and motion tracking tests have in some way proven successful in the end and so I'm one step closer to achieving the desired effect for my final project. Still need to make a few minor adjustment to the techniques I'm using though but practice makes perfect.
The motion tracking tests done so far have gone well in the end and I have learnt several techniques in MatchMover on how to achieve effective motion tracking for future projects. Of course, some problems with these tests took a certain amount of time to solve using different techniques and I still need to learn more on certain areas involving live action shooting and chroma keying but I do believe that all this information will assist me in improving my methods in making a good live action film that will be combined with a computer generated environment.
