Teemu Ylänne. 3D Camera Tracking for Low-budget Production - PDF

Description
Teemu Ylänne 3D Camera Tracking for Low-budget Production Helsinki Metropolia University of Applied Sciences Bachelor of Engineering Media Engineering Bachelor of Engineering Thesis 15 May 2011 Abstract

Please download to get full document.

View again

of 38
All materials on our website are shared by users. If you have any questions about copyright issues, please report us to resolve them. We are always happy to assist you.
Information
Category:

Books - Non-fiction

Publish on:

Views: 6 | Pages: 38

Extension: PDF | Download: 0

Share
Transcript
Teemu Ylänne 3D Camera Tracking for Low-budget Production Helsinki Metropolia University of Applied Sciences Bachelor of Engineering Media Engineering Bachelor of Engineering Thesis 15 May 2011 Abstract Author Teemu Ylänne Title 3D camera tracking for low-budget production Number of Pages 36 pages + 2 appendices Date 15 May 2011 Degree Bachelor of Engineering Degree Programme Media Engineering Specialisation option Digital Media Instructors Antti Laiho, Lecturer Harri Airaksinen, Principal Lecturer The purpose of the thesis was to find out if there is a free tool for 3D camera tracking and if such exists, to compare it with a commercial counterpart. Additionally, it was investigated whether the chosen tool is suitable for a low-budget production. To investigate this, a short film was made where a 3D creature created in 3D modelling software was composited with previously filmed live-action footage. To composite the creature with the footage, the footage was analysed with a 3D camera tracking software. An estimation of camera parameters and its movement were the result of the tracking process. The results were exported to 3D modelling software where a virtual camera for animation was created based on the results. Animation created in the 3D modelling software was exported to a video editing software for the final compositing process. A suitable 3D camera tracking tool was found for low-budget productions. It was competitive compared to a commercial software. It performed well in a normal desktop computer. There were still many problems with the camera tracking process. Many of them can be avoided with proper preparation before the shooting. Solution to some problems would require a bigger crew which is not always possible with lowbudget productions. Keywords 3D camera tracking, visual effect, short film, animation Tiivistelmä Tekijä Teemu Ylänne Otsikko 3D-kameranjäljitys pienen budjetin elokuvatuotannossa Sivumäärä 36 sivua + 2 liitettä Aika Tutkinto insinööri (AMK) Koulutusohjelma mediatekniikka Suuntautumisvaihtoehto digitaalinen media Ohjaajat tuntiopettaja Antti Laiho yliopettaja Harri Airaksinen Insinöörityön tavoitteena oli selvittää, löytyykö ilmaisia työkaluja 3D- kameranjäljitykseen, ja jos löytyy, verrata yhtä niistä kaupalliseen tuotteeseen. Lisäksi selvitettiin, onko ilmainen työkalu sopiva pienen budjetin elokuvatuotannon tarpeisiin. Asian selvittämiseksi valmistettiin lyhytelokuva, jossa valmiiksi kuvattuun materiaaliin liitettiin 3D-mallinnusohjelmalla tehty hahmo. Hahmon liittämiseksi valmiiseen materiaaliin analysoitiin materiaali 3D-kameranjäljitysohjelmalla. Analyysin tuloksena saatiin arvio kameran ominaisuuksista ja liikkeestä. Analyysin tulokset siirrettiin 3Dmallinnusohjelmaan, jossa tulosten perusteella luotiin tiedosto virtuaalikameroineen animaatioita varten. 3D-mallinnusohjelmassa luotu animaatio siirrettiin videoeditointiohjelmaan videokuvaan yhdistämistä varten. Pienen budjetin elokuvatuotantoihin sopiva ilmainen 3D-kameranjäljitystyökalu löytyi. Vertailussa kaupalliseen vaihtoehtoon se todettiin kilpailukykyiseksi vaihtoehdoksi. Se toimi hyvin tavallisessa tietokoneessa. Kuitenkin itse kameranjäljityksessä kohdattiin useita ongelmia. Monet niistä ovat vältettävissä kunnollisella valmistautumisella ennen varsinaisen materiaalin kuvaamista, mutta osan ratkaiseminen vaatisi työryhmän merkittävää kasvattamista, mikä ei pienen budjetin elokuvatuotannoissa ole aina mahdollista. Avainsanat 3D-kameranjäljitys, visuaalinen tehoste, lyhytelokuva, animaatio Contents Abbreviations and Terms... 1 Introduction Visual effects in cinema From Méliès to King Kong From Forbidden Planet to Futureworld From Judgement Day to the present Tracking Preparation and acquiring the footage Principles of tracking Automatic feature point detection Tracking Ihminen ja enkeli Using gathered data in 3D modelling software Combining the 3D scene and other materials Comparison Voodoo Camera Tracker Autodesk MatchMover Comparison of the chosen tools Conclusions...32 References...35 Appendices Appendix 1. List of Scenes Appendix 2. Sample of Tracking Data Abbreviations and Terms Alpha channel In digital images a channel where the transparency information is stored. Feature point A point in an image that is chosen either manually or by automatic detection to be tracked. Camera tracking A process of gathering information from previously filmed footage. CGI Computer-generated imagery. Visual effects created with a computer. Compositing In cinema a workphase where images from different sources are combined into a single image. SUSAN In mathematics an acronym for Smallest Univalue Segment Assimilating Nucleus. Python In programming an object oriented programing language. Visual effects (VFX) In cinema processes used to manipulate imagery in postproduction process. Includes optical, mechanical and digital effects. 6 1 Introduction This thesis has two purposes. Firstly, to find out if there is a non-commercial, free tool for 3D video tracking and if such exists to compare that tool with a commercial counterpart. Secondly, the purpose was to investigate if the chosen tool would be suitable for a low-budget short film project from the perspective of the artist. The short film, Ihminen ja enkeli, was created to experiment the tracking process on a real video footage. All video was shot without extra lighting and on location. The lighting varies from daylight to fluorescent and tungsten. Most of the shots were taken with a handheld camera, increasing the jitter in the footage. This was a deliberate decision in order to test how the tracking software performs when used with a less than perfect material. Inspiration for the script of Ihminen ja enkeli comes from an independent graphic short story depicting discussion between Jon Venables, Robert Thompson and a humanoid creature just before the abduction and murder of James Bulger. In the short story the creature plants the seed of violence in the heads of Venables and Thompson through discussion and subtle mindplay. The short story was seen in an exhibition in the mid 1990's and its idea of some creature controlling human mind and actions was haunting enough to be remembered after a decade. The script was written by Teemu Ylänne and Lassi Haaranen, short descriptions of the scenes can be seen in the Appendix 1. Ihminen ja enkeli was directed by Teemu Ylänne and the camera was operated by Lassi Haaranen. A rough cut of the original material was edited by Teemu Ylänne and used as the source for the camera tracking process. Teemu Ylänne did all the camera tracking. The 3D character was modelled by Lassi Haaranen and animated by Lassi Haaranen and Teemu Ylänne. Music was performed by Lassi Haaranen. Additional sound effects were created by Teemu Ylänne. All editing, mixing and compositing was done by Teemu Ylänne. Budget for the production was 300 euros which was mainly used for props and technical equipment. 7 The thesis firstly focuses on visual effects in cinema (chapter 2) and proceeds to discuss the process of 3D video tracking and the mathematics behind the process (chapter 3). Then it discusses the use of gathered data in a 3D modelling software (chapter 4) and combining the original video footage with a 3D model and other material (chapter 5). Finally, it focuses on the comparison of the used tracking tools and their features (chapter 6). 8 2 Visual effects in cinema 2.1 From Méliès to King Kong Visual effects have been an essential part of the movies for almost their entire history. George Méliès was a French magician whose movie A Trip to the Moon (1902) was one of the first visual effects movies. Méliès used in-camera trickery and special effects to visualise this science fiction tale written by Jules Verne. (Robbins 2010) Figure 1. Behind the scenes of Metropolis. (Kino International 2010) Other significant milestones in early visual effects history include Frits Lang's Metropolis (1926) and King Kong (1933), directed by Merian C. Cooper and Ernest B. Schoedsack. Frits Lang's use of mirrors to combine miniature's and actor's in full-sized sets creates a stunning city of the future. As seen in Figure 1, the miniature set included buildings taller than man and model vehicles on suspended roads. In King 9 Kong the use of miniatures, stop action photography and optical techniques helped the directors tell a story unlike anything ever seen on the silver screen. King Kong set a standard on visual effects that movies are trying to achieve even today. (Robbins 2010; von Bagh 1989: ) 2.2 From Forbidden Planet to Futureworld In the 1950's visual effects were an essential part of the science fiction movies such as Forbidden Planet (1956) directed by Fred M. Wilcox. Although the movie does not have ground breaking visual effects, it was still a great inspiration for future science fiction entertainment, including Gene Roddenberry's Star Trek. The science fiction films were collaborative works of entertainment and, therefore, the names of the artist are seldom remembered. This changed in 1968 with director Stanley Kubrick's and writer Arthur C. Clarke's collaboration 2001: A Space Odyssey. The film is a visual masterpiece with minimum of dialoque. Kubrick's and Clarke's tremendous visions were fullfilled by Douglas Trumbull. With 205 effects shots that took over half of the movies budget, the visual effects are one of the main reasons why the movie became a classic. (Robbins 2010; von Bagh 1989: ; IMDB 2011a) Michael Crichton's Westworld (1973) introduced audience to 2D computer-generated imagery illustrating a robots point of view. Its sequel Futureworld (1976), directed by Richard T. Heffron, was the first live-action film to use 3D computer-generated imagery. The 3D CGI was used to animate a hand and a face, the CGI face is seen in Figure 2. (Robbins 2010; IMDB 2011b) These movies began the slow transformation from the use of miniatures and special makeup to the modern visual effects done mainly on computers. 10 Figure 2. CGI from Futureworld (The Lightning Bug's Lair) Another significant milestone came from the studio that is best known for its animations. In 1982 Disney proved with its classic movie Tron that extensive use of computer generated imagery in a film was possible. Many of its scenes were computer enhanced and some purely CGI. Although the movie looked like a low-budget production compared to its competitors such as Ridley Scott's Blade Runner (1982) that relied in traditional effects, its value is in its role as an antecedent for the advance in visual effects history. (Robbins 2010) From Judgement Day to the present The use of computer in creation of visual effects significantly increased in the 1990's. Advances in the computer technology made creating complex effects easier and more affordable. James Cameron's Terminator 2: Judgement Day (1991) includes a lot of 3D animation and morphing effects. Figure 3 shows the T1000 made of liquid metal morphing into a police officer. In Death Becomes Her (1992) actresses head was removed from the shot and replaced with tracking another shot of a talking head onto it. (Kerlow 2004: 22-24) Figure 3. T1000 morphing into a police officer.(movie Mobsters. 2010) All the techniques to create visual effects for a movie were in use in The Lord of the Rings -trilogy directed by Peter Jackson. In The Return of the King over 1400 visual effects shots were made to create the movie. Pelennor Fields were completely created with computer-generated imagery, because there was no location to be found that would match J.R.R. Tolkien's description of the place. (Visual Effects. Weta Digital 2004) 12 Figure 4. Spaceships from Star Wreck: In the Pirkinning. (Energia Productions 2011) The development of the computer technology has made it possible to do visual effects that used to require expensive equipment on an ordinary workstation computer. This has brought the possibility of creating stunning visual effects available for low budget movies but the required software is still expensive. (Byrne 2009, Kerlow 2004) With limited resources it also requires significant amounts of time to do convincing visual effects. A good example of this is the Finnish science fiction parody Star Wreck: In the Pirkinning that was first released on the internet in The movie took seven years to complete. In that time many of the tools were upgraded and the hardware changed several times. Still everything was done with ordinary desktop computers. A sample of the results, a group of CGI spaceships, is seen in Figure 4. (Energia Productions 2011) 13 3 Tracking 3.1 Preparation and acquiring the footage Before shooting the actual video footage thought to be tracked, it is needed to prepare for the shoot as well as possible. In every phase of the preparation one must consider the requirements of tracking as well as everything else that might affect the shooting. In storyboarding it is crucial to visualise all the 3D elements to be included in the final composition. Otherwise the shots may be too tight to fit the intended 3D elements in or the shooting will lack proper purpose. A crude 3D animation of the scene is a good pre-visualisation tool also if there is enough resources to do it. Inadequate planning results in creative deficit. (Byrne 2009: 19-20; Kerlow 2004: 59,62-63) When acquiring the actual footage it is necessary to keep in mind the limitations of the tracking process. Shots should be made with a locked off camera or a camera on a tripod to avoid unnecessary jitter. The focal length of the camera should be constant if the camera is not on a tripod. A small focal length is preferable. Lighting should be good so that the footage will have enough contrasts. Unnecessary movements near anything that is supposed to be tracked needs to be minimised. When possible one should use suitable markers to help the tracking process. It is important that there is trackable objects both in the background and in the foreground of the shot. The shots should be as short as possible to enhance the possibility of succesful tracking process, preferably less than 400 frames. (Ferguson & Heron; Laboratorium für Informationstechnologie 2010; Kerlow 2004:377) In preparing for the shooting of Ihminen ja enkeli some storyboards were made to clarify the ideas. The 3D character's place in shots was planned to be in relationship with the Ihminen character. In Figure 5 there are three pictures from the crude storyboard made before shooting any material. In the storyboard the main movements of the characters are drawn as arrows pointing the direction of the movement. Only the necessary elements are shown in the storyboard and no consideration for set decoration was made at this stage of production. Some preliminary sketches of the enkeli were also made to visualise the overall tone of the film. There was no real 14 location scouting before the shooting. All locations were chosen based on easy access and proximity. No pre-visualisation animation was made for the short film. Figure 5. Pictures from crude storyboard It was decided that shootings would be made with as light equipment as possible. No additional lighting was used and notes of actual lighting conditions were made. The only light source that was excluded from a shot on location was a television because its flicker was disturbing. In one location the lighting was very diverse which made the shooting extra challenging. Because of the nature of the production, it was good that shooting on outdoor locations happened on cloudy weather. This made the lighting on the scenes even which made it easier to light the 3D scenes. Sounds were recorded with the camera's own microphone as a reference for editing. The plan was to discard all the sounds of the actual shots and replace them with sound effects and score. This was decided because there was no way of controlling the sounds of exterior shots. 15 Most of the shots were taken with a handheld camera although it increased jitter significantly. A tripod was used in some shots to make it easier to controll the movement of the camera. No markers were used to aid the tracking process during the shooting. Before the modelling of the creature or the tracking process started, a rough cut of the material was made. Many of the clips on the rough cut were quite long which was a potential problem for the tracking process but no consideration for shortening these was ever made. For research purposes these shots were perfect because they would really test the chosen tool's suitability for real productions. No color correction or adjustments to the contrast of the shots were made before the tracking process. 3.2 Principles of tracking Tracking is a process of gathering information from previously filmed footage. It has become one of the key visual effects techniques of the present. Tracking does not produce finished shots but numerical information that can be used to match virtual camera and animated objects and characters with live-action camera. Tracking markers are usually used to help the camera tracking process. (Kerlow 2004: 375, 377) Before starting the tracking process, it is good to preview the shot in real time. Making notes of good tracking targets and when they go out of frame or are occluded can save time and trouble in the actual tracking process. Any parallax in the scene will disqualify otherwise attracting tracking targets. (Wright 2002: 183) Many commercial tracking tools offer only solution for 2D tracking. 2D tracking tool is in fact a keyframe-generating assistant that analyzes images and calculates pixel shift on defined areas of interest. Visual effects artist could do all this manually but tracking tools are used to speed up the workflow. (Davies 2005: 196) 3D tracking resembles photogrammetry, a technique that can extract three dimensional models from two or more still images of a subject. Usually this is used to extract a depth map.(kerlow 2004: 383) In 3D camera tracking suitable points from the image are chosen as feature points either manually or automatically. The position of the feature points in the image plane 16 is measured and combined with the focal length of the camera. 3D tracking software analyses the footage frame by frame and estimates camera parameters from the paths of the chosen feature points. After analysing the acquired footage, the software can produce information usable in creation of a scene in 3D modelling software with virtual camera and the feature points placed in the 3D space. (Laboratorium für Informationstechnologie 2010) 3.3 Automatic feature point detection Automatic feature point detection chooses points that are bright, dark, edges or corners in the image. Voodoo Camera Tracker uses corner detection at sub-pixel accuracy. Corners are points where edges from at least two different directions meet at the same point. One of the algorithms Voodoo Camera Tracker uses for detection of the feature points is the SUSAN (Smallest Univalue Segment Assimilating Nucleus) corner detector, modification of the edge detector of the same name. (Laboratorium für Informationstechnologie 2010) In SUSAN corner detector the computation of the area of points inside a circular region N x, y have a brightness similar to the one of the central point x, y. The area is computed thus (1) 6 x, y = e I i, j I x, y t i, j N x, y The parameter t controls the sensitivity to noise which means it defines the similarity between the brightness values. The value of η(x,y) is compared to a fixed threshold equal to ηmax/2 where ηmax is the maximum value that η() can take. That maximum value is c s x, y = max 2 x, y ; if x, y 2 0 ; otherwise max (2) Computational simplicity makes this kind of algorithms based on brightness comparisons preferable solution for corner detection. (Laganière 1998) To track these points the software needs
Related Search
We Need Your Support
Thank you for visiting our website and your interest in our free products and services. We are nonprofit website to share and download documents. To the running of this website, we need your help to support us.

Thanks to everyone for your continued support.

No, Thanks