Lecture Notes
Dr. Tong Yu, April, 2010
    1. Introduction
    2. Color and Lighting
    3. Ray Tracing
    4. Intersection Testing
    5. Animation
     
    6. Sky and Terrain Rendering
    7. Particle Systems
    8. 3D Movies I
    9. 3D Movies II
    10. Radiosity

    An Introduction to 3D Movies

      Some of the materials in this Chapater are adopted from the book, "3D Movie Making", by B. Mendiburu, and materials from the Web.

    1. Stereoscopic Vision

      Stereoscopic Depth Perception

      Our 3 dimensional environment gets reduced to two-dimensional images on the two retinae.

      The "lost" dimension is commonly called depth.
      As depth is crucial for almost all of our interaction with our environment and we perceive the 2D images as three dimensional, the brain must try to estimate the depth from the retinal images as good as it can with the help of different cues and mechanisms.

      Depth perception is the visual ability to perceive the world in three dimensions (3D) and the distance
      of an object

      Depth sensation is the ability to move accurately, or to respond consistently, based on the distances
      of objects in an environment.

      There are a few kinds of depth cues:

      1. Oculomotor: based on the ability to sense the position of our eyes and the tension in the eye muscles.

      2. Monocular: Cues that work with one eye.

      3. Binocular: Cues that depend on two frontal eyes.

      4. Motion-based: Cues that depend on motion of objects.

    2. Oculomotor depth cues -- convergence and accommodation of our eyes induce 3D information.
        When looking at nearby objects our eyes move inwards. This is called convergence. and when looking at far away objects, our eyes move outward, as shown below:

        Left: Converged eyes   Right: Non-converged eyes

    3. Monoscopic depth cues -- we can extract a lot of 3D information from a single 2D view.

      • Perspective and Relative Size
        e.g. a building and a man look the same size; we can assume that the building is farther away.

        Retinal image size allow us to judge distance based on our past and present experience and familiarity
        with similar objects. As the car drives away, the retinal image becomes smaller and smaller. We interpret
        this as the car getting further and further away. This is referred to as size constancy. A retinal image of
        a small car is also interpreted as a distant car.

         

      • Linear Perspective
        When objects of known distance subtend a smaller and smaller angle, it is interpreted as being further away.
        Parallel lines converge with increasing distance such as roads, railway lines, electric wires.

      • Occlusion ( Interposition )
        e.g.if a house hides half a man, we know that the house is in front of him.

        (a)

        (b)

        (c)
        (a)Occlusion cues   (b)Contradicting Occlusion cues   (c)Contradiction resolved by Shadows

      • Aerial Perspective, Atmosphere Blur, Saturation, and Color Shift
        If the air is smoggy, distant objects look blur.
        Relative colour of objects give us some cues to their distance. Due to the
        scattering of blue light in the atmosphere, creating "wall" of blue light, distance
        objects appear more blue. (e.g. distant mountains appear blue.) Contrast of objects
        also provide clues to their distance. When the scattering of light blurs the outlines
        of objects, the object is perceived as distant. Mountains are perceived to be closer
        when the atmosphere is clear.
         


        Aerial perspective as viewed towards a sunset, with the colour shifting toward red as a result of scattering.
          Chinese landscape painting using "atmospheric perspective" to show recession in space

        In art, especially painting, aerial perspective refers to the technique of creating an illusion of depth by
        depicting distant objects as paler, less detailed, and usually bluer than near objects.

      • Texture Gradient
        If a texture has a repetitive pattern, the parts that are farther away appear smaller.
        A texture gradient is a surface pattern which provides information about the distance,
        depth and shape of an object.

        Texture gradient is carefully used in the painting [above], Paris Street: A Rainy Day by Gustave Caillebotte, to show that the street recedes in depth. Here square cobblestones are used. These cobblestones get progressively smaller as the road recedes in depth until the stones are not clearly distinguishable from each other. In the distance, only the general roughness of the street is noticeable and then this roughness becomes less easily noticed as well.

      • Cast Shadows and Specular Highlights
        The specular reflections on glossy objects are depth cues for surfaces
        facing the light source. e.g. a push button.

          Figure: Highlights and shadows provide information about depth

      • Previous Knowledge of Shapes and Size
        Since the visual angle of an object projected onto the retina decreases with distance, this information can be combined with previous knowledge of the object's size to determine the absolute depth of the object. For example, people are generally familiar with the size of an average automobile. This prior knowledge can be combined with information about the angle it subtends on the retina to determine the absolute depth of an automobile in a scene.

      • Position Relative to the Horizon
        As objects move away from us, objects get closer to the horizon.

    4. Motion-based depth cues -- Movement analysis in our brain
      reveals objects' speeds and directions, as well as their placement in 3D space.

      • Definition of Parallax
        Parallax is the relative position of an object's image in a set of pictures.
        It can be expressed as a motion vector.

      • Parallax Induced by Point-of-view (PoV) Movement
        e.g. train window
        Actually, every time we move our head, we generate motion parallax.

      • Parallaxes Induced by Objects' Movement
        Moving objects can show how far away they are by how fast they seem to move.
        e.g. airplane in sky looks stationary; on runway moves very fast
        e.g. some animals like birds rely on parallax to determine depth
        If information about the direction and velocity of movement is known, motion
        parallax can provide absolute depth information.

        Parallax of an object due to perspective shift: When viewed from "Viewpoint A", the object appears to be in front of the blue square. When the viewpoint is changed to "Viewpoint B", the object appears to have moved in front of the red square.
         

        An example of parallax: As the viewpoint moves side to side, the objects in the distance appear to move more slowly than the objects close to the camera.

    5. Stereoscopic depth cues
      Make comparison between the views of two eyes.
      Discrepancies between the two eyes are called binocular disparities. (Binocular disparity is converted to depth information by our brain.)

      • Convergence
        This is the muscle tension required to rotate each eye so that it is facing the focal point. horopter: a 3D curve consisting of the points for which the light falls on corresponding areas in the two retinas
        Moving an object slightly off the horopter creates a small amount of retinal disparity that gives rise to stereoscopic depth perception.

        (a)

        (b)

        (c)
        (a) The horopter.   (b) Points on the horopter are imaged at corresponding points of the two eyes.
        (c) The depth of the points not on the horopter can be estimated by their disparity.


          Disparity can be computed as the difference between the convergence angle
          to the fixation point and the stereoscopic stimulus (∠ α - ∠ β).

        We can compute the theoretical horopter based on the geometry of the eyes and viewing distance. We can also measure the actual horopter in a laboratory, which is referred to as an empirical horopter.

      • Horizontal Parallax
      • When we look at a stereopscopic picture, our brain extracts and computes
        the size of the disparities to access the distance of the objects.
        In the following figure, the projection for the left eye is on the left and the projection for the right eye is on the right, the distance between the left and right eye projections is called the horizontal parallax. Since the projections are on the same side as the respective eyes, it is called a positive parallax. Note that the maximum positive parallax occurs when the object is at infinity, at this point the horizontal parallax is equal to the interocular distance.


          Figure: Positive Parallax -- The blue and red points on the plane are
          referred to as homologous points.

        If an object is located in front of the projection plane then the projection for the left eye is on the right and the projection for the right eye is on the left. This is known as negative horizontal parallax. Note that a negative horizontal parallax equal to the interocular distance occurs when the object is half way between the projection plane and the center of the eyes. As the object moves closer to the viewer the negative horizontal parallax increases to infinity.


        Summary:

        • Objects with positive parallax are seen to be behind the stereo (projection) plane.
        • Objects with negative parallax fall in front of the plane.
        • Objects with zero parallax are at the stereo plane.
        • Parallax can be measured in inches or degrees. The parallax distance between homologous points would be in inches; the parallax angle θ describes the angle made by the homologous points and the viewer.
        • Parallax p = 2d tan ( θ / 2) , where d = the distance from the viewer to the stereo plane.
        • Maximum parallax ~ 1.5o
        • Standard computer monitor: p ~ 0.5 inches

        • Horizontal Parallax
      • Occlusion Revelations
      • Occlusion occurs when objects overlap with each other.
      • There is a thin stripe of the background object that is seen by only one eye.
      • Shape Change
      • The amount of side view perceived by each eye is different and is a function
        of an object's size and distance.
      • e.g. one eye may see the side of a die held in hand, but not the other eye;
        you may not see the sides of a building because its more than 2.5 inches wide
        ( width of your eyes )
      • Proprioception and depth perception

        • Proprioception
          from Latin proprius, meaning "one's own
          Proprioception is the sense of movement and spatial orientation arising from
          stimili within the body, indicating if the body is moving by its own effort.
          Its distinct from
        • The external senses, sight, smell, touch, hearing, and balance which describe
          the outside world to us.
        • The other internal senses, which inform us about pain in or stretching of
          internal organs.
        • Vergence, Divergence, and Convergence
          converging -- squint on a close up, or gaze on a direct parallel at a faraway landscape.

        • Accommodation
          The change in the lens of our eyes when focusing on objects would provide monoscopic
          depth cues if we were able to measure it.

        Double vision: Mounted on the front of the dolly shown here is a stereoscopic camera developed by PACE, a 3-D camera and editing firm in Burbank, CA. The device is actually a combination of two cameras: one corresponds to the viewer's left eye, the other to the right. Specialized software coordinates the cameras so that their zoom and focus settings, for instance, aren.t drastically different.
        Source: Making a Modern 3-D Movie of MIT Technology Review
    6. Rendering
      Setting up a virtual camera:
      • Toe-in (rotation) method
      • Off-axis (two centers) method

      Recall Perspective Projection:


      Viewer is at origin looking in the negative z-axis

      Ways to calculate stereo pairs (using a perspective projection):

      • Rotation, or the toe-in method:

        Problem: Causes vertical parallax and distorts the stereo plane. Not a friendly way to generate stereo.

      • Two Centers of Projection (Off-axis Projection):
        Correct method
        Choose a value e for the camera offset. Translate the scene e/2 to the left and right to generate the stereo pairs.

        Viewplane is at z = d.
        No vertical parallax:
        Use parametric equations to determine a point P = (x, y, z) on the line joining the points P0 = (x0, y0, z0) and the left eye, El = (e/2, 0, 0): P = P0 + t ( El - P0 ) At viewing plane, z = d. So we can solve for t. d = z0 + t ( 0 - z0 )
        => t = ( z0 - d ) / z0
        Knowing t, we obtain the x, y coordinates at the viewing plane: xl = d * x0 / z0 - e * d / (2z0) + e / 2
        yl = d * y0 / z0
        Similarly, we can solve for the case of right eye: xr = d * x0 / z0 - e * d / (2z0) - e / 2
        yr = d * y0 / z0
        In OpenGL, this shift is implemented by translation in the modelview matrix. The e term was recalculated for each frame in the animation based on a set parallax angle.
        An additional shift (yielding both positive and negative parallax) is created by altering the projection matrix so that the perspective view volume is asymmetrical.

      Objects that lie in front of the projection plane will appear to be in front of the computer screen; objects that are behind the projection plane will appear to be "into" the screen. It is generally easier to view stereo pairs of objects that recede into the screen. To achieve this, one would place the focal point closer to the camera than the objects of interest.

      A common measure of the stereo effect is the parallax angle defined as
        θ = 2 tan-1(DX / (2 d))
      where DX is the horizontal separation of a projected point between the two eyes and d is the distance of the eye from the projection plane. For easy fusing by the majority of people, the absolute value of θ should not exceed 1.5 degrees for all points in the scene. Note that θ is positive for points behind the scene and negative for points in front of the screen. It is not uncommon to restrict the negative value of θ to some value closer to zero since negative parallax is more difficult to fuse especially when objects cut the boundary of the projection plane.

    7. 3D Cinematography Fundamentals

      Cinema Setup for Controlling 3D Effect

    8. Interrocular (i.o.) Distance

      • .i.o. distance -- distance between the camera lenses The single most important parameter in steoscopy
        increasing i.o. distance => subjects grow
        decreasing i.o. distance => subjects shrink

        The amount of stereo effect is defined, in part,
        by the separation of the camera lenses, which defines
        the relative parallax differential between the left and
        right eye images, and can cause problems when attempting
        to bring the two images into focus.

        There is another effect of using a large inter ocular distance
        - that of miniaturisation.


        i.o. distance = 2 inches

        i.o. distance = 4 inches

      • 3 percent rule
        i.o. can be safely set at 1/30th of the distance from the camera to the foreground, i.e. 1 inch per 3 feet
        Most 3D still cameras use 2.5 inch i.o.

    9. 3D Cameras


        Parallel cameras with sensor shifted sideways, away from the optical axis.

    10. Some Applications
    11. Medical
      Stereo 3D images provide better guiduance of operating the needle used to treat severe liver disease.