An image or video sequence is a series of two-dimensional (2-D) images sequen tially ordered in time. Image sequences can be acquired, for instance, by video, motion picture, X-ray, or acoustic cameras, or they can be synthetically gen erated by sequentially ordering 2-D still images as in computer graphics and animation. The use of image sequences in areas such as entertainment, visual communications, multimedia, education, medicine, surveillance, remote control, and scientific research is constantly growing as the use of television and video systems are becoming more and more common. The boosted interest in digital video for both consumer and professional products, along with the availability of fast processors and memory at reasonable costs, has been a major driving force behind this growth. Before we elaborate on the two major terms that appear in the title of this book, namely motion analysis and image sequence processing, we like to place them in their proper contexts within the range of possible operations that involve image sequences. In this book, we choose to classify these operations into three major categories, namely (i) image sequence processing, (ii) image sequence analysis, and (iii) visualization. The interrelationship among these three categories is pictorially described in Figure 1 below in the form of an "image sequence triangle".

The processing of image sequences has a broad spectrum of important applica tions including target tracking, robot navigation, bandwidth compression of TV conferencing video signals, studying the motion of biological cells using microcinematography, cloud tracking, and highway traffic monitoring. Image sequence processing involves a large amount of data. However, because of the progress in computer, LSI, and VLSI technologies, we have now reached a stage when many useful processing tasks can be done in a reasonable amount of time. As a result, research and development activities in image sequence analysis have recently been growing at a rapid pace. An IEEE Computer Society Workshop on Computer Analysis of Time-Varying Imagery was held in Philadelphia, April 5-6, 1979. A related special issue of the IEEE Transactions on Pattern Anal ysis and Machine Intelligence was published in November 1980. The IEEE Com puter magazine has also published a special issue on the subject in 1981. The purpose of this book is to survey the field of image sequence analysis and to discuss in depth a number of important selected topics. The seven chap ters fall into two categories. Chapters 2, 3, and 7 are comprehensive surveys on, respectively, the whole field of image sequence analysis, efficient coding of image sequences, and the processing of medical image sequences. In Chapters 1, 4, 5, and 6 the authors present mainly results of their own research on, respectively, motion estimation, noise reduction in image sequences, moving object extraction, and occlusion.

Digital image sequences (including digital video) are increasingly common and important components in technical applications ranging from medical imaging and multimedia communications to autonomous vehicle navigation. The immense popularity of DVD video and the introduction of digital television make digital video ubiquitous in the consumer domain. Digital Image Sequence Processing, Compression, and Analysis provides an overview of the current state of the field, as analyzed by leading researchers. An invaluable resource for planning and conducting research in this area, the book conveys a unified view of potential directions for further industrial development. It offers an in-depth treatment of the latest perspectives on processing, compression, and analysis of digital image sequences. Research involving digital image sequences remains extremely active. The advent of economical sequence acquisition, storage, and display devices, together with the availability of computing power, opens new areas of opportunity. This volume delivers the background necessary to understand the strengths and weaknesses of current techniques and the directions that consumer and technical applications may take over the coming decade.

An image or video sequence is a series of two-dimensional (2-D) images sequen tially ordered in time. Image sequences can be acquired, for instance, by video, motion picture, X-ray, or acoustic cameras, or they can be synthetically gen erated by sequentially ordering 2-D still images as in computer graphics and animation. The use of image sequences in areas such as entertainment, visual communications, multimedia, education, medicine, surveillance, remote control, and scientific research is constantly growing as the use of television and video systems are becoming more and more common. The boosted interest in digital video for both consumer and professional products, along with the availability of fast processors and memory at reasonable costs, has been a major driving force behind this growth. Before we elaborate on the two major terms that appear in the title of this book, namely motion analysis and image sequence processing, we like to place them in their proper contexts within the range of possible operations that involve image sequences. In this book, we choose to classify these operations into three major categories, namely (i) image sequence processing, (ii) image sequence analysis, and (iii) visualization. The interrelationship among these three categories is pictorially described in Figure 1 below in the form of an "image sequence triangle".

This volume contains the proceedings of the NATO Advanced Study Institute on "Image Sequence Processing and Dynamic Scene Analysis" held 21 June - 2 July, 1982 in Hotel Maritim, Braunlage/Harz, Federal Republic of Germany. The organizing eommittee of the institute consists of T.S. Huang (Director), H.G. Musmann (Co Director), H.H. Nagel (Consultant), and C.E. Liedtke and W. Geuen (Local 'arrangement). This Institute was devoted to the rapidly emerging field of image sequence processing and dynamic scene analysis which has man! important applications in cluding target tracking, television bandwidth compression, highway traffic moni toring, and analysis of heart wall motion for medical diagnosis. The lectures and discussions in this Institute fell into three overlapping categories: Motion estimation; pattern recognition and artificial intelligence techniques in dynamic scene analysis; and, applications. 1) Motion estimation - One of the most important problems in image sequence analysis and dynamic scene analysis is displacement and motion estimation. For example, in interframe coding using temporal DPCM, displacement estimation and compensation can improve efficiency significantly. Also, estimated motion parameters can be powerful cues in target segmentation, detection, and classification. In this Institute, a number of recently developed techniques for displacement and motion estimation were discussed.

Motion and Structure from Image Sequences is invaluable reading for researchers, graduate students, and practicing engineers dealing with computer vision. It presents a balanced treatment of the theoretical and practical issues, including very recent results - some of which are published here for the first time. The topics covered in detail are: - image matching and optical flow computation - structure from stereo - structure from motion - motion estimation - integration of multiple views - motion modeling and prediction Aspects such as uniqueness of the solution, degeneracy conditions, error analysis, stability, optimality, and robustness are also investigated. These details together with the fact that the algorithms are accessible without necessarily studying the rest of the material, make this book particularly attractive to practitioners.

Many approaches have been proposed to solve the problem of finding the optic flow field of an image sequence. Three major classes of optic flow computation techniques can discriminated (see for a good overview Beauchemin and Barron IBeauchemin19951): gradient based (or differential) methods; phase based (or frequency domain) methods; correlation based (or area) methods; feature point (or sparse data) tracking methods; In this chapter we compute the optic flow as a dense optic flow field with a multi scale differential method. The method, originally proposed by Florack and Nielsen [Florack1998a] is known as the Multiscale Optic Flow Constrain Equation (MOFCE). This is a scale space version of the well known computer vision implementation of the optic flow constraint equation, as originally proposed by Horn and Schunck [Horn1981]. This scale space variation, as usual, consists of the introduction of the aperture of the observation in the process. The application to stereo has been described by Maas et al. [Maas 1995a, Maas 1996a]. Of course, difficulties arise when structure emerges or disappears, such as with occlusion, cloud formation etc. Then knowledge is needed about the processes and objects involved. In this chapter we focus on the scale space approach to the local measurement of optic flow, as we may expect the visual front end to do. 17. 2 Motion detection with pairs of receptive fields As a biologically motivated start, we begin with discussing some neurophysiological findings in the visual system with respect to motion detection.