T 0622/00 (Multi-view imaging system/CANON) of 2.7.2004

Summary of Facts and Submissions

I. European patent application number 95 06 97.3 claiming multiple priorities from 1994 and 1995 concerns image sensing and reproduction.

II. The European search report drawn up in respect of the application cited following documents, inter alia:

WO-A-93 14 467 (D1, published in 1993)

EP-A-0 498 542 (D2, published in 1992), and

FR-A-2 633 474, in the same patent family as document

US-A-4 956 706 published in 1990 (D3)

III. The examining division refused the application for lack of inventive step with decision of 29 November 1999. According to the reasons of the decision posted on 18 January 2000, the claimed invention was rendered obvious either by document D2 taken alone, or in combination with document D1.

IV. An appeal against the refusal decision, including a debit order for payment of the appeal fee, was filed by the applicant on 21 March 2000. The written statement setting out the grounds of appeal was filed on 26 May 2000.

V. With a letter dated 2 June 2004, the appellant replaced the claims on file, new claim 1 reading as follows:

"1. An image sensing apparatus for reproducing interpolated images for virtual viewpoints, the apparatus comprising: image input means (100) for inputting a plurality of actual images of an object from a plurality of different image pick-up positions as the apparatus is moved with respect to the object along a path, and storage means (310) for storing the images input by said input means from each of the plurality of different pick-up positions; and interpolating means for generating interpolated images by interpolating between stored actual images; and characterised in that the image sensing apparatus further comprises motion detection means (200), responsive of motion of said image sensing apparatus, for detecting motion of said image sensing apparatus as it is moved between the plurality of different pick-up positions and for generating relative position information for each actual image which is picked up; inputting means (2007) including viewpoint position means for inputting the coordinates of a plurality of arbitrary viewpoint positions different from the pick- up positions from which the plurality of stored actual images were input; and image generating means for successively generating individual images from the plurality of arbitrary viewpoint positions, said image generating means being operative to use said position information to successively generate the images from the plurality of arbitrary viewpoint positions by interpolating the plurality of actual images stored in said storage means."

VI. According to the appellant, the claimed invention differed from the cited prior art essentially in that the image sensing apparatus comprised motion detection measuring the moving amount of a hand-held camera, for example, as it was moved between the plurality of different image pick-up positions. In combination with the interpolation of the images taken, i.e. the construction of images from sensed images by some calculational method, the invention considerably reduced the amount of storage and processing requirements if compared to the prior art.

Document D1 merely provided a system for providing a smooth transition between such known control points indicative, for example, of the position of a walking person to produce an animated computer graphic.

The subject of document D2 was the generation of a visual 3-D database for generating, in real-time, perspective images for simulation. The system was based on aerial photographs including known ground control points and a Kalman filter for estimating the airplane position, altitude and orientation from the images taken. This piece of prior art was included into the scope of the first part of claim 1 but it did neither comprise the motion detection means nor the inputting means for inputting the coordinates of a plurality of arbitrary viewpoint positions nor the image generating means for successively generating individual images from the plurality of arbitrary viewpoint positions by interpolating the plurality of actual images stored in said storage means.

Document D3 used the photographed images and prior additional height information to generate a 3-D model and, therefrom, the images corresponding to a virtual viewpoint position. Therefore, as in the prior art of document D2, a sequence of previously known control points and additional information apart from the actual images and the detected position information were necessary to render the individual images.

However, none of the prior art documents disclosed the concept of a single image sensing unit having its own motion detection means for providing the positional information required to ensure correct projections and perspectives.

VII. The appellant requested that the decision under appeal be set aside and the application be allowed on the basis of the claims filed with the letter dated 2 June 2004. The appellant also requested the opportunity to put its case at oral proceedings.

VIII. In oral proceedings held on 2 July 2004 the Board announced its decision on the appeal.

Reasons for the Decision

1. The appeal complies with the requirements of Articles 106 to 108 and Rules 1(1) and 64 EPC and is thus admissible.

2. The appeal request, however, is not allowable on its merits since the requirement of inventive step (Articles 52(1) and 56 EPC) is not fulfilled.

3. In the practice of the EPO, inventive step is normally examined on the basis of the so-called problem and solution approach considering the technical contribution provided by the claimed invention to the prior art (Case Law of the Boards of Appeal of the European Patent Office, 4th edition 2001, chapter I.D).

3.1. Regarding the relevant prior art it is undisputed that Document D2 is an appropriate starting point for assessing inventive step since it concerns, like the invention as claimed, the reproduction of images for virtual viewpoints from a plurality of actual images of an object.

It is furthermore undisputed that an apparatus with the features of the first part of claim 1 is anticipated by this piece of prior art. Indeed, document D1 discloses an image sensing apparatus (aircraft 10 plus CGSI system) comprising image input means (camera 14) for inputting a plurality of actual images of an object (figure 1, building 13) from a plurality of different image pick-up positions (camera positions) as the apparatus is moved with respect to the object along a path (sequence of frames of imagery along the flight path), and storage means for storing the images input by said input means from each of the plurality of different pick-up positions (automatic database generating computer); and interpolating means for generating interpolated images on the basis of the stored actual images (generating a top down view). There is no basis in the application to construe the term "interpolation" differently than in its most general meaning of generating images by some calculational methods from stored images, i.e. by a method, for example, as used in the prior art CGSI system for generating real-time perspective images for use in simulators (see document D2, column 1, lines 5 to 18).

3.2. Furthermore, the "motion detection means" as defined in claim 1 are not clearly distinguished from the Kalman filter described in document D2, in particular column 5, lines 38 to 47. Since it lacks sufficient support by the application as originally filed the term is not be understood in the narrow sense argued by the appellant, namely that a hand-held camera should be considered, coupled directly to a motion detection means so as to produce information about the camera position in real-time.

3.3. According to the eighth embodiment, and like in document D2, the position information is derived on the basis of a plurality of corresponding input points (see the A1- publication, page 14, lines 41 to page 15, line 3). In the embodiments described first in the application, only "angular velocity sensors" are provided (see figures 3, 21, 23), the angular velocities being not sufficient information to calculate a linear length (moving amounts Bx, By, Bz), which is required by the invention to be carried out.

3.4. Nevertheless, the Board accepts the appellant's argument for the time being and construes the term narrowly in the sense of a motion sensor directly coupled to a camera or similar image sensing device, and thus as a feature distinguishing the claimed invention from document D2.

Regarding the remaining claim features the Board is not able to discover any further differences. According to document D2 an image database is produced by interpolating (in its broad sense) the images taken so to provide images as approximately seen from virtual viewpoints, i.e. in a top down view (see document D2, column 5, lines 48 to column 6, line 15). In addition such images may be produced by a CGSI system from the database for simulation, requiring appropriate inputting and viewpoint position means for inputting the coordinates of a plurality of arbitrary viewpoint positions different from the pick-up positions from which the database images were input. Like the top down view images such images for simulation of a battle field, for example, are generated successively from the arbitrary viewpoint positions by using said position information to successively generate the images from the plurality of arbitrary viewpoint positions by interpolating the plurality of actual images stored in said storage means.

3.5. Considering the above as the only difference to the prior art, it is first to be noted that the positional information about the image sensing device , i.e. x, y, z, roll, pitch and yaw are measured according to document D2 (see column 4, lines 19 to 25), which is done by the process disclosed in column 5, lines 24 ff.

The technical problem solved by the invention as claimed thus resides merely in proposing an alternative measurement concept, which consists in the light of the application as filed in measuring the (angular) degrees of freedom directly by means of a sensor. The Board considers such an alternative concept a normal design option which the skilled person would choose if such sensors are appropriate like in the prior art system of document D3 (see the "image pickup angle measuring means" 4 in figure 1 of document D3). An inventive step exceeding routine work is not necessary in conceiving such an alternative concept. The requirement of Article 52(1) and 56 EPC is thus not fulfilled.

ORDER

For these reasons it is decided that:

The appeal is dismissed.