T 0607/17 () of 24.9.2020

Summary of Facts and Submissions

I. The opponent appealed against the interlocutory decision of the opposition division maintaining European patent No. 1903303 in amended form.

Opposition had been filed against the patent as a whole and based on the grounds for opposition of Article 100(a) EPC 1973, together with Articles 54(1) and 56 EPC 1973, Article 100(b) EPC 1973, together with Article 83 EPC 1973, and Article 100(c) EPC 1973, together with Article 123(2) EPC.

The opposition division had found that the patent as amended according to the first auxiliary request then on file and the invention to which it related met the requirements of the EPC.

II. Oral proceedings before the board were held on 24 September 2020.

III. The appellant (opponent) requested that the decision under appeal be set aside and that the patent be revoked.

IV. The patentee is a party to the appeal proceedings as of right (Article 107, second sentence, EPC 1973) and the respondent in the present case and requested as a main request that the appeal be dismissed. As auxiliary requests, the respondent requested that the decision under appeal be set aside and that the patent be maintained as amended on the basis of the claims according to auxiliary requests 1, 1A, 2, 2B, 3, 4, 4C, 4D, 5, 6, 7, or 8, auxiliary requests 1 to 8 filed with the letter dated 6 September 2017 and auxiliary requests 1A, 2B, 4C and 4D filed with the letter dated 8 January 2019.

V. The following document relied on in the first-instance opposition proceedings will be referred to in the present decision:

D1: "Data Processing Algorithms for Generating Textured 3D Building Facade Meshes from Laser Scans and Camera Images", Christian Frueh et al., International Journal of Computer Vision 61(2), 159-184, 2005; XP 19216432.

VI. Reference is made to the numbering of the features 1a to 1g of claim 1 of the present main request, as defined in the opponent's statement of grounds of appeal, page 4.

VII. Independent claim 1 according to the patentee's main request reads as follows (the numbering 1a to 1g is placed at the beginning of the corresponding features):

" [1a:] A surveying method for acquiring point cloud data on a predetermined measurement range by a laser scanner (1) which is installed at a known point, characterized in that said surveying method comprises [1b:] a step of acquiring a main point cloud data on the predetermined measurement range by said laser scanner (1), [1c:] a step of detecting a range with no data acquired, [1d:] a step of acquiring a supplementary image data by taking the range with no data by stereoscopic image pickup from at least two directions by an auxiliary image pickup device (81) separated from said laser scanner, [1e:] a step of preparing a stereoscopic image based on the supplementary image data obtained by said auxiliary image pickup device, [1f:] a step of acquiring supplementary point cloud data on the range with no data acquired by an image measurement based on the stereoscopic image, and [1g:] a step of supplementing the range with no data acquired of said main point cloud data by matching of the main point cloud data and the supplementary point cloud data."

Reasons for the Decision

1. Main request - novelty

The subject-matter of claim 1 of the main request lacks novelty in view of D1 (Article 54(1) EPC 1973).

1.1 The wording of claim 1 is vague. Therefore, it has to be interpreted, namely as broadly as possible while remaining technically reasonable.

1.2 D1 discloses a surveying method for acquiring point cloud data on a predetermined measurement range by a laser scanner which is installed at a known point [feature 1a],

[The disputed features concern the expressions "predetermined" and "installed at a known point". The term "predetermined" has no precise limiting effect on the measurement range, such as implying a well-defined measurement range having specific properties. Claim 1 is silent about any such specific properties of the measurement range which would have been previously predetermined. As argued by the opponent during the oral proceedings, in D1, the predetermined measurement range corresponds e.g. to the "6769 meters long path in downtown Berkeley, starting from Blake street through Telegraph avenue" (D1, lines 1014 to 1016). This measurement range of D1 is "predetermined" in the sense that at a certain point in time before acquiring point cloud data, the path along which images would be taken was decided.

The expression "installed at a known point" has no other limiting effect as that the laser scanner is located at a known place. In particular, placing the laser scanner at a certain place on the truck of D1 or driving the truck of D1 through a certain street in Berkeley means that the laser scanner of D1 is installed at a known point on the truck and in the city of Berkeley. In D1, an approximate position of the truck is known from "an aerial image or a DSM as a global map" or by using a GPS (D1, lines 190 to 194 and 1153 to 1159). It is to be noted that, in D1, the laser scanner remains "installed at a known point" during the movement of the truck along the predetermined path. Moreover, whether the laser scanner is stationary or not during the execution of the claimed method is left open by the wording of claim 1.]

wherein the surveying method of D1 comprises

[feature 1b] a step of acquiring main point cloud data on the predetermined measurement range by said laser scanner,

[See D1, e.g. lines 160 to 165, disclosing data captured for reconstructing 3D geometry of the scanned street scenery.]

[feature 1c] a step of detecting a range with no data acquired,

[As argued by the opponent, the fact that holes in the captured data are filled in (see D1, e.g. lines 18 and 19), implies that these holes have been detected before being filled in.]

[feature 1d] a step of acquiring supplementary image data by taking the range with no data by stereoscopic pickup from at least two directions by an auxiliary image pickup device separated from said laser scanner,

[As argued by the opponent, the digital color camera of D1, line 152, corresponds to the auxiliary image pickup device of feature 1d of claim 1. The camera of D1 is installed on a moving truck so that it takes several images of an object from different directions while it is moved by the truck. This corresponds to a stereoscopic pickup. By this token, the camera images "often contain parts of the background invisible to the laser" (see D1, e.g. lines 425 to 428). The digital color camera of D1 is "separated from" the SICK LMS 2D laser scanners of D1 in the sense that they are structurally distinct items (D1, lines 149 to 152).]

[feature 1e] a step of preparing a stereoscopic image based on the supplementary image data obtained by said auxiliary image pickup device,

[As submitted by the opponent, D1, lines 428 to 432, discloses the provision of a stereoscopic image using stereo techniques based on images acquired by the digital camera.]

[feature 1f] a step of acquiring supplementary point cloud data on the range with no data acquired by an image measurement based on the stereoscopic image,

[As submitted by the opponent, D1, lines 530 to 535, discloses the provision of supplementary data based on stereo vision images in order to be used as additional information to fill in holes in the data in the range with no data acquired.]

[feature 1g] a step of supplementing the range with no data acquired of said main point cloud data by matching of the main point cloud data and the supplementary point cloud data

[As submitted by the opponent, D1 discloses filling in holes in the data obtained from laser scanning with data obtained from the digital camera (see e.g. D1, lines 425 to 432; lines 525 to 535; lines 547 to 559; lines 1032 to 1036; 1042 to 1045).]

1.3 The patentee is of the view that at least features 1a to 1d are novel over D1 and submitted essentially the following counter-arguments in favour of its view:

1.3.1 During oral proceedings, the patentee recalled the rationale and the basic working principles of the invention. The objective of the invention was to generate 3D data representing a landscape. In order to achieve this objective, there were several essential aspects, including the precise knowledge about the point from which images of the landscape were taken, i.e. the laser scanner had to be installed at a known point, defined as a Cartesian point with Cartesian coordinates. Moreover, the surveying method had to be executed according to a predefined time flow described in paragraphs [0163] to [0177] and figure 19 of the patent. In particular, it was essential that the step of detecting a range with no data (step 1c), preparing the subsequent step of taking exactly said detected range with no data by stereoscopic pickup (step 1d), was executed before said step 1d. In other words, after actively detecting in step 1c the holes in the image obtained from the laser scanner, a targeted request filling specifically these holes was made in step 1d. Due to such a targeted request, the holes in the image obtained from the laser scanner could be completely and reliably filled with data from the stereoscopic image (steps 1e to 1g), contrary to what happened in the surveying method of D1.

The board acknowledges that the patent describes a surveying method with certain technical details. However, the board cannot follow the patentee's argumentation based on the assumption that these technical details are somehow included in or implicitly derivable from the wording of claim 1. These technical details of a surveying method described only in the patent description do not limit the scope of claim 1 and, therefore, must be disregarded when comparing the subject-matter of claim 1 with the disclosure of the prior art document D1.

1.3.2 Feature 1a - "predetermined"

According to the patentee, in D1, point cloud data was acquired along an arbitrary path followed by the truck when driving along the streets in Berkeley. Depending on obstacles encountered by the truck, such as traffic jams or route deviations, the path followed by the truck was not predetermined but arbitrary. Therefore, the sub-feature relating to the term "predetermined" was not known from D1.

The board is not convinced by this argument because the term "predetermined" has no clearly limiting effect on the scope of claim 1 (see point 1.2 above). Specifying in D1 the acquisition of data of objects located "in downtown Berkeley, starting from Blake street through Telegraph avenue", corresponds to specifying the acquisition of data "on a predetermined measurement range" as claimed.

1.3.3 Feature 1a - "installed at a known point"

The patentee argued that the meaning of the term "installed" in the technical field of patents went beyond the simple meaning of the general term "placed". The term "installed" implied a kind of installation process, such as the levelling process described in the patent, paragraphs [0069] to [0073]. Moreover, in order to define a meaningful surveying method, it was clear to the skilled person that the point at which the laser scanner was installed had to be precisely known beforehand. Taking images with the laser scanner at a random location did not allow to supplement subsequently said data acquired by the laser scanner with data from a stereoscopic pickup separated from the laser scanner. Therefore, the Cartesian coordinates of the point at which the laser scanner was installed had to be known. Furthermore, from the wording "at a known point", it was derivable that the installation point may not move during the surveying method but had to remain stationary. Contrary to this, in D1, the laser scanner was installed on a truck whose position was unknown during the surveying method. The position only became known after the data was acquired and even then only an approximate value (D1, lines 194 to 198; lines 1154 to 1159). Moreover, the truck and the laser scanner were not stationary but moving during the surveying method. The patentee concluded that the sub-feature relating to the expression "installed at a known point" was novel over D1.

The board cannot follow the patentee's argumentation which is based on a too narrow interpretation of the claim wording. The laser scanner of D1 placed at a certain position on a truck driving through Berkeley falls under the wording "installed at a known point" of feature 1a. Claim 1 does not specify any of the aspects of concrete installation steps, knowledge of a precise position, moment when the position must be known or constraint of immobility of the laser scanner.

1.3.4 Features 1b, 1c, 1d - temporal order

For the patentee, it was clear from the wording of claim 1 that the claimed steps had to be executed according to a well-defined temporal order which was described in more detail in paragraphs [0162] to [0177] and figure 19 of the patent. As was apparent from claim 1, for acquiring supplementary image data by taking the range with no data, as defined to step 1d, the "range with no data" had to be known beforehand. Since the "range with no data" corresponded to holes in the main point cloud data, i.e. the "range with no data" was created during step 1b, the "range with no data" was detectable only after acquiring the main point cloud data as defined in step 1b. In order to be known and used in step 1d, the "range with no data" had to be detected first according to step 1c. It followed that steps 1b, 1c and 1d had to be executed in said temporal order. This was confirmed by the use of "a" in the expression "a range with no data" in step 1c and the use of "the" in the expression "the range with no data" in step 1d. Step 1d made explicit reference to the "range with no data" defined in 1c and previously created in step 1b. In contrast, in D1, the main point cloud data (step 1b) and the supplementary image data (step 1d) were acquired simultaneously.

The board is not convinced by the patentee's argument. While figure 19 of the patent discloses a chronological order as described by the patentee, the claim wording is so broad that it covers also a simultaneous image data acquisition by the laser scanner and the stereoscopic camera. The board cannot see any indication in claim 1 that step 1d is necessarily only carried out after step 1b. The fact that step 1c reads "a range with no data" and step 1d reads "the range with no data" merely signifies that the same "range with no data" is used in both steps, regardless of whether step 1d is executed after step 1c or simultaneously. It is to be noted that the list of steps 1b to 1g is introduced in claim 1 in general terms by reading "... said surveying method comprises ..." without any emphasis that one step is to be performed before or after another step.

1.3.5 Feature 1d - "separated from said laser scanner"

The patentee submitted that due to the stereoscopic pickup device being "separated from said laser scanner" images were taken from a different perspective than by the laser scanner. This different perspective allowed the stereoscopic pickup device to acquire image data corresponding exactly to the holes in the images taken by the laser scanner. In contrast thereto, in D1, the camera and the laser scanner formed one unit and could not be moved separately to obtain different perspectives, but were oriented in the same direction and synchronised with one another (D1, lines 167 to 177 and lines 706 to 708). This meant that in D1 the stereoscopic pickup device and the laser scanner were completely dependent on each other instead of separate.

This argument is not found convincing by the board. The expression in feature 1d, "an auxiliary image pickup device (81) separated from said laser scanner", in its broadest meaning, merely defines two items being structurally distinct. This is indeed the case in D1 disclosing, on the one hand, SICK LMS 2D laser scanners and, on the other hand, a digital color camera (D1, lines 149 to 152), the laser scanners not being integrated in the camera. Beyond that, in no way can it be inferred from the expression "separated from the laser scanner" that the camera takes images from a different perspective than the laser scanner.

1.3.6 Feature 1d - complete filling of all data-lacking portions

The patentee argued that feature 1d, when read in conjunction with paragraphs [0010], [0172] and [0173] of the patent and with a mind willing to understand, implied that the holes in the image acquired by the laser scanner were completely filled on the basis of data from the stereoscopic camera so as to leave no data-lacking portion. After having detected a "range with no data" in the preceding step 1c, the stereoscopic camera acquired said missing data in a targeted manner in step 1d so that 100% of the gaps were reliably filled according to step 1g. On the contrary, in D1, the gaps in the image acquired by the laser scanner could not be filled completely at all by the stereoscopic camera, since an additional interpolation process was required (D1, lines 1042 to 1049; figures 26(a) to 26(c)).

Contrary to the patentee's submission, step 1c does not define that all data-lacking portions in the image acquired by the laser scanner are to be detected, since already a single data-lacking portion falls under the wording of step 1c. Moreover, claim 1 does not specify the quality of the image supplementation. Therefore, the image supplementation of figure 26(b) of D1, based on "stereo vision vertices", does fall under the wording of features 1d and 1g in spite of the fact that "the outline of the original holes can still be recognized" (D1, lines 1042 to 1049).

1.4 It follows that the subject-matter of claim 1 is anticipated by the method disclosed in D1.

1.5 The decision under appeal, therefore, must be set aside.

2. Remittal of the case

2.1 The board notes that the patentee had filed a total of twelve auxiliary requests during the opposition-appeal proceedings and that currently it maintained all these requests. The board further notes that many of these requests are in substance identical to the requests filed by the patentee during the first-instance opposition proceedings so that there is no thorough reason for not admitting at least some of these requests into the opposition-appeal proceedings.

2.2 The subject-matter of a claim is generally defined by the features which are actually present in the claim. They have to be interpreted as broadly as reasonable. A limitation of the claimed subject-matter going beyond the wording of the claim, e.g. on the basis of explanations in the description of the patent or on the basis of a specific interpretation of the claim wording, is not allowable.

In the present case, the wording of claim 1 is interpreted fundamentally differently by the board than by the opposition division and the patentee, namely, the board interprets the terms of the claim in a substantially broader sense (see points 1.2 and 1.3 above). The facts of the case are fundamentally changed by this new interpretation of the claim wording. The consequence thereof is that the board comes to the conclusion that the subject-matter of claim 1 of the main request lacks novelty with respect to D1, whereas the appealed decision concluded that neither D1, nor any other available prior art document anticipated the subject-matter of claim 1.

2.3 The patentability of the subject-matter of claim 1 of the auxiliary requests has to be assessed on the basis of this new interpretation of the claim wording for the first time before the board of appeal. This amounts to a fresh case which alone justifies the case to be remitted to the department of first instance for further prosecution. It is to be noted that additional issues under Articles 83 or 84 EPC 1973 or Article 123(2) EPC might arise and have to be examined.

In addition, in view of the numerous objections of lack of novelty and lack of inventive step, based on a large number of prior art documents, raised by the opponent throughout the appeal proceedings and in view of the large number of auxiliary requests on file, the board judges that the patentability assessment of claim 1 of the auxiliary requests based on the new interpretation of the claim wording comprises a complexity in terms of the number of new issues not compatible with the primary object of the appeal proceedings to review the decision under appeal in a judicial manner (Article 12(2) RPBA 2020).

2.4 During the oral proceedings before the board, the opponent stated that even though it would prefer continuing the debate on patentability of the claimed subject-matter of the auxiliary requests, it agreed to the remittal of the case to the first instance. The patentee also agreed to the remittal of the case.

2.5 For the above reasons, the board decides to make use of its discretion under Article 111(1) EPC 1973 and Article 11 RPBA 2020 in remitting the case to the opposition division for further prosecution.

Order

For these reasons it is decided that:

1. The decision under appeal is set aside.

2. The case is remitted to the department of first instance for further prosecution.