T 2486/19 () of 26.10.2023

Summary of Facts and Submissions

I. The appeal is against the examining division's decision to refuse European patent application No. 10 844 834.1, published as international patent application WO 2011/093752 A1.

II. The prior-art documents cited in the decision under appeal were the following:

D1: EP 0 907 144 A2

D2: EP 1 612 733 A2

III. The decision under appeal was based on the ground that the claims of the main request and the first auxiliary request then on file did not meet the requirements of Article 83 EPC. The second auxiliary request was not admitted into the proceedings under Article 114(2) EPC.

IV. The applicant (appellant) filed notice of appeal. With the statement of grounds of appeal, the appellant filed claims according to a main request and an auxiliary request. It indicated a basis in the application as filed for the claimed subject-matter and provided arguments to support its opinion that the claims met the requirements of Articles 56 and 83 EPC. Furthermore, the appellant requested that the appeal fee be reimbursed because its right to be heard under Article 113(1) EPC had been violated.

V. The appellant was summoned to oral proceedings. In a communication under Article 15(1) RPBA 2020, the board gave the following preliminary opinion.

(a) The board could see no basis for reimbursement of the appeal fee under Rule 103(1)(a) EPC.

(b) Claim 1 of the main request and the auxiliary request did not meet the requirements of Articles 83 and 84 EPC.

VI. With its letter dated 28 August 2023, the appellant filed amended claims of a first and a second auxiliary request to replace all auxiliary requests on file. The appellant requested that the case be remitted to the examining division. It reasoned that the examining division made an error when raising fresh objections under Article 83 EPC. The appellant argued that the first and second auxiliary requests formed a basis for the appeal proceedings pursuant to Article 12(1)(d) RPBA 2020 and should be admitted into the proceedings under Articles 13(1) and (2) RPBA 2020. It also indicated a basis in the application as filed for the amendments and provided arguments to support its opinion that the claims of all requests met the requirements of Articles 83 and 84 EPC.

VII. With its letter dated 26 September 2023, the appellant withdrew its request for oral proceedings.

Thus, it appears from the file that the appellant requests that the decision under appeal be set aside and the case be remitted to the department of first instance, or that a European patent be granted on the basis of the claims of the main request filed with the statement of grounds of appeal or, alternatively, on the basis of the claims of either the first or second auxiliary request, both auxiliary requests filed with the appellant's letter dated 28 August 2023. Furthermore, the appellant requests that the appeal fee be reimbursed.

VIII. Claim 1 of the main request reads as follows:

"An automated three dimensional mapping method estimating three dimensional models taking advantage of a plurality of images (44), wherein

- the positions (x, y, z) and attitudes (alpha, beta, gamma) for at least one camera (2) are recorded when images are taken, wherein said at least one camera (2) is geometrically calibrated to indicate the direction of each pixel in an image (44),

- a stereo disparity is calculated for a plurality of image pairs (27-31) covering a same scene position setting a disparity and a certainty measure estimate for each stereo disparity

- the different stereo disparity estimates are weighted together to form a 3D height model,

- the stereo disparity estimates are reweighted automatically and adaptively based on the normal vectors of the estimated 3D height model, and wherein

- the certainty measure estimate for each disparity is set (46) in consideration of visibility, local contrast, and/or resolution"

IX. Claim 1 of the first auxiliary request reads as follows (features added compared with claim 1 of the main request are underlined and deleted features are [deleted: struck through]):

"An automated three dimensional mapping method estimating three dimensional models taking advantage of a plurality of images (44), wherein

- the positions (x, y, z) and attitudes (alpha, beta, gamma) for at least one camera (2) are recorded when images are taken, wherein said at least one camera (2) is geometrically calibrated to indicate the direction of each pixel in an image (44),

- a stereo disparity is calculated for a plurality of image pairs (27-31) covering a same scene position setting a disparity and a certainty measure estimate for each stereo disparity,

- the different stereo disparity estimates are weighted together to form a 3D height model taking notice of the certainty measure estimate,

- the stereo disparity estimates are reweighted automatically and adaptively based on the normal vectors of the estimated 3D height model, and wherein

- the certainty measure estimate for each disparity is set (46) in consideration of [deleted: visibility, ]local contrast and/or resolution."

X. Claim 1 of the second auxiliary request reads as follows (features added compared with claim 1 of the main request are underlined and deleted features are [deleted: struck through]):

"An automated three dimensional mapping method estimating three dimensional models taking advantage of a plurality of images (44), wherein

- the positions (x, y, z) and attitudes (alpha, beta, gamma) for at least one camera (2) are recorded when images are taken, wherein said at least one camera (2) is geometrically calibrated to indicate the direction of each pixel in an image (44),

- a stereo disparity is calculated for a plurality of image pairs (27-31) covering a same scene position setting a disparity and a certainty measure estimate for each stereo disparity,

- the different stereo disparity estimates are weighted together to form a 3D height model taking notice of the certainty measure estimate,

- the stereo disparity estimates are reweighted automatically and adaptively based on the normal vectors of the estimated 3D height model taking obscuration into consideration, such that for example an image taken from above is used to estimate the roof structure and not the sides of a building, and wherein

- the certainty measure estimate for each disparity is set (46) in consideration of [deleted: visibility, ]local contrast and/or resolution."

XI. The appellant's arguments relevant to the present decision may be summarised as follows.

Remittal and reimbursement of the appeal fee

(a) During the oral proceedings before the examining division fresh objections under Article 83 EPC, which took the appellant by surprise, were raised. In view of the examining division's statements in the annex to the summons to oral proceedings, the appellant could not have prepared to address these objections.

Due to the change of subject during the oral proceedings the appellant thus did not have sufficient opportunity to present counter-arguments during those oral proceedings before the examining division. The examining division therefore acted incorrectly by not postponing the oral proceedings.

In view of the above, the appellant's right to be heard under Article 113(1) EPC was violated. Thus, the case should be remitted to the examining division for further prosecution, and the appeal fee should be reimbursed.

Main request

(b) Using normal vectors for reweighting stereo disparities was common general knowledge of the person skilled in the art.

(c) As evidence that weighting of disparity estimates based on normal vectors was common general knowledge the appellant referred to document WO 2009/084993 A1 (see statement of grounds of appeal, section 2.1.2).

(d) Page 6, lines 9 to 15 of the description indicated at least one way to carry out the feature in question. From this passage it was clear that the normal vectors of the estimated 3D height model were used in the reweighting of the stereo disparity estimates to enable the use of stereo disparities originating from image pairs taken from obscured attitudes/positions to be avoided in the reweighting to form a 3D model.

First and second auxiliary requests

(e) The first and second auxiliary requests formed a basis for the appeal proceedings under Article 12(1)(d) RPBA 2020 because they had been submitted in reply to the board's communication (see letter dated 28 September 2023: sections II.1 and III.1, paragraph under the heading "Article 12(1)(d) RPBA 2020").

(f) The first and second auxiliary requests fulfilled the requirements of Article 83 EPC for the reasons set out in point (d) above and should therefore be admitted into the appeal proceedings (see letter dated 28 September 2023: sections II.3 and III.2).

Reasons for the Decision

1. The appeal is admissible.

2. Request for remittal of the case to the examining division for further prosecution (Article 111(1), second sentence, EPC, and Article 11 RPBA 2020)

2.1 Under Article 111(1), second sentence, EPC, the board may either exercise any power within the competence of the department which was responsible for the decision appealed or remit the case to that department for further prosecution.

Under Article 11 RPBA 2020 the board will not remit a case to the department whose decision was appealed for further prosecution, unless special reasons present themselves for doing so. As a rule, fundamental deficiencies which are apparent in the proceedings before that department constitute such special reasons.

2.2 The appellant argued that during the oral proceedings before the examining division fresh objections under Article 83 EPC, which took the appellant by surprise, were raised. In view of the examining division's statements in the annex to the summons, the appellant could not have prepared to address these objections.

To be in a position to argue against these objections, the appellant would have needed to prepare a basis for the standpoint of the skilled person at the date of filing and their common general knowledge at that point in time.

Due to the change of subject during the oral proceedings before the examining division, therefore, the appellant did not have sufficient opportunity to present counter-arguments during those oral proceedings. In this regard, the appellant referred to the decisions T 1164/00, T 2235/12 and T 248/92 (see statement of grounds of appeal, section 1).

The appellant submitted that it had tried to come up with arguments during the oral proceedings before the examining division but was indeed taken by surprise and, in this stressful situation, "did not find itself" to request postponement of the oral proceedings. Given the nature and extent of the fresh objections under Article 83 EPC, the appellant was not given enough time to respond to all of these objections. The examining division therefore acted incorrectly by not postponing the oral proceedings (see letter dated 28 August 2023: section I.1, first paragraph to third paragraph).

In view of the above, the appellant submitted that its right to be heard under Article 113(1) EPC had been violated (see point XI.(a) above).

2.3 The board is not convinced by these arguments, for the following reasons.

2.3.1 The board agrees with the appellant that fresh objections under Article 83 EPC were raised by the examining division during the oral proceedings. These objections were indeed different from the objection under Article 83 EPC mentioned in point 2.3.1 of the annex to the summons.

2.3.2 However, when faced with these fresh objections during the oral proceedings, the appellant was asked by the chairman of the examining division whether it needed a break to consider the new objections (see minutes of oral proceedings, point 4.8). There is no indication that the appellant asked for interruption or postponement of the oral proceedings. Rather, the appellant commented in substance on the new objections and provided several counter-arguments (see minutes of oral proceedings, points 5.1, 5.2 and 7).

2.4 Therefore, in the absence of any request by the appellant for the oral proceedings to be interrupted or postponed, the board finds that the appellant was not denied the time it deemed necessary to deal with the fresh objections and thus the appellant's right to be heard under Article 113 EPC was not violated.

2.5 The appellant referred to the following decisions: T 1164/00, T 2235/12 and T 248/92.

2.5.1 T 1164/00 relates to a situation in which the patent proprietor objected to a new ground of opposition being introduced by the opposition division for the first time during the oral proceedings. The appellant/patent proprietor reasoned that the opposition division had committed a substantial procedural violation by raising a new ground during oral proceedings. The board found that the opposition division should have adjourned the oral proceedings in order to give the appellant sufficient time to prepare a suitable defence against the new objections.

In T 2235/12 the examining division introduced two new documents two days before the oral proceedings and raised a new novelty objection based on one of those documents. The applicant requested postponement of the oral proceedings, which the examining division rejected. The board found that rejecting the appellant's request for postponement of the oral proceedings violated the appellant's right to be heard.

The board finds that cases T 1164/00 and T 2235/12 are clearly different from the present situation. In those cases the appellant objected to the introduction of a new ground of opposition or requested postponement of the oral proceedings when faced with a fresh objection and did not enter into a substantive discussion.

2.5.2 In T 248/92 the relevant objection of lack of inventive step based on common general knowledge was raised for the first time during the oral proceedings before the examining division. In that case the applicant did not present any arguments or requests in relation to the new objection raised by the examining division. In particular, the applicant did not request adjournment of the oral proceedings, and nor did it request that the proceedings be continued in writing. Therefore, the board found that the examining division was entitled to conclude that the matter had been thoroughly discussed and that a decision could be taken. In these circumstances, no violation of Article 113(1) EPC was found.

The board finds that it is even clearer in the current case than it was in T 248/92 that the appellant did not request interruption or postponement of the oral proceedings but entered into the substantive discussion of the fresh objections by providing several counter-arguments. Hence, at the end of that discussion the examining division was entitled to conclude that the matter had been thoroughly discussed and that a decision could be taken.

2.6 In view of the above, the board sees no violation of the appellant's right to be heard or any other fundamental deficiencies which are apparent in the proceedings before the examining division.

2.7 Therefore, the board does not remit the case to the department of first instance in accordance with Article 11 RPBA 2020 and Article 111(1), second sentence, EPC.

3. Request for reimbursement of the appeal fee (Rule 103(1)(a) EPC) due to a violation of the right to be heard (Article 113(1) EPC)

3.1 Under Rule 103(1)(a) EPC the appeal fee is reimbursed in full where the board deems an appeal to be allowable, if such reimbursement is equitable by reason of a substantial procedural violation.

3.2 For the reasons set out in point 2. above, the board finds that the appellant's right to be heard was not violated in the first-instance proceedings and thus no substantial procedural violation occurred. Therefore, the appeal fee is not reimbursed.

4. Main request - sufficiency of disclosure (Article 83 EPC)

4.1 Under Article 83 EPC, a European patent application must disclose an invention in a manner sufficiently clear and complete for it to be carried out by the person skilled in the art.

An invention is in principle sufficiently disclosed if at least one way is clearly indicated enabling the person skilled in the art to carry out the invention (see Case Law of the Boards of Appeal of the European Patent Office, 10th edition, 2022 ("Case Law"), II.C.5.2).

The claimed invention must be sufficiently disclosed as from the effective date of filing of the application. This disclosure is aimed at the person skilled in the art who may rely on common general knowledge to supplement the information contained in the patent application. Textbooks and general technical literature form part of the common general knowledge, which however does not normally include scientific articles and patent literature (see Case Law, II.C.4.1).

4.2 Claim 1 of the main request contains the following feature: "the stereo disparity estimates are reweighted automatically and adaptively based on the normal vectors of the estimated 3D height model"

4.3 The board finds that this feature is not disclosed in a manner sufficiently clear and complete for it to be carried out by the person skilled in the art. The reasons are as follows.

4.3.1 The application as originally filed contains the following disclosures relating to this feature:

- "The stereo disparity estimates could for example be reweighted based on normal vectors of the estimated 3D model" (see page 2, lines 17 and 18)

- "From this first model the original stereo estimates are reweighted automatically and adaptively based on normal vectors of the estimated 3D model taking information such as visibility, local contrast, resolution and visibility such as obscuration into consideration. In this connection for example an image taken straight above a building is used to estimate the roof structure and not the sides of a building. Another example could be to avoid mixing of the front side and back side of buildings" (see page 6, lines 9 to 15).

4.3.2 The board is of the opinion that the passages of the original application quoted above do not provide more insight than the wording of the feature itself.

In particular, the direction of the normal vector of a roof surface may differ significantly from the orientation of the optical axis of a camera taking an image from straight above a building, depending on the type and inclination of the roof.

4.3.3 Furthermore, the board is of the opinion that it is not evident that the person skilled in the art, using their common general knowledge, would understand how to use the normal vectors of an estimated 3D model to reweight the stereo disparity estimates.

4.3.4 The appellant submitted that using normal vectors for reweighting stereo disparities was common general knowledge of the person skilled in the art. The appellant argued that, if a stereo pair of images was recorded from camera angles deviating only slightly from the normal vector, the resulting disparity estimate would be given a high weighting based on a good correspondence between the images comprising essentially the same features (see statement of grounds of appeal, section 2.1.2 thereof, and point XI.(b) above).

The board is not convinced by this argument, for two reasons.

Firstly, the appellant has not provided any evidence, e.g. a reference to a textbook about stereo imaging, to corroborate its assumption.

Secondly, the board doubts whether the relationship submitted by the appellant - according to which camera angles that deviate only slightly from the normal vector will yield the most reliable disparity estimates - is common general knowledge and is generally true. The board finds it helpful to illustrate this with the following example. A human being with ordinary estimation skills can produce a disparity estimate, i.e. an estimate of a distance between themselves and objects they see with their eyes. If the distance between the object and the human being is small, this distance can normally be estimated quite accurately. However, as the distance between the object and the human being increases, the distance estimate becomes less accurate. A small distance between the human being and the object means that the angles between the rays connecting each eye with the object and the vector connecting the point between the eyes and the object are substantial. Conversely, a large distance between the human being and the object means that the rays connecting each eye with the object are almost parallel to the vector connecting the point between the eyes and the object.

4.3.5 As evidence that the weighting of disparity estimates based on normal vectors was common general knowledge, the appellant referred to document WO 2009/084993 A1 (see point XI.(c)).

The board is not convinced by this line of argument, for two reasons.

Firstly, this document discloses which image data to use when rendering a surface of a 3D model. In particular, the image data for rendering is selected such that the direction of the camera when capturing the image data is substantially normal to the surface to be rendered - see for example page 5, lines 19 to 24 of this document stating: "When displaying the virtual image in the most straight forward way each displayed pixel are chosen from the data base closest in angle to the direction of display of the three data bases. ... If the viewer looks straight south, data is only fetched from the data base recorded from north according to the example given above."

However, the question of which image data to use for rendering a surface of a 3D model is different from reweighting disparity estimates which would lead to a modification of the height of the 3D model.

Secondly, the board is not convinced that one particular published patent application constitutes sufficient proof of common general knowledge.

4.3.6 The appellant argued that the second passage of the description quoted in point 4.3.1 above indicated at least one way of carrying out the feature in question. In this passage it was disclosed that the reweighting could be based on obscuration using the normal vectors of the 3D height model, such that an image taken from above, for example, is used to estimate the roof structure and not the sides of a building. The appellant argued that the positions and attitudes for at least one camera were recorded when the images used in the claimed method were taken. Further, the normal vectors of the estimated 3D height model inherently contained information concerning which camera positions/attitudes had an obscured line of sight towards each point in the 3D height model. Hence, it was clear from the second passage of the description quoted in point 4.3.1 above that the normal vectors of the estimated 3D height model were used in the reweighting of the stereo disparity estimates to enable the use of stereo disparities originating from image pairs taken from obscured attitudes/positions to be avoided in the reweighting to form a 3D model, given that the positions and attitudes of the used images were known (see letter dated 28 September 2023: section I.1, paragraphs under the heading "Point 11.4 in the appealed decision", and point XI.(d) above).

The board is not convinced by this argument because it is not apparent that the normal vectors of the estimated 3D height model inherently contain information concerning which camera positions/attitudes have an obscured line of sight towards each point in the 3D height model. Usually, this information is only available once position and shape of any objects on a line of sight between a point on the 3D model and a camera position are known.

It may be assumed that the ensemble of normal vectors of the 3D height model defines a surface of that height model and that a particular point on that surface has an obscured line of sight towards a camera taking images. However, then it is not apparent how a disparity estimate for that particular point on the 3D model could have been created from image pairs taken by the camera, namely because the view of that camera towards the particular point on the 3D model had been obscured.

4.4 In view of the above, the board finds that claim 1 of the main request does not meet the requirements of Article 83 EPC.

5. First auxiliary request - admittance (Article 13(2) RPBA 2020)

5.1 The appellant argued that the first auxiliary request formed a basis for the appeal proceedings under Article 12(1)(d) RPBA 2020 because it had been submitted in reply to the board's communication (see point XI.(e) above).

Article 12(1)(d) RPBA 2020 provides that appeal proceedings are based on any communication sent by the board and any answer thereto filed pursuant to directions of the board.

The board is not convinced by the appellant's argument because Article 12(1)(d) RPBA 2020 states that those answers to communications from the board which are filed "pursuant to directions of the board" become a basis for the appeal proceedings.

However, in its communication the board gave merely a preliminary opinion, namely that claim 1 of the main request and the auxiliary request then on file did not meet the requirements of Articles 83 and 84 EPC. In its preliminary opinion, the board did not "direct" the appellant, i.e. did not give the appellant any instructions, to file amendments.

Therefore, the board finds that it cannot be concluded from Article 12(1)(d) RPBA 2020 that the first auxiliary request forms a basis for the appeal proceedings.

5.2 The first auxiliary request was filed after notification of the summons to oral proceedings. This auxiliary request is therefore an amendment within the meaning of Article 13(2) RPBA 2020.

5.3 Under Article 13(2) RPBA 2020, any amendment to a party's appeal case made after notification of a summons to oral proceedings is, in principle, not to be taken into account unless there are exceptional circumstances, which have been justified with cogent reasons.

Article 13(2) RPBA 2020 imposes the most stringent limitations on appeal submissions made at an advanced stage of the proceedings (see Supplementary publication 2, OJ EPO 2020, Explanatory remarks on Article 13(2), first paragraph, second sentence).

When exercising its discretion under Article 13(2) RPBA 2020, the board may also rely on criteria set out in Article 13(1) RPBA 2020 (see ibid., Explanatory remarks on Article 13(2), fourth paragraph).

Under Article 13(1) RPBA 2020 the board is to exercise its discretion in view of, inter alia, whether the appellant has demonstrated that any such amendment, prima facie, overcomes the issues raised by the board.

5.4 Since claim 1 of auxiliary request 1 contains the same feature as objected to in point 4. above for the main request, it does not, prima facie, overcome the objection under Article 83 EPC raised by the board against claim 1 of the main request.

In this respect, the appellant has not put forward any arguments beyond those already provided for the main request (see point XI.(f) above).

5.5 Therefore, the board exercised its discretion under Article 13(2) RPBA 2020, taking into account the criteria set out in Article 13(1) RPBA 2020, and decided not to admit the first auxiliary request into the appeal proceedings.

6. Second auxiliary request - admittance (Article 13(2) RPBA 2020)

6.1 The appellant argued that the second auxiliary request formed a basis for the appeal proceedings under Article 12(1)(d) RPBA 2020 because it had been submitted in reply to the board's communication.

The board is not convinced by this argument, for the same reasons as those given in point 5.1 above in relation to the first auxiliary request.

6.2 The second auxiliary request was filed after notification of the summons to oral proceedings. This auxiliary request is therefore an amendment within the meaning of Article 13(2) RPBA 2020.

6.3 As set out in point 5.3 above, the board may also rely on criteria set out in Article 13(1) RPBA 2020 when exercising its discretion under Article 13(2) RPBA 2020.

Under Article 13(1) RPBA 2020 the board is to exercise its discretion in view of, inter alia, whether the appellant has demonstrated that any such amendment, prima facie, overcomes the issues raised by the board.

6.4 Claim 1 of the second auxiliary request was amended to specify "the stereo disparity estimates are reweighted automatically and adaptively based on the normal vectors of the estimated 3D height model taking obscuration into consideration, such that for example an image taken from above is used to estimate the roof structure and not the sides of a building".

6.5 As set out in point 4.3 above, the board is not convinced that taking obscuration into consideration explains how disparity estimates can be reweighted automatically and adaptively on the basis of the normal vectors.

In this respect, the appellant has not put forward any other arguments than those already provided for the main request (see point XI.(f) above).

6.6 Hence, the board finds that the appellant has not demonstrated that the amendment to claim 1 of the second auxiliary request overcomes the objection under Article 83 EPC raised by the board.

6.7 Therefore, the board exercised its discretion under Article 13(2) RPBA 2020, taking into account the criteria set out in Article 13(1) RPBA 2020, and decided not to admit the second auxiliary request into the appeal proceedings.

7. Conclusion

The board decided not to remit the case to the department of first instance under Article 111(1) EPC and Article 11 RPBA 2020. Furthermore, the appellant's request for reimbursement of the appeal fee under Rule 103(1)(a) EPC is not granted. The main request is not allowable because claim 1 thereof does not meet the requirements of Article 83 EPC. The first and second auxiliary requests are not admitted into the appeal proceedings under Article 13 RPBA 2020. Since none of the appellant's requests is allowable, the appeal must be dismissed.

Order

For these reasons it is decided that:

1. The appeal is dismissed.

2. The request for reimbursement of the appeal fee is refused.