T 0749/19 (Time of flight camera / Rockwell Automation) of 20.3.2023

Summary of Facts and Submissions

I. Appeals were filed by the proprietor and the opponent against the interlocutory decision of the opposition division, that the auxiliary request 2 before it was allowable.

II. The opponent requested that the contested decision be set aside and the patent be revoked, or, as an auxiliary request, that oral proceedings be held.

III. The proprietor requested that the decision be set aside and the opposition rejected, i.e. that the patent be maintained as granted, or on the basis of one of the 12 auxiliary requests filed with the reply to the opposition. Oral proceedings were also conditionally requested.

IV. In a communication from the Board, the parties were notified that the patent had been surrendered or had lapsed with effect for all designated Contracting States and that the appeal proceedings would be discontinued, unless a request for continuation was filed by one of the parties within two months from notification of the communication.

V. The proprietor withdrew its appeal and indicated that it would not actively participate in the appeal proceedings, should the Opponent not withdraw its appeal. The opponent requested continuation of the appeal proceedings.

VI. In a subsequent communication under Rule 100(2) EPC, the Board set out its preliminary opinion on the substantive and legal matters of significance for the case and invited the parties to file their observations.

VII. The relevant sections of that communication referred to documents

E1: DE 4411448 B4,

E5: DE 3825757 A1,

E9: EP 2073035 A1,

E12: US 6323942 B1,

and read:

3. The opponent requests that the contested decision be set aside and the patent be revoked.

4. As the proprietor has withdrawn its appeal, its requests are those submitted with its response to the opponent's appeal, namely, that the opponent's appeal be dismissed (so that the patent can be upheld in the form according to auxiliary request 2, as in the appealed decision) or that the patent be upheld in the form of one of auxiliary requests 3 - 12, as before the Opposition Division.

5. Both parties also conditionally requested oral proceedings.

6. However, in view of the proprietor's subsequent indication that it will take no active part in the appeal proceedings, the Board is of the view that the proprietor no longer requests oral proceedings.

Opposition and appeal

8. On appeal the opponent argues that auxiliary request 2 lacks inventive step in view of a combination of E9 or E12 with E5 or E1, or of a combination of E1 with E9 or E12. The reference, found in the second paragraph of the opponent's statement of grounds, to the submissions made before the first instance, does not substantiate any other line of attack.

The Invention

9. The invention relates to optical surveillance systems and, in particular, to time of flight camera units for such systems, adapted to obtain information on the 3D contour of an imaged object (called depth images) based on the time-of-flight principles.

10. It is said to be based on the idea that, by incorporating at least one mirror in such a camera unit, indirect illumination of at least a part of the object shadowed under direct illumination can be achieved and, hence, a three-dimensional image of both directly visible and shadowed parts of the object can be obtained (Patent: paragraph [0014]; Figures 1 and 2).

Inventive step having regard to E1, E5, E9, and E12

11. E1, E9, and E12 relate to optical systems for surveillance of a space employing time of flight principles. E5 seems more distant, since the optical surveillance system employed does not employ time of flight techniques.

12. E9 and E12 disclose optical surveillance systems employing camera units adapted to obtain depth images of an object, but incorporating no mirror (E9: Figure 1, paragraphs [0044] to [0047]; E12: Figure 2, column 3 lines 12-22 and 30-34, column 4 lines 46-66, column 7 line 16-21).

13. E1 discloses the use of mirrors to illuminate parts of an object that are not directly illuminated (E1: Figure 2, paragraphs [0039] and [0044 to [0046]), but does not employ a 3D camera unit. Instead, in E1, a laser source is employed for illuminating one point of the object at a time, and the direction of the laser beam swept over an observation plane such that with a sweep out a line of points on the object and the respective distances are obtained, based on which the 2D contour of the object in the observation plane can be obtained (E1: figure 3, paragraphs [0048] to [0053]).

14. The main questions to be answered seem to be whether or not, and for which purpose, the skilled person, starting from either E9 or E12, would consider the teaching of E1, or vice versa; and what would be the results achieved from the combined teachings.

15. Starting from the disclosure of either E9 or E12, and attempting to reduce shadowing effects, the skilled person would find, in document E1, the solution of providing at least one mirror so as to illuminate the parts of the object that are not subject to direct illumination (E1: Figure 2, paragraphs [0039] and [0044] to [0047]). He would then also implement such a solution in the surveillance system of either E9 or E12.

16. The proprietor's argument that the skilled person would not consider E1, let alone a combination with either E9 or E12, because E1 does not employ a 3D camera unit such as those of E9 or E12, is not convincing because the skilled person would recognise that the solution to the shadowing problem, taught in E1, applies not only to the particular imaging device described in E1, but more generally.

17. It appears, furthermore, that, upon implementing at least one mirror in the 3D camera unit of either E9 or E12, as taught in E1, the skilled person would also adapt the processing unit, such that the reflections from the indirect illumination of the object via the mirror, would be processed to obtain a further depth image of the object under a different angle of observation; and would furthermore combine the two depth images for generating a three-dimensional image of the surveillance area, as defined in the last feature of the claim 1 of auxiliary request 2, so as to solve the objective technical problem.

18. The same result seems to be reached starting from the disclosure of document E1. The skilled person, seeking to provide a simpler and more comprehensive optical surveillance system, would replace the optical unit of E1 by a 3D camera unit such as those employed in either E9 or E12 and so would adapt the processing unit of such units to obtain different depth images from different points of view. A combination of the different depth images obtained for generating a three-dimensional image of the total surveillance area, appears likewise to be an implementation option that would be consider [sic].

19. Therefore, the Board sees the subject-matter of claim 1 of auxiliary request 2 as lacking an inventive step in view of the combined teaching of either E9 or E12 and E1.

Auxiliary requests

22. Auxiliary requests 4, 8, 9, 11, and 12 are not based on auxiliary request 2 and, hence, are inadmissible.

23. Auxiliary requests 3, 5, 7, and 10, are based on auxiliary request 2. They appear to also be not allowable for the following reasons:

(a) The camera unit of E9 also employs modulated radiation, as further defined in claim 1 of the auxiliary request 3, and its detecting unit also monitors, for each pixel, the time-of-flight of the light returned, as further defined in claim 1 of auxiliary request 5 (E9: abstract, paragraphs [0044], [0046] and [0047]).

(b) The feature added to claim 1 in auxiliary request 7 is also known from E1 (E1: paragraphs [0039] and [0044] to [0046]) and the feature added to auxiliary request 10 is also at least hinted at in E1 (Figure 2).

Hence the subject-matter defined claim 1 of any of the auxiliary requests 3, 5, 7 and 10 also lacks an inventive step, in view of the combined teaching of E9 and E1.

Final remarks on the procedure

24. In view of the appellant's conditional request for Oral Proceedings and the Board's view that the proprietor no longer requests oral proceedings, the Board would be in the position to issue a decision along the lines set out above.

25. The parties are hence requested to provide the Board with their comments on this communication, if any, within two (2) months.

VIII. After notification of this communication, the opponent expressed agreement with the issuance of the decision in written proceedings, and did not submit any additional substantive observations.

IX. No further submission was received from the proprietor.

X. Claim 1 of the patent reads:

Time of flight, TOF, camera unit for an optical surveillance system, said TOF camera unit (100) comprising:

a radiation emitting unit (102) for illuminating a first plane (112) of a surveillance area;

a radiation detecting unit (102) for receiving radiation reflected from said surveillance area and for generating a three-dimensional image from said detected radiation, wherein said radiation emitting unit and said radiation detecting unit are arranged within a common camera housing (102),

at least one mirror (120, 122) for partly deflecting said emitted radiation into at least one second plane (114) which is not directly illuminated by said radiation emitting unit (102), and for deflecting the radiation reflected from said second plane (114) to the radiation detecting unit (102) in addition to the radiation directly received from the first plane(112).

XI. Claim 1 of the auxiliary request 1 limits the radiation emitted and received to modulated radiation and further defines the three-dimensional image as generated from a phase shift of said detected modulated radiation.

XII. Claim 1 of the auxiliary request 2 adds to claim 1 of the patent the further limitations

...[camera housing (102)]

wherein the radiation detection unit comprises a phase sensitive detector chip and a signal processing unit for evaluating outputs of said detector chips;

[at least one mirror ... from the first plane (112);]

wherein said signal processing unit is operable to define and evaluate a first depth level representing the area directly illuminated by said radiation emitting unit, and at least one second depth level representing the area illuminated by said radiation emitting unit via said at least one mirror; and

wherein the processing unit is further operable to combine the images corresponding to the first and second depth levels for generating a three-dimensional image of the total surveillance area.

XIII. Claim 1 of auxiliary request 3 combines the limitations of claim 1 of auxiliary requests 1 and 2.

XIV. Claim 1 of the auxiliary request 4 adds, to claim 1 of the auxiliary requests 1, the limitation:

... [common camera housing (102),] wherein said modulated light is monitored by individually measuring the turnaround time of the modulated light individually for each pixel of a pixel array;

[at least one mirror] ...

XV. Claim 1 of auxiliary request 5 combines the limitations of claim 1 of auxiliary requests 2 and 4.

XVI. Claim 1 of the auxiliary request 6 adds to claim 1 of auxiliary requests 1, the limitation:

... [from the first plane (112)];

wherein the calculation of the time of flight takes into account an additional optical path length defined by the geometric position of the at least one mirror with respect to the radiation emitting unit and the radiation detecting unit and a background (104) of the surveillance area, wherein said generating of the three-dimensional image comprises correcting the optical path length of the modulated radiation reflected from said second plane.

XVII. Claim 1 of auxiliary request 7 combines the limitations of claim 1 of auxiliary requests 2 and 6.

XVIII. Claim 1 of the auxiliary request 8 combines the limitations of claim 1 of auxiliary requests 4 and 6.

XIX. Claim 1 of the auxiliary request 9 adds to claim 1 of auxiliary request 1, the limitation:

... [from the first plane (112);]

wherein a viewing angle of said receiving unit is larger than needed for monitoring said surveillance area, and wherein said at least one mirror is located in a peripheral region of said viewing angle.

XX. Claim 1 of auxiliary request 10 combines the limitations of claim 1 of auxiliary requests 2 and 9.

XXI. Claim 1 of the auxiliary request 11 combines the limitations of claim 1 of auxiliary requests 4 and 9.

XXII. Claim 1 of the auxiliary request 12 combines the limitations of claim 1 of auxiliary requests 6 and 9.

Reasons for the Decision

1. The preliminary opinion reproduced above expressed and explained the Board's view that the proprietor's requests were either inadmissible (paragraphs 4 and 22 of the opinion) or not allowable for lack of inventive step, having regard the state of the art (paragraphs 19 and 23 of the opinion).

2. The proprietor has not commented on, let alone contested, this preliminary opinion. The opponent has also made no substantive comment.

3. The Board does not see any reason to depart from its preliminary opinion.

4. It is noted that, even if the proprietor's main request and auxiliary request 1 were not regarded as withdrawn with the withdrawal of the proprietor's appeal, these requests too would be inadmissible, for the same reason as auxiliary requests 4, 6, 8, 9, 11, and 12, i.e. because the camera unit defined by claim 1 of these requests does not fall within the scope of claim 1 of auxiliary request 2, the request found allowable by the Opposition Division.

5. As the opponent became the sole appellant, maintaining the patent as granted, or as amended according to any of the auxiliary requests 1, 4, 6, 8, 9, 11, and 12, would have contravened the principle of prohibition of reformatio in peius (G9/92, OJ 1994, 875, point 2 of the order; Case Law of the Boards of Appeal, 10th ed., V.A.3.1 and V.A.3.1.5).

6. As all the proprietor's requests are either inadmissible or unallowable, the patent must be revoked.

7. The proprietor's indication that it would take no active part in the appeal proceedings, implies that the proprietor withdrew its request for oral proceedings (Case Law of the Boards of Appeal, 10th ed., III. C.4.3.1). This was pointed out in paragraph 24 of the Board's communication.

8. Additionally, in view of the conclusion that the patent must be revoked, the condition for the opponent's auxiliary request for oral proceedings does not arise.

9. Consequently, this decision is being issued on the basis of the parties' written submissions.

10. Both conditions of Rule 103(4)(c) EPC, that "any request for oral proceedings is withdrawn ..." and "no oral proceedings take place", are met (Case Law of the Boards of Appeal, 10th ed., III. C.4.3.4).

11. It makes not difference that the withdrawing party and the appealing party are not the same (T488/18, reason 8).

12. It also makes no difference that the withdrawal of the proprietor's request for oral proceedings was made prior to a "notification of the communication issued by the Board in preparation for the oral proceedings", or even of a summons. The Board shares the opinion expressed in T2361/18 at points 3.2 and 3.3, that the "within one month" of Rule 103(4)(c) EPC, rather than defining a time period starting with the notification of the communication, defines an "end point" for the withdrawal of the request for oral proceedings, if partial reimbursement of the appeal fee is to be obtained.

13. Consequently, the opponent's appeal fee is to be reimbursed at 25%.

Order

For these reasons it is decided that:

1. The decision is set aside.

2. The patent is revoked.

3. The opponent's appeal fee is reimbursed at 25%.