T 0690/13 () of 23.6.2016

Summary of Facts and Submissions

I. The appeal is against the decision of the examining division refusing European patent application No. 02755135.7 published as international patent application WO 03/017203 A1.

II. In the decision under appeal inter alia the following documents were cited:

D1: US 6,152,563 A,

D2: A.M. Brinicombe et al., "Direction of regard determination", Image Processing, 1996, Vol. 2, pages 169-172, XP010202621,

D3: D.H. Ballard et al., "Computer vision", Prentice-Hall, Inc., 1982, pages 123-131 and 151-152, XP002485299,

D4: V.F. Leavers, "Shape Detection in Computer Vision Using the Hough Transform", Springer-Verlag, 1992, pages 47, 138, 152 and 153,

D5: M.S. Nixon et al., "Feature Extraction and Image Processing", Newnes, 2002, pages 173-216, XP002485298

D6: L. Xu et al., "A new curve detection method: Randomized Hough Transform (RHT)", Pattern Recognition Letters 11 (1990), No. 5, May 1990, pages 331-338, XP000126171, and

D8: R. Klette et al., "Performance Characterization in Computer Vision", Kluwer Academic Publishers, 2000, pages 227-229.

III. The application was refused on the following grounds:

- claims 1 to 9 and 12 to 17 of the sole request did not meet the requirements of Article 84 EPC because essential features were missing;

- the application did not meet the requirements of Article 83 EPC; and

- the subject-matter of claim 1 did not involve an inventive step when starting from either D1 or D2 and taking into account common general knowledge (evidence of which could be found in documents D3, D4, D5, D6 and D8).

IV. With the statement of grounds of appeal the appellant filed amended claims according to a main request and an auxiliary request, replacing all claims previously on file. As a precaution, the appellant also requested oral proceedings.

V. With a letter dated 18 January 2016 the appellant requested accelerated processing of the appeal essentially because it was about to start negotiations with a customer with a view to licensing the technology described in the present patent application.

VI. In a communication under Article 15(1) RPBA (Rules of Procedure of the Boards of Appeal, OJ EPO 2007, 536) annexed to the summons to oral proceedings, the board informed the appellant of its decision to grant accelerated processing in accordance with the Notice from the Vice-President Directorate-General 3 dated 17 March 2008 concerning accelerated processing before the boards of appeal (OJ EPO 2008, 220). The board also gave its preliminary opinion that

- claim 1 of the main request did not meet the requirements of Article 84 EPC 1973 because essential features were missing,

- claim 1 of the main and auxiliary requests lacked clarity (Article 84 EPC 1973) because the expression "super-threshold pixels" had no well-defined meaning, and

- the subject-matter of claim 1 of both requests did not involve an inventive step in view of prior-art documents D2 and D6 and also in view of prior-art documents D1 and D6.

VII. With a letter dated 19 May 2016, the appellant withdrew its main request and filed amended claims according to first to fourth auxiliary requests.

VIII. With a letter dated 16 June 2016, the appellant filed amended claims according to a fifth auxiliary request.

IX. The board held oral proceedings on 23 June 2016. During the oral proceedings the appellant filed the following prior-art document D10 as evidence in support of its argument that the skilled person would not have wanted to apply the teaching of document D6 to the system of document D2:

D10: N. Kiryati et al., "Randomized or probabilistic Hough transform: unified performance evaluation", Pattern Recognition Letters 21 (2000), 2000, pages 1157-1164.

Document D10 was admitted into the appeal proceedings.

During the oral proceedings, the appellant also filed amended claims according to several new requests and withdrew or reordered other requests (see minutes of the oral proceedings).

At the end of the oral proceedings, the appellant's final requests were that the decision under appeal be set aside and that a patent be granted on the basis of the claims according to the main request filed as first auxiliary request with the letter dated 19 May 2016, or according to one of the first auxiliary request filed as second auxiliary request with the letter dated 19 May 2016, the second auxiliary request filed in the oral proceedings of 23 June 2016 or the third auxiliary request "15.40" filed in the oral proceedings of 23 June 2016.

Before closing the oral proceedings, the chairman announced the board's decision.

X. Claim 1 according to the appellant's main request reads as follows:

"An eye tracking system for monitoring the movement of a user's eye, the system comprising:

(a) video data input means for receiving video data produced by eye imaging means monitoring the user's eye;

(b) spot location means for determining from the video data the location of a reference spot formed on the user's eye by illumination of the user's eye by a point source of light, the spot location means including adaptive threshold means for providing an indication of parts of the image produced by the eye imaging means which have a brightness greater than a threshold value, and spot identification means for selecting a valid reference spot by comparing said parts of the image with predetermined validity criteria;

(c) pupil location means for determining from the video data the location of the centre of the pupil of the user's eye relative to the reference spot in order to determine the user's line of gaze, the pupil location means including selection means for selecting a pupil tracking window comprising a portion of the image produced by the eye imaging means containing the pupil relative to the valid reference spot location, edge determination means for determining an edge of the pupil by selection of those parts of said image portion having a gradient greater than a threshold value, centre determination means for location of the centre of the pupil by making reference to selected points about the edge of the pupil, wherein the centre determination means includes triad selection means for selecting substantially randomly three pixels deemed to have a gradient greater than the threshold value to form a triad for further processing, from among a plurality of pixels of pupil image data, and triad processing means for determining the centre and radius of a hypothetical circle passing through each of the selected pixels and further wherein the centre determination means includes accumulating means for accumulating a plurality of sets of values of the centre and radius determined by the triad processing means for a plurality of triads successively selected by the triad selection means, the centre determination means further including peak determination means for determining, from the sets of values accumulated by the accumulating means, the values of the centre and radius of the pupil; and

(d) display means for indicating the user's point of regard from the user's line of gaze determined by the pupil and spot location means."

XI. Claim 1 according to the appellant's first auxiliary request reads as follows (additions to claim 1 of the main request are underlined, long identical text portions are replaced by "[...]", no text was deleted):

"An eye tracking system for monitoring the movement of a user's eye, the system comprising:

(a) [...];

(b) spot location means for determining from the video data the location of a reference spot formed on the user's eye by illumination of the user's eye by a point source of light, and further spots from additional point light sources, the spot location means including adaptive threshold means for providing an indication of parts of the image produced by the eye imaging means which have a brightness greater than a threshold value, and spot identification means for selecting a valid reference spot by comparing said parts of the image with predetermined validity criteria, wherein the spot location means includes multiple spot classification means for identifying a reference spot by searching for the existence of further valid spots in search areas at defined locations relative to each reference spot indicated by the spot identification means;

(c) [...]; and

(d) [...]."

XII. Claim 1 according to the appellant's second auxiliary request reads as follows (additions to claim 1 of the main request are underlined, deletions are [deleted: struck-through], long identical text portions are replaced by "[...]"):

"An eye tracking system for monitoring the movement of a user's eye, the system comprising:

(a) [...];

(b) spot location means for determining from the video data the location of a reference spot formed on the user's eye by illumination of the user's eye by a point source of light along with additional point light sources, the spot location means including adaptive threshold means for providing an indication of parts of the image produced by the eye imaging means which have a brightness greater than a threshold value, and spot identification means for selecting a valid reference spot by comparing said parts of the image with predetermined validity criteria;

(c) [...]; [deleted: and]

and yet further wherein the pupil location means includes masking means for providing an output from which the influence of the reference spot and additional spots has been removed prior to selection of those parts of the gradient of said image portion in the pupil tracking window which have a gradient greater than the threshold value; and

(d) [...]."

XIII. Claim 1 according to the appellant's third auxiliary request "15.40" reads as follows (additions to claim 1 of the main request are underlined, long identical text portions are replaced by "[...]", no text was deleted):

"An eye tracking system for monitoring the movement of a user's eye, the system comprising:

(a) [...];

(b) [...];

(c) pupil location means for determining from the video data the location of the centre of the pupil of the user's eye relative to the reference spot in order to determine the user's line of gaze, the pupil location means including selection means for selecting a pupil tracking window comprising a portion of the image produced by the eye imaging means containing the pupil relative to the valid reference spot location, edge determination means for determining an edge of the pupil by selection of those parts of said image portion having a gradient greater than a threshold value, centre determination means for location of the centre of the pupil by making reference to selected points about the edge of the pupil, wherein the centre determination means includes triad selection means for selecting substantially randomly three pixels deemed to have a gradient greater than the threshold value to form a triad for further processing, from among a plurality of pixels of pupil image data, and triad processing means for determining the centre and radius of a hypothetical circle passing through each of the selected pixels and further wherein the centre determination means includes accumulating means for accumulating, in separate accumulators, a plurality of sets of values of the centre and radius determined by the triad processing means for a plurality of triads successively selected by the triad selection means, wherein the x and y coordinates of the centre and radius are each added to a separate accumulator, the centre determination means further including peak determination means for determining, from the sets of values accumulated by the accumulating means, the values of the centre and radius of the pupil; and

(d) [...]."

XIV. The appellant's arguments relevant to the decision are discussed in detail below.

Reasons for the Decision

1. The appeal is admissible.

The invention

2. The invention relates to an eye tracking system for monitoring the movement of a user's eye in order to determine the user's point of regard. It can be used for controlling a device by eye movements.

Main request - Article 56 EPC 1973 (inventive step)

3. Closest prior art

The appellant did not dispute the examining division's finding that document D2 could be regarded as the closest prior art for the subject-matter of claim 1 (see point 5.1 of the Reasons for the decision). The board concurs with the examining division on this finding.

4. Disclosure of D2

Document D2 discloses an eye tracking system for pilots of high-performance aircraft. The system determines the pilot's direction of regard, which can be used as a pointing mechanism, from the relative positions of the pilot's pupil and a reference spot formed on the pilot's eye by illumination with an infrared (IR) light source (see Subsection "Software" on page 169). In order to identify the pupil and the reference spot in a sequence of images of the eye captured by a camera, inter alia the following image-processing steps are performed: contrast enhancement, edge detection via Canny operators, gradient calculations, circular-feature detection based on a Hough transform and peak detection (see section "Description of System" on pages 169 to 171).

5. Distinguishing features of claim 1

The board concurs with the examining division (see points 5.2 and 5.2.1 of the Reasons for the decision) that the system of claim 1 underlying the decision under appeal differed from that of D2 essentially by the following features:

(B1) adaptive thresholding;

(B2) a tracking window;

(B3) indication of the point of regard;

(B4) random selection of three edge pixels and determination of a circle by processing these three pixels.

The additional features introduced into claim 1 of the present main request (from claims 10 and 11 underlying the decision under appeal) comprise the following further distinguishing features:

(B5) repetition of (B4) for a plurality of triads of pixels and accumulating means for storing the thus obtained sets of values of the centre and radius; and

(B6) peak determination means for determining, from the sets of values in the accumulating means, the values of the centre and radius of the pupil.

The appellant did not dispute that the system of claim 1 differed from that of D2 by the above distinguishing features.

6. Objective technical problem

The appellant submitted that the distinguishing features solved the objective technical problem of providing a more versatile approach in terms of accuracy and processing requirements (see points 1.7 and 2.4 of the statement of grounds of appeal).

Although the board has some doubts that the above problem really is solved by the distinguishing features of claim 1, it will assume arguendo that the objective technical problem can be formulated as proposed by the appellant.

7. Obviousness

Re: distinguishing features (B1) to (B3)

The board concurs with the examining division that features (B1) to (B3) are commonly-used features in the field of eye tracking systems and can be regarded as belonging to the skilled person's common general knowledge. More specifically, thresholding is a well-known technique for feature extraction and noise reduction in an image: see, for instance, column 6, lines 32 to 47, of D1 and figures 2a and 2b of D6. Window tracking is a conventional technique for reducing the processing requirements by limiting the image processing to a region of interest smaller than the whole image. Once the direction of regard has been determined, it is commonly known to display a marker on a display at the position where the direction of regard meets the display: see, for instance, figure 9 of D1.

The above conclusions regarding features (B1) to (B3) were not disputed by the appellant who, instead, focused its arguments in the statement of grounds of appeal and during the oral proceedings on distinguishing features (B4) to (B6) (see also point 5.2.2, last sentence, of the Reasons for the decision under appeal stating that the obviousness of features (B1) to (B3) was not disputed).

Re: distinguishing features (B4) to (B6)

The board is of the view that the skilled person, when starting from D2, would have arrived at distinguishing features (B4) to (B6) without an inventive step for the following reasons:

D6 discloses an improvement of the Hough Transform (HT) which makes it run faster, among several other advantages. The improved transform is called "Randomized Hough Transform (RHT)". Like HT, RHT operates on an image which has undergone some preprocessing, including edge detection using a gradient and thresholding (see D6, "Canny operator" in the middle of the left column on page 332, figures 2a and 2b and the sentence bridging pages 336 and 337). Like HT, RHT detects straight lines, circles and other curves in the image. As explained on page 334 of D6, it detects circles by randomly selecting three points from the edge pixels and calculates the centre and radius of the hypothetical circle passing through these three points. The value of an "accumulating cell" having an address corresponding to the centre and radius thus obtained is then incremented by 1. The process is repeated for successive sets of randomly picked edge pixels until the value in one cell reaches a given threshold. The cell thus obtained, which has a peak value, indicates the centre and radius of an identified circle in the image.

Hence D6 discloses distinguishing features (B4) to (B6). Since D6 teaches that RHT has a number of advantages, including being faster over HT for detecting circles (see sections 3.2 and 4 on pages 337 and 338), the skilled person would have wanted to apply the teaching of D6 to the system of D2, i.e. to replace HT by RHT.

The appellant presented the following arguments with regard to distinguishing features (B4) to (B6):

First, the skilled person would not have considered document D6 because it did not relate to eye tracking.

Even if the skilled person had considered document D6,(s)he would not have wanted to replace the HT of document D2 by the RHT described in document D6 because document D10 taught away from using RHT for noisy images; instead, D10 recommended using PHT ("probabilistic Hough transform"), another improvement on HT (see D10, Abstract and "Conclusions" section). In the system of D2, the images of the eye were noisy because they were produced from video data.

The appellant's arguments did not convince the board for the following reasons:

In the eye tracking system of D2, HT plays a central role in detecting the pupil and the reference spot. It would thus have been normal practice for the skilled person to look for improvements to HT. Moreover, D2 itself acknowledges the difficulty of achieving real-time processing for HT (see "Hardware" section on page 171). The board thus considers that the skilled person would have been looking for improvements to HT, in particular in terms of speed, in order to address the problem identified in D2. The skilled person would thus have found documents D6 and D10 which both deal with how to improve HT in the context of pattern recognition in images.

The appellant did not dispute that the RHT disclosed in D6 provided better results than HT, in particular in terms of speed and memory requirements. However, it argued that D10 taught that PHT, another improvement on HT, should be used instead of RHT in the system of D2 because of noisy images.

The board concurs with the appellant that D10, while acknowledging that both RHT and PHT are superior to HT, teaches that better results are achieved with PHT for noisy images and with RHT for low-noise images. D10, however, provides little information as to how noisy an image should be for PHT to be preferable to RHT. The only disclosure regarding this noise threshold is in the three images (hereinafter referred to as images A, B, C) shown in figures 1 and 2 which contain increasing levels of noise from A to C. For the least noisy image A, RHT is said to be superior to PHT, for image B, PHT and RHT are said to be comparable, while for the very noisy image C (see right half of figure 2), PHT is said to be superior to RHT (see page 1162, right column).

The board, however, is not convinced that the eye images used by the system of D2 would be as noisy as in image C, or even as in image B, of D10. Indeed, in D2 the camera is helmet-mounted (see Abstract). It is thus positioned near the eye, which should significantly limit the amount of noise in the image data, even if the image data is generated as a sequence of images, i.e. as video data. Also in a computer environment, with the camera positioned near the monitor, the noise should be relatively low. Moreover, the captured images are typically subjected to noise-reducing preprocessing before RHT is performed (see figures 2a and 2b in D6).

For the above reasons, the appellant did not convince the board that D10 would teach away from replacing the HT in the system of D2 by the superior RHT described in D6.

8. Conclusion on inventive step

In the light of the above, the subject-matter of claim 1 of the main request does not involve an inventive step in view of documents D2 and D6.

Conclusion on the main request

9. Since the subject-matter of claim 1 does not involve an inventive step, the appellant's main request is not allowable.

First auxiliary request - Article 56 EPC 1973 (inventive step)

10. Claim 1 of the first auxiliary request differs from claim 1 of the main request by the features underlined in section XI supra. In essence, the additional features are additional point light sources creating further spots and the spot location means taking into account the known relative positions of the light sources to improve the identification of the reference spot.

11. The appellant argued that neither D2 nor D6 disclosed the use of multiple light sources for improving the identification of the reference spot.

12. The board does not dispute this fact. However, the board notes that the first page of the description of the present application as filed acknowledges that it was known in the art to use multiple IR light sources to illuminate the eye for eye tracking (see "a near infrared source (or multiple sources)" on page 1). In the board's view, although the application is silent on the reasons for using several light sources instead of just one, the pros and cons would have been obvious to the skilled person: an advantage would be that several point light sources arranged in a known pattern, e.g. three point light sources arranged in a triangular pattern, would make it possible to distinguish reflection spots from the light sources from reflection spots from the sun, whereas a drawback would be an increase in the complexity of the system. Since D2 mentions sunlight reflections as a problem (see "noise, caused by sun-dazzle" on page 169, left column), it would have been obvious for the skilled person to further improve the system of D2 with additional light sources in order to distinguish the valid reference spot from reflection spots from the sun.

For the above reasons, the subject-matter of claim 1 of the first auxiliary request does not involve an inventive step in view of D2 and D6.

Conclusion on the first auxiliary request

13. Since the subject-matter of claim 1 does not involve an inventive step, the appellant's first auxiliary request is not allowable.

Second auxiliary request - Article 84 EPC 1973

14. Claim 1 of the second auxiliary request differs from claim 1 of the main request by the features underlined in section XII supra, in particular by the feature that "the pupil location means includes masking means for providing an output from which the influence of the reference spot and additional spots has been removed prior to selection of those parts of the gradient of said image portion in the pupil tracking window which have a gradient greater than the threshold value" (emphasis added by the board).

15. The appellant submitted the above feature was supported by figure 3 and the associated description of the application as filed.

16. The board, however, notes that according to figure 3 and the last four lines on page 7 of the description, the masking step (58) takes place after the gradient threshold comparison (52). It is thus the opposite of the above additional feature of claim 1, which states that the spots are masked prior to the selection of the image portions which have a gradient greater than a threshold value.

17. The appellant did not dispute that there was a contradiction between the wording of claim 1 and the disclosure in figure 3 and the associated description.

18. For the above reasons, claim 1 according to the second auxiliary request does not meet the requirement of Article 84 EPC 1973 that the claims be supported by the description.

Conclusion on the second auxiliary request

19. Since claim 1 does not meet the requirements of Article 84 EPC 1973, the appellant's second auxiliary request is not allowable.

Third auxiliary request "15.40" - Article 84 EPC 1973

20. Claim 1 of the third auxiliary request "15.40" differs from claim 1 of the main request by the additional text portions underlined in section XIII supra.

21. In the board's view, these additional text portions render claim 1 unclear for the following reasons:

The additional text portions (underlined below) state that the centre determination means includes accumulating means for accumulating, in separate accumulators, a plurality of sets of values of the centre and radius determined by the triad processing means for a plurality of triads successively selected by the triad selection means, wherein the x and y coordinates of the centre and radius are each added to a separate accumulator, the centre determination means further including peak determination means for determining, from the sets of values accumulated by the accumulating means, the values of the centre and radius of the pupil.

In the board's view, it is not clear from the above wording whether there are two accumulators (one for the centre and one for the radius) or three accumulators (two for the x and y coordinates of the centre and one for the radius).

It is also unclear how the peak determination means interact with the separate accumulators to determine the values of the centre and radius of the pupil.

22. The appellant argued that it was clear from the wording of claim 1 that there were three accumulators and that, as disclosed in the first paragraph of page 8 of the description of the application as filed, the peaks of the three accumulators were the x and y coordinates of the centre and the radius, respectively.

The appellant's arguments did not convince the board for the following reasons:

First, the wording of claim 1 is grammatically ambiguous in that it can be read as implying either two accumulators or three. The short disclosure in the description of the application as filed mentioning accumulators, in the first paragraph of page 8, fails to remove this ambiguity, because it uses essentially the same grammatical constructions.

Moreover, claim 1 contains no indication as to how the peak determination means use the separate accumulators for determining the centre and radius of the pupil, which results in a lack of clarity. And the description of the application (see first paragraph on page 8) states only that the peak of each accumulator is "used" for determining the centre and radius, without specifying how it is used. Hence, even in the light of the description, claim 1 still lacks clarity.

23. In the light of the above, the board considers that claim 1 of the third auxiliary request "15.40" does not meet the requirement of clarity under Article 84 EPC 1973.

Conclusion on the third auxiliary request "15.40"

24. Since the subject-matter of claim 1 does not meet the requirements of Article 84 EPC 1973, the appellant's third auxiliary request "15.40" is not allowable.

Conclusion

25. Since none of the appellant's requests is allowable, the appeal must be dismissed.

Order

For these reasons it is decided that:

The appeal is dismissed.