T 1662/09 (Eye control of computer apparatus/TOBII) of 19.2.2013

Summary of Facts and Submissions

I. The present appeal is against the decision of the examining division to refuse the European patent application no. 04 445 071.6, publication no. EP 1 607 840. The decision was announced during oral proceedings on 24 March 2009 and the written reasons were dispatched on 2 July 2009.

II. The decision under appeal was based on a main request comprising claims 1 to 20 filed during oral proceedings on 24 March 2009. In the decision it was found that claim 1 of the main request did not involve an inventive step in the light of the following document:

D1: US 5 649 061.

In the context of the discussion of inventive step, the decision referred to the skilled person's common knowledge which was said to include "well-known programming concepts such as the 'publish-subscribe' concept that is commonly used in event-based systems" (cf. decision, II. Reasons 12.1) and, in this regard, mentioned "the seminal work by Gamma et al. 'Design Patterns: Elements of Reusable Object-Oriented Software', Addison-Wesley, Professional Computing Series, 1994".

III. Notice of appeal was received at the EPO on 15 June 2009 with the appeal fee being paid on the same date. The notice of appeal included a written statement setting out the grounds of appeal. The appellant requested that the decision under appeal be set aside and that a patent be granted on the basis of a main request comprising claims 1-20 as submitted with the written statement setting out the grounds of appeal.

The claims of this request were substantially identical to those on which the decision under appeal was based.

IV. In a communication accompanying a summons to oral proceedings to be held on 19 February 2013, the board gave its preliminary opinion that the appellant's request was not allowable.

V. In said communication, the board introduced the following documents into the appeal proceedings of its own motion pursuant to Article 114(1) EPC 1973:

D4: US 5 850 211 B.

D5: Gamma E., Helm R., Johnson R., Vlissides J., "DESIGN PATTERNS: ELEMENTS OF REUSABLE OBJECT-ORIENTED SOFTWARE", pp.293-303, Addison-Wesley professional computing series, 1994, ISBN 978-0-201-63361-0.

D6: R. Wiener and L.J. Pinson, "Fundamentals of OOP and Data Structures in Java", Chapters 5 and 6, pp.77- 118, Cambridge University Press, 2000, ISBN: 0-521-66220-6.

D7: W. Barfield and T.A. Furness III (eds.), "Virtual Environments and Advanced Interface Design", "Chapter 7: Eye Tracking in Advanced Interface Design" by R.J.K. Jacob, pp.258-288, Oxford University Press, 1995, ISBN: 0-19-507555-2.

D8: R.J.K. Jacob, "A specification language for direct manipulation user interfaces", ACM Transactions on Graphics, Vol.5, No.4, October 1986, pp.283-317, ISSN: 0730-0301.

D4 is referred to in [0056] of the published application where it is erroneously cited as "US 5,850,221".

D5 is an extract from the textbook by Gamma et al. which was referred to in the decision under appeal (cf. II. above).

D6 is a textbook extract providing evidence of the general knowledge of the skilled person in the field of object-oriented graphical user interfaces.

D7 is a textbook extract which relates to the use of eye-tracking technology in human-computer interaction. D8 is an article cited as a reference in D7.

VI. In its communication, the board expressed the preliminary opinion to the effect that the documents D1, D4 and D7 appeared to be of particular relevance to the question of inventive step. The board also made reference to D5 which described the "publish-subscribe" notification paradigm in general terms and D6 according to which it appeared to be conventional practice to provide a graphical user interface comprising GUI-components designed to respond to a predetermined subset of "events". With respect to D7, the board further made reference to D8 which was cited on p.275 of D7 and which disclosed details of a user-interface management system (UIMS) for a direct-manipulation graphical user interface (cf. D8: 1. Introduction).

The board noted that the UIMS of D8 comprised an "executive" which appeared to provide functionality substantially similar to that of the "event engine" of the present application.

VII. The appellant did not file any written response to the board's communication.

VIII. Oral proceedings were held as scheduled on 19 February 2013.

IX. The appellant requested that the decision under appeal be set aside and that a patent be granted on the basis of the main request comprising claims 1 to 20 as submitted with the statement setting out the grounds of appeal or, alternatively, to remit the case to the department of first instance for further prosecution.

X. Claim 1 of the appellant's main and sole request reads as follows:

"An arrangement for controlling a computer apparatus (130) associated with a graphical display (120), the display (120) presenting a representation of at least one GUI-component (220) which is adapted to be manipulated based on user-generated commands, and at least one of the at least one GUI-component (220a, ..., 220n) is adapted to generate at least one respective output control signal (Ca, ..., Ca[sic]) upon a user manipulation of the component (220a, ..., 220n), the arrangement comprising an event engine (210) adapted to receive an eye-tracking data signal (DEYE) describing a user's (110) point of regard (x, y) on the display (120), and at least based on the eye-tracking data signal (DEYE) produce a set of non-cursor controlling event output signals (D-HIi) influencing the at least one GUI-component (220), each non-cursor controlling event output signal (D-HIi) describing a particular aspect of the user's (110) ocular activity in respect of the display (120), characterized in that the event engine (210) is adapted to:

receive a control signal request (Ra, ..., Rn) from each of the at least one GUI-component (220a, ..., 220n), the control signal request (Ra, ..., Rn) defining a sub-set of the set of non-cursor controlling event output signals (D-HIi) which is required by the GUI-component (220a, ..., 220n) to operate as intended;

produce the event output signals (D-HIi) which are requested by the at least one GUI-component (220a, ..., 220n) in the control signal request (Ra, ..., Rn), and

deliver non-cursor controlling event output signals (D-HIi) to the at least one GUI-component (220a, ..., 220n) in accordance with each respective control signal request (Ra, ..., Rn)."

Claim 10 seeks protection for a corresponding method.

XI. The written and oral submissions made on behalf of the appellant during the present appeal proceedings which are relevant to the present decision, may be summarised as follows:

(i) The invention relates to an arrangement and a method for controlling a computer apparatus using eye-tracking data signals as user input. A key feature of the invention is the "event engine" which is a module that receives eye-movement data from an eye-tracker and uses this to produce and deliver appropriate control signals to graphical user interface components (GUI-components).

The event engine is adapted to receive a control signal request from at least one GUI-component whereby the control signal request defines a subset of the available non-cursor controlling event output signals required by the respective GUI-component to operate as intended. The event engine then processes eye-movement data from the eye-tracker to generate and deliver control signals to the GUI-components in accordance with their respective control signal requests.

(ii) D1 describes a device and method for estimating a mental decision to select a visual cue based on eye fixation and associated event evoked cerebral potential. The device according to Dl functions as a prediction machine separating the two possible visual cue selections "select" or "reject" in a multivariate dimensional feature set space based inter alia on the eye fixation properties (cf. D1: col.16, lines 9-21). The prediction machine of D1 is in the form of a classification network which delivers a binary output having only two possible states. The function of the binary classification network of D1 is essentially different to that of the event engine of the present invention.

(iii) D4 relates to a computer system having an eyetracker which is utilized to control scrolling. However, D4 contains no identifiable disclosure of an event engine as specified in claim 1. According to the disclosure relating to Fig. 7 of D4 (cf. col.7 l.16-35), the eyetracker determines the location of a user's gaze and also determines whether the location of the user's gaze has gone outside of the boundaries of a selected object. On the basis of the description, it would appear that the eyetracker of D4 interacts directly with the scrolling application and sends commands directly to the application. Consequently, there is no requirement for an event engine as specified in claim 1.

(iv) D6 discloses subject-matter relating to event-driven object-oriented driven graphical user interfaces (GUIs). According to D6 the source of an "event" is the GUI-component itself. D6 discloses GUI-components which act as event sources by generating events. Each event source maintains an active list of interested listeners and posts events to an event queue from where they are extracted and "dispatched" by a centralised software component called an event handler. The event handler "dispatches" an event by telling the event source to notify its listeners. D6 does not disclose or suggest an event engine which receives "raw" eye-movement data from an eye-tracker and processes this to produce and deliver appropriate control signals to GUI-components in accordance with their respective control signal requests.

(v) Thus, even if the skilled person attempted to combine the teaching of D4 with the general knowledge disclosed in D6, he would still not arrive at the claimed invention.

(vi) D7 and D8 had not been considered during proceedings before the department of first instance as these documents had only been introduced into the appeal proceedings by the board. If the board maintained its view that these documents were of relevance to the question of inventive step, it was requested not to proceed to a decision on this point but to remit the case to the department of first instance in order to safeguard the appellant's right to have the matter decided by two instances.

XII. At the end of the oral proceedings the chair announced the board's decision.

Reasons for the Decision

1. The appeal is admissible. The board judges that the appeal is allowable for the reasons which follow.

2. Observations re D1 and D5

2.1 D1, which was cited as the closest prior art in the decision under appeal, relates to a device and method for estimating a mental decision to select a visual cue based on eye fixation and associated event evoked cerebral potential (cf. D1: Abstract). The device according to Dl functions as a prediction machine and employs a "classification network" for separating the two possible visual cue responses, i.e. "interest" or "no interest", in a multivariate dimensional feature set space based inter alia on eye fixation properties (cf. D1: col.3 l.22-28; col.16 l.9-21).

2.2 The classification network of D1 delivers a binary output having only two possible states and the preferred embodiment comprises a discriminant function in the form of an artificial neural network (ANN) based on Artificial Intelligence concepts (cf. D1: col.16 l.34-47).

2.3 In a preferred embodiment of D1, the system permits the user to control computerized machinery from a video display by using signals provided by an eye tracker device to select a visual icon shown on the display and to execute the computer program routine corresponding to the selected icon (D1: col.18 l.63 - col.19 l.8). The user interface of D1 thus comprises a series of display icons on which a simple selection action can be performed. There is no disclosure or suggestion of performing more complex event-driven operations on a plurality of GUI-components which can register a selective interest in specific events or, more particularly, of providing an event engine as defined in claim 1.

2.4 The board concurs with the appellant's submissions to the effect that the function of the binary classification network of D1 is essentially different to that of the event engine of the present invention (cf. Facts and Submissions, item XI(ii) above).

2.5 In the board's judgement, there is no discernible reason why the skilled person would choose to replace the binary classification network of D1 with an event engine providing the type of event-driven control of GUI-components specified in claim 1.

2.6 The examining division argued that "programming concepts such as the 'publish-subscribe' concept that is commonly used in event-based systems" were generally known at the claimed priority date (cf. Facts and Submissions, item II. above). The board agrees that, as evidenced by D5, this assertion is correct as far as it goes. However, mere knowledge of the aforementioned 'publish-subscribe' concept would not in itself have been sufficient to motivate the skilled person to replace the binary classification network of D1 with an event engine as specified in claim 1.

2.7 The board further judges that although it was common knowledge at the claimed priority date to provide graphical user interfaces comprising GUI-components designed to respond to a predetermined subset of "events" as evidenced by D6, this common knowledge would not in itself have induced the skilled person to replace the binary classification network of D1 with an event engine as specified in claim 1.

2.8 In view of the foregoing the board concludes that the subject-matter of claim 1 of the appellant's request is not rendered obvious by D1 whether considered in isolation or in combination with the general knowledge of the skilled person disclosed in D5 and D6.

3. Observations re D4 and D6

3.1 In the board's judgement, D4 discloses an arrangement for controlling a computer apparatus using input from an eye-tracking device which comprises, at least implicitly, all of the features of the pre-characterising part of claim 1.

3.2 The arrangement of D4 comprises a graphical display which presents a representation of at least one GUI-component (or "object" in the terminology of D4) which is adapted to be manipulated on the basis of user-generated commands. D4 refers to the use of object-oriented programming techniques (cf. D4: col.1 l.48 - col.2 l.6) from which the skilled person would infer that the graphical user interface of D4 (cf. D4: Figs. 2 to 5) is implemented using such techniques.

3.3 D4 does not, however, disclose an event engine as specified in claim 1. The board concurs with the appellant's submissions to the effect that the disclosure of D4 is to be interpreted as describing an arrangement according to which the eyetracker interacts directly with the scrolling application rather than via the intermediary of an event engine (cf. Facts and Submissions, item XI(iii) above).

3.4 Although it was generally known at the claimed priority date to provide graphical user interfaces comprising GUI-components designed to respond to a predetermined subset of "events" as evidenced by D6, the board concurs with the submissions made by the appellant to the effect that the event handling model disclosed in D6 is significantly different from that employed by the arrangement of claim 1 (cf. Facts and Submissions, item XI(iv) above).

3.5 In particular, D6 discloses an event handling model according to which the GUI-components act as "event sources" by generating events (cf. D6: 5.2.1.2, item 2. Source; 5.2.2.1, item 2. Source). Each event source maintains an active list of interested listeners and posts events to an event queue from where they are extracted and "dispatched" by a centralised software component called an event handler (cf. D6: 5.2.1.2, item 4. EventHandler). The event handler "dispatches" an event by telling the event source to notify its listeners.

3.6 Although the event handler of D6 is a centralised software component, it does not provide the same functionality as the event engine of claim 1 because it does not receive input data from an eye-tracker and process this to produce and deliver appropriate control signals to GUI-components in accordance with their respective control signal requests.

3.7 The board therefore takes the view that, although the skilled person might have considered using event-driven processing in the context of the system of D4, the event handling model disclosed in D6 would not have led him to provide an arrangement as defined in claim 1.

3.8 In view of the foregoing the board concludes that the subject-matter of claim 1 of the appellant's request cannot be arrived at in an obvious manner by the skilled person starting from D4 even when due consideration is taken of the general knowledge disclosed in D6.

4. Observations re D7 and D8

4.1 In its communication, the board indicated that it was of the opinion that the document D7 was also relevant to the question of inventive step, in particular when said document was considered in combination with D8 (cf. Facts and Submissions, item VI. above).

4.2 In view of the fact that D7 and D8 had only been introduced during the appeal proceedings, the appellant requested the board not to proceed to a decision on the relevance of these documents to the question of inventive step but to remit the case to the department of first instance (cf. Facts and Submissions, item XI(vi) above).

5. Conclusions

5.1 The board judges that, under the given circumstances, it is appropriate to grant the appellant's request for remittal, in particular to permit the relevance of D7 and D8 to the question of inventive step to be considered and decided upon at two instances.

ORDER

For these reasons it is decided that:

1. The decision under appeal is set aside.

2. The case is remitted to the department of first instance for further prosecution.