T 2488/13 (Detecting web-page gestures/APPLE) of 7.7.2015

Summary of Facts and Submissions

I. The appeal is against the decision of the examining division to refuse the present European patent application on the grounds of lack of inventive step (Article 56 EPC) with respect to the claims of a main request and an auxiliary request, having regard to the disclosure of

D1: US-A-2005/0162402

combined with the skilled person's common general knowledge as exemplified by

D2: "Document Object Model (DOM) Level 2 Events Specification", Version 1.0, W3C Recommendation, pp. 1-47, 13 November 2000.

The following prior-art documents were also cited in the decision under appeal as evidence of the skilled person's common general knowledge:

D10: "Java Programming/Applets/Event Listeners", Internet document, 1 May 2007;

D11: M.H. Brown and M.A. Najork: "Distributed active objects", Computer Networks and ISDN Systems, Vol. 28, No. 11, pp. 1037-1052, May 1996.

Further prior-art documents quoted in the course of the examination proceedings included the following:

D3: EP-A-1 517 228;

D6: US-A-2006/0026521.

II. With the statement setting out the grounds of appeal, the appellant filed amended sets of claims as a main request and two auxiliary requests. It requested that the decision of the examining division be set aside and that a patent be granted on the basis of one of the filed claim requests.

III. In an annex to the summons to oral proceedings pursuant to Article 15(1) RPBA, the board gave its preliminary opinion on the appeal. In particular, it raised objections under Article 123(2) EPC with regard to the first auxiliary request and under Article 56 EPC with regard to the other claim requests on file, mainly having regard to D2 combined with the disclosure of

D12: S. Ramachandran and R. Kashi: "An Architecture for Ink Annotations on Web Documents", Proceedings of the Seventh International Conference on Document Analysis and Recognition, pp. 256-260, August 2003.

Prior-art document D12 was introduced into the appeal proceedings by the board under Article 114(1) EPC due to its relevance for the assessment of novelty and inventive step of the underlying subject-matter.

IV. With a letter of reply, the appellant submitted amended claims according to four additional auxiliary requests as auxiliary requests A to D.

V. Oral proceedings were held as scheduled on 7 July 2015, during which the main request and the first auxiliary request ("auxiliary request A") were discussed. Following this discussion the appellant withdrew all other claim requests on file.

The appellant's final request was that the decision under appeal be set aside and that a patent be granted on the basis of the claims of the main request or of auxiliary request A.

At the end of the oral proceedings, the decision of the board was announced.

VI. Claim 1 of the main request reads as follows:

"A method for processing touches comprising:

receiving (302) simultaneous touch input signals associated with two or more regions of a web page (100) displayed on a touch sensitive device (400), wherein each region is associated with a different respective node (152-160) in a document object model (150);

determining (304) two or more separate touch events associated with the two or more regions of the web page based on the touch input signals;

detecting a gesture event by combining the two or more separate touch events associated with the two or more regions of the web page; and

sending the gesture event to the web page for processing."

Claim 1 of auxiliary request A reads as follows:

"A method for processing touches comprising:

receiving (302) first and second touch input signals simultaneously, wherein the first and second touch input signals are associated with first and second regions, respectively, of a web page (100) displayed on a touch sensitive device (400), wherein each region is associated with a different respective node (152-160) in a document object model (150), the first touch input signal corresponds to a first touch input at a first location that corresponds to the first region of the web page, the second touch input signal corresponds to a second touch input at a second location that corresponds to the second region of the web page, and the second location is distinct from the first location;

determining (304) a first touch event associated with the first region of the web page based on the first touch input signal;

determining (304) a second touch event associated with the second region of the web page based on the second touch input signal;

detecting a gesture event by combining the at least the first and second touch events; and

sending the gesture event to the web page for processing."

The further independent claim 12 of both claim requests is directed to a corresponding computer program.

Reasons for the Decision

1. MAIN REQUEST

Claim 1 of this request differs from claim 1 of the main request underlying the appealed decision in that it now specifies that

A) each region is associated with a different respective node in a document object model (emphasis added by the board).

This amendment was apparently made in response to the objection raised in the obiter dictum part of the appealed decision (cf. section 10.1) and is based e.g. on the disclosure of paragraph [0016] in conjunction with Figs. 1A and 1B of the application as filed, thus complying with Article 123(2) EPC.

1.1 Article 52(1) EPC: novelty and inventive step

The board judges that claim 1 of the main request does not meet the requirements of Article 52(1) EPC in conjunction with Article 56 EPC, for the following reasons:

1.1.1 The present invention concerns the implementation of a touch-event model to be used for processing user touch inputs on web pages displayed on a touch-screen device. Those web pages are supposed to be organised and handled based on the well-established Document Object Model (DOM) standard. According to the application, the problem to be solved by the invention is to correctly interpret touch events (rather than mouse events) on a touch-screen device and to allow web-page developers to fully utilise its capabilities (cf. [0003], last sentence of the application as filed).

1.1.2 Claim 1 of the main request is specifically directed to processing simultaneous user touch inputs, also called "multi-touch inputs". It is common ground that the subject-matter of claim 1 is novel over the cited prior-art documents. In the decision under appeal, document D1 was considered to be the closest prior art for the subject-matter of claim 1 on file (cf. appealed decision, section 7.1). However, the board agrees with the appellant that D1 is silent on implementational details about DOM-type touch/gesture processing of web pages depending on web-page regions and thus is not concerned with dynamic or active web pages, unlike the present invention. Since a DOM-based input processing scheme inherently includes a specific way of using input event types and corresponding event handlers at each predefined node (see e.g. D2), the board cannot subscribe to the finding of the decision under appeal that applying the DOM technique for designing, organising and managing various objects included in a web page had no technical effect (cf. appealed decision, section 7.3).

The board also agrees with the appellant that D2, although relating to input processing for HTML-based web pages according to the DOM standard, fails to address any simultaneous user input events. Rather, the board regards document D12, introduced by the board (cf. point III above), as the most suitable starting point for assessing inventive step, since it is indeed directed to DOM-type processing of user touch inputs (i.e. pen inputs representing ink annotations on documents) for HTML-based web pages, like the present invention.

1.1.3 The board holds that D12 discloses the following limiting features of claim 1:

A method for processing touches comprising the steps of:

a) receiving touch input signals ("ink points") associated with regions of a web page displayed on a touch-sensitive device (see e.g. section 3, first paragraph in conjunction with Fig. 2);

b) wherein each region is associated with a different respective node in a document object model (inherently performed in a DOM-based web page processing scheme as described in D12; see e.g. section 3.1);

c) determining separate touch events ("ink coordinates") associated with the regions of the web page based on the touch input signals (see e.g. section 1, last paragraph, first sentence: "... capturing ink coordinates ... and associating the ink with the underlying objects on the web pages" in conjunction with section 4, second paragraph, third sentence: "... ink points are first converted into coordinates ...");

d) detecting a gesture event (ink gestures "left", "right", "up" and "down"; see Table 1) by combining separate touch events ("ink coordinates") associated with the regions of the web page and determined based on the touch input signals (see e.g. section 1, last paragraph, last sentence: "... the ink captured is treated as a pen gesture ..." and section 3.1, first paragraph, second sentence: "The algorithm ... determines the slope of the best-fit line obtained by the ink coordinates captured in the gesture mode");

e) sending the gesture event to the web page for processing (see section 3.1, second paragraph, first sentence: "The ink-gesture is checked ... and on a match the appropriate gesture handlers are invoked" in conjunction with Table 1).

1.1.4 Hence, the difference between the subject-matter of claim 1 and the disclosure of D12 is that simultaneous touch input signals are received. Accordingly, the subject-matter of claim 1 is considered to be novel over D12 (Article 54 EPC).

1.1.5 The board considers that the objective problem associated with the above-identified distinguishing feature may be formulated as "how to extend the

single-touch gesture recognition scheme of D12 to multi-touch inputs on a touch-screen device".

1.1.6 When confronted with the above objective problem with which the skilled person in the field of touch-screen devices might realistically have been faced at the application's priority date, the person skilled in the art would know that recognising multi-touch user inputs on different regions of the screen was well known at the application's priority date (see e.g. D3, [0042] or D6, [0114] in conjunction with Fig. 17C). Therefore, and further considering that D12 relies on very basic gestures and aims at covering more complex gestures in the future (see section 3.1, second paragraph), the board finds that the skilled person would readily use, in an obvious way, the ink coordinates captured on one or more regions of the web page for also recognising multi-touch gestures.

1.1.7 The appellant argued at the oral proceedings before the board that e.g. D3 and D6 described merely how simultaneous touches are recognised, without providing any information about the actual transformation of touch input signals into touch events and then into gesture events as claimed.

The board notes, however, that D12 already teaches that kind of transformations, with the "ink points" (i.e. the actual pen positions) corresponding to the touch input signals and the "ink coordinates" corresponding to the touch events (see also the present description as filed, [0018], last sentence: "... each touch event can include a set of coordinates at which a touch is currently occurring") in respect of the phraseology of claim 1. Furthermore, documents D3 and D6 are solely intended to exemplify that simultaneously detecting multiple touch input signals and/or touch events was well within the reach of the skilled person at the application's priority date.

1.1.8 In view of the above, the subject-matter of claim 1 of the main request does not involve an inventive step having regard to D12 (Article 56 EPC).

1.2 In conclusion, this request is not allowable under Article 56 EPC.

2. AUXILIARY REQUEST A

This request was submitted for the first time after the appellant had filed its statement setting out the grounds of appeal, i.e. at a relatively late stage of the overall procedure. The board admitted it into the appeal proceedings under Article 13(1) RPBA, since it was in fact an appropriate and eventually successful attempt (see point 2.22.2 below) to overcome the objections raised by the board.

2.1 Independent claims 1 and 12 of this auxiliary request differ from those of the main request in that they now specify that (emphasis added by the board)

B) first and second touch input signals are received simultaneously and are associated with first and second regions, respectively, of the web page;

C) the first touch input signal corresponds to a first touch input at a first location that corresponds to the first region of the web page, the second touch input signal corresponds to a second touch input at a second location that corresponds to the second region of the web page, and the second location is distinct from the first location;

D) a first touch event associated with the first region of the web page is determined based on the first touch input signal and a second touch event associated with the second region of the web page is determined based on the second touch input signal;

E) the gesture event to be sent to the web page is detected by combining at least the first and second touch events.

Features B) to E) are taken in combination from the embodiment relating to the so-called "rotating gesture" based on moving two fingers placed on two different

web-page regions clockwise or counterclockwise (cf. [0024] of the description as filed). With respect to the feature of sending the gesture event to the web page, the corresponding passage in paragraph [0024] of the original description states:

"Gesture Events can be sent to the web page 100 before TouchEvents ... The gesture event can then be sent to the web page 100, followed by the touch events ..." (emphasis added by the board).

From this disclosure the board concludes that additionally sending the touch events, after sending the gesture event, is foreseen as an optional measure rather than disclosed as an essential feature. Hence, the board is satisfied that the above feature of claim 1 is sufficiently supported by the original application (Article 123(2) EPC).

2.2 Article 52(1) EPC: novelty and inventive step

The board holds that the independent claims of the present auxiliary request also meet the requirements of Article 52(1) EPC, for the following reasons:

2.2.1 The board considers document D12 to be the closest prior art also for the subject-matter of independent claims 1 and 12 of the present auxiliary request. As D12 is directed to a single-touch device, it inevitably fails to disclose features B) to E).

2.2.2 As to the wording of the independent claims, it is apparent to the board that distinguishing features B) and C) are related to the user's specific input behaviour, whilst distinguishing features D) and E) describe the specific reaction of the touch-screen device to the user input. With respect to the technical effect achieved by such a human-machine interaction scheme, the board accepts that distinguishing features B) to E), together with feature A), synergistically yield the overall effect of enabling the underlying touch-screen device to recognise gestures composed of simultaneous two-finger touches on distinct web-page regions instead of accommodating gestures applied only in a certain region. In practice, this may allow implementing gestures such as the described "rotating gesture" which involves different web-page elements e.g. for rotating them, thereby manifestly extending the usability of the touch-screen device in question for different user purposes. This may also be derived from the present description as originally filed (see e.g. [0024], last sentence).

2.2.3 In view of the above, the objective technical problem to be solved by the independent claims may be formulated as "how to improve the extendability of gesture-to-function mappings applied to web pages displayed on the touch-screen system of D12".

2.2.4 Starting out from D12 the skilled person would be aware that in a DOM-based input event processing scheme, event listeners or handlers are associated to each individual web-page region for collecting touch events (ink coordinates), possibly forming gestures and forwarding them to the web page for further processing. Hence, even if D12 were to support multiple touches, the skilled person would still assume that a gesture is recognised based only on the touch events of the associated event listener/handler, i.e. of the corresponding web-page region (see e.g. Fig. 2 in which different text ranges correspond to different web-page regions). Consequently, the skilled person would foresee that the touch events recognised within one and the same region are used for building the respective gestures to be sent to the web page rather than combining touch events arising from different web-page regions. In order to solve the above objective problem and in view of the fact that D12 provides the prospect of supporting more complex gestures (see D12, section 3.1, second paragraph), the person skilled in the art would rather devise more intuitive and/or sophisticated gestures and associated functions to be applied within the individual web-page regions, thus ending up with a solution different from that of claims 1 and 12. Therefore, the board holds that the present independent claims credibly provide a synergistic non-obvious effect which goes beyond the sum of the individual effects of their distinguishing features.

2.2.5 Moreover, the board finds that none of the other cited prior-art documents renders the subject-matter of claims 1 and 12 obvious, whether taken alone or in combination with the disclosure of D12:

Document D1 relates to gesture recognition for alphanumeric inputs made on a touch pad separate from the display based on display regions ("active objects") and visual feedback (partly also for web pages). However, it is - beyond optionally involving web browsing (see D1, [0068]) - completely silent as to the matter of the use of DOM-based dynamic web-page processing or event handler/listeners and in particular as to recognising gestures composed of touch events detected on distinct web-page regions (see also point 1.1.21.1.2 above).

Document D2 touches merely on the issue of web-page input processing for conventional desktop devices (e.g. mouse input events) based on different HTML object elements such as text or graphic objects, whilst failing to provide any incentive to implement recognition of multi-touch gestures composed of touch events detected on distinct web-page regions.

Documents D3 and D6, albeit relating to multi-finger gesture recognition based on different display regions, likewise does not provide a hint towards web-page processing based on recognising gestures composed of touch events detected on distinct web-page regions.

Lastly, documents D10 and D11 were cited in the decision under appeal solely as evidence of the skilled person's common general knowledge as regards using event handlers or active web-page objects and HTML tags in relation to D1 respectively. No motivation whatsoever is however given for the recognition of multi-touch gestures composed of touch events detected on various web-page regions.

Accordingly, even if the teachings of D12 and the above prior-art documents were combined, the skilled person would not arrive at the claimed solution.

2.3 Hence, having regard to the cited prior art, the subject-matter of the present independent claims is held to be new and to involve an inventive step within the meaning of Articles 54 and 56 EPC.

3. As all the other requirements of the EPC are also found to be fulfilled, the board decides that a patent is to be granted on the basis of the set of claims according to auxiliary request A.

Order

For these reasons it is decided that:

1. The decision under appeal is set aside.

2. The case is remitted to the department of first instance with the order to grant a patent on the basis of claims 1 to 19 of auxiliary request A submitted with the letter dated 5 June 2015 and pages 2 to 25 of the description as published, page 1 of the description as filed on 19 April 2011, and drawing sheets 1 to 6 as published.