T 0077/14 (Detection of scrolling gestures/PHILIPS) of 30.6.2015

Summary of Facts and Submissions

I. The appeal is against the decision of the examining division to refuse the present European patent application on the grounds of lack of inventive step with respect to the claims of a main request and twenty auxiliary requests, having regard to the combined disclosures of

D7: EP-A-0 880 091 and

D5: US-A-5 880 411

and the skilled person's common general knowledge as exemplified by

D10: ISO 9241-11: "Ergonomic requirements for office work with visual display terminals (VDTs) - Part 11: Guidance on usability", pp. 1-27, March 1998.

In an obiter dictum, the decision under appeal also stated that the claims of the requests on file were not supported by the description within the meaning of Articles 84 and 123(2) EPC.

II. With the statement setting out the grounds of appeal, the appellant re-filed the twenty-one sets of claims underlying the appealed decision. It requested that the decision of the examining division be set aside and that a patent be granted on the basis of one of those claim requests.

III. In an annex to the summons to oral proceedings pursuant to Article 15(1) RPBA, the board gave its preliminary opinion on the appeal. In particular, it raised objections under Article 56 EPC 1973 for all claim requests on file, mainly having regard to D7 and D5.

IV. With a letter of reply, the appellant submitted amended claims according to a main request and four auxiliary requests, replacing the former main and auxiliary requests on file, alongside counter-arguments to the objections raised in the board's communication under Article 15(1) RPBA.

V. Oral proceedings were held on 30 June 2015, during which the appellant filed a new main request and withdrew all the previous claim requests on file.

The appellant's final request was that the decision under appeal be set aside and that a patent be granted on the basis of the new main request submitted at the oral proceedings before the board.

At the end of the oral proceedings, the decision of the board was announced.

VI. Claim 1 of the new main request reads as follows:

"A touch-screen image scrolling system, comprising:

an electronic image display screen (40);

a microprocessor (42) coupled to said display screen (40) to display information thereon and to receive interactive signals therefrom;

timer means (43) associated with said microprocessor (42) to provide timing capacity therefore[sic];

a source of scroll format data capable of display on said display screen (40);

finger touch program instructions associated with said microprocessor (42) for sensing the speed and direction of a finger touch contact with said display screen (40);

characterized in that

said finger touch program instructions associated with said microprocessor (42) are also designed for sensing the time duration of a finger touch contact with said display screen (40); and in that said

touch-screen image scrolling system further comprises:

scrolling motion program instructions associated with said microprocessor (42) responsive to said duration of said[sic] stationary finger touch contact such that, when during a period having a duration which is greater than a first preset minimum time and less than a second preset minimum time motion of said finger touch along the surface of said display screen (40) is sensed, said display is first moved in correspondence with movement of the finger touch, and, following a separation of said finger touch from said screen (40), a scroll format display on said display screen (40) is caused to begin to scroll in said sensed direction and at said sensed initial speed;

wherein sensing a finger touch during scrolling displacement of the image on said display screen (40) acts solely as 'stop motion' regardless of the length of the touch;

time decay program instructions associated with said microprocessor (42) for reducing the rate of scrolling displacement on said display screen (40) at a predetermined rate until motion is terminated;

stopping motion program instructions associated with said microprocessor (42) for terminating scrolling displacement of the image on said display screen (40) upon first occurrence of any signal in the group of signals comprising:

(a) a substantially stationary finger touch on the display screen (40) enduring for a period longer than a preset minimum time, and

(b) an end-of-scroll signal received from said scroll format data source,

wherein said scrolling motion program instructions comprise instructions responsive to said duration of said stationary finger touch contact such that, when during a period having a duration which is greater than a first preset minimum time and less than a second preset minimum time motion of said finger touch along the surface of said display screen (40) is sensed, said display is first moved in correspondence with movement of the finger touch, and, if there is no finger motion at the time when the finger contact with the display screen (40) is broken, said display screen (40) will remain in the position it is at that time without further motion, and the system reverts to "waiting" status,

wherein said scrolling motion program instructions further comprise instructions responsive to said duration of said stationary finger touch contact such that, when said duration is less than said second preset minimum time and if no motion occurs before separation of said finger from said display screen, an item touched is selected, wherein upon selection the selected item is highlighted."

Independent claim 8 of the new main request reads as follows:

"A method of controlling a scroll-like display of data on an electronic display screen (40), said method comprising the steps of:

sensing (100c) the speed and direction of motion of said[sic] finger touch contact with said display screen (40);

sensing (100b) the duration of finger touch contact time with an electronic display screen (40) having scrollable data displayed thereon;

if the sensed duration of finger touch contact time is greater than a first preset minimum time and less than a second preset minimum time and is accompanied by motion along the surface of the display screen (40) moving said display in correspondence with movement of the finger touch, and - following separation of said finger touch from said display screen (40), - initiating (104) scrolling motion of said scrollable data on said display screen (40) in said sensed direction and at said sensed speed;

upon sensing a finger touch during scrolling displacement of the image on said display screen (40), regardless of the length of the touch, stopping the motion of said display;

slowing (106) the speed of said scrolling motion from the initiated speed thereof, at a predetermined rate; and

terminating said scrolling motion upon first occurrence of any conditions from the following group of conditions is sensed:

(a) a substantially stationary finger touch having

a finite duration is sensed;

(b) an end-of-scroll signal is sensed,

if the sensed duration of said stationary finger touch contact time is greater than a first preset minimum time and less than a second preset minimum time and is accompanied by motion of said finger touch along the surface of said display screen (40), and, if after subsequent moving of said display in correspondence with movement of the finger touch, there is no finger motion at the time that the finger contact with the display screen (40) is broken, maintaining said display screen (40) in the position it is at that time without further motion, and reverting the system to "waiting" status,

wherein said method comprises the further step (100) of selecting an item touched if the sensed stationary duration of the finger touch contact time is less than said second preset minimum time and if no motion occurs before separation of said finger from said display screen (40), wherein upon selection the selected item is highlighted."

Reasons for the Decision

1. MAIN (SOLE) REQUEST

Although this claim request was submitted for the first time during the oral proceedings before the board, i.e. at a very late stage in the overall procedure, the board admitted it into the appeal proceedings by virtue of Article 13(1) and (3) RPBA, since it was considered a legitimate and eventually successful attempt (see point 1.11.1 below) to overcome the objections raised by the board.

The main request differs from the claim requests refused by the examining division essentially in that present independent claims 1 and 8 now specify that (emphasis added by the board)

A) upon sensing a finger touch during scrolling displacement of the image on said display screen the motion of said display is stopped, regardless of the length of the touch;

B) if the sensed duration of a stationary finger touch contact is greater than a first preset minimum time and less than a second preset minimum time motion of said finger touch along the surface of said display screen, and if after subsequent moving of said display in correspondence with movement of the finger touch, there is no finger motion at the time when the finger contact with the display screen is broken, said display screen is maintained in the position it is at that time without further motion, and the system is reverted to "waiting" status;

C) if the sensed duration of a stationary finger touch contact said duration is less than said second preset minimum time and if no motion occurs before separation of said finger from said display screen, an item touched is selected and highlighted upon selection.

Feature A)is supported by page 5, line 28 and in particular page 6, lines 16-18, whilst feature B) is based on page 5, lines 8-10 of the description as originally filed. Feature C) finds its support at page 4, lines 11-16 and page 6, lines 12-14 in conjunction with Fig. 1, step 102 of the application as filed.

Hence, the board is satisfied that the above amendments comply with Article 123(2) EPC. In addition, as a result of the amendments made, the board finds that the objections raised under Article 84 EPC 1973 in the obiter dictum part of the appealed decision (under the heading "Additional Remarks" in section 12) are overcome and thus no longer apply.

1.1 Article 52(1) EPC: novelty and inventive step

The board judges that the independent claims of the present main request meet the requirements of Article 52(1) EPC, for the following reasons:

1.1.1 The present invention concerns a touch-screen device supporting gesture-based scrolling operations. According to the application, the problem to be solved by the claimed invention is to enable a user/viewer to access a desired portion of a long list of data by scrolling to the location of that portion rapidly and in a more natural manner (cf. page 2, lines 2-5 of the application as filed).

1.1.2 Independent claims 1 and 8 are directed to an electronic touch-screen display device with scrolling capability configured to detect different gestures for data display and selection. In particular, process claim 8 includes the following features (as labelled by the board):

A method of controlling a scroll-like display of data on an electronic display screen, said method comprising the steps of:

a) sensing the speed and direction of motion of a finger-touch contact with said display screen having scrollable data displayed thereon;

b) sensing the duration of finger-touch contact time with said display screen;

c) if the sensed duration of finger-touch contact time is greater than a first preset minimum time and less than a second preset minimum time and is accompanied by motion along the surface of the display screen, moving said display in correspondence with movement of the finger touch, and - following separation of said finger touch from said display screen - initiating scrolling motion of said scrollable data on said display screen in said sensed direction and at said sensed speed;

d) upon sensing a finger touch during scrolling displacement of the image on said display screen, regardless of the length of the touch, stopping the motion of said display;

e) slowing the speed of said scrolling motion from the initiated speed thereof at a predetermined rate;

f) terminating said scrolling motion upon first sensing either a substantially stationary finger touch having a finite duration or an end-of-scroll signal;

g) if the sensed duration of said stationary

finger-touch contact time is greater than a first preset minimum time and less than a second preset minimum time and is accompanied by motion of said finger touch along the surface of said display screen, and if, after subsequent moving of said display in correspondence with movement of the finger touch, there is no finger motion at the time that the finger contact with the display screen is broken, maintaining said display screen in the position it is at that time without further motion, and reverting the system to "waiting" status;

h) selecting an item touched if the sensed stationary duration of the finger-touch contact time is less than said second preset minimum time and if no motion occurs before separation of said finger from said display screen, wherein upon selection the selected item is highlighted.

1.1.3 More particularly, the gesture-based functionality (i.e. the gesture-to-function mappings) underlying the touch-screen system of claim 8 can be illustrated as follows (assuming that DELTAt = duration of finger touch; t1 = first preset minimum time; t2 = second preset minimum time; v0 = initial motion speed; d = motion direction):

feature |input(gesture) |required GUI processing |output(function) |

c |long touch + motion + separation |sensing

t1<DELTAt<t2 ;

v0 ; d ; DELTAt=0 |"start scroll" |

d,f |"start scroll" + touch or end-of-scroll |sensing DELTAt>0 |"stop scroll" |

e |"start scroll" +

no touch |sensing DELTAt=0 |"slow scroll" |

g |long touch + motion + no motion + separation|sensing

t1<DELTAt<t2 ;

v0 ; d ; v0=0 ; DELTAt=0 |"wait" |

h |short or long touch +

no motion + separation |sensing

DELTAt<t2 ; v0=0 ; DELTAt=0 |"select item" |

As shown above, the input gestures are made up of various ordered, sequence-sensitive touch events (like long/short touch, motion, separation, etc.) and are mapped to the corresponding output functions, thereby building the above "gesture matrix" of the underlying human-machine interaction scheme.

1.1.4 The board agrees with the decision under appeal (cf. section 7.7 of the reasons) that document D7 represents the closest prior art for the underlying subject-matter appealed decision, since it is related to the same type of functional and structural implementation as the present invention, namely controlling gesture-based scroll operations on touch-screen devices.

Document D7 teaches that the displayed part of a mobile terminal's touch screen is scrolled in the direction of the movement of the corresponding pointing means (e.g. finger) and at a rate proportional to the speed of that pointing means (see e.g. column 1, line 55 to column 2, line 3 and column 2, lines 18-26 in conjunction with Fig. 3, steps 33 and 34). Thus, in accordance with the finding of the decision under appeal, feature a) is known from D7. In addition, D7 states that an accelerated scrolling may be executed by touching the touch screen for a "longer" time (see column 7, lines 1-8 in conjunction with column 6, lines 50-52). From this the board concludes that, contrary to the finding in the appealed decision (cf. section 8.2 of the reasons), feature b) is also anticipated by D7.

With respect to features a) and b) of claim 1, the appellant argued that D7 did not disclose sensing any direction of finger motion and the time duration of a

finger-touch contact. As to feature a), the board however finds that, firstly, this feature does not properly indicate that an arbitrary direction is sensed, but only that the direction of the actual finger motion is to be detected, and that, secondly, D7 manifestly teaches that the corresponding display list is scrolled in the desired direction based on the moving direction of the pointing means (see e.g. D7, column 5, lines 24-31) implying that the actual direction of a finger movement is indeed sensed. Furthermore, as to feature b), D7 also teaches that the underlying touch-screen device may differentiate between "long" or "longer" time periods (see column 7, lines 1-8), thus, in the board's judgment, likewise implying that the corresponding touch durations have to be detected.

As regards feature e) and the scrolling-termination criteria according to features d) and f), D7 teaches that list scrolling is retarded according to a predefined formula (see e.g. column 2, lines 31-34 in conjunction with equation (1) in column 7) and is eventually stopped by touching the display control area or an element of the displayed list with the pointing means (see e.g. column 2, lines 44-46 and column 5, lines 37-38 in conjunction with Fig. 3, step 35). Therefore and in the absence of a detailed and solid definition of an "end-of-scoll" signal, the board holds that features d) to f) are known from D7 as well.

1.1.5 However, the board finds that D7 fails to directly and unambiguously disclose that the sensed duration of a touch contact within predetermined time limits is utilised to detect the gestures and perform the gesture-to-function mappings according to features c), g) and h). Consequently, the subject-matter of the present independent claims is found to be novel over D7 (Article 54 EPC 1973).

1.1.6 The rather detailed and somewhat academic assessment of inventive step with regard to document D7 in the decision under appeal (see sections 7.7 to 8.3 and 10 of the reasons), as far as relevant for the present independent claims as amended, may be summarised as follows:

The distinguishing features did not bring about a synergistic technical effect and did not relate to the actual data-scrolling procedure itself, which was essential to the present invention, but rather to the problem of "the selection of the gestures to which is to be 'attributed the meaning' (mapped) of launching the command data scrolling in a certain direction and speed, and the command for stopping the scrolling". The latter problem represented however a non-technical problem. Based on the COMVIK approach according to T 641/00, the objective technical problem to be solved was "bringing together the hardware and software elements implementing in the computer, in structural terms using technical means, the instructions attributing the meaning to the gestures and recognising them as defined by the non-technical members of the design team" (see appealed decision, sections 10.2.2.2, 10.2.3, 10.2.4 and 10.2.6 of the reasons). In this context, the board understands from the appealed decision (see in particular page 30, first paragraph in conjunction with section 5 of the reasons) that said non-technical members of the design team, according to the examining division, are attributed to the non-technical field of "human behavioural and perception psychology". Furthermore, since the use of a time-duration counter using the concept of thresholds, as demonstrated in D5, did not require the exercise of any inventive step, and since no further non-customary structural implementation aspect was disclosed in the present application, the presence of inventive step was denied (cf. appealed decision, sections 7.5, 10.3 and 10.4 of the reasons). It was added that for implementing additional functions common-sense design measures according to the standardised usability principles as defined in document D10 would force the corresponding design team "to more complex selection of the gestures in order to differentiate them" (cf. appealed decision, sections 11.2 and 11.4 of the reasons).

1.1.7 As regards the alleged non-technical aspects of the objective problem formulated in the decision under appeal by applying the approach adopted in T 641/00 (OJ EPO 2003, 352; cf. headnote 2 and point 7 of the reasons), the board notes that the application itself is silent as to whether the selection of specific gesture-to-function mappings based on features c) to h) - out of a virtually infinite variety of devisable gesture-based functions - is predominantly based on human perception phenomena, as the examining division seems to suggest, or whether it rather depends on purely technical considerations relating to performance-related improvements of the touch-screen device in terms of e.g. its gesture recognition speed, accuracy and the like. Therefore, the board holds that only through speculation could one derive from the application that the claimed gesture-to-function mappings are generally the mere result of non-technical considerations (i.e. "attribution of meanings" as referred to in the appealed decision).

However, the board accepts that distinguishing features c), g) and h) not only relate to the definition of gesture-to-function mappings as such (i.e. corresponding to the second and fourth columns labelled "input" and "output" of the gesture matrix depicted in point 1.1.31.1.3 above) but in particular and most importantly to actually enabling at the implementation level a proper recognition of and differentiation between three distinct scroll-related functions on a touch-screen device, namely scrolling initiation (relating to feature c)), scrolling interruption (corresponding to the "waiting" status according to feature g)) and data-item selection (relating to feature h)). This is notably achieved by adding a physical parameter involving the touch-contact durations in terms of different time intervals (i.e. t1<DELTAt<t2 regarding features c) and g) or DELTAt<t2 regarding feature h) of the independent claims). The parameter thus defined represents a further degree of freedom in the design of a gestural human-machine interface, in addition to the already utilised parameters relating to the occurrence of a touch, its motion as well as the motion's speed and direction (i.e. corresponding to the third column labelled "required GUI processing" of the gesture matrix illustrated in point 1.1.31.1.3 above). The board concurs with the appellant that thereby the number of distinct ordered (sequence-sensitive) touch events and feasible

gesture-based functions may be significantly extended. For example, based on the claimed first and second preset minimum times (i.e. t1 and t2), the stationary touches detected may readily be classified into short (DELTAt<t1), long (t1<DELTAt<t2) and longer touches (DELTAt>t2), thereby extending the resulting gesture-to-function space, which may be implemented on the underlying touch-screen device, by the order of three.

Moreover, even though the operation of data-item selection according to feature h) is not directly interlinked with the actual data-scrolling procedure itself, the board believes that it is well suited to readily selecting a particular data item after scrolling through a long data list according to feature c) or g). Hence, feature h) in fact seamlessly supplements the scrolling operation for the purpose of an efficient data search and selection on the touch screen.

1.1.8 In view of the above, the objective technical problem to be solved by present independent claims 1 and 10 may be formulated as "how to extend the number of recognisable gesture-based functions in the context of scroll-based data list search on a touch-screen device as given in D7".

1.1.9 Starting from the teaching of D7, the skilled person would notice that D7 teaches that a "long" and "longer" touch time may be distinguished according to a second embodiment relating to a touch surface located outside the mobile terminal's display screen (see column 6, line 44 to column 7, line 53 and Fig. 5). However, this distinction is exemplarily and exclusively made for the purpose of accelerating the scrolling speed in that embodiment (see column 7, lines 1-8), i.e. only for an operation within the already initiated data-scrolling mode rather than also for functions surrounding and/or built upon the actual scrolling process such as interrupting the scrolling mode (which is not addressed at all in D7) according to feature g) or selecting data items, whether highlighted or not, before or after scrolling according to feature h). Hence, D7 does not provide any hint towards duration-based discrimination between a variety of further gesture-based functionalities as regards scroll-based data list search.

The board agrees with the examining division that the implementation of detecting the individual time limits via the concept of thresholds belongs to the common general knowledge of the skilled person in the field of user interface design (see also application as filed, page 4, lines 8-10: "... The technology and methodology for sensing and determining the appropriate values for information of the type herein disclosed is well-known to persons having skill in this art ..."). It is also true that prior-art document D5 demonstrates detecting different touch-duration intervals (see e.g. D5, column 35, line 22 to column 39, line 9 in conjunction with Figs. 15A to 15E using "DragTime" and "TapTime" and including scrolling as a form of dragging). Therefore, the board concedes that the implementation of detecting time-related thresholds as such may be rendered obvious by the combination of D7 and D5. However, in the board's judgment, this does not suffice to cogently arrive at the claimed subject-matter.

Rather, the board finds that actually incorporating the additional detection of predetermined time intervals for the purpose of defining and applying gesture-based scrolling initiation, according to feature c), as well as additional gesture-to-function mappings, being directly or indirectly connected with the scrolling operation, according to features g) and h), goes beyond enabling mere differentiation between distinct gestures and their resulting functions at a conceptual level. In particular, there is no discernible encouragement whatsoever in D7 or D5 to markedly extend the number of detectable combinations of ordered touch events and thus the latitude for feasible gesture-based output functions. Nor is there any pointer to leverage distinct touch-duration intervals in addition to the physical touch-detection parameters already involved. This holds true even in the light of the standardised usability principles as defined in D10, which merely provides some very general guidance about interface design in terms of effectiveness, efficiency and user satisfaction. In view of the increased implementation complexity and difficulties (e.g. relating to gesture recognition speed, conflict resolution, calibration, noise tolerance) resulting from extending the number of physical parameters to be detected, the board believes that the skilled person would rather be led away from the claimed solution, especially at the time of D7's publication (i.e. 1998). Accordingly, starting from D7 and in view of the amendments made to claim 1, its distinguishing features cannot be considered as straightforward implementation measures which the skilled person in the field of touch-screen interface design would, depending on circumstances, inevitably choose at the application's priority date, without the benefit of hindsight knowledge of the invention.

1.2 Thus, having regard to the cited prior art, the subject-matter of present independent claims 1 and 10 is held to be new and to involve an inventive step within the meaning of Article 52(1) EPC.

2. As all the other requirements of the EPC are also found to be fulfilled, the board decides that a patent is to be granted on the basis of the claims of the present main request.

Order

For these reasons it is decided that:

1. The decision under appeal is set aside.

2. The case is remitted to the department of first instance with the order to grant a patent on the basis of claims 1 to 11 submitted as new main request at the oral proceedings before the board, the drawings and a description to be adapted thereto.