T 0606/17 (Image manipulation/NOKIA) of 20.8.2021

Summary of Facts and Submissions

I. This case concerns the appeal of the applicant against the decision of the examining division refusing the European patent application principally on the ground of lack of inventive step in the light of the disclosure of the following document:

D1: US 2003/0117501 A1.

II. The board's decision also mentions the following documents:

D10: US 6518957 B1

D11: US 6640113 B1.

III. The appellant requests that the decision under appeal be set aside and that a patent be granted on the basis of the claims of either a main request or one of five auxiliary requests, all as filed with the statement of grounds of appeal.

IV. Oral proceedings were held on 20 August 2021. The chair announced the board's decision at the end of the oral proceedings.

V. Claim 1 of the main request reads as follows:

"A method, comprising:

controlling a touch sensitive display (22; 32) to simultaneously display a first image (62) and a second image (60), wherein the second image is smaller than the first image and the second image is displayed in picture-in-picture format relative to the first image;

enabling a user to manipulate the first image, while the second image is smaller than the first image and is displayed in picture-in-picture format relative to the first image; and

enabling the user to manipulate the second image, using the touch sensitive display, while the second image is smaller than the first image and is displayed in picture-in-picture format relative to the first image."

VI. Claim 1 of the first auxiliary request is the same as claim 1 of the main request except that the second clause reads (amendments underlined by the board):

"enabling a user to manipulate the first image, while the second image is smaller than the first image independently of the second image and is displayed in picture-in-picture format relative to the first image".

VII. Claim 1 of the second auxiliary request reads as follows (amendments with respect to claim 1 of the main request underlined by the board):

"A method, comprising:

controlling a touch sensitive display (22; 32) to simultaneously display a first image (62) and a second image (60), wherein the second image is smaller than the first image and the second image is displayed in picture-in-picture format relative to the first image;

in response to a user input, manipulating the first image during the simultaneous display of the first image and the second image while the second image is displayed in picture-in-picture format relative to the first image at a smaller size than the first image; and

in response to another user input using the touch sensitive display, manipulating the second image during the simultaneous display of the first image and the second image while the second image is displayed in picture-in-picture format relative to the first image at a smaller size than the first image."

VIII. Claim 1 of the third auxiliary request is the same as claim 1 of the second auxiliary request except that the second clause reads as follows (amendments with respect to claim 1 of the second auxiliary request underlined by the board):

"in response to a user input, manipulating the first image independently of the second image during the simultaneous display of the first image and the second image while the second image is displayed in picture-in-picture format relative to the first image at a smaller size than the first image".

IX. Claim 1 of the fourth auxiliary request is the same as claim 1 of the main request except that the second clause reads as follows (amendment with respect to claim 1 of the main request underlined by the board):

"enabling a user to manipulate the first image, while the second image is not manipulated and is smaller than the first image and is displayed in picture-in-picture format relative to the first image".

X. Claim 1 of the fifth auxiliary request is the same as claim 1 of the second auxiliary request except that the second clause reads as follows (amendments with respect to claim 1 of the second auxiliary request underlined by the board):

"in response to a user input, manipulating the first image during the simultaneous display of the first image and the second image while the second image is not manipulated and is displayed in picture-in-picture format relative to the first image at a smaller size than the first image".

Reasons for the Decision

1. Technical context

The present application concerns a mobile communication device having a touch-sensitive display on which two images can be displayed simultaneously, namely a background image ("first image") and a smaller foreground image ("second image") displayed in

picture-in-picture format relative to the first image. The essential idea is that both images can be independently manipulated (e.g. moved or re-sized) while both images are displayed.

2. Main request - claim 1 - inventive step (Articles 52(1) and 56 EPC)

2.1 The closest prior-art document is considered to be D1. D1 discloses a portable communications device such as a mobile phone equipped with two cameras, a first "front" camera able to take a photo of a view and a second "back" camera pointed at the user able to simultaneously take a picture of a user. A small-sized image of the user (foreground image) is then superimposed on the background image of the view, i.e. displayed in picture-in-picture format. In D1, the display is conventional, i.e. not touch-sensitive. Various manipulations of the foreground image are possible using a keypad, e.g. re-positioning and

re-sizing (cf. paragraph [0048]).

2.2 Claim 1 of the main request includes the following features (feature labelling by the board):

A method, comprising:

A) controlling a touch-sensitive display

B) to simultaneously display a first image and a second

image,

C) wherein the second image is smaller than the first

image and

D) the second image is displayed in picture-in-picture

format relative to the first image;

E) enabling a user to manipulate the first image,

F) while the second image is smaller than the first

image and

G) is displayed in picture-in-picture format relative

to the first image; and

H) enabling the user to manipulate the second image,

I) using the touch-sensitive display,

J) while the second image is smaller than the first

image and

K) is displayed in picture-in-picture format relative

to the first image.

2.3 It is noted that, in accordance with feature E, the first image is not necessarily manipulated using the touch-sensitive display. However, for the sake of argument, the claim is interpreted in the sense that both images are manipulated using the touch-sensitive display.

2.4 The appellant argued that features A and I on the one hand and features E, F and G on the other hand are neither disclosed in D1 nor obvious in the light of D1 and common general knowledge.

2.5 As regards which are the distinguishing features, the board agrees with the appellant. D1 does not disclose a touch-sensitive screen (features A and I), and, in paragraph [0036], in contrast to the view of the examining division (cf. page 6, 2nd paragraph of the impugned decision), D1 does not unambiguously disclose that the background image can be manipulated whilst the foreground image is displayed (features E, F and G).

2.6 The board agrees with the appellant that the objective technical problem can be seen as how to provide improved user control over image manipulation of a picture-in-picture image.

2.7 Re features A and I:

The board fully agrees with the comments of the examining division on page 6, last paragraph of the impugned decision regarding the obviousness of replacing the display of D1 by a touch-sensitive display in order to solve this problem. In fact, a touch-sensitive display, as was well-known even at the priority date (year 2004), renders user control, including aspects of image control, much simpler. In particular, if the method of D1 were implemented using a touch-sensitive display, the foreground image could, like an icon, easily be moved across the display using a finger or stylus.

2.8 Re features E, F and G:

Once the person skilled in the field of GUI design has implemented the method of D1 using a touch-sensitive display and is viewing the superimposed image, it would have been an obvious further wish, in order to provide the user with increased control over image manipulation of the picture-in-picture image, to be able to manipulate the background image as well as the foreground image, e.g. in order to make quality adjustments, zoom (re-size) the image, or indeed "[o]ther manipulations of the first picture and the second picture, as understood by those in the art" (cf. the description of the present application, paragraph [0028]). Obvious other examples here, based on common general knowledge, would have been replacing the background image with another image or deleting the background image. Obviously, as with the foreground image, control of the background image would have involved touching directly the background image with a finger or stylus, e.g. to activate a menu of options. Furthermore, when designing an application for processing the background image, there would be no need to interfere with the displayed foreground image as the foreground and background images are stored and processed separately before being superposed (cf. D1, paragraph [0036]). Indeed, this principle of independent manipulation is already disclosed in D1 for the case of manipulation of the foreground image (cf. paragraphs [0037], [0038] and [0048]).

2.9 At the oral proceedings, the appellant argued mainly as follows:

The board's objection was based on an ex-post facto analysis and did not take account of the technological situation prevailing in 2004, the year in which the priority application was filed. In this respect, D1 concerned a mobile phone and the only touch-sensitive displays for mobile phones available at the priority date such as those cited in the examination proceedings (D10 and D11) were primitive, required pressing the screen very firmly, and were only conceived for dialling a number and thus not suitable for image manipulation. Indeed, easy manipulation of photo images on a touch-sensitive display of a mobile phone as understood today only became available with the advent of the iPhone**(TM) in about 2007. Moreover, D1 did not address at all the above objective technical problem. Thus, there was no prompting or signpost to be found in prior-art document D1 to use a touch-sensitive display for image manipulation of the background image, especially as D1 concentrated only on manipulating the foreground image.

2.10 The board however does not find these arguments convincing for the following reasons:

In the first place, it is noted that D1 is not limited to a mobile phone but more broadly discloses "a portable communication terminal". The skilled person would therefore consider a wider range of

touch-sensitive displays than those developed only for mobile phone use.

Secondly, the board recalls that the closest prior art does not have to disclose the objective technical problem, which is only determined in the second step of the problem-and-solution approach based on the technical effect(s) provided by those features distinguishing the invention as claimed from the closest prior art (see e.g. T 698/10, Reasons 3.4; T 910/90; Reasons 5.1, last sentence).

Further, the term "manipulation" used in the claim is extremely broad and includes simple menu-driven manipulation operations as indicated above. Therefore, the capabilities of an iPhone**(TM)are not required. Indeed, as regards the capabilities and properties of touch-sensitive displays at the priority date, the present application contains no implementation details at all. In particular, there are no implementation details as to how the implementation of the background image is independently performed while the foreground image is displayed. The board notes in this respect that if, arguendo, it is assumed that the requirement for sufficiency in accordance with Article 83 EPC is met, it also has to be assumed that the skilled person at the priority date would have been fully aware in the context of a touch-sensitive display how to carry out certain manipulations of the background image independently of the foreground image, e.g. using a pop-up menu activated by touching the screen. These implementation aspects therefore do not contribute to inventive step.

Finally, as regards a "prompt" for wishing to manipulate the background image independently of the foreground image, this arises from the obvious situation of a user of the device of D1 being not entirely satisfied with the background image although satisfied with the foreground image. In this situation, it would be clearly undesirable when changing the background image to have to change the foreground image, e.g. by capturing another foreground image with the camera at the same time as capturing a new background image, since the new foreground image may then be less good than before.

2.11 The appellant further argued in the written proceedings that if the person skilled in the art wished to manipulate the content of the background image, then he was explicitly taught by D1 to switch the content of the foreground image with the content of the background image in order that the new foreground image may be manipulated in the manner described in paragraphs [0037], [0038] and [0048].

The board however can find no teaching in D1 to this effect. Furthermore, even just contemplating this unwieldy procedure would lead the skilled person to consider the possibility of directly manipulating the background image, without having to switch the images. This is all the more obvious considering that the use of a touch-screen display clearly facilitates such manipulation.

2.12 It is concluded that the subject-matter of claim 1 of the main request does not involve an inventive step. Thus, the main request is not allowable under Articles 52(1) and 56 EPC.

3. First to fifth auxiliary requests - claim 1 - inventive step

3.1 Claim 1 respectively of each of the auxiliary requests essentially claims the same method as claim 1 of the main request using more precise wording. In this respect, the amendments make no difference to the assessment of inventive step because the more precisely defined features in respect of (i) independent manipulation of the first and second images, and (ii) manipulation "during the simultaneous display of the first image and the second image" have already been assumed when discussing the main request.

3.2 Hence, the reasoning given in respect of claim 1 of the main request applies, mutatis mutandis, to claim 1 of each of the auxiliary requests (Article 52(1) and 56 EPC).

Order

For these reasons it is decided that:

The appeal is dismissed.