T 1066/13 (Photo tag selection/BLACKBERRY) of 9.7.2018

Summary of Facts and Submissions

I. The appeal lies from the decision of the Examining Division to refuse European patent application No. 08170532.9. The application was filed as a divisional application of European patent application No. 07107866.1, with a filing date of 9 May 2007 and no priority claim.

The Examining Division decided that the subject-matter of the independent claims of a main request and of an auxiliary request lacked inventive step (Articles 52(1) and 56 EPC) over prior art document D3 in combination with document D5:

D3: Ahern, S. et al.: "ZoneTag: Designing Context-Aware Mobile Media Capture to Increase Participation", Yahoo! Research Berkeley, Berkeley, CA, USA, 6 September 2006, retrieved from http://groups.ischool.berkeley.edu/pics/papers/Ahern_et_al_zonetag_pics06.pdf;

D5: Tuffield, M. et al.: "Image annotation with Photocopain", Proceedings of the 15th International World Wide Web Conference, Edinburgh, Scotland, UK, 22 to 26 May 2006.

The Examining Division further expressed the view that the subject-matter of the dependent claims of both requests did not seem to involve an inventive step.

II. Document D3 consists of a three-page technical article and one page entitled "Index of /pics/papers" with a list of files. The first entry on this list is a file named "Ahern_et_al_zonetag_pics06.pdf", last modified on "6 September 2006".

III. In its statement of grounds of appeal, the appellant requested that the decision be set aside and that a patent be granted on the basis of one of three requests submitted with the grounds of appeal as the main request and first and second auxiliary requests.

IV. In a communication accompanying a summons to oral proceedings, the Board introduced into the proceedings a printout of the technical article of D3 with a better print quality than that on file and cited it as:

D3': Ahern S. et al.: "ZoneTag: Designing Context-Aware Mobile Media Capture to Increase Participation", Workshop on Pervasive Image Capture and Sharing (PICS 2006), Ubicomp 2006: Ubiquitous Computing, Orange County, CA, USA.

The Board noted that the Examining Division had not explained why it was convinced that the article of D3 had been published before the application's effective date of filing. The Board introduced into the proceedings printouts of web pages A1 to A5 captured by the Internet archive www.archive.org to be taken into account in establishing whether the disclosure of D3/D3' should be considered to form part of the state of the art:

A1: Ubicomp 2006 workshops program

(https://web.archive.org/web/20060816022423/http://www.ubicomp.org/ubicomp2006/conference_program/workshops/);

A2: PICS 2006 workshop web page, version of 21 August 2006 (http://groups.sims.berkeley.edu/pics/ and https://web.archive.org/web/20060821213626/http://groups.sims.berkeley.edu:80/pics/);

A3: PICS 2006 workshop papers web page (http://groups.sims.berkeley.edu/pics/papers.html), including

A3': version of 3 September 2006 (https://web.archive.org/web/20060903083156/http://groups.sims.berkeley.edu/pics/papers.html) and

A3": version of 6 July 2007 (https://web.archive.org/web/20070706154900/http://groups.sims.berkeley.edu/pics/papers.html);

A4: PICS 2006 Proposal (http://groups.sims.berkeley.edu/pics/PICS2006_proposal.pdf), version of 19 July 2006 (https://web.archive.org/web/20060719171929/http://groups.sims.berkeley.edu:80/pics/PICS2006_proposal.pdf);

A5: ZoneTag paper (http://groups.sims.berkeley.edu/pics/papers/Ahern_et_al_zonetag_pics06.pdf), version of 24 July 2007 (https://web.archive.org/web/20070724150448/http://groups.sims.berkeley.edu/pics/papers/Ahern_et_al_zonetag_pics06.pdf).

The Board briefly discussed deficiencies in terms of lack of clarity and added-subject-matter. The Board was of the preliminary view that the subject-matter of claim 1 of each of the requests was not inventive over the disclosure of D3'. Document D5 disclosed some aspects of the invention.

V. In a letter of reply, the appellant contested that document D3 was public prior art within the meaning of Article 54(2) EPC and filed new first and second auxiliary requests to replace the pending auxiliary requests.

VI. Oral proceedings were held on 9 July 2018. At the end of the oral proceedings, the chairman pronounced the Board's decision.

VII. The appellant's final requests were that the contested decision be set aside and that a patent be granted on the basis of the main request as filed with the statement of grounds of appeal or one of the first and second auxiliary requests as filed by letter of 8 June 2018.

VIII. Claim 1 of the main request reads as follows:

"A method of photo tagging and tag searching using a mobile communication device having a display, the method comprising:

presenting a tagging mode (148A) user interface for user selection of a location (302) for tagging within a photo (301);

presenting a tag entry field (406) for receiving, as a photo tag (412a, 412b, 412c), a text string entered by a user, the photo tag (412a, 412b, 412c) associated with the user selected location (302) within the photo (301);

searching a plurality of tag sources for tags that match the text string, wherein the tag sources correspond to one of a list of friends from an online service, a list of browser bookmark entries and a list of address book entries;

displaying in a matching tag list (412) any tags (412a, 412b, 412c) that match the text string; and

displaying a tag type for each tag (412a, 412b, 412c) appearing in the matching tag list (412), the tag type being associated with an icon, the tag type corresponding to one of a friend from an online service, a browser bookmark entry and an address book entry."

IX. Claim 1 of the first auxiliary request reads as follows:

"A method of photo tagging and tag searching using a mobile communication device having a display, the method comprising:

presenting a tagging mode (148A) user interface for user selection of a location (302) for tagging within a photo (301);

presenting a tag entry field (406) for receiving, as a photo tag (412a, 412b, 412c), a text string entered by a user, the photo tag (412a, 412b, 412c) associated with the user selected location (302) within the photo (301);

searching a plurality of tag sources for tags that match the text string, wherein the tag sources correspond to one of a list of friends from an online service, a list of browser bookmark entries in an Internet browser module (138) of the mobile communication device and a list of address book entries in an address book (142) of the mobile communication device;

displaying in a matching tag list (412) any tags (412a, 412b, 412c) that match the text string to allow user selection of a tag in the matching tag list (412) to complete the tag entry field (406), and displaying a tag type for each tag (412a, 412b, 412c) appearing in the matching tag list (412), the tag type being associated with an icon, the tag type corresponding to one of a friend from an online service, a browser bookmark entry in an Internet browser module (138) of the mobile communication device, and an address book entry in an address book (142) of the mobile communication device;

displaying a tag list (304) with the displayed photo (301), the tag list (304) including one or more tags (310) associated with subjects or objects within the photo (301), each tag (310) in the tag list having associated context depending on the tag type of the respective tag (310); and

activating, when the user scrolls over a tag (310) in the tag list (304), a menu including context sensitive menu options associated with the specific tag type of the respective tag (310)."

X. Claim 1 of the second auxiliary request differs from that of the first auxiliary request in that the following two text passages have been added after respectively "appearing in the matching list (412)" and "depending on the tag type of the respective tag (310)":

- "and associating a unique pointer for each tag type for highlighting the corresponding tagged user selected location (302) within the photo (301), the unique pointers having different shape and/or colour for different tag types," and

- ",wherein each tag in the tag list is displayed with the icon of its associated tag type;

upon user selection of a tag in the tag list, highlighting the associated tagged user selected location (302) within the photo (301) using the unique pointer associated with the specific tag type of the respective tag".

XI. The appellant's arguments, where relevant to this decision, are discussed in detail below.

Reasons for the Decision

1. The appeal complies with the provisions referred to in Rule 101 EPC and is therefore admissible.

The invention

2. The invention concerns a user interface for selecting a photo tag using a mobile communication device.

In a tagging mode of the user interface, the user may move a cross-hair pointer in a photo displayed on the screen to select a location on the photo to which to add a tag (paragraphs [0044] and [0045] of the original application and of the A1 publication). Different tag types are supported, e.g. free-form alphanumeric string, Facebook**TM friends, address book entries, or browser bookmarks (paragraph [0046]).

In one embodiment, the user may type text in a tag entry field and one or more tag sources, e.g. the user's address list or Facebook friend list, are searched for matching tags. The matching tags are displayed in a tag list for selection by the user. Each tag may have an associated icon or visual identifier indicating the tag type (paragraphs [0054] to [0060]).

In one embodiment, when the user scrolls over a tag in the tag list, a menu is activated with options associated with the tag. The menu may include context-sensitive menu options associated with the specific tag type. For example, for a Facebook friend there may be an item for viewing the friend's Facebook profile (paragraph [0050]).

Document D3 introduced by the Examining Division

3. Document D3 was retrieved from the internet. The Examining Division introduced it into the proceedings with a communication dated 11 January 2010, having cited D3 essentially as in section I above. Since the copies of document D3 on file were of poor print quality with blurred drawings, the Board obtained a new printed copy corresponding to D3 and introduced it as document D3'. The Board had difficulty in establishing the source and publication date of the technical article of document D3/D3' on the basis of the information on file. Taking into account prima facie evidence introduced by the Board into the appeal proceedings, document D3' was cited as originating from the PICS 2006 workshop (see section IV above), which took place before the present application's effective filing date of 9 May 2007. In its reply and at the oral proceedings, the appellant contested that document D3/D3' constituted prior art under Article 54(2) EPC. This question is dealt with below.

Internet disclosures, standard of proof and the public

4. Disclosures on the internet are generally regarded as part of the state of the art within the meaning of Article 54(2) EPC. Information disclosed on the internet is considered to be publicly available as of the date it was publicly posted (see decisions cited in Case Law of the Boards of Appeal, 8th edition, 2016, I.C.3.2.3).

4.1 Decision T 545/08 of 24 March 2017 included an in-depth review of the case law on the reliability of internet disclosures and the standard of proof to be adopted in order to establish that an internet disclosure formed part of the state of the art.

According to T 545/08, "the facts on which any finding of public availability is based must be established with a sufficient degree of certainty in order to convince the competent organ of the EPO in view of all the relevant evidence that they have indeed occurred. This holds true even if the determination is made on the basis of probabilities and not on the basis of absolute certainty ('beyond any reasonable doubt')" (reasons 11). It was thus correctly stated in the Guidelines (G-IV, 7.5.2) with respect to internet disclosures that according to the standard of the balance of probabilities it was not sufficient that the alleged fact (e.g. the publication date) was merely probable; the examining division had to be convinced that it was correct.

In T 545/08 the board also ruled that the burden of proof of the publication date for a cited document lay initially with the examining division, with at least prima facie evidence being required (reasons 12). Prima facie evidence is defined as evidence which is sufficient, on its own, to establish a fact or to raise a presumption of the truth of a fact unless controverted (see also T 750/94, OJ EPO 1998, 32, reasons 6; T 526/12 of 31 August 2015, reasons 1.4). It is then up to the applicant to prove otherwise or to at least submit evidence to displace the prima facie evidence (T 545/08, reasons 13; T 526/12, reasons 1.5).

4.2 In decision T 286/10 of 21 May 2014, the board described the Internet Archive Wayback Machine (www.archive.org) and considered that the fact that a document had been archived by the internet archive on a certain date, naturally barring special circumstances that justified suspicion, usually sufficed by itself to warrant a presumption that the document had been normally accessible to the public on the day of its downloading and made available to the public via the internet archive itself shortly thereafter (reasons 4.1 and 4.2).

This reasoning has been followed by several decisions, e.g. T 1040/14 of 24 April 2017 (see reasons 10), T 1711/11 of 9 November 2016 (see reasons 2.2) and T 2309/11 of 24 April 2017 (see reasons 6) and is therefore established case law of the Boards of Appeal (see also Case Law of the Boards of Appeal, 8th edition, 2016, I.C.3.2.3 c)).

4.3 The concept of "public" has been considered in several decisions. According to the case law of the Boards of Appeal, information is generally to be regarded as being made public already if it is made available to a limited circle of people, e.g. at a congress, who are in a position to gain access to and understand it, and if there is no obligation to maintain secrecy (see T 877/90 of 28 July 1992, reasons 2.1.5; T 202/97 of 10.2.1999, reasons 2.2.1.1; and other decisions cited in Case Law of the Boards of Appeal, 8th edition, 2016, I.C.3.3 and I.C.3.3.3).

Date of publication of document D3/D3'

5. Document D3 consists of a three-page technical article (corresponding to document D3') and a one-page listing of the entries in the directory in which the article is stored: "http://groups.ischool.berkeley.edu/pics/papers". According to this list, the file was last modified on 6 September 2006.

In the examination proceedings and in its final decision, the Examining Division did not explain why it was convinced that the article of D3 was published before the application's effective filing date of 9 May 2007. Nor did it provide any comments on the directory listing. In the decision under appeal, document D3 is cited as having a publication date of 6 September 2006, which is the date of the last modification of the file containing the article in the directory listing (see document D3, last page).

The Examining Division did not indicate why the directory listing was considered relevant. In the Board's view, a file's date of last modification in a directory listing cannot alone be considered prima facie evidence that the file was made public on or before that date, since the directory access rights could have been changed after the file's last modification. As explained in T 545/08, not every indication or hint qualifies as prima facie evidence (reasons 12).

6. Evidence introduced by the Board - A1 to A5

The Board has been able to establish that document D3 was accessible, on the date of writing the Board's communication, through the web page http://www.ubicomp.org/ubicomp2006/conference_program/workshops/#w7 of the PICS Workshop "W7: Pervasive Image Capture and Sharing". According to the website of the Eighth International Conference on Ubiquitous Computing Ubicomp 2006, the workshop took place on 18 September 2006 at that conference (http://www.ubicomp.org/ubicomp2006/conference_program/workshops/).

Archived web pages/documents A1 to A5 listed in section IV above concern that workshop and document D3/D3'.

6.1 Archived web pages A1 and A2

Web page A1 of the Ubicomp 2006 workshop programme was captured for the web archive on 16 August 2006. It includes a link to web page A2 of workshop W7 also captured by the web archive.

Web page A2 of the PICS Workshop "W7: Pervasive Image Capture and Sharing" was captured on 21 August 2006. It includes a link named "Papers" to web page A3 and a link named "Proposal" to web page A4.

6.2 Archived web page A3 - 3 September 2006 and 6 July 2007

Printout A3 introduced by the Board includes two captured versions of web page A3. Version A3' of 3 September 2006 shows an invitation to submit papers for PICS 2006 (see first page of A3). Version A3" of 6 July 2007 has links to the accepted papers (see second to last pages of A3).

On web page A3', the organisers announced that: the workshop papers would be posted there after acceptance; the deadline for submission, initially set at 16 June 2006, was extended to 30 June 2006; acceptances would be sent by 24 July 2006; and the workshop would be held on 18 September 2006.

Web page A3" has a link to document A5, which is an archived version of D3/D3'. The web archive did not capture any snapshots of web page A3 between 3 September 2006 (before the workshop) and 6 July 2007 (i.e. shortly after 9 May 2007, the date of filing of the present application).

6.3 Archived document A4

Document A4, captured on 19 July 2006 (see last page of A4), describes the goals for the PICS 2006 workshop and the process that would be followed to select the participants. It explains in particular that the organising committee would select the participants on the basis of position papers of 2 to 3 pages (page 2, right-hand column), and that the workshop proceedings "will be put together from all position papers" and "will be published as a technical report or an edited collection of papers and made available via the web" (page 3, left-hand column). It furthermore stated: "The position papers selected from participants will be published on the workshop website prior to the workshop."

The deadline for submissions was set to 16 June 2006 (see A4, page 3, first line, and A3, first page), but was later extended to 30 June 2006 (see A3, first page).

6.4 Archived document A5 - document D3/D3'

In the web archive, the earliest snapshot of online paper A5, which corresponds to article D3', seems to be that of 24 July 2007 listed above (see section IV).

7. Appellant's arguments - A1 to A5

In its reply to the Board's preliminary opinion, the appellant contested that document D3 could be considered prior art within the meaning of Article 54(2) EPC.

Web page A1 did not provide any information on the circumstances of the event, for instance whether the workshop had actually been held as scheduled and, if it had been held, what was discussed, who the participants were, under what circumstances the discussions were held and whether it was open to the public.

Web page A2 seemed to indicate that workshop W7 was due to be limited to 25 participants selected on the basis of position papers or extended abstracts. The appellant noted that the deadline for submission had been extended but not the date of announcement of acceptance. It remained unresolved whether either had actually happened.

While the organiser's goals and intentions expressed in A4 were ambitious, it could not be established whether any workshop proceedings or any corresponding technical report had been published. Since document A4 made reference to an edited collection of papers, it was likely that the papers had been edited after the workshop. In view of such considerations, the papers on web page A3 could have been made available to the public after, possibly even long after, the workshop in September 2006.

Even on the assumption that the archival dates of online documents according to the Internet archive would give indications meeting the criteria of "balance of probabilities", there were other sufficiently probable scenarios in which the actual publication date of D3 was after May 2007. It could at the most be established with a certain likelihood that paper D3/A5 had been publicly available from 6 July 2007, as potentially made evident by A3, or from 24 July 2007.

Any statements in documents A1 to A4 had to be viewed with caution, especially those relating to alleged goals or intentions. The statements in those documents could not be considered to represent reliable information. Document A4 announced that a report might be published in a major journal, but no such publication could be identified. Document A4 announced that the position papers would be published on the workshop website prior to the workshop, and web page A3 announced that the workshop papers would be posted there after acceptance. However, the version A3' archived on 3 September 2006 - only shortly before the workshop but long after the acceptance date - did not make the position papers publicly available.

At the oral proceedings the appellant further argued that it was unclear which version of the ZoneTag system had been shown at the workshop.

According to the case law, and following even the newly-established principles of decision T 286/10, the proven publication date of an online publication was still, in any case, on or shortly after the publication's earliest archival date, but not before it.

8. The Board's assessment of the evidence - A1 to A5

The Board does not agree with the appellant's argument that "it could not be established whether any workshop proceedings or any corresponding technical report had been published or not". Document A4 announced that the position papers would be "published on the workshop website" (see A4 page 3, left-hand column). There is no doubt that the workshop papers, including D3', were eventually published there, where the Examining Division and the Board were able to retrieve them.

In addition, web page A3" archived on 6 July 2007 has links to the position papers accepted for the PICS workshop, including the technical paper D3'. Following the principles established by the case law, document A5 captured on 24 July 2007 and corresponding to D3' can be considered sufficient evidence that the technical article D3/D3' was indeed published on the web page of the workshop as announced in A3 and A4, and that it was publicly available on 24 July 2007. The question remains whether it was published before the present application's effective filing date of 9 May 2007.

The Board agrees with the appellant that an archived copy does not prove online publication on a date before the publication's earliest archival date. However, the fact that no intermediate captures of web page A3 between A3' and A3", and no earlier captures of paper A5/D3', can be found in the Internet archive does not mean, without further evidence to the contrary, that web page A3" and paper A5 were not available online earlier. Furthermore, the archived copy A5 of D3' is not the only evidence in the present case.

Web page A1, which was captured by the Internet archive before the effective filing date of the present application, gives detailed information about the PICS workshop, including the date of the workshop, 18 September 2006. Document A4 and web page A3', both archived before the effective filing date of the present application, explain that the position papers selected from the participants would be published on the workshop website prior to the workshop.

From the above evidence, it is overwhelmingly probable that the PICS 2006 workshop took place on 18 September 2006 and that document D3/D3' is the published version of a position paper accepted for that workshop. It is also overwhelmingly probable that the paper of D3' was published either shortly before the workshop or within a period of approximately ten months between the date of the workshop, 18 September 2006, and the date of the A3" capture, 6 July 2007.

The Board further notes that the effective filing date of the present application is 9 May 2007, which is eight months into that period of about ten months between the date of the workshop and the captures of A3" and A5/D3' in July 2007.

The Board is not convinced that, as argued by the appellant, the workshop papers would have been made available to the public long after the workshop due to editing. It is not credible that editing the limited number of short papers of the workshop would require as long as eight months. Furthermore, the public nature of the workshop and of the papers accepted for it, and the intention to publish the papers on the website before the workshop, were announced in advance of the workshop (see A4 archived before the date of the workshop, page 3, left-hand column). In the Board's opinion, in the light of the evidence it is very probable that the papers were published shortly before the workshop, as announced, on the day of the workshop, or shortly thereafter. In particular, it is much more probable that document D3' was published before the end of the period of eight months between the workshop and the effective filing date of the present application than in the following two months up to the proven latest date of publication of 24 July 2007.

9. Document D3/D3' as prior art - conclusion

The Board therefore concludes that, if a decision is to be taken on the evidence currently on file, document D3/D3' has to be regarded as prior art within the meaning of Article 54(2) EPC for the present application.

However, at the oral proceedings, the appellant emphasised that in case the burden of proof had shifted to it, it would need more time to gather counter-evidence concerning the publication date of document D3'. The Board accepts that, since the question was first dealt with in the appeal, the appellant should be given an opportunity to submit such further evidence (see points 14 and 14.1 below). For procedural efficiency the Board nonetheless assesses inventive step of the claimed subject-matter over that document, on the assumption that it has to be regarded as prior art.

Main request

10. Inventive step - claim 1

10.1 Document D3' discloses an application called ZoneTag that can be used in a mobile communication device for assigning tags to a photo. Sources for tag suggestions include past tags from the user, the user's social network and the public, as well as names of real-world entities such as restaurants, events and venues near the user's location (abstract, Section 2, first two paragraphs). According to the abstract, the application offers a seamless interface to make it easy to assign tags to a photo.

The ZoneTag application gives the user "an option to select or type in tags (textual labels) to appear on the Flickr photo page" (abstract, page 2, right-hand column, first paragraph).

Document D3' therefore discloses a method of photo tagging and tag searching using a mobile communication device comprising the step of presenting a tagging-mode user interface for tagging a photo and a step of presenting a tag entry field for receiving, as a photo tag, a text string entered by a user, the photo tag associated with the photo.

On a tagging screen, the user is presented with a list of tags and a search box. The tagging screen presents a list of tags in order of likelihood of being selected for the current context. The tags are grouped into categories. An "All" category is also displayed, which includes tags from all the categories as well as tags the user has entered in the phone. Within a category, the user can search through the available tags by entering the first few letters of a tag in a search box (page 2, right-hand column, Figure 2 and following paragraphs). The method of document D3' therefore also includes a step of displaying in a matching tag list any tags matching the text string entered by the user.

The appellant argued that document D3 did not disclose searching tag sources, but provided tags from a server that were grouped into categories.

Document D3' indeed describes tag suggestions being "prefetched from the ZoneTag server". However, the tag suggestions are obtained from different sources, such as the user's social network, a local list of tags created by the user, or a list of nearby locations hosting events from "upcoming.org". The tag categories are said to reflect tag sources (abstract and page 2, right-hand column, paragraphs below Figure 2).

The two primary components of ZoneTag are the client application running on a mobile device and the ZoneTag server, which suggests tags to the client and passing of images and tags to Flickr (page 2, first paragraph). The ZoneTag client runs as a background process that continuously contacts the server to prefetch suggested tags appropriate to the current context. When a photo is captured, "the ZoneTag application comes to the foreground of the phone's user interface and provides an integrated tagging interface that allows the user to quickly annotate and upload the photo". The ZoneTag client sends the server the user-selected tags for a photo and the current location data. The ZoneTag server translates the location to human-readable labels (e.g. city) and propagates them to Flickr as tags (page 2, left-hand column). Document D3' also explains that "in addition to providing suggested tags to the ZoneTag client, the server also provides a web interface to tag suggestions, linked from the Flickr website" (page 2, left-hand column, third paragraph).

Document D3' therefore discloses a step of searching, at least indirectly, a plurality of tag sources.

In the method of document D3', "tags are grouped into categories, graphically organized as tabs" (page 2, right-hand column, first paragraph below Figure 2's caption). The Board is hence of the opinion that the tag categories of document D3' correspond to the tag types of claim 1.

10.2 The claimed method therefore differs from the method of document D3' in that:

(i) the tagging-mode user interface is also for user selection of a tag location on the photo;

(ii) the photo tag is associated with the user-selected location in the photo;

(iii) the tag sources correspond to one of friends from an online service, a list of browser bookmark entries and a list of address book entries;

(iv) a tag type for each tag in the tag list is displayed, the tag type being associated with an icon and corresponding to one of a friend from an online service, a browser bookmark entry and an address book entry.

10.3 At the oral proceedings, the appellant argued that all four distinguishing features contributed to the same technical problem of improving the user interface of the photo tagging application in a small device.

The Board is however of the view that the problem formulated by the appellant is too general. Features (i) and (ii) specifically solve the problem of associating tags with different elements in a photo, which is a non-technical requirement. Features (iii) and (iv) are not directed to that problem, since the support for tag types and tag sources of features (iii) and (iv) is independent of whether the tags are associated with a location as in features (i) and (ii). Consequently, no synergistic effect results from the combination of features (i) and (ii) with features (iii) and (iv).

10.4 In the Board's opinion, it would be obvious for the skilled person faced with the problem of supporting tags for different elements in a photo in the system of document D3' to let the user select a location on the photo as defined in features (i) and (ii).

The appellant argued that at the time of the invention, in 2006 or early 2007, the era of touch-screen smartphones had not yet begun and contested that at that time it was in any way obvious to provide distinguishing features (i) and (ii) in the Nokia Series60 devices of document D3'. That document acknowledged on page 1, right-hand column, that "easy input" was "especially challenging on a mobile device".

The Board however notes that, as can be seen from Figures 1 and 2, the mobile phones of document D3' support a graphical user interface of some complexity, in which the user may scroll and select items on a screen. In the Board's view, it is also possible in such a system to support a cursor moving up and down and left and right and to implement some form of location selection on a photo. The skilled person, who was aware of different user-interaction techniques to select items on a screen, would therefore consider supporting element tags by features (i) and (ii) in the device of D3'. The claim does not define more specific technical means for performing the selection.

10.5 In feature (iii), the fact that tags indicate friends or addresses is a non-technical aspect. Adding new sources of tags to the system of D3', such as a different online service or a list of bookmarks or addresses, in order to support such tags is a minor obvious modification.

The appellant argued that the tag categories listed in D3' were semantic categories that did not relate to particular applications. Unlike the claimed tag types, they did not provide a functional link between the tag string and a further application. The icon of claim 1 provided a technical effect by associating a tag (i.e. a text string) with a data structure within a separate application (i.e. an online service, a browser bookmark entry or an address book entry) and by enabling the user to launch the application by clicking on the icon.

The Board notes, however, that claim 1 does not specify a functional association established by the icons. An icon in claim 1 can be understood as a graphic representation of some information for the mere presentation of information. Feature (iv) thus lacks technical character and does not contribute to an inventive step.

10.6 In the light of the above, the subject-matter of claim 1 of the main request is not inventive over the disclosure of D3' (Articles 52(1) and 56 EPC), assuming that document is prior art within the meaning of Article 54(2) EPC.

First auxiliary request

11. Claim 1 of the first auxiliary request differs from that of the main request in that

(v) in the steps of searching a plurality of tag sources and displaying tags in a matching tag list,

- the list of browser bookmark entries is further specified as being in an internet browser module of the mobile communication device and

- the list of address book entries is further specified as being in an address book of the mobile communication device;

(vi) the step of displaying tags in a matching tag list is designed to allow user selection of a tag in the matching tag list to complete the tag entry field;

(vii) additional steps are performed to display a tag list with the displayed photo, the tag list including one or more tags associated with subjects or objects in the photo, each tag in the tag list having associated context depending on the tag type of the respective tag; and

(viii)when the user scrolls over a tag in the tag list, activating a menu including context-sensitive menu options associated with the specific tag type of the respective tag.

12. Inventive step - claim 1

12.1 The additional features establish a clear function of the tags displayed in the tag list that activates a context-sensitive menu when the user scrolls over a tag (see feature (viii)). Through the tag type and its associated context, they further establish a functional relationship between the tag and applications in the device, such as the internet browser (features (v), (vii) and (viii)).

Features (v), (vii) and (viii) are not described in document D3'. Together with distinguishing features (i) to (iv), they support facilitated access to functionality related to elements of a photo. The Board is not convinced that it would be obvious for the skilled person to add features (i) to (v), (vii) and (viii) to the photo tagging system of document D3'. Those features define an advanced and complex type of user interaction with menu options being activated during scrolling that would normally not be implemented in the more rudimentary user interface of the device of D3'.

12.2 Document D5 describes a semi-automatic image annotation system (page 1, abstract) that supports tags associated with different regions in a photo (page 8, section 4.4, Figure 2). However, document D5 does not describe any details of the user interface and does not disclose at least features (iv), (v), the context depending on the tag type of feature (vii), and feature (viii).

In particular, document D5 does not explain how the user inputs or selects a region, or searches for tags. Furthermore, document D5 does not describe tag types similar to those of features (iv) and (v) with associated contexts (as in feature (vii)), or the use of tags to facilitate user interaction in the manner described in feature (viii).

The Board further notes that, as the appellant argued, Figure 2 shows a sophisticated graphical user interface on a high-resolution display and document D5 does not mention any implementation for a device with a reduced display and limited input capabilities. In the Board's judgement, the skilled person would therefore not consider adding complex aspects of the user interface of document D5 to the system of document D3'.

12.3 The Board is therefore of the opinion that the subject-matter of claim 1 of the first auxiliary request is inventive over the disclosure of D3', alone or in combination with document D5.

Second auxiliary request

13. Inventive step - claim 1

13.1 Claim 1 of the second auxiliary request includes all the features of claim 1 of the first auxiliary request and the additional features do not change the inventive-step reasoning given above for the first auxiliary request.

The subject-matter of claim 1 of the second auxiliary request is hence also inventive over the disclosure of document D3', alone or in combination with document D5.

Further prosecution

14. The Board considers that the case should be remitted to the department of first instance for further prosecution (Article 111(1), second sentence, EPC), the reasons being as follows.

14.1 As explained under point 9 above, document D3' will have to be regarded as prior art for the present application, if the case is decided on the basis of the evidence presently on file. Nonetheless, since this crucial question was not discussed at all in the proceedings before the first instance and was first dealt with only in the Board's communication accompanying the summons to oral proceedings (see Section IV above), it is appropriate to give the appellant the opportunity to make an attempt to submit counter-evidence concerning the publication date of document D3'.

14.2 The subject-matter of claim 1 of the first auxiliary request involves an inventive step over the document D3'. The Examining Division may however wish to consider whether objections should be raised on the basis of other pieces of prior art. In addition, the text of the claims may need to be improved to resolve minor formal deficiencies and the dependent claims and the description may need to be adapted.

Order

For these reasons it is decided that:

1. The contested decision is set aside.

2. The case is remitted to the department of first instance for further prosecution.