T 2568/17 (Haptic effect synchronisation/IMMERSION) of 25.6.2019

Summary of Facts and Submissions

I. This appeal is against the decision of the examining division, posted on 12 June 2017, refusing

European patent application No. 06826378.9. The sole request was refused because of lack of clarity

(Article 84 EPC) and lack of inventive step

(Article 56 EPC), having regard to the disclosure of

D2: WO 2005/048541 alone, or in combination with

D4: US 2003/0002854.

II. Notice of appeal was received on 21 August 2017 and the appeal fee was paid on the same day. The statement setting out the grounds of appeal was received on

20 October 2017. The appellant requested that the decision be set aside and that a patent be granted based on a main request or on auxiliary requests 1 to 3, all requests submitted with the statement setting out the grounds of appeal. Oral proceedings were requested in the event that none of the requests were allowed.

III. A summons to oral proceedings was issued on

12 April 2019. In a communication pursuant to

Article 15(1) RPBA attached to the summons, the board gave its preliminary opinion that the requests did not meet the requirements of Article 56 EPC. The board also raised an objection under Article 123(2) EPC against the second auxiliary request.

IV. With a letter of response dated 24 May 2019, the appellant maintained the main request and auxiliary request 1 and submitted new auxiliary requests 2 to 5 to replace the previous auxiliary requests 2 and 3.

V. Oral proceedings were held on 25 June 2019, during which the appellant withdrew all its auxiliary requests.

VI. Claim 1 according to the main request (sole request) reads as follows:

"Method for synchronizing haptic effects with audio information and/or video information, characterized by comprising:

receiving a media transport stream; the media transport stream comprising an embedded master time code signal and a series of haptic frames including haptic information and a series of media frames including at least one of audio information or video information;

extracting the master time code signal from the media transport stream;

extracting the media frames and the haptic frames from the media transport stream;

using the master time code signal to synchronize the haptic information and at least one of the audio information or the video information by:identifying haptic information in the haptic frames of the media transport stream;determining time stamps corresponding to the haptic information in accordance with the master time code signal;assigning a time stamp to each haptic frame; wherein the time stamps indicate when to activate one or more actuators to generate haptic effects according to the haptic information stored in the haptic frames; anddetermining haptic effect information for each haptic effect in response to the haptic information in the haptic frames."

The request comprises a further independent claim (claim 10) relating to a corresponding apparatus.

Reasons for the Decision

1. The appeal is admissible (see point II).

2. Article 56 EPC

2.1 Prior art

D2 discloses a method for providing feeling effects, for instance haptic effects such as vibrations, to a user when audio/video information is played back (see the abstract). To this end, haptic information is transmitted within an audio/video stream to indicate to the receiver which haptic effect is to be provided with which audio/video data. Audio and video packets comprise headers containing data for controlling time synchronisation such as time stamps (see page 10, lines 3 to 12). The time stamps contained in the headers of the audio and video packets thus represent a master time code signal which can be extracted from the media stream and can be used by the receiver to synchronise the audio and video data by playback. The audio and video packets are grouped into frames and in each frame a packet comprising haptic information, corresponding to the audio and video data in the frame, is added at the end of the frame (see page 11, lines 10 to 33 and page 18, lines 9 to 12). Since the packet comprising the haptic information is transmitted within the frame comprising its corresponding audio/video data, the haptic information can be provided at the receiver, synchronised with the played-back corresponding audio/video data (see page 19, lines 26 to 34).

D4 discloses a system for transmitting audio and video frames to be played back at a receiver (see paragraph [0039]). A time stamp is included in each audio frame and each video frame for synchronizing audio and video frames which have to be played back concurrently (see paragraphs [0040], [0043] and [0051]).

2.2 It was agreed in the examination and appeal proceedings that D2 represents the closest prior art.

The difference between the subject-matter of claim 1 and the disclosure of D2 is that time stamps corresponding to the haptic frames can be determined in accordance with the master time code signal and assigned to the haptic frames, these time stamps indicating when the haptic effects are to be generated according to the haptic frames. In D2, no time stamps are directly assigned to the haptic frames but the time at which haptic effects have to be generated is the time at which the corresponding audio and video data of the same frame are played back, this time being determined by the time stamps of the audio and video packets in the frame.

The appellant argued that the technical effects of this distinguishing feature are that a frequent re-synchronisation of the frames can be permitted, which avoids a drift out of synchronisation between corresponding haptic and media (i.e. audio and video data) playback signals.

The objective technical problem, formulated by the appellant, is thus to enable a more reliable synchronisation between the generation of the haptic effect and the playback of the corresponding audio and video data.

The board maintains that a skilled person desiring to improve the synchronisation in D2 would consult prior art documents related to synchronisation issues within a stream of frames to be played back, irrespective of the kind of the information contained in the frames. The skilled person would find in D4 a method of assigning a time stamp to each frame in a stream of audio and video frames to achieve synchronisation (see paragraph [0040]). By applying the teaching of D4 to the method of D2, the skilled person would assign a time stamp to each haptic frame of D2, not only to media frames and thus arrive at the subject-matter of claim 1 without exercising inventive skills.

2.3 The board is in particular not convinced by the appellant's argument that the master time code signal defined in claim 1 is a separate signal received independently from the audio, video and haptic frames and defining an "absolute time line" [sic], as illustrated in Figure 4. First of all, claim 1 recites extracting the master time signal from the media transport stream, which contradicts construing the master code signal as a separate signal. Secondly, the master time code signal is mentioned only in paragraphs [[0007] and [00014] of the description, and defined therein as being embedded in the media transport stream and not as a separate signal. The other paragraphs mentioning a master time code, and not a master time code signal, state that the code is embedded in or transmitted via the media transport stream (see paragraphs [00021], [00024], [00031] and [00040]), for instance in the audio component (see paragraph [00024]).

The appellant also argued that assigning a time stamp to each haptic frame at the receiver enables a timely rearrangement of the haptic frames so that they can be played at the receiver independently from the sequence of received media frames. The board cannot follow this line of argument. Firstly, such a mode of operation at the receiver is not mentioned at all in the description. Secondly, the very aim of the system described in the application is to enable the activation of haptic effects at the proper time corresponding with the playback of corresponding audio and video signals, the timing relationship between haptic effects and audio/video data being enshrined in the received media transport stream (see for instance paragraphs [00014], [00016] and [00021]).

2.4 For these reasons, the board maintains that the subject-matter of claim 1 does not involve an inventive step (Article 56 EPC), with regard to the disclosure of D2 in combination with D4.

Order

For these reasons it is decided that:

The appeal is dismissed.