T 1410/12 () of 20.10.2017

Summary of Facts and Submissions

I. The appeal is directed against the decision to refuse European patent application No. 06 736 531.2, published as international application WO 2006/094035 A1.

II. The patent application was refused by the examining division on the grounds that the independent claims of the main request did not comply with Article 84 EPC. The examining division added a chapter entitled "Further Observations" to the decision and stated that the subject-matter of the independent claims lacked novelty (Article 54(1) and (2) EPC) in view of:

D1: Chu, Chung-Tao et al.: "Bi-directional object-based coding in video compression at very low bitrate", in: Proceedings of the 3rd International Conference on Signal Processing, vol. 2, pages 986 to 989, 14 October 1996, XP010209494, ISBN: 0-7803-2912-0; or

D2: WO 2004/023819 A2.

Even if the independent claims were interpreted in a restrictive sense as argued by the applicant, the subject-matter of the independent claims would still lack an inventive step (Article 56 EPC) in view of:

D8: US 2004/0240554 A1 in combination with

D9: LEE, J. W.: "Bit allocation for MPEG-4 video coding with spatio-temporal tradeoffs", IEEE Transactions on Circuits and Systems for Video Technology, IEEE Service Center, Piscataway, NJ, US, vol. 13, no. 6, June 2003, pages 488 to 502, XP002351714, ISSN: 1051-8215.

III. The applicant (appellant) appealed against this decision and requested that it be set aside. With the statement of grounds of appeal, the appellant submitted amended claims of a main request and of auxiliary requests I to V and requested that a patent be granted on the basis of these claims. It also requested reimbursement of the appeal fee.

IV. In a letter dated 18 September 2017 in response to a communication annexed to a summons to oral proceedings the appellant submitted claims of an amended first auxiliary request replacing the previous auxiliary request I.

V. Oral proceedings were held before the board on 20 October 2017. At the oral proceedings, the appellant filed the claims of a new main request. It requested that the decision under appeal be set aside and that a European patent be granted on the basis of the claims of the main request filed at the oral proceedings of 20 October 2017, or the first auxiliary request filed with the letter dated 18 September 2017, or the auxiliary requests II to V filed with the statement of grounds of appeal. Moreover, it requested that the appeal fee be reimbursed.

VI. Independent claims 1, 18 and 35 of the main request read as follows:

"1. A method comprising:

obtaining (66; 88) a video frame;

encoding (74, 76; 96 - 100) a region of interest (ROI) and a non-ROI within the video frame by allocating bits between the ROI and non ROI areas;

determining content activity in the ROI area; determining content activity in the non-ROI area; and determining (74; 91) whether to skip encoding of the non-ROI area of the video frame based on content activity of the ROI (90) and the content activity of the non-ROI (92) area and accumulated distortion (94) due to skipping of encoding of non-ROI areas in one or more other frames."

"18. A device comprising:

a region-of-interest mapper (52) that generates a definition of a region of interest (ROI) and a non-ROI within a video frame;

a video encoder (58) that encodes the ROI and the non-ROI within the video frame by allocating bits between the ROI and non-ROI areas;

a frame analyzer (60) for determining content activity in the ROI area and for determining content activity in the non-ROI area; and

a skipping module (50) that determines whether the encoder will skip encoding of the non-ROI area of the video frame based on content activity of the ROI and the content activity of the non-ROI area and accumulated distortion due to skipping of encoding of non-ROI areas in one or more other frames."

"35. A computer-readable medium comprising instructions to cause a processor to:

obtain (66; 88) a video frame;

encode (74, 76; 96 - 100) a region of interest (ROI) and a non-ROI within the video frame by allocating bits between the ROI and non-ROI areas;

determine content activity in the ROI area;

determine content activity in the non-ROI area; and

determine (74; 91) whether to skip encoding of the non-ROI area of the video frame based on content activity of the ROI (90) and the content activity of the non-ROI (92) area and accumulated distortion (94) due to skipping of encoding of non-ROI areas in one or more other frames."

The remaining claims 2 to 17, 19 to 34 and 36 to 51 are dependent on claims 1, 18 and 35, respectively.

The wording of the claims of the auxiliary requests has no bearing on this decision.

VII. The examining division held that the determining step of claim 1 of the main request underlying the decision under appeal ("determining (74; 91) whether to skip encoding of a non-ROI area of the video frame based on content activity of the ROI (90) and the non-ROI (92) area and accumulated distortion (94) due to skipping of encoding of non-ROI areas in one or other frames") rendered claim 1 unclear. In claim 1 there was "no feature of the algorithm or of the mathematical function used to obtain the decision" (to skip encoding of non-ROI areas), "leaving the skilled person in doubt even about whether all the parameters" (content activity in the ROI and non-ROI and accumulated distortion) "are used in the decision". It also noted that the wording "based on" in the determining step was interpreted as "based on at least one of ..." because of the lack of features of the claims excluding this interpretation (see decision under appeal, sections 1, 2 and 4).

On the basis of that interpretation, the subject-matter of claim 1 was not new in view of D1 and D2.

In case the independent claims were interpreted restrictively, i.e. in the sense that the determining step was dependent on all three parameters (content activity in the ROI and non-ROI areas and accumulated distortion), then D8 would be regarded as the closest prior art to the subject-matter of claim 1. It disclosed all features of claim 1 except that the accumulated distortion due to skipping of encoding of non-ROI areas in one or more other frames was taken into account in the skipping decision. The associated technical problem was how to improve encoding of the sequence based on rate-distortion criteria. This problem and the solution were disclosed in D9 (see first page, right column, second paragraph, and chapters V.A and V.D).

VIII. The appellant's arguments, as far as they are relevant to the present decision, may be summarised as follows:

The decisive phrase in the independent claims was analogous to "based on A and B and C", which was grammatically structured as an AND statement and could by no means be construed as being an OR statement.

The examining division's assessment of inventive step based on D8 as closest prior art did not take into account that D8 did not disclose any of the "determining" steps of claim 1. It was specified in the claim that the content activity was determined in the ROI and the non-ROI area. This implied that the frame was separated into an ROI and a non-ROI area before the content activity was determined.

The objective technical problem was how to provide a dynamic and adaptive skipping coding of non-ROI areas which provided higher visual quality more pleasing to human perception.

D9 related to providing improved coding efficiency in order to achieve a target bit rate and maintain a stable buffer level. It did not disclose the distinguishing features of claim 1. The claim also specified that the "accumulated distortion due to skipping of encoding of the non-ROI areas" in one or more other frames was considered for the decision of whether to skip encoding or not. This was different from D9, which minimised the overall coding distortion.

The appeal fee should be reimbursed because in the oral proceedings the examining division had not provided arguments why claim 1 was not clear, except that its scope was too broad. This was not a clarity issue. The analysis of the examining division contained errors of judgment going against the basic rules of claim interpretation to such an extent that they precluded a discussion on novelty and inventive step during the oral proceedings before the department of first instance. The scale of the error of judgment also became apparent from the fact that - once the claims were considered to be clear - they were novel with respect to all ten documents cited in the decision under appeal. The misinterpretation of the claims by the examining division had truncated the examination proceedings and led to a procedural stoppage, such that the applicant's right to be heard was violated.

Reasons for the Decision

1. The appeal is admissible.

The invention

2. The application relates to a technique of content-adaptive background skipping which may be useful in video telephony applications, particularly in low bit-rate applications such as mobile video telephony (see abstract).

In order to reduce the data rate of such applications and to improve the subjective image quality, a region of interest (ROI) such as a human face may be preferentially encoded at the expense of the background area of the video scene (non-ROI) by allocating a greater proportion of encoding bits to the ROI than to the non-ROI (see paragraphs [0003] and [0004]).

The present application discloses a method containing steps to allocate bits between the ROI and the non-ROI (see, for instance, paragraph [0025]). For this purpose the content activity in the ROI and in the non-ROI is determined. On the basis of the measured content activities and the accumulated distortion it is decided whether the non-ROI encoding may be skipped such that more bits are available to encode the ROI. Content activity may be determined by measuring the shape deformation or motion in the ROI or the motion or texture complexity in the non-ROI (see paragraphs [0018], [0019], [0031], [0091], [0107] to [0109] and figure 8).

The application also relates to a device corresponding to claim 1 and a computer-readable medium comprising instructions to cause a processor to carry out the steps of claim 1.

Main request, clarity

3. According to Article 84 EPC 1973 the claims must be clear and concise and be supported by the description.

3.1 The examining division held in the decision under appeal that the step of claim 1 "determining whether to skip encoding ... based on content activity of the ROI (90) and the content activity of the non-ROI (92) area and accumulated distortion (94) ..." was unclear.

In claim 1 there was "no feature of the algorithm or of the mathematical function used to obtain the decision" (to skip encoding of non-ROI areas), "leaving the skilled person in doubt even about whether all the parameters" (content activity in the ROI and non-ROI areas and accumulated distortion) "are used in the decision".

The examining division also interpreted the wording "based on" in the determining step as "based on at least one of ..." (see decision under appeal, sections 1, 2 and 4).

3.2 The board is not convinced by this reasoning.

The interpretation of the feature hinges on the expression "based on ... and ... and ...". It is apparent from the context of this expression in claim 1 that the expression defines a logical link between the parameters "content activity of the ROI", "content activity of the non-ROI" and "accumulated distortion". The result of the expression's evaluation is taken as the condition that determines the decision "whether to skip encoding of the non-ROI area". Hence, the expression "based on ... and ... and ..." has to be construed as specifying a logical relation. In logic the word "and" is used to form a conjunction, as opposed to "or", which is used as a sentential connective that forms a disjunction. This implies that the decision "whether to skip encoding of the non-ROI area of the video frame" is based on all three parameters: content activity of the ROI, content activity of the non-ROI area and accumulated distortion due to skipping of encoding of non-ROI areas in one or more other frames.

3.3 The board considers this interpretation to be unambiguous and clear.

Skipping encoding of the non-ROI based on the combination of three parameters is a feature of the algorithm used to obtain the decision on skipping encoding of the non-ROI. Hence, the board cannot agree with the decision under appeal that there was "no feature of the algorithm or of the mathematical function used to obtain the decision". The examining division also did not designate features of the algorithm that it considered essential to render claim 1 clear, and the board sees no necessity to include further features of the algorithm or of the mathematical function in the claim. As argued by the appellant, different embodiments of the invention are conceivable and presented in the application, see paragraphs [0095] to [0111] and figure 8.

3.4 The decision under appeal also referred to claim 1 of the fourth auxiliary request filed on 9 June 2011 as an example of an algorithm not including the dependency on the accumulated distortion (see decision under appeal, Reasons, point 1.1). Claim 1 of that request contains the additional features:

(i) "... further comprising skipping (90) encoding of the non-ROI area if the content activity within the ROI exceeds a first threshold;

(ii) skipping (92) encoding of the non-ROI area if the content activity within the ROI does not exceed the first threshold but the content activity within the non-ROI area does not exceed a second threshold."

These features are based on figure 8 (see steps 90 and 92) and are also present, for example, in claim 13 of the present main request.

The board agrees with the decision under appeal that for some input images with for example a content activity of the ROI area exceeding the first threshold, the content activity in the non-ROI area and the accumulated distortion are not taken into account in deciding on skipping encoding of the non-ROI area. It is, therefore, correct that features (i) and (ii) specify decisions of the algorithm taken independently of the third parameter, "accumulated distortion".

This notwithstanding, the method of claim 1 is to be separated from the input data processed by the method. The method as a whole requires the determining step of whether to skip encoding to be based on all three parameters. For certain input images the determining step may depend on only one or two parameters, and it may even be conceivable that for each image there is always only one parameter considered (each one of the three parameters for one of three kinds of image). As long as the method as a whole is based on all three parameters, the requirement of claim 1 for the method to be based on all three parameters is fulfilled.

Hence, the board considers the objection of lack of clarity raised in the decision under appeal to be unfounded as far as claim 1 is concerned. Moreover, the above considerations also apply to the other independent claims 18 and 35.

3.5 The board sees no further clarity objection. Hence, the set of claims 1 to 51 is clear (Article 84 EPC 1973).

Main request, added subject-matter

4. Compared to claim 1 of the main request underlying the decision under appeal, present claim 1 has been amended to contain the following underlined features (deletions are indicated by strike-through):

"A method comprising:

obtaining (66; 88) a video frame;

encoding (74, 76; 96 - 100) a region of interest (ROI) and a non-ROI within the video frame by allocating bits between the ROI and non ROI areas;

determining content activity in the ROI area; determining content activity in the non-ROI area; and determining (74; 91) whether to skip encoding of [deleted: a] the non-ROI area of the video frame based on content activity of the ROI (90) and the content activity of the non-ROI (92) area and accumulated distortion (94) due to skipping of encoding of non-ROI areas in one or more other frames."

4.1 The wording "by allocating bits between the ROI and non ROI areas" is based on, for instance, paragraphs [0006], [0018] and [0025] of the application as filed. The steps of determining content activity in the ROI and the non-ROI areas are disclosed in paragraphs [0007], [0019] together with [0088] to [0094].

4.2 Three additional dependent claims 15 to 17 were added, these being based on paragraphs [0003], [0022], [0026] to [0038], [0043], [0046] and [0060] to [0062]. Dependent claims 7 to 11 were amended by elimination of the restriction to the rho-domain, which is supported by paragraphs [0039] and [0048]. The further independent claims 18 and 35 and their dependent claims 19 to 34 and 36 to 51 contain amendments corresponding to those of claims 1 to 17. Finally, the amendments made in claims 24, 27, 30, 31, 47, 48 and 49 of the new main request filed at the oral proceedings on 20 October 2017 concerned only textual or grammatical corrections without affecting the content of said claims; in particular, claims 30 and 31 (referring back to independent claim 18) as well as claims 47 and 48 (referring back to independent claim 35) were adapted to the corresponding claims 13 and 14 (referring back to independent claim 1) by adding the missing words "the first" [threshold].

4.3 Hence, the board is satisfied that the requirements of Article 123(2) EPC are fulfilled.

Main request, inventive step

5. The board agrees with the "Further Observations" made by the examining division that D8 may be considered the closest prior art (see decision under appeal, point 5.1).

5.1 D8 discloses a method for image coding in image communications equipment, for instance for video telephony and teleconferencing (see paragraphs [0002] and [0089]). The encoding method comprises a first step of image segmentation which may separate regions containing a human face from its background (see paragraphs [0083] and [0084]). Subsequently, the image segments are classified or "hierarchised" according to their degree of importance, such that for example the segment containing the human face is classified as being of highest importance, whereas the background segments are assigned the lowest degree of importance (see paragraphs [0086] to [0090]). D8 also discloses that a time interval of coding may be set longer for a class having a lower degree of importance. For example, a segment of the highest degree of importance may be transmitted every frame, whereas a signal of the lowest degree of importance may be coded once per several tens of frames (see paragraph [0093]).

5.2 Hence, D8 discloses a method comprising obtaining a video frame, encoding an ROI and a non-ROI within the frame and skipping encoding of the non-ROI in one or more frames.

D8 does not disclose:

(a) allocating bits between the ROI and the non-ROI,

(b) determining content activity in the ROI area,

(c) determining content activity in the non-ROI area,

(d) determining whether to skip encoding of the non-ROI area of the video frame based on content activity of the ROI and the content activity of the non-ROI area and accumulated distortion due to skipping of encoding of non-ROI areas in one or more other frames.

5.3 Claim 1 of the main request differs from claim 1 underlying the decision under appeal by features (a) to (c). Concerning feature (d) the examining division argued in the decision under appeal that D8, paragraph [0129], disclosed a step of determining whether to skip encoding of the non-ROI area of the video frame based on the content activities of the ROI and non-ROI areas.

It is correct that D8 refers to content activity (see paragraph [0103]). In contrast to the present application, the content activity is determined for an image in order to segment and classify the image into an ROI and a non-ROI area, i.e. the content activity is not determined separately in the ROI and non-ROI areas, but in the whole image so as to be able to subsequently segment the image into these areas. The decision whether to skip encoding of the non-ROI is also not based on the content activity in these areas, but on the assignment of coding time intervals to the non-ROI area (see paragraph [0093]), i.e. the non-ROI area is always transmitted at a "certain coding time interval" independently of its content activity.

Paragraph [0129] of D8 addresses the case where a person moves and uncovers background, which is then transmitted and updated in the receiver's memory if it is not already stored therein. The decision whether to skip encoding is based on the determination of whether the background has been transmitted previously and not whether its content activity is high or low. Hence, at least parts of feature (d) are not disclosed in paragraph [0129].

5.4 The technical effect caused by the distinguishing features may be taken as improving the skipping encoding of a video sequence. The improvement concerns the rate-distortion characteristics of coded images.

5.5 The technical problem can therefore be formulated as how to improve the skipping encoding of a video sequence in view of its rate-distortion characteristics.

5.6 D9 discloses the use of accumulated distortion as a quality measure for coding a video sequence. It discloses several rate-distortion models that use a flexible bit allocation for frames and objects, thus balancing the coded frame distortion with the distortion incurred by frames or objects that have been skipped (see page 488, abstract and introduction, and page 495, chapter V.D).

D9 does not disclose features (b) and (c) as well as feature (d) as far as the content activity of the ROI and non-ROI is concerned.

5.7 Hence, the combination of D8 and D9 does not render the claimed subject-matter obvious.

5.8 The further documents on file are no more relevant than D8. They also do not disclose the distinguishing features (a) to (d).

5.9 The board concludes that the subject-matter of claim 1 and the corresponding independent claims 18 and 35 involves an inventive step in view of the documents on file.

Reimbursement of the appeal fee

6. The appellant requested reimbursement of the appeal fee. According to Rule 67, first sentence, EPC 1973 the reimbursement of appeal fees is to be ordered in the event of interlocutory revision or where the board of appeal deems an appeal to be allowable, if such reimbursement is equitable by reason of a substantial procedural violation.

6.1 The appellant argued that the examining division had not provided arguments why claim 1 was not clear, except that its scope was too broad. This was not a clarity issue.

It is established jurisprudence that the broadness of a claim is not to be equated with its clarity (see Case Law of the Boards of Appeal of the European Patent Office, 8th edition 2016, section II.A.3.3). Nevertheless, a broad claim may be understood as lacking essential features or as specifying the claimed subject-matter in vague terms. In the present case the examining division provided reasons for its opinion why claim 1 of the main request underlying the decision under appeal was considered ambiguous (see page 1, third paragraph, of the minutes of the oral proceedings before the examining division and the decision under appeal, Reasons, points 1.1 and 2.1). The board did not follow this reasoning (see section 3 above). Nevertheless, it considers the reasoning of the examining division to be an error of judgment on a substantive issue and not a procedural violation.

6.2 The appellant also argued that the analysis of the examining division contained errors of judgment going against the basic rules of claim interpretation to such an extent that they precluded a discussion on novelty and inventive step during the oral proceedings before the department of first instance. The misinterpretation of the claims by the examining division had truncated the examination proceedings and led to a procedural stoppage.

The board notes that - according to the minutes of the oral proceedings before the examining division - inventive step for claim 1 of auxiliary request 6 was discussed in the oral proceedings (see the minutes following the break from 13:50 to 14:30). More importantly, there is no obligation on the department of first instance to discuss all the issues in oral proceedings which may become relevant if - in the ensuing appeal proceedings - the board is not convinced by the reasoning in the decision under appeal.

6.3 It is not apparent from the file or argued by the appellant that the appellant had no opportunity to present its comments on the grounds or evidence on which the decision is based (Article 113(1) EPC 1973). On the contrary, the appellant's reasoning in the statement of grounds of appeal confirms that arguments were exchanged, but that the appellant's arguments did not convince the examining division that their interpretation of "based on" in the present case was incorrect.

6.4 Hence, the board cannot see that the procedure before the department of first instance was tainted by a substantial procedural violation. As a consequence, the reimbursement of the appeal fee is not equitable in the present case.

Order

For these reasons it is decided that:

1. The decision under appeal is set aside.

2. The case is remitted to the department of first instance with the order to grant a patent with the following claims and a description to be adapted thereto: claims 1 to 51 of the main request, received during the oral proceedings of 20 October 2017.

3. The request for reimbursement of the appeal fee is refused.