T 1137/16 () of 12.2.2021

Summary of Facts and Submissions

I. The appeal is against the decision of the Examining Division refusing European patent application No. 07 018 320 on the following grounds:

(a) the main request comprised subject-matter extending beyond the content of the application as filed (Article 123(2) EPC) and lacked clarity (Article 84 EPC);

(b) auxiliary request 1 lacked clarity (Article 84 EPC) and did not involve an inventive step within the meaning of Article 56 EPC; and

(c) auxiliary request 2 lacked clarity (Article 84 EPC) and comprised subject-matter extending beyond the content of the application as filed (Article 123(2) EPC).

II. The appellant requested in writing that the decision under appeal be set aside and that a patent be granted on the basis of the main request or auxiliary request 1, both filed with the statement of grounds of appeal, or on the basis of auxiliary request 2 filed on 25 September 2015.

III. The following documents are referred to:

D5: |Floyd and Steinberg: An Adaptive Algorithm for Spatial Greyscale; Proceedings Of The Society Of Information Display, volume 17, number 2, Second Quarter 1976, pages 75-77|

D6: |CS 559: Computer Graphics Floyd-Steinberg Dithering, University of Wisconsin-Madison, retrieved from the internet at https://research.cs.wisc.edu/graphics/Courses/559-s2004/docs/floyd-steinberg.pdf|

IV. Claim 1 of the main request reads as follows:

"An apparatus for improving qualities of motion and still images to be output in a mobile communication terminal, the apparatus comprising:

a display (206) configured to display an applied image in a color representation scheme based on a predefined number of bits;

an error diffuser (220) configured to:

sequentially select pixels constructing an original image in a predetermined processing sequence when the original image is input;

compute quantization errors for color channels by quantizing a currently selected pixel;

select neighboring pixels around the currently selected pixel and determine error-reflected ratios based on positions of the neighboring pixels;

determine error-reflected values of the neighboring pixels according to computed quantization errors and error-reflected ratios of the neighboring pixels;

accumulate the determined error-reflected values to pixel-by-pixel error values of the neighboring pixels;

generate an error-diffused image by adding the quantization values in a current color representation scheme for the pixels of the original image to pixel-by-pixel error values related to the pixels of the original image when quantization errors of all the pixels of the original image are completely computed; and

a controller (200,600) configured to input a selected original image to the error diffuser (220) when a user selects the original image to be output and to control the display (206) to display the error-diffused image in place of the original image when the error-diffused image is generated;

wherein the error diffuser (220) comprises an error memory (304) configured to determine whether pixel-by-pixel error values related to at least one neighboring pixel are pre-stored until error-reflected values of all pixels of the neighboring pixels are completely determined, storing currently determined error-reflected values as pixel-by-pixel error values when pixel-by-pixel error values are not pre-stored, and storing pixel-by-pixel error values computed by adding pre-stored pixel-by-pixel error values to determined error-reflected values related to the current quantized pixel when the pixel-by-pixel error values are pre-stored,

wherein the error diffuser (220) comprises an error calculator configured to determine the error-reflected values of the at least one neighboring pixel around a current quantized pixel using an error-reflected-value table storing error-reflected values pre-computed according to quantization errors and the error-related ratios related to positions of neighboring pixels."

V. In preparation for oral proceedings, the Board issued a communication under Article 15(1) RPBA 2020. The appellant was informed that, in addition to the objections raised by the Examining Division, the question whether the application met the requirements of Article 83 EPC would be discussed at oral proceedings, and the Board explained why it was of the provisional opinion that the invention was inadequately disclosed. For the main request, problems also appeared to remain in relation to the requirements of Articles 123(2), 84 and 56 EPC. The auxiliary requests appeared to give rise to many clarity problems, but, to the extent to which they could be understood, they did not appear to overcome the Board's objections in relation to the requirements of Article 83 EPC.

VI. The appellant's arguments in the statement of grounds of appeal, insofar as they are relevant to the present decision, may be summarised as follows:

The independent claims of the main request, at least by virtue of the amendments made, had a basis in the application as originally filed and were clear. They also defined novel and inventive subject-matter over the cited prior art.

The auxiliary requests also defined subject-matter which had a basis in the application as originally filed, and was clear, novel and inventive.

VII. In response to the communication of the Board, the appellant submitted a further letter dated 23 November 2020. The Board's doubts regarding sufficiency of disclosure and other substantive matters were not addressed; the letter simply stated the following:

"With regard to the above-identified appeal case T1137/16-3.4.03, the Applicant recently instructed us not to attend the oral hearing scheduled for December 3, 2020."

As a result, the Board cancelled the oral proceedings.

Reasons for the Decision

1. The appeal is admissible.

2. In both the notice of appeal and the statement of grounds of appeal, the appellant made a conditional request for oral proceedings. The subsequent statement that the appellant would not be represented at oral proceedings (see above, point VII) is normally treated as equivalent to a withdrawal of the request for oral proceedings. The Board decided to cancel the oral proceedings and proceeded to issue the present decision based on the written procedure (see Case Law of the Boards of Appeal, 9th edition 2019, III.C.4.3.2).

3. The Scope of Examination in the Present Appeal

3.1 In the contested decision the claimed subject-matter was rejected for failing to meet the requirements of Articles 123(2) and 84 EPC (main request and auxiliary request 2) or Articles 84 and 56 EPC (auxiliary request 1). No reference was made in the "Reasons of the decision" to the requirements of Article 83 EPC, nor, according to the "Summary of Facts and Submissions", was this matter raised in the written procedure.

3.2 The Board, nevertheless, has the power to examine the application for compliance with this, or any other provision of the EPC, as confirmed by the Enlarged Board of Appeal in G 10/93, the Headnote of which reads as follows:

"In an appeal from a decision of an examining division in which a European patent application was refused the board of appeal has the power to examine whether the application or the invention to which it relates meets the requirements of the EPC. The same is true for requirements which the examining division did not take into consideration in the examination proceedings or which it regarded as having been met. If there is reason to believe that such a requirement has not been met, the board shall include this ground in the proceedings."

3.3 Moreover, an examination for compliance with the requirements of Article 83 EPC 1973 does not violate the appellant's rights under Article 113(1) EPC 1973. The Board's concerns in relation to the requirements of Article 83 EPC 1973 were set out at some length in the communication under Article 15(1) RPBA 2020, dated 12 June 2020. Since the date for oral proceedings had already been fixed for 3 December 2020, the appellant had ample opportunity to present its comments on Board's objections either in writing or at the oral proceedings, the cancellation of which was due to the appellant's stated intention not to attend.

4. The Nature of the Claimed Invention

4.1 Claim 1 defines an "apparatus for improving qualities of motion and still images to be output in a mobile communication terminal" and comprises (in summary) a display configured to display a colour image based on a predefined number of bits, an error diffuser and a controller. The error diffuser comprises an error memory and an error calculator, and is configured to sequentially quantize a currently selected pixel, compute the quantization errors, and process the quantization errors according to a scheme which will be examined in detail below.

4.2 The claimed subject-matter encompasses both digital to digital quantization and analog to digital quantization. In this regard, the Board notes that the following is stated on page 7, lines 18-21:

"It is assumed herein that quantization does not indicate a process for converting analog data into digital data, but indicates a process for converting a pixel represented by a quantization value of a larger number of bits into a quantization value of a smaller number of bits."

However, this statement of what is "assumed herein", appearing in the section "DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS", has no limiting effect on the invention as defined by claim 1, which places no such limitation on the nature of the input image, and therefore encompasses embodiments in which the error diffuser is configured to carry out digital to digital processing (e.g. 24 bit to 16 bit), and also embodiments in which the error diffuser is configured to carry out analog to digital processing.

5. The Background to the Present Invention

5.1 Before analysing the claimed invention in detail, the Board firstly offers the following summary of conventional error diffusion according to the prior art.

5.2 Error diffusion is a technique first introduced in a paper by Floyd and Steinberg in 1976 (document D5, see point III, above). The particular algorithm of Floyd and Steinberg is now generally referred to as "Floyd-Steinberg Dithering" (hereinafter "FSD").

Although D5, for simplicity, describes quantization to a binary image (two levels: dark = zero, bright = 1), it is also applicable to multi-level outputs (D5, page 76, left-hand column, last paragraph). Again, for simplicity, the (analog) input in D5 ("desired brightness") is normalized to values between 0 and 1; where a desired brightness in the range of zero to 0.5 results in a dark (0) binary output, and a desired brightness above 0.5 results in a bright (1) binary output.

The image is quantized (binarized) pixel-by-pixel, and the fundamental idea of conventional error diffusion as set out in D5 is that, for any given pixel (apart from the first), the input for the quantization is not just the brightness data for the corresponding input pixel, but this input brightness plus weighted quantization errors from neighbouring pixels, according to the scheme shown in Fig. 1 of D5 (and Fig. 1 of the present application).

5.3 FSD may be illustrated by considering the application of the algorithm to a uniform greyscale image, where every pixel has a desired brightness of 0.5. Such an application is mentioned in D5 (page 76, left-hand column, lines 6-8), and is dealt with more fully in D6. The Board is not asserting that D6 forms part of the state of the art (no date appears to be available), it is cited merely to elucidate the nature of FSD, a technique which has been known since the 1970s.

5.4 Applying binarization without error diffusion would result in every pixel being rendered dark (each pixel has an input of 0.5 which maps to dark). Hence, the resulting binary output would be a uniformly dark image, which would clearly be a poor representation of half tone grey.

Using FSD, the first pixel, P(1,1), having a desired brightness of 0.5, would be rendered dark (quantized brightness = 0) by quantization, and the quantization error (quantized brightness minus desired brightness) for P(1,1) would be -0.5 (i.e. 0 - 0.5). This pixel has a negative error (it has been rendered too dark, i.e. rounded down), and so it would be desirable for a positive error to be added to neighbouring pixels to compensate by rounding up (see D5, page 75, right-hand column, first paragraph). Hence, according to the scheme, a weighted quantization error of 7/16 * (+0.5) = 0.219 is propagated to pixel P(1,2).

The desired brightness for pixel P(1,2) is therefore modified to 0.719 (i.e. 0.5 + 0.219), which, after quantisation, results in a bright (1) binary output and a quantization error of 0.281 (i.e. 1 - 0.719). As this is positive, a negative weighted quantization error of 7/16 * (-0.281) = -0.123 is propagated to pixel P(1,3).

For pixel P(1,3), the desired brightness is modified to 0.377 (i.e. 0.5 - 0.123), which, after quantisation, would result in a dark (0) binary output.

The drawings on the final page of D6 demonstrate how the algorithm would continue, the final result being a chequerboard pattern of dark and bright pixels. The averaging property of the human eye would result in this pixel chequerboard being perceived as a good representation of the original 50% grey image.

5.5 In more complex images the quantization errors are not necessarily opposite in sign for each neighbouring pixel; however, the fact that pixels accumulate errors from four neighbouring pixels (disregarding edge effects) makes it more likely that quantization will result in a pixel being rounded downwards if a number of neighbouring pixels have been rounded upwards, and vice versa, thus reducing the average quantization error.

6. Main Request: Article 83 EPC 1973

6.1 Claim 1 defines an "apparatus for improving qualities of motion and still images to be output in a mobile communication terminal". This improvement is said to be achieved by the error diffusion process provided by the claimed error diffuser (see e.g. page 8, lines 10 to 12 and 24 to 28). Hence, the technical effect of the invention (improved image quality) is stated in claim 1.

6.2 In Decision G 1/03 the Enlarged Board of Appeal stated the following (See Reasons, point 2.5.2.):

"a lack of reproducibility of the claimed invention ... may become relevant under the requirements of inventive step or sufficiency of disclosure. If an effect is expressed in a claim, there is lack of sufficient disclosure. Otherwise, ie if the effect is not expressed in a claim but is part of the problem to be solved, there is a problem of inventive step (T 939/92, OJ EPO 1996, 309)."

This passage has since been cited in, for example, T 1079/08 (Reasons, point 4) and T 2001/12 (Reasons, point 3.3).

6.3 To comply with the requirements of Article 83 EPC 1973, the application must disclose the invention in a manner that would enable a skilled person to achieve the claimed improvement in the quality of the quantized image. Clearly, this condition could not be met unless it is credible that the disclosed invention is actually capable of providing the claimed improvement, and the Board therefore turns to this question.

6.4 The error diffuser of claim 1 is configured to perform a series of processing steps on the data of an original input image. Several of these steps are the same as (or at least include embodiments which would correspond to) conventional FSD.

6.5 Thus, a currently selected pixel is quantized and the quantization error is computed; neighboring pixels around the currently selected pixel are selected, and error-reflected ratios based on positions of the neighboring pixels are determined (the selected neighboring pixels and error-reflected ratios shown in Fig. 1 of the application correspond precisely to those of FSD).

The "error reflected values" are then determined for these neighbouring pixels according to the RGB quantization errors for the selected pixel and the error-reflected ratios of the neighbouring pixels. The term "according to" includes embodiments in which the error reflected values are determined by multiplying the RGB quantization errors for the selected pixel by the error-reflected ratio of a given neighbouring pixel, exactly as in FSD.

The next step in claim 1 is "accumulate the determined error-reflected values for pixel-by-pixel error values of the neighboring pixels corresponding to the determined error-reflected values", which appears to mean that for each pixel, the error-reflected values from four error-diffusing neighbouring pixels are accumulated (step 410 in Fig. 4). Again this corresponds to conventional FSD.

6.6 The error diffuser is then configured to move on to the next selected pixel. In claim 1 this is expressed in the phrase "sequentially select pixels", but more details are given in Fig. 4. According to step 412 there is a check whether "error values" (which appears to mean the same as "error-reflected values") have been fully computed for each pixel. If this is not the case, the next pixel to be quantized is selected in step 418, and the error diffuser returns to step 404, where the RGB quantization errors for the newly selected pixel are calculated "by quantizing a currently selected pixel", according to the wording of claim 1. The input for this quantization can only be the original image data, which has not, according to the disclosed scheme, been modified at this point by any error diffusion. This represents a first difference over conventional FSD.

6.7 The next claimed feature is as follows:

"generate an error-diffused image by adding the quantization values in a current color representation scheme for the pixels of the original image to pixel-by-pixel error values related to the pixels of the original image when quantization errors of all the pixels of the original image are completely computed".

Hence, the step of adding the error values to the quantization values takes place only "when quantization errors of all the pixels of the original image are completely computed". The quantization values to which the error values are added are simply those obtained "by quantizing a currently selected pixel". This is confirmed in step 414 of Fig. 4, according to which pixel-by-pixel error values are added to color-by-color quantization values of pixels of the original image. This represents a second difference over conventional FSD.

6.8 As a result of these two differences, the claimed error diffusion scheme is radically different to conventional error diffusion.

In conventional FSD, the weighted errors from a selected pixel are added to the neighbouring pixels prior to quantization of the neighbouring pixels, in order to influence the quantization result.

According to the claimed error diffusion scheme, each pixel is quantized according to the input from the original image only; the weighted errors (the "error-reflected values") are determined and stored (accumulated) for each pixel, but they are not used until quantization of the entire image is complete. Only then are the error-reflected values added to the quantization values of the respective pixels.

6.9 The question whether the invention meets the requirements of Article 83 EPC 1973 therefore boils down to asking whether it is credible that the disclosed error diffusion scheme would, in practice, achieve the claimed improvement in image quality.

6.10 The application comprises examples of how to carry out the error diffusion of the invention, at least in the case of conversion of 24 bit digital data to 16 bit digital data (e.g. in table 1). However, the Board sees nothing in the application, which would make it credible that this type of error diffusion would actually lead to the claimed improvement in image quality, for example, test results or comparative images.

6.11 For the requirements of Article 83 EPC 1973 to be met, the application must disclose the invention in a manner which allows it to be performed (including the achievement of any claimed technical effect) in the whole range claimed (see Case Law of the Boards of Appeal, 9th Edition, 2019, II.C.5.4). In the present case, far from providing evidence that the technical effect is reproducible over the whole range claimed, the application fails to provide any indication that the alleged improvement in image quality was checked and verified even for a single embodiment of the invention.

The Board does not believe that subsequently submitted evidence could have remedied this deficiency (see G 1/03, Reasons, point 2.5.3), but in any event, no such evidence of the alleged improvement in image quality has been submitted in the appeal procedure.

6.12 In the simple case of binarization of a uniform greyscale image (see above, points 5.3 and 5.4) it is possible to determine the effect of the claimed scheme on image quality. For this case, it will be recalled that applying binarization without error diffusion would result in every pixel being rendered dark, which would clearly be a poor representation, while applying FSD would give a chequerboard pattern which would be a much better representation of the original 50% grey image.

6.13 Applying the method of the invention, a general pixel, P(n,m), in row n and column m, having a desired brightness of 0.5, would be rendered dark (quantized brightness = 0) with a quantization error of -0.5. The error reflected values of the neighbouring pixels would then be determined based on the quantization error of P(n,m) and the error-reflected ratios. For example, following the scheme of Fig. 1, the neighbouring pixels would be P(n,m+1), P(n+1,m-1), P(n+1,m) and P(n+1,m+1), with error-reflected ratios 7/16, 3/16, 5/16 and 1/16, respectively. These error reflected values would be accumulated but not (yet) added to anything.

Pixel P(n,m+1) would then be quantized on the basis of its original desired brightness only, and so it would also be rendered dark (quantized brightness = 0) with a quantization error of -0.5, and so the error reflected values of the four neighbouring pixels would be the same as those of the neighbouring pixels of P(n,m). Again, these error reflected values would be accumulated but not (yet) added to anything.

The end result of this procedure would be a totally dark quantized image, with each pixel having (discounting any possible edge effects) the same accumulated quantization error values.

6.14 According to claim 1, once every pixel has been quantized, and all error values determined, an image is generated by adding "the computed pixel-by-pixel error values" to the quantization values for the respective pixels. Since the accumulated error (reflected) values for all pixels would be the same, the same thing would be added to the quantization value of every pixel. Hence, either all pixels would remain dark, or all pixels would flip to bright. Either way, there would be no improvement on what would be achieved by a simple binarization without taking error values into account, and a considerable deterioration compared to FSD.

6.15 This example of uniform greyscale to binary is not chosen at random. In Fig. 1 of the original paper of Floyd and Steinberg (D5) the selected pixel is denoted by P and the neighbouring pixels are A, B, C and D, each receiving 7/16, 1/16, 5/16 and 3/16 of the error respectively. The following is stated on page 76, left-hand column, lines 6-8:

"These parameters were found mostly by trial and error, guided by the desire to have a region of desired density .5 come out as a checkerboard pattern."

Thus, from the earliest days of error diffusion, converting a uniform 0.5 density greyscale image to a checkerboard binary pattern was seen as a basic requirement of (analog to digital) error diffusion schemes, a requirement which the scheme defined in claim 1 of the main request fails to meet.

6.16 For the reasons given above, the Board judges that it has not been credibly demonstrated in the application that the apparatus of claim 1 would actually achieve the claimed improved quality of motion and still images. Since obtaining the said improved image quality is a feature of the claimed invention, the Board concludes that the application does not disclose the invention in a manner sufficiently clear and complete for it to be carried out by a person skilled in the art. Hence, for the invention defined in claim 1 of the main request, the application does not meet the requirements of Article 83 EPC 1973.

7. Auxiliary Requests

7.1 In both auxiliary request 1 and auxiliary request 2, claim 1 defines an "apparatus for improving qualities of motion and still images ...". For the reasons given above for the main request, the Board judges that it has not been credibly demonstrated that any apparatus capable of achieving the claimed improvement has been disclosed in the application.

7.2 Hence, for the inventions defined in claim 1 of auxiliary request 1 and in claim 1 of auxiliary request 2, the application does not meet the requirements of Article 83 EPC 1973.

Order

For these reasons it is decided that:

The appeal is dismissed