T 0840/18 () of 10.2.2021

Summary of Facts and Submissions

I. The appeal was filed by the appellant (patent proprietor) against the decision of the opposition division to revoke the patent in suit (hereinafter "the patent").

II. Oral proceedings before the Board were held on 10 February 2021.

III. The appellant-proprietor requests that the decision under appeal be set aside and that the patent be maintained in amended form according to the main request, filed as auxiliary request 1 at the oral proceedings before the Board.

The respondent-opponent requests that the appeal be dismissed.

IV. The independent claims that play a role in this decision are worded as follows:

Claim 1 of the 2nd auxiliary request (no longer maintained), filed with the appellant-proprietor's grounds of appeal:

"1. A system (158), comprising:

a first camera (158a);

a second camera (158b), the second camera (158b) having a higher resolution than the first camera (158a);

a processor (200) communicatively coupled to the first camera (158a) and the second camera (158b), the processor (200) operable to:

determine a center coordinate (712) of an udder (802) of a dairy livestock (800) based at least in part upon visual data captured by the first camera (158a) from a rear of a dairy cow; and

determine a position of a teat (804) of the dairy livestock (800) based at least in part upon the center coordinate (712) and visual data captured by the second camera (158b) from the rear of the dairy cow; and

a memory (240) communicatively coupled to the processor (200), the memory (240) operable to store a plurality of coordinates comprising an x-coordinate, a y-coordinate, and a z-coordinate for each teat (804) of the dairy livestock (800), wherein each coordinate is in relation to the center of the udder (802) of the dairy livestock (800)."

Claim 1 of the main request, filed at the oral proceedings as auxiliary request 1:

"1. A system (158), comprising:

a first camera (158a);

a second camera (158b), the second camera (158b) having a higher resolution than the first camera (158a);

a processor (200) communicatively coupled to the first camera (158a) and the second camera (158b), the processor (200) operable to:

determine a center coordinate (712) of an udder (802) of a dairy cow (800) based at least in part upon visual data captured by the first camera (158a) from a rear of the dairy cow; and

determine a position of a teat (804) of the dairy cow (800) based at least in part upon the center coordinate (712) and visual data captured by the second camera (158b) from the rear of the dairy cow; and

a memory (240) communicatively coupled to the processor (200), the memory (240) operable to store a plurality of coordinates comprising an x-coordinate, a y-coordinate, and a z-coordinate for each teat (804) of the dairy cow (800), wherein each coordinate is in relation to the center of the udder (802) of the dairy cow (800)."

V. In the present decision, reference is made to the following documents:

D1 : US2008/0257268 A1

D2 : US2007/0215052 A1

D3 : WO01/30134 A1

D5 : WO2010/046669 A1

D6 : WO2005/015985 A2

D8 : White, J "Design of a robotic manipulator for automatic application of milking cups", Master of Engineering thesis, Dublin City University, 25.10.2006

VI. The appellant-proprietor argued that the main request filed during oral proceedings should be admitted into the proceedings. Furthermore, it argued that it did not add subject-matter extending beyond the application as filed and that it involved an inventive step.

VII. The respondent-opponent argued that the main request should not be admitted into the proceedings. That it added subject-matter that extended beyond the application as filed and that its subject-matter lacked an inventive step.

Reasons for the Decision

1. The appeal is admissible.

2. Background

The patent relates to dairy farming, and more particularly to a vision system that can be used by a robotic attacher. Amongst other things, the system determines the centre coordinate of an udder (see published patent specification, paragraphs [0001] and [0004]).

3. Admittance of the main request

3.1 The respondent-opponent argued in its reply to the proprietor's appeal (see page 8) that the amendment to claim 1 of the second auxiliary request as filed with the appellant's appeal grounds, rendered the claim as a whole unclear.

3.2 In particular, the respondent-opponent argued as follows:

"Claim 1 of the second auxiliary request includes the amendment that the visual data captured from the first camera and the visual data captured from the second cameras is captured "from a rear of a dairy cow" and "from the rear of the dairy cow", respectively.

It is submitted that this amendment renders the claim as a whole unclear. The amended text uses the term "dairy cow". However, the claim also refers to "dairy livestock". It is not apparent from the claim whether the dairy cow and dairy livestock relate to the same animal.

In addition, the first portion of amended text that reads, "captured by the first camera from a rear of a dairy cow" is also unclear. By specifying a, or one, rear, it suggests that the cow has more than one rear and the image is captured only from one".

3.3 The Board understood the respondent-opponent's argument as to why the claim was unclear to turn on the use of the indefinite article "a rear" rather than "the rear". The Board did not interpret the claim as possibly meaning that an animal could have more than one rear (cf. third paragraph of the respondent's submissions). Nor was there any formal objection made under Article 84 EPC, which would have made it clear that the clarity objection is independent from the subsequent formal objection under Article 123(2) EPC.

The only comment the respondent-opponent made with regard to the terms dairy livestock and dairy cow was that it was not apparent whether this was the same animal, without saying that this rendered the claim unclear. Bearing in mind that a dairy cow is a specific species of dairy livestock in general the Board saw no inconsistency of terminology arising as a result of both the terms appearing in the claim, as such. All the more so, since the animal itself is not part of the system.

3.4 In other words, the Board did not recognise from the respondent-opponent's original submission that the claim could be directed to a system operable to determine the center coordinate of the udder of a dairy livestock other than a cow having one rear.

3.5 That said, it is regrettable that the Board did not comment in its communication on the respondent's original submissions with regard to these points.

3.6 During the discussion of the clarity of this second auxiliary request at the oral proceedings before the Board, the appellant-proprietor argued that the use of two terms for the dairy animal was inconsistent but that the claim itself was clear.

3.7 The respondent-opponent explained that the claim was unclear because an ambiguity as to whether the dairy cow and dairy livestock mentioned in the claim related to the same animal arose because the term dairy cow was introduced into the claim with an indefinite article "from a rear of a dairy cow" (emphasis added by the Board), and was thus introduced as a separate entity to the previously introduced dairy livestock, rather than merely specifying the dairy livestock to be a dairy cow.

3.8 This detailed explanation of an objection under Article 84 EPC, not in conjunction with "a rear", but in conjunction with "a dairy livestock" and "a dairy cow", was made for the first time at the oral proceedings. Therefore, it was not until the oral proceedings that it became apparent that the respondent-opponent's objection was not merely that two terms were used (dairy livestock and dairy cow) but rather that the latter was preceded by an indefinite article (a dairy cow) and that it was this that made the claim ambiguous as to whether the dairy cow was the dairy livestock.

3.9 During the oral proceedings before the Board, the appellant-proprietor filed a new main request (as auxiliary request 1), based on the original second auxiliary request.

3.10 According to claim 1 of this request, the animal with which the claimed system operates is consistently referred to as [a] dairy cow and, after its first introduction (a dairy cow) is referred to using the definite article (the dairy cow). Thus the claim clearly defines that the system operates in relation to a single animal (a dairy cow). Consequently, the present main request, prima facie, overcomes the objection of lack of clarity (Article 13(1) RPBA 2020).

3.11 Moreover, since the amendment merely removes an ambiguity as to what animal the system operates on without changing any feature of the system, the further issues of added subject-matter and inventive step raised by the respondent-opponent against the system features of the second auxiliary request (irrespective of animal type), remain unchanged for the new main request. Therefore, the amendment, prima facie, does not give rise to further issues, in accordance with Article 13(1) RPBA 2020. That the amendment also solved the clarity problem without raising new ones was also confirmed by the statement of the opponent conceding that the main request was seen as clear.

3.12 The appellant-proprietor argued that, since there was no formal objection made under Article 84 EPC by the respondent-opponent in its reply to the appeal to the second auxiliary request, and the new main request clearly overcame the objections, it should be admitted into the proceedings.

The respondent-opponent argued that, although claim 1 of the new main request was clear, it should not be admitted because it was first filed at the oral proceedings whereas its clarity objection to the second auxiliary request had been on file from the beginning of the appeal proceedings.

3.13 The Board agrees with the appellant-proprietor that no formal objection under Article 84 EPC was made by the respondent-opponent against the original second auxiliary request with its reply to the appeal. Moreover, the reasons why the respondent considered the claim to lack clarity were only explained in detail, and thus only became apparent, for the first time at the oral proceedings.

The Board held that, without a formal objection under Article 84 EPC having previously been on file and the essence of why the respondent considered the claim unclear only becoming apparent at the oral proceedings, it would be unfair not to allow the appellant-proprietor the opportunity to formulate a suitable response to the clarity objection. In view of the above, the Board considered that the circumstances of the present case were exceptional in the sense of Article 13(2) RPBA 2020. Therefore, the Board decided to admit the appellant-proprietor's main request (filed at the oral proceedings as auxiliary request 1) into the proceedings.

4. Main request, added subject-matter

4.1 In the following, reference is made to the application as published (WO2012/149077A2).

4.2 According to established jurisprudence, see Case Law of the Boards of Appeal, 9th edition, 2019 (CLBA), II.E. 1.3.1, any amendment to a European patent can only be made within the limits of what a skilled person would derive directly and unambiguously, using common general knowledge from the whole of these documents as filed, see in particular CLBA, II.E.1.2.1, and in particular G 2/10, reasons 4.3.

4.3 Furthermore, (see CLBA II.E.1.9), it will normally not be allowable to base an amended claim on the extraction of isolated features from a set of features originally disclosed only in combination, e.g. a specific embodiment in the description. Such an amendment results in an "intermediate generalisation".

Extracting a feature from a disclosed combination of features is justified only in the absence of any clearly recognisable functional or structural relationship among the features of the specific combination or if the extracted feature is not inextricably linked with those features.

4.4 Claim 1 is based on claims 1 and 3 as originally filed and furthermore adds the features (as summarised by the Board) that the first and second cameras capture visual data from the rear of the dairy cow.

4.5 The respondent-opponent argued that an intermediate generalisation exists because the claim as amended refers only to capturing visual data from a rear of the dairy cow. Thus, this feature covers all angles from the cow's rear. At the same time, the claim requires that a center coordinate of the udder is determined which, according to the respondent, can only be done by obtaining visual data as for example shown in figure 7 from a position directly to the rear of the cow, that is, aligned with the cow's longitudinal axis. So, the respondent argued, this (directly to the rear) position, which has not been claimed, is structurally and functionally linked to the claim feature of determining the center coordinate of the udder.

In more detail, so the argument goes, determining the centre coordinate of the udder involves defining it to be equidistant between the points where the udder intersects with the right and left leg (page 20, line 28 to page 21, line 9), which can only work when directly to the rear of the cow. Because this aspect has not been claimed, the claim constitutes an inadmissible intermediate generalisation. The Board disagrees.

4.6 The appellant-proprietor argued that the added features (visual data captured from a rear of the dairy cow) are based on the description, in particular the paragraph on page 13, lines 27 to 31.

4.7 This paragraph belongs to the description of the various embodiments. It discloses that the vision system may locate the teats from a position to the rear of the dairy cow without further qualifying that this position should lie on the longitudinal axis of the cow.

Nor does the Board consider this to be implicit from the rest of this paragraph, which merely explains that there may be multiple cameras such as a first and a second camera and there may be a robotic attacher.

Therefore, the Board considers that the paragraph directly and unambiguously discloses capturing visual data with first and second cameras from a rear of a dairy cow in a general sense, without specifying that this must be from a position directly to the rear of the cow.

4.8 Moreover, in the Board's view, the feature does not constitute an inadmissible intermediate generalisation because it is claimed in conjunction with determining the center coordinate of the udder.

4.8.1 The general idea of using as a reference point the center location of the udder (cf. figure 7, point 712) is introduced on page 15, lines 23 to 30 with reference to figure 7, but without explaining any particular orientation of the cameras, let alone that they must be directly to the rear of the cow. It is only said that a reference point may be defined relative to certain features of the dairy cow such as the hind legs and/or the udder and that in certain embodiments it can be the centre point [of the udder] 712. Nor, in the Board's view, would the skilled person see from figure 7 that the cameras must be directly to the rear of the cow, because the figure is merely an example snapshot identifying various portions of a dairy livestock (see page 3, lines 21 to 22), rather than purporting to explain how the cameras are arranged.

4.8.2 Therefore, at its most general, the idea of determining the center coordinate of the udder is not originally disclosed in a structural or functional relationship with capturing visual data from a position directly to the rear of the dairy cow.

4.8.3 As to the more detailed detailed explanations of how the system generates a reference point that may be the center point of the udder, it is likewise not explained that the cameras must be directly to the rear of the cow. At most, the only information provided from which any camera location might be derived (see page 16, lines 20 to 22) appears to be that the first camera should generally depict the rear of the dairy cow.

4.8.4 Furthermore, whilst it might well be that to obtain the center location of the udder by calculating a coordinate that is equidistant from each udder edge (see page 20, line 30 to page 21, line 2), it would be advantageous to capture visual data from a position directly behind the cow (though it is not stated), this aspect of the determination has not been claimed and is only presented there as a possible way, rather than the only way, of determining the coordinates of center of the udder.

4.8.5 For all these reasons, the Board is not convinced by the respondent-opponent's argument that claim 1 constitutes an unallowable intermediate generalisation.

4.9 The respondent-opponent has furthermore argued that, since the claim only specifies capturing visual data [at any angle] from a rear of the dairy cow, it covers the possibility of obtaining visual data that does not actually show the rear of the animal, but rather the animal's flank. According to this argument, subject-matter is added because the application as filed neither discloses this possibility nor describes how the center coordinate can be determined using such image data.

4.10 The argument boils down to the claim covering a non-disclosed hypothetical embodiment in which the cameras could be arranged at the rear of the cow but at such an oblique angle that the invention could not be carried out. This may be so.

However, the question of compliance with Article 123(2) EPC is not one of whether or not a specific hypothetical embodiment falls within the scope of claim 1. Rather, the relevant question is whether an amendment adds new subject-matter, that is, new information.

4.11 In the present case, the embodiments hypothesised by the respondent-opponent showing only the animal's flank are manifestly not originally disclosed and have no relevance for assessing added subject-matter.

Moreover, for the reasons already explained, the claimed subject-matter is directly and unambiguously disclosed in the application as filed.

4.12 For the above reasons, the Board concludes that claim 1 meets the requirements of Article 123(2) EPC.

5. Inventive step starting from D1 with the skilled person's general knowledge, including D8 and/or D5 and/or D3

5.1 The respondent-opponent has argued that, starting from D1 with the skilled person's general knowledge, which includes D8, and possibly also taking into account the teaching of D5 or D3, the subject-matter of claim 1 lacks inventive step. The Board disagrees.

5.2 In the present case D1 (see abstract and figure 1) discloses a system comprising a first camera (second visual detecting means 21 in the words of D1) and a second camera (first visual detecting means 17 in the words of D1).

5.3 It is not in dispute that D1 does not show visual data captured from the rear of the dairy cow. As can be seen in figures 1 and 2, visual data is captured from the side of a dairy cow.

5.4 In the Board's view, as will now be explained, capturing visual data from the side is fundamental to D1's teaching. The summary of the invention (see paragraph [0007]) opens by explaining a problem with the prior art being associated with using a physical sensor to detect the back of a milking animal. It is stated that the sensor has to be provided for the single purpose of measuring the longitudinal position of the milking animal.

5.5 The document then explains the invention in general terms. Amongst other things (see paragraph [0018]), it teaches to provide a second visual detection device (first camera in the patent's terms) that is provided for detecting a position of the milking animal in a longitudinal direction when being located in the milking box.

5.6 Similarly, in paragraph [0025] it is explained that, although first detection of teats may be complex, in subsequent milkings, only a position of the milking animal in the longitudinal direction is needed.

5.7 Likewise, independent device claim 1 defines that the second visual detection means is provided for detecting a position of said milking animal in a longitudinal direction (x), and the independent method claim 14 has a corresponding method step feature.

5.8 The capturing of visual data in the longitudinal direction-x of the cow (that is from the side) is also consistently presented in the detailed description of the embodiments (see for example paragraphs [0036], [0038], [0040], [0042] with figures 1 and 2).

5.8.1 In this respect, whilst it is true that D1's teaching (see paragraph [0042]) is not limited to a system that visually detects the very back of the cow - any other part such as a hind leg can be used instead - this statement has to be read in the context of the first two lines of the paragraph that explains that it concerns establishing the position of the cow in the longitudinal direction (x).

5.9 From the above, the Board considers that establishing the position of the cow in the longitudinal direction (x) is fundamental to D1's teaching.

5.10 According to established jurisprudence, see CLBA, I.D.3.6 with for example T0570/91, reasons 4.4, although a person skilled in the art is completely free in choosing a starting point from which to assess inventive step, afterwards they are bound by that choice. In the present case, this starting point is D1.

5.11 The Board agrees with the respondent-opponent that it belongs to the skilled person's general knowledge to attach milking cups to a cow's udder from the side, rear or underneath a cow (see for example D8, page 53, lines 1 to 4). Attaching from the rear and capturing an image from the rear is certainly known from D5 (see abstract and figure 2), and allegedly also from D3.

5.12 However, in the Board's view this knowledge (milking from the rear and capturing an image from the rear) would not lead the skilled person to modify the system of D1 so that visual data was captured from the rear of the dairy cow. This is because it is fundamental to D1's teaching to establish the cow's position in the longitudinal direction (x), which means viewing it from the side rather than the rear. Therefore, how ever the skilled person might develop or improve D1's system, they would always capture visual data from the side, rather than from the rear of the dairy cow.

5.13 It follows that, starting from D1, the skilled person would not arrive at the subject-matter of claim 1 as a matter of obviousness.

6. Inventive step starting from D5 in combination with D6

6.1 D5 (see abstract and figures 1 and 2) discloses a system used in milking a cow. In more detail (see page 13, lines 1 to 11 with figure 1), the system comprises a first camera 16. It is also not in dispute that the first camera 16 (see figures 1 and 2 and page 9, lines 6 to 11) is located behind the cow and thus captures visual data from a rear of the dairy cow as claimed. The system comprises a processor 18 that receives images from the first camera and is thus communicatively coupled to it.

6.2 In the Board's view, D5 (see page 14, line 9 to page 15, line 28 with figure 3) also discloses that the processor is operable to determine a centre coordinate of an udder as claimed. It scans image data captured by the first camera to find the cow's legs (page 15, lines 14 to 20), once these are found, it scans upwards to find the udder (page 15, lines 22 to 28), including its height. Bearing in mind the general shape of the udder (see figure 3), finding the height of the udder means finding its lowest point, which is in the middle of the udder. Moreover, finding the distance between the camera and the udder means anchoring this point in three dimensional space relative to the first camera. In other words, the center coordinate of the udder is determined based upon the visual data captured by the first camera.

6.3 The object of this calculation is to manipulate a robot arm 14 to move in the same three dimensional space between the legs of the cow and towards the udder so that it is nearby the udder (see page 16, lines 1 to 7).

6.4 D5 (see page 16, lines 9 to 15) also discloses that the system comprises a robot teat disinfection apparatus with teat locating sensor as described in D6 - WO2005/015985. Thus D5 includes the teaching of D6 by reference.

6.5 D6 (see the abstract, page 9, lines 1 to 9 and figures 1 and 2) discloses a vision system 10 that has a camera 12 mounted behind the robot arm. When incorporated into the system of D5, this would be a second camera. Moreover, given the location of D5's robot arm 14 behind the cow (see figure 2), the Board considers it obvious that the skilled person would incorporate D6's camera 10 into D5's system behind the robot arm and thus also behind the cow to capture visual data from the rear of the cow, rather than pointing in the direction shown in D6, figure 2.

The Board also considers it obvious to provide the teat detecting second camera with a higher resolution than the first camera, because the latter only needs to detect the much larger udder.

6.6 D6 does not explain how the second camera 10 is directed towards the cow's udder prior to detecting the teats. It is merely said (see page 6, lines 1 to 3) that the teat locating sensor (which includes the camera 10) should be positioned in the vicinity of the udder. However, it must prepare to look for the teats by pointing somewhere. This can either be where it happens to be pointing or towards a candidate target area.

6.7 In the Board's view, it would be obvious to use the center coordinate of the udder found by the first camera to provide a candidate target area for the second camera, in order to more quickly find the cow's teats. In so doing, the skilled person would arrive at the claim feature of determining [in a subsequent step] a position of a teat of the dairy cow based at least in part upon the center coordinate (found using data from the first camera) and visual data captured by the second camera, as claimed.

6.8 Therefore, in the Board's view, starting from D5/D6 in combination, the question of inventive step hinges on whether the combination discloses or renders obvious the last feature of the claim (in summary: a memory operable to store x, y, z teat coordinates in relation to the center of the udder). The Board answers this in the negative.

6.9 At this juncture it is helpful to consider how this claim feature is to be interpreted.

The respondent-opponent has argued that the feature could simply mean that the [Cartesian] x, y and z coordinates of the teats are stored with respect to a real world origin, and that the coordinates of the centre of the udder are also defined with respect to the same origin. According to this argument, because the relationship between the teats and the centre of the udder can be calculated, each teat coordinate is in relation to the centre of the udder as claimed. The Board disagrees.

6.10 In the Board's view, defining that each stored teat coordinate is in relation to the center of the udder means that the coordinates themselves define their relationship to the center of the udder. Thus, the feature is more specific than merely defining that [by some calculation] teat coordinates could be related to the centre of the udder.

6.11 Moreover, the Board holds that since Cartesian x, y, z spatial coordinates define lengths in three orthogonal directions from an origin, if these coordinates are to be in relation to the centre of the udder, then the origin must be the center of the udder.

6.12 In other words, according to the last feature of the claim, the memory stores coordinates of a given teat that directly specify the distances from the center of the udder to that teat in the x, y and z directions.

6.13 The description confirms this interpretation (see published patent specification, column 24, lines 29 to 32 with figure 6).

6.14 Turning again to D6, the document discloses generating a 3D teat location map (see for example the abstract). The Board agrees with the respondent-opponent that this implies storing teat coordinates using Cartesian x, y, z coordinates, otherwise they could not be used to direct the robot arm (see page 11, lines 20 to 23 and figure 3, last step).

6.15 Although D6 does not specify the origin of these coordinates, it does explain in relation to an embodiment that uses two cameras (see page 14, line 15 to page 15, last line with figure 8, in particular the paragraph bridging pages 14 and 15) that the cameras are at known positions with respect to a determined world coordinate system. Therefore, the Board holds that it is implicit that the udder coordinates are expressed relative to a Cartesian coordinate origin that is fixed in the real world.

6.16 Moreover (see page 10, line 30 to page 11, line 5), the maps only contain information on the position of the teats, not any surrounding detail. This, D6 explains, reduces the amount of data to be manipulated and stored. This means that information on the rest of the udder, including its center coordinate, is not part of the teat map.

6.17 In the Board's view, what ever obvious improvements and modifications the skilled person might make to the D5/D6 combination, they would not express and store the coordinates in relation to the centre of the udder.

This is because D6 teaches (see page 16, lines 5 to 7) that a principle advantage of its invention is to locate teats in real time, and another advantage is that the teats can be tracked (see page 16, lines 16 to 19). Such real-time tracking - enabling the robot arm to quickly apply a teat disinfecting unit for example - can happen because the teat coordinates are stored relative to a fixed origin in the real world within which the robot arm moves.

This real time tracking would appear to at least be much more complex if the coordinates were stored in relation to a part of the animal that could move (the centre of the udder).

Moreover, expressing the coordinates in relation to the centre of the udder would require not only determining the position of the centre of the udder but also calculating the position of the teats in relation to this center position. Since D6 expressly teaches, in relation to the teat map, only to detect the coordinates of the teats in order to reduce the amount of data to be manipulated and stored, to then store the teat coordinates relative to the centre coordinate of the udder would run contrary to D6's teaching, even considering that D5's system derives such a position (see again paragraph bridging pages 15 and 16) for the purpose of moving the robot arm to be near the udder. Therefore, the Board holds that from the combined disclosure of D5/D6 it would not be obvious for the skilled person to arrive at the last feature of claim 1 (storing teat coordinates in relation to the centre of the udder).

7. From the above, it follows that the subject-matter of claim 1 is not obvious in the light of the cited prior art and thus involves an inventive step.

8. No further objections were raised or are apparent against the claims according to the main request.

8.1 The Board concludes that, for the reasons explained above, the claims meet the requirements of the EPC. But for the necessary adaptation of the description to bring it in line with the new definition of the invention in claim 1, the patent can be maintained with claims according to the main request pursuant to Article 101(3)(a) EPC.

Order

For these reasons it is decided that:

1. The decision under appeal is set aside.

2. The case is remitted to the Opposition Division with the order to maintain the patent in an amended form on the basis of claims 1-3 of the main request filed as Auxiliary Request 1 during the oral proceedings before the Board and a description to be adapted to these claims.