CLR I D 9.1.6.A Mental activities based on data visualisation

The jurisprudence of the boards of appeal provides little support for the idea that screen representations inherently address technical problems (T 1143/06, T 95/86). A feature which relates to the manner in which cognitive content is conveyed to the user on a screen normally does not contribute to a technical solution to a technical problem. An exception would be if the manner of presentation can be shown to have a credible technical effect (T 1143/06, T 1575/07, T 1741/08 and T 1562/11). Features aimed exclusively at improvements regarding the way information is perceived or processed by the human mind are regarded as non-technical (see e.g. T 1567/05, T 125/04, T 579/11). A GUI layout (Graphical User Interface) as such is considered to be non-technical, being a "presentation of information" (Art. 52(2)(d) EPC ( T 1741/08 citing T 1143/06) (see also T 1214/09).

Features concerning the graphic design of user interfaces do not have a technical effect, because their design is not based on technical considerations, but on general intellectual considerations as to which design is particularly appealing to a user (Guidelines G‑II, 3.7.1, November 2015 version). For example, the colour (see T 1567/05, T 726/07, T 1734/11), shape (T 677/09), size (T 823/07, T 1237/07), layout (T 756/06, T 1741/08, T 1214/09), arrangement of items on the screen (T 643/00, T 1237/10) or the information content of a message displayed is usually not a technical aspect of a graphical user interface. An exception would be if these features contribute to achieving a particular technical effect (T 1741/08, T 1143/06).

In T 244/00 the board stated that the graphic design of menus was, as a rule, not a technical aspect of a menu-driven control system. Nor was the practical use of such menus genuinely a problem with which the skilled person, in his function as a technical expert, was confronted. For the purpose of the problem and solution approach, the problem had to be a technical problem which the skilled person in the particular technical field might have been asked to solve at the priority date. The board therefore concluded that, in the case at issue, the technical problem had to be formulated in a more limited way than on the basis of the alleged advantages of moving a cursor diagonally over the TV screen (see T 154/04, OJ 2008, 46; T 125/04, T 1143/06).

In T 1237/10 the board pointed out that the displayed arrangement of images as defined by claim 1, including its temporal changes (a calendar-type layout), constituted a presentation of information, which is excluded "as such" from patentability under Art. 52(2)(d) EPC. Such a presentation of information contributes to inventive step only to the extent to which it interacts with the technical subject-matter of the claim for solving a technical problem (see T 154/04, OJ 2008, 46, and T 1214/09). The claimed presentation of images was aimed at showing the images to the user essentially for informational purposes and not, for example, at enabling a new mechanism for inputting an image selection.

The board considered that, in the context of the claimed invention, the idea of sequentially replacing the images shown in a display region by other images classified into the same time slot was not technical. Dealing with limited available space is part and parcel of the design of presentations of information for human viewing and is therefore per se not an indication of technicality (cf. decision T 1562/11). Although the issues of image size and resolution might play a role in the idea's implementation, the idea itself was independent of such issues. The objective technical problem solved by the claimed selection means was therefore that of implementing this idea. The board came to the conclusion that the skilled person would have trivially implemented it by providing suitable software "selection means" for sequentially replacing the images shown in each display region. The skilled person would hence have arrived at the subject-matter of claim 1 without the exercise of inventive activity. Thus the subject-matter of claim 1 lacked inventive step (Art. 52(1) and 56 EPC).

In T 643/00, the underlying GUI was configured, upon a user instruction, to simultaneously display hierarchically encoded image data in a side-by-side manner at a low resolution in order to make the search process easier for the user. Although human activities were involved in solving the problem, which activities might relate to the information contents of images looked for and might be motivated by personal interests and/or other non-technical preferences, the board considered that the claimed features relating to the format of images displayed were not presentations of information as such. The board noted that it was true that non-technical aspects may be found in the design and the use of an interface through which the user interacts with a system (see decision T 244/00). Indeed, presenting information through a user interface, if the only relevant effect of the presentation relates to the visually attractive nature of the graphic design or artwork, does not have technical character. However, in its decision the board had not excluded the possibility that an arrangement of menu items (or images) on a screen might be determined by technical considerations. Such considerations might be intended to enable the user to manage a technical task, such as searching and retrieving images stored in an image processing apparatus, in a more efficient or faster manner, even if an evaluation by the user on a mental level was involved. Although such evaluation per se did not fall within the meaning of "invention" pursuant to Art. 52 EPC 1973, the mere fact that mental activities were involved did not necessarily qualify subject-matter as non-technical, since any technical solutions in the end were intended to provide tools which served, assisted or replaced human activities of different kinds, including mental ones. The board referred to T 1177/97 where it was stated that the use of a piece of information in a technical system, or its usability for this purpose, may confer a technical character on the information itself in that it reflects the properties of the technical system, for instance by being specifically formatted or processed. Additionally, the board referred to T 1194/97 where it was stated that functional data (line numbers, coded picture lines, addresses and synchronisations) recorded on a record carrier to be used in a picture retrieval system were to be distinguished from the cognitive content encoded. Even if the overall information could be interpreted in an infinite number of different ways in other technical or human contexts, this did not detract from its technical function in the relevant context of the claimed invention (see also T 424/03). Finally, the prior art lacked any clear hint of the claimed combination of image processing functions, so that the presence of inventive step as required by Art. 56 EPC had to be acknowledged.

In T 336/14 claim 1 of the invention was concerned with a user interface for an extracorporeal blood treatment machine. The parties quoted a large number of decisions regarding the assessment of inventive step, in particular the matter of presentations of information as such with respect to graphical user interfaces. The board, after summarising those decisions, stated that in the assessment of inventive step of a claim which comprises technical and non-technical features ("mixed invention") and in which the non-technical features relate to cognitive content presented to the user of a graphical user interface (GUI), i.e. relate to "what" is presented rather than "how" something is presented, it has to be analysed whether the GUI together with the content presented credibly assists the user in performing a technical task (related to "why" that content is presented) by means of a continued and/or guided human-machine interaction process (see point 1.2.4 of the Reasons).

The board noted that it was immediately apparent that the information presented according to features of claim 1, i.e. the operating instructions and the corresponding pictographs, were cognitive rather than functional data in the sense of T 1194/97, since they directly address the user of the blood treatment machine and were consequently meaningful only to a human mind. Further, the distinguishing features were related to the content of the information, i.e. to "what" was presented, rather than to the manner in which the information was presented, i.e. to "how". Next, the question was whether the underlying user interface and the content presented credibly assisted the user in performing a technical task by means of a continued and guided human-machine interaction process. So, this question was basically related to "why" (i.e. "for what purpose") the content was presented. In other words, it had to be established whether the information presented constituted "technical information", which credibly enabled the user to properly operate the underlying technical system and thus had a technical effect, or rather "non-technical information", which was exclusively aimed at the mental activities of the system user as the final addressee.

The board noted that not everything that supports a technical task has itself a technical character (cf. T 1741/08). Rather, the information of relevant features was evidently not related at all to any internal system state concerning the proper functioning of the underlying machine in the sense of T 115/85, let alone to a desirable or valid state within the meaning of T 362/90 or T 887/92. Thus, in accordance with T 1143/06, making reference to T 619/98, an action (possibly) performed by a user in response to a message concerning the technical functioning of an apparatus did not necessarily render technical the information conveyed (T 1143/06). The board came to the conclusion that the information provided according to the relevant features could not credibly support a continued and guided human-machine interaction process. Thus, it could not assist the user in performing the technical task. It only addressed the human mental process of an "average user", however the latter was supposed to be defined based on personal skills and preferences (see e.g. T 407/11). This had to be considered a non-technical effect.

In T 125/04 the board stated that, in general, the task of designing diagrams is non-technical. This is so even if the diagrams arguably convey information in way which a viewer may intuitively regard as particularly appealing, lucid or logical. In T 125/04 the invention, like T 643/00 above, also concerned an arrangement of images but was different in that only the information conveyed by the images, ie their "cognitive content" (see T 1194/97, OJ 2000, 525), was relevant. The new features had to do with how this content was represented. Unlike T 643/00, the invention provided no information about the computer system itself, such as the location where the data are stored. According to decision T 641/00 (OJ 2003, 352), features of a claim making no contribution to the technical character of an invention cannot support the presence of an inventive step. Implementing the claimed steps by merely providing means for carrying out these steps is obvious. Accordingly, the board could not find that the steps performed by the features in claim 1 had any technical effects which went beyond those obtained by the normal use of a computer. It followed that the subject-matter of claim 1 lacked an inventive step.

In T 49/04 the application related to a method and an apparatus for enhancing the presentation of a text in a natural language on a (computer) display. The board followed T 643/00 rather than T 125/04 and stated that technical aspects could not be ruled out in the design and use of a graphic interface. Furthermore, the presentation of natural language text on a display in a manner which improved readability, enabling users to perform their task more efficiently, related to how, i.e. by what physical arrangement of the text, cognitive content was conveyed to the reader and could thus be considered as contributing a technical solution to a technical problem. Therefore, the board concluded that the claimed features, viewed as a whole, did not relate to a non-invention listed in Art. 52(2) EPC 1973 as such.

In T 1214/09 the appellant had argued that the arrangement of thumbnail file images defined by feature (i) contributed to improved evaluation of a large number of thumbnail file images by the user and thereby solved the problem of how to enable more efficient image retrieval. The appellant referred to decisions T 49/04 and T 1749/06. In decision T 49/04, a particular way of dividing text segments based on linguistic considerations was considered to produce a technical effect in that it improved the readability of text on a display. The deciding board did not consider it relevant that the improvement in readability was essentially of a cognitive nature. In decision T 1749/06, providing an icon with a three-dimensional appearance by modifying its edge with alternate dark and light stripes was found to be a technical effect. The deciding board was of the view that the test "happens in the brain of the viewer", which had been invoked by the examining division, was not useful for deciding whether a feature contributed to the technical character of a claim. The board recognised that both of these decisions could be understood as challenging the notion that improving a human's cognitive evaluation of certain information by changing the manner in which the information is presented is non-technical. The board, however, noted that the alleged effect of feature (i), i.e. the improved evaluation of thumbnail file images by the user, was due solely to the claimed arrangement of thumbnail file images. In the board's judgment, this arrangement was not based on considerations other than those proper to the field of designing presentations of information for human viewing and is hence not an expression of any technical principle. The board therefore considered the alleged effect not to be a technical effect. The board held that, since in the context of this invention any improvement in the efficiency of image retrieval can only be the result of the non-technical improvement in the user's evaluation of the displayed thumbnail file images, feature (i) did not contribute to a technical solution of the problem of enabling more efficient image retrieval (see T 1143/06 and T 1741/08). The board therefore followed the approach taken in decision T 1143/06 and found that feature (i) at most contributed to the technical character of the invention through the details of its implementation.

In T 1235/07 the invention used a tree diagram to view and navigate through the dimensions and levels of data in a multidimensional database. The board took a wider view of "presentation of information" than just the actual information that is displayed, the so-called cognitive content, to include also structural aspects of how the information is displayed. This broader meaning was supported by almost the only reference to the presentation of information in the travaux préparatoires. In the board's view, such additional aspects can only contribute to inventive step if they have technical character. The board noted that there was jurisprudence in this field holding that similar presentations of information are not technical. In particular, T 1143/06 concluded that representing, by the speed of an element moving on a display, the relevance of data in a database to sort statements had no technical effect. The case was similar in that the information was presented to the user in a way that made it easier to evaluate and the user could then respond by selecting and displaying data. The board noted that the information was known per se, but only differed in the visual form in which it was displayed; i.e. speed of a moving element instead of in tables. In the board's view, the case before it was even less convincing because the invention was merely a visual display of a known analysis technique using known means, whereas in T 1143/06 the idea of varying the speed of the element was at least not known. The board found that the solution was the implementation of user choices using known techniques that would be matters of routine design. Accordingly, the clamed invention did not involve an inventive step.

In T 1741/08 the application was concerned with a method of entering data into a data processing system. The question was whether a technical effect could be attributed to a particular layout of a Graphical User Interface (GUI). It was undisputed that according to the established case law of the boards of appeal, the subject-matter of a claim cannot be inventive if there is no additional technical effect over and above any technical effects present in the closest prior art. The appellant argued that the required technical effect was to be seen in the reduction of computer resources resulting from the layout of icons which made it easier, particularly for an inexperienced user, to identify the stage reached in a process of data input requiring a number of steps and sub-steps. The board made clear that "lowering the cognitive burden of the user" is not of itself a technical effect (T 1143/06 followed, T 49/04 not followed). It found that the reduction in use of resources would be caused by the way the brain of the user perceives and processes the visual information given by a particular way of presenting information. The appellant also argued that there was a chain of effects. But in terms of technical effects this was a broken chain. The board did not accept that such a broken chain could be used as evidence of the required technical effect overall. Following T 1143/06, the board considered a GUI layout as such to be non-technical, being a "presentation of information" (Art. 52(2)(d) EPC). In the case at issue, the arrangement of icons displayed was intended to convey information, namely at which step in the input process the user found him- or herself. The board stated that the mere fact that a particular choice of information to display or of how to display it is particularly clear, lucid, or "lowers the cognitive burden" of the user was not sufficient to demonstrate that the choice has a technical effect (see also T 306/10). The case law of the boards of appeal was entirely consistent with this principle. The fact that not all GUI-related applications are considered to be inventive does not mean that there is a contradiction between the decisions. As to the one possible exception, T 49/04, a single case deviating from the general case law was not a sufficient reason for a referral to the Enlarged Board of Appeal (see G 3/08, OJ 2011, 10, point 10.12 of the Reasons). The cases T 643/00, T 928/03 and T 333/95 which had been cited by the appellant were different from the case under consideration in that there was something other than the simple choice of what information to display and with what layout to display it, so that in these exceptional cases the displayed information might play a part in the assessment of inventive step of the claimed invention. The board made it clear that not everything that supports a technical task itself has a technical character.

T 333/95 concerned using a computer to produce animations. In order to produce a scene in which a particular object moves, the selected object takes the place of the cursor, so that the user can move it around with the mouse as desired, while the system records the movements of the mouse and translates them into a script of commands for moving the object in the later display of the animated scene. However, the board in that case clearly considered the feature of "making said graphics object the current cursor" as a technical feature per se. This "graphics object cursor" replaced the normal cursor (also technical) and its movements were recorded and translated into a kind of programming language. There was at least a technical effect in allowing the user to move the selected object around under control of the mouse. The decision did not say that every GUI design that makes user's inputting more efficient has a technical character (see T 1741/08).

In T 1073/06, the underlying GUI was configured, upon user input, to display objects of a simulation model, including graphical link representations to improve the ease of a user's comprehension of the model. The contribution of the claimed subject-matter to the prior art was related to the utilisation of association data stored in the memory to cause the link between the objects in the simulation model to be displayed with the associated graphical link representation. The deciding board held that "an improvement in the comprehension of a model is a purely mental effect, so that the problem solved was not seen as being technical. The claimed 'graphical link representations' related to the state of the simulation model, rather than to the state of the claimed simulation apparatus, and thus constituted presentations of information and are therefore also non-technical" (see T 336/14).

In T 1834/10 the contribution provided by the application had non-technical and technical aspects: a web page is made more attractive to potential customers by presenting images in a lively manner, and the images to be presented are selected and displayed automatically so that the presentation can be changed rapidly. The board noted that a web page designer may consider various approaches of selective content presentation as promising with respect to attracting customers' attention. However innovative an attractive content selection may be, it related to a presentation of information which was a priori non-technical (Art. 52(2)(d) EPC), even if it lowers a user's cognitive burden (T 1741/08) or prompts the user to start some technical action (broken technical chain, T 1741/08, T 1670/07). The desire to present a selection of images, to limit the selection to a predetermined number of images and to present the selected images in some preferred layout is a non-technical aim. The meaning of the display data does not convey any technical character to the presentation. Effects resulting from an image presentation depend on the user's perception and/or constitute indirect technical effects and/or relate to administrative and economic aspects. The board noted that the only aspect that can enter into the examination for an inventive step is a technical implementation of the desired image presentation. However, program elements (tags, widgets) for transferring parameters from a markup language document to a page-generating program (e.g. browser) were well-known and were only used by the application in suit for their conventional purpose. The board had general doubts about whether a piece of software serving a non-technical purpose (presentation of information) can be considered as a technical implementation; such software rather constitutes a computer program as such (Art. 52(2)(c) EPC) as it fails to provide any further technical effect beyond the elementary interaction between software and hardware (T 1173/97, OJ 1999, 609). The combination of two types of non-inventions (presentation of information, computer program) is not enough to define a technical contribution (T 1755/10, "software implementation fallacy").

In T 677/09 the appellant argued that in the context of vehicle information systems and the user action of actuating the component, the information provided about differences between vehicles went beyond the mere provision of the information because it resulted in enhanced safety, which was technical. The board considered that such an effect would depend on the content of the information and the user’s reaction to it. This effect was thus not the direct effect of the feature and could not be used to formulate the technical problem. The board noted that the appellant invoked non-technical aspects as a reason for not modifying the prior art and that in T 1670/07, this kind of argument was referred to as a "non-technical prejudice fallacy" (point 16 of the Reasons). The fact that this was not possible followed from the fact that the non-technical features relating to the information content could not contribute towards an inventive step at all. The question was not whether the skilled person would consider providing these features because that had already been decided in formulating the technical problem, but whether it would be obvious to implement the features in the claimed manner. The appellant pointed out that in data processing a signal generally has cognitive information content, but according to the jurisprudence nevertheless has technical character. However, the board considerd that this technical character was due to the so called "functional data" implied by the signal, which inherently comprises technical features that interact with those of the system in which the signal is operating, such as synchronising data (see for example T 1194/97, point 3.3 of the Reasons). In the case at issue, there were no such inherent technical features of the difference information so that in the board's view it did not have a functional part, but remained purely cognitive.

The appellant considered that such an approach would rule out patents for all types of advanced driver assistance systems. The board did not share this concern as it is easy to imagine systems with features that might have a direct technical effect, such as giving information about the status of the engine, or about an imminent collision, or how to park the vehicle. The board noted that the information did not specifically relate to any technical condition of the vehicle, but simply differences between "features of a component", the effect of which depended on the information and covered any number of non-technical possibilities, such as the colour or the shape of the component. Accordingly the board held that the claim in question did not involve an inventive step.

In T 1567/05 the examining division argued that computing the largest stress value was obvious since this value would determine whether a building structural component was likely to fail or not, and the use of colours to represent ranges of values was well-known to the skilled person. In the board's view, however, there was no need to assess the degree of originality of these features since they have no technical effect. Art. 52(2) EPC includes "presentations of information" in the list of subject-matters that shall not be regarded as inventions. As was noted in decision T 154/04 (point 8 of the Reasons), this list covers subject-matters whose common feature is a substantial lack of technical character. That this is true for presentations of information was observed in decision T 119/88 (OJ 1990, 395), which states in point 4.2 of the Reasons that the classification of objects by colour represents a non-technical effect. Thus, the indication of "strength levels" in the form of predetermined display colours - which is a classification - has no technical effect and cannot contribute to an inventive step. Nor does the choice of information to be presented have a technical effect. Art. 52(2)(d) EPC does not distinguish between different kinds of information. Therefore the presentation (as such) of any information must be regarded as a "non-invention". It is thus irrelevant that the present invention is arranged to display the largest stress value for each building structural component (rather than some other stress value). Although relating to technical phenomena, the stress values are mere pieces of information aimed exclusively at the human mind. Further, the problem indicated in the description was not of a technical character because it depends on the skills and preferences of the particular group of users for whom the invention is intended. (see also e.g. T 579/11).

In T 1734/11 the board stated that a reduction of user interaction does not necessarily convey technical character to the means for achieving the reduction (following T 1741/08). For example, a travel agent who acts as an intermediary between providers of travel services and a prospective traveller also has the effect of reducing the interaction between the end consumer and the providers of those services. The travel agent collects and processes information, and presents the results to the customer in a suitable form. This is, in the board's view, an administrative process which can be performed without technical means and without the need of technical skill. Inputting information to a machine and reducing the burden of doing so, while these may be technical tasks a priori (at least in the sense that they are not listed in Art. 52(2) EPC), not everything that supports a technical task has itself a technical character. On the other hand, presenting (i.e. outputting) information was deemed to be non-technical a priori (Art. 52(2) (d) EPC). However, in contrast to the presentation of the state of a technical machine, which may have technical character (T 115/85, OJ 1990, 30), the board considered that the presentation of pricing information, for example by colour coding, no matter how skilful that might be, is a non-technical aspect even if it helps the user to conduct a price-sensitive travel query more efficiently.

In T 726/07 the feature in question was the use of a colour to represent a state of the cache. The board stated that even assuming that this could be considered as representing "conditions prevailing in an apparatus" in the sense of T 115/85 (OJ 1990, 30), the board considered that the use of a colour was a common and obvious implementation of a status indication.

47 references found.

Click X to load a reference inside the current page, click on the title to open in a new page.

EPC Articles

EPO Guidelines - G Patentability

Case Law Book: I Patentability

Case Law of the Enlarged Board

General Case Law