Information Relevance

Saracevic (1996) defines relevance as “a very basic human cognitive notion in frequent, if not even constant, use by our minds when interacting within and without in cases when there is a matter at hand” (p.3). Maron (1977) discussed relevance through the notion of aboutness, relating it to the meaning making and interpretation process: “Perhaps we should say that about can be interpreted from several different points of view and that each different viewpoint gives rise to a different interpretation of the meaning of about” (Maron, p.40). This notion of multiple interpretations of the meanings of how things are related has produced numerous viewpoints on what relevance is and how can one go about determining if two things are relevant to each other. This challenge has been addressed by the scholars and researchers in the field of information science in response to the need to determine whether a certain document is related to user’s information need, request or a query: “We all are able to think and understand and know what some piece of writing is about, yet we can’t say exactly what is going on and, certainly, we cannot prescribe to another how he or she ought to do it” (Maron, p.40). By stating that “we cannot prescribe to another how he or she ought to do it,” Maron has raised and identified one of the most important and the most fundamental question pertinent to the act of being able to say when two things are about each other, i.e. are related to each other. This is the very question that IR research has addresses for a long time, suggesting and developing various IR models and techniques, based on different senses of how aboutness and relevance is perceived in different contexts.

The complex nature of relevance and the evolution (as well as branching) of the concept over time is clearly presented by Mizzaro (1997). He divides the history of relevance “into three periods (‘Before 1958,’ ’1959-1976,’ ’1977-present’) and, inside each period, the papers on relevance are analyzed under seven different aspects (methodological foundations, different kinds of relevance, beyond-topical criteria adopted by users, modes for expression of relevance judgment, dynamic nature of relevance, types of document representation, and agreement among different judges)” (p.810). Similarly to Saracevic (1996), Mizzaro defines relevance in terms of interaction and “relation between two entities of two groups. In the first group, we have one of the following three entities: (i) Document… (ii) Surrogate… (iii) Information [as a (re)source]. … In the second group we have on of the following four entities: (i) Problem … (ii) Information need… (iii) Request… (iv) Query… Now, a relevance can be seen as relation between two entities, one from each group” (p.811). Further, Mizzaro presents another dimension pertinent to relevance that is the questions of aboutness in relevance. Is the about in relation to a topic/theme, a task, or a context (Mizzaro, p. 811)? Considering the necessity to search for information and the potential resources that can satisfy the necessity, and its aboutness, Mizzaro suggests that “each relevance can be seen as a point in a fourth-dimensional space, the values of each of the four dimensions being: (i) Surrogate, document, information; (ii) query, request, information need, problem; (iii) topic, task, context, and each combination of them; and (iv) the various time instants from the arising of the problem until its solution” (Mizzaro, p, 812). The dimension of aboutness (task, topic, and context) is rather incomplete in a sense that aboutness in relation to time could have been included, in addition to including time as the fourth dimension. The difference between time as a fourth dimension and time related to aboutness, is that time aboutness would give us relevance related to the passage of time. For example, a document might be less relevant today in a certain organizational context compared to the earlier relevance it might have had, resulting from the fact that other documents appearing latter have superceded it; something like the induced difference in relevance judgments as a result of two points in time, and the additional difference in relevance when these two points in time are moved together to another time. This could be considered different than the fourth dimension where the relation between the need for information, the resource to satisfy the need and its aboutness all three change in the way they are related at different points in time. One could argue however that the time aboutness is part of the context. Nevertheless, I think time aboutness should be treated separately as is the task, the topics, and the context.

A critical juncture in the discourse on relevance is the relevance judgment. Which and whose relevance judgment is more ‘relevant’ to the issue at hand? Mizzaro defines relevance judgment as “an assignment of a value of relevance (now we know that it is more correct to say ‘a value of a relevance’) by a judge at a certain point of time” (p. 812). In the attempt to elaborate on this question, Maron distinguishes between objective, subjective and retrieval about: the “… S-about (for subjective about) is a relationship between a document and the resulting inner experience of its readers”, “The interpretation of this concept of about [the O-about, for objective about] is obtained by considering and external or observer’s point of view, as opposed to the internal or the subjective point of view”, “This third concept [R-about, for retrieval about], like O-about, is objective and behavioral. However, unlike O-about, it does not refer to the behavior of a single individual, but rather to the information searching behavior of a class of individuals; viz., the class of those patrons who constitute the population of any given library or retrieval system” (Maron, p.41). The retrieval about (R-about) is most directly related to aiding IR system designs where serving a broad base of users is very important. For the purposes of IR systems design, it should be noted that the design functionality satisfying the R-about should be not very restrictive as close off the functionality needed by S-about (user-centric), and at the same time should not be to open ended to encompass enormous functionality that could potentially be introduced by the O-about (sort of system-centric), i.e. the S-about and O-about should play the balancing roles to enable building a system that will accommodate wide user base but also help each user to individually benefit from the system.

Saracevic (1996) identified four major pertinent issues that need to be addressed by researchers in elaboration of relevance: its nature (what is the appropriate framework within which relevance can be defined), manifestation (the ways and contexts in which relevance manifests itself), behavior (what is the variability in observable contextual human behavior), and effect (how to utilize relevance in theory, experiment and practice) (p.2), with four frameworks on nature of relevance have emerging over time: systems, communication, situational and psychological (p.1). The limitations of the systems centric and the user centric frameworks are that they are blind to each other: “Systems framework is user blind, and situational framework is system blind” (p.7). To overcome these limitations, starting with Ingwersen’s Cognitive Model and Belkin’s Episode Model, Saracevic suggests and develops the Stratified Model of IR interaction (p.9). This model involves the user (physiological, psychological, cognitive processes) on one side and the computer (physical, symbolic, and algorithmic) on the other. The user and the computer interact at the surface level/stratus through an interface, then the user further interfaces with the information resources on a cognitive, situational, and affective level, as well as the content, processing and engineering on the computer side (p.9). Then, Saracevic proposes a fifth, the interactive framework to study relevance as a better fit to accommodate the interactive nature of IR: “based on stratified model of information retrieval (IR) interaction, where interactions are viewed as involving levels or strata. It is suggested that there is not only one relevance a play, but that exist an interdependent system of relevances, dynamically interacting within and between different strata or levels, with adaptation as necessary” (p.1). This is a very profound suggestion with the aim to present an easier way to design, develop, and evaluate IR systems. The difficulty however is actually in the ability to decompose the relevance into its separate relevances and assigning measures for experimentation and operationalisation attempts, give that relevance has been traditionally used to asses the effectives of IR systems (Saracevic, p.1).

So far I have attempted to show the complexity of the frameworks within which relevance has been studied. It is evident that there has been evolution over time. However, even within this framework it is not easy to study relevance. What are the attributes of relevance that matter? Based on the intuitive understanding of relevance as well as the philosophical, and cognitive, and communicative explication of relevance, Saracevic suggests the following attributes of relevance: relation, intention, context, inference, and interaction (p.5). These attributes of relevance manifest themselves at various levels. In relation to the stratified model, Saracevic distinguishes between the following manifestations of relevances: system or algorithmic relevance, topical or subject relevance, cognitive relevance or pertinence, situational relevance or utility, motivational, or affective relevance (p.12).

The multiplicity of relevance attributes and multiplicity of relevance manifestation suggests it is very simplistic to refer to relevance in singular. Even when the word relevance is used in singular probably refers to set of relevances related to each other in some meaningful way, unless it is specifically referred to a particular manifestation of relevance. How are the various relevances related to each other? At what level are they related to each other? Cosijn and Ingwersen “examine one of the possible ways to model relevance manifestation (as defined by Saracevic) within a system of relevance attributes” (p.534). They describe “manifestation of relevance within a system of relevance attributes to show that attributes function in different dimensions for the different manifestations of relevance” (p.533). Further “the manifestations of relevance are plotted against attributes of relevance that show that attributes function in different dimension for the different manifestation of relevance” (p.534) to show that “relevances are moving … [order: relation, intention, context, inference, interaction] from a systems approach to a user-and socially-oriented approach” (p.536). It is not surprising that relevances are meaningfully related to the systems-centric approach and the user-centric approach. The nature of relevance has evolved over time as the problems related to relevance has been recast in new form and level of importance and as information systems have become more pervasive in our daily lives. The pervasiveness of various information systems is perhaps partially responsible for bringing forth aspects of the user-centric approach in the design of information systems.

Conclusion

At its basics, relevance is about matching the pertinent thing to an information need. It is established and evaluated by matching the representation of texts and representation of information need (Saracevic, p.6). As the basic and one of the most important concepts in information retrieval (IR) systems, relevance has evolved and in many ways led the evolution in the research, design, and development of IR systems. The concept of relevance has evolved from the system centric approach and into the more user centric approach floating in the discourse of the cognitive viewpoint of information science.

The intuitive understanding of relevance and aboutness has been rather recognized to be very complex, as Mizzaro and Saracevic have shown. In the system centric approach, “Relevance is considered to be a property of the system – it depends on how the system acquires, represents, organizes and matches texts, or in other words on the internal manipulation of the system” (Saracevic, p.6). With the move towards the cognitive viewpoint, research has elaborated on the various relevance attributes and the various manifestations relevance exhibits itself. So, which relevance should IR systems designers, developers and researchers deal with? The paradoxical answer is the relevant relevance at the appropriate level/dimension of manifestation. Based on the intuitive understanding of relevance, Saracevic derives that: “as a cognitive notion relevance involves an interactive, dynamic establishment of a relation by inference, with intentions toward a context” (Saracevic, p.5)

From the above it is evident that context matters. Relevance cannot be addresses without a context especially in relation to the interactive IR systems with the user(s) as the central element affecting multiple manifestation of relevance: “Relevance is a dynamic phenomena: For the same judge, a document may be relevant at a certain point of time and not relevant later, or vice versa” (Mizzaro, p. 814).

If there are multiple manifestations of relevance, is it feasible to identify relevance, a composite one, which perhaps can give us an insight into the relevance as it pertains to a particular situation and task? A challenge like this would perhaps require understanding the relation among the various manifestations of relevance. [194:610]

References:

Cosijn, E. & Ingwersen, P. (2001). Dimensions of relevance. Information Processing & Management, 36 (4), 533-550

Maron, M.E. (1977). On indexing, retrieval and the meaning of about. Journal of the American Society for Information Science, 28 (1), 38-43

Mizzaro, S. (1997). Relevance: the whole history. Journal of the American Society for Information Science, 48 (9), 810-832

Saracevic, T. (1996). Relevance reconsidered. Information science: Integration in perspectives. Proceedings of the Second Conference on Conceptions of Library and Information Science. Copenhagen (Denmark), 201-218