ISIS-Tutor: An Intelligent Learning Environment for CDS/ISIS Users

Peter Brusilovsky

Dept. of Cognitive Psychology, University of Trier, D-54286 Trier, Germany Tel: 49-651-201-391 Email address

Leonid Pesin

School of Applied Mathematics and Cybernetics, Moscow State University, Moscow, Russia Email address

Table of contents

Integrating a hypermedia component into ILE: the knowledge based approach
ISIS-Tutor: The domain and the architecture
ISIS-Tutor components
Summary of the knowledge-based approach
Student model centered architecture for the integration of system components
Adaptive navigation support as a kind of adaptive hypermedia


Abstract: The paper discusses the prospects of building educational systems which integrate the capabilities of hypermedia, learning environments, and intelligent tutoring systems. We present the ISIS-Tutor system, which provides an example of integrating hypermedia technology into an intelligent learning environment. ISIS-Tutor is intended for learning the print formatting language of an information retrieval system, CDS/ISIS.


Many developers of educational systems consider Intelligent Tutoring Systems (ITS) and learning environments as different and even contradictory ways of using computers in education. The recent success of such well-known Intelligent Learning Environments (ILE) Smithtown (Shute & Glaser, 1990) and Sherlock (Lajoie & Lesgold, 1989) showed that these ways are not contradictory, but rather complimentary. ITSs are able to control learning adaptively on various levels, but generally do not provide tools to support free exploration. Learning environments and microworlds support exploratory learning, but they lack the control of an intelligent tutor. Without such control the student often works inefficiently and may never discover important features of the subject.

The same situation exists now with ITSs and educational hypermedia systems. They are often considered as two different approaches to using computers in education, but these approaches are in fact complimentary. Recent research has demonstrated that hypermedia can provide the basis for an exploratory learning system but that, by itself, such a system is insufficient, needing to be supplemented by more directed guidance (Hammond, 1989). The guidance can be provided by an intelligent tutoring component. By comparison, hypermedia can add new dimensions to traditional ITS/ILE by providing a tool for student-driven acquisition of domain knowledge.

We think that in many domains it is possible to achieve a good result by developing an educational computer system which integrates the capabilities of an intelligent tutor, a learning environment, and a hypermedia system. Such an integrated system can support the learning of both procedural, and declarative knowledge and provide both system-controlled and student-controlled styles of learning (Tab.1). The latteris essentially important because recent studies reported that optimal balance between student-controlled and system-controlled strategies depends on the student's abilities and knowledge level. Only an integrated system can support the whole continuum of learning styles ranging from an unstructured instructional environment to a guided curriculum.

Table 1. Knowledge, styles of learning and supporting systems

Procedural knowledge Declarative knowledge
TeachingITS (coach)ITS (tutor)
LearningLearning environmentsHypermedia systems

Our research at the Moscow State University are centered around two problems of creating integrated ILE: the problem of adaptation and the problem of integration. As for adaptation, the problem is to make all the components of an integrated ILE adaptive. Most of ITS and tutoring components of ILE can adapt its work (tutoring) to the given student, however very few environment and manual components can do that. It was one of our goals to build adaptive environment and manual components of ILE. As for integration, our position is that an integrated system should be not just a sum but a real integration of its components. In particular, it requires the continuity of student work in an integrated ILE. The results of students' work with any of the components during the session should be taken into account by other components to adapt their performance to the changed knowledge level and current interest of the particular student.

As a solution for the above two problems we designed a knowledge based approach and a simple student model centered architecture for building integrated ILE. By this approach all the components of ILE including the environment use the same central knowledge base (which include central student model, a traditional part of ITS) to coordinate its behavior and to adapt it to the given student. Specific structure of the knowledge representation and and specific techniques for work with it forms the specifics of the approach. Since 1985 we applied the approach and the student model centered architecture in several ILE designed by our group for different domains. We use these ILE to investigate various aspects and problems of integrated ILE. During the last few years we extended our approach by integrating hypermedia technology into our existing ILEs.

This paper describes briefly our knowledge based approach to building a hypermedia-based intelligent learning environment (Brusilovsky, Pesin & Zyryanov, 1993) and then presents the ISIS-Tutor system, which is an intelligent learning environment intended for learning the print formatting language of an information retrieval system CDS/ISIS/M. From our point of view, ISIS-Tutor provides an interesting example of integrating a hypermedia technology into an intelligent learning environment. Separately we discuss some interesting features of ISIS-Tutor: the use of student model centered architecture and adaptive navigation support.

Integrating a hypermedia component into ILE: the knowledge based approach

Our approach to building integrated ILE is based on the ideas from the domains of intelligent tutoring systems and adaptive interfaces. This approach was first implemented in 1985 in the ITEM/IP system (Brusilovsky 1992b) which integrates an ITS for programming, a programming environment, and adaptive on-line help facilities on the basis of represented knowledge about the domain and the student. The following kinds of knowledge are represented. The domain model in ITEM/IP is a net with nodes corresponding to programming concepts and mini-language constructs (domain knowledge elements), and with links reflecting several kinds of relations between nodes. The overlay student model reflects by a set of integer counters the extent to which the student has mastered the concept or the construct. The student model is always kept up-to-date and supports adaptive work of all modules. All the teaching material is stored in the knowledge base of teaching operations in the form of frames interlinked with the domain model. These frames are used to generate several kinds of teaching operations (presentation, example, problem, etc).

ITEM/IP uses the represented knowledge to support adaptive guidance and adaptive presentation. At any moment of work the student can ask the system for the next "best" teaching operation. The strategy module applies an embedded tutoring strategy and uses the domain knowledge and the student model to select such a teaching operation. A student who is not satisfied with the optimal operation suggested by the system, can choose any relevant teaching operation from menus. The presentation module uses the represented knowlege about the domain and the student to provide adaptive presentation of the selected teaching operation. Briefly speaking, the presentation of teaching material is adapted to the current level of student knowledge, so a repeated explanation is usually more concise and complete than the explanation presented when a concept is initially introduced.

The ITEM/IP approach was later generalised and applied sucessfully in a number of other domains. However we were not completely satisfied with the menu-based interface for student-driven learning. In our most recent works we have been trying to extend our approach by integrating the ITS and hypermedia technologies. The problem here is not just to design a hypermedia component to support student-driven learning for one of our existing ILEs, but to find the ways to integrate this component into a particular ILE. It means that the hypermedia component must both use the student model to adapt its performance to the given student, and update the student model to reflect the results of the student's work with the component.

We found that the knowledge based approach is very suitable for integration of intelligent tutoring and hypermedia technologies. We need only "two steps" to make a hypermedia network really integrated in our ILE. The first step is to take the domain model network of an ILE as a basis for the hypermedia network. Actually, different ideas about creating the hypermedia network from the domain concept network are quite popular and have been discussed elsewhere (Hendley, Whittington & Jurascheck, 1993). What we suggest is to design the main part of the hypermedia network just as the visualized (and externalized) domain network. Each node of the domain network should be represented by a node of the hyperspace, while the links between domain network nodes constitute main paths between hyperspace nodes. Thus the core structure of the overall hyperspace resembles the pedagogic structure of the domain knowledge. In addition to that, each teaching operation is also represented as a node of the hyperspace and interlinked with all domain concepts listed in its spectrum. It provides natural interface for student-driven learning.

The second step concerns the content of hypermedia pages. We decided not to duplicate the learning material in the form of static hypermedia pages, but to have the hypermedia pages generated from the material stored in the knowledge base of learning material. Thus, each concept and each teaching operation will have a hypermedia page (or block of pages) as an external representation, and have a frame as an internal representation. What the student will see on the screen visiting a hypermedia node is really generated from the corresponding frame by a special program which can take into account the student's current state of knowledge. This approach not only saves page design time but also provides space for adaptation.

The knowledge-based approach enables bi-directional communication between the heart of the system - the student model and the hypermedia component. First, any student's 'visits' to a particular node of the network can be reflected in the corresponding compone nt of the overlay student model (which is based on the same domain network). Second, the current state of the student model for the given node of the network and its links can be used by the hypermedia component to adapt the screen layout of the corresponding hyper-node, as well as the number and status of visible links to other nodes. More considerations about integration of hypermedia into an intelligent learning environment, including a review of related works, can be found in (Brusilovsky, Pesin & Zyryanov 1993; Brusilovsky, 1994).

The following section explains our way of building hypermedia-based ILE on the example of ISIS-Tutor system.

ISIS-Tutor: The domain and the architecture

ISIS-Tutor is an intelligent learning environment to support learning the print formatting language of the well known information retrieval system CDS/ISIS/M for IBM PCs (ISIS for short). This system is supplied by UNESCO and used widely in Russia and in many information centres in the world. The print formatting language is key to many CDS/ISIS operations and mastering the language is important for effective use of the system. In some ways, it is a kind of programming language. To display or print the result of a search, or the content of a database, an ISIS user has to write a sequence of print formatting commands, really, a more-or-less complex program in print formatting language. This format program is used by ISIS produce an external presentation of the record when displaying or printing it. To print the selected records of a database in the specified format ISIS applies the print formatting program to every record being printed. Print formatting commands, for example, can type a field of the current record or a part of field, can manipulate the current output position, type a constant character string, and so on. Print formats are also used in indexing and sorting. There are over 50 different commands and modifiers in the print formatting language, so a tutoring system for the language is really helpful.

The ISIS-Tutor system is designed in International Centre for Scientific and Technical Information (ICSTI) and Moscow State University. ISIS-Tutor is written in embedded ISIS-Pascal and uses the power of ISIS for knowledge bases storage and access. Earlier versions of ISIS-Tutor was described in (Pesin & Brusilovsky, 1992; Brusilovsky & Pesin, 1994) In this paper we present the most recent version of ISIS-Tutor which provides an example of integrating hypermedia technology into an intelligent learning environment (Figure 1).

Figure 1

Figure 1. Top-level menu of ISIS-Tutor

ISIS-Tutor architecture resembles the original architecture of ITEM/IP (Brusilovsky 1992b) in many ways: it also contains an environment to experiment with the language, and a tutor component. It uses similar domain and student models. However the domain model, which is a network of 69 concepts and constructs, is twice as complex than this one in ITEM/IP. The overall 'space' of teaching material is bigger as well. Unlike ITEM/IP, ISIS-Tutor contains the hypermedia component. Main components of ISIS-Tutor are the following:

The integration of guided tutoring mode (provided by the tutor) with free exploration mode (provided by the hypermedia and the environment) is the most interesting feature of ISIS-Tutor. The following section describes ISIS-Tutor components in more details.

ISIS-Tutor components

The central part of ISIS-Tutor architecture is the interrelated domain model and student model. The domain model representes the material being taught (knowledge about language). The material is divided into a set of elementary concepts, which represents various features of the language, and structured as a directed graph (concept map) representing prerequisite relationships between concepts. The overlay student model (SM) reflects the extent to which the student has mastered language concepts by providing an integer counter for each concept. All learning material is also interlinked with the domain model network. There are three kinds of teaching operations in ISIS-Tutor: concept presentations, problems to solve and examples to analyze. The teaching operations (or learning tasks from student's point of view) are stored in the knowledge base of learning material. Each teaching operation is represented as frame. One of its slots contains the list of domain concepts related with the task; for example, the list of concepts required to solve the problem. This list (called the spectrum) provides the link with the domain model.

Each step of the learning process in ISIS-Tutor is an application of one of two main teaching operations: selection and solving of a problem, or selection and presentation of a new concept. The choice can be made by the system or by the user. In the first case, the system uses the student model to select the optimal teaching operation. This mode gives an adaptive sequence of tasks and concepts. In the second case the user selects the next teaching operation using the hypermedia component. This mode can be useful for an experienced student or for a teacher, who wants to set a learning task manually for the student (Figure 1 and Figure 2). Important is that the student can always request a repeated presentation of a learned concept or a solved problem. This feature is very useful, essentially for a weak student, who has not mastered the material very well.

Figure 2

Figure 2. Topic index in ISIS-Tutor (only first 44 topics is shown). Link annotation with different colors is used for adaptive navigation support. Hiding is disabled.
Bottom line - annotation keys: +(green) Well learned, (green) Learned, (red) Ready to be learned, (no annotation) Not ready
Line above the bottom - menu options: Enter - learn, F4 - practice, F6 - teach me, F8 - problem index, F9 - back, PgDn - next index page

In any case the selected teaching operation is a concept presentation (Figure 3 and Figure 4) or a problem solution (Figure 5). The example presentation is at present not an independent teaching operation (as it was in ITEM IP). It follows a concept presentation and can be requested from the concept presentation window (Figure 3).

Figure 3

Figure 3. Topic presentation page for topic 8 "Field Output". A menu of related examples is shown in the center. Each example is a link to the exploratory environment (see Figure 6).

The learning process is managed by the intelligent tutor. Using SM and an embedded strategy, the tutor can select an optimal teaching operation - new concept presentation or problem, depending on the current state of user knowledge. The tutor usually tries to offer a problem for mastering a concept which is known but not mastered enough by solving problems. If there are no relevant problems, then a new concept for study is selected. After a teaching operation is completed the tutor updates SM. Initially SM is empty. While the user works with the system the model is updated in parallel with the growth of the user's knowledge.

Figure 4

Figure 4. Topic presentation page for topic 5 "New Line". A menu of related topics is shown in the center. Each menu line is a link to the presentation of this topic. Link annotation with different colors is used for adaptive navigation support. Hiding is disabled.
Bottom line - annotation keys (same as on Figure 2): +(green) Well learned, (green) Learned, (red) Ready to be learned, (no annotation) Not ready

The hypermedia module supports student-driven access to the teaching material. The teaching material in ISIS-Tutor forms a hypermedia network. Presentation of any concept or construct includes providing generated lists of related concepts, examples and problems (Figure 3 and Figure 4). Important is that all the above lists includes both known and new concepts and examples, as well as solved and not solved problems (as we will see later, to distinguish known concepts from new they are marked in two different colours). The student can select any related concept or example from the generated list to move to a related page of teaching material (Figure 4). The selected concept, example, or problem is presented to the student, who can read information about the concept, experiment with an example, or try to solve the problem (Figure 5). When selecting an example (usually a program fragment using the original concept) the student moves to the programming environment component with the selected example loaded for interpretation. The student can play with the example using the visualizing interpreter, changing the data and the program example itself. When solving a problem the student also can use the programming environment. Presentation of both examples and problems includes "reverse" links: from a problem or an example to all concepts from the spectrum of these teaching operations.


Figure 5. Problem presentation page for problem 3. The answer line shows the correct answer. Answers can be prepared and debugged with the exploratory environment (see Figure 6).
Bottom line - menu options (same as on Figure 2): Enter - learn, F4 - practice, F6 - teach me, F8 - problem index, F9 - back.

Using the described approach we build a naturally structured and tightly interlinked hyperspace of educational material, which supports advanced navigation. For example, the user can start from a domain concept, then move to a related construct, then to some example of it's application. Here the user can enter the environment to play with the example, then move back to the construct and repeat it with another example. Then the user can select one of the problems related with the construct to master the obtained knowledge. If the problem appears to be hard, the student can analyse the list of concepts in the problem spectrum and move from a problem to the concept which is not clear yet (and which can be far away in the network from the the starting concept. Thus the user in ISIS-Tutor has many ways of navigation and many paths going from the current node to related nodes.

To help the user to navigate in this tightly interlinked hyperspace the hypermedia component applies an adaptive navigation support technique. The idea of adaptive navigation support in ISIS-Tutor is to annotate the set of links leading from the current node to related nodes (and from index page to all nodes) according to the current user knowledge and educational goals. The student model and the hypermedia component distinguish four knowledge states for each concept and related hypernode: not-ready-to-be-learned (i.e. has unlearned prerequisites), ready-to-be-learned, in-work(learning started), and learned (user demonstrate that he knows the concept by solving the required number of problems). Thus, at any moment the hyperspace is divided implicitly into four zones with different educational status. Our idea is that different zones have different meanings for the student and marking these zones visually would help the student in hyperspace navigation. To mark the zones the hypermedia component just marks the hyper-links of each node using different colours and some special characters. For example, the links to the nodes which are not-ready-to-be-learned can be dimmed so as not to distract the student. Similar technique is used to adapt the link presentation to current educational goal.

The educational goal in ISIS-Tutor is just the set of concepts which the student expected to learn at the current session. The goal can be prescribed by the human teacher of by the student himself. We use two kinds of goal adaptation: hiding all the concepts outside the current goal (which really restricts the student to this goal) and outlining the goal concepts. Thus, the links from the index or current pages to related concept pages have can have different colours and special marks attached to it, and that tells the student about the educational and goal states of the related pages (Figure 2). Similar ways are used the represent different educational states of related problems and examples (Figure 4).

Using colours to support adaptive navigation is definitely not the best way. Generally, colours are meaningless for the student. In addition, too many different colours on the screen is very distractive. We think that using icon-based (de La Passardiere & Dufresne, 1992) or text-based (Zhao, O'Shea & Fung, 1993) annotation is better in many ways. Unfortunately, these require advanced display facilities, while we were limited to IBM PCs. However, our preliminary experimental data shows that even colour-based adaptation can improve student performance: to reduce the time and the number of visited nodes while keeping the same level of results. These results show that adaptive navigation gives the user enough information about related nodes to avoid unnecessary visits to them.

Thus ISIS-Tutor provides a hypermedia-like way of investigating the teaching material. Navigating to related concepts and examples, the students can repeat learned pieces of knowledge as well as learn new material. While the student works in hyperspace, the hypermedia component uses the the stdent model for adaptive navigation support. On the other hand, according to the stundent model-centred architecture, the student-driven navigation is tracked by the hypermedia component and results are reflected in SM by increasing level of browsed concepts.

The environment is a tool for exploratory learning of the language and the acquisition of procedural knowledge. Regular CDS/ISIS/M facilities for exploratory learning are quite weak. The user can see only one of three main things on the screen: either the print format, a database record to be printed, or the results of printing. Also an ISIS user usually gets formatted text as the result of the application of the whole format to all database records and can hardly understand the contribution of any operator to the overall result. It is not so bad for regular work but it is very bad for a novice user exploring the language. It's very hard to learn from experience in ISIS, because the semantics of each print command can't be understood well.

Figure 6a

(Figure 6a)

Figure 6b

(Figure 6b)

Figure 6. The ISIS-Tutor exploratory environment. The environment is used for problem solving support as well as for a free exploration of the language to be taught. Parts (a) and (b) show different steps in processing and visualizing a print format example statement. On each part: Top line - the example statement (a processed portion is colored red); Central part - a database record to be printed; Bottom lines - the result of formatting.

The ISIS-Tutor environment was specially designed to support exploratory learning. Working with the learning environment the student can see at the same time a record of the sample database, a print format string, and the result of formatting the given record by the given format. The user can change record (select other database record or create a new one) or change format (select another example from a list suggested in the concept presentation window, edit or enter his own format). Another important feature of the environment is the ability to see the execution result of each operator in the format separately. In ISIS-Tutor environment a student can execute the format step-by-step or command-by-command thus learning what changes the executed command adds to the output (Figure 6).

Summary of the knowledge-based approach

Here we summarize main features of our approach to integrating a hypermedia component into an intelligent learning environment:

We will be working further testing and improving our approach. In designing a new system by our approach, a major part of our work is just repeating and re-coding the same things that we did designing older systems. We are now going further and starting the design of an authoring system based on our approach.
The following two section discuss in more details two special features of ISIS-Tutor - the student model-centred architecture and the adaptive navigation support.

Student model centered architecture for the integration of system components

In our work on student model centered ILE we were going from the ITS side, thus we adopted traditional ITS architecture as a basis for the architecture of student model centered ILE. The traditional ITS architecture includes three main components: the expertise component, the tutoring component, and the student modeling component. Each of the components localizes one of the three kinds of knowledge important for intelligent tutoring: knowledge about domain, knowledge about tutoring, and knowledge about student and student modeling (Wenger, 1987). According to this architecture, the student model represents the student's understanding of material to be taught. The student model is used by the tutoring component to provide adaptive tutoring on various levels. The results of student's work with teaching operations are returned back to student modeling (diagnostic) component and used to update the student model. This is called the student modelling loop.

To use ITS experience in student modelling we decide to apply the regular student model being used by the tutoring component of an ILE as the central student-user model for overall ILE. In our first systems the tutoring component provides regular student modelling loop, while other components of the ILE just use this central student model for adaptation. The only problem was to choose the kind of student model which can be used by all the components. We use overlay model, which contains one integer counter for each subject knowledge element measuring student's understanding of this element. This kind of overlay model is powerful and general enough to be used by different components of ILE. The student model is kept updated by the special evaluation module which analyzes the results of student' problem solving activity.

The above overlay model is accessible for all the modules of ILE and can be used by each of them to adapt its behavior to the student knowledge. However to avoid using senseless numbers and to provide more flexibility we suggested a threshold technique. Each of the ILE components can distinguish several distinct knowledge states for each knowledge elements. Each of these states has special meaning for the module from the adaptation point of view. The more states a module can take into account the more complex adaptation it can provide. Simple modules can distinguish only two states - for example unknown and known, while the most adaptive tutoring module of ITEM/IP can distinguish six states (Brusilovsky 1992a). To map a particular integer value of the overlay model into a set of states each module use integer thresholds which divide the possible range of values of the counter into required number of intervals corresponding to knowledge states recognizable by the module. Thus simple modules use one threshold only, while the tutoring module uses five thresholds. Each module use own set of thresholds over the central student model. These thresholds can be different for different knowledge elements and different students. The threshold technique provides a good flexibility, giving the way to adapt the student modelling mechanism to the knowledge elements of different difficultly and to different classes of students.

We applied the above student model centered architecture in several ILE for different domains. These ILE have the same overall architecture, but use different sets of modules and demonstrates several possible ways of applying the overlay student model for adaptation. For example, ITEM/IP contains the following adaptive modules: the strategy module which supports adaptive sequencing of teaching operations, the visual interpreter which uses the student's current knowledge level to provide adaptive error handling and adaptive visualization, and the presentation module which generates an adaptive description of a concept or a construct when introducing or repeating it. All these modules refer to the same six knowledge states (five thresholds) for each domain knowledge element in its adaptation rules. ISIS-Tutor adds the new adaptive module - the hypermedia component. As we already mentioned, it distinguish four knowledge states for each concept: not-ready-to-be-learned, ready-to-be-learned, in-work, and learned.

The methods of adaptation used in the our ILE are rather simple. The goal was not to improve the known methods of adaptation of various components, but to build a system where most of the modules can use the same student model to adapt their performance, in various ways, to the knowledge of the given user. On the further steps some simple methods of adaptation can be replaced by more sophisticated technologies developed in the fields of intelligent interfaces and intelligent help systems.

Adaptive navigation support as a kind of adaptive hypermedia

Here we put adaptive navigation support in more general context of adaptation in hypermedia. What can be adapted in adaptive hyperemedia (AH) are the content of a hypermedia page and the links from a page (including index pages and maps) to related pages. We distinguish these two techniques of adaptation and call the first technique adaptive presentation (or content-level adaptation) and the second technique adaptive navigation support (or link-level adaptation). Adaptive presentation is the most popular and the most studied way of hypermedia adaptation (Boecker, Hohl & Schwab, 1990; de Rosis, De Carolis & Pizzutilo, 1993; Boyle & Encarnacion, 1994).

With adaptive presentation the content of a hypermedia page is generated or assembled from pieces according to the user's class and knowledge state. Generally, qualified users receive more detailed and deep information, while novices receive more additional explanation. By adaptive navigation support we mean all the ways to play with visible links which can support hyperspace navigation. Previous works (Boecker, Hohl & Schwab, 1990; Kaplan, Fenwick & Chen, 1993) suggest adaptive ordering technique for adaptive navigation support. This technique apply user model and some user-valuable criteria to adapt the order of presentation for all possible links. It gives the user a hint which link to follow (the more close to the top, the more relevant the link is). We think that adaptive ordering technique provides a good way to support user navigation in the pages with dozens of possible links, but has less sense in educational context when the number of links is smaller. Some research also shows that the stable order of options in menus is important for novices. We apply another technique for adaptive navigation support in educational hypermedia: visual adaptive annotation of links (augmenting links with dynamic comments in any form) according user's goal and knowledge state (Brusilovsky, Pesin & Zyryanov, 1993; Brusilovsky & Pesin, 1994).

We expect that adaptive annotation can give the users some additional information about accessible nodes and thus reduce their floundering in the hyperspace. In particular, we hope that it reduce the number of "orientation" visits when the user visits related nodes just for several seconds to see what is around him. At present there are very few studies which investigate the effectiveness of AH. The experiments, reported in (Boyle & Encarnacion, 1994) shows that adaptive presentation increase user performance. The work (Kaplan, Fenwick & Chen, 1993) reports positive experimental results of adaptive ordering technique. By now there was no experiments with adaptive annotation technique, though some related research shows that even non- adaptive annotation, which tells the user more about the nodes designated by annotated links, can increase students performance (Zhao, O'Shea & Fung, 1993). It was the goal of our recent experiment with ISIS-Tutor to check the effectiveness of adaptive annotation in educational context.

As we described above, hypermedia component of ISIS-Tutor uses colours and special marks to annotate the set of links leading from the current node to related nodes (and from index page to all nodes) according to the current user knowledge and educational goals. In the version which was used in experiment the links to concepts which are the goal of the current lesson are marked with a sign "-", links to not-ready-to-be-learned concepts were not specially coloured, ready-to-be-learned were coloured red, both in-work and learned were coloured green, and learned concepts was additionally marked with sign "+". Links to problems were adaptively annotated by the same way.

Figure 7

Figure 7. Topic index in ISIS-Tutor with hiding enabled. All "ready-to-be-used" links occupy just one half of the screen. Compare with Figure 2 showing the same index with hiding disabled.
Bottom line - annotation keys: +(green) Well learned, (green) Learned, (red) Ready to be learned.
Line above the bottom - menu options: Enter - learn, F4 - practice, F6 - teach me, F8 - problem index, F9 - back.

Twenty six subjects (first year computer science students of the Moscow State University) took part in the experiment. They were briefly introduced to ISIS-Tutor and then had up to 45 minutes to work with the system. The same educational goal (ten concepts and ten test problems) was set to all the students. To finish the course, each user had to solve all ten problems. The subjects were divided randomly into three groups. Group A worked with hypermedia without any adaptation, the students however were given the numbers of goal concepts (in index and all menus the name of the concept is always preceded by its number). Group B worked with adaptive hypermedia as described above. Group C worked with restrictive version of the same adaptive hypermedia: the links to all not-ready-to-be-learned concepts and problems and to all concepts and problems outside the learning goals were excluded from index and all other menus (Figure 7). The idea of this restriction is to reduce the cognitive load of the student by excluding "not useful" information. All actions of the students working with the system are recorded and then analyzed to compare various aspects of user performance.

GroupOverall number
of steps
of studied concepts
concept -> concept
index -> concept

The results of the experiment are shown in the table above (all data are average numbers for each group). As we can see, the overall number of navigation steps, the number of repetitions of previously studied concepts, the number of transitions from concept to concept and from index to concept are seriously less for AH. Moreover, this difference is usually bigger for non-restrictive hypermedia. The results of our experiment shows that adaptive visual annotation of hypermedia links in educational context can really reduce user's floundering in the hyperspace and make the learning with hypermedia more goal-oriented. With adaptive annotation the user can achieve the same result with less navigation steps and by less visits to hypernodes. It is interesting to compare our results with the results presented in (Boyle & Encarnacion, 1994).

This work reports that adaptive presentation in hypermedia can reduce the time for learning the material and improve the comprehension of it, but can not reduce the number of nodes visited in the process of learning. In the same time, adaptive annotation of links can hardly improve the quality of learning, but can reduce the number of visited nodes thus further reducing the learning time. These techniques looks complimentary and can be used together for further improvement of the effectiveness of learning with hypermedia.

We plan further experiments to investigate various aspects of adaptive educational hypermedia.


Boecker H.-D., Hohl H. and Schwab T. (1990)
Hypadapter - Individualizing hypertext, In Proc. INTERACT'90, North-Holland, Amsterdam, pp.931-936.

Boyle C. and Encarnacion A.O (1994)
MetaDoc: an adaptive hypertext reading system. User models and user adapted interaction, 4(1), 1-19.

Brusilovsky P. (1992a)
A framework for intelligent knowledge sequencing and task sequencing. In Frasson C., Gauthier G. and McCalla G.I. (eds.) Intelligent Tutoring Systems. Proceedings of the Second International Conference, ITS'92. Springer-Verlag, Berlin. 499-506.

Brusilovsky P. (1992b)
Intelligent Tutor, Environment and Manual for Introductory Programming. Educational and Training Technology International, 29(1), 26-34.

Brusilovsky P. (1994)
Adaptive hypermedia: the state of the art. In Proc. of MHVR'94 Multimedia, Hypermedia and Virtual Reality (Moscow, Sep. 14-16, 1994), ICSTI, pp.24-29.

Brusilovsky P., Pesin L., and Zyryanov M. (1993)
Towards an adaptive hypermedia component for an intelligent learning environment. In Human-Computer Interaction. Lecture Notes in Computer Science #753, Springer-Verlag, Berlin, 348-358.

Brusilovsky P. and Pesin L. (1994)
ISIS-Tutor: An adaptive hypertext learning environment, Proc. JCKBSE'94, Japanese-CIS Symposium on knowledge-based software engineering. (Pereslavl- Zalesski, May 10-13, 1994). Tokyo, pp 83-87

de La Passardiere B. and Dufresne A. (1992)
Adaptive navigational tools for educational hypermedia. In Tomek I. (ed.) Computer Assisted Learning. Proceedings of the 4th International Conference, ICCAL'92. Springer-Verlag, Berlin. 555-567.

de Rosis F., De Carolis N. and Pizzutilo S. (1993)
User tailored hypermedia explanations, in Adjunct Proc. INTERCHI'93 (Amsterdam, April 24-29 ,1993), pp. 169-170.

Hammond N. (1989)
Hypermedia and learning: Who guides whom? In Maurer H. (ed.) Computer Assisted Learning. Proceedings of the 2-nd International Conference, ICCAL'89. Springer-Verlag, Berlin. 167-181.

Hendley R.J., Whittington C.D. and Jurascheck N. (1993)
Hypermedia generation from domain representations. Computers and Education, 20(1), 127-132.

Kaplan C., Fenwick J. and Chen J. (1993)
Adaptive hypertext navigation based on user goals and context. User models and user adapted interaction, 3(2).

Lajoie S.P. and Lesgold A. (1989)
Apprenticeship training in the workplace: computer-coached practice environment as a new form of apprenticeship. Machine Mediated Learning, 3(1), 7-28

Pesin L.A., Brusilovsky P.L. (1992)
Adaptive task sequencing in ISIS-Tutor. In Abstracts of East-West Conference on Emerging Computer Technologies in Education, Moscow, 6-9 April 1992, Moscow. 176.

Shute V.J. and Glaser R. (1990)
A large-scale evaluation of an intelligent discovery world: Smithtown. Interactive Learning Environments, 1(1), 51-77.

Zhao Z., O'Shea T., Fung P. (1993)
Visualization of semantic relations in hypertext systems, In Proc. ED-MEDIA'93, AACE, 556-564.


Dr. Peter Brusilovsky

12.1994 to 3.1996 @ Department of Cognitive Psychology, FB I-Psychologie, Universitaet Trier, D-54286 Trier, Germany.; fax: +49 651 201-2955; phone: +49 651 201-2976

since 4.1996 @ School of Computer Science, Carnegie Mellon University, Pittsburgh, PA 15217, USA.