CALL AND TEACHING 10

Factors influencing the success of computer mediated communication (CMC) environments in university teaching: a review and case study
Andrew Tolmie and James Boyle
Department of Psychology, University of Strathclyde

Abstract
Despite its potential benefits, the effectiveness of CMC when used to support learning in higher education isvery variable, making it important to identify those factors which best predict successful implementations. A review of the literature from the past few years, presented in the first half of this paper, suggests that, consistent with Activity Theory (Leont'ev, 1978), the critical factors are those which provide a context and rationale for online communication by helping users to establish a shared purpose. However, generating empirical support for this hypothesis presents two kinds of methodological problem: specifying the methodsand measures necessary to discern the existence and impact of shared purpose; and dealing with the difficulties of making controlled comparisons in this area. The second half of the paper illustrates, via an implementation case study, something of how these methodological problems might be resolved, and presents evidence in favour of the central importance of shared purpose.
Key words
computer-mediated communication; distance education and telelearning; evaluation methodologies; pedagogical issues; post secondary education.
Running title
Factors influencing the success of CMC
Contact
Dr Andrew Tolmie,
Centre for Research into Interactive Learning,
Department of Psychology,
University of Strathclyde,
40 George Street,
Glasgow G1 1QE
email: a.k.tolmie@strath.ac.uk
Fax: 0141 552 6948
2
Factors influencing the success of computer mediated communication (CMC) environments in university teaching: a review and case study
Andrew Tolmie and James Boyle
Department of Psychology, University of Strathclyde
Introduction
The past five years have seen rapidly increasing use of CMC (i.e. email and text-based conferencing, particularly in asynchronous mode) within higher education (HE). This growth reflects attempts to circumvent the constraints of distance (Kaye, 1992a; Daniel, 1996) and increased student numbers (Pincas, 1995a; Tolmie & Anderson, 1998) on educational interactions, by introducing alternative forms of student-student and student-tutor contact, such as virtual tutorials. However, it is also fuelled by a belief that CMC promotes engagement and productive discussion (see e.g. Kaye, 1989; Harasim, 1989; Odasz, 1992; Bates, 1995; Henri, 1995; Jones, 1998). As Beattie (1982) notes, in face-to-face tutorials tutors typically pose questions to individual students, students respond, and tutors comment on students' responses, in what is known as the Initiation--Response--Evaluation (IRE) sequence (Wells, 1997; see also Sinclair & Coulthard, 1975). Dialogue between students is more-or-less non-existent. Online, however, the IRE participation structure seems harder to maintain, and teacher initiations are often ignored. Berkenkotter (1997) reports of one instance that "instead the electronic forum served as an open space for plethora of conversational topics introduced by students...[and] brought forth many voices and many student issues, feelings, and agendas that would have never surfaced in classroom talk."
Central to explanations of these differences is the notion of affordances: properties of objects or systems which allow certain actions to be readily performed with them, and which therefore push behaviour in certain directions. The concept was first defined in
3
the context of ecological psychology, but explicitly applied to the use of electronic media by Gaver (1992). Dede (1991) argues that a general characteristic of mediated interaction is that technologies create environments which shape interpersonal exchange. With CMC, one result is "more heterogenous discourse forms, more 'whispering' among the 'students'" (Cole, 1997; see also Graddol, 1989; Kaye, 1992b), with less vocal students in face-to-face settings becoming active contributors online (the "democratising influence" of Steeples, Goodyear & Mellar, 1994; see also Harasim, 1989; Kaye, 1992a). Light, Colbourn & Light (1997), for instance, report that whilst male dominance was common in face-to-face tutorials, gender differences in contributions disappeared online (see also Steeples, Unsworth, Bryson, Goodyear, Riding, Fowell, Levy & Duffy, 1996; Trushell, Reymond, Herrera & Dixon, 1997; though Kirkup, 1996, gives contrary evidence). Less "face" seems needed to become involved, and the presence of authoritative voices is less obvious (Bates, 1995; Pincas, 1995a) so these cease to be a conduit for interaction. Having space to reflect before responding may amplify these effects (Leach, 1996).
All this might be of limited interest, were it not for theoretical models of conceptual growth which stress the role of communication between collaborating learners, and studies of peer interaction that support these (e.g. Howe, Tolmie, Anderson & Mackenzie, 1992). Notable amongst such models are those derived from the socio-cognitive conflict theories of Piaget and Doise (Piaget, 1932; Doise & Mugny, 1984; see also Wolf, 1988; Henri, 1995). Central to these is the argument that when peers engaged in an activity disagree over some decision, their equal status means no one viewpoint is accorded intrinsically greater merit. As a result, each participant has to make explicit the basis for their ideas, so that their respective qualities can be assessed. This dialogue, of the kind Berkowitz & Gibbs (1983) call "transactive" discussion (i.e. the articulation, critique and defence of ideas), exposes inadequate understanding and creates pressure for conceptual growth. According to Piaget and Doise this growth occurs via recombination of the best elements of existing ideas
4
(one's own, post-task, for Piaget; those which have been discussed, on-task, for Doise) into a formulation which resolves the initial conflict (cf. Harasim, 1990, on idea generation, linking and structuring). Research indicates that conflict not only produces conceptual growth as predicted, but that this can happen through both individual post-task reflection and on-task negotiation, depending on, amongst other things, prevailing task conditions (e.g. Tolmie, Howe, Mackenzie & Greer, 1993). This framework pinpoints the value of asynchronous email exchange: it is not just that it facilitates discussion between students, but that any disagreements which occur will promote growth in understanding. Indeed, CMC might be the ideal medium for maximising the positive effects of conflict, since it can support both reflection before responding (McNeil, 1992; Steeples et al., 1994; Nalley, 1995; Light et al., 1997; Wilson & Whitelock, 1998a) and on-task recombination of ideas, by providing the text of messages in reviewable and manipulable form (McConnell, 1988; Harasim, 1989; Henri, 1995).
It is these properties which lie at the core of the positive effects associated with CMC, and explain its potential to promote the right "quality of interaction" (Wolf, 1988). However, despite this apparent potential, the practice often falls short. Crook (1994), Light et al. (1997), Philips & Santoro (1989) and McAteer, Tolmie, Duffy & Corbett (1997), for instance, all report implementations in residential HE where take-up by students was very patchy, with a few making much use of the system, and many making little or none. Where this pattern occurs in distance education (Thomas, 1989; Kaye, 1992a, 1992b), it is attributed to students not having the time to try using the resource (Mason & Bacsich, 1998). However, as Rapaport (1991) points out, skewed participation rates appear to be the norm across educational and corporate contexts, suggesting that more fundamental processes are at work. Even where take-up is less patchy, usage may still bear little resemblance to transactive discussion. Crook & Webster (1997), for example, describe an implementation where all students used the system, but most traffic was not work-related (see also Crook, 1988, 1994). If work-
5
related interaction occurs, this often takes the form of questions to, and responses from tutors (Steeples et al., 1994; Light et al., 1997; Stainfield, 1997; Wilson & Whitelock, 1998b; Rada, 1998; Trushell, Reymond & Burrell, 1998), hardly an improvement on face-to-face IRE exchanges. Finally, if this limitation is avoided, the outcome can still be "desultory, 'chaining' interactions that 'don't amount to much'." (Cole, 1997; see also Henri, 1995). Plainly, given such variation, productive use is not just a function of generic system affordances. To avoid much wasted effort, then, it is important to identify the factors above and beyond these that determine successful implementations.
Factors associated with successful CMC resources in HE
Given the possible influences, the task does not seem easy. However, the literature that details the variations in outcome noted above also provides strong leads as to which factors, and which levels of these, are associated with 'successful' CMC resources:
1) Size of group: smaller is better. Light et al. (1997) found that small groups of six students working in a self-selected area made more frequent, longer and balanced use of CMC than a large group of 80. Comparison across McAteer et al. (1997), Tucker, Whately & Hackney (1997), Issroff & Eisenstadt (1997), and Wilson & Whitelock (1998b) yields a picture consistent with this (see also Mason & Bacsich, 1998). Bates (1995) reports that in practice CMC is commonly used for relatively small groups, and Steeples et al. (1994) indicate that where students are part of larger groups, they sometimes seek to set up smaller private conferences (see also Kaye, 1995). Against this is evidence that a certain amount of activity is needed or people will not log on to the system (see e.g. Wibe, 1994). However, large groups may achieve this by having many people to draw on, despite a low proportion of active contributors; whilst small groups may reach the critical level by increasing contributions per individual, and by promoting sustained engagement by all participants.
6
2) Knowledge of other participants: it is better if participants know each other. The evidence here is often more suggestive than direct, but McAteer et al. (1997) report a case study where use of CMC for identical tasks was greater where participants were familiar with each other than where they were not. Drysdale & Creanor (1998) found poor use of a HyperNews discussion facility on a staff development course by a group of lecturers, researchers and postgraduates who were unfamiliar with each other. Similarly, Rada (1998) reports that the balance of student-student and student-teacher interactions was overwhelmingly towards the latter in a group who had had no face-to-face meetings and little other contact with each other. Conversely, Lewis (1997) argues that the computer supported cooperative work (CSCW) literature points to the importance of early face-to-face meetings for later online interaction (see also Friedman & McCullough, 1992, on work with teacher-researchers). Levinson (1989) makes the same point in an HE context. Calvani, Sorzio & Varisco (1997) found good communication within two groups of students at different locations, whose members knew each other, but incomprehension and even antagonism between them.
3) Student experience: it is better if students are experienced communicators under the task conditions involved. Light et al. (1997) found more use and greater spread of use of CMC amongst third years than first, although experience was confounded here with size and familiarity. However, they also report that mature first years made more contributions than younger, both online and in face-to-face tutorials, consistent with an effect of experience at expressing personal viewpoints. Drysdale & Creanor's (1998) participants were all relatively experienced and this did not seem to help, but coming from different categories of staff, their experience of communicating with each other may have been limited. Certainly, Trentin (1997) found that teachers with similar backgrounds on a high-level course showed none of the problems reported by Drysdale & Creanor or by Calvani et al. (1997), despite also undertaking between and within location interaction.
7
4) Clarity about task: it is better if students understand how to go about the task they are engaged in, especially if this understanding is shared. Lewis (1997), again drawing on CSCW work, argues that consensus over what the task involves is a major determinant of smooth online contact (see also Friedman & McCullough, 1992). Part of the problem for the students in Calvani et al. (1997) was that they were not clear what they were supposed to be doing, and had particular problems with disagreement cross-location on this point. Pincas (1995b) notes that of four modes of CMC use in one course, the most successful was that in which students were allocated a specific task. More generally, Mason & Bacsich (1998) report that simply leaving students to get on with using CMC resources only produces successful interaction if tutors provide extensive and well structured support, and students may actually request set tasks. A number of authors (see e.g. Kaye, 1992b) conclude that a leader or moderator is crucial to the quality of CMC activity, but it may be that this support is needed precisely because it is not clear to students what the task is. Where CMC is embedded in other online resources (e.g. web materials), this has been argued to promote its use (Steeples et al., 1994; Crook & Webster, 1997; Crook, 1998), as has the UK Open University "online wrap around" model (Mason & Bacsich, 1998), which grounds conferencing in set workbook activity. In both cases this may be because such embedding serves to define more clearly the reasons for communication, and what it is intended to achieve.
5) Ownership of task: it is better if students have the chance to negotiate what the task is to involve. The third year students in Light et al. (1997), who made good usage of CMC, established their own task framework early on. Lewis (1997) and Calvani et al. (1997) both indicate an agreed division of labour is important, and Lewis argues it is best if this is negotiated face-to-face at the outset, with opportunities for renegotiation later on. In Issroff & Eisenstadt (1997), there are signs that students became more active online as they negotiated roles for themselves. Students may not
8
decide to do what teachers think they should be doing, however (SHARP Project Team, 1998): with increased ownership, they are more likely to generate the alternative agendas noted by Berkenkotter (1997).
6) Need for system: it is better if there is a clear function for CMC which cannot be served more easily in another way. Riel & Levin (1990) argue there has to be good reason for students to interact electronically, if this is to happen (see also Levinson, 1989). This explains the greater incidence of CMC use in distance education relative to comparable residential contexts (e.g. the introductory computing students in Wilson & Whitelock, 1998a vs the first year psychology students in Light et al., 1997): these students are necessarily more reliant on the email system for a range of contact (Bates, 1995; Mason, 1989). Even here, though, usage depends on what students see the system as being for: if they regard it as a support mechanism to overcome isolation, it tends to be difficult, for example, to get them to engage in online discussion of course content (Mason & Bacsich, 1998). In residential contexts, Tucker et al. (1997) found much use of email by student groups who were able in this way to exchange updated versions of crucial documents outside of scheduled laboratory sessions. This was seen as valuable because it allowed them to continue working through the week. In contrast, McAteer et al. (1997), Crook (1997) and Canning & Swift (1992) all found that use of conferencing resources was minimal or non-existent when students had little explicit reason for doing so.
7) Type of system and 8) prior experience of CMC. These may be factors, but the important levels are unclear. Some authors (e.g. Harasim, 1989; Ambrosius, 1992; Davies, 1995) have argued system configuration is a crucial influence on CMC use, but the growing similarity and sophistication of systems as technology develops means that their features have become generic in important ways, and this may now be less of an issue. Canning & Swift (1992), Steeples et al. (1994), Silvennoinen & Kerttula (1994) and Bates (1995) argue technical preparation is important, but in
9
many of the successful instances noted above this was students' first use of CMC, with little apparent training. Again, improvements in the usability of systems may have helped here.
Processes underlying the operation of the identified factors
Having isolated a number of factors associated with the success of CMC resources in HE, it is pertinent to ask what processes might underlie the influence of these factors. There are a variety of possibilities. For example, group size might be important because users are less self-conscious in small groups. Mason & Bacsich (1998) report students will go so far as to back out of sending messages if they read others which they think are better, but since there would be less chance of seeing such contributions in a small group, its members might not feel so exposed to potential ridicule. They might also feel more pressure to become involved, since hiding any lack of input would be harder.
Explanations of this type are of limited scope, though, and at best may only describe processes of marginal importance. Those detailed by a unified account of the role of all the factors would be more likely to be central, and such an account can in fact be made out. The start point for this is knowledge of other participants. Steeples et al. (1994) argue that face-to-face meetings help users because they reduce the anonymity of subsequent exchanges. CSCW research suggests the reason this is important is that "people prefer to know who else is present in a shared space, and they use this awareness to guide their work" (Ackerman, 1997). One way they do so is by modelling others' concerns, in order to anticipate their needs, how these will inform what they say, and how they will react to messages aimed at them. Such models are argued to be the basis of fully effective communication (see Krauss & Fussell, 1990; Morgan & Schwalbe, 1990; Happé, 1993), and one means of building them up is via personal knowledge of those being communicated with.
10
This explanation can be extended to the influence of the other factors. With regard to group size, for instance, if participants lack a model of each other at the outset it will require less effort to build one up in a small group, even without face-to-face meeting. Similarly, communicative experience will provide users with a model of the general needs and behaviours of those engaged in particular tasks. This explains why the impact of this factor depends, as noted above, on common past experience, otherwise different models will be drawn on, and participants' expectations of each other will fail to correspond. Understanding of the immediate task would provide a localised model of the needs of others in the absence of communicative experience. The influence of task negotiation would follow from, but go beyond this, in that it would allow this model to be actively constructed rather than tacitly assumed.
Thus all these factors can be argued to facilitate use of CMC by providing participants with models which assist the mutual anticipation of communicative needs. One reason why this might be important is provided by the deficit account. In face-to-face interaction, cues such as gesture, facial expression and intonation yield information that aids interpretation of communications. Rutter, Stephenson & Dewley (1981) proposed that shifting from face-to-face communication to e.g. a textual medium results in loss of this information (a state called "cuelessness"), causing messages and authors' intentions to seem more ambiguous. Rutter et al. noted that this apparently led communicators to feel socially distant from each other, dropping all social niceties (see also Kiesler, Siegel & McGuire, 1984) and focusing on the basic messages that had to be exchanged to carry out the task in hand. If communicators held good models of each other, this might ameliorate cuelessness, since intentions could be worked out, and supporting information about the significance of communications would be less critical.
11
This account does not explain the need for CMC use to serve a clear function, though. Moreover, it implies successful online interaction depends on users possessing good mutual understanding at the outset. This sits oddly with the notion of CMC promoting transactive discussion, where participants learn by working through their differences. There is a broader account, however, which suggests how mutual understanding could provide the medium with a function and permit transactive discussion, as well as aiding message interpretation. Rutter et al. (1981) explained their observation that cue loss leads to an increased focus on task in terms of social distance effects. A better reading may be that it is the task in hand which gives communications their shape and function, and that shared perception of what that task involves, rather than anything else, sustains interaction when other information is lost. If so, then perhaps the identified factors promote CMC by helping users establish not just any mutual understanding, but a shared purpose: certainly, communicators’ models of each others’ needs are likely to go hand-in-hand with models of the task from which those needs arise, and the more consensual the latter, the more accurate the former. This shared purpose might then also determine whether and how CMC is used.
The rationale for this account is provided by activity theory (Leont'ev, 1978, 1981; see also Cole, 1996; and Lewis, 1997, for a summary in relation to CMC usage). Activity theory situates behaviour within social contexts, via three levels of description: activity system, action, and operation. The activity system is the basic unit of analysis of group and individual behaviour, and comprises a subject (the group or individual) using tools (including writing and speech) to pursue an object (a global intention or purpose). Tools and objects are not invented from scratch when an activity system comes into being. Cultures store up defined objects and prescribed methods of using particular tools to achieve these, and members of a culture are inducted into knowledge of these objects and methods. When individuals interact, they use shared knowledge provided by their culture to reconstruct the activity system pertinent to their intended object. For instance, writing this paper constitutes an activity system,
12
and the authors' culture provides a shared understanding of the object of this exercise and how to go about it. Part of this understanding focuses on actions directed at specific goals, which subjects take to move toward the overall object. Actions are usually conscious, but comprise relatively unconscious operations, through which they are carried out. Thus if the object of the activity system is to produce this paper, composing this paragraph is a contributory action, and typing this word is a constituent operation. It is important to emphasise that it is the perceived activity which organises actions, and gives them meaning: in the absence of an identifiable activity, actions are meaningless, and will not be performed.
Activity systems and their constituent actions and operations are not static, though, but evolve. Cultural transmission is not perfect, and systems are reconstructed by individuals who differ at least slightly in their understanding of that system's object, and the actions required to achieve it. Engestrom (1987), for example, emphasises that activity systems may contain various viewpoints, which serve both as a resource and a source of conflict. Conflict is overcome by reconciling these viewpoints into new (and potentially more adaptive) formulations (cf. Doise & Mugny, 1984), but this can only occur within a certain range of convenience (to borrow the term used by Kelly, 1963) i.e. where there is moderate disagreement within a shared framework. If differences are too great, the activity is unlikely to get off the ground; if they are small, agreement is tacitly assumed. In between, productive discussion may take place about concepts and procedures, depending on what task progress requires. An illustration of this is provided by Howe et al. (1992), who found that undergraduates' strategies for solving a series of computer presented problems improved most after discussion between those with different strategies but similar ideas about underlying principles i.e. where there was a shared framework but one with enough tension in it to create new understanding.
13
From an activity theory standpoint, shared purpose is critical to student usage of CMC in a number of ways. Firstly, lack of shared purpose would indicate the absence of any agreed activity system to organise and give meaning to joint action, leaving exchanges uninterpretable, because founded on differing conceptions (if any) of the object of the task and each others' roles. This is consistent with the previous points about mutual understanding, but puts shared perception of the task at the core of any mechanism for construing the communications of others and anticipating their needs. It also explains why a variety of factors influence CMC use: there are many different ways in which shared purpose (i.e. a working activity system) can be established. This is more likely to happen if participants have similar prior experience of the task in hand or if they have had similar instructions on what they should be doing. In the absence of these, though, it is more likely that participants will be able to establish a shared purpose in a small group, if they know each other, or if they have the chance to meet and negotiate.
Secondly, even if there is a shared purpose, unless CMC can be readily used to carry out actions and operations required by the activity, it will not be taken up. This explains the importance of the resource's perceived function, and within the same framework as the other factors affecting usage. Thirdly, even if the shared purpose encompasses use of CMC, this may not lead to transactive discussion. In this account such interaction depends not on system affordances (although these can undoubtedly facilitate it), but on the perceived nature of the activity and the actions it requires, which might include such discussion, either in its own right or as a means of resolving differences between users. Thus the account deals with both how and why the generation of new understanding via transactive discussion happens, but it is only one possible outcome. This explains part of the gap between potential and actual CMC usage, and implies that the definition of productive usage, and hence successful resources, needs to be considered in broader terms, according to whatever shared purpose is in operation.
14
Problems of testing the shared purpose hypothesis regarding the use of CMC resources
Thus activity theory provides a fully unified account of why the identified factors affect CMC usage, centred on the hypothesised role of shared purpose. Putting this to the test, though, presents various methodological problems. One is how to measure whether the effectiveness of CMC does vary with the extent and nature of shared purpose. In activity theory terms, an effective resource is simply one which produces significant contributions towards the object of the activity. This makes it difficult to fix on specific types of usage as indices, since any might be productive, depending on the task in hand. Moreover, merely examining the incidence of different types of contribution would be inadequate. A given type might, for example, be infrequent, and yet serve a critical function (see e.g. Howe et al., 1992). In order to discern whether useful contributions are being made, then, these must be looked at in direct relation to the broader activity of which they form a part, and the role they play, if any, in furthering this.
Thus what is required is an integrated range of measures, covering both CMC use and other actions and interactions, including negotiations about the activity and what it is intended to achieve. This stipulation contrasts with a common tendency to employ (or at least report) only limited and unrelated snapshots of CMC use, making it hard to judge the significance of observations. In this respect, it echoes other authors' concerns with triangulation between different information sources, and the identification of a coherent story (see e.g. Breen, Jenkins, Lindsay & Smith, 1998). However, an analysis in terms of activity theory specifically entails that this approach be adopted, since it is only by examining the whole activity system that CMC use (or the lack of it) can be understood, and that the hypothesised role of shared purpose can be examined.
15
Beyond this is the problem of controlled comparison. An experimental study would manipulate the extent to which shared purpose is achieved by different groups, and note the impact of this on CMC usage. However, if shared purpose is determined by what makes sense to students themselves, the effect of such manipulation will be unpredictable. Moreover, establishing a meaningful purpose will almost certainly require that the activity take place in the "real world". Thus controlled laboratory studies are not likely to be informative, if even achievable (Draper & Brown, 1998). Naturalistic experiments, in which spontaneous variations in shared purpose are capitalised on to compare outcomes, would be an alternative. These would surrender strict control, but would preserve the real world context, and avoid the ethical problems of actively steering some students away from potentially beneficial experiences (Oliver & Conole, 1998a). This approach is similar to "cluster evaluation" (Barley & Jenness, 1993), where related implementations are examined using common measures. Such studies are difficult to conduct, however, because of the need to locate similar sites at a convenient time, and coordinate work between them. This in practice leaves research reliant on single case studies, surrendering direct comparison, let alone control. Worse yet, if the cases are real world ones, those responsible for teaching will naturally want any CMC resource to be effective. Thus, in studying the impact of shared purpose it is hard to avoid proceeding by looking at a resource in which it is encouraged as far as possible, and examining whether this works i.e. introducing a confirmatory bias, as well as abandoning controlled comparison.
In an attempt to clarify how to deal with these issues, the remainder of this paper focuses on an case study involving students at the University of Strathclyde. This study examines the importance of shared purpose for CMC use, and illustrates the form that integrated measures of activity might take, whilst suggesting an alternative perspective on the problem of confirmatory bias.
16
MSc Educational Psychology Case Study
The MSc in Educational Psychology is a two-year postgraduate professional training course involving formal teaching, group and individual project work, and practice placements. Trainees attend the University two days per week, and spend another two days in Psychological Service offices across central Scotland. They organise the timetabling of project work for themselves. Since for much of the week the trainees are either in separate locations or have a full schedule of classes, it was decided in 1997, with their agreement, to introduce an online conferencing system that would allow them to communicate with tutors and each other without having to worry about constraints of place and time. This system operated from networked machines in the University and modem links in Psychological Service offices using First Class 3.5. This software was chosen because it had a relatively transparent interface and menu-driven facilities, important considerations given that only three of the cohort of twelve trainees at that time had used email before. In view of this, training was also provided in its use, backed up by paper-based notes. This focused on sending messages to a shared conference space and private mailboxes, and on replying to these.
During the set up of the system, networking problems meant that connections from placement offices proved difficult to establish, and five weeks after implementation, only eight of the twelve trainees were fully online. Two others continued to have problems for five weeks after that. As connections began to settle down, the trainees were given a specific incentive to make use of the system. One month after its introduction, they were divided into two groups of six and given the task of collaborating within these groups to conduct a literature review and write a seminar paper on a set theme, making use of the conference as appropriate. The groups were allowed approximately five weeks to prepare their paper, and worked consecutively, with the first group (Group 1) presenting their paper at a face-to-face session one week after the second (Group 2) had been set their task. In addition to the seminar task, the trainees were encouraged to use the conference to collaborate on their group
17
projects, and prepare a poster for a professional meeting. No restriction was placed on less formal communication.
Evaluation
Although the conferencing system was set up to facilitate trainee communication, its implementation also provided an opportunity to examine whether the existence of a shared purpose amongst users was central to the success of CMC resources. Thus the literature reviewed earlier was used as a guide to set up a system that would be well-used (cf. the point about confirmatory bias). For instance, this particular group of students was identified as having all the characteristics of group size, familiarity with each other, communicative experience, and task-related expertise that would predict a successful implementation. In addition, care was taken to ensure that at least one clear task was set, that ownership of this task and perception of need for the system were fostered, and that, as far as possible, the system itself was readily usable. Within the constraints imposed by a case study approach, this allowed examination of the validity of the theoretical framework outlined earlier. If this were correct: 1) the group and task characteristics would help trainees establish some discernible shared purpose to motivate their online interactions; 2) the type of CMC usage that occurred (if any) would correspond primarily to the requirements of this shared purpose; and 3) the resource would be successful in terms of its use, and would be perceived to be successful, to the extent that it helped trainees achieve this shared purpose.
Evaluation work was aimed at gathering information relevant to these predictions, and the measures employed were chosen accordingly, bearing in mind particularly the need for triangulation between system use and the broader context of activity. An outline of the measures, and the data that they generated over the five month period from the conferencing system being set up until the ensuing summer vacation, is laid out below. This is followed by a summary of how far the predictions were met, and what conclusions can be drawn about the activity theory framework. The implications
18
of the research with regard to the methodological issues raised earlier will also be considered at that point.
The introduction of the conferencing system and framing of subsequent activity
Given the theoretical significance of initial activity for subsequent use of CMC, it was important to monitor the introduction of the conferencing system and the framing of its use, since this might provide insights into what needs the trainees saw it as serving. Thus written records were kept of meetings where the introduction and use of the system were discussed, in the form of notes made at the time, or during interviews with course tutors immediately afterwards.
In mooting the introduction of a conferencing system, the course director noted various potential benefits: a) easier communication on administration and project work; b) improved means of generating joint writing; c) contact with other professionals and trainees; and d) experience of using resources of this kind. He also mentioned that, to establish whether it was effective, the use of any such system would need to be monitored, and this might involve the completion of questionnaires and logs, and inspection of online messages, but only with trainees’ agreement. There was a cautious reaction to the idea of monitoring, and a second meeting was required to reassure the trainees that intrusion would be kept to a minimum, and online messages would not be inspected without consent. Concerns were also voiced at this meeting about use of the system involving extra work on top of an already heavy load; and conversely, about whether they would actually find things to use it for, and if so, whether it would benefit everyone. Despite these reservations, though, the trainees expressed a favourable overall opinion, and agreed to proceed.
Once the system was set up and they had been instructed in its operation, the trainees were given time to establish for themselves something of how they might use it. Similarly, when the seminar task was introduced, the groups were given a familiar
19
objective, and left to decide how exactly to pursue this, although they were steered towards use of email. The aim in both cases was to promote ownership of online activity and encourage perception of a role for the system, although the outcome was not directly monitored.
The pattern and content of online and offline interaction
A variety of methods were employed for monitoring how the conferencing system was used, and examining how online interaction related to offline activity. Observation of a face-to-face seminar confirmed both the trainees' relative intimacy, and the fact that they were experienced communicators under familiar conditions. Over a two hour period almost all made substantial contributions, and they typically addressed each other rather than the tutor who was chairing the session. They also organised floor-shifting amongst themselves, those who had not yet contributed picking up a point from the current speaker, developing it and carrying it on into their own presentation.
Online interaction was examined in terms of: a) the frequency and timing of messages; b) the relative contribution of individuals; and c) message function and length. On the first, a count of messages per week showed a small flurry of activity for the first four weeks after start-up (see Table 1), as those who could gain access tested their connections. This subsided over the next four weeks, despite Group 1's seminar task having been set, although this period did coincide with the Easter vacation. There was then a sharp climb as the rest of the trainees established connection, Group 2's seminar task was set, and Group 1 finalised their paper. The four week period leading up to Group 2's seminar saw a drop back from this peak, and there was a further decline during the period after that, leading into the summer vacation. At no point, then, was message frequency high, but this was not surprising, given that the trainees had much else to do. Of greater significance was the fact that there was a clear coincidence between online activity and the seminar task, although
20
this trend was more marked for Group 1 than for Group 2. Thus, whilst it was not necessary to use email for the task, the indications were that it had proved helpful to do so, and for this activity more than any other.
-- Insert Table 1 about here --
Data on the relative contribution of different individuals confirmed and qualified the apparent impact of the seminar task. Use of the conference varied widely across trainees, from no contributions at all in two instances, to a maximum of 24, with an average of 6.5. Even taking initial connection problems into account, only five of the twelve could be classed as regular users (i.e. those who made a number of contributions over a period of time). However, the number of contributors did increase during the seminar task, although there were further differences in this respect between Group 1 and Group 2 (see Table 1): for the former, four out of the six group members became regular contributors for the period preceding their face-to-face seminar; for the latter it was three at most, and then more briefly. This pattern contrasts markedly with the participation observed in the face-to-face seminar. It is of course possible that trainees divided into "movers and shakers" and "lurkers", for the set task at least, and these were unevenly distributed across the two groups. In fact, one person did acknowledge lurking in a message to the conference. However, they were a member of Group 1, not Group 2. It also seems unlikely that any lack of engagement extended to all aspects of the task, since there were none of the claims about "passengers" which would typically accompany this. The alternative, then, is that all members of both groups were contributing to their seminar task, but a number of Group 2 were doing so outside of the online conference.
Codings of the function and length of messages helped fill out the emerging picture. All messages were categorised in one of nine ways: 1) connection and hardware/ software issues; 2) seminar-related matters; 3) tutor contact and advice; 4) peer
21
contact and advice; 5) social exchange; 6) project work; 7) conference preparation; 8) course administration; and 9) professional issues. The results confirmed the impact of the seminar task: for six of the eight weeks leading up to the face-to-face seminars for Group 1 and then Group 2, communications relevant to the task were amongst the most frequent categories of message, and average length jumped from 1K to 8K (see Table 1). These seminar-related messages were of two kinds: brief exchanges of information, or attachments of draft sections with requests for feedback (hence the jump in length). Thus use of the conferencing system did not parallel face-to-face discussion, making any departure from face-to-face patterns of interaction less remarkable. Moreover, whilst Group 2 made fewer online contributions, the characteristics of their seminar-related messages were similar to Group 1's. In other words, email served the same type of function for both groups, if not to the same extent.
Outside the seminar task, the system had some usage for project work and conference preparation, and occasional social, advice, and course administration messages occurred throughout. In each case, information exchange was the primary objective. Overlaying all else, however, were exchanges about software and network connections, pointing up problems, or announcing their solution. During 11 of the 17 weeks from the start-up of the system until the summer vacation this was the most frequent or joint most frequent category of message. Thus all other activity took place against a background of perceived system difficulties.
A final source of information about interaction was provided by logs of contact and activity which the trainees were asked to keep in relation to the seminar task. These consisted of proformas for noting down relevant activity, who else (if anyone) was involved, the medium used, the date, and the time taken. The logs provided information on the crucial issue of how online communication fitted into other activity and contact, and although the return rate was patchy, comparison between
22
completed logs for Group 1 and Group 2 was highly informative vis à vis the differences between their online activity.
-- Insert Table 2 about here --
For Group 1, four logs were returned, from which was calculated the mean time spent by group members on each of four broad categories of activity: 1) face-to-face communication; 2) email communication (including failed attempts); 3) telephone communication (including sending of faxes); and 4) independent activity. As Table 2 shows, members of Group 1 spent roughly the same time on face-to-face and email communication, made little use of the telephone, and devoted the largest proportion of work to independent activity, such as accessing information and preparing draft sections of the seminar paper. Face-to-face communication took place at points of negotiation, most typically at the outset, whilst email was used more for the exchange of drafts, and updating on progress. This confirms email was used mainly for information exchange, whilst discussion was reserved for face-to-face contact, perhaps because there was opportunity throughout for such contact (cf. Tucker et al, 1997).
For Group 2, only two logs were returned, but these are revealing. Total time devoted to the seminar task was similar to Group 1, but proportionately much less was spent on email, in line with the message data, and more on face-to-face and fax communication (see Table 2). Face-to-face communication for Group 2 included working together, on top of the strategic uses made by Group 1. Thus lack of email contact via the shared conference did not signify lack of engagement with the task, but a use of other methods of working and other channels of communication. The reason for these differences is not immediately apparent, but one factor which might have been significant was that Group 2 reported more failures in sending attachments,
23
in part because of mismatches between the word processing software on the group members' machines.
Trainees' perceptions of the system
In order to obtain direct information on trainees' perceptions of the conferencing system, groups were asked to complete a questionnaire after the seminar task. This covered past experience of email; the perceived usability of the system, its main advantages or disadvantages, and what other support they felt might have been useful; which aspects of the seminar task had been seen as most useful, and which as least; and whether there were any surprising or disappointing outcomes. There were five respondents for Group 1, of whom only one reported having utilised email previously. All respondents reported having experienced access problems due to network/modem difficulties or lack of availability of machines, and all felt more instruction (about attachments especially) would have been helpful. However, there were positive aspects. Everyone liked the speed of exchange which email allowed, and most also mentioned the convenience of asynchronicity. All felt learning about conferencing was the most useful aspect of the seminar task, and that keeping in touch or discussing were the least useful (cf. the relative lack of use of the conference for this, and the availability of face-to-face contact). The successful joint production of the seminar paper was spontaneously mentioned by almost all as a surprising, and positive outcome.
For Group 2 also, five questionnaires were returned. Two respondents had had previous experience with email, confirming that the differences between Group 1 and Group 2 were not attributable to the latter containing fewer experienced users. In general, the pattern of responses was similar to those given by Group 1. However, there was a greater emphasis on technical problems, no mention of the benefits of asynchronicity (perhaps reflecting paucity of use of the system), and a less positive overall reaction. One other point of interest was that the group reportedly split into
24
pairs to carry out the seminar task, hence the greater incidence of face-to-face working noted in the logs, and the lower use of email. The implication is that technical difficulties led Group 2 to adopt a structure for the seminar task which minimised the need for online exchange, and this was the reason for the differences in interaction patterns between Group 1 and Group 2.
Implications and conclusions
The activity theory approach predicted that: 1) the characteristics of the trainees and the task they were set would help them establish a shared purpose which would motivate online interaction; 2) the CMC usage that occurred would reflect the requirements of this shared purpose; and 3) the conferencing system’s success would be a function of how far it helped the shared purpose to be achieved. The first prediction requires consideration of whether trainee and task characteristics did promote interaction via CMC; and, if so, whether this was mediated by the emergence of a shared purpose. Certainly, group size was small, knowledge of others and students' level of experience (in similar contexts) were both good, and trainees' face-to-face interaction was consistent with these factors operating as expected. As far as CMC was concerned, the rate of contribution per individual was patchy and overall usage was not high, but this is attributable in part to the trainees' workload and opportunity for face-to-face contact. The effect on message frequency of the seminar task demonstrates that they did communicate online when there was reason, underscoring the particular influence of clear tasks. In the absence of this, use was minimal, for all the trainees' facilitating characteristics. Individual differences in prior experience of CMC had no direct impact, and despite perceptions that more training would have been helpful, when there was a need e.g. to send attachments, Group 1 at least worked out how to do so. The influence of task ownership is visible here: when the group themselves wanted to achieve something using CMC, they made it happen. Overriding all other influences, though, were the perceived need for the system, and the type of system in the sense of its accessibility and reliability. All the trainees had
25
problems with the system, but Group 1 saw an important role for it in spite of these, especially as part of the seminar task, and so used it. Group 2 apparently mistrusted it, declined to give it a role, and so used it very little.
Overall, then, the identified factors did have much of the expected impact on CMC use. This leaves open the question of whether this was a function of the factors helping the trainees establish a shared purpose; and also whether use of the conferencing system and its perceived value were driven by the nature of this purpose (the second and third predictions). It is necessary to concede a reliance on indirect data on these points, since the trainees were not specifically questioned on their perceived objectives and the extent to which these were shared (a limitation that should be rectified in subsequent research). This said, the strong relationship of the seminar task to online activity, and more particularly the way in which the groups tackled it both provide evidence for the role of shared purpose. That the trainees all saw the object of the task as being to produce a paper for the face-to-face seminar is unsurprising, since it was explicitly stated to be their goal. However, a working activity system requires agreement about not just the object of the activity, but also the actions to be taken to achieve that object (cf. Lewis, 1997), and 'shared purpose' therefore entails both. The central point, then, is that the evidence of the logs of contact and activity suggests that: a) both groups used initial face-to-face contact to establish their object and working procedures; and b) consistent with the predicted benefits of small group size, good knowledge of each other, high level of expertise, clarity about task and sense of ownership, the time spent on this was brief (although there were later points of renegotiation).
Thus, Group 1 decided to work individually on different aspects of their paper, and to use the system to exchange drafts of these and piece the final product together; whereas Group 2 agreed to work face-to-face in pairs, to achieve the same end. In line with the shared purpose account, both groups were consistent in using their agreed
26
procedures, there being clear differences in activity between, but similarities among them. That it was these working procedures which mostly defined use of the conferencing system is consistent with the second prediction. In support of the third prediction, not only did Group 1 make more use of the system, having given it a central role, they also had more positive perceptions of its value, seeing it as having contributed to a successful outcome.
In general, then, there is good support for the activity theory perspective, and for the role of shared purpose in defining system use. What is less clear is why the two groups differed so markedly in the working procedures they adopted, given that they had the same goal and similar backgrounds and experiences. In part, this is because there has been a tacit assumption thus far that the groups made firm, clear choices to adopt their different procedures, and this may be overly simplistic. One sign that it is unlikely to have been the case is the fact that Group 2 did make tentative efforts to use the system in the same way as Group 1. Similarly, the renegotiation points and their surprise at achieving the goal both suggest that Group 1 did not start out with a fully developed model of how to use the system effectively for joint writing.
Instead, then, it seems likely that both groups had clear ideas about how joint writing should be conducted, based on past experience, but, pursuing the suggestion of the course director that the conference could be used to support this activity, they tried to work out how to adapt their existing practices to CMC. They may have been aided in this by the affordances the system offered for the exchange of text: if affordances are secondary to purpose they may not be perceptible in the absence of a goal, but once an object and set of actions have been established, it will be easier to envisage how those actions could be supported by novel operations. This suggests that Group 1 exhibited a transformation of an existing activity system, but Group 2 failed to achieve this, because it depended on the reliability of the system, as well as on its affordances. Thus the division between the groups may have come down in the end to a marginal
27
difference in just this respect: both saw the system as unreliable, but Group 1 found it trustworthy enough to persevere, and Group 2 did not. It is worth noting the ease with which the perceived instability of the system deflected Group 2 "off-course". Although lack of prior experience with email did not appear to be of much consequence in itself, the interaction of this with small differences in perceived reliability may have affected the outcome. New users are perhaps in this sense unstable themselves: any influence, especially a negative one, has a larger impact.
There are a number of pointers for future practice flagged up by this case study. Firstly, it underlines the need to plan implementations within which students can readily establish a shared purpose. The discussion above identifies factors and processes that are likely to be important in achieving this. Secondly, though, there is a need to secure the reliability of the system across all points of use at the outset (see Watson, Blakely & Abbott, 1998, on this point), especially if the students involved are relatively inexperienced. It may also be helpful to establish the optimum operating parameters of the system, including e.g. the format for attachments; and to ensure that students are made familiar with these. Thirdly, even when these criteria are met, the nature of the interaction using the system will be heavily dependent on what the shared purpose is. In particular, 'productive discussion' will only occur if it fits that purpose, or if a degree of conflict arises en route. Given students' greater familiarity with concrete tasks than with discursive ones (Crook & Webster, 1997), if they are left to negotiate what they do for themselves, it is perhaps unlikely that their use of CMC will involve transactive discussion. It might be possible to engineer it, given research showing that conflict and discussion are promoted by task structures which require individuals to make judgements and then come to a group consensus on these (e.g. Tolmie et al., 1993; Tolmie & Anderson, 1998). There are certainly examples of CMC being successfully used in this way (e.g. McAteer et al., 1997), although the imposition of such structures could undermine task ownership and genuine shared purpose. The same outcome might be achieved without this risk if students are set a
28
familiar objective where they can reliably be expected to disagree over how to reach it, but such disagreement may in practice be hard to predict.
The main point is to beware of rigid expectations of outcomes. Transactive exchanges will occur if tasks require them, but other types of activity may also be productive. This brings us back to the methodological issues raised earlier. On the question of what measures to use to identify effective resources, the study illustrates that tying effectiveness to specific functions is likely to be unhelpful. Discursive indices alone, for instance, would have yielded misleading conclusions, since the main CMC function that emerged was not of this kind, and yet it was clearly valued, at least when it got off the ground. Thus, from a methodological as well as a pedagogical viewpoint, it is important to be open to any activity which is consistent with task progress, and this may not be predictable. As to the data collection techniques to use to achieve this, the study outlines one package of possible methods and measures. Rather than dwelling on the specific merits of these, though (see Oliver & Conole, 1998b, for a comprehensive treatment of potential methods), it is more useful to focus on the broad message that to get a rounded picture of CMC activity it is necessary to use a variety of measures, and these must include detail about its context, including offline activity. In the absence of this information, online indices are barely interpretable, for example leaving Group 2 looking like lurkers or passengers, whereas with contextual information it can be seen that they merely adopted a different (but appropriate) set of working procedures. This "back-up" also permits cross-validation between measures and compensation for the inevitable gaps in data collected in natural settings (see Breen et al., 1998). In the present case, for instance, given their return rate, the logs of contact and activity would have been less useful without the information on message frequency.
Turning to the issue of control and confirmatory bias, it has already been noted that the system reported on was set up to work, and broadly speaking it did. The question
29
is whether, in the absence of any direct point of comparison, let alone a controlled one, this provides valid information about the factors that were "manipulated". In fact, there are at least two reasons to argue that it does. The first is that sufficient data was collected to thoroughly examine the system in use, and it could then be considered whether these observations made sense in terms of the predicted operation of the factors and the underlying theoretical framework. The second is that the resource did not actually work perfectly, and the same data permitted consideration of whether that outcome was also explicable within this framework. This approach bears strong similarities to what can be called the "engineering model" (see O'Neill, 1991): i.e., in broad terms, keep doing what works (often bigger) until it fails, collect enough data to work out why, and start over again. There is a long tradition in the applied sciences of "confirmatory hypothesis testing" of this kind, not as a result of poor scientific methods, but because this is all practicalities allow. This tradition blends scientific and historical techniques, because it is desirable both to obtain rigorous data, and to monitor the object of enquiry over time, since no final end point (no defining experiment, no perfect bridge) can ever be arrived at. The complexities of this approach are increased when applied to humans, because they are self-adaptive (they can examine their own data and change themselves), but it seems eminently applicable to research in educational contexts.
However, an interpretative framework to construe data, and one that can be modified (or rejected) at points of failure, is an essential part of this. For the moment, as far as CMC is concerned, activity theory appears to work well in this regard, and better than the older deficit model. There is still much to be understood, though, and fed back into practice. There is a need in particular to look in more detail at the negotiation and renegotiation (however conducted) of goals and working procedures, and to identify ways of unobtrusively supporting this process, in order to promote more effective use of CMC across the board in HE settings.
30
Acknowledgements
Thanks are due to: the MSc Educational Psychology trainees at Strathclyde University for their willingness to be involved in this research; Ingeborg Stobie for her role in the development of the Educational Psychology CMC resource; Jen Harvey, for her advice and assistance during data collection; and Erica McAteer, for her many and invaluable comments on earlier versions of this paper and the arguments developed therein. A version of this paper was also presented at a School of Cognitive and Computing Sciences Seminar, University of Sussex, in October 1998, and the authors gratefully acknowledge the valuable comments and suggestions made by the audience on that occasion. Some elements of the data reported above were initially published in N. Mogey and the Learning Technology Dissemination Initiative Team (Eds.), LTDI Evaluation Studies, Heriot-Watt University, 1998.
31
References
Ackerman, M. (1997). Communication and collaboration from a CSCW perspective. Contribution to XMCA Discussion List (xmca@weber.ucsd.edu), 8.3.97.
Ambrosius, J. (1992). Hardware and software architecture in computer conferencing systems. In A.R. Kaye (Ed.), Collaborative Learning through Computer Conferencing: The Najaden Papers. Berlin: Springer-Verlag.
Barley, Z. & Jenness, M. (1993). Cluster evaluation: a method to strengthen evaluation in smaller programs with similar purposes. Evaluation Practice, 14, 141-147.
Bates, A.W. (1995). Technology, Open Learning and Distance Education. London: Routledge.
Beattie, G.W. (1982). The dynamics of university tutorial groups. Bulletin of the British Psychological Society, 35, 147-150.
Berkenkotter, C. (1997). Re: Affording. Contribution to XMCA Discussion List (xmca@weber.ucsd.edu), 20.3.97.
Berkowitz, M.W. & Gibbs, J.C. (1983). Measuring the developmental features of moral discussion. Merrill-Palmer Quarterly, 29, 399-410.
Breen, R., Jenkins, A., Lindsay, R. & Smith, P. (1998). Insights through triangulation: combining research methods to enhance evaluation of IT-based learning methods. In M. Oliver (Ed.), Innovation in the Evaluation of Learning Technology. London: LTID, University of North London.
32
Calvani, A., Sorzio, P. & Varisco, B.M. (1997). Inter-university cooperative learning: an exploratory study. Journal of Computer Assisted Learning, 13, 271-280.
Canning, C. & Swift, K. (1992). Connecting the university and the field of practice: computer conferencing in education at the University of Michigan. In M.D. Waggoner (Ed.), Empowering Networks: Computer Conferencing in Education. Englewood Cliffs, N.J.: Educational Technology Publications.
Cole, M. (1996). Cultural Psychology: A Once and Future Discipline. Cambridge, MA: Harvard University Press.
Cole, M. (1997). IRE/internet. Contribution to XMCA Discussion List (xmca@weber. ucsd.edu), 20.3.97.
Crook, C. (1988). Electronic media for communications in an undergraduate teaching department. In D. Smith (Ed.), New Technologies and Professional Communications in Education. London: National Council for Educational Technology.
Crook, C. (1994). Computers and the Collaborative Experience of Learning. London: Routledge.
Crook, C. (1997). Making hypertext notes more interactive: undergraduate reactions. Journal of Computer Assisted Learning, 13, 236-244.
Crook, C. (1998). Computer networks in education. The Psychologist, 11, 378-380.
Crook, C. & Webster, D.S. (1997). Designing for informal undergraduate computer mediated communication. Active Learning, 7 (December 1997), 47-51.
33
Daniel, J.S. (1996). Mega-Universities and Knowledge Media: Technology Strategies for Higher Education. London: Kogan Page.
Davies, D. (1995). Learning network design: coordinating group interactions in formal learning environments over time and distance. In C. O'Malley, (Ed.), Computer Supported Collaborative Learning. Berlin: Springer-Verlag.
Dede, C.J. (1991). Emerging technologies: impacts on distance learning. Annals of APPSS, 514, 146-158.
Doise, W. & Mugny, G. (1984). The Social Development of the Intellect. Oxford: Pergamon.
Draper, S.W. & Brown, M. (1998). Evaluating remote collaborative tutorial teaching in MANTCHI. In M. Oliver (Ed.), Innovation in the Evaluation of Learning Technology. London: LTID, University of North London.
Drysdale, J. & Creanor, L. (1998). Leading new teachers to learning technology. In N. Mogey & LTDI Team (Eds.), LTDI Evaluation Studies. Heriot-Watt University, Edinburgh: Learning Technology Dissemination Initiative.
Engestrom, Y. (1987). Learning by expanding: An activity-theoretical approach to developmental research. Helsinki: Orienta-Konsultit.
Friedman, L.B. & McCullough, J. (1992). Computer conferencing as a support mechanism for teacher-researchers in rural high schools. In M.D. Waggoner (Ed.), Empowering Networks: Computer Conferencing in Education. Englewood Cliffs, N.J.: Educational Technology Publications.
34
Gaver, W.W. (1992). The affordances of media spaces for collaboration. In CSCW ‘92: Sharing Perspectives. Proceedings of the Conference on Computer Supported Cooperative Work, Toronto.
Graddol, D. (1989). Some CMC discourse properties and their educational significance. In R. Mason & A. Kaye (Eds.), Mindweave: Communication, Computers and Distance Education. Oxford: Pergamon.
Happé, F.G.E. (1993). Communicative competence and the theory of mind in autism: a test of relevance theory. Cognition, 48, 101-119.
Harasim, L. (1989). Online education: a new domain. In R. Mason & A. Kaye (Eds.), Mindweave: Communication, Computers and Distance Education. Oxford: Pergamon.
Harasim, L. (1990). On-Line Education: Perspectives on a New Environment. New York: Praeger.
Henri, F. (1995). Distance learning and computer-mediated communication: interactive, quasi-interactive or monologue? In C. O'Malley, (Ed.), Computer Supported Collaborative Learning. Berlin: Springer-Verlag.
Howe, C.J., Tolmie, A., Anderson, A., & Mackenzie, M. (1992). Conceptual knowledge in physics: the role of group interaction in computer-supported teaching. Learning & Instruction, 2, 161-183.
Issroff, K. & Eisenstadt, M. (1997). Evaluating a virtual summer school. Journal of Computer Assisted Learning, 13, 245-253.
35
Jones, C. (1998). Evaluation through ethnography: context, content and collaboration. In M. Oliver (Ed.), Innovation in the Evaluation of Learning Technology. London: LTID, University of North London.
Kaye, A. (1989). Computer-mediated communication and distance education. In R. Mason & A. Kaye (Eds.), Mindweave: Communication, Computers and Distance Education. Oxford: Pergamon.
Kaye, A.R. (1992a). Computer conferencing and mass distance education. In M.D. Waggoner (Ed.), Empowering Networks: Computer Conferencing in Education. Englewood Cliffs, N.J.: Educational Technology Publications.
Kaye, A.R. (1992b). Learning together apart. In A.R. Kaye (Ed.), Collaborative Learning through Computer Conferencing: The Najaden Papers. Berlin: Springer-Verlag.
Kaye, A.R. (1995). Computer supported collaborative learning in a multi-media distance education environment. In C. O'Malley, (Ed.), Computer Supported Collaborative Learning. Berlin: Springer-Verlag.
Kelly, G.A. (1963). A Theory of Personality: The Psychology of Personal Constructs. New York: W.W. Norton.
Kiesler, S., Siegel, J. & McGuire, T. (1984). Social psychological aspects of computer-mediated communications. American Psychologist, 39, 1123-1134.
Kirkup, G. (1996). The importance of gender. In R. Mills & A. Tait (Eds.), Supporting the Learner in Open and Distance Education. London: Pitman.
36
Krauss, R.M. & Fussell, S.R. (1990). Mutual knowledge and communicative effectiveness. In J. Galeghar, R. Kraut & C. Egido (Eds.), Intellectual Teamwork: Social and Technological Foundations of Cooperative Work. Hillsdale, NJ: Lawrence Erlbaum.
Leach, J. (1996). Learning in practice: support for professional development. In R. Mills & A. Tait (Eds.), Supporting the Learner in Open and Distance Education. London: Pitman.
Leont'ev, A.N. (1978). Activity, consciousness, and personality. Englewood Cliffs: Prentice-Hall.
Leont'ev, A.N. (1981). Problems of the development of the mind. Moscow: Progress.
Levinson, P. (1989). Media relations: integrating computer telecommunications with educational media. In R. Mason & A. Kaye (Eds.), Mindweave: Communication, Computers and Distance Education. Oxford: Pergamon.
Lewis, R. (1997). An Activity Theory framework to explore distributed communities. Journal of Computer Assisted Learning, 13, 210-219.
Light, P., Colbourn, C. & Light, V. (1997). Computer-mediated tutorial support for conventional university courses. Journal of Computer Assisted Learning, 13, 228-235.
McAteer, E., Tolmie, A., Duffy, C., & Corbett, J. (1997). Computer mediated communication as a learning resource. Journal of Computer Assisted Learning, 13, 219-227.
37
McConnell, D. (1988). Group communications via computer conferencing: the educational potential. In D. Smith (Ed.), New Technologies and Professional Communications in Education. London: National Council for Educational Technology.
McNeill, D.P. (1992). Computer conferencing: the causes for delay. In M.D. Waggoner (Ed.), Empowering Networks: Computer Conferencing in Education. Englewood Cliffs, N.J.: Educational Technology Publications.
Mason, R. (1989). An evaluation of CoSy on an Open University course. In R. Mason & A. Kaye (Eds.), Mindweave: Communication, Computers and Distance Education. Oxford: Pergamon.
Mason, R. & Bacsich, P. (1998). Embedding computer conferencing into university teaching. Computers & Education, 30, 249-258.
Morgan, D.L. & Schwalbe, M.L. (1990). Mind and the self in society: linking social structure and social cognition. Social Psychology Quarterly, 53, 148-164.
Nalley, R. (1995). Designing computer-mediated conferencing into instruction. In Z.L. Berge & M.P. Collins (Eds.), Computer Mediated Communication and the Online Classroom, Volume II: Higher Education. Cresskill, N.J.: Hampton.
Odasz, F. (1992). Grassroots networking on Big Sky Telegraph: empowering Montana's one-room rural schools. In M.D. Waggoner (Ed.), Empowering Networks: Computer Conferencing in Education. Englewood Cliffs, N.J.: Educational Technology Publications.
38
Oliver, M. & Conole, G. (1998a). The evaluation of learning technology: an overview. In M. Oliver (Ed.), Innovation in the Evaluation of Learning Technology. London: LTID, University of North London.
Oliver, M. & Conole, G. (1998b). Evaluating communication and information technologies: a toolkit for practitioners. Active Learning, 8 (July 1998), 3-8.
O'Neill, W. (1991). Bridge design stretched to the limits. New Scientist, No. 1792, 36-43.
Philips, G.M. & Santoro, G. (1989). Teaching group discussion via computer-mediated communication. Communication Education, 38, 151-161.
Piaget, J. (1932). The Moral Judgement of the Child. London: Routledge.
Pincas, A. (1995a). Views of learning based on the experience of computer conferencing. In C. O'Hagan (Ed.), Empowering Teachers and Learners through Technology. Birmingham: SEDA Publications.
Pincas, A. (1995b). Analysis of face-to-face and computer conferencing interactions. In C. O'Hagan (Ed.), Empowering Teachers and Learners through Technology. Birmingham: SEDA Publications.
Rada, R. (1998). Efficiency and effectiveness in computer-supported peer-peer learning. Computers & Education, 30, 137-146.
Rapaport, M. (1991). Computer Mediated Communications. New York: John Wiley.
39
Riel, M.M. & Levin, J.A. (1990). Building electronic communities: successes and failures in computer networking. Instructional Science, 19, 145-169.
Rutter, D.R., Stephenson, G.M. & Dewley, M.E. (1981). Visual communication and the content and style of conversation. British Journal of Social Psychology, 20, 41-52.
SHARP Project Team (1998). Sharable Representations of Practice - An Introduction to the SHARP Project. URL: http://www.lancs.ac.uk/users/edres/research/sharp/index.htm
Silvennoinen, H. & Kerttula, E. (1994). The TOMU project: a distance learning programme on computer science and information systems using e-mail for computer-mediated communication. In W. Veen, B. Collis, P. de Vries & F. Vogelzang (Eds.), Telematics in Education: The European Case. De Lier: Academic Book Centre.
Sinclair, J.McH. & Coulthard, R.M. (1975). Towards an Analysis of Discourse. London: Oxford University Press.
Stainfield, J. (1997). Using IT to manage a third year module. Active Learning, 6 (July 1997), 30-34.
Steeples, C., Goodyear, P. & Mellar, H. (1994). Flexible learning in Higher Education: the use of computer-mediated communications. Computers & Education, 22, 83-90.
Steeples, C., Unsworth, C., Bryson, M., Goodyear, P., Riding, P., Fowell, S., Levy, P. & Duffy, C. (1996). Technological support for teaching and learning: computer-mediated communications in Higher Education (CMC in HE). Computers & Education, 26, 71-80.
40
Thomas, R. (1989). Implications of electronic communication for the Open University. In R. Mason & A. Kaye (Eds.), Mindweave: Communication, Computers and Distance Education. Oxford: Pergamon.
Tolmie, A. & Anderson, A. (1998). Information technology and peer-based tutorials. The Psychologist, 11, 381-384.
Tolmie, A., Howe, C.J., Mackenzie, M. & Greer, K. (1993). Task design as an influence on dialogue and learning: primary school group work with object flotation. Social Development, 2, 183-201.
Trentin, G. (1997). Tellematics and on-line teacher training: the Polaris Project. Journal of Computer Assisted Learning, 13, 261-270.
Trushell, J., Reymond, C. & Burrell, C. (1998). Undergraduate students' use of information elicited during e-mail "tutorials". Computers & Education, 30, 169-182.
Trushell, J., Reymond, C., Herrera, R. & Dixon, P. (1997). Undergraduate students' use of information communicated during e-mail "tutorials". Computers & Education, 28, 11-21.
Tucker, D, Whately, J. & Hackney, R. (1997). Group project work through information technology: issues of infrastructure and interaction. Active Learning, 7 (December 1997), 30-34.
Watson, D., Blakely, B. & Abbott, C. (1998). Researching the use of communication technologies in teacher education. Computers & Education, 30, 15-21.
41
Wells, G. (1997). From guessing to predicting: progressive discourse in the learning and teaching of science. In C. Coll & D. Edwards (Eds.), Teaching, Learning and Classroom Discourse. Madrid: Fundacion Infancia y Aprendizaje.
Wibe, J. (1994). The SPINN project: Norway. In W. Veen, B. Collis, P. de Vries & F. Vogelzang (Eds.), Telematics in Education: The European Case. De Lier: Academic Book Centre.
Wilson, T. & Whitelock, D. (1998a). What are the perceived benefits of participating in a computer-mediated communication (CMC) environment for distance learning computer science students? Computers & Education, 30, 259-269.
Wilson, T. & Whitelock, D. (1998b). Monitoring the on-line behaviour of distance learning students. Journal of Computer Assisted Learning, 14, 91-99.
Wolf, D.P. (1988). The quality of interaction: domain knowledge, social interchange, and computer learning. In G. Forman & P.B. Pufall (Eds.), Constructivism in the Computer Age. Hillsdale, N.J.: Lawrence Erlbaum.
42
Table 1. Frequency of online messages week-by-week (key time points marked), together with mean message length, most frequent message categories, and number of contributors from Group 1 and Group 2.
Week
no.
Key time
points
No. of
messages
Mean
message
length
Most frequent
message categories
No. of
Group 1
contributors
No. of
Group 2
contributors
1
8
1k
1) connection;
2) social
1
0
2
2
1k
1=) tutor, social
0
0
3
4
1k
1) connection
0
1
4
5
2k
1) connection;
2) tutor
1
0
5
Group 1
task set
0
-
-
0
0
6
1
1k
1) professional
0
0
7
2
1k
1=) professional,
connection
2
0
8
1
1k
1=) seminar, social
1
0
9
4
5k
1=) connection,
course admin, social
2
0
10
8
6k
1) connection;
2) social;
3) seminar
4
0
11
Group 2
task set
20
17k
1) connection;
2) seminar
3
3
12
All now
connected;
Group 1
face-to-face
seminar
14
4k
1) social;
2) connection;
3) seminar
4
2
13
3
1k
1=) seminar, connection,
social
0
2
14
3
1k
1) social;
2) seminar
0
2
15
3
2k
1) social
1
1
16
Group 2
face-to-face
seminar
4
7k
1) connection;
2) seminar
2
1
17
2
2k
1=) connection,
conference
1
1
18
1
1k
1) social
1
0
19
3
1k
1) social
1
1
20
Start of
summer
vacation
0
-
-
0
0
21
1
17k
1) course admin
0
0
22
0
-
-
0
0
23
0
-
-
0
0
24
1
1k
1) conference
0
1
25
0
-
-
0
0
26
1
1k
1) conference
1
0
43
Table 2. Average time (in minutes) spent by members of Group 1 and Group 2 on different activities whilst working on the seminar task.
Activity
Group 1
Group 2
Face-to-face communication
119
200
Email communication
97
21
Telephone communication
12
29
Independent activity
281
242