VOL. 15, No. 1, 39-62
This study focuses on the feasibility of using authentic problem-based collaboration at a distance. In order to understand this question fully, we compared a traditional residential introductory instructional design (ID) course with another that was offered as a modified distance education course. Students from a traditional university (Trad U) in an introductory ID course were assigned a major complex ID project with an authentic context and SME to work with. The same faculty member gave students from a distance education institution (Dist U) a similar project with similar resources. Data collected included background surveys, reflective student journal entries, e-mail records, pre- and post-individual interviews, classroom observations, and actual class products from both groups. In addition, for the distance education group audioconference transcripts, IRC chat records, and Web discussion board artifacts were collected and analyzed. The study found that it is possible to enact authentic learning and distance collaboration within modified distance education when learners are advanced in their studies, have had previous experiences collaborating at a distance with smaller projects, and are prepared for the challenge of authentic experiences by seeing a strong relevance to their own work.
Cette étude se concentre sur la « faisabilité » de la résolution de problème collaborative et authentique à distance. Pour mieux comprendre cette question, nous avons comparé un cours conventionnel d'introduction au design pédagogique (DP) avec un autre, modifié de façon à être offert à distance. Dans le cadre d'un cours d'introduction au DP, des étudiants d'une université traditionnelle (UT) avaient à réaliser un projet d'envergure assez complexe s'appuyant sur le contexte authentique d'une moyenne entreprise. Le même enseignant a donné aux étudiants d'une institution à distance un projet semblable avec des ressources semblables. Les informations ont été recueillies grâce à des sondages, des réflexions inscrites par les étudiants dans leur journal de bord, des traces de courriel, des entrevues individuelles, avant et après, des observations en classe et des travaux du cours réalisés par les deux groupes. De plus, dans le cas du groupe à distance, nous avons recueilli et analysé les transcriptions d'audioconférence, la trace des séances de bavardage (Chat) et des artefacts résultant des forums de discussion sur le Web. L'étude a démontré qu'il est possible de concrétiser un apprentissage collaboratif authentique à distance quand les apprenants sont suffisamment avancés dans leurs études, quand ils ont déjà une expérience préalable de collaboration à distance dans des projets de moindre importance, quand ils sont préparés au défi des expériences authentiques et capables d'en apprécier la pertinence dans leur propre travail.
This study examines the potential for collaborative, authentic problem-based learning and solving at a distance. As educators we have begun the task of learning about and understanding what distance education is and is not capable of and in particular how Web-based education, the least expensive1 form of distance education, can be employed to best advantage (Khan, 1997). This study was undertaken to explore the possibilities for distant, problem-based collaboration among student groups of 4-5 learners in both distance and traditional residential instruction. The primary research question focused on the viability of authentic projects in distance collaboration. Informing questions included what the experience was like for both groups of students, the particular obstacles they encountered and positive experiences they enjoyed, and the distinct differences between the two groups in terms of previous experiences, and their motivations for seeking graduate education.
Distance education is one of the fastest growing areas of education today (Daniel, 1996). Distance education, Web-based instruction, and virtual classrooms are all popular new forms of education being adopted at all levels of schooling, and they are generating a great deal of interest in the instructional technology R&D community (Kahn, 1997, in press; Hackbarth, 1997; Hiltz & Wellman, 1997). Particularly at the graduate level, distance education models provide access to working and nontraditional students and efficient delivery of educational products from courses, to certificates, to degrees (Jones, 1997). Situated cognition, collaborative or cooperative learning, problem-based learning, and constructivist learning models are increasingly recognized as advances in human learning theory (Jonassen & Land, in press). What happens when we pose real problems to groups of students who have to collaborate with one another over long distances?
Web-based forms of educational delivery are appealing to administrators because potential economies of scale make delivery efficient, and it is attractive to instructors because continuing advances in development tools enable quick development and student use. Encouraging a student-centered, learning-focused approach to distance education, however, has been difficult in the face of increasing pressures by administrators in higher education to get courses online quickly. Because the use of cases, problem-based learning, and authentic approaches require much more instructional development and preparation, much of the Web-based education today is reverting to more and more information-rich instruction—some might even say dispensation. Although some work has been done in distance education and collaborative communication (Lopez & Nagelhout, 1995; Heeren, 1996; Hiltz, Johnson, & Turoff, 1986; Harasim, 1994), substantive evaluations for the purposes of decision-making on the effectiveness of distance education for collaborative learning have not yet been adequately addressed. Heeding the calls of constructivism (Jonassen, 1991), situated cognition (Brown & Duguid, 1994), cooperative learning (Johnson, Johnson, & Holubec, 1984), and problem-based learning (Septien & Gallagher, 1993; Barrows, 1986) in the context of distance education is quite challenging.
The research in distance education has little to say about authentic problem-based collaboration at a distance. Some of the recent studies in distance education have found that videoconferencing and other older technologies are not adequate to meet the challenges of higher-order learning and thinking in accordance with newer learning theories. Larson and Bruning (1996) as well as Barker (1987) find that students prefer the face-to-face mode, particularly where collaboration is a course expectation. Jonassen, Davidson, Collins, Campbell, and Bannan Haag (1995) advocate for computer-mediated communication in distance education as a method to reach constructivistic learning goals. Although some policy-makers and distance education advocates see technology itself as a way to transcend the distance while still maintaining constructivist and problem-based learning approaches (Mingle, 1995), other theorists see pedagogy as the main element in improved learning rather than technological innovation (Seaton, 1993). Yakimovicz and Murphy (1995) and Seaton recognize that CMC technologies are more appropriate for collaborative and constructivist goals than older forms of distance education. Seaton writes, “Computer-mediated communication has become a significant technology in distance education during the past decade. It has been cited as overcoming learner isolation ... One of the most successful pedagogical uses of CMC in establishing such learning communities at a distance is collaborative learning” (p. 51). However, such works do not address the benefits of CMC over face-to-face instruction for constructivistic ends, and none of this work approaches the problems of authentic problem-based collaboration at a distance.
Brookfield (1987) suggests that such critical thinking skills are central to self-directed learning, which is a core requirement of effective distance education. Some work has been conducted on the question of learner centeredness as it relates to self-directed learning such as Burge and Howard’s (1988) survey examining the attitudes and practices of Canadian distance educators. Abrami and Bures (1996) suggest that interaction is imperative for complex problem-solving and collaborative learning, and they describe computer-supported collaborative learning (CSCL) as encouraging only “asocial interaction.” They call into question whether students in CSCL environments can truly accomplish the goals set out for them in complex problem-solving tasks such as the authentic learning activities that are the focus of this study. “Consequently, asocial interactions may complicate the completion of collaborative tasks that require coordinated and concerted input of all group members. Students using CSCL for distance learning must develop strong commitments to the group and its task” (p. 41). Therefore, this study informs the literature in distance education on the viability of authentic problem-based, collaborative learning at a distance while also comparing this approach with face-to-face instruction and collaboration. This study is based on qualitative research paradigms (Lincoln & Guba, 1985) as well as recent developments in student-centered learning theories (Savery & Duffy, 1996; Wilson, 1996) and systemic approaches to distance education (Collis, 1996; Moore & Kearsley, 1996).
This study took place in two settings. The first, a traditional residential instructional systems program (Trad U) with six full-time faculty, four or five affiliated or adjunct faculty, and well over 100 graduate students at both the master’s and doctoral level. Many students in this residential program travel from remote parts of the state to take classes at night, and almost all courses are offered during the evening hours to facilitate working schedules. Students are given extensive choice among coursework, and sequence offering flexibility and requiring contact with an advisor. Along with the faculty, several full-time students with assistantships made up the primary core of a scholarly community. This community extended to the social community in the form of attendance at lectures, potluck poster sessions, and weekly happy hours at the local watering hole.
The second, a modified distance education institution (Dist U) houses two faculty full time and a large pool of adjunct faculty from all over the country. This adjunct faculty represents experts and emerging scholars from some of the finest programs in the field. During the study, full-time faculty were housed in a temporary university facility that took the form of a low-slung office building in an office park approximately a mile from the main campus buildings.2 Students in this modified distance education program visit campus three times during each year of the program for face-to-face instruction, which takes the form of three-, four-, or 10-day experiences with full-time or adjunct faculty. Students are asked to maintain enrollment with a cadre of colleagues and are not given choices as to their course sequence or course selections. As a result of the cadre model, this academic program enjoys a strong sense of identity within cadres and pride for the program as a whole. The community seems revitalized and strengthened with each cadre visit to campus for instruction. On-campus instruction takes place in university buildings or in local hotel conference rooms depending on availability of facilities.
The course, an introduction to instructional design, was taught by the same faculty member and facilitated or observed by two graduate students from the traditional residential program. The course is founded on the basic premise that authentic learning of instructional design principles is inherently advantageous to the learners as well as to the outside community (Carr, 2000). Real-world projects are the centerpiece of the course every semester, and all learning of basic concepts such as needs assessment, instructional strategies, instructional development, and evaluation is conducted within the context of the real-world ID problem. These problems are posed by subject matter experts (SMEs) from three contexts: higher education, K-12 education, and business and industry—the most likely contexts for students’ future employment.
Early in the course SMEs presented their problems to both the residential and distance students. In the traditional course all SMEs were present to describe their instructional problem. In the case of the distance education course, one SME happened to be in the area and was present in person, whereas the rest were all contacted and made their presentations by telephone conference call. As many as seven problems were posed to each class, and learners were asked to create their own groups of three or four colleagues. Problems included firefighting training in Colorado; FAA training in Washington, DC; course design for university faculty in libraries, mathematics, and fuel sciences; and professional development for teachers learning to use new technologies in two school districts.
In neither case was the purity of traditional residential (pure face-to-face without reliance on distance technologies such as e-mail, telephone, etc.) nor the purity of distance education (absence of any face-to-face interaction with complete reliance on distance technologies) models lived out. In fact, in several cases the residential student teams included members from remote locations as far as two hours away, and some residential teams had distant SMEs. However, the magnitude of the distance in the Dist U students’ case was substantially larger, opening a new set of issues such as time zones, which were not a part of the experience of the residential students. Because of market forces, the blurring of traditional residential and distance education becomes an important marker of adequately describing the context in which this study took place (Carr-Chellman, in press).3
This study employed a naturalistic inquiry orientation and used case study and interview methodologies (Stake, 1986; Lincoln & Guba, 1985; Merriam, 1988). Based on Collis’ (1995) categorization of three phases of computer-related research, our work would fit within the “describe-understand phase” or the early testing. As Heeren (1996) concluded, we also believed that “with regard to research on technology support for collaborative distance learning ... an exploratory research approach that aims at contributing to the `mosaic of insights’ that is in development in the domain” (p. 56) seems appropriate. Thus, although we try to draw conclusions based on this matched-case study that can inform decision-makers and administrators of distance learning programs, we also forward this work as part of an initial attempt to understand the broader context of authentic distance collaboration.
The first group of 28 learners were enrolled in Trad U’s Instructional Technology program at the graduate level and met as a group once a week for three-hour sessions in which problems were posed, mini-lectures were delivered, and groups worked on their problems.
The second group of 23 learners were enrolled in the Dist U’s Instructional Technology program, which is a modified distance education program. The program employs face-to-face meetings at its campus in Florida along with telephone, IRC chat, Web discussion boards, and e-mail as a management system. For this course all Dist U students met for a three-day intensive workshop in which there was problem-posing, lectures, and limited opportunity for in-person group work on the problems.
Both groups were enrolled in the basic instructional design course in the fall semester of 1997. Both groups were presented a similar array of problems for which they were to create a complete instructional design plan. The primary instructional difference between the groups was that the Dist U group dispersed after their three-day workshop and had to meet via phone, Web board discussions, e-mail, and IRC chats. It may be argued that the course was not explicitly redesigned for the purposes of distance or modified distance delivery; however, this study was an attempt to discover the viability of the use of collaborative, authentic learning strategies in distance education. The Dist U students had to solve their instructional design problems at a distance, whereas the Trad U. students were able to meet each week in person in class and were able to arrange some out-of-class, face-to-face meetings as well. There were additional differences between these student groups in terms of their place in program and previous experiences, as are explored in the results section of this study.
The two groups of students were followed for one semester. A research team made up of two graduate students and a faculty member in Instructional Design conducted the study. Surveys were administered early in the semester to determine entry-level skills and prior experience with group work, authentic problem-solving, and various aspects of instructional design. Volunteers from each group (12 from Dist. U. and 8 from Trad U.) were interviewed early in the semester to determine their prior experiences in more depth and were contacted late in the semester by e-mail questionnaire to get feedback on the problem-solving processes. Both groups were observed in class through focused participant observations conducted by the collaborating graduate students. Field notes were collected on these observations as well as a variety of documents such as class projects, problem solutions, e-mails, Web board discussion records, and promotional materials from both institutions. All documents were analyzed along with interview transcripts, student journals, surveys, audioconference transcripts, and observational field notes. This analysis was primarily content analysis (Weber, 1990) and thematic analysis, with both preconstructed and emergent themes resulting from the analysis. Survey results were tabulated and analyzed using SPSS by a member of the team.
We entered into this study without a strong set of preconceived notions about distance education. As instructional technologists for more than a decade and as change advocates, we were interested in finding out whether distance education would afford adequate opportunity for advanced learning theories and instructional methodologies to be lived out. We hoped in effect that this would be the case, but were concerned that it might not be. Each of us enjoys classroom contact with our students in the current traditional program. We see ourselves as enthusiastic, personable teachers and were afraid that distance education might change that self-perception. In the case of the first author, her technology skills were a little rusty and she was not wholly confident of their application to distance instruction. However, she was energized at the outset of this study by the opportunity to work with a new type of student in a new program, and we were all somewhat predisposed to a positive outcome of the study. Before the beginning of the course, there was an orientation session in which the lead instructor and first author were trained in all the technologies to be used in the distance delivery modes. As we lived through the experience of distance education, our hopefulness was tempered by the realities. However, throughout the data collection process, we suspended biases, journaling as a method for self-reflection. We continued to believe that there would be important learnings from this study in terms of what characteristics made authentic, collaborative distance problem-solving effective.
There was a complex interrelationship here that is important to sort through in this study. During the data analysis process the first author’s (and primary instructor for the course) view changed, and it is important to be honest with the reader that this was the case. As the data analysis process went on, she found herself mired in a discussion in her (residential) academic program about the wisdom of distance education as her program faculty took up the task of offering several courses on the Web through Penn State’s Wold Campus initiative. At this same time, she came to read many of the critics of distance education such as Noble (1998) and found strong opposition to a distance education approach in the traditional residential program. The first author felt negative partly because of the experience of distance educating, but at the same time she was analyzing the data of that experience. It cannot be said that these newfound opinions did not color the data that was highlighted during the analysis procedures. However, balance was continually sought, and the first author worked hard to maintain impartiality in reporting the data.4
Perhaps the most important outcome of this work is that the Dist U students were able to complete a complex instructional design project with equal success to that of the traditional residential students. Although we found that the Dist U students were farther along in their programs and were significantly different in terms of their previous levels of experience (see survey results below), this is nevertheless an important accomplishment. In particular, it indicates that it is possible to collaborate successfully on authentic problems at a distance. The important question becomes then, not whether it is in fact feasible to enact collaborative authentic learning at a distance, but rather what circumstances make such an instructional model effective.
A professional development survey was administered early in the semester to both groups. All 28 Trad U students filled out the questionnaire. Two of the 23 Dist U students did not answer all the questions and were excluded from the analysis. To analyze the results, t-tests for Equality of Means were used. Not surprisingly (because Dist U students had more completed coursework and experience), the Dist U students reported a significantly higher skill in Instructional Design (M=4.03) than did the Trad U students (M=3.26), t(47)=5.50, p<.00001. Looking at the subskills underlying the overall Instructional Design rating showed that the Dist U students scored significantly higher on general computer use, t(47)=2.66, p<.05, use of other technology in general, t(47)=2.80, p<.01, and their use of telecommunications in particular, t(47)=4.37, p<.0001. They also showed a higher assessment of their policy knowledge, t(47)=6.54, p<.0001, project management knowledge, t(47)=3.84, p<.0001, and evaluation knowledge, t(47)=2.49, p<.05. Given that they were well into their program, these survey results are not at all surprising.
Interviews that were conducted with volunteers from both groups focused on their previous educational experiences. These interviews confirmed the findings from the surveys. The Dist U students reported much higher levels of previous education. The majority of Dist U students had completed a master’s degree before beginning the Dist U program, and many had begun traditional doctoral programs. Trad U students were typically taking their first course of their master’s or doctoral work. A few had attempted doctoral programs in other areas, and some had completed master’s degree coursework. However, many of the students in Trad U came directly out of their undergraduate studies to begin a master’s program in Instructional Design. No students of this sort were present in the Dist U group. The Dist U students also reported higher previous experience with authentic learning and group work in particular than the Trad U students. Both groups reported positive and negative group work experiences, but the articulations of group dynamics principles were far more advanced among Dist U students than those reported by Trad U students. This finding may be a function of the increased levels of prior graduate work or the fact that the Dist U students were in general a little older.
Almost all the Dist U students believed the most positive aspect of the Dist U experience was the people, and particularly the diversity of people. However, there was a conflict therein because many of them also expressed that the most negative aspect of the experience was the diversity or heterogeneity of technology experience among the students, voicing frustration when this diversity held back progress in courses or group work. In contrast with the Dist U students, the Trad U students looked for commonality in their community of colleagues. Whereas Dist U students may have found more power in diversity (possibly because of their previous work experiences with diverse colleagues), Trad U students felt more comfortable finding others like themselves. This may be partly because they are at different stages in their respective programs.
Almost all the Dist U students cited relevance, real-world, and applicability as positive aspects of their graduate education.
Well, the best part of my graduate school experience in the traditional background was that it got me the job I have right now. ... The best thing that I would say about Dist U is that it has enabled me, in my current job to do a number of projects that I would describe as leadership projects for the district. (Dist U Student Pre-Interview)
There was far less emphasis among Trad U students on the applicable nature of their educational experiences. Trad U students articulated more general interest in “the field,” an interest in the area, not so much emphasis on the practical applications.
I guess just having an opportunity to learn stuff that I didn’t’ know before. To broaden my background in a different area or an area that I’ve been interested in but didn’t have the background to do anything. I’ve done instructional design but sort of by the seat of your pants. (Trad U Student Pre-Interview)
During the pre-interviews, students from the Dist U group consistently noted lack of time in terms of personal or work obligations that often kept them from making the most of their educational experiences. This theme recurs in their follow-up e-mail questionnaire late in the semester. One student also noted a lack of personal free time during the initial workshop. Students reported that in forming groups for distance problem solving, working with those who were located nearby was important. The students preferred working face-to-face whenever possible. Dist U students described their work as more self-directed, and they preferred that mode. This finding is supported by Jones’ (1997) examination of distance education student characteristics. They had higher prior experience with self-directed learning in general than the Trad U students did. The Trad U students wanted more lectures, and more direct information sharing even if it was theoretical or abstract information.
Dist U students were fairly critical of their faculty on the whole. They reported disappointment in the faculty’s inability to make time for them, to keep appointments, and to be available. They also were strongly critical of the faculty’s inability to “teach as they preach.” Trad U students, on the other hand, did not report the same frustrations with faculty. Instead, Trad U students reported annoyance with the level of bureaucracy and internal politics that they had to deal with in the traditional program. Trad U students also felt ill-advised, but the reasons were less clear than those of the Dist U students, where time and deadlines seemed to be clear causes.
Dist U students really like the convenience of the distance education opportunity they have chosen. They dislike the expense and inconvenience of traveling (although it is actually less than in most traditional doctoral programs because of the way credit hours are structured), particularly the expense of traveling to the workshop site.
The results in this study of follow-up e-mail questionnaires, e-mail exchanges, journals, Web discussion boards, class artifacts, and audioconference transcripts are discussed thematically and represent both strong thematic trends and outlying themes. The strongest themes include stress, time, distance, project frustrations, technological limitations, the importance of face-to-face interactions, and authenticity.
Many of the students in both groups reported high levels of stress associated with the workload of the project. For instance,
I am a little stressed since we have not yet handed in a complete needs assessment yet and we still need to do a lot more work to get all the needs assessment information. I am also a little stressed because it is so difficult to get together with our group. Since I am the contact, it seems that right now I am doing most of the work. Hopefully, later we will be able to distribute the work better. (Trad U journal entry)
In the case of the residential students, their proximity actually heightened these fears and frustrations, as in certain class sessions groups would share their progress on their final projects.
I think the confusion that was experienced in the middle of the course was the most stressful time for me and my team. It appeared that everyone was at different stages and it was a little difficult to apply the information we were learning in class to our project since our team was not that far along. (Trad U journal entry)
For the Dist U students, the frustrations ran high, and the distance, along with the complexity of the content of Instructional Design, made for a fairly volatile mixture. What seemed to help them was their prior experiences with team work, collaboration at a distance, and instructional design. In the reflection of all the researchers involved in this project, it became apparent that although the participants in both groups were able to produce approximately equivalent projects as evidenced by examining various outcomes including grades, final project components, and reflective products, the Dist U students were much more experienced in terms of group work.
When working at a distance with a team, it is important that everyone first understand the process of ID. I have worked in this way for several years successfully with ID and project teams. I think it worked (in these other contexts better than in this one) because the people I work with are experienced instructional designers and developers for corporations. It is too difficult to learn a new process and try to learn to work with each other at a distance all at the same time. Learning ID is difficult enough. (Dist U Follow-up E-mail response)
Dist U and Trad U students all cited time constraints and life or personal issues as problematic in getting their projects done and interfering with their learning processes. However, the Dist U students mentioned this constraint far more often than did the Trad U students. In referring to the worst part of the experience, Dist U students wrote,
Time management—having practicum preparation, full-time work, family, and the course. I found it difficult to juggle everything and find time slots to accomplish all that needed to be accomplished. (Dist U Follow-up E-mail response)
The worst part is there isn’t enough time to see the project through to completion.... Time is an issue, I know we can’t take forever to complete these projects, but in every class we’ve had in this program we’ve been rushed. I’ve managed to get good results, but it’s a lot of pressure. I guess it comes with the territory. (Dist U Follow-up E-mail response)
The worst part of the experience was the timing. I had 10 house guests over 13 of the December days of the assignment ... and I was the cook. The end came right as we were planning a family holiday (as a respite from all the house guests).... We learned discipline and time management for sure. (Dist U Follow-up E-mail response)
The Dist U students are focused on the pressures of family, work, and learning experiences. This balancing act can be extraordinarily difficult and demand a high level of self-discipline when distance learning is the mode.
In some cases, the Trad U students also complained about time constraints. However, the commitment evidenced by the Trad U students in terms of setting aside personal time commitments came through more clearly. For example,
Crazy things like my job, family in Pittsburgh, my hospitalization in October, and other classes keep getting in the way. However, one thing is for sure—you’ve got to prioritize. Enjoying the project and team helps this class be enjoyable as well as a priority. In reality it is truly the challenges that build character. In the thick of things I often forget to remind myself about this. This fall has been the ultimate character building experience (with this class being one of many aggregates). (Trad U journal entry)
Some of the Trad U students were working at relatively small distances, but they noted that this complicated the time issues that were pressuring them as they developed their projects.
One of the most frustrating things in relationship to the project was the difficulty to schedule meetings—not only with the group members, but also with the SME. Time is so precious. And when all of your group members work full-time and live outside of State College it gets especially difficult. I am thankful that all of the members were so understanding. (Trad U Follow-up E-mail response)
This complicating factor of both time and distance leads to a further discussion of the problems encountered by both groups in terms of distance.
Clearly there were much greater obstacles for the Dist U students to overcome in terms of distance. Dist U students had to struggle with time, place, and language. Students came from all across North America including Canada, and so time zones and English as a second language were complicating distance education factors.
I think probably logistic-wise it is difficult working with a group when you’re from all over the world and trying to get together, but we’ve done, I think, a good job. (Dist U Audio transcript 11/11/97)
The content for this course was difficult to learn, especially at a distance. (Dist U Follow-up E-mail response)
Although the Dist U students cited the distance as problematic more often than did the Trad U students, it might be surprising that Trad U students found distance a problem at all, as presumably one of the advantages of traditional residential instruction is the co-location of learners.
It is strange how we haven’t exchanged phone numbers, only e-mail addresses. A commentary on the mode of preferred communication? Since we are all living in different places ... I think this will be our most effective way to communicate. The distance between us may be one of our greatest obstacles to overcome. (Trad U journal entry)
As mentioned above, Trad U students did cite distance as an issue more often than we might expect. It is possible that the Dist U students were less likely to voice their difficulties with the distance because it was an accepted part of their program. Their prior experience collaborating at a distance probably helped here, and the high value placed on distance education and technology as a part of their preparation:
The program in which we are participating is called Instructional Technology and DISTANCE EDUCATION. Everything that can be done to emphasize the learning of how distance education works should be done. We think the way the ID course was set up with groups working at a distance can be used as a model for all the courses in the program. (Dist U Follow-up E-mail response)
I felt that this particular method worked fine with our group. We five have been a “team” since our first R&E course when we also completed projects in that manner. We are used to daily e-mail and even phone calls about our work.... In all, I think we were not at all hindered by the distance factor. (Dist U Follow-up E-mail response)
The Dist U students dealt remarkably well with the distance obstacle. At times they found innovative ways of working at a distance (using telephone, makeshift whiteboard technologies, and the like). All of the Dist U groups felt that a face-to-face site visit with the SME was critical. Almost all of the Dist U groups had members of the team who lived within driving distance of their project site—so they drove and visited.
So he missed it (an audio-bridge with the group) but we’ve had real good correspondence with him because Ed lives near there and he’s gone to the site and he took pictures and put them on the Web. (Dist U Audio transcript 11/11/97)
We appointed Sammy as our SME contact as he was in close proximity (75 miles) of the school. He made visits there with his digital camera and posted pictures for us to see on the Web site. (Dist U Follow-up E-mail response)
The Dist U students found other innovative ways around the distance obstacle. Whereas the Trad U students struggled to make sure everyone in their groups knew how to use e-mail and had an account, the Dist U students were creating new ways to communicate effectively for true team collaboration at a distance:
And we’ve got that pretty well worked out and what we’ve been doing because of the distance between Jeannie, we’ve actually been using the Web pages as a method to work back and forth. Kind of an asynchronous white board process. (Dist U Audio transcript 11/11/97)
Despite their positive attitude toward the flexibility of technology and the distance obstacle, Dist U students were highly frustrated by the process and voiced more general project frustration than did Trad U students.
Many distance education studies are beginning to uncover the student frustrations that accompany Web-based course projects (Hara & Kling, 1999). In this study, both groups expressed a preference early in the course to work on projects of their own devising rather than on those provided by their SMEs. In both groups, as the first author has found is typical in past classes of this sort, a couple of students strongly voiced the “need” to work on something from their own workplace. In my past experience teaching this course residentially, there are typically two factors that motivate this resistance: fear of group collaboration and desire for efficiency in learning. In the first case students are concerned about trusting group members with their grades; they prefer to maintain more control over their environment. In the second case working on something from another class or from their workplace will make their job more time-efficient—an all-important value in both groups of learners. Dist U students were slightly more resistant to the approach than were Trad U students:
I guess I’m looking for some validation because as I’m going through this process normally I should think that for every course that I’ve taken so far at Dist U I’ve kind of been able to understand or comprehend more of the expectations and what’s required, but because this is something that I haven’t gone through before I have the anxious feeling or the frustration, but I guess because it’s the learning process I really don’t understand fully. (Dist U Audio transcript 11/11/97)
One of the biggest frustrations voiced by the Dist U group was centered on the problems that arose when technology failed them. They were strongly committed to seeing the project through and using the most up-to-date technologies to assist them in that goal. In fact they routinely become frustrated with their adjunct professors when they do not know how to use updated technology. It is understandable that for a group so dedicated to distance education technological problems result in high frustration.
We had initial difficulty with ISP functionality and operationalizing chat software, but adapted fairly easily to telephone conferencing. (Dist U Follow-up E-mail response)
I would say maybe one of the problems that I found with distance education was as the project got bigger and bigger was being able to e-mail it to each other. And especially when we weren’t all using the same format. I know Angie was using Macintosh and we were constantly trying to open files and I found that probably the most time-consuming part of it that seemed to take so much of my time anyway. (Dist U Audio transcript 2/13/97)
One of the important findings of this study was that all groups felt that face-to-face interaction was preferable and critical. Despite the Dist U students’ dedication to distance education as a mode, all the teams visited their SME in person. Two Dist U teams had some members co-located (two married couples on one team and three employees of Dist U on another). They cited this contact as crucial:
Of course we had one small advantage that a lot of others didn’t have. Three out of the four participants live within 45-minute ride of each other. And although we like distance, we got together. So we have to let you all know that right up front. (Dist U Audio transcript 2/13/97)
I’d also like to mention something about the collaborative project as we are members of the group. I think we were also fortunate in that we had two hubs. One in Houston and one in Kansas and I think that made it a little easier to write collaboratively rather than everyone in a different part of the world. (Dist U Audio transcript 2/13/97)
The Trad U students were quite vocal about the necessity of face-to-face contact:
We conducted some “business” by e-mail, but normally ended up meeting face-to-face to get the bulk of our work done. I think if we had been unable to do that, it would have made the project more difficult. (Trad U Follow-up e-mail response)
When distance for the team was an issue, we overcame it with phone calls, e-mails, and face-to-face meetings at a nearby University (a halfway point for all team members. The best meetings were face-to-face. (Trad U Follow-up e-mail response)
We were effective, but only because we minimized the distance element by meeting face-to-face periodically. (Trad U Follow-up e-mail response)
Thus although all students valued face-to-face interaction, the residential students were more vocal about their support for this need, and the distance students were more vocal about the appropriateness of the distance program. This is probably due to their own rationalizations about the choices they have made in their educational experiences. Thus for the residential students valuing face-to-face confirms their choice and fits in with their value system, whereas for the distance students overcoming the distance obstacle as a goal in itself rewards them for a choice made to study at a distance.
Both Dist U and Trad U students enjoyed the authentic nature of the projects, although Trad U students expressed satisfaction and a higher level of reflective processing about the authenticity of the project. Specifically, the Dist U students were primarily interested in how these ideas would help them in their workplace, whereas Trad U students were more generally interested in the authentic learning as it would contribute to their development. For example,
Reflecting on what we have been doing for the past two weeks, I am beginning to sense what instructional design is like. We are not only learning from books about instructional design process, but are also actually experiencing the process by working on an authentic problem. This is just a beginning, and we are learning how to plan for the whole project, things as specific as contacting the SME and arranging time for the meeting, making action plans, writing project statement and carrying out needs analysis. It is a very good learning experience. (Trad U journal entry)
Trad U students also were able to connect with the service element of the project. They did not voice as much frustration as Dist U students over the fact that the projects were not in their own workplace.
However, it was good to know that what we were working on was “real life” stuff. There’s something very gratifying to know that all of the time and effort spent on our projects may be used to benefit someone; it’s not just a hypothetical situation. (Trad U journal entry)
Students in the Dist U program were more likely to speak about their work in terms of the applicability to their work situation.
I was not just being kind when I said that your course was the most valuable of all I took at Dist. I am using what I learned already. (Dist U Follow-up e-mail response)
In a word, relevance. The project enabled us to experience, first hand, the role of instructional systems designer. (Dist U Follow-up e-mail response)
Thanks for fostering such professional growth in me.... I have already begun to use the methodology in my work setting. (Dist U Follow-up e-mail response)
And talking about getting to work with a real world problem ... I really appreciated that. Because I don’t like to work with a pretend stuff. I really enjoyed actually working on a real problem. (Dist U Audio transcript 2/13/97)
Data analysis showed several important concerns with authentic problem-based collaboration at a distance. First, the students in both groups exhibited a certain amount of resistance to the problem-based curriculum. However, the students working at a distance exhibited more frustration over the problem-based approach because of the added difficulties of truly collaborating at a distance in a consistent fashion. They were concerned about their ability to perform all tasks involved in instructional design because they felt they had learned only the part they were assigned (or had negotiated as their responsibility with their team) rather than the whole process. This feeling was more prevalent among the distance students than the residential students. This heightened level of concern may be the result of increased division of labor among distance students or possibly because they saw more direct applications of their learning to their current work situations because more of them were employed full time in related work contexts. Further research would be necessary to confirm these explanations for the data.
In addition, the data confirmed that the distance students were farther along in their program, generally a little older, employed full time in a related position, and had a good deal more experience with education and instructional design than did the residential students. The products that both groups created in response to the authentic problem were of approximately equivalent quality5 and evidenced similar levels of understanding of the instructional design process.
The students in the distance group consistently pointed to the importance of the audio-bridges (phone) contact with the group where all members could work together on the problem and speak to one another. Many felt the project would have been impossible without the audio-bridges and felt that relying solely on the Web board or e-mail (less expensive options) would have been extremely frustrating if not impossible. In addition, they listed a plethora of technical difficulties that emerged from their attempts to transfer documents, graphics, information, and pieces of their projects back and forth between multi-platform users. These technical difficulties were their biggest frustration and one that was not mentioned at all by the residential students. Perhaps the biggest surprise of the study was the lack of group dynamics problems among the distance students as compared with the residential students. Because it was not the focus of the study to examine this particular question, little emerged from the e-mails, journals, or interviews that would speak directly to this phenomenon. It is possible that a feeling of access to the instructor or the ease of face-to-face contact with the instructor led to more sharing by the residential students in terms of interpersonal problems, difficulties with group dynamics, and lack of collaboration or members’ noncompliance than was shared by the distance students. The other possible explanation for this is the deep relationships that had formed among students in the distance section. These students were part of a cohesive cadre that had been together in several courses before this experience and appeared more committed to one another as a result.
Because these settings do not represent pure distance or pure residential instruction, the comparisons may not be as powerful as if we had conducted this study in a setting that did not include any face-to-face contact and relied more heavily on Web-based technologies for instructional delivery. However, in many ways this actually reflects the realities of distance and residential education today. There has been much more blurring between these two distinctions, and this will probably continue. Although the access to distance education over the Web to complete degree programs without any residential or on-site requirement seems appealing, most programs still require some face-to-face interaction between learners and instructors. Nevertheless, this study could be replicated looking at a slightly more pure distance program where Web-based learning is the primary mode and no residential component is present.
This work has several specific implications for those hoping to employ distance education at the university level. These graduate students have told us, rather clearly, that they needed the audio support, and they needed to have small experiences with distant collaboration before authentic problem-based collaboration at a distance could be undertaken. It is important to reflect the reality that most students in both groups are substantially familiar and comfortable with the traditional models of instruction and learning and are thus in need, at least for a time, of scaffolding for these new methods. The students also are clearly pointing out the problems that are inherent in such collaboration when the technology is not capable of fully, seamlessly supporting their efforts. In designing such experiences, the use of advanced learning theories such as collaboration and authentic problem-based learning should be approached with caution by the designer-instructor. Based on the findings of this study, it would seem that only students with appropriate technological support, prior experience with the content, and mature levels of group processing should be given authentic problem-based learning experiences to collaborate on at a distance. Online course designers should consider the following list of implications of this research.
Attempt Web-based collaborative authentic learning in stages. Problem solving and collaboration are skills to be learned in stages. Embedding these experiences into an authentic problem should be an ultimate goal, not a starting point.
Provide audio support. Although this may limit the efficiency of online courses in higher education, it seems clear from these findings that audio support is a vital link for authentic group-based learning at a distance. Students did not find the Web discussion boards or e-mails sufficient.
Provide technical support. Providing adequate support for their learning, seeing that the technology functions for them in a seamless fashion is an important first step in effective Web-based training for computer-assisted learning. This may go without saying, but when students are engaged in high-level problem-solving, working through group problems at a distance, and trying to apply their ideas to real-world contexts, they are easily frustrated by technological problems.
Ensure the learners have appropriate entry-level skills. Learners should have prior experiences with the content before attempting collaborative, high-level problem-solving in authentic contexts at a distance. In addition, this learning must be effectively assessed to be certain learners have not only read Web screens, but are able to apply their learning to some fictional or contrived context before to being able to apply it to a real-world context.
This research contributes to the growing base of literature in distance education, collaborative learning, and authentic problem-based learning. It is extremely important that combining current learning theories and current technologies is approached with not only pragmatic eyes toward efficiencies to be discovered, but also implications for better learning. Only in this way will the true potentials of distance education be realized and appropriately used.
Abrami, P.C., & Bures, E.M. (1996). Computer-supported collaborative learning and distance education. American Journal of Distance Education, 10(2), 37-42.
Barker, B.O. (1987). Interactive instructional television via satellite: A first year evaluation of the TI-IN network. Journal of Rural and Small Schools, 2, 18-23.
Barrows, H.S. (1986). A taxonomy of problem-based learning methods. Medical Education, 20, 481-486.
Brookfield, S. (1987). Developing critical thinking: Challenging adults to explore alternative ways of thinking and acting. San Francisco, CA: Jossey Bass.
Brown, J., & Duguid, P. (1994). Practice at the periphery: A reply to Steven Tripp. Educational Technology, 34(8), 9-11.
Burge, E., & Howard, J. (1988). Learner centredness: Views of Canadian Distance Education Educators. Unpublished manuscript.
Carr, A.A. (2000) I have a problem! The use of cases in educating instructional designers. Tech Trends, 43(6), 15-19.
Carr-Chellman, A. (n.d.). Long distance CAL (collaborative authentic learning): Recommendations for problem-based training on the Web. Unpublished manuscript.
Carr-Chellman, A. (n.d.). To distance educate or not to distance educate: That is our question. Unpublished manuscript.
Collis, B. (1995). Evaluating electronic performance support systems: A methodology focused on future use-in-practice. Innovations in Education and Training International, 32(1), 23-30.
Collis, B. (1996). Tele-learning in a digital world. London: International Thomas Computer Press.
Daniel, J.S. (1996). Mega-universities and knowledge media: Technology strategies for higher education. London: Kogan Page.
Hackbarth, S. (1997, May-June) Web-based learning. Special Issue of Educational Technology Magazine.
Hara N., & Kling, R. (1999). Students’ frustrations with a Web-based distance education course: A taboo topic in the discourse. Available: http://php.indiana.edu/~hara/
Harasim, L (1994). Collaborating in cyberspace: Using computer conferences as a group learning environment. Interactive Learning Environments, 3(2), 119-130.
Heeren, E. (1996). Technology support for collaborative distance learning. Centre for Telematics and Learning. University of Twente, Enschede, Netherlands.
Hiltz, S.R., Johnson, K., & Turoff, M. (1986). Experiments in group decision making. Human Communication Research, 13(2), 225-252.
Hiltz, S.R., & Wellman, B. (1997). Asynchronous learning networks as a virtual classroom. Communications of the ACM, 40(9), 44-49.
Johnson, D. W., Johnson, R.T., & Holubec, E. J. (1984). Circles of learning: Cooperation in the classroom. Alexandria, VA: Association of Supervision and Curriculum Development.
Jonassen, D. (1991). Objectivism versus constructivism: Do we need a new philosophical paradigm? Educational Technology Research and Development, 39(3) 5-14.
Jonassen, D.H., & Land, S.M. (in press). Theoretical foundations of learning environments. Mahwah, NJ: Erlbaum.
Jonassen, D.H., Davidson, M., Collins, M., Campbell, J., & Bannan Haag, B. (1995). Constructivism and computer-mediated communication in distance education. American Journal of Distance Education, 9(2), 7-26.
Jones, G.R. (1997). Cyberschools. Englewood, CO: Jones Digital Century.
Khan, B. (Ed.). (1997). Web-based instruction. Englewood Cliffs, NJ: Educational Technology Publications.
Khan, B. (Ed.). (in press). Web-based training. Englewood Cliffs, NJ: Educational Technology Publications.
Larson, M.R., & Bruning, R. (1996). Participant perceptions of a collaborative satellite-based mathematics course. American Journal of Distance Education, 10(1), 6-22.
Lincoln, Y.S., & Guba, E.G. (1985). Naturalistic inquiry. New York: Sage.
Lopez, E.S., & Nagelhout, E. (1995). Building a model for distance collaboration in the computer-assisted business communication classroom. Business Communication Quarterly, 58(2), 15-20.
Merriam, S.B. (1988). Case study research in education: A qualitative approach. San Francisco, CA: Jossey-Bass.
Mingle, J.R. (1995). Vision and reality for technology-based delivery systems in postsecondary education. Report to the Governor’s Conference on Higher Education. St. Louis, MO: State Higher Education Executive Officers.
Moore, M.G., & Kearsley, G. (1996). Distance education—A systems view. Belmont, CA: Wadsworth.
Noble, D. (1998). Digital diploma mills: The automation of higher education. First Monday, 3(1).
Savery, J., & Duffy, T. (1996). Problem-based learning: An instructional model and its constructivist framework. In B. Wilson (Ed.), Constructivist learning environments: Case studies in instructional design (pp. 135-148). Englewood Cliffs, NJ: Educational Technology Publications.
Seaton, W.J. (1993). Computer-mediated communication and student self-directed learning. Open Learning, 8(2), 49-54.
Septien, W., & Gallagher, S. (1993). Problem-based learning: As authentic as it gets. Educational Leadership, 50(7), 25-28.
Stake, B. (1986). Quieting reform. Urbana, IL: University of Illinois Press.
Weber, R.P. (1990). Basic content analysis. Newbury Park, CA: Sage.
Wilson, B. (Ed.). (1996). Constructivist learning environments: Case studies in instructional design. Englewood Cliffs, NJ: Educational Technology Publications.
Yakimovicz, A.D., & Murphy, K.L. (1995). Constructivism and collaboration on the Internet: Case study of a graduate class experience. Computers in Education, 24(3), 203-209.
Interview Protocol
Describe your graduate school experience
Best Worst
What one thing has been invaluable in your graduate studies? (e.g., experience, information, specific skill or knowledge)
What has helped you succeed here?
What has contributed to any sense of frustrations you have?
Has anything specific contributed to any failure you may have had?
What one thing would you change about your graduate school experience?
Before this course what was your experience with problem-based or project-based learning?
Before this course what was your experience with group work?
Read each statement below and circle the number on the continuum to represent your experiences with group work. Add comments to qualify your response as needed.
Before this course, how much experience had you had with group work in classes?
No Experience Lots of Experience 1 2 3 4 5 6 How many in-class group projects have you worked on in the past three years? 0 1-3 4-6 7-9 10 or more Evaluate your skills in the following areas Low High Group work 1 2 3 4 5 6 Task and personal support 1 2 3 4 5 6 Teamwork 1 2 3 4 5 6 Constructive conflict 1 2 3 4 5 6 Clear group goals 1 2 3 4 5 6 Commitment to self and others 1 2 3 4 5 6 Assessment of Individual and group efforts 1 2 3 4 5 6 Continuous improvement of process 1 2 3 4 5 6 Before this course, how much experience had you had with real-world problems used as major course projects? No Experience Lots of Experience 1 2 3 4 5 6 In the last three years how many course projects have you been involved in that use real world problems? 0 1-3 4-6 7-9 10 or more Of those projects from question #5, how many of those involved group work? 0 1-3 4-6 7-9 10 or more Evaluate your skills in the following areas Low High Identifying the problem 1 2 3 4 5 6 Identifying problem issues 1 2 3 4 5 6 Identifying possible solutions 1 2 3 4 5 6 Identifying necessary resources to solve the problem 1 2 3 4 5 6 Synthesizing findings 1 2 3 4 5 6 Evaluating hypotheses 1 2 3 4 5 6 Implementing problem solution 1 2 3 4 5 6 Evaluating problem solution 1 2 3 4 5 6
1. Note that although Web-based distance education is one of the least expensive ways to deliver instruction, it can be quite expensive in terms of development costs and initial investment in technological infrastructure.
2. Since then the program has been moved to the south campus in University buildings.
3. For a more complete description of the differences between these settings in terms of locus of instruction and locus of learning, see Carr's chapter in Khan (in press) book on Web-based training.
4. For additional insights in terms of a personal retelling of this experience, please see Carr-Chellman, n.d.
5. The final projects for both groups were graded using the rubric for evaluating the course projects from previous (residential) sections of the course and include creativity, comprehensiveness, systemic nature of the design (that is, does it all work together?) and several other criteria.
Alison A. Carr-Chellman is an assistant professor of instructional systems at Penn State University’s College of Education. Her research and writing interests center on systemic change in organizations, including higher education. Her e-mail address is aac3@psu.edu.
Dean Dyer is currently working on his doctoral dissertation at Penn State University while working as an instructor in teacher education at State University of New York Potsdam. His e-mail address is dyerdr@potsdam.edu.
Jeroen Breman is a doctoral candidate at Penn State University. His e-mail address is jxb66@psu.edu.