This chapter discusses the design of the research in the VMDL Project and the way that the work evolved over a number of phases.
The main stages of the Project are outlined in Section 3.2; the main data sources are described in Section 3.2.1 which is followed (Section 3.2.2) by a description of the various methods of investigation used. Finally case study selection (Section 3.3) and the data analysis (Section 3.4) are explained.
The VMDL Project team comprised researchers who had experience in a number of different though, from the Project perspective, complementary fields. Hence, the team 'inherited' a range of methodological approaches and a diverse literature of research findings. This contrasts with much previous research which has focused almost exclusively upon one or other of the following:
¨ technological innovation and description;
¨ theoretical exploration of possible technology-based futures;
¨ organisational/developmental investigation into non-traditional ways of management as alternatives to the bureaucratic/hierarchic model.
VMDL set itself the challenge of working with disparate investigatory paradigms and their associated methodological approaches.
Despite the different methodological backgrounds of the team members, the choice of a multiple case study investigation was straightforward. Complex and naturally occurring settings were to be studied and their dynamics synthesised without intervention by the researchers.
Such an approach is rooted in a qualitative methodological orientation which means that data were not weighted on the basis of frequency of occurrence, but on the basis of their descriptive usefulness. In other words, an intervention was not necessarily considered more important because it was repeated by more people, but because it provided useful understanding of the how the community worked. This notion corresponds to Van Maanen's (1983) definition of qualitative methodology as:
"an array of interpretative techniques which seek to describe, decode, translate and otherwise come to terms with the meaning, not the frequency, of certain more or less natural occurring phenomena in the social world". (p. 9)
Whilst developing the detail of the case-study methodology, the VMDL community itself was used as the first case study. This approach was adopted in the expectation that additional insights might be uncovered that would be inaccessible to external observers. Data derived from this line of investigation both informed and was compared with data derived from the investigations of the other communities. However, the complexity of the VMDL Project was increased by the use of the research partnership itself as a 'live case' of a distributed research community.
Two other research communities were chosen according to the criteria developed by the team and the details of each of the three case studies is given in Chapter 4.
The methodologies adopted by the Project were influenced by the nature of the research question and by the nature of the other distributed research groups that the Project selected to investigate. As described in Chapter 2, the research question itself evolved partly as a result of the self-reflection of the VMDL Project, and partly due the parameters which seemed important in getting to understand the other case study groups. Hence, the methodology itself evolved during the course of the Project (despite the short period available for the research).
An iterative design was adopted that had two main stages, each of approximately six months duration. In the first period the basis for the case studies was prepared following a traditional pattern of literature study, development of a conceptual framework, and refinement of research questions. Less traditionally, this investigation was informed by a deliberate self-reflection on the functioning (and development of organisational and operational structures) of the VMDL partnership itself as a distributed research community (see Section 4.1). During the last six months, the case studies were conducted (largely by interview) on the selected distributed research communities, followed by data analysis and reporting. More detail is provided below.
The VMDL Project drew upon four main sources for its case study data:
¨ documents prepared by the research communities which were to be studied, in particular their statements of objectives and independent evaluation reports when available;
¨ records of interactions between participants in the research groups from electronic conferences;
¨ semi-structured face-to-face (recorded) interviews with some members of the research groups;
¨ observation of some research groups in action (either through physical presence or by 'eavesdropping' on electronic interactions).
All these methods were used with the consent of the research groups as a whole and of individual members.
Naturally, the reflective case study (VMDL itself -Section 4.1) had by far the richest sources of data as the researchers were aware of forms of interaction which left no trace, such as occasional telephone discussions, the informal sub-group discussions and the many 'private' one-to-one interactions, face-to-face and by electronic mail, which were not systematically available for analysis. However, while introspection is increasingly being recognised as a useful tool within qualitative research (Lee, 1995a) it is recognised that the introspector will interpret the situation according to his/her own 'unique mapping of meaning'. Schatzman and Strauss (1973) say that the possible effects of a introspector can be that the introspector may be not able to balance what he sees, experiences, and 'feels', and he will absorb certain viewpoints rather than challenge them. Thus case study data largely derived from introspection in the first of the VMDL case studies (though interviews were also carried out) was balanced by more traditional methods of investigation in the other two cases.
Literature-based research and introspection. As mentioned above, the first six months of the Project were largely devoted to shared introspection and literature-based research. This process helped in identifying the variables on which the case studies would focus and in arriving at a shared understanding of the meaning of these variables in the context of the distributed research communities.
Computer conference records. In the case study of VMDL, rich data were (more than 1300 messages) available in the form of the database of computer-conference interventions. The generic interest in these is not only in the actual content of the interaction but also the form of the interactions. The parameters of interest include: the frequency of intervention; the nature of the interventions; the group structure; and changes in patterns over time. These data were used to some extent in Section 4.1 and in the cross-case analyses in Chapter 5 but time and resources did not allow full exploitation of this data source.
Interviews. The majority of data were gathered through in-depth face-to-face interviews, allowing direct interaction, thereby facilitating the elaboration of answers, and the adoption of flexible and context-sensitive questioning procedures. Whilst this approach offers the potential of richer data than do structured questionnaires, it places reliance upon interviewer sensitivity and bias. This approach was piloted to minimise unwelcome side-effects and sharpen the way the main interviews were carried out.
Communication channels (media) were explored in detail during the interviews in the light of group member's motivations using contextual factors (community, task, organisation), as discussed in Chapter 2, with a focus on:
¨ within media referring to the characteristics, methods and effectiveness of a particular communications channel;
¨ between media focusing on the complementarity of various communication channels;
¨ over time referring to the changes of the use and effectiveness of communications channels over time.
In the first stage of the research (referred to as the pilot stage) the VMDL research fellows interviewed each other and in the second stage they interviewed other members of the VMDL team. The first purpose was to evaluate and refine the understandings derived from introspection, thereby cyclically exploring each other's perceptions. The second was to pilot the interview techniques, approaches and questions iteratively, thereby attempting to minimise interviewer bias and maximise question relevance. The interviews were recorded, distributed among the interviewers and analysed jointly through closed discussion.
Observations. In an opportunist way, VMDL members observed and/or participated in a number of other research communities. Of particular interest was their use of communications media which were not present in the case studies. These included video conferencing and real-time electronic conferencing on the World-Wide Web.
Criteria were established for the selection of communities to be used as case studies. A matrix of possible communities was developed with rows relating to aim, partners, media, orientation, duration and forthcoming face-to-face meetings. From a range of possible candidates, three communities were identified as meeting the diversity of needs in the VMDL study, one of them being VMDL itself.
The case studies were selected on the following criteria:
¨ The communities were distributed and learning/research-oriented (in contrast to production/task-oriented);
¨ The communities already used several different communications channels including face-to-face meetings (or other synchronous medium) and electronic media.
The case studies were set up by a process aimed to encourage researchers in the two non-VMDL communities to 'enter into' the research interest of VMDL. Initial introductory letters were sent to a known contact person in each community and a first meeting arranged with that person. Following this a further letter was distributed among the members of the chosen community to explain VMDL and to ask for their co-operation. The contact person advised on the selection of interviewees (the criteria for which were to establish a range of levels of involvement, roles, motivation, needs, and context within each community) who were then contacted personally by the interviewer through e-mail or by phone, and the interview arranged. The communities are described fully in the sections of Chapter 4.
The analysis focused on the key variables as identified in Chapter 2 and represented by Fig. 2.2 which is reproduced for convenience.
The interview data collected by the VMDL researchers were assembled for common access in electronic files with due concern for the confidentiality of any sensitive data. The way in which the interviews were undertaken allowed interviewers to compare the recorded interview data by topic, by person and by community regardless of where in the interview any particular topic was addressed. The data were analysed within-case (see Chapter 4) and across-case (Chapter 5), with different members of the VMDL team taking lead responsibility for different areas of analysis and reporting.
This process itself, however, also provided further data for the continuing introspective analysis of the VMDL Project itself, and is reported in Chapter 4.1
Documentary evidence, enriched by the interview data, allowed case study profiles to be created (see each section in Chapter 4), influenced by common parameters (derived from Hughes and Christie (1995)).
Following the case study analysis, members of the VMDL team formed sub-groups to work on three cross-case themes: nature of the research tasks, individual and group profiles, and management and organisational issues. In each of these (Chapter 5), the common theme of 'use of communications media' remains central.
The less formal, yet insightful, studies of other communities (Chapter 6) contribute a more technologically focussed description of distributed communities than those which formed the main thrust of the VMDL Project.
Having set the scene for the VMDL Project, it is now possible to turn to the details of the case studies (Chapter 4) and the thematic analyses (Chapter 5).