Within the domain of the use of ICT to stimulate and support collaborative pan-European research a pilot study was conducted to investigate the question: 'What factors influence the success of a distributed research community?'.
The project has resulted in a number of conclusions. These will be discussed below in Section 7.2. However, in this first section some contextual variables and characteristics of the project will be discussed. As the followed research methodololy was a situated action research approach using case-studies, those charecteristics may have had a significant - maybe even dominant, influence on the results of the project. Knowing them can be helpful in interpreting those results.
The following aspects will be discussed:
a Was the choice of the research question appropriate?
b What about the choice of the research methodology?
c What about the choice of the cases, in particular the choice of the project itself as a case?
d What about the data gathering?
e What about the sequence the research was conducted?
Although the research question is a very common and reasonable one, the scope of the project (time-frame and resources) did not really allow to expect stable and generalizable results, and therefore 'success' cannot be evaluated or measured. Therefore, the research question is maybe somewhat overstated. What the project has done is investigating factors that influence the life cycle of a distributed research project.
The choice of a situated action research methodology using case studies seems to be a correct decision. The multivariate and complexity of the domain of the research and the need for an exploratory approach supports the use of qualitative research methods. In addition the reflections based upon Action Theory in combination with Information-richness Theory offers very interesting scientific hypotheses.
Three communities were chosen as case-studies in the project. The first case was the VMDL-project itself. The main reason to choose this project was because of the expectation that additional insights might be uncovered that would be inaccessible to external observers. The second case was a typical European Project involving 30 researchers, working on workpackages using computer-conferencing and face-to-face workshops. The area was about choosing modules to be attended at different European universities. Most of the researchers were participating in the project part of their time. The third case was a specialist research group of forty physicists from three research laboratories in three countries, preparing for an experiment to be conducted in 1999. The research community started its work in 1993. While the first and second case study dealt with social science topics, this case study was about hard-sciences: particle physics.
In retrospect one can question if the additional insights did not became so overwhelming that there was a kind of transition from 'objective insight' to a very subjective involvement of individual 'states-of-minds'. Even more questionable was the effect of this introspection on the interpretation of the research question. The research question in itself became object of introspection. Where a shift of emphasis in a research question during a research project is not unacceptable by default, it remains questionable if in the VMDL project the involvement of the main players (researchers) in the major case study has not strongly overshadowed the results.
It was the intention that the two other case studies would balance the introspection during the first case. Given the time pressure that was put on those two cases, and the fact that because of external and private circumstances the personal situation of the prime researchers were shifting, it remains unclear if the same research framework that was used in the beginning of the project, at least in a virtual way, was still the same at the end of the project. Therefore, it remains unclear if the intented balance of the external cases towards the results of the first case is really reached.
During the first case study data were gathered by analysing logbooks of computer conferencing and e-mail activities, as well as by face-to- face interviews of community members. As the total number of actively participating members was low (3 in the so-called inner circle, and 3 in the so-called outher circle) and the interviews took one hour on the average, the dominance of the logbook information was rather high. In addition there was a specific personal relationship between the members of the research community (long-standing colleagues, or teacher-learner relationships), which made the research community and thus the data gathering very specific.
The data gathering for the other projects was rather limited. Again, it remains an open question how specific these data are, and therefore how generalizable the results of the project are.
As it was necessary for the VMDL-project team to start knowing each other, focussing on the VMDL-project itself is understandable. However, it could have been a better idea to have investigated one of the other cases first (after the 'knowing-each-other period' within VMDL). That would have forced the VMDL-team to better clarify what were the goals of the project and more particular, further specifying the research methodology. The potential large influence of the introspection within the VMDL-project as a case, could then be reduced.
The objectives of the project was to assess and come to an understanding of the conditions necessary for collaborative working in a community comprising researchers from different disciplines and traditions, and using ICT. The metaphor of the research was expressed by the notion of virtual mobility, by creating distributed research laboraties and communities.
To answer this question, it is needed to know the answer to the question 'what determines success of a research community?'. In a simple way the answer is that (a) the research community has to contribute to the development of the field, (b) the community has to do this in an effective and efficient way, and (c) the community has to create and attract sufficient (financial) support in order to be able to continue its efforts. The added value of a distributed research community should be visible in each of these three aspects. This leads to the following three questions.
As more scientists are involved in a distributed research community than there are in a local group, and assuming that involvement of more people rises the quality, the answer to this question is 'yes'. However, as will be described later in this chapter, the decision- making activity in a distributed research community creates particular problems. This could hamper scientific progress. In any case, this project did not enough time and resources to seriously answer this kind of question.
Most of the researchers in the case studies were involved part of their time in the particular research project under investigation. This situation can be seen as rather natural, as few researchers will have long-standing research obligations in only one project. There is a danger however, that the easiness to participate in distributed research projects, leads to many engagements, which then, taken together, limits the engagement in each of the particular projects.
Another issue is the language of communication between researchers. In the investigated projects English was used as working language, even without questioning, although not all of the participating researchers had English as their mother tongue. This is certainly an significant advantage.
Based upon the experience within this project and based upon the consortium already established, an application for continuation was sent to the EU. The new application was not succesful. In principle an already established consortium should have a better chance for approval. But....
What factors can be identified contributing to the succes of a distributed research community? Factors can be identified in different areas: (a) organisational environment, (b) technological environment, (c) task and content environment, (d) composition and characteristics of the community members.
The predominant working method has been through electronic mail and (asynchronous) computer conferencing. Four face-to-face workshops and other informal sub-groups meetings have been organised. The intensity of the electronic communication has been very high (5-10 interactions each day) using a robust but not 'latest state-of-the-art' technology.
In the first case (the VMDL project itself) decisions were taken based upon a mixed approach (hierarchical in the beginning, then more democratic in the middle, and again more hierarchical at the end). This approach more probably evolved because of availability of time with the senior researchers and time pressure at the end of the project to come up with results, then because of deliberate decisions to find an optimum decision-making approach. It seems to be clear that the most active (young) researchers went to a very interesting learning experience for themselves. It is doubtful however if such an approach is advisable when 'real' research items are at stake. It seems to be that a more direct guidance from senior researchers determining the research framework and sequence of events is a more productive approach. Of course, senior researchers should therefore carefully plan their time involvement and live up to it. In Stanchev's models of communication, the star model clearly is preferable.
In the second case (European project) decision-making was described in the project proposal, whereby the workpackage leader took formal responsibility. Further decision making was done during face-to-face meetings. As in the first case study, decision-making while communicating through a computer conference appeard to be difficult. It was necessary that someone took a leading role. In order to keep being and feeling 'democratic' taking turns in moderating the discussions and decision-making could be a solution. As there was a serious time-pressure within this project, most members welcomed strong coordination.
In the third case the decision-making was done during the face-to-face meetings. Although the importance of a leading figure was acknowledged, in most cases the scientific relevance of the solution determined the way how to come to a conclusion. The abscence of clear decison-making authority was made up for by a clear, common goal.
There is no single medium that adequately supports the collaborative process. A distributed research group needs access to a variaty of tools for interaction. Such media, however do not need to be the latest state-of-the-art technology. Robustness, adequacy and costs are the appropriate criteria for the choice of the media mix. It is unclear of 'richer' media would have facilitated decision-processes. However, it is essential that all members of the research community have easy access to the media used.
As the second case study involved partly the same parent organisation as the first case study, very comparable technological means were available for this project. Therefore it is not a surprise that comparable results were found in this case study with respect to the technological factors.
The media used in the third case were again synchronous and asynchronous media. In addition face-to-face meetings were organised. Because of the distance between the European and US members of the community and the costs involved having face-to-face meetings, video- and audioconferencing was used occasionally. Because of lack of quality the videoconferencing was felt to be disappointing.
Both synchronous and asynchronous media were used throughout the first project. In particular the computer conferencing system (Caucus) was use extensively. Because of the hugh volume of the number of messages, it became soon apparent that a well thought out organisational structure of this stream had to be put into place. Lots of efforts were involved in this task. A major question is if the efforts to organize the hugh number of messages and the different structures that were established, should not be replaced by a more substantial and content-oriented effort to limit the communication between the members of the research community and focus on the core of the research activities: more product instead of procedure oriented. Stating it more generally, the information overload, also created by the ease of communication, is becoming a major problem. Maybe communication should not made very easy, or better, should have a serious price tag so that users will think twice when becoming involved in communication.
It was expected that motivation could have an important influence on the execution of research. The data gathered during the first case study showed that members of research communities had indeed strong motivations to participate, but that the kind of motivational factors were very dispered, changed over time, and were strongly steered by personal objectives. The motivation, partly steered by social cohesion and pressure, also depends on the interdepence of research activities and another's performance. If in a distributed research project cohesion and interdependence of partial tasks is not very strong, personal and changing motivation will strongly determine the amount of active involvement.
The motivations within the second case study were more content-related, but also personal and career-related. This coincides with the structure of European projects and the decision to participate. It was felt that researchers gained from the international flavour of the experience, getting more English language profiency, better insight in cultural differences. These are of course interesting results, however, results that were not aimed at in the original research proposal.
In the third case study working together towards a common research goal was a dominant motivation. Member's personal motivations were largely synonymous with the goals of the community.
It was concluded that all group members had various reasons for joining the research project, and that once the project started their motivation changed, either to maximise upon the career enhancement it might provide, or to finish what they had started. This is not a uncommon situation in research projects. However, did the distributed aspect of the research community added to the motivation or distracted participants from their research task? It seems that most group members preferred to work/meet face-to-face first before working at a distance from the start. However, the distance did not seem to influence the teamwork mainly because the technology made it easier to cross the cultural and language barrier, as non-native English speakers could work more easily through a computer system that gave them the time to react in a foreign language. On the other hand, working at a distance appears to (over) emphasize those aspects of individual commitment and group development (process of decision-making, handling of power issues, encouragement of commitment and involvement) that might otherwise be subconsciously 'negotiated' in face-to-face work.
In the VMDL case study, most of the researchers were very familiar which each other, or did not know each other at all. As the VMDL case study was rather short in its execution (a couple of months) it is an interesting question how the familiarity (or the lack of familiarity) of the team members had strong impact on ways of communicating which each other. Both situations (familiarity and lack of familiarity) had a certain impact on the communication between team members (in particular between the so-called 'young' and 'old' fellows). It seems to be that a rather large discrepancy of level of familiarity between team members certainly is not a factor contributing to success. On the contrary, it seems to be that it is better that the 'between-familiarity' of members of a distributed research community is not too strong nor too low. A typical colleagual level of familiarity between members of groups of researchers seems to be advisable.
The main conclusion of the project is that the major success factor for a distributed research community is the need for regular face-to-face meetings in order to reach agreement about main issues. If main issues are clearly defined and operationalised, there is less need for a clearly established authority to make the final decision as scientific arguments will determine the outcome of the process. However, and certainly in the case of social sciences where clearly defined and operationalised issues are more difficult to formulate, the opinion and choice of a specific leading figure is often necessary in order to be able to proceed. In any case, during the time between the face-to-face meetings, regular electronic communication between participants can be held in order to work out the decisions taken during the face-to-face meeting. Such communication should be well-structured and well-organised, whereby a strong moderator will keep discussions on track. A more interesting conclusion in this respect is the relation of specific outcomes as described above, with a more abstract level of considerations based upon action theory and information-richness theory. The conclusion that 'group activity at the intentional level can be hypothesized to require rich media, activities at the functional level would require moderately rich media, and activities at the operational level would be best served with media of low richness' and its rationale in relation to reduction of uncertainty or equivocality, is a major contribution to the complex area of the task/media combinations and its rationalization.
The ease of use of different media and the switch from one to another can not be underestimated. Participants in a distributed research community must feel that the technology is 'at their service' all the time, without any doubts or misfunctioning. As a consequence, the robustness of the media has to be proven before it is included in the regular work of a distributed research community. This is not yet the case with video-conferencing.
The risk of organising a research project in a distributed way is that, because of the freedom of time and place with respect to personal involvement, local events become more dominant in the determination of an individual's time schedule. What is nearby gets more attention...Unless one is able to implement a carefully planned time schedule in which each task is given its fair share, distributed activities will always lose out over local activities.
Although there is some indication that the clearer the task and the more homogeneous the group, the better a distributed research community can work, this project did not allow itself to come to this kind of conclusion. Instead a number of 'possible guidelines' are given which remain rather 'open'. The important message however is that working at a distance makes some aspects which are 'hidden' in face-to-face meetings, explicit. There has to be an awareness and a willingnes to deal with such aspects. A clear statement is given about the necessity to have specific decision-makig rules, certainly in a asynchronous discussion. As an alternative, asynchronous discussions followed by a final synchronous decision-making (for instance through telephone conversation) is suggested, in particular when management decisions had to be taken.
The overview of technical facilities to support collaborative work results in the recommendation to use MOO or WOO environments, because they can be remodelled on the basis of current users' needs.
An interesting consequence of activities in a distributed research community, are the not-planned results, such as getting more proficiency in a foreign language (English), getting better insight in cultural differences, improved feeling about being a member of a larger (European or even world) community. Getting an easier way to cross the cultural and language barrier using technology even seems to be a major motivator to use a distributed research approach.