Software System Development: A Gentle Introduction
Software System Development: A Gentle Introduction
Information, Systems and Information Systems: Making Sense of the Field
Information, Systems and Information Systems: Making Sense of the Field
Ten steps applied to development and evaluation of process-based biogeochemical models of estuaries
Environmental Modelling & Software
Computers and Electronics in Agriculture
A review of the factors which influence the use and usefulness of information systems
Environmental Modelling & Software
Simple software processes and tests improve the reliability and usefulness of a model
Environmental Modelling & Software
Environmental Modelling & Software
Forage rummy: A game to support the participatory design of adapted livestock systems
Environmental Modelling & Software
Position paper: Characterising performance of environmental models
Environmental Modelling & Software
Information systems for marine protected areas: How do users interpret desirable data attributes?
Environmental Modelling & Software
Innovative approaches to integrated global change modelling
Environmental Modelling & Software
Exploring the application of participatory modeling approaches in the Sonora River Basin, Mexico
Environmental Modelling & Software
Hi-index | 0.00 |
The intention of this paper it to open up debate within the environmental modelling and software (EMS) community on how best to respond to the increasing desire to evaluate the success of EMS projects in terms of outcomes rather than outputs. Outcomes in these regards are changes beyond the walls of the research organisation (typically to values, attitudes and behaviour). The authors recognise that outcome evaluation is essential in ensuring the relevance and effectiveness of activities. To date, however, there is a limited appreciation within the EMS community of the nature of the challenge inherent in outcome evaluations. The paper presents an exploratory analysis of the challenges that outcome assessment raises for EMS. It does so using mutually reinforcing conceptual and practical perspectives. The paper presents a conceptual framework of three loosely coupled phases - research, development and operations. The nature of activities and their interactions within these phases is outlined and the forms of evaluation associated with each stage set out. The paper notes how existing forms of evaluation (e.g. peer review, validation and relevance) underpin the delivery of outcomes but do not of themselves evaluate outcomes. The paper proposes that outcomes need conceptually to be seen as an element of complex social processes mediated by government, regulation, markets and the media rather than as simply another form of output from research and development projects. As such outcomes of EMS are: less easily tangible than are outputs; more likely to occur at a significant time lag after any intervention; more difficult to assign causality for and to be subject to significant contestation. Thus EMS activity, however well conducted technically, may only have a minor influence on outcomes and EMS practitioners will have limited control over those outcomes that do occur. The paper uses a series of linked EMS projects to populate the conceptual framework showing the role of evaluations in research, development and operations phases. The paper then presents two forms (quantitative and qualitative) of outcome evaluation used as part of an operational phase evaluation of a project communicating the consequences of climate change to remote-rural land managers in Scotland. The authors conclude that while the challenges of EMS evaluation can be met, there needs to be care from the EMS community not to raise expectations of outcomes that cannot be met.