Measure Effects: Reality Check
WE'RE EXCITED THAT... |
|
|
BUT ARE CHALLENGED BECAUSE... |
| There are many ways to do citizen science, and many ways projects can make an impact. |
Different project models demand the identification of specific, measurable outcomes. Need to define the "continuum" of the citizen science engagement model (see some different models). |
||
|
|
|
|
|
| New projects are well suited to start with the end in mind, identifying project goals around which evaluations can be designed. |
Having a broad view of the "end" as more than just collecting data. Projects exist because of a complex network of individuals who need to be impacted in some meaningful way in order to keep up their involvement. |
WE'RE CHALLENGED... |
|
|
BUT SOME IDEAS ARE... |
| Thinking of effectiveness in the practice of science is not necessarily
intuitive for scientists. Scientists and evaluators are professional partners coming from
dramatically different backgrounds. |
There are similarities between science and evaluation. Partners need to find a way to speak a common language. |
||
|
|
|
|
|
| Determining the scope of what you will evaluate... you can't evaluate everything! | Evaluation provides incentives to focus project efforts around targeted goals. |
||
|
|
|
|
|
| Getting buy-in for evaluation from program developers. Finding and funding the infrastructure for evaluation. | |||
|
|
|
|
|
| Knowing when you have enough data for statistical significance. | Do a cost/benefit analysis to determine the most efficient scale of data collection. | ||
|
|
|
|
|
| Form fatigue/resistance... participants feel like they are being tested, rather than making contributions to project improvement. | |||
|
|
|
|
|
| Tech support staff changing systems in the middle of testing. | Communicate and work with technology staff to better understand the system, and to appropriately time changes to receive effective feedback. |
||
|
|
|
|
|
| Variability of participant experiences (background knowledge; experience with project). |
|||
|
|
|
|
|
| Evaluating multiple objects with a single measure. | |||
|
|
|
|
|
| Evaluating hard-to-measure outcomes, such as long-term effects. |
|||
|
|
|
|
|
| Using terms precisely (community, volunteer, participant...). |
Know of any opportunities for or challenges to this step? Soon you will be able to share them through our discussion forum.
 Previous Step |
Toolkit  Steps |
Next Step  |
|
(refine protocols) |
(train participants) |