Richard A. Krueger

ABOUT US

FOCUS GROUP
INTERVIEWING


EVALUATION

STORY-TELLING

LINKS

HOME

OBSERVATION

How to Use Observation in Evaluation

by Richard A. Krueger

  1. Begin with a framework, a theory or an idea of what you are observing
    • Something is driving your study. You suspect that something is happening but you don't have evidence. Describe it in writing. This will clarify your thinking and allow others to offer feedback.
    • Too much is going on to observe everything. You must focus your attention on a limited number of areas, but stay open to new developments.
  2. Develop your observational strategy--develop a plan
    In your plan you should briefly describe: Who will observe, how will observers be prepared, what is the location, timing and duration of observation, how the study is explained to the subjects and how the subjects will be selected?
  3. Consider an observation checklist
    • Consider if a checklist would be helpful. Often checklists are useful when observing specific, unambiguous behaviors. The checklist usually has a rating scale that asks the observer to indicate presence or absence, frequency, intensity, speed, etc. As the behavior gets more complex checklists are less helpful.
    • In some situations you may be attempting to identify patterns, connections, sequences, etc. In these cases, use field notes to document what you observe. This latter case might involve observing sequence of events, how something is done, dexterity, skill, concentration, distractions, etc.
  4. Consider comparison
    A research strategy that has often been helpful is to make comparisons. Sometimes important findings don't emerge until you compare different locations or situations. Consider comparing:
    • from one location to another
    • from one time period to another
    • from one group (age, class, gender, etc.) to another
  5. Take detailed field notes
    Always take field notes, even when using a checklist. Record the date, time, situations and describe what you've observed. Sometimes you can take notes while you are observing, but in other situations the note taking might disturb the subjects. In these cases delay your note taking and then later as soon as possible record your notes in detail.
  6. Issues that often surface in observational studies
    • Do participants act differently because they are being observed?
    • How unobtrusive should the researcher be?
    • What do you tell participants when they ask what you are doing?

Suggestions for quality observations

  • Practice observing with others on the research team to be sure that the same things are being observed and recorded in a consistent manner.
  • Put full attention on what you are observing.
  • Do not draw attention to yourself.
  • Write detailed field notes. Describe what you see and also note the hunches or ideas that come to you while you're observing.
  • When in doubt, invite other members of the team to observe and offer suggestions.
  • Analysis and reporting begin with a brief statement of the problem that gave rise to the study. Write a paragraph describing how the study was conducted. In this paragraph describe the situation, environment, timing of observation, research team, subjects, etc.
  • Analysis can be in several forms. Aggregate the results if you are using a checklist. Look over your field notes. Look for patterns, themes, or overarching concepts. Draw sketches of concepts or relationships. Draw a picture of force fields or flow charts or time lines, etc. to illustrate the concepts.
  • When reporting results use bullets to highlight key points. If possible, compare and contrast findings across sites, categories, etc.

References

Patton, M. Q. (1987). How to use qualitative methods in evaluation. Newbury Park, CA: Sage.

Patton, M. Q. (1990). Qualitative evaluation and research methods (2nd ed.). Newbury Park, CA: Sage.

Taylor, S.J. & Bogdan, R. (1998) Introduction to qualitative research methods (3rd ed) New York: Wiley.

University of Wisconsin Extension Collecting Evaluation Data: Direct Observation http://www.uwex.edu/ces/pdande/Evaluation/evaluat.html

Back to EVALUATION

 
     

The views and opinions expressed in this page are strictly those of the page author.
The contents of this page have not been reviewed or approved by the University of Minnesota.