resources
Using Interviews and Focus Groups in Language Program Evaluation
I. How-To Books on Interviewing
II. How-to Books on Focus Group Techniques
III. How-to Books on Qualitative Data Analysis
IV. Online Advice on Designing and Analyzing Interviews
V. Online Advice on Focus Group Techniques
VI. Examples of Interviews and Focus Groups in Language Program Evaluation
References marked with an asterisk (*) indicate those sources that we have found particularly helpful.
I. How-To Books on Interviewing
Bourque, L. B., & Fielder, E. P. (2002). The Survey Kit (2nd ed.): Vol. 4. How to Conduct Telephone Surveys. Thousand Oaks, CA: Sage.
Fowler, F. J., &
Mangione, T. W. (1990). Standardized
survey interviewing: Minimizing interviewer-related error.
Newbury Park, CA: Sage.
Frey, J. H., & Oishi, S. M. (1995). How
to Conduct Interviews by Telephone and in Person
(Survey Kit, Vol. 4). Thousand Oaks, CA: Sage.
Gubrium, J. F., & Holstein, J. A. (Eds.). (2002). Handbook of interview research: Context & method. Thousand Oaks, CA: Sage.
*Kvale, S. (1996). InterViews: An introduction to qualitative research interviewing. Thousand Oaks, CA: Sage.
NOTE: Kvale walks the reader through seven methodological stages of qualitative interview studies: thematizing, designing, interviewing, transcribing, analyzing, verifying, and reporting.
Maso, I., & Wester, F. (Eds.). (1996). The deliberate dialogue: Qualitative perspectives on the interview. Brussels: VUB University Press.
Mishler, E. G. (1986). Research
interviewing: Context and narrative.
Cambridge, MA: Harvard University Press.
Rubin, H. J., & Rubin, I. (2005). Qualitative
interviewing: The art of hearing data
(2nd ed.). Thousand Oaks, CA: Sage.
Seidman, I. E. (2006). Interviewing as qualitative research: A guide for researchers in education and the social sciences (3rd ed.). New York: Teachers College Press.
Weiss, R. S. (1995). Learning from Strangers: The Art and Method of Qualitative Interview Studies. New York: Free Press.
Wengraf, T. (2001). Qualitative research interviewing: Biographic narrative and semi-structured methods. London: Sage.
II. How-to Books on Focus Group Techniques
Fern, E. F. (2001). Advanced focus group research. Thousand Oaks, CA: Sage.
*Krueger, R. A., &
Casey, M. A. (2000). Focus
groups: A practical guide for applied research
(3rd ed.).Thousand Oaks, CA: Sage.
NOTE: A non-technical step-by-step guide on how to design focus group
studies, select participants, moderate focus group discussion, analyze
focus group data, and report results.
Morgan, D. L. (1997). Focus groups as qualitative research (2nd ed.). Newbury Park, CA: Sage.
*Morgan, D. L., &
Krueger, R. A. (Eds.). (1997). Focus
Group Kit (Focus Group Kit,
Vols. 1-6). Thousand Oaks, CA: Sage.
NOTE: This six volume set provides a straightforward blueprint for
people conducting focus groups, illustrating a number of key steps
involved in conducting focus group research. Below is the list of
volumes included in the kit.
(1) Morgan, D. L.
(1997). The focus group
guidebook (Focus Group Kit,
Vol. 1). Thousand Oaks, CA: Sage. (2) Morgan, D. L., & Scannell, A. U. (1997). Planning focus groups (Focus Group Kit, Vol. 2). Thousand Oaks, CA: Sage. (3) Krueger, R. A. (1997). Developing questions for focus groups (Focus Group Kit, Vol. 3). Thousand Oaks, CA: Sage. (4) Krueger, R. A. (1997). Moderating focus groups (Focus Group Kit, Vol. 4). Thousand Oaks, CA: Sage. (5) Krueger, R. A., & King, J. A. (1997). Involving community members in focus groups (Focus Group Kit, Vol. 5). Thousand Oaks, CA: Sage. (6) Krueger, R. A. (1997). Analyzing and reporting focus group results (Focus Group Kit, Vol. 6). Thousand Oaks, CA: Sage. |
*Stewart, D. W., &
Shamdasani, P. N. (1990). Focus
groups: Theory and practice.
Thousand Oaks, CA: Sage.
NOTE: The book helps the researcher attend to the intra- and inter-
personal and environmental variables when designing focus group
studies.
Vaughn, S., Schumm, J. S. & Sinagub, J. M. (1996). Focus group interviews in education and psychology. Thousand Oaks, CA: Sage.
III. How-to Books on Qualitative Data Analysis
*Boyatzis, R. E. (1998). Transforming
qualitative information: Thematic analysis and code development.
Thousand Oaks, CA: Sage.
NOTE: This book provides clear guidelines on the process of thematic
analysis, a method to encode and find patterns in qualitative data.
Coffey, A., & Atkinson, P. (1996). Making sense of qualitative data analysis: Complementary strategies. Thousand Oaks, CA: Sage.
Denzin, N. K., & Lincoln, Y. S. (Eds.). (2005). The Sage handbook of qualitative research. Thousand Oaks, CA: Sage.
Erlandson, D. A., Harris, E. L., Skipper, B. L., & Allen, S. D. (1993). Doing naturalistic inquiry. Newbury Park, CA: Sage.
Glesne, C. (2005). Becoming qualitative researchers: An introduction (3rd ed.). New York: Allyn & Bacon.
Maso, I., & Wester, F.
(Eds.). (1996). The
deliberate dialogue. Brussel:
VUB Press.
Miles, M. B., & Huberman, A. M. (1994). Qualitative
data analysis: An expanded sourcebook
(2nd ed.). Thousand Oaks, CA: Sage.
Patton, M. Q. (2002). Qualitative research & evaluation methods (3rd ed.). Thousand Oaks CA: Sage.
Silverman, D. (2001) Interpreting qualitative data: Methods for analysing talk, text and interaction (2nd ed.). London: Sage.
IV. Online Advice on Designing and Analyzing Interviews
General
Topics on Interviewing & Needs assessment decision aid
http://www.ceismc.gatech.edu/MM_Tools/analysis.html
Heidler, Reeves, and Brackett describe the advantages and disadvantages
of focus groups, interviews, and questionnaires, for the purpose of
identifying needs.
Overview
of qualitative methods and analytic techniques: Common qualitative
methods
http://www.nsf.gov/pubs/1997/nsf97153/chap_3.htm
Mahoney describes and compares the purposes, advantages, and
disadvantages of common qualitative methods: observation, in-depth
interviews, and focus groups. This site includes sample instruments in
the Appendices.
Online
interviews
http://www.geog.le.ac.uk/ORM/interviews/intcontents.htm
O'Connor introduces an online module for conducting synchronous
interviews (a real time online chat interview).
Designing
structured interviews for educational research
http://pareonline.net/getvn.asp?v=5&n=12
This article in Practical Assessment, Research & Evaluation
provides a brief introduction to problems and solutions for using
interviews in program evaluation.
Developing
interviews: Administering interviews
http://oerl.sri.com/module/mod7/m7_p1.html
Colker provides a module on how to set-up, conduct, follow-up, and
analyze interviews.
Developing
interviews: Preparing an interview protocol
http://oerl.sri.com/module/mod6/m6_p1.html
In conjunction with “Administering Interviews,”
Colker offers a module on how to prepare and structure interview
questions and interview protocols.
The
use of qualitative interviews in evaluation
http://ag.arizona.edu/sfcs/cyfernet/cyfar/Intervu5.htm
Sewell summarizes interview types, the designing of an interview, types
of analysis, and ethical issues in interviewing. This site also has a
short annotated bibliography which can be useful for learning more
about interview methodology.
Evaluate
programs: Interviews
http://www.utexas.edu/academic/diia/assessment/iar/programs/plan/method/interview.php
This very useful site from the University of Texas provides a step by
step guide to planning and conducting interviews, and transcribing and
coding data from interviews. In addition, it suggests ways of using
interview findings for evaluative purposes, including how to report.
V. Online Advice on Focus Group Techniques
General
Topics on Focus Groups & Focus group fundamentals
http://www.extension.iastate.edu/Publications/PM1969B.pdf
Grudens-Schuck, Lundy-Allen, and Larson summarize the features of focus
group methodology for evaluators to use in generating program-related
information.
Can
you call it a focus group?
http://www.extension.iastate.edu/Publications/PM1969A.pdf
Larson, Grudens-Schuck and Allen compare the use of focus groups with
small and large discussion groups when developing and evaluating
programs.
What
is a survey?
http://www.whatisasurvey.info/
Scheuren illustrates planning of and data collection in a survey study,
types of surveys, and concerns about error. Focus group techniques are
described in chapter 5.
Use
of Focus Groups in Program Evaluation
Using focus groups for evaluation
http://ag.arizona.edu/sfcs/cyfernet/cyfar/focus.htm
Marczak and Sewell describe the effective use of focus groups in
program evaluation and outline how they can be used in
Jacob’s (1988) five-tiered approach to evaluation.
Jacobs, F. H. (1988). The five-tiered approach to evaluation: Context
and implementation. In H. B. Weiss & F. H. Jacobs (Eds.),
Evaluating family programs, New York: Aldine DeGruyter.
Focus
groups
http://www.programevaluation.org/focusgroups.htm
New York State Teacher Centers Evaluation and Strategy Committee
introduces focus group questioning strategies and methodological
strategies when teacher development centers have limited time and
resources available.
VI. Examples of Interviews and Focus Groups in Language Program Evaluation
Using Interviews
Alderson, J. C., & Scott, M. (1992). Insiders, outsiders and
participatory evaluation. In J. C. Alderson & A. Beretta
(Eds.), Evaluating second
language education (pp. 25-57).
Cambridge: Cambridge University Press.
Chaudron, C., Doughty, C., Kim, Y., Kong, D., Lee, J., Lee, Y., Long, M. H., Rivers, R., & Urano, K. (2005). A task-based needs analysis of a tertiary Korean as a foreign language program. In M. H. Long (Ed.), Second language needs analysis (pp. 105-124). Cambridge: Cambridge University Press.
Gattullo, F. (2000). Formative assessment in primary (elementary) ELT classes: An Italian case study. Language Testing, 17(2), 278-288.
Horwitz, E. K. (1985). Formative evaluation of an experimental foreign-language class. Canadian Modern Language Review, 42(1), 83-90.
Karava-Doukas, K. (1998). Evaluating the implementation of educational innovations: Lessons from the past. In P. Rea-Dickens & K. P. Germaine (Eds.), Managing evaluation and innovation in language teaching: Building bridges (pp. 25-50). London: Longman.
Kieley, R. (1998). Programme evaluation by teachers: Issues of policy and practice. In P. Rea-Dickens & K. P. Germaine (Eds.), Managing evaluation and innovation in language teaching: Building bridges (pp. 78-104). London: Longman.
Liskin-Gasparro, J. E. (1995). Practical approaches to outcomes assessment: The undergraduate major in foreign
languages and literatures. ADFL Bulletin, 26(2), 21-27.
Long, M. H. (2005). Methodological issues in learner needs analysis. In M. H. Long (Ed.), Second language needs analysis (pp. 19-76). Cambridge: Cambridge University Press.
Lynch, B. K. (1992). Evaluating a program inside and out. In J. C. Alderson & A. Beretta (Eds.), Evaluating second language education (pp. 61-99). Cambridge: Cambridge University Press.
Lynch, B. K. (2000). Evaluating a project-oriented CALL innovation. Computer Assisted Language Learning, 13(4-5), 417-440.
Mitchell, R. (1989). Second language learning: Investigating the classroom context. System, 17(2), 195-210.
Mitchell, R. (1990). Evaluation of second language teaching projects and programmes. Language, Culture, and Curriculum, 3(1), 3-15.
Mitchell, R. (1992). The "independent" evaluation of bilingual primary education: A narrative account. In J. C. Alderson & A. Beretta (Eds.), Evaluating Second Language Education (pp. 100-140). Cambridge: Cambridge University Press.
Pawan, F., & Thomalla, T. G. (2006). Making the invisible visible: A responsive evaluation study of ESL and Spanish language services for immigrants in a small rural county in Indiana. TESOL Quarterly, 39(4), 683-705.
Rea-Dickins, P. (2001). Mirror, mirror on the wall: Identifying processes of classroom assessment. Language Testing, 18(4), 429-462.
Roberts, C. (1998). Language and cultural issues in innovation: The European dimension. In P. Rea-Dickens & K. P. Germaine (Eds.), Managing evaluation and innovation in language teaching: Building bridges (pp. 51-77). London: Longman.
Slimani, A. (1992). Evaluation of classroom interaction. In J. C. Alderson & A. Beretta (Eds.), Evaluating second language education (pp. 197-221). Cambridge: Cambridge University Press.
Snow, M. A., & Brinton, D. M. (1988). Content-based language instruction: Investigating the effectiveness of the adjunct model. TESOL Quarterly, 22(4), 553-574.
Weir, C., & Roberts, J. (1994). Evaluation in ELT.Oxford: Blackwell.
Using focus groups
Dassier, J-L. P., & Powell, W. (2001). Formative foreign language program evaluation: Dare to find out how good you really are. In C. C. Maurice (Ed.), The Odyssey Continues: Dimension 2001. Selected Proceedings of the Joint Conference of the Southern Conference on Language Teaching and the South Carolina Foreign Language Teachers' Association (pp. 91-110). GA: Valdosta State University.
Lynch, B. K. (2000). Evaluating
a project-oriented CALL innovation. Computer
Assisted
Language Learning, 13(4-5),
417-440.