The following are publications and presentations produced by staff and participants in conjunction with the Foreign Language Program Evaluation Project. Where possible, we will provide direct links to related web-based documents. This page is updated frequently.

 

Publications

BACK to TOP

 

Norris, J. M., & Watanabe, Y. (under contract). Program evaluation. Blackwell Encyclopedia of Applied Linguistics.

Norris, J. M., & Ortega, L. (under contract). Assessing learner knowledge. In S. M. Gass & A. Mackey (Eds.), Handbook of second language acquisition. NewYork: Routledge.

 

Davis, J. McE. (2011). Using surveys for understanding and improving foreign language programs. (NetWork #61) [PDF document]. Honolulu: University of Hawai´i, National Foreign Language Resource Center. doi: http://hdl.handle.net/10125/14549

 

Byrnes, H., Maxim, H., & Norris, J. M. (2010). Realizing advanced FL writing development in collegiate education: Curricular design, pedagogy, assessment. Modern Language Journal, Monograph. Cambridge: Blackwell.
 

Norris, J. M., Davis, J. McE., Sinicrope, C., & Watanabe, Y. (Eds.) (2009). Toward useful program evaluation in college foreign language education. Honolulu: University of Hawai‘i, National Foreign Language Resource Center.

 

Norris, J. M. (2009). Introduction to the volume. In J. M. Norris, J. McE. Davis, C. Sinicrope, & Y. Watanabe, (Eds.), Toward useful program evaluation in college foreign language education (pp. 1-3). Honolulu: University of Hawai‘i, National Foreign Language Resource Center.

 

Davis, J. McE., Sinicrope, C., & Watanabe, Y. (2009). College foreign language program evaluation: Current practice, future directions. In J. M. Norris, J. McE. Davis, C. Sinicrope, & Y. Watanabe (Eds.), Toward useful program evaluation in college foreign language education (pp. 209–226). Honolulu: University of Hawai‘i, National Foreign Language Resource Center.

 

Watanabe, Y., Norris, J. M., & González-Lloret, M. (2009). Identifying and responding to evaluation needs in college foreign language programs. In J. M. Norris, J. McE. Davis, C. Sinicrope, & Y. Watanabe, (Eds.), Toward useful program evaluation in college foreign language education (pp. 5-58). Honolulu: University of Hawai‘i, National Foreign Language Resource Center.

 

Norris, J. M. (Ed.) (2009). Special issue: Understanding and improving language education through program evaluation. Language Teaching Research, 13(1).

 

Norris, J. M. (2009). Understanding and improving language education through program evaluation: Introduction to the special issue. Language Teaching Research, 13(1), 7-13.

 

Watanabe, Y. (2009). Naibusha niyoru naibushanotameno jituyoujyuushigata genngokyouiku program hyouka. [Utilization-focused evaluation by and for the internal staff]. Proceedings of the Society for Teaching Japanese as a Foreign Language, 92-95.

 

Watanabe, Y., Sylwester, B., & Norris, J. M. (2009). Foreign language program evaluation: An annotated bibliography of resources for foreign language educators. Honolulu: University of Hawai‘i, National Foreign Language Resource Center. http://www.nflrc.hawaii.edu/evaluation/biblio/index.cfm

 

Norris, J. M. (2008). Validity evaluation in foreign language assessment. New York: Peter Lang.

 

Faculty Working Group on Foreign Language Program Evaluation. (2007). Developing useful evaluation practices in college foreign language programs: Showcase session. (NFLRC NetWork #48) [PDF document]. Honolulu: University of Hawaii, National Foreign Language Resource Center. http://nflrc.hawaii.edu/networks/NW48.pdf

 

Mackay, R. (2007). Foreign language program evaluation project: External evaluation report. Honolulu, HI: Foreign Language Program Evaluation Project, University of Hawaii at Manoa.[view report]

 

Sinicrope, C., Norris, J.M., and Watanabe, Y. (2007). Understanding and assessing intercultural competence: A summary of theory, research, and practice. Honolulu, HI: Technical report for the Foreign Language Program Evaluation Project. [view report]

 

Norris, J. M. (2006). Assessing foreign language learning and learners: From measurement constructs to educational uses. In H. Byrnes, H. Weger-Guntharp, & K. Sprang (Eds.), GURT 2005: Educating for Advanced Foreign Language Capacities: Constructs, Curriculum, Instruction, Assessment (pp. 167-187). Washington, DC: Georgetown University Press.

 

Norris, J. M. (2006). Development and evaluation of a curriculum-based German C-test for placement purposes. In R. Grotjahn (Ed.), Der C-Test: Theoretische Grundlagen und praktische Anwendungen (vol. 5) (pp. 45-83). New York: Peter Lang.

 

Norris, J. M. (2006). The why (and how) of student learning outcomes assessment in college FL education. Modern Language Journal, 90(4), 590-597.

 

Norris, J. M. (2006). [Review of B. Lynch (2003). Language assessment and programme evaluation.] Studies in Second Language acquisition, 28(3).

 


Presentations

BACK to TOP

 
Watanabe, Y. (2011, November). Fostering evaluative thinking, values, and identity through outcomes assessment in college humanities programs. Paper presentation and panel organizer for the panel session: "Using evaluation perspectives to develop organizational capacity to conduct useful student learning outcomes assessment in higher education" at the American Evaluation Association 2011 Conference, Anaheim, CA.

 
Watanabe, Y. (2011, October). Fostering evaluative thinking through outcomes assessment in academic departments. Paper presentation at the Assessment Institute 2011, Indianapolis, IN.

 
Norris, J. M., Davis, J. McE., Sylwester, B., & Watanabe, Y. (2011, July). Making the most of program evaluation. Workshop organized for the annual meeting of the Western Consortium of Middle Eastern Language Programs, University of Texas, Austin, TX.

 
Norris, J. M. (2011, July). High-value evaluation strategies in foreign language education. Keynote address presented at the annual meeting of the Western Consortium of Middle Eastern Language Programs. University of Texas, Austin, TX. [view powerpoint]

 
Davis, J. McE. (2011, July). Using surveys for understanding and improving foreign language programs. Workshop delivered at the annual meeting of the Western Consortium of Middle Eastern Language Programs. University of Texas, Austin, TX. [view powerpoint and handout]

 
Akbari, N. (2011, July). Using program logic models to understand and improve Persian language programs. Paper presented at the annual meeting of the Western Consortium of Middle Eastern Language Programs. University of Texas, Austin, TX. [view powerpoint]

 
Raizen, E. L., & Caravita, J. (2011, July). The use of 'Sabras' as mentors for advanced Hebrew students. Paper presented at the annual meeting of the Western Consortium of Middle Eastern Language Programs. University of Texas, Austin, TX. [view powerpoint]

 
Sylwester, B., & Watanabe, Y. (2011, July). Why outcomes-based evaluation? In search of value and impact. Paper presented at the annual meeting of the Western Consortium of Middle Eastern Language Programs. University of Texas, Austin, TX. [view powerpoint & handout]

 
Norris, J. M., Akbari, N., Al-Batal, M., Belnap, K. (2011, July). How can we make the most of the mandated program review process? Roundtable discusssion at the annual meeting of the Western Consortium of Middle Eastern Language Programs. University of Texas, Austin, TX. [view handout]

 
Norris, J. M., Okal, A., Raizen, E., & Schulte-Nafeh, M. (2011, July). Assessing and otherwise gathering data on diverse program outcomes: Moving beyond 'how do we measure?' Roundtable discusssion at the annual meeting of the Western Consortium of Middle Eastern Language Programs. University of Texas, Austin, TX. [view handout]

 
Norris, J. M. (2011, May). Linking assessments to actions: Moving from doing to using assessment. Workshop delivered at the Center for the Study of Languages, Rice University, Houston, TX.

 
Norris, J. M. (2011, May). Constructs and consequences: Toward useful assessment in the humanities. Invited presentation at Rice University, Houston, TX.

 
Norris, J. M. (2011, March). The value of evaluation in language education. Plenary address delivered at the XIII Jornadas de Lingüística, Universidad de Alicante, Spain.

 
Davis, J. McE. (2010, November). Proactive and productive reactions to externally mandated university outcomes assessment. Round table session at the American Council on the Teaching of Foreign Langauges, Boston, MA. 

 

Watanabe, Y. (2010, November). Organizational readiness for outcomes assessment in higher education. Paper presentation at the American Evaluation Association Conference: Evaluation 2010, San Antonio, TX. 

 

Norris, J. M. (2010, October). Outcomes assessment in higher education: Impetus, utility, and consequence. Invited roundtable presentation at the Consortium on Useful Assessment in Language and Humanities Education, Georgetown University, Washington, DC.

 

Norris, J. M. (2010, October). A decade of work on task-based language assessment. Paper presented on the invited colloquium “Reprising the role of tasks in language assessment” at the annual Second Language Research Forum, College Park, MD.

 

Norris, J. M. (2010, October). Validity evaluation of assessments for understanding and improving language education. Invited presentation to the Linguistics Department at Georgetown University, Washington, DC 

 

Norris, J. M. (2010, September). Task-based language teaching. Invited roundtable in Applied Linguistics, New York University, New York. 

 

Norris, J. M. (2010, September). Understanding instructed SLA: Constructs, contexts, and consequences. Plenary address delivered at the Annual Conference of the European Second Language Association (EUROSLA), Reggio Emilia, September 2, 2010.

 

Watanabe, Y. (2010, September). A multiple-case study of factors affecting outcomes assessment in higher education. Paper presentation at the Hawaii-Pacific Evaluation Association Conference, Honolulu, HI.


Norris, J. (2010, March). Outcomes assessment in higher education: Impetus, utility, and consequence. Panel discussion at the University of Notre Dame, South Bend, IN.

Pfeiffer, P. (2010, March). Practicing assessment: An example case study from Georgetown University. Panel discussion at the University of Notre Dame, South Bend, IN.

Watanabe, Y. (2010, March). Building a scholarship of outcomes assessment in college foreign language programs. Panel discussion at the University of Notre Dame, South Bend, IN. 

 

Norris, J. M. (2009, September). Reframing the evaluation of task-based language education. Paper presented at the refereed colloquium “Evaluating task-based language programs”, at the 3rd International Conference on Task-Based Language Teaching, Lancaster, UK (September 14, 2009). [view powerpoint]

 

Watanabe, Y. (2009, September). Culturally-appropriate focus group for program evaluation. Workshop for the Kauai Community College, University of Hawaii Kauai Community College, Lihue, HI.

 

Norris, J. M. (2009, May). Assessment and evaluation: Alternatives for inquiry into language learning and teaching. Invited address for the Language Learning Distinguished Scholar in Residence Award, Katholieke Universiteit Leuven, Belgium (May 19, 2009).

Watanabe, Y. (2009, May). Utilization-focused evaluation for the intended users, by the intended users: Values, steps, and examples. Colloquium paper a the Society for Teaching Japanese as a Foreign Language Spring Conference, Chiba, Japan. 

 

Norris, J. M. (2009, April). Prioritizing the use and usefulness of evaluation in understanding and improving student learning. Keynote address given at the Multi-Language Western Consortium Workshop, University of Texas at Austin. (April 17, 2009). [view powerpoint]


Watanabe, Y., & Davis, J.McE. (2009, April). A utilization-focused approach to stating and evaluating student learning outcomes. Workshop given at the Multi-Language Western Consortium Workshop, University of Texas at Austin. (April 17, 2009). [view workbook]

 

Davis, J.McE., & Watanabe, Y. (2009, April). Lessons learned from a college-wide student learning outcomes evaluation project: Balancing external and internal impetuses. Presentation delivered at the Multi-Language Western Consortium Workshop, University of Texas at Austin. (April 18, 2009). [view powerpoint]

 

Norris, J. M. (2009, April). Assessment and evaluation: Alternatives for inquiry into language learning and teaching. Plenary address delivered at the annual Spring Second Language Studies Symposium, Michigan State University, East Lansing (April 3, 2009). [view powerpoint]

 

Norris, J. M. (2009, March). Organizer of the invited colloquium “Critical issues at the interface between assessment and U.S. language education policy”, at the annual conference of the American Association of Applied Linguistics, Denver, CO (March 22, 2009). [view powerpoint]

 

Norris, J. M., & Liskin-Gasparro, J. (2009, March). The consequences of accreditation and outcomes assessment for college foreign language programs. Paper presented at the invited colloquium “Critical issues at the interface between assessment and U.S. language education policy”, at the annual conference of the American Association of Applied Linguistics, Denver, CO (March 22, 2009).[view powerpoint]

 

Watanabe, Y. (2009, February). Utilizing focus group ("Talk-story") for evaluation purposes. Workshop for the Achieving the Dream Initiative, University of Hawaii Community College System, Honolulu, HI. 

 

Norris, J. M. (2009). A conversation on accreditation and assessment. Invited presentation to the Theology Department, University of Notre Dame, South Bend, IN (March 30, 2009). [view handout]

 

Norris, J. M. (2008, August). Evaluating task-based language teaching and learning (research) from an educational programs perspective. Paper presented at the refereed colloquium “Task-based language learning, and task-based teaching. How are they related”, at the 15th World Congress of Applied Linguistics (AILA), Essen, Germany (August 25, 2008).

 

Norris, J. M. (2008, April). Constructs and consequences: Toward useful assessment in the humanities. Invited presentation at the University of Notre Dame, South Bend, IN (April 3, 2008). [view handout]

 

Norris, J. M., & Ortega, L. (2008, April). Measurement for understanding: The case of complexity, accuracy, and fluency. Paper presented at the invited colloquium “Complexity, accuracy, and fluency in second language acquisition: Theoretical and methodological perspectives”, annual conference of the American Association of Applied Linguistics, Washington, DC (April 1, 2008).

 

Norris, J. M. (2008, March). Making the most of assessment and evaluation in college foreign language programs. Invited presentation at the Language Resource Center, Cornell University, Ithaca, NY (March 26, 2008).

 

Norris, J. M. (2008, March). Assessing intercultural (communication) learning: Indicators, methods, uses. Invited workshop at Duke University, Durham, NC (March 21, 2008). [view workbook]

 

Norris, J. M. (2008, March). Assessment and evaluation in the humanities: Retrospect and prospect. Invited presentation at Duke University, Durham, NC (March 20, 2008).

 

Watanabe, Y., & Sinicrope, C. (2008, March). Linking the how to the why: Methods decisions in foreign language program evaluation. Invited workshop at the Foreign Language Program Evaluation Summit, University of Hawaii, Honolulu, HI (March 13, 2008). [view powerpoint] [view handout] [view resources] [view worksheets]

 

Norris , J. M., & Watanabe, Y. (2007, November). Roles and responsibilities for evaluation in foreign language programs. Invited session delivered at the annual meeting of the Japan Association of Language Teachers, Tokyo, Japan (November 24, 2007). [view powerpoint] [view booklet]

 

Norris, J. M. (2007, November). Learning to value evaluation in language teaching. Plenary address delivered at the annual meeting of the Japan Association of Language Teachers, Tokyo, Japan (November 23, 2007). [view powerpoint] [view handout]

 

Norris, J. M. (2007, October). Using assessment for understanding and improving language education. Invited presentation at the Berkeley Language Center, University of California, Berkeley, CA (October 12, 2007). [view handout]

 

Norris, J. M. (2007, September). Making the most of assessment and evaluation in college foreign language programs. Invited presentation at the Language Institute, University of Wisconsin, Madison, WI (September 10, 2007). [view handout]

 

Norris, J. M. (2007, September). Use-oriented assessment and evaluation. Invited workshop at the Language Institute, University of Wisconsin, Madison, WI (September 10, 2007).

 

Watanabe, Y., & Hill, Y. (2007, September). User and use focused conference evaluation: Collaboration, negotiation, and action. Poster presented at the 2nd Annual Hawaii-Pacific Evaluation Conference, Honolulu, HI. [view poster pdf]

 

Norris, J. M. (2007, June). Understanding program evaluation in college foreign language education. Plenary address delivered on the invited panel “Toward useful program evaluation in college FL departments” at the annual summer meeting of the Association of Departments of Foreign Languages, University of Hawaii, Honolulu (June 8, 2007).

 

Norris, J. M. (2007, May). Making the most of assessment and evaluation in college foreign language programs. Invited presentation at the Department of Romance Languages, University of Notre Dame, South Bend, IN (May 4, 2007). [view handout]

 

Norris, J. M., & Liskin-Gasparro, J. (2007, May). Linking assessments to actions: Moving from doing to using assessments. Invited workshop at the Department of Romance Languages, University of Notre Dame, South Bend, IN (May 4, 2007). [view handout]

 

Norris, J. M., (2007, April). Educational context, epistemological competition, and the future of task-based research: A program evaluation perspective. Paper presented at the refereed colloquium “Towards and educational agenda for research into task-based language teaching”, Social and Cognitive Perspectives on Language Learning and Teaching conference, Auckland, New Zealand (April 13, 2007).

 

Watanabe, Y., & Pang, A. (2007, March). Program evaluation: Utilizing needs analysis for decision making. Paper presented at the 41st TESOL Annual Convention, Seattle, WA. [view handout]

 

Watanabe, Y., Gonzalez-Lloret, M., Hill, Y., & Yang, W. (2007, February). Putting Useful Program Evaluation into Practice. Invited presentation at the Second Lantuage Teaching & Curriculum Center, Demos & Discussion Workshop, University of Hawaii at Manoa, Honolulu, HI (February 9, 2007). [view handout]

 

Watanabe, Y. (2007, Februrary). Utilizing program evaluation for program improvement and articulation. Poster presented at the The 2007 Hawaii TESOL Conference, Honolulu, HI.

 

Norris, J. M. (2006, November). Assessment and evaluation in college foreign language programs: Past, present, and future tense. Invited presentation at the School of Foreign Languages, University of Maryland, College Park, MD (November 27, 2006).

 

Norris, J. M. (2006, November). The transformative potential of assessment in college foreign language education. Paper presented at the invited colloquium “The outcomes of collegiate FL programs: Specification, curricular implementation, assessment”, annual conference of the American Association of Teachers of German, Nashville, TN. [view handout]

 

Norris, J. M., Watanabe, Y., & González-Lloret, M. (2006, November). Program evaluation in college foreign language education: Need, capacity, development. Session presented at the annual convention of the American Council on the Teaching of Foreign Languages, Nashville, TN. [view brochure]

 

Watanabe, Y. (2006, November). Taking programmatic actions through program evaluation. Poster presented at the Dynamic Teacher Communities: Professional & Organizational Development Symposium, Saitama, Japan.

 

Norris, J., Gonzalez-Lloret, M., & Watanabe, Y. (2006, September). Identifying and responding to evaluation needs in college foreign language programs. Poster presented at the 1st Conference of the Hawaii-Pacific Evaluation Association, Honolulu, HI. [view poster pdf]

 

Norris, J. M. (2006, June). Using assessment for understanding and improving college FL curriculum. Paper presented in the invited colloquium "Assessing Student Competencies, the Curriculum, and Faculty Members in New Research Areas", at the annual summer meeting of the Association of Departments of Foreign Languages, University of Wisconsin, Madison (June 30, 2006). [view handout] [view powerpoint]

 

Norris, J. M. (2006, February). First things first: Why assess in the liberal arts? Paper presented in the invited colloquium "How to assess success in a liberal arts education", at the Arts and Sciences Symposium: The value/values of a liberal arts education, University of Hawai'i (February 7, 2006).

 

Norris, J. M., & Watanabe, Y. (2005, November). Responding to program evaluation demands in college foreign language education. Session presented at the annual convention of the American Council on the Teaching of Foreign Languages (ACTFL), Baltimore, MD (November 18, 2005). [view handout]

 

Norris, J. M. (2005, September). The essential roles of assessment, measurement, and evaluation in task-based language teaching. Plenary address presented at the First International Conference on Task-Based Language Teaching, University of Leuven, Belgium (September 23, 2005). [view powerpoint]

 

Norris, J. M. (2005, September). Invited discussant for the symposium "Taking text to task: Issues and choices in curriculum development", at the First International Conference on Task-Based Language Teaching, University of Leuven, Belgium (September 23, 2005).

 

Norris, J. M. (2005, March). Assessing advanced foreign language learning and learners: From measurement constructs to educational uses. Plenary address delivered at the 54th annual Georgetown University Round Table on Languages and Linguistics, Georgetown University, Washington, DC (March 12, 2005).

 

Norris, J. M. (2004, June). Meeting the diverse challenges of assessment in college foreign language programs. Plenary address delivered at the annual summer meeting of the Association of Departments of Foreign Languages, University of New Mexico, Albuquerque (June 11, 2004). [view handout]

 

 

 

© 2007 John Norris, Yukiko Watanabe, Marta Gonzalez-Lloret & Hye Ri Joo