September/October 08

Testing Tips: Announcing a Free Course on Oral Proficiency Assessment
By Margaret E. (Meg) Malone, Ph.D. - Center for Applied Linguistics

By now, many of us have settled into a routine for the school year (at least until weather and illnesses disrupt it!). Along with these routines come opportunities to assess students’ progress, as well as the challenges involved in regular assessment.

For many reasons, assessing students’ Interpersonal Communication and Presentational Speaking is among the greatest challenges instructors face. One challenge is that of time and student management; assessing Interpersonal Communication in a class of 25 students is difficult. Similarly, knowing what to listen for and grade students on while keeping the remaining 24 students engaged in meaningful tasks can be exhausting. Many of us have little or no background in assessment and even less in assessing speaking.

To help give language instructors a background in a nationally used rating scale, the National Capital Language Resource Center, through the Center for Applied Linguistics, is offering a one-month, five module course on oral proficiency assessment. The course, Assessment Training Online (ATOL), will be available November 1-30, 2008 and is offered free to the first 20 participants who sign up. The course includes the following five modules:

Module 1 introduceshe technology used throughout the course.

Module 2 provides an overview of oral proficiency testing.

Module 3 focuses on recognizing and describing the four major ACTFL levels.

Module 4 explores the structure of oral proficiency testing.

Module 5 reviews the nature of rating and holistic assessment.

You should expect to spend 3-5 hours per week on each module, including reading the material, listening to speech samples, participating in the weekly live chat and completing assignments. You must have a high-speed internet connection to participate.

Our hope in offering this course is to remove the mystery of oral proficiency testing and provide a solid background to language teachers.

Assessment Training Online (ATOL)

Purpose: The course provides language instructors with skills in oral proficiency assessment.

Dates: November 1 - 30, 2008

Cost: FREE to the first 20 participants

How to participate: Email Meg Malone . In the email, please include:

1. Your name;

2. Where you teach;

3. The language(s) you teach; and

4. One sentence about why you want to participate in the course.

Testing Tips Summer 2008
By Margaret E. (Meg) Malone, Ph.D. - Center for Applied Linguistics

Welcome to the latest issue of Testing Tips! In addition to other projects at CAL, I have spent much of the past couple of months conducting face-to-face and online workshops on assessments for language instructors. On issue that often emerges is what test (or tests) to use and whether one test can be used for every need you have. The more language teachers I meet, the more amazed I am by how much teachers accomplish every day, and how scarce the assessment resources are to help these teachers. As we’ve discussed before, it is important to align your purpose for testing with the test you choose. We’ve also talked about how to align the audience for testing with the test you choose. For example, if Mr. Enriquez wants to find out how well his students listen and read in Spanish, he should administer them a test of Spanish listening and reading. But that’s not enough; we need to make sure that the test is appropriate for his students’ and their age level, how much Spanish they have learned, and so on.

I receive calls from teachers like the mythical Mr. Enriquez every day. I often refer such calls to two major online resources. One resource is the Center for Advanced Research on Language Acquisition’s Virtual Assessment Center (VAC). The VAC includes a series of modules that will give you background, guidance and hands-on practice to develop your own or think about how to select your own test.
http://www.carla.umn.edu/assessment/VAC/index.html

The second resource is the Foreign Language Assessment Directory, which is a free, searchable directory of language tests. It provides information on over 150 language tests in over 60 languages other than English.
http://www.cal.org/CALWebDB/FLAD/

I recommend that you look at both resources. Then, go to the FLAD and look for a test that you think you could use with your students. Then ask yourself the following questions about the test,

Back to top

Testing Tips: Testing needs

Last month, I wrote about audiences for testing and focused on students and the testing process. This month, we're going to think about a larger range of audiences and their needs both in the actual testing situation and in reporting student outcomes. To maximize test results, it is most important to plan for assessment to make sure that the needs of all groups- students, instructors, administrators, parents, the business community- will be met by the results. It's often easier to imagine how to include different audiences in the testing process if the purpose and context of a test are clearly defined. Here’s an example: Ms. Parks, a middle school teacher, is testing her students. The purpose of the assessment is an end-of-year test in a sixth grade Spanish class to test how well students have learned the content of the course. The context is one middle school, in a mid-sized district which has been teaching a pilot Spanish course to sixth graders instead of waiting to begin language instruction in seventh grade.

Questions Ms. Parks may want to consider:
• Who might want information about the results of the test?
• How will the test outcomes potentially impact these groups??
Students have a direct relationship with their parents, and instructors have a direct relationship with the administrators. Therefore, to clarify the issue, we might wish to list all the audiences for the test and its results, and describe their needs for both the actual assessment process and for reporting results of assessments.

If you want more information on this topic, I have listed a few good references below for thinking about assessment and stakeholders. Now that we've discussed all the stakeholders' needs, do you think there is one test that will meet the purpose and address everyone's needs? Stay tuned...

Annotated References:
Stoynoff, S. & Chapelle, C. (2005) ESOL tests and testing. Alexandria, VA: TESOL.
Short, accessible book which introduces basic concepts of language testing and reviews 20 English language tests

Bachman, L. & Palmer, A. (1996). Designing and developing useful language tests. Oxford: Oxford University Press.
Book with in-depth information on measurement, language test uses and methods, reliability, and validity

Brown, H.D. (2003). Language assessment: principles and classroom practice. New York: Pearson ESL.
A practical guide to developing your own classroom assessments

Hughes, A. (2003). Testing for language teachers (2nd edition). Cambridge: Cambridge University Press.
A book which provides a thorough but accessible overview of foundational concepts in language testing

McTighe, J. & Wiggins, G. (2005). Understanding by design (2nd ed). Alexandria, VA: Association for Supervision and Curriculum Development.
Handbook which explains the principles of backward design for classroom assessment

Back to top

Audiences for Testing

Welcome to the latest issue of Testing Tips! I've met many of our newsletter readers face-to-face at the whirl of conferences that comprises the late winter and early spring. As I have met many of you and discussed testing with our colleagues, I thought a great deal about audiences for testing.

I promised to write about audiences for testing this month. Audience includes the students you're testing, in addition to all those who are impacted by the results: parents, students, you, other teachers, administrators, the community, and more. This month, I will focus on the student audience, and next month, we'll talk about everyone else.

WHOM am I testing ? That may seem very straightforward, because we're usually testing our students. However, it's important to think about all of our students and all of their needs. Here are the questions we should ask about our students. Number of Students in a class. This is important for you and for them. It's important to design and select tests for the right number of students. With some of our oral proficiency tests, we find that students have difficulty concentrating if they are sitting too close to another student who is also responding aloud.

Student developmental level. In mid-March, I was trying out test items with students in grades 1-11. It amazed me, as it always does, to see how differently students understand what they are supposed to do on a test based on their age, developmental level and experience. Developmental growth, especially reading, is especially important for directions. Sometimes, students can't (or don't!) read the directions, so they respond incorrectly. Then, we don't know if the student didn't know the material or if s/he jus didn't understand the directions. This is especially important if you are working with a new group of students or if you are using a standardized test. It's important to make sure that students understand what they are supposed to do so that they can show you how much they have learned. Experience. What kind of experience do students have with testing? If we're presenting them with the first oral proficiency test of their life, we need to provide them with support and experience so that they can show what they can do. If they are taking a computer-based test, we need to make sure that they know how to use the computer.

Special needs. If our students have special needs, we need to consider these and accommodate them in our tests as well as our classes. This is a lot to think about. Next months, we'll talk about how to incorporate students' needs with your needs, as well as our other audiences: parents,, other teachers, administrators, the community, the school board.....

Back to top

The Process of Making a Test

As promised, this month's Testing Tips discusses the process of making a test. Teachers all over the world create tests, grade them, record the results, return the results to students and field questions from students, parents and administrators about these results. Organizations and assessment professionals also design tests. A great deal of time goes into test development, administration, scoring, recording, reporting and revising the tests.

So, what do you do first? This column deals with the first step in selecting or creating a test, which is to define your purposes for testing. Next, you can explore the type of test that might work for your situation.

Define your purpose. Before any person, group or organization makes a test, the first step is to define the purpose for testing. The first question is basic: why are you testing your students? Classroom teachers have a number of reasons for conducting assessments, and they may use test results to assign grades, place students in a course, reflect on their own teaching or make other decisions about their students and programs. Here are some example responses, based on emails I receive from language teachers:

1. I want to know if students have learned what I taught;
2. My program needs to place students from all over the country in the language program;
3. I'm hiring someone, and I need to make sure they can communicate in the language; and
4. I need to show the taxpayers that my program is working.

Before we talk about some basic types of tests, reflect on whether we should use the same kind of test for (1) as for (3). Would we use the same test for (2) as (1)? The answer: it depends.

Types of tests. Once you’ve defined why you are testing your students, you can begin to think about the type of test that might match your needs. There are many types of tests, and I won't describe them all (although the short bibliography refers to colleagues who have defined tests much better than I ever could).

• Do you want to know how much students have learned (or to what extent they have internalized the wisdom of their teacher) in a day, week, chapter, unit or course? An achievement test provides information about how much students have learned and is directly related to the content of a course or program. Many classroom tests are achievement tests. #1 above is an example of an achievement test.


• Do you want to know how students can perform a function in a close-to-real-life setting, such as (my personal favorite) ordering a meal or buying a ticket in the target language? Or, how about the example of interpreting for a doctor, nurse or other medical personnel to make sure that the patient understands her diagnosis? A performance test requires examinees to demonstrate knowledge or a skill through an activity or performance.


• Do you want to know which level of language class is right for students entering one institution from various feeder institutions ? Placement tests are used to make decisions about student placement in a course or program. #3 is an example of a placement test; another is placing students from a number of middle schools into high school language classes.


• Do you want to find out about students’ general ability in the target language? Proficiency tests measure general language ability. A proficiency test may be appropriate for example #4 because the results would show students’ general ability rather than achievement in a specific course.

As I mentioned in the first column, one size does not fit all in testing. It’s important to match the reason you’re testing with the type of test you select or create.

Next month, we'll discuss audiences for test results and how that may affect the testing process

References

- Bachman, L. & Palmer, A. (1996). Designing and developing useful language tests. Oxford: Oxford University Press.
- Brown, H.D. (2003). Language assessment: principles and classroom practice. New York: Pearson ESL.
- Hughes, A. (2003). Testing for language teachers (2nd edition). Cambridge: Cambridge University Press.
- International Language Testing Association Code of Practice. http://www.iltaonline.com/code.htm
- McTighe, J. & Wiggins, G. (2005). Understanding by design (2nd ed). Alexandria, VA: Association for Supervision and Curriculum Development.
- Stoynoff, S. & Chapelle, C. (2005) ESOL tests and testing. Alexandria, VA: TESOL.

Back to top

Testing Tips
By Margaret E. (Meg) Malone, Ph.D. - Center for Applied Linguistics

Welcome to NCLRC’s newest offering "Testing Tips!" Offered every month in the NCLRC newsletter, it will provide insight into foreign language testing for teachers, administrators and researchers. I hope you will send me your questions, and I will try to address them here.

Why is the newsletter focusing on testing? Since long before the passage of No Child Left Behind, testing has been important to provide feedback to learners and teachers. Since No Child Left Behind, K-12 testing has moved from the classroom to the front page. In this column, I’ll write about good testing resources (many available from our sister Language Resource Centers), the basics of testing, and your questions. Note that I haven’t said I would answer your questions but address them, because testing is not a “one size fits all” endeavor. Instead, it should be tailored to the specific context in which it occurs. Next month, I’ll write about the testing process and what goes into making a test.

If you have a testing question for Dr. Malone, please email it to her at: info@nclrc.org