Student Assessment and Program Evaluation

Program:

The Immigrant Learning Center, Inc. (ILC)
442 Main Street
Malden, Massachusetts 02148
Tel. 781.322.9777 Fax. 781.321.1963
www.ilctr.org
facebook.com/ImmigrantLearningCenter
twitter.com/ilctr

Contact:

Karen Oakley, Director of English Language Programs
koakley@ilctr.org

“Useful programming ideas can and do come from any number of sources—research, students, teachers, other staff, volunteers, the Board of Directors and student council. But if you don’t have systems in place not only to elicit that input, but then to respond to it, it’s as if you never asked those people, those resources for feedback in the first place.”

~ Diane Portnoy, Co-Founder and Director

Mission

Established in 1992, the mission of The Immigrant Learning Center, Inc. (ILC) is to provide enough knowledge of the English language to immigrant and refugee adults to help them lead productive lives in the United States and to become successful parents, workers and community members. The ILC also educates the public including policy makers, teachers, and media about the positive impact immigrants have on the country, the economy and our communities.

Services

Since inception, The ILC has served more than 7,600 immigrants and refugees. With our current programming, approximately 900students enroll annually. Students speaking 69 native languages have come from 118countries and 81 communities in the Boston area. The ILC offers five levels of ESOL classes as well as ESOL Literacy for students with low literacy in their native languages. Classes meet either four or five days a week for a total of 10 to15 hours of weekly instruction. All classes are complemented by computer literacy classes. In addition, The ILC provides Citizenship preparation for approximately 65 students per year, and workshops for parents on a wide variety of family literacy topics. The ILC provides support services and education/career advising to students through two full-time counselors.

Rationale and Background of the Practice

The ILC views on-going student assessment and program evaluation as the means to demonstrate that its mission and goals are being reached. Data on student performance and other information direct program and policy decisions. The ILC’s consultative process has led the agency to refine program evaluation and student assessment processes and tools over the years.

On-going program evaluation and student assessment allow the agency to be accountable to all its stakeholders (students, staff, funders and trustees). Furthermore, adult literacy research shows that adults persist in their studies when they can state their learning needs and goals and can see evidence that they are reaching those goals (Comings et al, 2000).

Description of the Practice

Student Assessment: The ILC’s assessment model translates into a three-part cycle. It includes initial intake and placement through an interview process; on-going, in-class articulation of student learning and vocational goals and subsequent refinement of classroom instruction, and a final stage of assessment that identifies students’ learning gains.

1. Initial intake and placement

A counselor meets with each incoming student for a 20-minute interview that determines which waiting list is appropriate for each student. No standardized tests are used during the initial intake. The interview consists of English conversation, reading and writing exercises.

Once the student is called for enrollment, he or she may be reassessed, depending on the length of time spent on the waiting list, to ensure that students are placed in the correct class level. Basic information about education background and work experience is also updated and provided to teachers and counselors for use in goal-setting activities.

2. Classroom assessments of student progress

Goal Sheet: Students complete a goal sheet that includes plans for meeting their personal, family and workplace literacy goals. Goal sheets become part of each student’s permanent record and are reviewed at the beginning and end of each session.

Teachers not only elicit students’ personal goals, but also their class goals: what they want to study, what they hope to learn in the class. Students discuss and share activities that help them in class and out of class with pronunciation, conversation, listening, grammar, reading and writing. A variety of techniques are used to obtain this information: worksheets, problem posing activities, information grids, facilitated discussions, etc.

3. End-of-session assessment and placement

Testing: In addition to ongoing assessment by teachers, students' conversation and writing skills are assessed in order to determine whether promotion to the next level is appropriate. Depending on the class the students attend, conversation skills may be assessed through the Best Plus or through teacher-developed tools. Writing skills may be assessed by means of a standardized written test(REEP) or through teacher-developed tools.
Student-Teacher Conference: Each student meets with his or teacher for an individual conference to discuss progress toward goals and placement for the following session.
Achievement Form: Teachers lead students through a series of questions that provide information about workforce development goals (e.g., “Did you start your own business?”), educational goals (e.g. “Did you take the GED test?"), and civic goals (e.g. Did you take the citizenship exam?”).

Program Evaluation: The results of the student assessment process inform overall program evaluation efforts at The ILC. The ILC identifies ongoing program evaluation and subsequent refinement of its services as one of the keys to successful organizational development and stability. The program evaluation consists of:

End-of-session classroom evaluations that inform teacher practices. This evaluation asks six questions. Following are two examples: “Please tell me what you think about the difficulty of the lessons. Are they too easy? Too difficult?” and, “Are the lessons helpful to you outside of school? Do you ever use what we learn in class after you go home? Can you give any examples?”
Yearly student surveys administered by upper level students and analyzed by staff. The survey instrument was initially developed by a subcommittee of the Board and refined over the years with input from students and teachers. This school-wide survey is six pages long and covers student demographic data, reasons for attending The ILC, areas in which students felt they had improved and questions about instruction. Questions about instruction ask: how well students understand the lessons; how often class materials are useful and interesting; how often the teacher works with small groups or individual students; how often the teacher provides practice time; how often the teacher encourages and answers question; and how has homework been helpful. Questions are primarily multiple choice except for the last two, which ask students what they like about the Center and what they would change. Students complete the survey anonymously.
Follow-up: After students leave the program, counselors attempt to stay in touch with students with letters, email or phone calls to keep track of continuing achievement of goals.

Challenges

Implementing this type of student assessment and program evaluation system presents two challenges:

Program evaluation, systematic student assessment and follow-up create additional staffing and data collection expenses for the agency. For example, a second full-time counselor/advisor was hired. This is addressed through fundraising efforts.
Staff must “buy in” to the need for a student assessment and program evaluation process and trust the effectiveness of the particular assessment and evaluation tools that the agency chooses to use. This is addressed through staff input. Teachers have created some of the assessment tools. The culture of The ILC encourages staff suggestions and initiatives. Monthly staff meetings, scheduled teacher sharing time and the easy accessibility of the director enhance the process.

Evidence of Impact and Effectiveness

“Detailed documentation of diverse outcomes has bolstered the ILC’s fundraising efforts. The ILC draws its funding from a very diverse pool of public and private sources, namely foundations, businesses and individuals.”

~ Diane Portnoy, Co-Founder and Director

The student assessment and program evaluation efforts enable The ILC to monitor and measure its impact and effectiveness in quantitative and qualitative ways. The ILC is able to present to all its stakeholders a myriad of countable outcomes as well as narrative student and program success stories. For example:

In fiscal year 2013, 136 students achieved their employment goals such as gaining employment, moving from part-time to full-time work, being promoted and starting businesses; 43 students entered colleges or training programs; and 28 students became United States citizens.
Hundreds of other goals were also achieved by students in FY 2013, including getting driver's licenses, getting library cards, buying homes, volunteering, attending parent-teacher conferences, and being able to see doctors without the need for interpreters.

This detailed documentation of diverse outcomes has bolstered The ILC’s fundraising efforts. Statistics gathered by teachers, counselors and from student surveys are entered into a database. Reports can be generated for combined or specific school programs and for the full or partial fiscal year. These reports accompany grant proposals to donors and ILC information packets to the media and other interested parties. They also are shared with The ILC Board of Trustees and staff.

Implications for Practice, Policy and Research

Nationally and locally, there is much debate about best ways to assess student progress. While the search for the perfect assessment tools and processes continues, publicly funded programs must find feasible and adequate ways of assessing their impact and being accountable to their students and funders. The ILC’s integrated and ongoing program evaluation system, which includes a multi-pronged student assessment process, can serve as a model for other adult literacy programs. Programs contemplating the adoption of a similar system would do well to reflect on what those tools do and do not capture, and whether there are other outcomes that are important to the program and the students.

References

Comings, J., Parrella, A. &Soricone,.S. (2002). Helping adults persist: Four supports in Focus on Basics, Volume 4, Issue A. Boston, MA: National ILC for the Study of Adult Learning and Literacy.
http://www.ncsall.net/index.php?id=332