Depending on where the information is gathered and for what purpose it will be used, assessment of student learning processes at the University of Puerto Rico Rio Piedras Campus can be classified in one of three levels: at the College of General Education level, at the academic program (undergraduate or graduate) level, or at the institutional level. The latter aims at assessing students across the institution at a previously determined time focusing, usually, in one aspect of interest. However, its purpose depends mostly on its participants. If sophomore, junior or senior students are assessed it can be argue that its objective will be to determine how well are they achieving (or have achieved) in certain institutional learning outcomes. But, if they are freshmen students, more than often, its intention will be to establish a baseline of their knowledge, skills or disposition with which future comparisons can be made, specially if they are assessed before starting their academic year.
There are two additional benefit of assessing students just before their higher education experience starts. First, findings can help professor pinpoint criteria at the start of the semester in which students need more help in order to take immediate action in their courses. Second, if the information can be provided to students in a timely manner, they can be encouraged to actively participate in educational experiences such as courses or workshops that are offered constantly that foster the development of skills, abilities and knowledge that can have a positive impact on their college experience as well as in their future professional careers.
After four months of organization, on August 16th 2012, days before starting the new academic year, the OEAE personnel and a group of volunteers made up of assessment coordinators, professors and students, worked together to administered a written communication test to freshmen students, in which the students were required to write a short essay based on a specific topic. Their essays will be evaluated using a four point rubric by a group of professors from the College of General Studies and the College of Humanities that have worked previously assessing the same kind of essays on the Spanish Advance Placement Test of the College Board. Its purpose was to determine a reasonably accurate profile of their written communication skills in Spanish, in terms of their strengths and areas of need.
We are please to report that almost 1,700 freshmen students (approximately 80% of the cohort) took the exam. We plan to send the students their assessment results by the end of September with a list of available workshops and courses that they can take to strengthen their skills. By polishing their writing communication skills at an early stage in their higher education, students are more likely to acquire a set of skills that will provide for a better college experience and for a more complete preparation for their professional careers. We will also provide the results to the Spanish Department of the College of General Studies in order to for them to take the corresponding actions at the curricular level. Finally, the information gathered as a result of this initiative will be used for continuous institutional research to improve student services.
This institutional effort was grounded on a common set of educational values that provided for an cost-effective initiative using in-kind resources. Findings will be useful for a handful of stakeholders (e.g., students, professors, department chairs, administrators, etc.) and will be employed to improve curricular activities as well as student services throughout the institution. Faculty led data driven decisions will be made continually to provide for an organized, systematized, and sustained assessment of student learning process at the University of Puerto Rico Rio Piedras Campus.
We will update this post as milestones are achieve in this project.
Phase I: Evaluation, analysis and results - Completed on 19-12-2012.
Update (09-19-2012): Essays were submitted for evaluation. Five college professors will participate in this process. Each essay will be assessed blindly by two professors (they will not know each others' assessment). An inter-rater reliability measure will be calculated to assess how consistent were the evaluations.
Update (10-10-2012): First round of evaluations is complete. The second round of evaluation will be completed by October 29th.
Update (11-20-2012): Second round of evaluations is complete. Data entry and analysis will begin next week.
Update (12-03-2012): Data entry and analysis is complete. Reliability analysis (using an inter-rater correlation coefficient, with an absolute agreement parameter) shows a high consistency (.70+) between evaluators, and thus their ratings. Results will be sent to different academic units and students.
Update (12-19-2012): Results were sent to students via their institutional emails. The communication included: 1) individual results, 2) an holistic interpretation of five performance levels and recommendations, 3) general results for comparison purposes, and 4) a list of recommended courses available in our institution geared to strengthen written communication in Spanish as well as English.
Phase II: Reports and second measure - to be completed in the second semester
On December 17th, 2012 a meeting was held between the OEAE personnel, Dr. Marta Charrón, Assistant Dean of Academic Affairs, and the Spanish Department Chair of the College of General to go over the results of the project and plan future actions. Additional meetings will be held during the second semester with key personnel to implement the second phase of the project.
