Assessment of Student Learning Outcomes
A quality educational institution should be committed to assessing the outcomes of student learning and using the results of that assessment to improve the educational experiences of its students.
A plan to assess student learning should be rooted in the College's mission and its core values- specifically, integrity, academic excellence and commitment to teaching and learning. The plan should reflect the recommendations of our Institutional Self-Study (2004) on Standard 7 - Institutional Effectiveness and Standard 14 - Assessment of Student Learning and meet the Fundamental Elements defined in the Characteristics of Excellence by the Middle States Commission on Higher Education. These Elements include:
To address this assessment goal from a department level, each academic department was asked to develop a measurable assessment question that would yield some data on student learning outcomes related to its curriculum. The questions developed, will be explored in the Spring 2007 semester.
I appreciate the work department faculty did to further this Academic Master Plan goal.
Academic Master Plan 2006-2009
Judith Gay, Vice President for Academic Affair
Division of Mathematics, Science, and Health Careers
Allied Health
The Department of Allied Health will utilize a newly developed Course Review by
Students evaluation tool. The instrument was developed to measure the effectiveness of a
course in meeting stated outcomes and was administered to students during the Spring
2006 semester. The instrument will be used again in the Spring 2007 semester. Questions that
will be addressed in relation to the Course Review by Students are:
Biology
The Biology Department will examine the impact of BIOL 106: General Biology on achieving a successful outcome for BIOL 109: Anatomy and Physiology I. In particular the department will measure the retention rate for students who had taken BIOL 106 prior to BIOL 109 in comparison to those who did not.
Chemistry
Beginning with the Fall 2005 semester, the Chemistry Department has given all CHEM 101, CHEM 110 and CHEM 121 students a departmentally developed "pre-course" diagnostic math questionnaire during the student's first day in the laboratory. This math score is not included in the calculation of a student's semester grade average. However, a student's score is shared with the student to indicate that the basic math skills required for success in chemistry courses are either present or lacking. In Fall 2006, students were given the opportunity to attend specially arranged math workshops based upon the results of this test and offered by the Math Learning Lab faculty during the second and third weeks of the semester. The Chemistry Department administered the test during the Fall 2006 semester and initiated a post-test the week before 11/20/06. The following will be explored:
Mathematics
The Mathematics Department, recognizing the need to address concerns regarding student performance in developmental mathematics courses convened a subcommittee (CEMEC: Committee on Elementary Mathematics and its Effect on Curriculum) and developed a pilot plan to address these concerns. The plan was the result of intensive review of the literature, examination of the current curriculum and the review of developmental mathematics at twelve community colleges which were selected by the League for Innovation of Community Colleges as vanguard colleges in the nation. Faculty worked during the Spring 2006, Summer 2006 semesters to prepare materials to implement the pilot program during the Spring 2007 and Fall 2007 semesters. The Mathematics Department proposes to assess how the new approach and new materials utilized in MATH 016 affect student outcomes. Student outcomes to be considered:
Nursing
The Nursing Department has converted the lecture, seminar, and lab sections of the nursing courses to a Hybrid, distance learning format. The Department proposes to assess the following in relation to this new format:
Measurement strategy - compare statistical data of specific test questions from previous years, compare overall grades on tests, and compare overall grades in the course.
Measurement strategy - survey students for amount of time spent online.
Measurement strategy - survey students regarding time usage.
Measurement strategy - Survey students and faculty about time and amount of face to face encounters. - PASS sessions, faculty/student discussions, one to one office time, time on email, time in forums, etc.
Physics
The Physics Department has been utilizing an online homework system in certain sections of physics courses. The system has been used in PHYS 140 during the Spring 2006 semester and in PHYS 111 and PHYS 241 during the Fall 2006 semester. The department proposes to assess the overall effectiveness and convenience of using an online homework grading system.
Division of Liberal Studies
Art
Are the objectives in ART 105 - Drawing I consistent from teacher to teacher and are they clearly stated in each teacher's syllabus? Rationale
ART 105 is arguably the most important class in the art curriculum. It is required of all art majors and it is the class most often taken by non-majors. More sections of ART 105 are taught in a semester than any other studio art course.
Art majors are required to take four semesters of drawing for the Art and Design curriculum. It is essential that every student gets the same foundation in ART 105 and is prepared for the following three semesters of drawing.
In any given semester there are more teachers teaching ART 105 than any other studio art course. This group of teachers is comprised of full time and part time faculty. It is important that the faculty teaching ART 105 have the same objectives in their classes.
Are these objectives shared by all faculty teaching ART 105? Classes evolve and assignments vary from teacher to teacher, however the objectives must be shared.
Evaluation
To answer our department's assessment question we will do two things. The first step will be to have all drawing faculty meet and clearly define the exit criteria/objectives for ART 105. The second step will be to review all ART 105 syllabi to ensure that the class objectives are consistent and clearly stated.
Behavioral Science
There is considerable debate in the department as to whether or not we should be allowing ENGL 098 Regular students in our PSYC 101 classes. These students have college-level reading test scores but their writing is weak. We plan to look at the pass rates and grades of 098 regular students and see how they compare to ENGL 101 ready students.
BHHS faculty are interested in determining the effectiveness of the new practicum courses they developed over the last few years. They wish to assess whether the practica are effective in showing/demonstrating to BHHS students how BHHS theory is applied in real work settings.
All students who major in Education will eventually need to take the Praxis exams. The education faculty plan to assess whether the praxis workshops were helpful to students in preparing for the test. Specifically, we will ask our participants to discuss which topics were most helpful and for which topics we should have devoted more time.
Knowledge of English vocabulary, grammar, and semantics is an important aspect of learning a second language, and is especially vital for developing more advanced skills in interpretation. While advanced vocabulary skills may not be significant for the 100 level ASL classes, they are vital for success in the 200 level and higher ASL classes. However, there is little to no room in the INT curriculum for teaching vocabulary and general English skills, so we rely on previously learned and developed English ability which becomes increasingly important as students move through the curriculum.
The purpose of this project will be to assess students' English vocabulary as a predictor of success in both 100 and 200 level ASL coursework.
At the beginning of the Spring 2007 semester we will collect scores on English entrance exams, and/or other English coursework up to ENGL 101. In addition, at the beginning of the semester each student will take a brief vocabulary test to assess knowledge of words that might commonly be found in professional discourse. Some examples could be the following: disparity, impediment, precocious, reiterate, fastidious, feckless, consolation, mortify, aversion.
At the end of the semester 1) English/test scores will be correlated with final grades, and 2) we will compare the students who completed the course with those who dropped out based on the vocabulary test and previous English coursework grades.
ECE faculty have been working to discuss writing across the curriculum and to develop new class writing assignments. The faculty would like to assess the impact of these workshops. Faculty plan to assess how faculty writing assignments have changed as a result of these workshops and how effective they have been in improving student writing.
The leadership curriculum and courses are being revised. We want to assess whether the greater flexibility in coursework counted toward the degree increase enrollments in the program.
Women's Studies faculty have been developing online sections of courses in response to student interest in distance education. Student response to date indicates that the certificate program will be more attractive to students if we increase online options.
The Women's Studies faculty would like to assess the effectiveness of the online sections vs. traditional classroom instruction sections.
Assessment question :
WS faculty will compare the grades and withdrawal rates in traditional vs. on-line sections of the Introduction to Women's Studies courses in order to determine the effectiveness of the new on-line courses.
Survey students to determine why they are drawn to the field of youth work. What do they view as the role of a youth worker? What skills and knowledge do they see as necessary for successful work with young people? Where do they envision their career path going in the future?
Design Technologies
Currently the department offers the first two semesters of design (ARCH 109 and ARCH 111) in both day and evening sections. There is increasing interest in providing alternative delivery of courses, particularly as the department sees some loss of students to Drexel's evening architecture program. The department will produce and review questionnaires to understand students' expectations for the courses, reasons for preferring day or evening sections, interest in taking evening sections of second-year design courses, and issues such as work and family commitments that may affect selection of courses. The department will also look for "self-selection": for example are the students in evening sections older and potentially more self-directed?
The focus will be on second semester design studio, ARCH 111, Digital Design and Delineation with data from 2005-06 and 2006-07 academic years, four sections total. Reason: same instructor over two years for both day and evening, eliminating one variable. Also this course has had relatively consistent student retention to future semesters.
The data needed to undertake this study will include class lists from 2005-06 and 2006-07 for ARCH 111 and datasheets for these students to understand subsequent performance. This includes a total of four sections: 2 day and 2 evening, approximately 60 students. Data from surveys of students will be conducted at the end of the Fall 2006 and Spring 2007 semesters.
This question will inform our understanding as to how we can better serve our students and enhance their performance, and it will provide guidance to the department regarding future course offerings.
English
College Writing How do students who have passed English 101 before enrolling in Psychology 101 and Sociology 101 perform in those courses compared to students who enroll in the same courses while they are taking unlinked English 098?
Developmental English
What percent of former English 098 students pass English 102 the first time?
Literature/Creative Writing
What are the proportions of students in English 205 (Creative Writing) who have taken no literature before English 205, those who are taking literature concurrently with English 205, and those who have literature prior to taking English 205? Are there differences in grades in English 205 by these cohorts?
English-as-a-Second-Language
Of the first time student cohorts identified by Institutional Research from Fall 2001, Fall 2002 and Fall 2003 who failed to persist at Community College of Philadelphia, what are the current findings in terms of age, race, gender, and specifically, their earned credits at point of departure. Did these departing students subsequently enroll in/transfer to a four-year institution?
Speech and Theater
Does the cohort of students who takes English 142 fare better in terms of GPA, transfer rate, and/ or graduation rate than other students within the theatre option/ and or other students in com arts or liberal studies?
Foreign Languages
Several years ago, the Foreign Language Department lifted the "ENGL 101 ready" prerequisite for students taking our courses at the 101 level. The idea behind this change was that in a first semester language class, since basic grammar concepts are covered, students' skills in developmental English classes could be enhanced. Our suspicion was that after taking a first level language class, developmental English students would be better prepared. We hoped that the two courses taken in conjunction would supplement each other. Anecdotally, at least, this seems to be the case. We don't seem to see any measurable difference between developmental students taking language courses and their English 101 ready peers. We would like to test these hunches with hard data. Using data from the Spring 2006 and Fall 2006 semesters, we will compare:
History and Philosophy
Student evaluation in both history and philosophy is heavily dependent on writing in-class essays, take home essays and research essays. The department would like to know whether training in composition writing enhances student success in history and philosophy, whether it is incoming writing ability that determines student success in history and philosophy, or whether success in history and philosophy is independent of writing abilities. Further, we would like to know whether student success in history is correlated with student success in philosophy, i.e., are there writing or intellectual abilities that are discipline specific or are these generic abilities. In order to measure the relationship between writing abilities and success in history and philosophy classes, the Department will compare grades in history and philosophy courses with scores on in-coming writing and reading examinations and with grades in ENGL 101 and ENGL 102. History and philosophy courses already require students to be ENGL 101 ready. This research should help the Department to determine whether that requirement makes sense, whether we should require ENGL 101 as a prerequisite, or whether, if writing abilities are discipline specific, faculty members should be spending more time in teaching discipline specific writing in class. (There are currently a number of discipline specific writing texts that might be used for this purpose.)
Music
Does the curriculum in MUS - 100 Music Reading adequately prepare students for success in Harmony I? Can the Grades in this course help to predict future success in all levels of advanced harmony? General Hypothesis If the Results are True: If the Results are False:
Photographic Imaging
When the course PHOT 151 - Digital Imaging was created in late 1995, the introductory course PHOT 101 - Basic Photography, was identified as its prerequisite. In the meantime, the prerequisite has often been waived for students with a variety of alternate experiences. Is there a substantial difference in the outcomes of students who were permitted to take PHOT 151 with a prerequisite waiver, and those who preceded their PHOT 151 enrollment with successful completion of the prerequisite course PHOT 101?
Social Science
Each discipline and program developed its own assessment questions. Even so, interestingly, discipline groups working independently came up with similar assessment concerns in several areas: a determination of the impact of successful completion of various English courses on success in discipline courses. The goals of all of those assessments are the evaluation of current prerequisites for the discipline courses. Of course, with each of these assessments, the actual sampling size needs to be determined as well as ways of ensuring consistency of evaluation. No question was submitted for political science. Additionally, the disciplines of GIS and Computer Forensics are so new that assessment is premature. Fire Science will be working on its assessment question early next semester. Anthropology
Faculty members in anthropology are concerned with determining the appropriate prerequisites for two of its introductory sections.
What are the success rates of students in ANTH 101 and 112 who come in under the following conditions?
Geography / Earth Science
Faculty in the disciplines of geography and earth science are concerned with evaluating whether the current prerequisites are the appropriate ones for its offerings. Accordingly, this assessment strives to determine the difference in outcomes among those who have successfully completed English 101 (or tested at English 102 level), those who are taking English 101 concurrently and those who have not successfully completed English 101.
A comparison in outcomes (based on final grades)of these three groups of students will be compiled. The results will inform faculty discussion of appropriate prerequisites and a determination of whether any alteration in prerequisites is needed at this time.
Sociology
Sociology faculty are concerned with prerequisites for advanced sociology classes. Specifically, the question the sociologists wish to examine is whether successful completion of English 101 should be a prerequisite for advanced Sociology classes. Do students who have passed English have higher grades in advanced sociology courses than those students who have not passed English 101?
Additionally, the faculty are concerned with grades in English courses as a predictor for success in higher level sociology sections and would like to also consider the following: Do students who have completed English 101 with grades of "C" or better perform better than students who have not completed English 101 with grades of "C" or better?
Justice Curriculum
The overall thrust of this project for the Justice Curriculum is consistency. The Justice program offers courses at a variety of times, at the main campus, Northwest Regional Center, Northeast Regional Center, the Philadelphia Police Academy, the Philadelphia Prison Academy, and in online and traditional formats. We need to study whether the student outcomes vary by location, time of day of class, and/or course format. If there is a marked difference in these outcomes, what adjustments need to be made to ensure that all justice students, regardless of the time, location or method of delivery, receive comparable instruction producing comparable student outcomes? Of course this involves several assessments, and it is anticipated that in the Spring semester, one part of this evaluation will be undertaken.
To begin this project justice faculty will consider outcomes (grades and retention) in an on-line section of a justice course (to be determined) as compared with a live section of the same course. To further ensure that a proper comparison is being made, sections taught by the same instructor will be considered.
Paralegal Studies
Paralegal Studies is concerned with consistency with respect to outcomes in its traditional offerings as compared to its distance offerings. (This comparison is also mandated by its accreditation agency, the American Bar Association). Accordingly outcomes (grades and retention) in sections of traditional courses will be compared with outcomes in sections of distance courses.
Division of Business and Technology
CIS 103
The primary goal of CIS 103 is to provide the student with skills in Microsoft Office applications consistent with the International Computer Driving License (ICDL) syllabus. The ICDL is the worldwide standard of basic computer competence. Student success in CIS 103 is measured by the student's final grade, and CIS faculty want to ensure consistency in measuring student achievement regardless of the instructor. The faculty will evaluate students' grades for Fall 2006 CIS 103 sections, compare the results with the Fall 2005 and Spring 2006 CIS 103 grades when Alice programming was taught and make recommendations for CIS 103 based on the findings. The team supporting the Alice project has already analyzed the final grade distribution for seven instructors who were teaching the most sections of CIS 103 during the Fall 2005 and Spring 2006 semesters.
Economics and Accounting
At the end of the academic year each faculty member completes the course evaluation questionnaire used for Act 335 documentation for every course he/she teaches and submits the evaluations to the Department Head who reviews them. If faculty have expressed any concerns with courses and/or students' learning in the courses, the Department meetings are used to evaluate the problems and resolve the issues. Within each discipline (Accounting, Economics, Finance) general student progress and performance in each course is discussed. If a 100 level course has a sequential course, e.g. ACCT 101/ACCT 102, then the student performance in the sequential course is evaluated as well. Final grades are used as an indicator of student success.
Transportation Technologies Management
The Automotive Technology program is using many assessment tools to review different aspects of the program. The Advisory committee meets biannually and one of the agenda items is to discuss whether the program is preparing our graduates for successful transitions into the workforce. At our Fall 2006 meeting the consensus was that our students are technically competent but are missing some soft skills. The Automotive faculty is reviewing how we can address this issue in our courses. We will evaluate the Automotive courses and will make recommendations to add communication and interpersonal skills to identified courses.
The Automotive Technology (AT) program also conducts a self evaluation every 2½ years and sends this documentation to our accrediting agency, the National Automotive Technicians Education Foundation (NATEF). We are also reviewed by NATEF every 5 years (next evaluation 2008). This evaluation makes sure the technical competencies are up to date and the curriculum is current. It also reviews how AT conducts our technical competencies evaluations.
Automotive students in each Automotive course are evaluated by their instructor on their ability to complete specific technical tasks. The students must complete the tasks within a specific time period, are given credit once the tasks are completed and the faculty member records the successful completion. These records are stored in a student database for all NATEF required tasks. The Advisory committee members also review these completed tasks and assist the faculty in adding tasks to the competency list if necessary.
The faculty meet biweekly to discuss any issues with AT courses, curriculum or students. This enables the faculty to quickly identify students who are having difficulties handling the workload. The discussion also allows the faculty to discuss how we can help the student, provide tutoring in any automotive area if necessary plus address any academic issues that may be affecting the student.
Each AT course also assesses students using conventional testing methods, hands-on evaluations and computer based manufacturer testing (CBT). The CBT format ensures the student is assessed at the same level as a manufacturer tests their technicians. Although not always the best assessment method, it gives the student a sample of how they will be evaluated on the job. It also allows the student to earn valuable technical certifications for various manufacturers. The conventional style resembles what the student will see at an ASE (Automotive Service Excellence) certification test.
In all, the student outcomes for the Automotive program are extremely important to the success of our program and our students.
Library and Learning Resources
Library Services
Are there significant differences in the grades of Spring 2007 ENGL 102 students who participated in ENGL 101 library instruction (LI) in Fall 2006 and those who did not? (Special programs are not included.)
Library faculty will gather the names of Fall 2006 ENGL 101 students (no special programs). These students will be divided into two groups: those whose section participated in an LI session and those who did not.
Of these two groups, comparisons of Spring 2007 ENGL 102 grades will be made to determine if participation in LI in ENGL 101 improved grades in ENGL 102.
Division of Educational Support Services
Learning Labs
Students who receive a high grade in MUS100 should also receive higher grades in the advanced written theory classes.
If MUS 100 prepares students adequately, we should see a direct correlation in their grades between Music Reading and Harmony I.
If the results do not show a correlation, then the Music Department must look at both the grading system and the curriculum.
