Evaluation of Course

Evaluation of Course by Students

At the end of each semester, UAF asks its students to evaluate each course.  The evaluation process consists of a computer-scored form that the students use to rate the course semi-quantitatively, and a comment sheet where the student may write additional, text comments.  Both forms and the evaluation process are anonymous.  After the comment sheets are collected and the computer forms collated, a "Student Evaluation of Instruction" which summarizes the IAS form and the Comment Sheets are routed through the Dean's office, the Department Chair and then to the Instructor.   For computer scored evaluation, UAF uses the Instructional Assessment System (IAS) developed by the University of Washington.

The commonest scoring system in the IAS is

The percentage of students scoring each rating is reported and from that a median score reported.   For example, if 50% of the students rated the item "Very Good," and 50% of the students rated the item "Excellent," that item would be reported as 4.50.  There is often a slight difference between the average and the median for grouped data; IAS reports medians.  It is difficult and probably inappropriate to use statistics to compare the students' responses between different courses.  For example, when I taught EQE 697 in Spring 1999 there were 7 students registered, but only 4 of them completed the assessment.  For comparison purposes below are displayed the evaluations from two other courses, 3 classes of EQE 649 and one class of ESM 450.  The demographics, engineering vs. scientist and full-time vs. professional student, are about the same in EQE 649 and EQE 697.  For  ESM 450 the demographics are quite different; all the students are undergraduate engineers, usually in their last semester before graduation.  I teach ESM 450 each semester and the data are reported from the last time I taught it, the 7th time. Presumably "the bugs are all worked out" of ESM 450.

The first four items of the IAS are very general and their average is often used to gauge the students' overall opinion of the course. (The entire IAS is available.)

  * per IAS report, glitch noted.

Without attempting statistics with this, it appears that the combined score of the first four items is higher than when EQE 693 was taught in the classroom in Spring 1999, and higher than the average of the three times that EQE 649 was taught, and higher than ESM 450, the course with which the instructor has the most experience.

Items 5 through 22 of the IAS ask the student in more detail about the course.    Here are summarized only the items that differed more than 0.2 between the average of the EQE 649 and the EQE 693, or if EQE 693 was lower than 4.0.

There were five questions to which the students' answers differed by more than 0.2 points or that were lower than 4.0, the divide between Very Good and Excellent.  I will attempt to discuss these and avoid the temptation to defend the score.  Since almost all the scores were between very good and excellent, the students felt they received adequate value.  The point here is that if there are substantial differences between the web-based course and the classroom-based courses, differences in the student evaluation of these details might provide insight.

Question 6, regards the course organization, and the score is 3.68,.  Here this course is the same as the EQE 649 average, and higher than the EQE 693 from Spring 1999, the first time it was taught.  All contrast poorly with the high mark in ESM 450, a highly organized class, with a rigid syllabus.  I interpret this to mean that the all-electronic format did not improve the students' perception of overall organization.  This may have been due to several changes in the scheduling of modules and homework as the semester progressed.

Question 10 regards the questions or problems raised by the instructor.  Here this course is essentially a 4.0 and close to the EQE 649 average.  This could mean the questions were not difficult enough or that the students disagreed with the grading of certain questions.

Question 12 regards the instructor's enthusiasm.  This was slightly less than the EQE 649 average.  The written portion of my student evaluations sometimes cites my enthusiasm in class.  Presumably this refers to personal manner in class.  This would not come across in web-bases pedagogy.

Question 17 regards the instructor's interest in whether or the students learned.  This is 4.0 is the dividing between very good and excellent, so it is not a bad mark, but is troubling because it is lower than both the EQE 649 average and considerably less than the 450.  I don't have a good explanation for this.  It bears some research and watching in the future, if there is some reason the distant students do not perceive the instructor is interesting in whether or not they learn.

Question 19 regards the relevance and usefulness of the course content.  Here the grade is satisfactory, but 0.3 lower than the EQE 649 average.  It is the same as the ESM 450 grade. 

Question 22 regards the clarity of the students' responsibilities.  Here the grade is higher than that the others.  This was probably due to the necessity of organizing the homework at the same time I was preparing the modules.

Questions 23 through 27 ask the student to compare this course to other college course they have taken.  Here the rating was a little different.  IAS uses a 7-point scale for these comparisons. 1 means much lower, 4 means average, and 7 means much higher.

All the items are above 4.0, average.  It would appear that the students expect to get lower grades (Question 23), perceived the intellectual challenge as less (Question 24), but put more effort into the course (Question 26) relative to other UAF courses.  Note the paradoxical response that amount of effort put into the course was greater, but amount of effort to succeed was less than average.  The involvement of the students was greater.  Since the students have a varied background and these questions specifically ask them to compare with other college courses, interpretation is more difficult.  If you look at the mode of the responses, the results appear more consistent, the grade, intellectual challenge, and effort to succeeds being average compared to other college courses, the amount of effort put in and the involvement being higher than average.  I do have a speculative explanation for the difference between the amount of effort put into the course being greater and the amount to succeed being less.  The course was set up with most of the credit obtained from the homework, and the homework required a lot of work, but if the students did the homework, they received a passing grade.  Also some of the discussion work students received a passing grade for contributions, but were not graded if their answer was right or wrong.  Hence a diligent student might have perceived that they could pass with less effort then they were making.

Questions 28 and 29 regard the amount of time the students spent on the course.  Question 28 asks about the total time, while Question 29 asks how much of the time was useful.  Their quantitative responses correspond to the qualitative answers to 25 and 27 - more time was required.  Also, relative to other courses, they did not feel a lot of time was wasted.  The quantitative result below compares with the 3.6, 3.9, 4.33 hours per week for Modules 2,3, and 4.  One explanation for this is that the students were working on their papers more toward the end of the semester, at the time the survey was done.  Both the classroom based EQE 693 and EQE 649 have a paper due at the end of the semester and would demonstrate this effect.

TOC Resources