How do you rate your University experience? We’re at the half way point in the University Semester, a time when academics take stock and plan for their final classes for the year. We explore whether they could also plan to improve their teaching scores.
At the end of each Semester students are asked to rate their subjects with a Course Experience Questionnaire (CEQ) and following completion of their degree take a further Graduate Destinations Survey (GDS) – both surveys look at the teaching quality of academics and are an important factor in universities determining their policies around academic performance and teaching output.
Econometrics, or the application of statistical methods to economic data, is increasingly important to organisations for its ability to translate qualitative economic statements into quantitative ones. Joe Hirschberg and Jenny Lye, used Econometric techniques to unpack the data of 10,433 students and look at the effects on Academic Departments and also the impact of CEQ scores on GDS results.
Top marks for good teaching
A major finding of Hirschberg and Lye’s study was that a student’s average score for the question ‘Was the class well taught?’ greatly influenced their scores for the other questions in CEQ. This outcome was only apparent when also testing against the departments in which students took most of their classes. This confirms widely reported observation that average teaching quality instrument scores are strongly influenced by the department in which a class is taught, indicating that students who take classes from a particular department form expectations about the delivery of those classes.
Lesson: Getting academic staff on board with your distinct Department identity is important in setting a clear impression for students.
Is it worth going the extra mile?
Hirschberg and Lye also discovered that students who rated the quality of teaching higher than the average didn’t actually score the subject higher overall in the subjects CEQ results. By contrast, students who had a less than average experience regarding the teaching quality, reacted with a significantly low CEQ score for the subject.
Lesson: Maintaining a consistent standard of teaching, which avoids major negative dips, should result in better overall teaching performance than aiming for high scores from a small number of students.
Part-time vs. Full-time
By matching responses from both the GDS and CEQ, Hirschberg and Lye were able to look at whether the responses by students were affected by their part-time or full-time student status and if their status had any implications for further study. Their findings contrasted with previous research in this field by revealing that students who studied full time in their final year and continued studying for a higher degree were more positive regarding teaching standards than their part-time equivalents who did not go onto further study.
Lesson: Part-time students, often have significant commitments outside of study, such as work or family, so maintaining strong teacher-student engagement vital and different methods to the full-time student cohort may need to be employed.
By using Econometric techniques for analysis Lye and Hirschberg were able to build up a robust picture of these student’s views and experiences. They were also able to construct a complete set of academic results for each respondent as well as the full distribution of all grades in the classes they took, creating a comprehensive data model. Teaching and research are at the heart of what universities and academics do, but ensuring they do the former well is something that can constantly be refined and improved.
Melbourne Business School is launching a new Master of Applied Econometrics in 2017. The degree will allow students to acquire the knowledge and skills required for data modelling for a future career in policy analysis within government, large corporate organisations, or academia. Associate Professor Jenny Lye is the Program Director for the degree.
This article is based on the findings by Hirschberg and Lye, the University of Melbourne, presented in “The influence of student experiences on post-graduation surveys”, published in Assessment & Evaluation in Higher Education (2016).
Jenny Lye, is an Associate Professor and Reader (Economics) in the Faculty of Business and Economics.- Joe Hirschberg, is an Associate Professor (Economics) in the Faculty of Business and Economics.