How often do students . . .

This week we are looking at some of the results of the Community College Survey of Student Engagement (CCSSE) and the Community College Faculty Survey of Student Engagement (CCFSSE). You can find the CCSSE report and the full results on the IR intranet page.

The first comparison has to do with how often students come to class unprepared.

 

table 4

The second question concerns how often students skip class

table 1
The final example concerns class participationtable 2

The point of surveying both the faculty and the students on similar questions is to give insight into how those two groups view similar issues. I am interested to know how people interpret the differences in responses.

My own first response is methodological and was initially pointed out to me by my colleague Bonnie January.

While the questions address similar issues, faculty and students are not being asked precisely the same question. Faculty are asked about students collectively, while students are asked about themselves individually. The questions being asked of faculty are a bit unclear. Does a “very often” mean that most students in most classes ask questions, skip, or come unprepared or that some student does those things most classes or something else altogether? It isn’t exactly clear, and some of these results may differ for that reason.

If we wanted them to be the same, we would have to ask faculty a series of questions: What percentage of students asks questions or contributes to class discussions very often? What percentage of students asks questions or contributes to class discussions often? What percentage of students asks questions or contributes to class discussion sometimes? And so on. The problem here is that a survey that is already longish would become four times as long.

My own view, however, is that there is more to these differences than question wording. How would you interpret these differences? Please share your thoughts by commenting on this post.

 

10 responses to “How often do students . . .

  1. Kristine

    Wow, that last set of numbers is a pretty big mismatch. I thought the first two sets agreed ok. Makes me wonder… Perhaps instructors are hearing from the same 1-3 people and interpreting that as an inclusive discussion when it really isn’t…

  2. Anne

    Alec,
    I wonder if the results for skipping class (students think they never do, faculty think they often do) is actually about the person’s definition of what “skipping” class means. Faculty might define skipping as equivalent to “missing” or “absent for any reason” while students might define skipping as “not attending class for a reason of little value”. Students may perceive the word skipping with negative connotations and think “well, I’ve never skipped a class”, yet they have missed class due to valuing something other than attendance more highly, i.e. work, family, etc. In other words, to a student, missing class because your little brother was ill and your mom couldn’t stay home so you had to, wouldn’t qualify as skipping.

    • Anne,

      I think that you are probably correct. I think it would be interesting to dig into these differences and as professors and students what absences are acceptable and what kinds are not. I would expect agreement. Hospitalization would be acceptable, staying home to play video games would not be acceptable. In the middle I would expect more disagreement. What kinds of illnesses lead to acceptable absences? What about work or child care? If we had that data I think we could learn a lot about our students and vice versa. It wouldn’t be too hard to study this but it would require a survey of students and faculty dedicated to this single issue.

  3. It’s obvious what’s going on with these survey data. Although Ms. January is right that asking about group behavior is different from asking about individual actions, the disparity between students’ and faculty’s perceptions is due to the filter we all apply when we evaluate our own behavior.

    Ask me if I ever exceed the speed limit on the freeway, and my “Occasionally” response actually translates to “Pretty much every trip I take.” Likewise, ask a teacher about students posing questions and participating in discussions, and an overly positive response is understandable: If a majority of students didn’t participate, it would imply that the teacher’s not all that great.

    Any survey that asks people to assess potentially negative behaviors will invariably result in somewhat skewed data. Even in the privacy of the confessional, we tend to polish our failures and smooth the rough edges off of the selective transgressions we acknowledge committing.

    Uh, or so I’ve heard.

    • Dan,

      It is certainly true that people tend to overstate their own virtuousness when responding to survey questions. People report voting and recycling much more often than they engage in those activities. So, students probably do under report skipping classes. I still think Anne is on to something with the question of definitions though.

  4. Andrew Vanture

    Alec,
    How much of the difference in the responses do you think is a selection bias? So the students that are likely to respond to the survey are the ones that show up, ask and answer questions. I know you touched on this in your comment about collective vs. individual experiences, but I guess I am wondering how drastically the varying response rate of students vs. faculty will have.

    • Andrew,

      This is a complicated question with a lot of considerations and a lot of possible answers. I should start by saying that if I knew for sure ( or was even pretty sure) how selection bias was operating then I could think about statistical corrections. That is a fancy way of saying that what follows is an educated guess. Here are a few considerations in the evolution of that guess.

      1) selection bias operates on both surveys since the faculty response rate was well below 100%.

      2)On the other hand, your proposed selection mechanism (asking and answering questions) is plausibly related to motivation, engagement and thereby to the skipping of classes. The same mechanism won’t apply to the accurate observation of skipping by faculty so selection bias might matter more for students

      3) The sample was random (classes weighted by enrollments were selected) in selection but not in application since not all selected classes chose to participate (that is a selection bias produced by faculty decisions to participate or not). Thus, like most social science samples it wasn’t really random.

      4) within selected classes the response rate was high. Nonetheless, students who skipped class on the day of the survey are unrepresented and it is plausible to assume that students who skip class on any given day are more likely to skip on other days than those who were in class on that day. Thus class skippers are almost certainly underrepresented

      5) on individual surveys the item response rate was high.

      Conclusion 1: The student reports of skipping are lower than the true rate because of the absence of skippers on the day of the survey. (I’m confident in this one)

      Conclusion 2: #4 and #5 lead me to believe that there is limited selection bias with regard to taking the survey or answering specific questions on the survey once it was offered to students. The primary bias is in classes participating not individuals participating. I think this will limit the impact of the motivation mechanism that you are suggesting. (I’m less confident in this one)

      Conclusion 3: The difference between faculty observation and student self report is larger than I would expect to occur just from conclusion 1 so my guess is that selection bias does not account for all of the observed differences. (I’m least confident of this one)

  5. Kristen McConaha

    Interesting how some are similar and others are quite varied showing the gap between faculty and student perceptions. Thanks for the specific share out.

  6. KenMar

    Given the questions asked faculty about students as a whole while the students were asked about themselves…could it be that only the more studious, attendant students take the time to complete the survey?