Tuesday, March 13, 2007

An interesting comment about Student Evalutations.

At last, someone wrote an interesting comment with some real details. The strange thing was the email blogger sends me about it was caught up in a bunch of BoundlessGallery.com orders, so I didn't see it until later. Here is the text -

Anonymous said…

In my college, the typical role allocation is 40% teaching, 40% research, and 20% service.

On average, student evaluations make up approximately 20% to 30% of the faculty member's overall annual evaluation.

In my view this weighting is a little high, but current students should have some say.

Having said that, I would prefer a different evaluation metric to dominate in my teaching evaluations.

Perhaps the following weighting scheme could be used to evaluate the teaching part of my job: (1/4) current student evaluation; (1/4) peer evaluation [faculty member]; (1/4) outside evaluation [perhaps a local alum who wants to give back to SIU]; (1/4) former student evaluation [poll recent graduates who are 3 or more years out].

Admittedly, this suggestion would be expensive relative to the current system. However, once in place, and if used widely, could be feasible. Former students and alumni would be involved, and therefore, remain attached and interested in how the university is performing.

I believe the benefits would outweigh the costs.

I'm not claiming that this idea is the best idea possible, but it is far better then what SIU is doing today. I have written, back in my "first, fire all the chairmen" orbit, that the chairmen should go into the classroom and watch a class unannounced (like they do at every other level of school). I also suggested that in classes that have common finals, you could easily check final scores against grades to see which teachers are giving a free letter grade or two away (yes, they exist and are considered top teachers based on student evals).

What an idea, do a little more work and try to get better results. Great comment Anonymous, you are the first professor who has written an idea of how to improve the situation.

It is hard to disagree with the comment's last line, "I believe the benefits would outweigh the costs."

Your comments are always welcome.


Scott McClurg said...

You might get more accurate evaluations this way, but you won't stop grade inflation because it has little to do with the evaluations in my opinion. I have nothing against better evaluations, FWIW, but I'm not all that convinced that they're all that meaningful in the grand scope of things. Most instructors are above par, in my experience, and the ones who aren't are widely known. In other words, the evaluations are non-informative as to the key criteria -- sorting out the professors who can't teach. They would be useful for spreading best practices and the like but, again, I think that goes on anyway.

Anonymous said...

scott mcclurg said "Most instructors are above par..." What is par for an SIU-C faculty member? What is par for a course that has more to do with "goofy golf" than Pebble Beach. SIU-C has long since ceased to be a Cathedral of the Culture. Might the "Priesthood"and "Bishops" have had anything to do with that?

Peter in Carbondale said...

Hi Scott -

I think your theme song on this issue is that old Paul Simon song, "I am a Rock."


I like this link too -

Peter in Carbondale said...

If you do a quick review of Dr. McClurg, he seems to check out as a good guy. My feeling is he is just rationalizing because he is stuck here for the long term. I like his comments in general and appreciate his views.

When you start talking about above average instruction, it depends what group you are averaging against.

It is interesting when you don't agree on something important. For me the study and execution of management is very important. Scott doesn't think that management is important (based on his comments here) and doesn't understand management.

Oh well, it is OK to disagree. At least we have an intelligent voice, that has an understood difference of opinion added to the mix.

Scott McClurg said...

To clarify "above par" -- I was referring primarily to the folks I know in the social sciences, which is my area of expertise. Maybe everyone outside of there just stinks, but I'm skeptical in the face of actual evidence.

Plus, I'm not "stuck." I had the option to leave to a program widely perceived as "better" but decided to stay based on the administration's committment to my department and me. And, I understand management better than you think (why is that dig necessary?).

Peter in Carbondale said...

There is no dig, you disagree on a fundamental level with everything I write about SIU.

We all get stuck in a place and need to "buy in" and try to make it better. Dan Worrell was Dean of Business, until the day his son graduated from high school and was accepted at a college elsewhere, then he was gone. Was he "stuck" here, well yes he was. I have been stuck in Carbondale for a couple of years, it takes guts to examine where you are and removed yourself. Maybe you are stuck because your department has hired well and you have a great group of people to work with, a reasonable chair, a sunny office or a good PhD student? Maybe one day the things that make you stay here will change and you will get unstuck? If you don't think you are rationalizing, you are mistaken, we all do.

More importantly, I believe what you write is what you believe (I am willing to accept changes, typos, and misstatements too. This is just a blog) and I believe what I write is what I believe. Not that either of us can't change, learn, or grow, which is why I spend my time doing this blog.

Having been in town for years, I have been having this kind of back and forth with my Father for all of those years. He believed when I moved back to town that Professors were a rock and weren't effected by management. He also knew that management at SIU was deeply flawed and he had worked very hard to create systems that allowed professors to do something when management screwed them over. For example, when I was 15-18 he as working hard on creating what is now the JRB. He has known for years that SIU has been on the way down, he came here in 1972 and knows a whole lot about the slide. His predictions about the future of SIU have been scarily accurate through the years (for example, he and I both met Wendler right after he came. He predicted Wendler was going to fail after a 30 minute meeting. I adapted a wait and see approach. He was right, Wendler just didn't get university leadership.).

After years of discussion, I think he finally is beginning to understand the power of management. It isn't that is makes people smarter, but it allows the best people to soar to great heights. Bad management makes results sink. He really likes Jo Anne Argersinger, because she was a real leader and she got what SIU needed to do. It was interesting to me that you know that Argersinger would be better for the whole university then Wendler, but you don't believe in the power of management. If management doesn't effect you, why did Glenn Poshard becoming President of SIU lift so many spirits.

I think it is interesting that SIU's decline is so easy to document, but some people claim there is no decline. When you go from 25,000 students on campus to 19,000 students, that isn't luck. It could be that research productivity is up, but I don't evidence of that. What I see is an organization with pockets of people working hard, but far, far too many, do far too little.

Like talking about instruction, it depends on what group you are talking about when you refer to stinks. Are the students coming into SIU worse students then showed up here 20 years ago? Well, yes they are, but they can do Facebook and play Quake far better. Are the professors still smart and from great schools (in general)? Well, yes they are. Are the research results better now then 20 years ago? Well, no they are not.

When you compare SIU to its peer group of Public Schools in Illinois, SIU has to receive a failing grade. Every other school is up, and SIU is at the very bottom of the curve in results. So, when comparing Apples and Apples, SIU is the worst performing Public University in Illinois based on revenue producing units (students). No one can argue about that, can they?

If we assume that everyone can understand that SIU's results are way down. The question about why SIU is failing compared to everyone else is a key question of this blog. Hopefully, it will become a key question for the professors at SIU. They are the group of "rocks" most likely to fix it. The sad thing is that people who see themselves as "rocks," generally don't move until someone breaks them to pieces with a hammer.

Congratulations, I have written you personally a long and complicated reply to a comment. A statement that you are worth spending time on, I almost never do it. You are exactly my target market, you can help turn SIU around if you care to. You are a good professors. You care. Now figure out if part of your mission is your own work or the whole institution. Influence management, reach beyond your scope. Figure out the truth as you see it and work to improve things.

I get tired of comments that say things like, I don't know and don't care about anything outside my area, I haven't bothered to compare so I don't understand, I haven't talked to anyone... but I don't agree with with your analysis, even though I know very little about any of the issues.