Nobody is Perfect

During our visit, the guy tried to up-sell us, downplayed the value of our trade-in, did the, “Let me talk with the manager” nonsense, permitted a massive error (that we caught immediately) in the financing agreement, and  periodically reminded us that we would be sent an assessment form via email that we needed to fill out.

Oh – and if he didn’t get perfect scores on the form, he would be fired. In other words, if we didn’t write that he was God’s gift to car buyers, his supposed firing would be on us, forever.

How do we respond to that?

The goal of assessment is no longer to use meaningful tools to gather sensible data by which we can evaluate the condition of an educational program (or of a business, or a teacher’s effectiveness via SROIs, or even an individual car sale).

Rather, it is now to cook the books with nonsense data to make the educational program or business or car sale look perfect. “Truth” is what I say it is, the facts be damned.

Assessment has gone off the deep end, so evaluation has become advertising, not judging based on sensible data. If we cannot trust our data to be representative of our system, then science, which relies on credible data at its core, ceases to be science.

Let’s dig into these ideas.

First, assessment and evaluation.

Assessment is gathering sensible data that measure the effectiveness of a program, organization, person, etc. The data can take all kinds of forms, from experiments to questionnaires to focus groups – small groups who discuss specific questions – to one-on-one visits.

Evaluation is the interpretation of the assessment data. You find meaning in the evaluation of assessment data.

An example I often use to illustrate these concepts is that of a runner who is having trouble with their knees.

An orthopedic specialist whose professional understanding dictates the tests to (forgive me, pun intended) run.

These tests might include:

• bending the leg to look for mobility;
• taking an MRI to see inside the knee;
• looking at the wear pattern on the running shoes.

Each of these tests provide sensible and meaningful data. That’s the assessment part.

Maybe the knee can bend only a few degrees, and the MRI reveals an unexpected white line of fluid in the meniscus. Her running shoes might show unusual pronation. The assessment does not interpret – it only gathers data.

Now the doctor is ready for the evaluation part. This is where she uses the assessment data. Evaluating the MRI data shows a clear tear. The other tests can help confirm this evaluation – this diagnosis. Based on the evaluation, the doctor can recommend treatment, such as surgery.

Garbage in, garbage out.

Recall above that I discussed meaningful data. What makes data meaningful or not meaningful? Why is the doctor’s data meaningful and the car salesman’s, or, often, Yelp’s, not?

When the assessment is geared toward a certain outcome, the data is biased.

The car salesman was not interested in my opinion of his work. He was interested in having his opinion of his work transmitted to the survey using me as a ventriloquist’s dummy.

The data would have been nonsense – not meaningful – because his opinion of his work is not what the survey wanted to know.

Garbage in, garbage out. We could accurately call it Fake News.

Has assessment gone off the rails?

I love the apartment complex in which I live. We have a beautiful view of the regional bluffs near the place. It’s a walk to shopping and a lite rail to the city center.

Some weeks back, after the maintenance staff did some timely repairs (we love the maintenance staff as well!), I was asked to rate the place on Yelp. I was happy to do it. Great rating. It was my personal assessment of the place and the staff, which I believe readers can rely on. But can they rely on all the assessments? Food for thought. What about the Ratemyprofessors website? Hotels on TripAdvisor? Are those meaningful assessments that lead to reasonable evaluations? Why?

When the assessment is geared toward a certain outcome, the data is biased.

These days, you and I and so many others are flooded with assessment requests from restaurants, chain stores, cable TV service, even lite rail operators. How many restaurants have you visited in which the waitstaff says that their job is dependent on getting a superior assessment from you?

Assessments are getting a deserved bad rap because they are becoming a cudgel to use against employees in fear of their jobs, and, therefore, against customers, who are asked to protect these jobs.

Assessments are becoming the anti-assessments, in which evaluation means nothing. Regarding the data, it’s nonsense in, nonsense out. Assessment and evaluation are too important to be part of the flatlining of meaning that’s going on now. We must find ways – and I don’t know what those are – to give assessment and evaluation back to the realm of science. Only then can they have their meaning to the realm of education.

The Grizzled Teacher (TGT) has taught at eight public universities, one private college, two “flagships”, and several regional state schools during 39 years in post-secondary education. TGT has directed and taught in a 6,000-student first-year science program with a typical class size of 300 students, has taught graduate courses with 5 students, and worked with a thousand faculty, instructional staff, and more than 10,000 students. TGT was on short-term contracts for many years, has been tenured for many more, and has won two-dozen university, state, and national teaching awards. TGT hasn’t seen it all, but has seen a heckuva lot.

After all these years, I still don’t know all the answers, but I’m getting better at knowing the questions.

To quote the late, great Joan Rivers, “Can we talk?”