Educational Metrics

Charles Hugh Smith wrote an essay that I think is very perceptive.  All metrics are abstractions of a more complicated reality.  It is easy to get caught up in trying to manage the metric or "the business model" rather than manage the process or the business.

Education is grappling with the methods involved in becoming more outcome focused. Industry went through a similar convulsion as foreign competitors became more capable in terms of volume and quality...even as they retained the currency exchange rate advantage.  Those convulsions thrashed North American industry for about thirty years.  Much resources were squandered.  Many blind alleys were investigated.

Education seems to have stalled out in the "measure it to death" stage of process improvement.  Some of the educators I talk with are exhausted by the constant testing and vaguely defined need to "analyze" the results.

The act of measuring does not automatically improve quality.  Nor does the act of "analysis" guarantee effective action.  Perhaps it is not an accident that "analysis" rhymes with "paralysis".  In most cases, data collection and analysis beyond what is required to diagnose specific, high-runner root-causes is not cost effective.

Root causes of educational failure

Not all causes of educational failure rest with the teacher.  But some causes do.  "Testing" and data gathering should be designed to capture evidence, in real time, of those failures.  The declaration of those shortcomings should be in plain English to better suggest the actions that must be taken to fix them.

There are a limited number of teaching failures. Testing and analysis in excess of what is required to effectively identify those failures is of very limited value.  (In retrospect, I am describing a Process Failure Mode and Effect, PFMEA, approach to education.)

Examples:

Failure to spend enough time presenting the specified content. I.e., time-on-task.

According to my recollection those teachers fell into two main categories.  The highly distractable:  "Gee Mr. Finch, tell us about the time you went skydiving."  And the terminally disorganized.

Solution: This problem can be diagnosed with a stop-watch.  How many minutes-of-the-hour is the instructor actually presenting the lesson?

Failure to control the classroom behaviors, allowing a chaotic non-learning environment to evolve.

Teachers who fail to break the downward spiral are in trouble.  A businesslike learning environment keeps the students busy and engaged.  Failure to control troublemakers interrupts the learning environment and causes more kids to disengage and start causing trouble.  Use technology...there is no reason to turn your back on the class.  Get the class invested by doing problems on the board.  Bust the troublemaker's chops.  Ask for help.

Solution: This problem can be diagnosed with a sound level meter or by asking the custodians whose room is a mess.

Failure to interact with the class to verify that learning actually occurs during the lesson.  "Push" teaching.

Some teachers call this "teaching the book".  It is an efficient use of instructional time as long as the majority of the students are tracking the material and reassembling the information in their heads.  The problem is when a significant number of students do not learn the required concepts.  Then there is an air bubble in the siphon.  The lesson needs to be retaught, perhaps using an approach that may be more relevant to the student's life experiences.

Every class has a few learners who do not learn efficiently via standard "push" teaching.  Push also becomes a failure when the teacher does not take ownership of teaching the students who require other teaching modalities.

Personal Aside:  Students practicing problems on the chalkboard paired well with "push" teaching.  The students were attentive because they did not want to embarrass themselves in front of their classmates.  Doing the problem provided immediate practice in recalling the concepts and using the skills....most learning failure is not because the knowledge did not get stuffed into the brain but rather because the student cannot access that knowledge when they needed it.

Watching the students perform the tasks provided the instructor of individual and group soft-spots.

Most of the really proficient engineers would do college homework problems on the board with two or three friends.  Doing the problems provided double transparency.  Poor scholarship was exposed and corrected.  Excellent scholarship was copied.

Solution: Testing is a good way to make adjustments to push teaching but the tests must be frequent and focused on the material that was recently presented.  Mid-lesson summary by class holds them accountable for the material and can provide intra-lessson feedback to the instructor.

I open the floor to comments.  I would dearly love to hear of additional "failure modes" that I missed.