Value Above Replacement: Sabermetrics for EdTech

Have you seen the movie, read the book, or heard of Moneyball? Here's a quick primer if not - be sure to keep reading after the excitement of this trailer. :)

Billy Beane, real-life general manager of baseball's Oakland Athletics, was faced with a challenge I'm sure you recognize: get a ton of performance with what never seems like enough money. Knowing he was never going to get ahead (or feel satisfied) trying to make conventional, "sexy" decisions, Beane recruited a statistician to go behind what his scouts' eyes were seeing, and make his personnel decisions based on parameters, or metrics, that were going to provide the most improvement for the best price. Generally, this outlook in baseball is referred to as sabermetrics, and is first attributed to Bill James.

the specialized analysis of baseball through objective evidence, especially baseball statistics that measure in-game activity.

I very simply substitute "technology" for "baseball" and "learning" for "in-game activity" in the above definition from wikipedia and I have definition for the principles we should be following when we choose technology tools for purchase on any level, and when we plan our everyday lessons.

the specialized analysis of TECHNOLOGY through objective evidence, especially TECHNOLOGY STATISTICS that measure LEARNING.

I'll be the first to argue in favor of spending as much money as possible on district technology budgets, but only if an actionable research plan has been followed first. Bad money spent just takes away from sports, theatre, music, teacher salaries, etc.

My favorite sabermetrics stat is WAR - Wins Above Replacement. (Here's how Baseball Reference calculates) Basically, WAR aims to compare any baseball player on a team against what an "average" player at the position would do. How can we apply that to education technology? I don't have numbers for it yet, but using my new sabermetrics definition for technology, (and some intuition and reflection), I can compare any piece of technology against the "average" instructional tool or method.

Wet-erase overhead vs. SMARTboard

iPads vs. laptops vs. computer labs

Graphing calculators vs. Graphing software

Graphing calculators vs. graphing by hand

student clickers vs. paper response cards

students on whiteboard apps vs. small, physical whiteboards

The observations would be simple, the CHANGE or DECISIONS would be more difficult. If the technology method as you're applying has insignificant statistical difference between traditional methods, then you have 2 options:

1. Transform the way you use the tool, get new data, compare again
2. Pass on that tool and continue using the "old" method

Its usually fairly easy to justify purchase or use of technology, even if students are more successful. Our culture ingrains within us such a reverence of new technology that often the first impulse is not to question the technology, but rather, search for other factors. The students? The teacher? Hour of the day? Grade-level?

Could you commit to making anti-technology decisions in favor of future, more capable tools or un-flashy, proven methods? What's the VAR of that technology in your classroom?