Consider Learning

I still remember when I got my smartphone.  I’d used some friends’ smartphones to some degree, and so I knew the basics of moving things with my finger, using two fingers to zoom, etc.  However, I lacked a deep understanding of smartphones and apps, and hence, I was very nervous when using my phone.  I didn’t know what effect installing certain apps would have, nor agreeing to various licenses.  I didn’t know what the preinstalled apps did.  Essentially, I was scared of doing much with my phone because I was worried that I would mess it up, and I didn’t want to ask for help because I’d be demonstrating my ignorance.

I’ve felt math anxiety exactly three times before.  The first time was my first summer at CTY, when I first learned proof by induction.  We had morning quizzes, and I had only a surface understanding of the topic; I was worried that I would demonstrate my ignorance to my teachers, whom I greatly respected.  The second time was my first summer at Mathcamp.  Throughout the entire program I felt less prepared than my peers because I was seeing all this math that I didn’t really understand.  I knew the words behind group theory; I could talk about subgroups and say the words “quotient group,” but I didn’t know what they meant.  I was terrified that this would become apparent and that my friends would lose respect for me.  The final time was taking algebraic topology at MIT; I had failed to do my reading and I was worried again that I would be discovered for not having understood what we were learning.

To me, the experience with the smartphone and my experiences with math anxiety are very similar.  In both cases, I knew some words and some surface features of how to work with something, but I knew nothing about what made it tick.  I did not understand, if I poked in one spot, which other spots would be bumped out.  Without understanding that structure, I couldn’t do anything beyond what had been rigidly prescribed to me, and hence I felt like I was lost in a world that I did not truly understand.

I believe that this is what math anxiety is like.  If you do not understand the mathematics, and you are not comfortable repeating a given procedure to get an answer (perhaps because you sense that there should be something more, or perhaps because you are not very good at memorization), then you’ll feel like you’re in a foreign country where you don’t understand the language — or the laws.  You’re terrified that you’ll make a mistake and get arrested, but you can’t communicate with anyone to ask what you should do.  Thus, you mimic what everyone else is doing and hope that it’s good enough.

If you want to teach someone a foreign language and set of laws, we have an established way of doing so.  Start by giving them basic language tools (reading picture books, if you will) and by communicating the basis of the laws (the morality that the people share).  The corresponding aspects of mathematics are understanding the precision with which mathematics is communicated, and understanding basic arithmetic and why it works.

If we move students to more advanced mathematics without those basics, then it is that much more difficult to keep understanding, and they are forced to resort to mimicking what they’re taught.  They’re lost in a foreign country, scared of what’s happening to them.  They’re terrified someone will see them do something wrong, but they won’t really know why it’s wrong.

That’s the disservice that I believe we do to students when we force them to go to more advanced mathematics before they understand the earlier levels.  It’s the disservice we do when we try to teach how to do something to those who don’t understand the earlier steps.

Why do boys so often outperform girls at the top end of mathematics achievement?  This question perplexes academics, is the source of much consternation in hiring, and has caused no end of trouble for those (like Larry Summers) who have waded incautiously into the debate.  As en educator, I want to understand if we as a society are not doing enough to help girls enter math and science.  But I’m not convinced if someone tells me that girls don’t get as many math PhD’s as boys, because it doesn’t address the root cause.  I need to see something about society’s impact on those numbers.

First, some background.  Yes, in school, boys do test higher in math than girls on average, but the difference is usually very small.  If you don’t like relying on tests, girls tend to have higher marks in school.  The key thing to understand is the score distribution: more boys score at a high level, but more boys also score at a low level, bringing the average back to about the same score as girls. Hence, men are overrepresented as top achievers, but also as bottom achievers.  (There is a provocative argument to be made that there are evolutionary reasons for men to be greater risk-takers and to have greater variation in their performance.)

Nevertheless, there is considerable evidence for strong cultural factors that impact girls’ performance.  For example, here’s some evidence for factors that may affect girls in the aggregate (but which, fortunately, are also overcome by many girls):

• There is clear evidence of stereotype threat.  For example, if the College Board asked for test takers’ gender after the AP calculus exam, rather than before, some 5000 more girls would pass it nationwide.  Surely this stereotype danger affects not just test-taking in the moment, but also impacts students in each classroom, each day, little-by-little.  (Full disclosure: there are also some doubts about the impact of stereotype threat.)
• Besides, in general, women don’t do as well on tests, and yet tests define a large part of our success and advancement in such subjects.
• Even when sexism is not overt, we all have mental biases of which we’re not aware.  Just as double-blind auditions significantly boosted women’s admission to symphony orchestras (i.e. having potential candidates audition behind a screen), the same is true of peer review for journal articles.  Again, this effect must surely play out again and again throughout school, with many little encouragements or discouragements building upon one-another.
• The clearest evidence of social factors affecting girls’ performance might be this study that shows that top-performing girls tend to consistently come from just a few high schools, while top-performing boys come from many different high schools.  In theory, this means that those high schools have figured out how to more fully develop girls’ abilities.
• Another strong example is this study in which the math anxiety of elementary school teachers (almost all women) strongly affected girls’ math performance and perception of gender roles, but had no impact on boys.
• Another, similar study that might find causation is this recent one in which it was found that American parents talk to boys age 20-27 months about numbers much more often than they do to girls.
• Women who are exposed to romantic cues report less interest in mathematics, perhaps a reflection of what society views as “feminine” or desirable in women.
• This study demonstrates several countries in which girls and boys have the same variance in performance, and also notes that there are more girls on serious IMO teams when measures of gender equity are higher.
• Another cute example comes from a study that gave spatial puzzles to two tribes, one patrilineal, one matrilineal.  The gender gap disappeared in the matrilineal society.  (Although one has to ask, why was it merely eliminated and not reversed?)

If we search for a core issue, the popular perception of girls’ math ability seems to be a major factor.  Perception of girls’ math ability can explain many of the above bullet points: it is likely the core cause of stereotype threat, it may affect parents’ conversations with children, and it likely impacts the kind of role model that an elementary teacher might be.  If we could just address this one issue, if we could telepathically make it clear to each person in the world that women can be tremendous achievers in math and science, then we might eliminate the gender gap or significantly close it.

Now, I want to be clear: the jury is still out.  Political correctness dictates that it’s much easier to publish a study that explains the gender gap through social rather than biological factors.  The preponderance of evidence points to clear social factors that influence girls’ performance, and those social factors all seem tied to our view of girls’ ability to achieve, but there’s no way to judge what would change if we could correct for social factors.  Women might choose other careers for very legitimate reasons.  There might, yes, be biological differences.  We don’t truly know.

However, this lack of evidence should make no difference to our policies!  If it turns out that the impact of social factors is small, then not very much is lost by making a concerted effort to change the cultural perception of women’s achievement in math and science.  On the other hand, if it turns out that the impact of social factors is large, then we gain tremendous value from repairing cultural perception of women in math and science.  Moreover, it seems likely that the impact of social factors is large.

We should get rid of the cultural factors that prevent girls from making good on their ability, or we should at least strengthen the paths for girls to succeed independent of cultural factors.  Which begs the question: how can we effectively do these things?

Note: Post has been updated to add the Wisconsin study in which some countries have the same variance in performance for both boys and girls.  Post has also been updated to add the meta analysis that calls stereotype threat into question.

Let’s face it: educational technology is overhyped.  There’s nothing out there that gives the same education as a great teacher in a great environment.  But education technology is also underhyped!  Nothing else gives the potential for reaching so many people with the full richness of learning.

Yes, there’s a lot of potential in edtech.  But most startups fail, and frankly they fail because they’re often pretty dumb about what education really needs.  If we’re going to improve, we have to stop using technology “because we can” and instead use technology where it actually makes a positive difference.

I’ve long been an EdTech skeptic who believes that great things are possible.  While I am far from an expert on the field, here are my thoughts on the two big categories of EdTech, the failure points they’ve experienced so far, and the potential for excellence.

Technology for the Classroom

AKA “working with teachers,” this is technology designed to aid teachers.  Two favorite examples include Reasoning Mind which offers a computer-based math curriculum using Russian ideas in math education, and the “inverted classroom” model, in which students get content from videos or other sources outside of class and then do problems in class.  (The idea is that you can learn the basics from anyone, but time with a teacher is precious and should be used where the teacher is most valuable.)  I also like BLOSSOMS, videos made by MIT folks where the in-classroom teacher shows part of the video, does an activity with students, shows the next part, does another activity, and finishes up with the rest of the video.  BLOSSOMS is great because it brings experts into the classroom while working well with the classroom structure.

Why do these fail?  It depends on the tech.  Sometimes, the big innovation is trying to give teachers access to more data so that they can diagnose what their students don’t understand and help them—these fail because entering the data is too clunky, and takes too much time.  Sometimes the innovation is giving the teachers better ways to present material—although the SMART board has succeeded, most other technologies fail when the complexity of creating lesson plans goes up.  I’ve seen tech based on helping teachers decide which student to call on (often not enough benefit for time/cost), tech based on giving all students iPads (not enough educational material to make it worthwhile, and too distracting), and so forth.  Everyone has their idea for what teachers really need to become more effective, and they’re usually wrong.  That’s not entirely true—sometimes the innovators are right, but they haven’t taken the time to align to standards, nor have they made it easy for teachers to find and use their resources.

What are the opportunities?  Some innovations have potential for greater efficiency in classroom time (such as the inverted classroom), and technology should be able to help here.  Technology could also allow for better in-class problems customized to each student (adapting to their prior work) or better assessments.  We’ll eventually find a way to get teachers better data about their students.  Finally, people keep talking about well-designed interactive apps that would allow students to explore material in a non-linear fashion or to do projects where students analyze real-world data.  Although nothing seems to have taken hold just yet, it’s a tremendous opportunity.

Technology for Outside the Classroom

This is designed to skip the in-person component altogether, optimized for self-paced study or for online classrooms.  The motivating forces tend to be either “make education accessible to everyone” or “take the work of the best teachers, distill it, and scale it.”  Favorite examples include Khan Academy (even though I think there’s lots of room for improvement) and Art of Problem Solving.  There are also tons of online schools of various sorts.

Why do these fail?  Often, these technologies don’t account for how much less engaging it is when you’re not in front of a real person.  (That’s why live sporting events and live plays are still popular.)  They also don’t account for our short attention spans.  And, most critically, they don’t account for the importance of being around other learners, part of a community that spurs you to greater learning.

What are the opportunities?  The trend towards adaptive learning is a big plus here, although the give-and-take of adaptive multiple-choice questions remains less compelling than real human interaction.  For very motivated learners or those with someone (e.g. a parent) watching over them, online schools may be compelling.  There’s also an opportunity in short, awesome online content that students can watch, share on Facebook, etc., and then perhaps follow up on in greater depth.  However, I think the real future is in social networking, where students can watch a video and chat live as the video plays, or share questions with a small group of trusted online friends who can help them through the material.  As near as I can tell, that’s the only hope for a real online community of learning that will keep students learning.

The problem with education technology is that there’s a lot of “cool stuff” out there that is genuinely very cool—but it just doesn’t actually educate that well.  I think we’ll get there eventually, as more collaborations evolve between knowledgeable educators and really awesome tech innovators.  I’ll even share some of my own ideas in a future post, places I think there’s space for real innovation.

Meanwhile, as an educator, I’m not holding my breath.  I’m happy to adopt things that come around that will really make a big difference, but until they do, don’t expect me (or anyone else) to jump on your bandwagon!

Recently, I attended the Building a Better Commonwealth forum hosted by the Boston Globe on “Building the Talent Pipeline.” In other words, how do you produce more high-level STEM jobs? Here are some of the things that I learned:

• Massachusetts is unique for having a statewide plan for developing STEM talent.   Like the (national) report from the President’s Council of Advisors on Science & Technology, the MA plan discusses the need for nurturing curiosity, suggests a strong attention to standards, wants to get more people pursuing STEM, and wants to more effectively prepare teachers.  Unfortunately, it seems that both of these reports fail to address allowing students to excel.  (See below.)  That said, the Massachusetts plan is very impressive, with an eye towards creating coherent curriculum, good experiences, role models, and much more, and a plan for implementation.  If MA pulls this off, it could be quite significant.
• Apparently STEM learning in MA is particularly pushed by Lt. Gov. Timothy P. Murray. It’s interesting to see how this kind of project gets high-level support.
• There is a dramatic hiring shortage in STEM. (Well, I already knew that.) But the STEM jobs are not just the desk jobs that you might have in the financial industry or at Google. They’re also repairing and servicing power lines, because the job is so dangerous that you need to know what you’re doing. They’re also factory jobs, which are so automated that they require special expertise. And so forth.
• It’s really weird to explain that you live in Boston but run a summer program for New York City students.

Here are three reflections:

• Parents must know what their kids need and push them towards it.  It’s great that there are all of these initiatives, but ultimately, it’s very hard to make up for low-knowledge parents or communities.  There is too much that students learn about from parents; too much push that parents need to give to apply for or attend other programs (how many times did your parents wake you up on a Saturday morning to make sure you got to your activity?); too much information where parents must be part of the process.  Usually, it’s not that parents don’t care deeply for their kids (just the opposite), but that they don’t have a model from their own parents of pushing kids in this way and so it’s not natural to them to provide the push.
• Rigid curriculum can be severely limiting.  Enforced curricula from a state or national level can block hands-on learning, customization of a student’s work to their interests, or customization of a class to a teacher’s skills.  I think there are good ways to design a curriculum that is flexible to this kind of work, although it hasn’t been done yet.
• The community does not understand the difference between baseline achievement and excellence.  Most people in education do not realize that getting all A’s does not make you a world-class student.  Even taking lots of AP courses does not necessarily do so.  Students today do research, attend summer programs, do various competitions, and more.  There’s a national infrastructure set up by small independent groups designed to help students achieve excellence, but no one in education pays attention to it.  In part, it’s because there’s a perception that all-A students have “already made it.”  In part, it’s because it can be politically daunting to want to help top students succeed more.  In part, it’s because of a lack of knowledge.  But this kind of structure is necessary to create the best-possible STEM workforce, and it doesn’t have to be based around selective admissions.  There are many good opportunities that anyone can participate in and gain the opportunity to excel.  Yet the constant focus on creating new STEM standards and testing is never going to produce a STEM workforce because it will always be based around broad ideas that don’t develop the top students.

As you can tell, I’ve been thinking a great deal about what a good STEM pipeline should be.  Look for some systemic thoughts in the coming weeks.

Apologies for the long absence.  The past few months have featured intense work and family obligations.  I have missed blogging, and now I’m back!  My hope is to spend some of the next few weeks thinking carefully about what a good pipeline looks like for developing a STEM workforce.  We begin with the next post, my reflections on a recent Building a Better Commonwealth event.

It’s Splash season, so my blogging has fallen behind.  To keep you satiated, here are some interesting articles I’ve read recently and some thoughts on each.

• Harvard Education Letter writes about the Waldorf model of schooling.  A small movement, but one that even has some Gates Foundation support, Waldorf schools have a very long-term outlook to children’s success.  They do a lot more art, and a lot more personal exploration.  “In early grades, strict Waldorf classrooms delay overt academic work in favor of imaginative play and movement centered on myths and fairy tales. Multiplication tables, for example, are not taught until fourth grade, although kindergarteners may gain early math skills as they knit. Even high school students studying science find a narrative focus as a teacher describes how Charles Darwin struggled to conceive his theory of evolution. Students may draw muscle cells to learn about them. There are no textbooks; students create their own “lesson books” to chart their learning.”  Test scores are rising at this small sample of schools, although that might just be because their leaders are particularly passionate.  Regardless, the article provides an important reminder that when you focus in single-mindedly on one goal like test success, you can forget what’s really important to bringing a child into adulthood, and in the process sabotage your own test results with short-term thinking. (*)
• Washington Post reporter Jay Mathews blogs about improvements at his old high school.  The twist?  He’s strongly reform minded (use tests to evaluate teachers, more charter schools, etc.) and the school’s reforms are precisely the opposite.  I normally hate writing from partisans in this debate because they just talk past each other.  Here is one partisan showing how the other side has done something significant.
• The Wall Street Journal writes about students who switch to easier majors despite lower pay.  College courses in science and math are a huge step up from high school courses in terms of abstraction and the independence required to succeed, and so students change majors.  Some places seem to want to make the college courses easier, but that leads to insufficiently prepared graduates.  Naturally, I think the solution is for high school students to study more abstract topics in greater depth.  That’s not on the horizon, so a lot of colleges offer remedial work—which is just like a high school course, but maybe sped up!  Most students fail their remedial course, or at least that’s what happened when I was at the University of Illinois.  So instead of offering more of the same, I propose that colleges should develop deep, abstract “remedial” courses that teach math and science the way it should be taught.  These courses will adequately prepare students and the ideas within these courses might trickle down to high schools, becoming the standard of college preparatory work.

(*) A more common example of short-term thinking: teachers cram test-prep into the end of the school year, because it’s so important to them and students that the students pass.  But students forget the test prep and lose learning time from it, so in future years, they have less to build on.

I’ve spent much of this week doing interviews for Early Action applicants to MIT.  This volunteer role lets me meet a lot of very cool young people.  Moreover, although I live in Boston, I do interviews in my home town of Vestal, NY to a radius of a couple hours out.  It’s a sparsely-populated region, so I meet a wide mix of folks.

This year, I’ve been struck by how much applicants are shaped by their surroundings.  The difference in resources between schools and communities is stark.  At one extreme, there are famous magnet schools like as Stuyvesant or Thomas Jefferson which have highly-advanced courses, advisers to help students complete Intel research projects, training for national-level competitions, and high awareness of advanced summer programs.  At the other end, rural or inner-city schools that have virtually no AP classes.  But right near Vestal there is also wide variation.  Vestal, my alma matter, has a number of academic clubs, advanced study through both AP courses and an IB program, and, for those students who take initiative, the opportunity to take classes at SUNY Binghamton (one of the top New York public universities) or even to do research there.  In contrast, I’ve interviewed students from 30-60 minutes out whose schools have fewer than 5 AP courses available, and where awareness of out-of-school opportunities are minimal.  There, if the family has sets the right kind of tone, then kids can discover serious resources and potentially make a lot of progress.

It’s easy to forget in all of the debates about testing, teachers, and curriculum that our learning and achievements are strongly defined by our communities, their awareness and emphasis on learning, and our opportunity to learn.  The best students I’ve met from more remote schools had families who provided them with reading suggestions and role models.  Creating opportunity to learn (and the inspiration to pursue it) has been a driving force of my career in education from Learning Unlimited to the Summer Program in Mathematical Problem Solving to Mathcamp.

Unfortunately, you can’t just say “here, kids, have an opportunity!”  There are innumerable influences on students’ time.  To pursue real studies, they need some mix of older peers to be role models, teachers who will be supportive, knowledge of how to pursue those studies, a sense of where serious study will take them, and more.  Yet even those opportunities aren’t quite enough.  Most of what we choose to do comes from serendipity.  We hear about an opportunity right when we have a moment to Google it.  We have an argument with a friend and, feeling isolated, we immerse ourselves in our work—just when an interesting topic comes up—and because we happen to do well on that test, we decide to pursue that topic more seriously.  Our attention catches on random things, and if our world isn’t permeated with topics of interest, we’ll miss those serendipitous moments.

Which is to say that we have a long way to go if we’re going to get more people to pursue learning outside of school.  For my part, now I’m thinking about organizing a “math day” for anyone within 90 miles of Vestal sometime in the spring so they can get a taste of advanced math.  What are you going to do?

Why mathematics?  What purpose does it serve?  Why should we teach it to students?  There tend to be roughly three broad categories of answers to this question:

1. It is useful.  Math is needed both in daily life and in work, and without a strong math education you are closing doors to excellent careers.  In this vision, the math curriculum should be designed to teach specific knowledge and skills.
2. It trains you to think.  Math poses deep and abstract problems.  Facility in solving these problems strengthens your critical-thinking skills across the board.  In this vision, the math curriculum should be designed to build abstraction and pose challenging problems.  (Warning: there may be some research that says that critical thinking in mathematics does not transfer to other disciplines.)
3. It is pretty.  Math is gorgeous; it should be explored and enjoyed.  It is one of the great achievements of humankind, and every student should have the opportunity to appreciate and understand it.

I believe deeply in all three.  It is dangerous to ignore any of them.  If you design a class that explores the beauty of pure and applied mathematics, builds abstract thinking, and poses deep problems, you can easily meet all three criteria together.  But when you ignore one of them, mathematics suffers.

It’s for this reason that I am not nearly as enthralled by Lockhart’s Lament as many of my fellow mathematicians.  Lockhart’s Lament is a powerful and eloquent description of why mathematics is beautiful.  I couldn’t agree more with all of the points Lockhart makes for the beauty of mathematics.  However, when he proposes to scale all teaching towards the exclusive goal of beauty, I believe the result is both dangerous and unrealistic.  (Lockhart later wrote a clarification that he did not mean to be making a proposal nor to imply a dichotomy between pure and applied mathematics; although he walks back from what the Lament says, it fairly clearly speaks for itself and it is what is being passed around between mathematicians.)

Teaching only for beauty is dangerous because without encountering specific content, students would not learn mathematics that is important for their lives.  Those who will not pursue math-oriented careers would not have the tools to manage their lives or interpret the world around them, while those who will pursue math-oriented careers would find themselves without the necessary tools.  Besides, not everyone finds mathematics beautiful for the same reasons.  Some find it beautiful for its applications and relation to the physical world.  Some might not find mathematics beautiful at all, and I do not consider that a failing.  Just as I do not particularly enjoy opera and do not want its appreciation forced upon me, I would not want to force appreciating mathematics on others.

Teaching only for beauty is unrealistic because we simply do not have the teaching force that is capable of teaching mathematics purely for its beauty (as Lockhart notes).  Moreover, even if teachers were in a position to do this, if each teacher designed their own curriculum with no guidance then classes would be filled with students with very different background knowledge and designing coherent lessons would be impossible.

Paul Lockhart teaches at Saint Ann’s School, a private school in New York City whose tuition ranges to over $30,000/year.  It has a free-form curriculum focused on the arts (and with many children of artists attending).  It is a wonderful school, ideal for Lockhart’s self-described teaching style, and I have known several teachers from there who are quite remarkable people.  However, to take that experience and generalize it to all of K-12 education is neither wise nor truthful.

In the end, I believe that it is possible to construct a school mathematics curriculum that combines all of points (1)-(3) above.  We should not focus on one at the expense of another; we should instead see the wonderful ways that they can all work together.  Lockhart’s essay is a wonderful demonstration of the power of mathematical beauty in teaching.  It is now time for the discussion among mathematicians to move beyond this very satisfying lament and on to incorporating all aspects of mathematics.

In his follow-up/reply to critics, Lockhart writes, “My point is that at present we have neither Romance nor Practicality – nothing but a jumbled, distorted mishmash of pseudo-mathematical vocabulary, symbols, and mindless procedures.”  With this I completely agree.  With his proposed solutions and measures of success—well, I’m not there yet.

Which of the following makes for a good history lesson about the underground railroad?

1. A traditional lesson in which the background of the underground railroad is presented, major players in the story are introduced, an explanation is given of the politics of the time, and the conditions that escaped slaves went through to get to freedom are presented.
2. For a fourth-grade classroom, have the students bake biscuits, which were a common staple on the underground railroad, in order to gain an understanding of what runaway slaves did and ate.
3. Give tenth-grade students original documents from the underground railroad that they use to draw and debate conclusions about the underground railroad (like real historians do).
4. Run a class discussion in which students are asked questions to elicit their understanding of life within the underground railroad.  How did runaway slaves obtain food?  How were they able to prepare it?  How were they able to pay for it?  What might the consequences have been for someone who housed runaway slaves?  And so forth.

Easy answers: (1) sounds fine if you need to transmit lots of knowledge, but boring.  A good lecture will communicate facts as well as connections between those facts, and it will build important neural connections to help students remember and process data.

Option (2) is nonsense.  The students will spend their time thinking about measuring flour, not the underground railroad.  (Math teachers, take note—this happens in your classes, too!)

Option (4) is great.  It prompts students to reflect on what they have learned and it leads them to make connections between topics.  By giving each of these items real thought and relating it to other facts they might know about the time period, their overall understanding will be greatly strengthened.

In his fascinating book Why Don’t Students Like School?, Daniel Willingham argues that option (3) is a mistake unless your goal is building student excitement.  But… surely not, you say!  Is this not what real historians do?  If history class was about analyzing real documents, would that not more deeply engage students’ minds and show them what the field was really like?

Not so, says Willingham.  This kind of exercise is asking students to think like experts.  However, he says, the cognition of experts is fundamentally different from non-experts.  Here’s how:

• Experts think in terms of functions or deep structure, while novices think in terms of surface features.  For example, suppose you ask novices and expert chess players to memorize a chess board.  When replacing the pieces on the board, the novices will put them down in clumps organized by location on the board.  Expert players will put them down in clumps organized by function—pieces that are threatening each other will go down all at once, even if on different ends of the board.
• Experts’ focus on deep structure allows them to ignore unimportant details and immediately see the useful information.
• Experts have procedures that are essentially automatic from repeated practice, so they can execute those procedures/thought patterns without the mental overhead of novices.

Willingham then notes that becoming an expert takes 10,000(ish) hours of practice.  He reflects on judging science fairs where students churn out lots of basically useless science experiments, with huge flaws in experimental design of which they are blissfully unaware.  Besides, for this particular history exercise, students don’t have nearly the background knowledge to make useful connections; they’ll never get into the deep structure, and will instead be stuck on surface features of the documents.

I’ve been wrestling with the idea that we should not challenge students to think like experts.

On the one hand, when I look at the classes I teach, I don’t generally challenge students to think like experts.  I’m not asking students to discover new mathematics in my math classes, and with good reason—I don’t think most discovery problems encourage reflective practice.

On the other hand, I don’t believe the 10,000 hours claim, because it fails to look at why someone would spend 10,000 hours playing chess.  They will probably only do so because they see some of the deep structure of the game, which inspires them to want to explore it further.  Someone’s decision to pursue a topic is a complicated interplay of factors.  To become an expert, you need to feel like you’re already on that track.  So if we want students to pursue expertise, we must challenge them sometimes to achieve it and give them experience with how an expert thinks.

Also, the science fair conclusion seems totally wrong to me.  Just because students do not do something well does not mean that there isn’t value in doing it!

Which leads me to conclude: challenge students to think like experts, but rarely and judiciously.  Option (3) is appropriate, but only after students have built up enough background knowledge of the underground railroad that they have connections and context to draw on.  If you can build up enough experience with that aspect of history that students can work with the documents, then they might be able to draw some new conclusions and learn from the experience.  But most of the class can’t be structured this way, or the basic connections students require to set documents in context will never have time to develop.

Yesterday, the New York Times had an article by Paul Tough about developing character in schools.  Citing the academic success of a middle school KIPP charter in NYC at achieving excellent test scores for its low-income students and getting them into selective private schools, the article then went on to show how still only 33% of the KIPP graduates got a college degree.  (Frightening fact: only 31% of middle school graduates nationally get college degrees, according to the article.)  KIPP founder Dave Levin’s solution: help his students develop character.

The article inadvertently demonstrates how education reform is often like a game of whack-a-mole.  We find a target to aim at and we focus our energies on it, only to see that there is another obstacle that pops up.  For example, the No Child Left Behind Act was instituted to guarantee minimum competency for all students regardless of background, but the nature of the tests used to guarantee this “proficiency” focused learning so tightly on a few specific subjects (and narrowly within those subjects) that schools left out important topics and skills students would later need.  Similarly, in the article, KIPP achieved real learning and proficiency on the part of their students, only to discover that the students didn’t have the grit to stick it through college.

Tough goes on to describe how the NYC KIPP schools have started teaching character.  Not “moral character,” but rather “performance character”: things like effort, diligence, and grit.  They’re whacking the next mole.

Unfortunately, I’ll bet that there will be yet more moles after this one.  For example, some of my own work deals with developing education-dependent adult identities for students to understand why they’re going to college.  Students also need a broader understanding of what opportunities exist for them in order to find the right path in college and afterward.

So how do you build a program that really helps students succeed?  It’s not easy to find effective methods.  Imagine trying to test out the effectiveness of this kind of character education.  There’s no test that will show immediate improvement just because some students have a bit more grit each year, so you need to do a longitudinal study (very hard and expensive!) to see how grit helps them in college and afterward.  It would take at least a decade.  And then, what if you made the mistake of doing your study on an ineffective school?  Students without sufficient knowledge and skills wouldn’t benefit nearly as much from character education, so we might conclude that this intervention doesn’t work at all, when in fact it does work in a school like KIPP that has a good grasp on the basics.

But let’s assume we got lucky and tested character education in the right setting.  We’d no doubt find many different interventions that are successful, and now we’d need to understand how these interventions work together.  A rigorous and controlled academic curriculum like KIPP’s might pair well with this kind of character education, while the effort might be wasted elsewhere.  It would take centuries to test all of these combinations.

Building a good program is hard.  (Kudos to KIPP for tracking its students, discovering their failures, analyzing them, and finding an underlying reason.  That takes incredible leadership.)  We can’t possibly test every combination of in-school methods.  Successful methods are often contradictory, or work against each other when implemented together.  Rigorous studies can often provide some clues as to what works, but we still need to trust ourselves to make good choices.  We still need to look at each program holistically and not let ourselves be so beholden to research that we can’t use our common sense to design a curriculum and a school.  Nor can we measure outcomes with a test at the end of the year and conclude that we’ve understood the impact on students.

In the end, this is a flaw with the slew of programs that are developing with laser-precision focus on specific outcomes.  Without thinking about all aspects of a child’s education, we’ll miss things.  The need to explain a program in a single sentence, to give a “statement of need” that explains what niche you fill, misses the real breadth that a successful program has.

We can and should measure outcomes, but we should also admit that our measurement is incomplete and trust our gut.  I really do wish that there was an easy way to find effective methods and combine them into a coherent program.  But because there isn’t, we will have to instead go back to basics: have good people at all levels of the education system and enable them to do good work that is guided by the research but not beholden to it.