Can students think like experts?

Which of the following makes for a good history lesson about the underground railroad?

  1. A traditional lesson in which the background of the underground railroad is presented, major players in the story are introduced, an explanation is given of the politics of the time, and the conditions that escaped slaves went through to get to freedom are presented.
  2. For a fourth-grade classroom, have the students bake biscuits, which were a common staple on the underground railroad, in order to gain an understanding of what runaway slaves did and ate.
  3. Give tenth-grade students original documents from the underground railroad that they use to draw and debate conclusions about the underground railroad (like real historians do).
  4. Run a class discussion in which students are asked questions to elicit their understanding of life within the underground railroad.  How did runaway slaves obtain food?  How were they able to prepare it?  How were they able to pay for it?  What might the consequences have been for someone who housed runaway slaves?  And so forth.

Easy answers: (1) sounds fine if you need to transmit lots of knowledge, but boring.  A good lecture will communicate facts as well as connections between those facts, and it will build important neural connections to help students remember and process data.

Option (2) is nonsense.  The students will spend their time thinking about measuring flour, not the underground railroad.  (Math teachers, take note—this happens in your classes, too!)

Option (4) is great.  It prompts students to reflect on what they have learned and it leads them to make connections between topics.  By giving each of these items real thought and relating it to other facts they might know about the time period, their overall understanding will be greatly strengthened.

In his fascinating book Why Don’t Students Like School?, Daniel Willingham argues that option (3) is a mistake unless your goal is building student excitement.  But… surely not, you say!  Is this not what real historians do?  If history class was about analyzing real documents, would that not more deeply engage students’ minds and show them what the field was really like?

Not so, says Willingham.  This kind of exercise is asking students to think like experts.  However, he says, the cognition of experts is fundamentally different from non-experts.  Here’s how:

  • Experts think in terms of functions or deep structure, while novices think in terms of surface features.  For example, suppose you ask novices and expert chess players to memorize a chess board.  When replacing the pieces on the board, the novices will put them down in clumps organized by location on the board.  Expert players will put them down in clumps organized by function—pieces that are threatening each other will go down all at once, even if on different ends of the board.
  • Experts’ focus on deep structure allows them to ignore unimportant details and immediately see the useful information.
  • Experts have procedures that are essentially automatic from repeated practice, so they can execute those procedures/thought patterns without the mental overhead of novices.

Willingham then notes that becoming an expert takes 10,000(ish) hours of practice.  He reflects on judging science fairs where students churn out lots of basically useless science experiments, with huge flaws in experimental design of which they are blissfully unaware.  Besides, for this particular history exercise, students don’t have nearly the background knowledge to make useful connections; they’ll never get into the deep structure, and will instead be stuck on surface features of the documents.

I’ve been wrestling with the idea that we should not challenge students to think like experts.

On the one hand, when I look at the classes I teach, I don’t generally challenge students to think like experts.  I’m not asking students to discover new mathematics in my math classes, and with good reason—I don’t think most discovery problems encourage reflective practice.

On the other hand, I don’t believe the 10,000 hours claim, because it fails to look at why someone would spend 10,000 hours playing chess.  They will probably only do so because they see some of the deep structure of the game, which inspires them to want to explore it further.  Someone’s decision to pursue a topic is a complicated interplay of factors.  To become an expert, you need to feel like you’re already on that track.  So if we want students to pursue expertise, we must challenge them sometimes to achieve it and give them experience with how an expert thinks.

Also, the science fair conclusion seems totally wrong to me.  Just because students do not do something well does not mean that there isn’t value in doing it!

Which leads me to conclude: challenge students to think like experts, but rarely and judiciously.  Option (3) is appropriate, but only after students have built up enough background knowledge of the underground railroad that they have connections and context to draw on.  If you can build up enough experience with that aspect of history that students can work with the documents, then they might be able to draw some new conclusions and learn from the experience.  But most of the class can’t be structured this way, or the basic connections students require to set documents in context will never have time to develop.

Knowledge, Character, and the Next Mole

Yesterday, the New York Times had an article by Paul Tough about developing character in schools.  Citing the academic success of a middle school KIPP charter in NYC at achieving excellent test scores for its low-income students and getting them into selective private schools, the article then went on to show how still only 33% of the KIPP graduates got a college degree.  (Frightening fact: only 31% of middle school graduates nationally get college degrees, according to the article.)  KIPP founder Dave Levin’s solution: help his students develop character.

The article inadvertently demonstrates how education reform is often like a game of whack-a-mole.  We find a target to aim at and we focus our energies on it, only to see that there is another obstacle that pops up.  For example, the No Child Left Behind Act was instituted to guarantee minimum competency for all students regardless of background, but the nature of the tests used to guarantee this “proficiency” focused learning so tightly on a few specific subjects (and narrowly within those subjects) that schools left out important topics and skills students would later need.  Similarly, in the article, KIPP achieved real learning and proficiency on the part of their students, only to discover that the students didn’t have the grit to stick it through college.

Tough goes on to describe how the NYC KIPP schools have started teaching character.  Not “moral character,” but rather “performance character”: things like effort, diligence, and grit.  They’re whacking the next mole.

Unfortunately, I’ll bet that there will be yet more moles after this one.  For example, some of my own work deals with developing education-dependent adult identities for students to understand why they’re going to college.  Students also need a broader understanding of what opportunities exist for them in order to find the right path in college and afterward.

So how do you build a program that really helps students succeed?  It’s not easy to find effective methods.  Imagine trying to test out the effectiveness of this kind of character education.  There’s no test that will show immediate improvement just because some students have a bit more grit each year, so you need to do a longitudinal study (very hard and expensive!) to see how grit helps them in college and afterward.  It would take at least a decade.  And then, what if you made the mistake of doing your study on an ineffective school?  Students without sufficient knowledge and skills wouldn’t benefit nearly as much from character education, so we might conclude that this intervention doesn’t work at all, when in fact it does work in a school like KIPP that has a good grasp on the basics.

But let’s assume we got lucky and tested character education in the right setting.  We’d no doubt find many different interventions that are successful, and now we’d need to understand how these interventions work together.  A rigorous and controlled academic curriculum like KIPP’s might pair well with this kind of character education, while the effort might be wasted elsewhere.  It would take centuries to test all of these combinations.

Building a good program is hard.  (Kudos to KIPP for tracking its students, discovering their failures, analyzing them, and finding an underlying reason.  That takes incredible leadership.)  We can’t possibly test every combination of in-school methods.  Successful methods are often contradictory, or work against each other when implemented together.  Rigorous studies can often provide some clues as to what works, but we still need to trust ourselves to make good choices.  We still need to look at each program holistically and not let ourselves be so beholden to research that we can’t use our common sense to design a curriculum and a school.  Nor can we measure outcomes with a test at the end of the year and conclude that we’ve understood the impact on students.

In the end, this is a flaw with the slew of programs that are developing with laser-precision focus on specific outcomes.  Without thinking about all aspects of a child’s education, we’ll miss things.  The need to explain a program in a single sentence, to give a “statement of need” that explains what niche you fill, misses the real breadth that a successful program has.

We can and should measure outcomes, but we should also admit that our measurement is incomplete and trust our gut.  I really do wish that there was an easy way to find effective methods and combine them into a coherent program.  But because there isn’t, we will have to instead go back to basics: have good people at all levels of the education system and enable them to do good work that is guided by the research but not beholden to it.

Math Literacy vs. Math Workforce Development

Here are two complaints often heard in any math department.  The first is that people do not understand any mathematics; this complaint is typically inspired by some horrendous misuse of statistics or basic misunderstanding about numbers.  The second complaint is about students who have generally succeeded at basic mathematics but who do not truly understand the advanced tools that they’re using.  These complaints are about very different aspects of math education, and we need to keep them straight.

Ok, secret educator speak decoding time!

  • “math literacy” == understanding basic mathematics to get through life and interpret your world
  • “workforce development” == preparing people for jobs in STEM fields (Science, Technology, Engineering, and Math)

People often don’t give enough thoughts to their goals.  Say, for example, that you’re debating if calculus should be the pinnacle of high-school learning.  If you think the purpose of math education is math literacy, then it clearly shouldn’t be: statistics would be much more valuable.  If you think the purpose of math education is workforce development, then calculus probably should be the goal: you need calculus for engineering, for math research, for any kind of science, and without it you’re sunk.  (Leaving aside other goals such as exposing students to beauty or to deeper thinking, which might culminate a discrete math course.)

Schools basically push everyone to a workforce development track until they get to college.  Even assuming that this track adequately prepares people to study STEM fields (it doesn’t, judging from most incoming college freshmen), it’s pretty bad for anyone who doesn’t want to pursue a STEM field: they don’t get a lot of the skills they need for life (interpreting mortgages, research studies, and politicians) and they’ve learned to thoroughly hate math.

Yet we shouldn’t put everyone into a math literacy track, either, because it’s quite helpful to build up good abstract thinking skills early-on if you do plan to pursue a STEM career.

Naturally, you think, we should just let students choose or perhaps sort them somehow into different tracks.  But this opens up its own host of problems.  For example, a student might actually be quite good at math but lack the self-motivation to pursue it seriously, so it is a disservice not to push them harder.  There are students who develop mathematical skill later in life, but it is very hard to catch up in the workforce development track if you haven’t been given the opportunity to learn advanced algebra or trigonometry.  Let alone that low-income and minority students will track disproportionately into math literacy, and you’ll also create student self-images of being “not good” at math in the literacy track.

Which brings me to the recent New York Times op-ed, How to Fix Our Math Education.  This is probably the best proposal I’ve seen for addressing the situation, but it’s still not good enough.  The authors come up with credible ideas for courses that would demonstrate the connection between the “real world” and mathematics.  However, I don’t believe that the resulting courses would cover enough for workforce development.  Also, the plan feels too much like a retreat: since we can’t actually get most students to deeply understand calculus, let’s find a way to learn topics that don’t involve calculus.  But we know that calculus is possible from the results of other countries!

I do not know how to resolve this.  I don’t know how to make calculus and advanced, abstract mathematics accessible to those who are ready for it without the collateral damage of convincing other people (many of whom could excel in mathematics) that they are not good at mathematics.  Any ideas?

Are Math Competitions Good or Bad?

Mathbabe wrote a very interesting two posts about math competitions and the harm they do.  In summary, her argument gives three negatives to competitions:

  • Math competitions discourage most participants because low scorers conclude they are not “good” at math.
  • Those who do well at math competitions get an inaccurate picture of success, only to be stymied in (for example) grad school, which requires more sustained attention, gives less instant feedback, and does not always have a “right” answer.
  • Competitions particularly discourage girls, who in general are not as “into” competitions and are more susceptible to feeling mathematically inferior.

Her posts are interesting and provocative (just see the comments thread!).  Yet there are also numerous advantages to competition.

Perhaps the biggest advantage is that competitions are an incredibly scalable model: a single competition can involve tens of thousands of participants at low cost.  Good competition problems and preparation books offer a solid curriculum for math team coaches across the country (even those who are not very good at math) and hence create mathematical communities where they could not otherwise be.

There are other advantages as well.  Competition really does appeal to something primal in young boys, and so it can get them into doing math when other techniques won’t work.  (Is this infrastructure why boys are still good at math when they fall behind in many other subjects?)  In fact, being on a math team can help raise your social standing relative to just being a generic math geek—it provides a cover similar to being on a sports team, if somewhat less revered.  Competition math also spurs the creation of lots of good math problems(*) that help students to reflect on deeper mathematics.

(*) Some argue that competition problems are not good problems, but that they are forced and arbitrary exercises that do not arise in nature.  I see the point, but I still find them intellectually stimulating.

I believe that competition math is valuable up to a point.  I think that intense studying for high-level performance is generally not very useful; it’s not learning real math (just competition strategies) and it exposes competition problems as arbitrary.  But for someone who has not spent hours upon hours studying, competition problems are engaging and a good way to challenge yourself.

However, mathbabe’s arguments point to two very important changes that we should bring about in the pathways towards studying mathematics.

  1. We must create more alternate ways to become mathematically successful.  Math circles are a start, but they still rely on a strong mathematician/educator in the area.  Some specific, scalable ideas:
    • Can we harness the internet to create better opportunities with video lectures and online communities?  Art of Problem Solving does a great job at this (see, e.g., Alcumus) although still with competitions as the main focus.
    • Instead of timed competitions, can we distribute deep problems for students to unravel over the course of a month, submit, and receive feedback without grading?  The best work could be published, sufficient reward in itself.
    • Can we develop deep mathematical curricula for the formation of “math clubs” that could be used out of the box by motivated students or teachers?
    • Can we expand math summer programs and create more national opportunities for students?  Successfully doing this also requires new selection mechanisms, as right now most summer program quizzes are competition-like problems.
  2. We must change the culture around math competitions.  We should emphasize the joy of the problems over numerical success and treat competitions for what they are—fun, stimulating, but arbitrary collections of problems.  Students should understand that competition math is an inaccurate reading of their chances at becoming a real mathematician.  In particular, they should understand that studying hard and doing well at the top levels of competition math is like training hard to be a good sprinter—you’re great at running 100 meters, but it doesn’t help for the marathon.  There is nothing wrong with that and the competitive spirit should be encouraged for those who want it.  However, it should not be forced on those who don’t.

I feel that throwing out competitions is throwing out the baby with the bathwater, and mathbabe notes much the same towards the end of her second post.  But building a new infrastructure for mathematical success is not easy, and will require some real initiative on our parts to make it happen.

Specialization in the Summer: the Long Tail

The long tail has been frequently discussed with respect to online companies.  The idea is simple: yes, there are lots of very popular things (like, say, Lady Gaga or Justin Bieber), but there are actually a lot of people with specialized tastes (for example, computer nerds like Jonathan Coulton).  The internet enables those specialized interests to take root: on the local level, fans are isolated, but combine people nationally or internationally and you can get big business.  Amazon and Netflix are often credited for having taken advantage of this: Amazon because it can sell lots of books that local bookstores can’t carry, and Netflix because it can have lots of special-interest DVDs.  The online program Art of Problem Solving, for students with talent and interest in mathematics, is quite similar: it creates a mathematical community that can’t exist locally.  Other forums, Twitter, Facebook, LinkedIn, etc. do the same for other fields.  And magazines with national distribution fill the same niches, even moreso prior to the internet.

Summer programs that draw statewide or nationally can provide much of the same advantage.  Programs for students with interest in particular academic fields (from mathematics to writing to astronomy) allow communities to form based on these common interests.  Without the inter-school draw, there would not be enough students to create these communities.  (The same argument can be made for programs for students with learning disabilities, for sports, etc.)

My view is that these opportunities are crucial to developing young learners.  There should be systemic support for exploring your interests.  Schools alone cannot offer this support, because they don’t have the staff or the size to take advantage of the long tail.  Hence, we must have stronger support for summer programs that can bring students with common interests together, and also for programs like Splash that draw across schools, where the variety encourages students to find their part of the long tail.  (I also admire Citizen Schools for the same reasons: they encourage students to deeply pursue new interests, although because they are within a single school it’s not always the case that everyone in a project is deeply invested.)

Which brings me to the discussion recently about “expanded learning time” (ELT).  The discussion is here and here, and the participants quite nicely talk right past each other.  In a bid to share some of the contradiction myself, I am both a strong supporter and opponent of ELT.

I am a strong supporter when ELT means exploring and developing new interests.  But I am not a supporter when it means, simply, more time learning the way you do in school.  I don’t believe that additional time addresses the root problem that students do not self-identify as scholars, and it is a tremendous missed opportunity for students who cannot explore their part of the long tail.

While I can appreciate arguments that students should not choose their in-school curricula (because there are basic things that they should know), I firmly believe that ELT should center around choice and following your interests.  Summers, weekends, and after-school programs provide fantastic opportunities to do so.  We should make it expected that students pursue these opportunities.

Stereotypes and Self-Image

This talk from Joshua Aronson is simply outstanding:

I recall a graduate school course where a guest speaker was discussing the importance of opportunity to learn.  This self-explanatory idea can easily be forgotten in debates about education.  However, contrary to the speaker’s perspective, opportunity to learn is not enough.  Students must also have the drive to learn.

To understand the importance of having drive to learn, consider your own life as you balance different things that you “want” to do.  For example, I want to learn German.  However, I do not want to learn German more than I want to help Learning Unlimited succeed, more than I want to read the new George R.R. Martin book, or more than I want to maintain a blog.  Hence, I do not learn German.  Similarly, all students want to learn and want to go to college—but perhaps not more than they want to hang out with friends, monitor their social status, or earn some extra money.  That’s why wanting to learn or go to college is not enough.  You have to be driven to learn; your self-identity must be that of someone who learns, a scholar, in order to make it a priority for you.

Although Aronson’s talk is about “stereotype threat”, the impact of students’ self-identity as it is reinforced by stereotypes of their groups (gender, race, etc.), one conclusion to draw is the importance of self-identity to academic success.

In case you didn’t watch his talk, here is just some of the evidence he cites:

  • Fear has a big impact on intelligence.  “In the two weeks following a homicide in the South Side of Chicago, the IQ scores of the black children in a pretty large radius around the homicide will suffer a half a standard deviation decrement” (time index 15:30).  [Aronson goes on to talk about “threatened belongingness” having the same effect, showing how important social life is to learning.]
  • You probably already know that if you’re asked to indicate your race prior to a test and you’re African American or Latino, then your score goes down.  On math tests, Asians asked about their race see their scores go up (but their English scores go down).  Women asked about their gender on math tests see their scores go down, men see their scores go up.  Older people asked about their age prior to a test of memory see their scores go down.  (Back to the math scores for a moment.  Because the AP calculus exam asks for gender before the exam, some 5000 women do not pass who would otherwise have passed — time index 36:30.)
  • What may be more surprising is the same effect for miniature golf.  If, prior to playing miniature golf, the group is told that it is a measure of “kinesthetic intelligence” (emphasis mine), then you get a better score if you’re white and a worse score if you’re black.  In contrast, if you’re told that it’s a measure of “natural athletic ability”, then this reverses and black participants see their performance go up!  (Time index 26:30.)
  • White male engineering students from Stanford were given a test.  In one group, they were told that the study was on “math ability — what makes some people smarter at math than others”.  In the other group, they were told that the study was about why “Asians are good at math”.  This second group scored 30% less well.  This is a full standard deviation, the size of the black-white gap.  (Time index 29:40.)
  • Let’s move on to solutions.  “Students who are high self-monitors are much less likely to experience stereotype threat.”  “People who are high self-monitors are really good at thinking about the situation they’re in and considering the consequences, and the stereotype threat just sort of bounces right off of them.”  (Time index 35:45.)  Teaching self-monitoring might get around the stereotype threat!
  • Asking students questions that make them think about membership in intellectual communities changes their scores.  For example, reminding students that they go to a selective college before the test (via the question “Can you tell us a few things a student might like about being a student at a highly-selective liberal arts college?”) nearly eliminates the gender gap.  (Time index 52:40.)  In other words, building a positive community membership can raise students’ achievement.
  • Finally, if you listen to nothing else, I highly encourage you to listen to the description of Crellin Elementary School at time index 56:00.  It is a school that has truly managed to give students an intellectual identity, and to create an intellectual community.  In three years, the principal turned the school around from 0% to 100% proficiency on the state exam, with 50% reaching an advanced level.  Seven years after leaving the school, its students form 75% of the AP courses at the high school, despite the extreme poverty and tough home conditions of the students.  I won’t try to describe what the principal does.  You should hear it yourself.

I have been appreciating more and more the importance of students’ self-identity.  Do they see themselves as scholars?  Do they see themselves as people who enjoy learning?  (This is a critically different question from “do they enjoy learning?”)  I’m sure that it matters not just to immediate scores on tests, but also to learning over time.

How much stronger would students be with better self-identities?  How much would their performance improve if we gave them the ability to self-monitor and thereby get around their stereotypes?  One thing is clear: we must design our programs to build students’ self-identity as scholars, and we must do so as early in their lives as possible.

Update 02/29/12: A recent meta analysis calls stereotype threat into question, although there are claims that the meta analysis is, itself, biased.  The meta analysis still found effects among top performers, and was unclear on effects for average performers.

The Power of Intellectual Communities

The most powerful experiences of my life have come from membership in intellectual communities.  We draw our identities as people from community membership, and being a part of a community that prides itself on learning inspires us to strive for learning in our lives.  I have been privileged to be a part of intellectual communities at MIT, Canada/USA Mathcamp, Splash, CTY, and SPMPS, among many others.  I have modeled my communities in life after the communities I grew up in, seeking friends who value intellectual achievement as I do.

When I design a new educational program, I give great thought to shaping the community.  A carefully-designed community will turn students’ focus to academics and make them celebrate their growth.  Community should be a conscious part of designing any place of learning.

But how?  For this inaugural blog post, I would like to share my developing thoughts on how to create a successful intellectual community.  I believe that the key is not to make the community yourself but to drive students towards creating an intellectual community themselves.

  • Celebrate learning and knowledge, not achievement.  If the primary focus is on test scores, class rankings, or even going to college, then learning is not valued.  If the focus is on learning, the usefulness of the material, or the beauty of the material, then it will be valued by the community.
  • Make students feel the value of being here.  For most of the programs above, this was easy: the programs are selective, and so students feel “special” for making it in.  However, any program can make learning feel like something you are privileged to be able to do in this place, and students will in turn value the experience and create a community around that valuation.
  • Design the space to center on learning.  MIT lines its corridors with robots, posters, and labs.  Math departments everywhere have mathematical sculptures and posters.  At the Summer Program in Mathematical Problem Solving, I made sure that books were prominent along the walls.
  • Give students choices in what they learn, so that they take ownership over it and want to share it with their peers.
  • Show passion over the subject.  Give students role models who love learning.  Do not be afraid to digress from the material (within reason); that digression demonstrates love of the topic.
  • Treat each student question as sacred.
  • Make teachers accessible for out-of-classroom discussion, and create opportunities for a vibrant intellectual life to take root outside of class.
  • Humor and weirdness stemming from academics make those academics a part of the culture.
Of course, none of these bullet points are easy.  How do you drive students to create an intellectual community in your places of learning?