For that, he's caught a great deal of flack. But, according to some recent research, there may indeed be differences in the way boys and girls learn.
Correspondent Elizabeth Kaledin reports, in the CBS News Sunday Morning cover story.
Ask any kid: Boys and girls are different.
"Girls are smarter," one girl says.
"Boys are better at sports," another girl suggests.
"Guys are talking about baseball or sports. Girls might be talking about guys, or whatever else," a boy observes.
But, asks CBS News Correspondent Elizabeth Kaledin, where do these differences come from? Are they hard-wired into the brain, there from the moment of birth? Or are they taught -- reinforced by a culture filled with stereotypical images: girls dreaming of that perfect doll and boys playing with trucks and guns?
At a recent conference on diversity, Harvard University's outspoken president, Lawrence
MIT biology professor Nancy Hopkins was so offended, she walked out: "When you send this message that maybe it's in your genes, you're asking women every day to prove over and over, 'Yes, I really am good enough to be here.' It's just exhausting."
"I could not believe what I was hearing, that women can't do science," another audience member says. "That is so surprising to me."
At Barnard College, an Ivy League all-women's school in New York -- a school where about one-third of graduates get science degrees – some young organic chemistry students were outraged.
"I'm kinda surprised he would say that and that people would still think that way," one female student told Kaledin. "The fact that he's president of Harvard, perpetuating snobby upper class male notions of women -- it's just not so."
You might say that Summers had fired a shot heard 'round the academic world. There were even calls for his resignation.
Summers' comments have reignited the age-old debate about genetics and gender, a debate that's been simmering for decades: Are male brains hard-wired in a way that's different than female brains? Can those differences be examined and mapped scientifically? And just what do those differences mean?
Women are well represented in certain scientific fields, such as medicine, Kaledin points out. Half the graduates of the nation's medical schools are women. But it's also true that in certain sciences, such as physics and engineering, women fill only a tiny fraction of all jobs.
"Somewhere along the road," Barnard President Judith Shapiro commented to Kaledin, "girls get turned off to science. …If we have stereotypes about what girls can and cannot do, they operate by having girls themselves believe it."
Put another way, says Shapiro, sure there's nature -- but don't forget about nurture: "I think we already know so much about the cultural factors and social factors in the way science is organized, that makes it (such) a problem for women, that to start focusing now on innate differences is really a diversion."
On the other hand, Kaledin notes, when it comes to nature, some scientists studying human brain structures think Summers might just have a point.
At the University of California at Irvine, psychologist Richard Haire's findings show striking differences in the way male and female brains work.
"Can these differences be related to other cognitive or mental differences, like mathematical ability?" Haire asks. "It's not much of a stretch."
In men, Haire found, the smarter they were, the more grey matter they had, and used, to solve problems, while in women, the white matter connecting different parts of the brain does most of the work.
What does that mean? Haire says it's just too soon to be certain: "Should women be worried? …(For them), there seems to be less brain involved with intelligence. …(Women I've studied) are just as intelligent as these men, but they seem to accomplish it with even less brain tissue, so this suggests that there's something about women's brains that's more efficient. So more is not necessarily better."
At the National Institutes of Health, in Bethesda, Md., Dr. Jay Geidd has been mapping the development of the adolescent brain for 13 years.
Using the most advanced MRI imaging processes to peer into the brains of twins, such as 15-year-old Kimberly and Allison Hast, and their 18-year-old brother Eric. Geidd is in search of answers to some of the age-old questions about boys and girls -- and intellectual development.
"The male brain is about 10 percent larger than the female brain," Geidd says, "and if you adjust for that size difference, things are pretty much in proportion. …I can't tell a boy brain from a girl brain from looking at it. I don't think most people could."
So far, his 4000 scans of more than 2000 brains haven't begun to answer the big questions about nature versus nurture. He has, however, mapped different development paths in the brains of girls and boys.
"In general," Geidd continues, "we think the girls' brains are maturing a bit faster than the boys' brains."
But not in all areas.
"Certain parts of the brain involved in mechanical skills or projectile estimations actually mature somewhat faster in boys," Geidd says.
What does that mean in terms of the ways kids learn?
Dr. Leonard Sax, a psychologist and family practitioner, says it means it's important to pay more attention to the differences between the way boys learn and girls learn -- the differences in the ways they develop.
"Both boys and girls are being shortchanged as a result of the neglect of hard-wired gender differences," Sax asserts. "By the age of 12, the geometry area of a girl's brain looks like an eight-year-old boy's brain: They're four years behind. "The language area of a boy's brain is three-to-four years behind the language areas of a girl's brain.
"Girls and boys differ profoundly in how they hear, how they see, how they respond to stress --and those differences are present at birth."
In fact, in his book, "Why Gender Matters," Dr. Sax argues that to ignore gender differences puts us at risk of perpetuating them.
"If you teach the same subjects in the same way to girls and boys -- in other words, if you ignore gender, by the age of 13, you're gonna have girls who think they can't do geometry, and boys who hate to read and write and think poetry is a waste of time," Sax remarks.
But, he cautions, those differences have nothing to do with the innate ability to do science.
"The fact that a boy prefers a truck over a doll doesn't mean he's gonna be a better physicist than the girl who just wants to stare at something interesting, whether it be a truck or a doll," Sax says, "because physics is not necessarily about trucks accelerating or things exploding. That's just the way you were taught physics. Physics is about contemplating the universe."
"You go to any bookstore today and buy a high school physics book and open it up and what do you see? You'll see a rifle shooting a bullet and calculate the acceleration of the bullet, three seconds after it is fired. You'll see a baseball hitting a ball, a football player kicking a football. …It's saying (to women), 'You don't belong here.' "
In Rochelle Barry's 7th grade class in Ardsley, N.Y., both boys and girls are enthusiastic about their science projects. The girls, like 12 year-old Nica Siegel, whose parents are scientists, don't seem to see any differences with boys at all.
Asked by Kaledin whether she's felt discouraged in any way to pursue math and science, Siegel responded, "No, I really haven't. You know, in liking science and math -- I just enjoy it."
But, Kaledin says, the boys think they might have a slight edge in science, and they think they know why.
"Why do you think there aren't as many women in science as there are men today," Kaledin asked them.
"I guess some women don't like to be doing the science stuff," one boy answered.
"I think it's like -- women used to discouraged from jobs like that, but it's changing," another said.
"People were stereotyped. They believe that men are better than women."
"There's no doubt there are differences between boys and girls, Kaledin says. "Science is bearing that out. But the consensus seems to be that the reason there aren't that many women working in science has much more to do with attitudes, than aptitudes."
And, says Barnard's Shapiro, 'twas ever thus: "I am a great believer in science, and I think science is one of the greatest of human achievements. But if you look back in the history of scientific studies of sex differences, it can get very embarrassing.
"I mean, at the time when women's colleges like Barnard were being founded, there were prominent male scientists and doctors, saying that if women took on challenging courses in academic study, their reproductive organs would shrivel up and they would not be able to have babies," she laughed.