Knowing what we don’t know

Courtesy of Alan Jacobs, here is a terrific article by Errol Morris about a fundamental but largely ignored truth: there is a critical difference between knowing that you don’t know something and not knowing that you don’t know something.

Morris came across a paper with a wonderful title, “Unskilled and Unaware of It: How Difficulties of Recognizing One’s Own Incompetence Lead to Inflated Self-assessments.” It was written by two psychologists, David Dunning and Justin Kruger [emphasis added throughout]:

Dunning and Kruger argued in their paper, “When people are incompetent in the strategies they adopt to achieve success and satisfaction, they suffer a dual burden: Not only do they reach erroneous conclusions and make unfortunate choices, but their incompetence robs them of the ability to realize it.  Instead, they are left with the erroneous impression they are doing just fine.”

It became known as the Dunning-Kruger Effect — our incompetence masks our ability to recognize our incompetence.

Morris contacted Dunning to ask him a few questions:

DAVID DUNNING:  Well, my specialty is decision-making.  How well do people make the decisions they have to make in life?  And I became very interested in judgments about the self, simply because, well, people tend to say things, whether it be in everyday life or in the lab, that just couldn’t possibly be true.  And I became fascinated with that.  Not just that people said these positive things about themselves, but they really, really believed them.  Which led to my observation: if you’re incompetent, you can’t know you’re incompetent.


DAVID DUNNING:  If you knew it, you’d say, “Wait a minute.  The decision I just made does not make much sense.  I had better go and get some independent advice.”   But when you’re incompetent, the skills you need to produce a right answer are exactly the skills you need to recognize what a right answer is.  In logical reasoning, in parenting, in management, problem solving, the skills you use to produce the right answer are exactly the same skills you use to evaluate the answer.  And so we went on to see if this could possibly be true in many other areas.  And to our astonishment, it was very, very true.

ERROL MORRIS: Many other areas?

DAVID DUNNING: If you look at our 1999 article, we measured skills where we had the right answers.  Grammar, logic.  And our test-subjects were all college students doing college student-type things.  Presumably, they also should know whether or not they’re getting the right answers.  And yet, we had these students who were doing badly in grammar, who didn’t know they were doing badly in grammar.  We believed that they should know they were doing badly, and when they didn’t, that really surprised us.

ERROL MORRIS:  The students that were unaware they were doing badly — in what sense?  Were they truly oblivious? Were they self-deceived?  Were they in denial?  How would you describe it?

DAVID DUNNING:  There have been many psychological studies that tell us what we see and what we hear is shaped by our preferences, our wishes, our fears, our desires and so forth.  We literally see the world the way we want to see it.  But the Dunning-Kruger effect suggests that there is a problem beyond that.  Even if you are just the most honest, impartial person that you could be, you would still have a problem — namely, when your knowledge or expertise is imperfect, you really don’t know it.  Left to your own devices, you just don’t know it.   We’re not very good at knowing what we don’t know.

ERROL MORRIS:  Knowing what you don’t know?  Is this supposedly the hallmark of an intelligent person?

DAVID DUNNING:  That’s absolutely right.  It’s knowing that there are things you don’t know that you don’t know.

At the risk of being pedantic, let me point out that at the end of the interview Dunning says two important things which he doesn’t distinguish very well. First, he points out that it is very difficult to know exactly where the gaps lie in your knowledge; put another way, it is difficult for us to keep track of which of our operating assumptions are grounded in knowledge and which are based on sheer guesswork, received wisdom, wishful thinking, and so on.

However, when Morris asks him if such an ability is the hallmark of an intelligent person, in agreeing Dunning actually puts forward a different hallmark: an awareness that there are gaps in your knowledge. It’s not enough to be good at keeping track of what you don’t know. You have to be humble enough to acknowledge that gaps are the default state, that what you actually know is vastly outweighed by what you don’t know. And you have to be vigilant about staying clear on what fits into which category.

I think this is more or less what Socrates meant when he said, “The only true wisdom is in knowing that you know nothing.” The statement by itself is misleading, but the context makes it clear. Since I’ve written about this before, I’ll resort to quoting myself:

Part of the accusation against Socrates was that he purported to be wise, and was misleading the youth of Athens and otherwise raising a ruckus via his wisdom. His response was that in fact he had no wisdom, and that the rumor that he considered himself wise was baseless.

He went on to trace the source of the rumor to a prophecy by the oracle at Delphi, who once proclaimed that Socrates was the wisest of all men. Socrates was astonished, knowing he had no wisdom, and alarmed, not wanting such a reputation. He figured the quickest way to disprove the prophecy was to find someone wiser than he was. The search for such a person didn’t go well, though; each time he found folks who had reputations for being wise—politicians, philosophers, poets, artisans—Socrates’ questioning of him revealed that in fact there was no wisdom there at all. (Which didn’t please the politicians, philosophers, poets, and artisans, turning them into enemies who floated calumnies against him, leading to his false reputation for wisdom.)

The puzzle remained: was the oracle correct? Giving the oracle the benefit of the doubt, Socrates finally understood the prophecy to mean that he was the wisest of men simply because he, unlike the rest, understood that he had no wisdom at all.

In saying that he has no wisdom at all, I think that Socrates is using excessive modesty as a rhetorical device; clearly he thought he had wisdom to share with others. But his wisdom is actual wisdom because he has done the hard work of identifying and junking the baseless assumptions that others pass off as wisdom, thereby making room for the truth.

I will often challenge my children to defend a claim they make, especially an offhanded one, not because the claim is false—it may be true—but because I don’t think they know it to be true. And even so I am not insisting that they speak the truth, but that they speak carefully. I want them to develop the awareness that Dunning speaks of, as well as the humility and consideration needed to keep silent when one has nothing useful to say.

Generalize, dehumanize, marginalize

I’m reading a very good short book by John Carey, The Intellectuals and the Masses. It touches on a subject that greatly interests me, namely which blessings are available to the average person, and which are not. Literature, for example—clearly not all of what passes as literature is going to be accessible to everyone, or even potentially accessible. Which leaves me wondering, are some joys just beyond the reach of most people, or do we make an important mistake when we assume that such things are joys?

I like a writer who is upfront about his intentions. Carey introduces his book as follows:

This book is about the response of the English literary intelligentsia to the new phenomenon of mass culture. It argues that modernist literature and art can be seen as a hostile reaction to the unprecedentedly large reading public created by late nineteenth century educational reforms. The purpose of modernist writing, it suggests, was to exclude these newly educated (or ‘semi-educated’) readers, and so to preserve the intellectual’s seclusion from the ‘mass’.

Sometimes changing circumstances can shine a bright light into the dark recesses of our hearts. As long as access to “the best that has ever been thought or said” was limited by practicalities, intellectuals could view their special social status (and the lifestyle it required) as noblesse oblige. But once the public became literate, that status had to suddenly be defended; there’s not much special about a club that just anyone can join.

The end point of this attitude, of course, is the old Groucho Marx quip: “I refuse to join any club that will have me as a member!” Most of us are not so secure in our conviction that we are right and the rest of the world is wrong, so we like to have at least a few others in our club for reassurance (and for purposes of mutual admiration). But there must be a gate, and it must be manned vigilantly.

As an element in the reaction against mass values the intellectuals brought into being the theory of the avant-garde, according to which the mass is, in art and in literature, always wrong. What is truly meritorious in art is seen as the prerogative of a minority, the intellectuals, and the significance of this minority is reckoned to be directly proportionate to its ability to outrage and puzzle the mass. Though it usually purports to be progressive, the avant-garde is consequently always reactionary. That is, it seeks to take literacy and culture away from the masses, and so to counteract the progressive intentions of democratic educational reform.

What has surprised me in reading Carey’s book is how transparently contemptuous the intellectuals were of the newly educated public, what they referred to as the masses. Writers we all admire (or think we admire) wrote some pretty vile things about these folks. And Carey claims that it was made possible only by conjuring up a notion of “the masses” which had no basis in reality.

The ‘mass’ is, of course, a fiction. Its function, as a linguistic device, is to eliminate the human status of the majority of people—or, at any rate, to deprive them of those distinctive features that make users of the term, in their own esteem, superior.

Intellectuals aside, isn’t this a fairly accurate description of the rhetorical strategy of our age? We are quick to seize on any group characteristic that doesn’t include us, whether it be racism or environmental wackoism or climate change denialism or easy believism or fundamentalism or feminism or capitalism or socialism or partriachalism or egalitarianism or clericalism or individualism. We apply the name to ‘people’ who don’t actually have names or faces or histories, who we have no personal acquaintance with. And then we proceed to point out to those inside the gates how ridiculous these ‘people’ are, how absurd, how irrational, how lacking in character, how … inferior they are to us.

Carey makes a telling observation about the idea of the masses.

Its usage seems to have been originally neither cultural nor political but religious. St Augustine writes of a massa damnata or massa perditionis (condemned mass, mass of perdition), by which he means the whole human race, with the exception of those elect individuals whom God has inexplicably decided to save. Even in modern times, the belief that God is implicated in the condemnation of the mass lingers on among intellectuals […]

Outer darkness comes in many forms, and there is no small pleasure to be had in consigning others to it, since it reinforces the idea that God has very strict standards for choosing those He will love—and it’s sweet to have made the cut.

Lately I’ve been exploring the opposite attitude, namely that if it is something that separates me from a (potential) brother—and who isn’t a potential brother?—then it is adiaphora, neither mandated nor forbidden. I don’t want to deal with the difficult cases by dehumanizing them in order to consign them to outer darkness, I want to confront them in their rich, complex, confusing, mixed-bag humanity.

Aleksandr Solzhenitsyn once wrote this:

Gradually it was disclosed to me that the line separating good and evil passes not through states, nor between classes, nor between political parties either, but right through every human heart, and through all human hearts. This line shifts. Inside us, it oscillates with the years. Even within hearts overwhelmed by evil, one small bridgehead of good is retained; and even in the best of all hearts, there remains a small corner of evil.

It reminds me of a friend I had once, a very kind and sweet older woman who along with me attended the most liberal Episcopal church in Austin, the most liberal city in Texas. She was a dedicated parishioner, participating fully in the life of the church. At one retreat we both attended, somehow the conversation turned to overpopulation, and she told me that even though it was pretty tough-minded she could see the wisdom in China’s one-child policy. In that moment it was very, very important to me that we were sitting across a picnic table from one another, that I could see the thoughtfulness in her face, that I had a history with her and knew her to be kind and sweet and very serious about her faith—because it forced me to wrestle truthfully with the things in our thinking that separated us, rather than simply write her off as an abstraction.

Why I don’t write much

If you like to think about the acts of reading and writing, you really ought to follow the Text Patterns weblog by Alan Jacobs. He doesn’t write too often or at length, but over the past year I’ve benefited from reading this weblog more than any other; I’ve read several of the books Jacobs recommended, and started looking into things that wouldn’t have otherwise shown up on my radar screen.

Today he points to an excellent short essay on writing by Jed Perl, which considers the question of whether everything which is written ought to be read. First, Perl notes that the value of a piece of writing is not directly correlated with its success in some marketplace:

But the speed with which words, once written, are now being read … has set me to thinking about the extent to which writing, for the writer, ought to have a freestanding value, a value apart from the reader. There is too much talk about the literary marketplace, the cultural marketplace, and the marketplace of ideas. We need to remember that a book—or a painting or a piece of music—begins as the product of an individual imagination, and can retain its power even when largely or even entirely ignored.

I can certainly vouch for this as a reader. If I were to list the fifty pieces of writing that have had the greatest impact on my thinking, forty of them would be works that are largely forgotten today, and some were never valued at all in the marketplace of ideas. No matter; the power of those works remains available to the few who are fortunate enough to encounter them.

Perl goes on to observe that when we assume that only what is read is valuable, we run the risk of sacrificing value for eyeballs, giving the reader what titillates rather than what edifies.

But writers who live for their readers—or for what their editors imagine their readers want—may end up with an impoverished relationship with those readers.

And when I can’t stand the thought of my words going unread, I face the even greater danger of inflicting half-baked thoughts on our readers. Why is this a danger? Because it risks reinforcing the assumption that readers read my words because they are my words, that what matters isn’t what I say in those words but just that they emerged from my pen phrased in my own special way, that the reader delights in me as a personality (scamp, curmudgeon, innocent, sage, iconoclast) rather than in the ideas I convey. Soon enough I won’t bother baking those thoughts at all, or even worrying whether they qualify as thoughts; any fodder will be sufficient, since it is the sausage machine of my style that provides the significance.

I’ve always been attracted by the thought of being a writer, but not so much by the thought of being a stylist—it didn’t come naturally to me, and I had other ways of scratching my itch to show off. Once I gave up on style, I discovered that all that remained to interest the reader was content—and valuable content was not only hard for me to come by, it was even harder fore me to put into readable shape. At that point I mostly gave up on being a writer.

But I never gave up on writing. What kept me at it was the growing realization that I couldn’t say with any confidence that I understood something if I was unable to convey it to someone else—essentially, putting an idea into words (or failing to do so) is an important test of its soundness. Moreover, putting an idea into words often requires that it be researched and reshaped, with the final product being different that the vague original—and often much less valuable.

Writing, before it is anything else, is a way of clarifying one’s thoughts. This is obviously true of forms such as the diary, which are inherently solitary. But even those of us who write for publication can conclude, once we have clarified certain thoughts, that these thoughts are not especially valuable, or are not entirely convincing, or perhaps are simply not thoughts we want to share with others, at least not now.

I am not someone who does the bulk of my thinking on paper. Usually by the time I sit down to write, the basic ideas have been turned over at length in my mind. But writing still serves as the final, cruelest test of my thinking. I use a few guidelines that will frequently force me to revise my thinking on a certain point, often encourage me to put a idea back in the oven for further baking, and sometimes lead me to abandon certain assumptions. One of those (which I am employing right here) is to always follow a general claim with at least one concrete example. Frequently I need to modify the claim in order to come up with such an example, and often enough I can’t come up with a convincing example at all; both occasions teach me something, and spare the reader a bit of nonsense.

For many of us who love the act of writing—even when we are writing against a deadline with an editor waiting for the copy—there is something monastic about the process, a confrontation with one’s thoughts that has a value apart from the proximity or even perhaps the desirability of any other reader. I believe that most writing worth reading is the product, at least to some degree, of this extraordinarily intimate confrontation between the disorderly impressions in the writer’s mind and the more or less orderly procession of words that the writer manages to produce on the page.

When I think about the writers I loved to read when I was in high school and college, I know what mattered most to me was the one-on-one relationship I felt I was developing with the writer’s thoughts. It was fantastic to feel I was alone with a writer, engaged in a splendid intellectual or imaginative conversation. (The wonder of reading Henry James’s late prose was in seeing his magnificent, disorderly thoughts achieve their infinitely complex order.) [Emphasis added]

I love this observation. I’ve spent enough time trying to think clearly about various topics to know that it is a precious and hard-won thing. And so I bond strongly with a writer who not only has done the hard work of thinking things through, but has gone on to the even harder work of putting those thoughts into readable form. Such writing not only reveals important things to me, it teaches me how to think through such things for myself.

I don’t publish here in order to put my thinking to the test. I am confident that I can read my own writing honestly and critically enough to judge its soundness. I publish in the hope that some will benefit from reading about what I’ve concluded, and that some others from seeing how I reached those conclusions.