Self-Insight Roadblocks and Detours on the Path to Knowing Thyself by David Dunning

This is the Dunning from the Dunning-Kruger Effect, who had this book in his authors' tagline in the Do your own research (NYT article).

How well you know yourself seems to all boil down to specifics, ie. is there direct evidence of knowledge or capacity? The easier it is to concretize, the more likely we are to know our true state, eg. can I juggle three things, four, five?

Unfortunately, there will always be some horizon where the tested fades into the untestable and then into the unknown.

As reported by Diogenes Laertes, the Greek philosopher Thales, one of the first Western philosophers whose thinking we still have records of, lamented that knowing thyself was one of the most difficult tasks people face certainly more arduous than the easiest task he could think of, which was, of course, giving advice to others.
The correlation between perception and reality is often modest to meager, if it appears at all. For example, peoples ratings of their intelligence tend to correlate roughly around between .2 and .3 with their scores on IQ tests and other intellectual tasks (Hansford & Hattie, 1982). When tested in their ability to tell when other people are lying, their impressions of performance correlate only .04 with actual proficiency (DePaulo, Charlton, Cooper, Lindsay, & Muhlenbruck, 1997). When expressing emotions to others, peoples estimates of success fail to be related at all to actual success (Riggio, Widaman, & Friedman, 1985).
Nurses estimates of their proficiency at basic life support tasks fail to correlate with their actual level of knowledge (Marteau, Johnston, Wynne, & Evans, 1989). Doctors beliefs about their understanding of thyroid disorders does not corre spond at all to their actual level of understanding (Tracey, Arroll, Richmond, & Barham, 1997). Family practice residents views about their patient interviewing skills do not correlate highly (roughly .30) with what their instructors and other experts think (Stuart, Goldstein, & Snope, 1980).
College students do a better job at predicting the longevity of their roommates romantic relationships than they do their own (MacDonald & Ross, 1999). What employees think of their job and social skills in the workplace tends to correlate only .36 with what their peers think of them and, perhaps more important, only . 35 with what their supervisors think. However, both supervisors and peers seem to be spotting something in common, in that the correlation between supervisor and peer impressions tend to be rather high, roughly .62 (Harris & Schaubroeck, 1988).
Similarly, in a study of surgical residents, self-ratings of surgical skill failed to predict how well residents did on an object test of those skills. However, ratings by superiors and peers (which, of course, tended to be highly correlated with each other) successfully predicted performance on the objective test (Risucci, Tortolani, & Ward, 1989).
In the social psychological literature in recent years, there has been a continuing debate about whether, in essence, the Greeks were right when they asserted that knowing thyself was at the core of living a happy, productive, and gracious life. Some researchers have asserted the good life does not flow from accuracy, but rather from bias. They argue that having unrealistic and positive views of self are the golden keys to the good life, that mistaken self-images are essential for generating motivation, good moods, creativity, and generosity with others (e.g., Taylor & Brown, 1988). Needless to say, this position has its discontents (e.g., Colvin & Block, 1994).
It is not that people performing poorly fail to recognize their incompetence. Instead, our argument is that people performing poorly cannot be expected to recognize their ineptitude. They are simply not in a position to know that they are doing badly. The ability to recognize the depth of their inadequacies is beyond them.
Several domains in the intellectual and social realms share this propertythe skills needed to produce a correct response are also the skills needed to evaluate the adequacy of a response. In a phrase, the skills needed to perform the cognitive task (producing the response) are the exact same ones necessary for metacognitive tasks (judging the response).
Much work in cognitive psychology reveals that the intellectual skills a person has can differ dramatically from domain to domain. For example, the most skilled horserace handicappers, those who set the odds of each horse winning a race, tend not to be particularly bright when it comes to more academic tasks (Ceci & Liker, 1986).
Because top performers do so well, they just assume that other people are doing well, too. As a result, they do not believe their performances to be that unique or special, leading them to underestimate their percentile ranking of skill relative to their peers. In short, top performers suffer from a false consensus effect, which refers to the tendency for people to overestimate the commonness of their own responses and experiences (Marks & Miller, 1987; Ross, Greene, & House, 1977). Top performers, being knowledgeable, overestimate how knowledgeable their peers are.
In short, the grading study showed that observing the performances of others helps the competent to achieve more accurate views of themselves, but does not necessarily help the incompetent.
The social psychological literature is replete with examples of how people avoid, distort, or spin information to reach favorable conclusions about themselves and their abilities (for reviews, see Baumeister & Newman, 1994; Dunning, 2001; Kunda, 1990).
Thus, with ill-structured tasks, one makes errors of omission, but the set of omission errors is not well defined.
One might write a book about, lets say, self-insight, but one will never know if the observations made in the book were the best set of observations that could have been made on the topic. In short, with ill-structured tasks, people are never really in a position to know just how well they have done with the solutions they reached.
Such superstitions are not constrained to birds. Nearly 40 years after B.F.Skinners (1948) experiment, Ono (1987) repeated something like it with humans. College students were brought into a laboratory and placed in front of an apparatus with three levers, three lights, and a counter. They were told to try to get as many points on the counter as they could. They did try, even though their efforts had no impact on how many points rang up on the counter. Despite this, one student began pulling levers in a specific sequence, followed by keeping a hand on one lever. Another student began laying her hands on the levers, the apparatus containing them, a nail in the wall, as well as other objects in the room, in a vain attempt to conjure forth more points.
Police investigators are no better than laypeople at picking out liars; both groups do just a tad better than chance. See Ekman & OSullivan, 1991.)
People tend to construe themselves as special, as unique, as more invulnerable to the situational pressures that cause others to act selfishly and unethically. As Goethe, the German poet, observed, self-knowledge comes from knowing other men, but this is an observation we often do not take advantage of. Perhaps this is not surprising. As the anthropologist Clifton Geertz once observed, one of the most difficult tasks bedeviling self-perception is to see ourselves as no more than a local example of the forms human life has locally taken, a case among cases.
The situation was further well defined for participants in that they possessed direct experience with the task before they made their predictions, allowing them to know what the situation would look like concretely (gee, the snake does not writhe all that much) and what their emotional reaction would feel like (gosh, I do not faint when I touch it). Second, the actions participants were asked to perform were, in a sense, completely under their control. Each had the physical capability to walk toward the snake, extend their hand to its skin, and to grasp it sufficiently to pick it up out of the cage. To be sure, their emotions may run unchecked, but if they had made the prediction to perform some task they did have it within their physical capability to complete it. Third, the predictions that participants made were specifically and concretely tailored to the tasks at hand. They were asked precise questions about rather concrete behaviors, such as whether they could touch the snake with their bare hand. They were not asked more vague and indirect questions, such as whether they would be a good snake handler. The specificity of the questions asked of people matters. A long litany of research findings suggests that people tend to be much more accurate when they are asked concrete questions about precisely defined behaviors rather than when asked more global and abstract queries less specifically tailored to the particular task at hand (Bandura, 1986).