Here’s something I wrote a while ago. It was written while I was taking basic psychology classes because I thought I was going to become a therapist.
8/29/24
In class we’re talking about personality tests. We’re talking about the Rorschach. We’re talking about reliability and validity.
Validity means: are you measuring what you think you’re measuring or what you mean to be measuring.
Reliability means: do you get consistent results.
With regards to the Rorschach, the question of validity can be made to make sense, I think. The question is, maybe, does this lead in a meaningful direction? And if you believe in the Rorschach and on the meaningfulness of unconscious image or story creation the answer can be yes. (But maybe it is only yes conditionally, depending on the ability of the interpreter so then it is not a question of instrument at all but of the talent or competence of the psychotherapist administering the test. The Rorschach can lead you in a direction but can you tell which direction it is pointing?)
Reliability is potentially more confusing. Perhaps the strength of the Rorschach is in its unreliability, its lack of right answer. If the Rorschach was reliable it would be a photograph of a butterfly, and although it would be reliable it would not be useful. I think maybe reliability in the case of the Rorschach or a psychological or personality test like it would be a weakness… but then I think I’m misunderstanding or misrepresenting.
In a reliable personality test, it’s not that you want everyone to get the same result, but that you want everyone who really is that way to get the same result as everyone else who really is that way. You want people with the same strengths and deficiencies and tendencies to respond the same way. I don’t know that I see that as a feasible or even worthwhile goal. The alternative to this for me is that each individual is individual, you can only begin and end with the person in front of you, everything else is projection, but that might just be me being a binary thinker. Of course both are useful.
Science and statistics are a marvel, the attempt to isolate variables and reliably explain behavior, to categorize and diagnose and qualify and explain, are all part of what makes grounded theory possible, but I can’t seem to be convinced that it is the only way to develop good theories. Or certainly the only way to do good clinical work. Case studies are ungeneralizable but not useless. What can be extrapolated? Why do we need to extrapolate?
I think maybe the way to get my head around the statistical problems of generalizability is to think of it this way: if you develop a correlation, 99% of people who are like A are also like B, it doesn’t mean that in your practice all or any of the people who are like A will also be like B, because statistics do not describe people, they describe statistical tendencies within a population. So it may be useful to look into B if someone is A, but not to diagnose or prescribe or project B onto them. Although a case study cannot be used to argue that anything is likely to happen to anyone else, I’m not sure that the most valuable research needs to be extrapolable, or that it is valuable to think of anyone as fitting within even an empirically grounded model of their behavior. The only thing empirical about an individual is what can be observed about them specifically. I guess this ends up just being against broad theoretical projection in general instead of in favor of less “scientific” methods like the Rorschach.. I just like “un scientific” methods.. I can’t help it..