Uncertainty Wednesday: Avoiding Strong Claims on Intelligence

Today’s Uncertainty Wednesday will be the last post for a while about matters related to intelligence following my posts on the problems of sample correlation under fat tails and on dynamic versus static models. Now might be a good time to state that I firmly believe we should be researching human intelligence including its genetic component. This should not be off limits to science, especially at a time when we are actively building artificial intelligence and making major progress in neuroscience and in understanding and modifying the human genome. My point is simply that this is an area in which we must proceed with extreme caution and avoid strong claims. That is both because of the sordid history of using intelligence claims to justify all sorts of horrors and because we are at the beginning of our understanding of the extraordinary complexity of our genome and its expression as well as of our brain.

The point of all three prior posts was to point out methodological problems that many people making or repeating strong claims today seem either to be unaware of or to dismiss. To close out this mini series, I want to mention two more such issues. The first relates to so-called genome-wide association studies (GWAS) that are currently being done to come up with polygenic scores for all sorts of measures, including intelligence. Now given the size of the genome we are faced with tens of millions of potential features as the difference between two humans is currently estimated at about 20 million base pairs. So this is very much a situation where any analysis faces the curse of dimensionality and it is possible to wind up with random loadings that still appear to have explanatory power. Again, I am not saying we shouldn’t do such analyses, just that we need a ton of data, a lot of careful analysis and repeatable independent replication of results.

Second, intelligence, unlike say height, is something we have a hard time defining and measuring (if it were easy we would be a lot further with creating artificial intelligence). In fact one of the central concepts of lot of intelligence research is the so called g factor, which is a measure of correlation among performance on a variety of cognitive tasks. In other words, the g factor is a computed value, which adds some interesting methodological challenges (e.g., what should be a Bayesian prior for the g factor?). Added to this is the challenge that the rise of intelligence testing is co-incident with the industrial age and so a lot of “validation” measures of intelligence tests has been their ability to predict success in an industrial society, such as degree of educational attainment or lifetime income. For a species that has hundreds of thousand of years of history before the industrial age and will (hopefully) have hundreds of thousand of years of history after it, that is likely way too narrow a way of thinking about intelligence. Another problem arising from this historic coincidence is that we really don’t have much of an idea of how intelligence could be developed in a radically different educational system, since our current system was also developed during the same time (and in no small part influenced by strong claims about intelligence). The current system is heavily biased towards people who are fast learners out of the gate. By contrast slow starters are usually cut off from most prolonged learning opportunities and of course many people are excluded entirely. As a result we have very little to no data on how someone who is interested in math but might be off to a slow start (or no start) could do if they were able to stick with it for a decade or more.

So by all means let’s do research. But let’s approach this field with the extreme care and degree of skepticism that is called for. And until we really know much more than we do today, let’s avoid strong claims entirely. This is a responsibility not just for those working in the field but also anyone covering it or simply re-tweeting it.