New Year's Resolution: Spot Some Bad Science

 

Here's your assignment: using one (or more) of the 12 methods in the infographic above spot some bad science this year (from COMPoUND iNTEREST download their graphic here). Let me know your findings. Enjoy and Happy New Year. Via datavizblog.

Adding Economic Noise

Two months ago the New York Times had a very informative visualization of monthly economic data with added variability due to sampling error. In the above screen shot we can see how repeated sampling variability can change a steady job growth graph to many different shapes of what it might have looked like with repeated sampling.

Here is another, an accelerating job growth graph and one possible result of the same graph with sampling variability showing very stable job growth, via Statistical Modeling ... One commenter there notes how easy it is for us to reading meaning into random noise.

Dancing Statistics

A still image from the project Communicating Psychology to the Public through Dance, produced by Lucy Irving, Elise Phillips, and Andy Field supported by the British Psychological Association and IdeasTap. Four videos: my favorite Frequency Distributions, Sampling and Standard Error, Variance, and Correlation.

In this image from the first of the videos the dancers start our in one large unorganized group, some dancing with very slow movements, some with very quick movements, and as one would then expect more with movements of a more intermediate speed. As they dance they sort themselves out, from the slower movement dancers on the left, to the more rapidly moving dancers on the right, building up a sample from a bell-shaped distribution. Very clever.

There is a video about correlation with dancers performing the same movements together or nearly opposite movements together. As they mention in the text of the video, these movements are just co-occurrences, one does not cause the other: correlation is not causation.

There is a video about variation with dancers performing variations on the same set of movements.

Another is about sampling and standard error. In this dance, a single blue-shirted dancer performs his movements to indicate the four corners of a rectangle. He and the rectangle defined by his  movements are termed the population. Then several red-shirted dancers mark four corners in their own styles producing various quadrilaterals that estimate the rectangular shape of the blue-shirted dancer.

I do think calling that initial, single blue-shirted dancer a population could be misleading, especially since at the beginning of this video the text mentions "a large group (a population)". This, of course, is the usual view: the large group is the population from which we observe samples to estimate it. But perhaps better, in this setting, would be to talk more generally about a statistical model. This is a model for a dancer's movements. These movements depend on the physical aspects of the dancer: height, limb length, reach, flexibility, etc. They also depend on artistic intent, style, technique, etc.

The blue-shirted dancer specifies the results of a certain collection of all of these aspects. This becomes a parameter, a target. The red-shirted dancers sample from the model of movements to estimate this parameter. The variety and range of their movements display the sampling variability as they attempt to match the governing shape (parameter) of the blue-shirted dancer. This view is more general than the viewing of sampling as from a fixed large group population.

Artificial Population Sampler

I found this brochure in an old file. It dates from 1978. It shows a physical model for teaching sampling and quantitative methods in ecology. Click on the images for larger views. Developed by Arnold M. Schultz, a Professor of Systems Ecology at the University of California, Berkeley and used as a teaching tool beginning in 1958. His 1961 patent application seen here was awarded in 1965.

The model contains two plates of plastic containing arrangements of vinyl discs of various sizes and colors that represent populations in a square region of the Cartesian plane. One plate is called the random population plate made up of six colors that seem to be distributed uniformly over the square. The other plate is called the nonrandom population plate made up of 19 colors that are arranged in rough clusters to illustrate more contiguous, local populations. Instruments are provided for measuring locations and size. Along with color they provide both qualitative and quantitative data for estimating means, variances, density, diversity from various random sampling methods. The population parameters are fixed in each of the plates allowing assessment of measurement error and bias. A paper by Schultz et al. from the Journal of Range Management (1961,v14,n5,pp 236-242) illustrates its use. I've found one listing of the color distribution (from the paper "A comparison of techniques for assessing dispersion patterns," D.W. Goodall and N.E. West, Vegetatio, (40)1:15-27, 1979) but other than a reference to it having six sizes, nothing explicit about the size distribution.

A real hands-on approach to sampling that is very useful for both teaching and research. An example in teaching is illustrated here. Its use in ecological research in 2006 is shown here.