Interpreting Results and Drawing Conclusions

First general question: Have we accomplished our experimental goal?

¨ Did you have measurement problems?

· Scale attenuation

* Ceiling effects (not sensitive to variability in upper range; “too easy”)

* Floor effects (not sensitive to variability in lower range; “too hard”)

· Regression artifact

* Selection of subjects or scores reflecting extreme values

¨ Did you find statistically significant results and is it in the direction that supports the hypothesis?

¨ Is your experiment internally valid [i.e., not confounded] -

Internal validity—mostly determined before the fact by the experimental design, being reasonably assured with appropriate design and control techniques

¨ Is your experiment internally valid [i.e., not confounded]

¨ Are effects on DV attributed solely to IV manipulation?

· Did you prevent intrusion of effects of extraneous variables

* random assignment of subjects

* constancy of conditions

* counterbalancing; control of order/carryover effects

· Was it possible that something in experiment (usually an extraneous variable) was different from what was originally intended. If this happens, the possibilities are that the extraneous variable

* did not compromise the internal validity; the results are not confounded.

* resulted in a serendipitous finding [finding a result which was not intended with the experimental manipulation]

* confounded the experiment; a decisive interpretation is not possible

¨ Is your finding reliable?

· Test reliability: consistently measures a variable or construct across subjects and time; an unreliable test provides measures that are not valid

· Experimental reliability: repeating an experiment yields the same results; repeatability usually means performing the experiment with a different set of subjects (except for small n designs); replication is the basis to establish experimental reliability

* Direct replication: same experiment in new subjects (limit the changes0

* Systematic replication: many factors that are considered irrelevant to the phenomenon of interest, but the results should remain the same; key to testing generality and external validity

* Conceptual replication: test the phenomenon (or underlying construct) in a way entirely different from the original test. (example of hypothesis of neuronal vulnerability to alcohol during period of initial dendritic outgrowth and synaptogenesis)