AERA-I: Education in the Professions Forum

In response to my post "Re: pre/post testing to determine student 
progress," [Hake (2006)], Eugene Komaroff [Director of the 
Biostatistics and Clinical Trials at the Kessler Medical 
Rehabilitation Research & Education Corporation of The New Jersey 
Medical School] posted on AERA-D:

"Thank you for the most informative post on controversies surrounding 
RCT in education.  However, seems to me the problem is lack of 
randomization, not a lack of comparison/control group, for drawing 
causal conclusions with pre-post designs.  I do not understand the 
resistance to randomization.  Why does random assignment to 
intervention appear difficult to achieve in Educational Research - 
ethics, politics?

Physicist Robert Ehrlich (2002) in an American Journal of Physics 
(AJP) article "How do we know if we're doing a good job in physics 
Teaching?" raised a similar point. In my response [Hake (2002a) I 
wrote [bracketed by lines "HHHHHHHHHHH. . . .", see that article for 
references other than Ehrlich (2002) and Hake (1998a,b; 2002b)]:

HHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHH
In his recent perspective ["How do we know if we're doing a good job 
in physics Teaching?"], Robert Ehrlich set forth some good ideas, 
but, in my opinion, faltered in his criticism of my survey article 
[Hake (1998a,b)]
. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . 
Here are Ehrlich's remaining reasons 1, 2, and 4 for not accepting my 
claims at face value, followed by brief rebuttals:
. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .  
2. ". . . the IE versus non-IE comparison is hardly a double blind 
one, because both Hake and the course instructor knew both the 
category the course is being placed into (IE or non-IE), as well as 
the FCI gain for that class."

In the case of a survey such as mine, it is not clear that blindness 
to differences in T [Traditional] and IE [Interactive Engagement] 
physics instruction could have been found in any potential surveyors, 
physics teachers, or students who were not medically 
institutionalized.

Non-double-blind education research experiments may be less 
convincing than some double-blind medical experiments, but that 
doesn't mean that the education results should necessarily be taken 
at LESS than face value. In the case of my survey, I think that the 
results merit acceptance at FULL face value, especially considering 
the fact that normalized gain differences between T and IE courses 
that are consistent with those I reported, have now been obtained by 
physics education research groups [referenced in Hake (2002b)] at the 
University of Maryland [Redish et al. (1997), Saul (1998), Redish and 
Steinberg (1999), Redish (1999)], the University of Montana [Francis 
et al. (1998)], Rennselaer and Tufts Universities [Cummings et al. 
(1999)], North Carolina State University [Beichner et al. (1999)], 
Hogskolan Dalarna - Sweden [Bernhard (2001)], Carnegie Mellon 
University [Johnson (2001)], and City College of New York [Steinberg 
and Donnelly (2002)].
HHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHH


And then in "Re: Should Randomized Control Trials Be the Gold 
Standard of Educational Research? [Hake (2005)] I wrote:

HHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHH
Could physics education researchers (PER's) whose work is 
predominately in UNDERGRADUATE education utilize RCT's? PER's deal 
with populations of UP (university professors) and US (Undergraduate 
Students). Most UP's demand autonomy in the way they teach courses 
since they obviously know best how to lecture. Most of the US's (or 
their parents) paid good money to be lectured at. No one that I know 
of has been insane enough to even suggest that subjects from 
populations UP and US be randomly assigned to different curricula in 
a RCT, especially if one curriculum deemphasizes lectures.  Also the 
average UP, thrown into an IE course would be a total disaster. If 
anyone has some ideas on how to accomplish an RTC among UP's and US's 
while avoiding dismissal or execution please let me know. Of course 
one could PAY the subjects, but this might bias the results towards 
the greedy and/or impecunious.
HHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHH


Richard Hake, Emeritus Professor of Physics, Indiana University
24245 Hatteras Street, Woodland Hills, CA 91367
<[log in to unmask]>
<http://www.physics.indiana.edu/~hake>
<http://www.physics.indiana.edu/~sdi>


REFERENCES
Ehrlich, R. 2002. "How do we know if we're doing a good job in physics
Teaching?"  Am. J. Phys. 70(1), 24-29 (2002); online to subscribers 
at 
<http://scitation.aip.org/dbt/dbt.jsp?KEY=AJPIAS&Volume=70&Issue=1>.

Hake, R.R. 1998a. "Interactive-engagement vs traditional methods: A
six-thousand-student survey of mechanics test data for introductory 
physics courses," Am. J. Phys. 66: 64-74; online as ref. 24 at
<http://www.physics.indiana.edu/~hake>, or simply click on 
<http://www.physics.indiana.edu/~sdi/ajpv3i.pdf> (84 kB).

Hake, R.R. 1998b. "Interactive-engagement methods in introductory 
mechanics courses," online as ref. 25 at 
<http://www.physics.indiana.edu/~hake>, or simply click on 
<http://www.physics.indiana.edu/~sdi/IEM-2b.pdf> (108 kB) -   a 
crucial companion paper to Hake (1998a).

Hake, R.R. 2002a. "Comment on 'How do we know if we are doing a good 
job in physics teaching?' by Robert Ehrlich," Am. J. Phys. 70(10): 
1058-1059; online as ref. 17 at 
<http://www.physics.indiana.edu/~hake> or download directly by 
clicking on 
<http://www.physics.indiana.edu/~hake/HakeOnEhrlich-2.pdf> (40 kB).

Hake, R.R. 2002b. "Lessons from the physics education reform effort," 
Ecology and Society 5(2): 28; online at
<http://www.ecologyandsociety.org/vol5/iss2/art28/>. Ecology and 
Society (formerly Conservation Ecology) is a free online 
"peer-reviewed journal of integrative science and fundamental policy 
research" with about 11,000 subscribers in about 108 countries.

Hake, R.R. 2005. Re: Should Randomized Control Trials Be the Gold 
Standard of Educational Research? online at
<http://lists.asu.edu/cgi-bin/wa?A2=ind0504&L=aera-l&T=0&O=D&P=2100>.
Post of 17/18 Apr 2005 to AERA-C, AERA-D, AERA-G, AERA-H, AERA-J,
AERA-K, AERA-L, AP-Physics, ASSESS, Biopi-L, Chemed-L, EvalTalk, 
Math-Learn, Phys-L, PhysLrnR, Physhare, POD, STLHE-L, & TIPS.

Hake, R.R. 2006. "Re: pre/post testing to determine student 
progress," online at
<http://lists.asu.edu/cgi-bin/wa?A2=ind0609&L=aera-l&T=0&F=&S=&P=1718>. 
Post of 24/25 Sept 2006 to AERA-A, AERA-B, AERA-C, AERA-D, AERA-I, 
AERA-J, AERA-K, AERA-L, ASSESS, Chemed-L, Biopi-L, Biolab (rejected), 
DrEd, EvalTalk, IFETS, Math-Learn, PsychTeacher (rejected), 
TeachingEdPsych, Phys-L, PhysLnrR, POD, RUME, STLHE-L, & TIPS.

AERA Division I: Education in the Professions Forum
-----------------------------------------------------------------
 AERA Home Page on the World Wide Web: http://www.aera.net
 List Service Info http://lists.asu.edu/cgi-bin/wa
 To cancel your subscription address an email message to
 [log in to unmask] containing only the message UNSUB AERA-I
 Address problems with your subscription to: [log in to unmask]
-----------------------------------------------------------------