WEBVTT

1
00:00:09.280 --> 00:00:11.105
- Thank you all for turning out on this

2
00:00:11.105 --> 00:00:14.505
very rainy Monday night.

3
00:00:14.505 --> 00:00:17.651
We are very pleased to have, at Smith,

4
00:00:17.651 --> 00:00:20.089
Professor John Creswell.

5
00:00:20.089 --> 00:00:22.412
He is professor of Educational Psychology

6
00:00:22.412 --> 00:00:25.592
at the University of Nebraska-Lincoln.

7
00:00:25.592 --> 00:00:28.612
In addition to teaching at the university,

8
00:00:28.612 --> 00:00:30.306
is the author of numerous articles

9
00:00:30.306 --> 00:00:32.327
on mixed-methods research,

10
00:00:32.327 --> 00:00:34.230
qualitative methodology,

11
00:00:34.230 --> 00:00:36.865
general research design.

12
00:00:36.865 --> 00:00:39.500
He's also the author of 22 books,

13
00:00:39.500 --> 00:00:41.799
including many new editions,

14
00:00:41.799 --> 00:00:44.180
which focus on research designs,

15
00:00:44.180 --> 00:00:47.500
comparisons of qualitative
research methodologies,

16
00:00:47.500 --> 00:00:51.667
and the nature and use of
mixed-methods research.

17
00:00:51.667 --> 00:00:54.801
His books are wildly popular,

18
00:00:54.801 --> 00:00:57.356
and have been translated
into many languages,

19
00:00:57.356 --> 00:01:00.630
and hat are used around the world.

20
00:01:00.630 --> 00:01:02.209
Now I get to have a little fun.

21
00:01:02.209 --> 00:01:04.845
You all know who Dan Brown is?

22
00:01:04.845 --> 00:01:06.783
"Angels and Demons?"

23
00:01:06.783 --> 00:01:09.755
He's number one on Amazon.

24
00:01:09.755 --> 00:01:11.855
John Creswell, writing a book on

25
00:01:11.855 --> 00:01:14.584
mixed-methods methodology,

26
00:01:14.584 --> 00:01:17.790
is number 1,005.

27
00:01:17.790 --> 00:01:20.041
(wooing)

28
00:01:20.041 --> 00:01:23.074
I have a book that's, like, 4,000,067.

29
00:01:23.074 --> 00:01:24.987
(laughter)

30
00:01:28.495 --> 00:01:30.564
You rarely get to compare your speaker

31
00:01:30.564 --> 00:01:33.527
to the Amazon fame list of Dan Brown

32
00:01:33.527 --> 00:01:35.535
and Stephen King, who are number one

33
00:01:35.535 --> 00:01:37.266
and number two at the minute.

34
00:01:37.266 --> 00:01:40.289
Not only does he have number one,

35
00:01:40.289 --> 00:01:42.911
not only is he 1,005 in books,

36
00:01:42.911 --> 00:01:45.334
but he's number one in

37
00:01:45.334 --> 00:01:47.947
qualitative, quantitative
and mixed methods

38
00:01:47.947 --> 00:01:50.908
in the social sciences.

39
00:01:50.908 --> 00:01:53.485
A prior book of Dr. Creswell's,

40
00:01:53.485 --> 00:01:55.354
"Qualitative Inquiry and Research Design:

41
00:01:55.354 --> 00:01:57.629
"Choosing Among Five Approaches,"

42
00:01:57.629 --> 00:02:01.171
which, for a long time,
was the most widely read

43
00:02:01.171 --> 00:02:03.701
qualitative research book in the world,

44
00:02:03.701 --> 00:02:05.300
is now number three.

45
00:02:05.300 --> 00:02:08.347
So he's populating the top 10 very fully.

46
00:02:10.813 --> 00:02:13.216
John Creswell held the Clifton Institute

47
00:02:13.216 --> 00:02:15.040
Endowed Professor Chair for five years

48
00:02:15.040 --> 00:02:17.802
at the University of Nebraska-Lincoln.

49
00:02:17.802 --> 00:02:19.636
For the past five years, he's served

50
00:02:19.636 --> 00:02:20.865
as co-director of the

51
00:02:20.865 --> 00:02:23.049
Office of Qualitative and
Mixed Methods Research

52
00:02:23.049 --> 00:02:25.765
at Nebraska-Lincoln,
and has provided support

53
00:02:25.765 --> 00:02:27.785
for scholars, incorporating qualitative

54
00:02:27.785 --> 00:02:30.130
and mixed methods into research projects

55
00:02:30.130 --> 00:02:32.905
for external funding.

56
00:02:32.905 --> 00:02:35.354
Another big achievement and real area

57
00:02:35.354 --> 00:02:37.305
of leadership is that he has served

58
00:02:37.305 --> 00:02:40.025
as the founding co-editor
of the Sage journal

59
00:02:40.025 --> 00:02:43.310
"The Journal of Mixed Methods Research."

60
00:02:43.310 --> 00:02:45.656
He's also been an adjunct
professor of family medicine

61
00:02:45.656 --> 00:02:48.373
at the University of
Michigan, where he's worked

62
00:02:48.373 --> 00:02:52.467
on investigations in the
health sciences and education,

63
00:02:52.467 --> 00:02:54.200
and research methodology for the

64
00:02:54.200 --> 00:02:55.943
National Institutes of Health

65
00:02:55.943 --> 00:02:58.962
and National Science Foundation projects.

66
00:02:58.962 --> 00:03:00.981
He's served extensively as a consultant

67
00:03:00.981 --> 00:03:02.804
in the health sciences research area

68
00:03:02.804 --> 00:03:05.301
for the Veterans Administration.

69
00:03:05.301 --> 00:03:07.994
Dr. Creswell has also been
a Senior Fulbright Scholar

70
00:03:07.994 --> 00:03:11.988
to South Africa, and in 2008, lectured

71
00:03:13.388 --> 00:03:15.897
to faculty at five
universities on education

72
00:03:15.897 --> 00:03:18.092
and health sciences.

73
00:03:18.092 --> 00:03:21.261
In 2012, he was again a
Senior Fulbright Scholar

74
00:03:21.261 --> 00:03:23.153
to Thailand.

75
00:03:23.153 --> 00:03:26.101
In 2011, he served as co-leader

76
00:03:26.101 --> 00:03:27.959
of a national working group for the

77
00:03:27.959 --> 00:03:30.142
National Institutes of Health, developing

78
00:03:30.142 --> 00:03:32.778
best practices for mixed-methods research

79
00:03:32.778 --> 00:03:34.820
in the health sciences.

80
00:03:34.820 --> 00:03:37.897
In the spring of 2013, Dr. Creswell

81
00:03:37.897 --> 00:03:39.638
has served as a visiting professor

82
00:03:39.638 --> 00:03:42.100
at Harvard School of Public Health.

83
00:03:42.100 --> 00:03:44.736
In summer 2013, he'll conduct

84
00:03:44.736 --> 00:03:46.626
mixed-method training workshops

85
00:03:46.626 --> 00:03:49.494
at Cambridge University in the U.K.

86
00:03:49.494 --> 00:03:52.976
In 2014, he'll be awarded
an honorary doctorate

87
00:03:52.976 --> 00:03:56.868
from the University of
Pretoria in South Africa.

88
00:03:56.868 --> 00:04:00.038
As you can see, he mightily deserves that.

89
00:04:00.038 --> 00:04:02.522
We're very proud to have Dr. John Creswell

90
00:04:02.522 --> 00:04:04.484
with us tonight at Smith.

91
00:04:04.484 --> 00:04:07.312
Dr. Creswell. (applause)

92
00:04:08.443 --> 00:04:10.376
- Thank you. Very good.

93
00:04:17.012 --> 00:04:20.284
Well, thank you for coming out tonight.

94
00:04:20.284 --> 00:04:24.452
Jim, thanks for the
great introduction here,

95
00:04:24.452 --> 00:04:26.682
and the dean for your support

96
00:04:26.682 --> 00:04:28.853
to come here,

97
00:04:28.853 --> 00:04:31.093
and my daughter, Joanna Baez, who is

98
00:04:31.093 --> 00:04:32.834
a third-year doctoral student,

99
00:04:32.834 --> 00:04:35.533
who loves the doctoral
program here at Smith,

100
00:04:35.533 --> 00:04:38.714
and has just thrived so much.

101
00:04:38.714 --> 00:04:41.523
I appreciate all that
Smith has done for her.

102
00:04:42.696 --> 00:04:45.527
She'll be a great asset when she goes out

103
00:04:45.527 --> 00:04:47.797
to the social work community.

104
00:04:49.834 --> 00:04:53.131
I'm going to introduce
mixed-methods research tonight

105
00:04:53.131 --> 00:04:56.092
in a way that, hopefully,
will be very user friendly

106
00:04:56.092 --> 00:04:57.535
and applied.

107
00:04:59.921 --> 00:05:01.829
I'm going to start

108
00:05:03.519 --> 00:05:06.154
with a very broad question.

109
00:05:07.668 --> 00:05:09.584
What evidence do we use

110
00:05:09.584 --> 00:05:12.574
to really makes sense of our world today?

111
00:05:14.506 --> 00:05:17.244
Now I'm going to bring in an illustration.

112
00:05:18.988 --> 00:05:22.752
This will, perhaps, tap into

113
00:05:22.752 --> 00:05:24.942
your emotions.

114
00:05:24.942 --> 00:05:28.065
It will tap into the affective side.

115
00:05:28.065 --> 00:05:30.746
Hopefully it won't re-traumatize people

116
00:05:30.746 --> 00:05:32.290
in the audience.

117
00:05:32.290 --> 00:05:34.310
It's not my intent.

118
00:05:34.310 --> 00:05:35.924
I'd have to say, when I put together these

119
00:05:35.924 --> 00:05:38.306
next slides, though,

120
00:05:38.306 --> 00:05:39.663
I just felt

121
00:05:41.289 --> 00:05:44.230
a churning of my stomach.

122
00:05:44.230 --> 00:05:46.493
It was very hard to put these together,

123
00:05:46.493 --> 00:05:49.484
but I think it illustrates mixed methods.

124
00:05:52.321 --> 00:05:54.321
So we have the Boston Marathon

125
00:05:54.321 --> 00:05:55.942
on April 15th.

126
00:05:57.674 --> 00:06:00.504
We can look at this event
in a number of ways.

127
00:06:02.508 --> 00:06:05.073
The weather was in the 50s.

128
00:06:05.073 --> 00:06:07.313
There were 23,000 runners.

129
00:06:08.486 --> 00:06:12.713
The prize was $108,000.

130
00:06:12.713 --> 00:06:14.545
71 countries.

131
00:06:16.072 --> 00:06:19.294
75%, three-quarters, of
the individuals finished.

132
00:06:21.207 --> 00:06:24.644
There were 18 hand cycle contestants

133
00:06:24.644 --> 00:06:28.540
and 52 wheelchair contestants in that.

134
00:06:31.958 --> 00:06:34.419
After the bombs,

135
00:06:34.419 --> 00:06:36.951
which occurred at 2:50 in the afternoon,

136
00:06:38.563 --> 00:06:41.506
there were three people
killed, 170 wounded.

137
00:06:42.750 --> 00:06:45.553
Taken to the Brigham Hospital, which was

138
00:06:45.553 --> 00:06:48.839
close to where I had my office,

139
00:06:48.839 --> 00:06:51.253
were 31 individuals.

140
00:06:51.253 --> 00:06:53.830
Mass General had 31.

141
00:06:53.830 --> 00:06:56.652
Boston Medical Center had 23.

142
00:06:56.652 --> 00:06:59.508
Beth Israel had 21.

143
00:06:59.508 --> 00:07:02.207
Boston Children had 10.

144
00:07:04.117 --> 00:07:06.137
The first responders are a group

145
00:07:06.137 --> 00:07:08.610
that I work with, actually, at Harvard,

146
00:07:08.610 --> 00:07:11.061
the Harvard Humanitarian Initiative.

147
00:07:12.372 --> 00:07:13.678
They were some of the first doctors

148
00:07:13.678 --> 00:07:15.559
at these hospitals.

149
00:07:15.559 --> 00:07:17.428
Because they were prepared for dealing

150
00:07:17.428 --> 00:07:20.667
with crisis situations, did a great job

151
00:07:20.667 --> 00:07:23.831
in responding to the
people that were injured.

152
00:07:30.059 --> 00:07:33.911
Now, I was at the mile 24 marker.

153
00:07:33.911 --> 00:07:35.835
That morning, I decided not to go down

154
00:07:35.835 --> 00:07:37.398
to the finish line.

155
00:07:39.014 --> 00:07:42.547
24 mile marker was a place,

156
00:07:42.547 --> 00:07:45.113
just as you would expect, two miles

157
00:07:45.113 --> 00:07:47.759
from the finish line.

158
00:07:47.759 --> 00:07:50.789
It was a point where the
runners were getting tired

159
00:07:50.789 --> 00:07:53.158
and wanting to drop out.

160
00:07:53.158 --> 00:07:54.887
There must have been a thousand people

161
00:07:54.887 --> 00:07:56.385
at this corner.

162
00:07:56.385 --> 00:07:58.545
You can't see all of the people here

163
00:07:58.545 --> 00:08:00.831
cause this is my picture.

164
00:08:00.831 --> 00:08:02.491
There were about a thousand people.

165
00:08:02.491 --> 00:08:05.418
Someone had set up a
couple of big speakers.

166
00:08:05.418 --> 00:08:09.120
They were blasting out
this music on the speakers,

167
00:08:09.120 --> 00:08:11.104
"Sweet Caroline,"

168
00:08:11.104 --> 00:08:13.262
Chubby Checker's "Shout."

169
00:08:14.498 --> 00:08:16.829
Imagine standing there with about

170
00:08:16.829 --> 00:08:19.883
a thousand people, half of 'em singing

171
00:08:19.883 --> 00:08:23.093
at the top of their lungs those songs.

172
00:08:25.781 --> 00:08:27.991
So this was a moment of jubilation.

173
00:08:29.866 --> 00:08:31.688
Actually, some of the runners

174
00:08:31.688 --> 00:08:33.325
were doing that YMCA when the

175
00:08:33.325 --> 00:08:35.462
YMCA song came on, you know,

176
00:08:35.462 --> 00:08:36.866
as they were running by.

177
00:08:36.866 --> 00:08:38.997
They were starting to smile.

178
00:08:40.987 --> 00:08:43.472
A few minutes later, I was standing there,

179
00:08:43.472 --> 00:08:45.097
and I watched the people that went

180
00:08:45.097 --> 00:08:47.071
to the 24 mile marker and were asked

181
00:08:47.071 --> 00:08:48.858
to be turned around.

182
00:08:48.858 --> 00:08:50.077
Did you know that?

183
00:08:50.077 --> 00:08:51.679
The police stopped 'em

184
00:08:51.679 --> 00:08:53.490
at mile 24 because they couldn't go on

185
00:08:53.490 --> 00:08:55.897
to miles 25.

186
00:08:55.897 --> 00:08:59.982
I stood there, after this
moment of jubilation,

187
00:08:59.982 --> 00:09:01.607
and watched people, you know how

188
00:09:01.607 --> 00:09:05.450
they have those aluminum capes around,

189
00:09:05.450 --> 00:09:07.060
walking back

190
00:09:08.468 --> 00:09:10.738
right through this area.

191
00:09:15.446 --> 00:09:17.025
I've just presented two different kinds

192
00:09:17.025 --> 00:09:18.775
of evidence for you.

193
00:09:22.841 --> 00:09:25.506
One was the numbers,

194
00:09:25.506 --> 00:09:27.608
which was informative, wasn't it?

195
00:09:27.608 --> 00:09:28.908
I'll bet you didn't know all those

196
00:09:28.908 --> 00:09:31.346
wheelchair people were in this race,

197
00:09:31.346 --> 00:09:33.308
or even that there was a prize for over

198
00:09:33.308 --> 00:09:35.313
$806,000.

199
00:09:37.766 --> 00:09:39.802
Then, I told you

200
00:09:39.802 --> 00:09:42.280
my personal story.

201
00:09:45.166 --> 00:09:47.092
So which one did you like the best,

202
00:09:47.092 --> 00:09:49.861
or which one was the
best evidence for you?

203
00:09:52.630 --> 00:09:55.074
People might be divided on this question.

204
00:09:56.693 --> 00:10:00.193
In fact, if I just presented
the quantitative statistics,

205
00:10:02.150 --> 00:10:04.904
it probably wouldn't tell the whole story.

206
00:10:04.904 --> 00:10:07.749
Or if I just told you my personal story,

207
00:10:07.749 --> 00:10:10.767
without those statistics,

208
00:10:10.767 --> 00:10:12.840
it wouldn't be as complete.

209
00:10:16.804 --> 00:10:18.313
What I've given you here

210
00:10:18.313 --> 00:10:20.786
is a mixed-methods rendering

211
00:10:20.786 --> 00:10:22.959
of the Boston bombing situation,

212
00:10:24.548 --> 00:10:27.572
where both statistics
and person experiences

213
00:10:27.572 --> 00:10:29.797
come together.

214
00:10:29.797 --> 00:10:32.792
Are you beginning to understand
mixed-methods research?

215
00:10:32.792 --> 00:10:34.690
(audience agrees)

216
00:10:38.016 --> 00:10:40.895
It's a very simple idea.

217
00:10:40.895 --> 00:10:43.298
We don't need to complicate this.

218
00:10:43.298 --> 00:10:46.096
Mixed methods is using qualitative

219
00:10:46.096 --> 00:10:50.309
and quantitative data together

220
00:10:50.309 --> 00:10:52.214
to have a larger understanding

221
00:10:52.214 --> 00:10:54.318
of the problem we're working with.

222
00:10:55.592 --> 00:10:58.759
That each one might give
us a partial picture,

223
00:10:58.759 --> 00:11:01.824
but when we put these
two pictures together,

224
00:11:01.824 --> 00:11:04.541
we have a more complete understanding.

225
00:11:04.541 --> 00:11:06.863
Here's a formal definition.

226
00:11:06.863 --> 00:11:08.905
It's an approach to
investigating behavior,

227
00:11:08.905 --> 00:11:11.007
social and health science
problems by collecting

228
00:11:11.007 --> 00:11:12.563
and analyzing

229
00:11:12.563 --> 00:11:14.583
quantitative and
qualitative data in response

230
00:11:14.583 --> 00:11:17.009
to research questions, and integrating

231
00:11:17.009 --> 00:11:19.505
or mixing the two to generate

232
00:11:19.505 --> 00:11:22.593
a new and more complete insight

233
00:11:22.593 --> 00:11:24.312
than what would have been obtained

234
00:11:24.312 --> 00:11:27.064
by either than quantitative
or the qualitative.

235
00:11:28.683 --> 00:11:30.895
That's the pitch.

236
00:11:32.561 --> 00:11:34.244
I've come to this

237
00:11:34.244 --> 00:11:38.086
position, starting out as
a quantitative researcher.

238
00:11:38.086 --> 00:11:41.117
I spent the first 12 years of my life

239
00:11:41.117 --> 00:11:43.973
doing correlational analysis,
experimental studies,

240
00:11:43.973 --> 00:11:46.701
survey projects, et cetera.

241
00:11:46.701 --> 00:11:49.963
Then I started teaching
qualitative research.

242
00:11:49.963 --> 00:11:51.809
Then I added to that mixed methods

243
00:11:51.809 --> 00:11:54.107
because I really felt that putting

244
00:11:54.107 --> 00:11:56.185
the two approaches together would give us

245
00:11:56.185 --> 00:11:57.927
greater insight.

246
00:11:57.927 --> 00:12:01.549
This book, which first came out in 1994,

247
00:12:01.549 --> 00:12:03.394
had a chapter called

248
00:12:03.394 --> 00:12:06.166
"Combined Quantitative
and Qualitative Research."

249
00:12:07.632 --> 00:12:10.097
I got more emails

250
00:12:10.097 --> 00:12:12.372
about that one chapter

251
00:12:12.372 --> 00:12:15.339
than any other part of this book.

252
00:12:17.376 --> 00:12:19.199
I didn't really have a lot to work from

253
00:12:19.199 --> 00:12:22.466
at that point in the early '90s.

254
00:12:24.028 --> 00:12:26.814
I've expanded this to different
qualitative approaches,

255
00:12:26.814 --> 00:12:28.718
five approaches here,

256
00:12:28.718 --> 00:12:31.318
11 different approaches here.

257
00:12:31.318 --> 00:12:34.138
This is the mixed methods
intro book that I use

258
00:12:35.351 --> 00:12:36.999
in all my workshops.

259
00:12:36.999 --> 00:12:39.658
This is a reader of good studies.

260
00:12:39.658 --> 00:12:43.152
This is an applied
educational research book.

261
00:12:43.152 --> 00:12:46.286
I've got six books that
are very active now,

262
00:12:46.286 --> 00:12:47.967
that really are beginning to focus

263
00:12:47.967 --> 00:12:50.382
more and more on how you combine

264
00:12:50.382 --> 00:12:52.182
multiple approaches,

265
00:12:52.182 --> 00:12:54.399
how you use different
approaches to research,

266
00:12:54.399 --> 00:12:57.331
how you can bring quantitative
and qualitative together.

267
00:12:59.763 --> 00:13:03.002
Then I started looking
into the social work field.

268
00:13:03.002 --> 00:13:05.886
I've worked with Deborah
Padgett down at NYU.

269
00:13:08.400 --> 00:13:11.469
I've noted that she's
done some writing here

270
00:13:11.469 --> 00:13:14.093
on qualitative and mixed
methods and social work.

271
00:13:14.093 --> 00:13:17.669
"Knowledge Development," which
is a guest editorial piece

272
00:13:17.669 --> 00:13:20.059
where she begins to introduce people,

273
00:13:20.059 --> 00:13:22.025
back in 2009,

274
00:13:22.025 --> 00:13:25.682
to mixed-methods research.

275
00:13:25.682 --> 00:13:28.710
I've looked at Jim's article,

276
00:13:28.710 --> 00:13:30.487
Professor Driscoll's article, on

277
00:13:30.487 --> 00:13:32.459
utilization-focused evaluation

278
00:13:32.459 --> 00:13:35.487
of two intensive family
preservation programs.

279
00:13:36.836 --> 00:13:38.533
It's an article where he brought in

280
00:13:38.533 --> 00:13:42.125
both qualitative and quantitative data.

281
00:13:42.125 --> 00:13:43.648
He was ahead of his time because

282
00:13:43.648 --> 00:13:46.897
this was in 1998

283
00:13:46.897 --> 00:13:48.836
that he's been thinking about

284
00:13:48.836 --> 00:13:51.262
how you might utilize both forms of data,

285
00:13:51.262 --> 00:13:53.456
both forms of evidence, together.

286
00:13:54.849 --> 00:13:56.748
There are more recent pieces.

287
00:13:56.748 --> 00:14:00.265
This came out in 2012, looking at

288
00:14:00.265 --> 00:14:02.413
group work for social work students

289
00:14:02.413 --> 00:14:04.757
and the types of competencies.

290
00:14:04.757 --> 00:14:06.313
You can see they're calling this a

291
00:14:06.313 --> 00:14:08.973
mixed-methods study.

292
00:14:08.973 --> 00:14:10.736
So now the words mixed methods

293
00:14:10.736 --> 00:14:12.908
are beginning to flow into

294
00:14:12.908 --> 00:14:15.856
the social work articles.

295
00:14:15.856 --> 00:14:19.143
This one looks at
school-based social workers,

296
00:14:19.143 --> 00:14:20.721
and really compares those that

297
00:14:20.721 --> 00:14:24.332
are school-hired versus agency-hired,

298
00:14:24.332 --> 00:14:26.886
in terms of their competencies.

299
00:14:26.886 --> 00:14:28.604
You start reading this.

300
00:14:28.604 --> 00:14:32.227
First of all, it's described
as a mixed-methods study.

301
00:14:32.227 --> 00:14:34.582
They gathered survey
data and they gathered

302
00:14:34.582 --> 00:14:37.300
focus group data, and then they compared

303
00:14:37.300 --> 00:14:39.342
those two databases.

304
00:14:39.342 --> 00:14:43.482
This is a fairly recent article, in 2011.

305
00:14:44.601 --> 00:14:46.320
I began finding some

306
00:14:46.320 --> 00:14:49.026
mixed-methods studies in social work.

307
00:14:49.026 --> 00:14:52.352
This research is being
done right in your field.

308
00:14:55.190 --> 00:14:57.047
Just a little bit about

309
00:14:57.047 --> 00:14:58.766
the development of this.

310
00:14:58.766 --> 00:15:02.387
It's often called the third
methodological movement.

311
00:15:02.387 --> 00:15:05.476
The first was quantitative,
the second was qualitative,

312
00:15:05.476 --> 00:15:07.179
and now mixed methods.

313
00:15:08.332 --> 00:15:10.444
Depending on when

314
00:15:10.444 --> 00:15:12.664
you view quantitative
coming in, it definitely

315
00:15:12.664 --> 00:15:15.520
came in around the turn of the century.

316
00:15:15.520 --> 00:15:18.400
Qualitative, you could argue it stretched

317
00:15:18.400 --> 00:15:21.140
way back to

318
00:15:21.140 --> 00:15:24.042
some of the early anthropologists.

319
00:15:24.042 --> 00:15:26.736
In the social sciences,

320
00:15:26.736 --> 00:15:29.161
we often associate qualitative with

321
00:15:29.161 --> 00:15:32.621
the Chicago school case study work

322
00:15:32.621 --> 00:15:35.013
of the 1950s and '60s.

323
00:15:35.013 --> 00:15:38.415
But mixed methods is a much more recent

324
00:15:38.415 --> 00:15:39.528
development.

325
00:15:41.052 --> 00:15:43.037
In 2009, one of my students

326
00:15:43.037 --> 00:15:45.753
did an analysis of doctoral dissertations

327
00:15:45.753 --> 00:15:47.355
using ProQuest.

328
00:15:47.355 --> 00:15:49.434
As you can see here, dissertations

329
00:15:49.434 --> 00:15:52.185
that had mixed methods in the title,

330
00:15:52.185 --> 00:15:55.645
it increased from three, from 1984,

331
00:15:55.645 --> 00:15:58.374
to over 2,500

332
00:15:58.374 --> 00:16:00.373
in 2009.

333
00:16:00.373 --> 00:16:02.173
One of my colleagues, Plano Clark,

334
00:16:02.173 --> 00:16:03.796
says that the number now is over

335
00:16:03.796 --> 00:16:07.002
6,000 dissertations.

336
00:16:07.002 --> 00:16:09.823
Plano Clark also mapped out

337
00:16:09.823 --> 00:16:11.971
the NIH funding,

338
00:16:11.971 --> 00:16:14.154
National Institute of
Health funding awards,

339
00:16:14.154 --> 00:16:16.614
that self-identify as mixed methods.

340
00:16:16.614 --> 00:16:20.121
You can see it's an upward climb here

341
00:16:20.121 --> 00:16:22.465
to the number of 226

342
00:16:22.465 --> 00:16:24.636
by 2008.

343
00:16:24.636 --> 00:16:26.899
It's still tending to climb.

344
00:16:26.899 --> 00:16:28.930
So there's interest, tremendous interest

345
00:16:28.930 --> 00:16:31.589
across the country in
different disciplines

346
00:16:31.589 --> 00:16:33.802
for mixed-methods research.

347
00:16:35.629 --> 00:16:38.024
It's only 25 years old.

348
00:16:38.024 --> 00:16:40.169
I roughly tag this back to the

349
00:16:40.169 --> 00:16:44.012
1985, 1990 period.

350
00:16:44.012 --> 00:16:45.708
We had different people working.

351
00:16:45.708 --> 00:16:48.145
Alan Bryman was working over in England.

352
00:16:48.145 --> 00:16:50.607
He wrote a book on quantity and quality.

353
00:16:52.047 --> 00:16:55.023
Jennifer Greene, out
of the evaluation field

354
00:16:55.023 --> 00:16:57.556
at Illinois, wrote an article

355
00:16:57.556 --> 00:16:59.600
in 1989.

356
00:16:59.600 --> 00:17:01.621
There were about eight people,

357
00:17:01.621 --> 00:17:05.858
including myself, during
this 1985, '90 time period,

358
00:17:05.858 --> 00:17:07.482
that were beginning to think about how

359
00:17:07.482 --> 00:17:10.501
we could put qualitative
and quantitative together.

360
00:17:10.501 --> 00:17:12.801
I felt it was just a matter of time before

361
00:17:12.801 --> 00:17:15.732
it would develop into a
full-blown methodology.

362
00:17:17.245 --> 00:17:20.857
This book came out in 1998,
called "Mixed Methodology."

363
00:17:22.232 --> 00:17:24.135
The handbook on mixed-methods research

364
00:17:24.135 --> 00:17:27.876
from Sage first came out in 2003.

365
00:17:29.283 --> 00:17:32.170
That's a little over a decade ago.

366
00:17:33.940 --> 00:17:36.007
Then in 2006,

367
00:17:36.007 --> 00:17:39.385
the large, international Mixed
Methods Conference started.

368
00:17:39.385 --> 00:17:43.039
It was sponsored by a
couple of universities

369
00:17:43.039 --> 00:17:44.860
in England.

370
00:17:44.860 --> 00:17:46.730
The "Journal of Mixed Methods" came in

371
00:17:46.730 --> 00:17:48.428
in 2007.

372
00:17:49.981 --> 00:17:52.891
In 2011, NIH developed some

373
00:17:52.891 --> 00:17:56.931
best-practice standards
for mixed-methods research

374
00:17:56.931 --> 00:17:59.947
across all 17 institutes

375
00:17:59.947 --> 00:18:02.131
and 11 centers,

376
00:18:02.131 --> 00:18:04.011
talking about how to do good

377
00:18:04.011 --> 00:18:05.984
mixed-methods research.

378
00:18:05.984 --> 00:18:09.397
Then the latest event to unfold

379
00:18:09.397 --> 00:18:11.465
is happening right now.

380
00:18:11.465 --> 00:18:12.255
There's the

381
00:18:12.255 --> 00:18:14.497
International Association
of Mixed-Methods Research

382
00:18:14.497 --> 00:18:17.794
that's forming that'll
be interdisciplinary

383
00:18:17.794 --> 00:18:19.542
and it'll be worldwide.

384
00:18:21.079 --> 00:18:23.235
Those are some of the major events.

385
00:18:24.923 --> 00:18:28.097
The characteristics of this methodology.

386
00:18:28.097 --> 00:18:30.652
I think there are only just a few

387
00:18:30.652 --> 00:18:32.662
that are key ones to look at.

388
00:18:34.634 --> 00:18:36.641
You ask quantitative, qualitative

389
00:18:36.641 --> 00:18:37.943
and mixed-methods

390
00:18:37.943 --> 00:18:40.822
research questions

391
00:18:40.822 --> 00:18:44.440
using open- and close-ended
or mixed information.

392
00:18:45.639 --> 00:18:47.476
Collecting and analyzing both quantitative

393
00:18:47.476 --> 00:18:49.408
and qualitative data.

394
00:18:50.760 --> 00:18:54.180
Using rigorous methods of data collection.

395
00:18:55.376 --> 00:18:58.809
Integrating or bringing
the two databases together.

396
00:18:58.809 --> 00:19:01.606
We're gonna talk about some of these ways.

397
00:19:01.606 --> 00:19:04.776
Using a specific mixed-methods design,

398
00:19:06.586 --> 00:19:08.530
and often framing the study within

399
00:19:08.530 --> 00:19:12.891
some larger theory or some
philosophical assumptions.

400
00:19:14.929 --> 00:19:17.308
You have quantitative
research and qualitative.

401
00:19:17.308 --> 00:19:19.490
We're gonna talk about integrating them,

402
00:19:19.490 --> 00:19:21.964
bringing the two together.

403
00:19:21.964 --> 00:19:23.623
Through that process of integration,

404
00:19:23.623 --> 00:19:26.432
we're going to have stronger,
a better understanding

405
00:19:26.432 --> 00:19:28.847
of our research problem
that what either one

406
00:19:28.847 --> 00:19:30.897
would yield in itself.

407
00:19:34.524 --> 00:19:36.209
Some people will come up say to me,

408
00:19:36.209 --> 00:19:39.771
"I've been doing mixed
methods since 1970, John."

409
00:19:39.771 --> 00:19:41.814
What they've been doing
is they've been gathering

410
00:19:41.814 --> 00:19:44.346
qualitative and quantitative data,

411
00:19:44.346 --> 00:19:46.633
usually very informally,

412
00:19:46.633 --> 00:19:48.212
not using some of the features I'm gonna

413
00:19:48.212 --> 00:19:50.128
show you here that have evolved in this

414
00:19:50.128 --> 00:19:52.126
methodology of mixed methods,

415
00:19:53.536 --> 00:19:55.731
often not seriously thinking about how

416
00:19:55.731 --> 00:19:58.297
they're gonna integrate
the two forms of data.

417
00:20:00.629 --> 00:20:03.005
It's more informal approaches

418
00:20:03.005 --> 00:20:06.811
rather than mixed methods,
as we see it today.

419
00:20:06.811 --> 00:20:08.987
Mixed methods is not just

420
00:20:08.987 --> 00:20:11.285
taking your qualitative data and reporting

421
00:20:11.285 --> 00:20:14.733
the results, and your
quantitative and reporting 'em,

422
00:20:14.733 --> 00:20:16.858
and keeping 'em separate.

423
00:20:16.858 --> 00:20:19.610
We're gonna talk about
integrating the two,

424
00:20:19.610 --> 00:20:21.059
that step.

425
00:20:22.513 --> 00:20:24.245
There are a lot of people
that are just using

426
00:20:24.245 --> 00:20:27.646
the word mixed methods

427
00:20:27.646 --> 00:20:30.140
because it's a popular concept.

428
00:20:30.140 --> 00:20:33.043
Somebody mentioned a
software program today

429
00:20:33.043 --> 00:20:36.491
where they're billing it as mixed methods,

430
00:20:36.491 --> 00:20:38.268
but they're really not doing much in terms

431
00:20:38.268 --> 00:20:41.095
of the methodology that
I'm gonna be talking about.

432
00:20:43.027 --> 00:20:44.953
There's a big controversy around the world

433
00:20:44.953 --> 00:20:47.377
right now, especially in England,

434
00:20:48.611 --> 00:20:51.088
about whether mixed methods is collecting

435
00:20:52.422 --> 00:20:55.126
multiple quantitative forms of data

436
00:20:55.126 --> 00:20:58.040
or multiple qualitative forms of data.

437
00:20:58.040 --> 00:21:01.105
It's certainly multiple research.

438
00:21:01.105 --> 00:21:03.124
We call this multi-method research,

439
00:21:03.124 --> 00:21:04.795
not mixed methods.

440
00:21:06.688 --> 00:21:09.304
If you go to the ICQI Conference,

441
00:21:09.304 --> 00:21:12.078
they are collecting
multiple qualitative forms,

442
00:21:12.078 --> 00:21:14.923
such as interviews and observations,

443
00:21:14.923 --> 00:21:17.105
but they're not collecting
quantitative data,

444
00:21:17.105 --> 00:21:18.986
so it's really not mixed methods.

445
00:21:18.986 --> 00:21:21.210
It's more multi-method research.

446
00:21:23.259 --> 00:21:24.501
There's this whole field out there

447
00:21:24.501 --> 00:21:26.684
called content analysis.

448
00:21:26.684 --> 00:21:28.146
It's where you do some interviews,

449
00:21:28.146 --> 00:21:31.560
and then you analyze
the data quantitatively.

450
00:21:31.560 --> 00:21:33.637
You count how many times people

451
00:21:33.637 --> 00:21:35.772
mention a code or a theme.

452
00:21:37.864 --> 00:21:40.979
In this scenario, you can see

453
00:21:40.979 --> 00:21:42.836
you're collecting qualitative data,

454
00:21:42.836 --> 00:21:45.973
but you're not collecting
quantitative data.

455
00:21:47.347 --> 00:21:48.856
When those articles came in to our

456
00:21:48.856 --> 00:21:50.214
"Journal of Mixed Methods Research,"

457
00:21:50.214 --> 00:21:52.188
they were rejected.

458
00:21:52.188 --> 00:21:54.232
We recommended that they be published

459
00:21:54.232 --> 00:21:56.833
in a content analysis type of journal.

460
00:21:58.100 --> 00:21:59.433
There are a lotta people
around the country

461
00:21:59.433 --> 00:22:01.362
that thinks mixed methods
is just summative,

462
00:22:01.362 --> 00:22:03.115
informative evaluation.

463
00:22:03.115 --> 00:22:06.679
Certainly it could be, but
it's much more than that.

464
00:22:06.679 --> 00:22:08.641
It's how you would actually

465
00:22:08.641 --> 00:22:10.904
integrate your qualitative
and quantitative data

466
00:22:10.904 --> 00:22:14.150
at both that summative and
at the formative stage.

467
00:22:17.557 --> 00:22:20.320
So what's emerged in the last 25 years

468
00:22:20.320 --> 00:22:22.955
are a series of strategies.

469
00:22:22.955 --> 00:22:24.778
I call these strategies that are developed

470
00:22:24.778 --> 00:22:28.001
in the science of a rigorous
mixed-methods study.

471
00:22:29.354 --> 00:22:30.849
If you were to do a mixed-methods study

472
00:22:30.849 --> 00:22:32.568
here at Smith, I would be going through

473
00:22:32.568 --> 00:22:34.845
this checklist of 10 features

474
00:22:34.845 --> 00:22:37.305
to see if you've incorporated them.

475
00:22:37.305 --> 00:22:38.942
I think if you have, it would probably

476
00:22:38.942 --> 00:22:42.419
be published in one of
the leading journals.

477
00:22:44.050 --> 00:22:46.274
(hum)

478
00:22:47.440 --> 00:22:51.161
Asking, am I getting ahead here?

479
00:22:51.161 --> 00:22:53.100
Here I go.

480
00:22:53.100 --> 00:22:54.678
Asking questions.

481
00:22:54.678 --> 00:22:57.198
We'll talk about framing within theory,

482
00:22:57.198 --> 00:22:59.334
writing a rationale,
collecting qualitative

483
00:22:59.334 --> 00:23:01.818
and quantitative data
using rigorous procedures,

484
00:23:01.818 --> 00:23:03.432
selecting a mixed-methods design,

485
00:23:03.432 --> 00:23:05.952
drawing a visual of it,
adding in procedures

486
00:23:05.952 --> 00:23:09.609
and integration,
specifying validity threats

487
00:23:09.609 --> 00:23:12.079
and creating joint displays of the data.

488
00:23:14.257 --> 00:23:16.463
Are you tired yet? (laughter)

489
00:23:16.463 --> 00:23:18.645
I'll go through these very quickly

490
00:23:18.645 --> 00:23:20.002
and give you some illustrations.

491
00:23:20.002 --> 00:23:22.080
In fact, the most important thing for me

492
00:23:22.080 --> 00:23:25.442
is to adapt this to the
field of social work.

493
00:23:25.442 --> 00:23:27.439
What I always do in my presentations

494
00:23:27.439 --> 00:23:30.249
is I go into the field
that I'm presenting to,

495
00:23:30.249 --> 00:23:33.012
and I look for good mixed-methods studies.

496
00:23:33.012 --> 00:23:34.718
I looked for quite a while.

497
00:23:34.718 --> 00:23:36.805
I found several that I thought

498
00:23:36.805 --> 00:23:39.127
were good mixed-methods projects.

499
00:23:39.127 --> 00:23:41.611
This is one that comes out of Connecticut.

500
00:23:41.611 --> 00:23:44.351
Anne Farrell and Preston.

501
00:23:44.351 --> 00:23:46.859
I understand Preston Britner was here

502
00:23:46.859 --> 00:23:48.239
on the faculty

503
00:23:49.691 --> 00:23:52.353
a few years ago.

504
00:23:52.353 --> 00:23:54.050
He or she

505
00:23:54.050 --> 00:23:55.929
said that he had been at Connecticut

506
00:23:55.929 --> 00:23:57.926
for about 16 years, but prior to that,

507
00:23:57.926 --> 00:24:00.371
they were here in Smith teaching.

508
00:24:01.711 --> 00:24:03.639
This is a good mixed-methods study.

509
00:24:03.639 --> 00:24:07.076
It's a very recent one, 2012.

510
00:24:07.076 --> 00:24:10.404
I found it in the "Child and
Family Social Work" journal.

511
00:24:11.751 --> 00:24:13.161
Basically, what they're doing here

512
00:24:13.161 --> 00:24:15.913
is they're talking about
a program for families

513
00:24:15.913 --> 00:24:18.153
with a dual vulnerability in housing

514
00:24:18.153 --> 00:24:19.836
and child welfare.

515
00:24:21.299 --> 00:24:24.785
They're very interested
in how parents react.

516
00:24:26.220 --> 00:24:28.169
They used a mixed-methods approach.

517
00:24:28.169 --> 00:24:30.224
They gathered quantitative data using

518
00:24:30.224 --> 00:24:32.279
an engagement measure,

519
00:24:32.279 --> 00:24:35.635
which they labeled
right here quantitative.

520
00:24:35.635 --> 00:24:38.410
I hope I'm not in the way of anyone here.

521
00:24:38.410 --> 00:24:41.533
Then they used open-ended
interviews, qualitative.

522
00:24:41.533 --> 00:24:44.413
They gathered data from 41 parents.

523
00:24:44.413 --> 00:24:47.313
They then converged these two databases.

524
00:24:48.509 --> 00:24:51.673
Then compared the two in terms

525
00:24:51.673 --> 00:24:55.327
of high and low engagement scores.

526
00:24:55.327 --> 00:24:57.659
A mixed-methods project, and it would fit

527
00:24:57.659 --> 00:25:00.319
those characteristics that
I talked about earlier,

528
00:25:00.319 --> 00:25:02.118
gathering quantitative data, gathering

529
00:25:02.118 --> 00:25:04.910
qualitative, integrating.

530
00:25:06.715 --> 00:25:08.844
I'm gonna use this as an illustration.

531
00:25:10.363 --> 00:25:13.157
I'm gonna present first
their research questions.

532
00:25:13.157 --> 00:25:15.478
The point that I'm trying
to make in mixed methods,

533
00:25:15.478 --> 00:25:17.684
what we typically do in mixed methods,

534
00:25:17.684 --> 00:25:20.667
is we state both a quantitative,

535
00:25:20.667 --> 00:25:23.734
a qualitative and a
mixed-methods question.

536
00:25:25.544 --> 00:25:28.191
The first question was, what is the level

537
00:25:28.191 --> 00:25:31.395
and quality of client engagement

538
00:25:31.395 --> 00:25:35.772
in this particular program

539
00:25:35.772 --> 00:25:39.243
for individuals that are,

540
00:25:39.243 --> 00:25:41.911
welfare families that
are looking for housing?

541
00:25:43.297 --> 00:25:46.129
Would you call this
quantitative, qualitative

542
00:25:46.129 --> 00:25:47.879
or mixed methods?

543
00:25:51.354 --> 00:25:52.529
- [Voiceover] - Qualitative.

544
00:25:52.529 --> 00:25:54.507
- What is it?
- [Voiceover] Qualitative.

545
00:25:54.507 --> 00:25:56.846
- Qualitative.

546
00:25:56.846 --> 00:25:57.810
Okay, let's see.

547
00:25:57.810 --> 00:26:00.364
We have two more response
possibilities here.

548
00:26:00.364 --> 00:26:02.648
(laughter)

549
00:26:02.648 --> 00:26:04.656
- [Voiceover] You're measuring levels.

550
00:26:04.656 --> 00:26:06.661
- Yeah, yeah, it's a measurement type

551
00:26:06.661 --> 00:26:07.892
of question, isn't it?

552
00:26:07.892 --> 00:26:10.701
Looking at the level and quality

553
00:26:10.701 --> 00:26:12.977
of client engagement.

554
00:26:12.977 --> 00:26:15.612
Here's the variable being measured,

555
00:26:15.612 --> 00:26:17.075
and they're gonna measure that

556
00:26:17.075 --> 00:26:19.367
in terms of level and quality.

557
00:26:21.660 --> 00:26:24.819
The second question they
posed in this article,

558
00:26:24.819 --> 00:26:26.897
are there trends in engagement across

559
00:26:26.897 --> 00:26:28.881
different program enrollments?

560
00:26:28.881 --> 00:26:31.239
So people enrolled in this
program at different times,

561
00:26:31.239 --> 00:26:32.794
and after the program is all over,

562
00:26:32.794 --> 00:26:35.013
then they're doing their study now.

563
00:26:36.370 --> 00:26:38.749
Quantitative, qualitative or mixed?

564
00:26:44.686 --> 00:26:46.214
- [Voiceover] Quantitative.

565
00:26:46.214 --> 00:26:47.630
- Quantitative.

566
00:26:47.630 --> 00:26:49.719
Why did you say that?

567
00:26:49.719 --> 00:26:51.796
- [Voiceover] Because the
trends are being measured.

568
00:26:51.796 --> 00:26:53.456
- Right.

569
00:26:53.456 --> 00:26:56.981
Usually in quantitative,
we assess trends, don't we?

570
00:26:56.981 --> 00:26:59.024
So knowing some of the terms.

571
00:27:01.068 --> 00:27:03.465
How do clients perceive their relationship

572
00:27:03.465 --> 00:27:05.334
with their case managers, especially

573
00:27:05.334 --> 00:27:07.552
with respect to engagement?

574
00:27:07.552 --> 00:27:09.386
- [Audience] Qualitative.

575
00:27:09.386 --> 00:27:11.940
- That's an easy one,
isn't it? Qualitative.

576
00:27:11.940 --> 00:27:13.843
Yes, good, qualitative.

577
00:27:13.843 --> 00:27:15.271
Now look at this one.

578
00:27:16.758 --> 00:27:18.876
Do quantitative measures of engagement

579
00:27:18.876 --> 00:27:22.000
match client narrative reports?

580
00:27:22.000 --> 00:27:23.286
- [Audience] Mixed.

581
00:27:23.286 --> 00:27:24.670
- Mixed.

582
00:27:26.445 --> 00:27:28.813
They're looking at both the measures

583
00:27:28.813 --> 00:27:31.413
and all of these narrative reports.

584
00:27:31.413 --> 00:27:33.874
In fact, that's a very good

585
00:27:33.874 --> 00:27:36.928
mixed-methods question, and it was

586
00:27:36.928 --> 00:27:39.331
probably clear, what you've learned

587
00:27:39.331 --> 00:27:41.419
about mixed methods up to this point.

588
00:27:42.988 --> 00:27:44.350
Here's their last one.

589
00:27:44.350 --> 00:27:45.941
Does the parent engagement measure

590
00:27:45.941 --> 00:27:48.341
effectively capture client engagement?

591
00:27:50.410 --> 00:27:53.035
Quantitative, qualitative or mixed?

592
00:27:53.035 --> 00:27:55.014
- [Audience] Mixed.

593
00:28:00.791 --> 00:28:03.120
- I'm gonna give ya a little hint on this.

594
00:28:03.120 --> 00:28:05.238
Construct validity.

595
00:28:06.730 --> 00:28:09.035
They've studied that in the courses.

596
00:28:11.071 --> 00:28:12.802
This is quantitative.

597
00:28:12.802 --> 00:28:14.706
Is this a valid instrument?

598
00:28:14.706 --> 00:28:17.485
Does it have a good construct validity?

599
00:28:17.485 --> 00:28:18.784
- [Voiceover] Just curious.

600
00:28:18.784 --> 00:28:20.677
I'm really very new at
this, but I was wondering

601
00:28:20.677 --> 00:28:23.032
if number one could also,

602
00:28:23.032 --> 00:28:25.405
I got thrown off by the
quality of client engagement.

603
00:28:25.405 --> 00:28:26.846
- I know.

604
00:28:26.846 --> 00:28:28.586
- [Voiceover] I wondered if
that could be qualitative

605
00:28:28.586 --> 00:28:31.443
because who's deciding
what the quality is?

606
00:28:31.443 --> 00:28:33.549
- They probably had
some measure of quality,

607
00:28:33.549 --> 00:28:35.187
would be my guess.

608
00:28:35.187 --> 00:28:37.914
That word, but that's important.

609
00:28:37.914 --> 00:28:39.772
You're looking at the wording

610
00:28:39.772 --> 00:28:42.709
in these questions to try to decipher

611
00:28:42.709 --> 00:28:45.054
whether it's quantitative or qualitative.

612
00:28:45.054 --> 00:28:47.528
Just because they say the word quality

613
00:28:47.528 --> 00:28:50.440
doesn't mean it's gonna be qualitative.

614
00:28:50.440 --> 00:28:53.183
They're trying to measure quality.

615
00:28:53.183 --> 00:28:54.738
In fact,

616
00:28:54.738 --> 00:28:57.362
here's a little primer
on research questions.

617
00:28:58.918 --> 00:29:01.791
What we hope to learn from
qualitative questions.

618
00:29:01.791 --> 00:29:05.017
Individual experiences, person meanings,

619
00:29:05.017 --> 00:29:07.259
individual perceptions.

620
00:29:07.259 --> 00:29:09.756
Let's look for key words in the questions

621
00:29:09.756 --> 00:29:12.889
such as meaning, experiences, exploration,

622
00:29:12.889 --> 00:29:14.551
individual views.

623
00:29:14.551 --> 00:29:16.026
There's probably some other ones,

624
00:29:16.026 --> 00:29:17.662
but you see that's how I was learning

625
00:29:17.662 --> 00:29:20.067
which ones were quantitative
and qualitative.

626
00:29:20.067 --> 00:29:22.794
I was looking at their
wording very carefully.

627
00:29:22.794 --> 00:29:24.907
There are words that we often associate

628
00:29:24.907 --> 00:29:26.606
with qualitative research.

629
00:29:28.272 --> 00:29:30.977
Quantitative questions.

630
00:29:30.977 --> 00:29:32.127
What we hope to learn

631
00:29:32.127 --> 00:29:34.924
is cause and effect,
probable cause and effect,

632
00:29:34.924 --> 00:29:37.165
relationships among variables,

633
00:29:37.165 --> 00:29:40.184
comparing groups, right?

634
00:29:40.184 --> 00:29:41.948
The words that are gonna tip us

635
00:29:41.948 --> 00:29:43.968
in that direction, factors, causes,

636
00:29:43.968 --> 00:29:46.407
measures, determinants, correlations,

637
00:29:46.407 --> 00:29:49.088
trends, level,

638
00:29:49.088 --> 00:29:50.968
magnitude.

639
00:29:50.968 --> 00:29:52.907
We could probably go on, but I was looking

640
00:29:52.907 --> 00:29:56.320
for those types of words to see

641
00:29:56.320 --> 00:29:58.954
whether they were really
trying to measure something.

642
00:29:58.954 --> 00:30:00.556
Your mixed-methods questions,

643
00:30:00.556 --> 00:30:03.534
as you've saw, as you perceptively

644
00:30:03.534 --> 00:30:05.891
identified, really combines both

645
00:30:05.891 --> 00:30:08.360
experiences and some type

646
00:30:08.360 --> 00:30:11.414
of quantitative measure,

647
00:30:11.414 --> 00:30:13.957
relationship among
variables or between groups.

648
00:30:13.957 --> 00:30:15.558
It's some combination

649
00:30:15.558 --> 00:30:17.353
of this and this.

650
00:30:21.863 --> 00:30:23.615
Number two.

651
00:30:23.615 --> 00:30:25.241
What we want in every mixed-method study

652
00:30:25.241 --> 00:30:26.912
would be qualitative, quantitative

653
00:30:26.912 --> 00:30:28.688
and mixed-methods questions.

654
00:30:28.688 --> 00:30:30.616
The second thing is we need to frame

655
00:30:30.616 --> 00:30:33.885
the study within some
theory or philosophy, often.

656
00:30:36.014 --> 00:30:38.674
This is from Crotty's book in 1998.

657
00:30:38.674 --> 00:30:40.206
He has a nice flow chart.

658
00:30:40.206 --> 00:30:41.761
He said, "You know, we have these

659
00:30:41.761 --> 00:30:43.781
"paradigms and worldviews,

660
00:30:43.781 --> 00:30:46.823
"epistemologies,
ontologies, methodologies,

661
00:30:46.823 --> 00:30:49.852
"that inform theoretical
lenses that we use,

662
00:30:51.722 --> 00:30:54.466
"that then inform our methodology,

663
00:30:54.466 --> 00:30:56.741
"such as mixed methods,

664
00:30:56.741 --> 00:30:58.982
"that then informs

665
00:30:58.982 --> 00:31:01.311
"our method of data collection."

666
00:31:05.566 --> 00:31:08.849
There are different worldviews out there.

667
00:31:08.849 --> 00:31:10.638
In fact, if you look at the handbook

668
00:31:10.638 --> 00:31:13.412
on qualitative research
from Denzin and Lincoln,

669
00:31:13.412 --> 00:31:15.699
and track, over the years, through their

670
00:31:15.699 --> 00:31:18.025
different volumes the worldviews,

671
00:31:18.025 --> 00:31:19.952
we know that they're changing.

672
00:31:19.952 --> 00:31:21.449
I think in the latest one they have

673
00:31:21.449 --> 00:31:24.270
critical theory as one of the worldviews.

674
00:31:24.270 --> 00:31:27.487
I especially wanted to bring in,

675
00:31:27.487 --> 00:31:30.145
we bring to research certain

676
00:31:30.145 --> 00:31:32.788
predispositions, certain perspectives

677
00:31:34.061 --> 00:31:36.394
that inform how we're
going to do our research.

678
00:31:38.135 --> 00:31:41.043
We may feel that there's
a theory that's important,

679
00:31:41.043 --> 00:31:43.925
and so we're most postpositivist.

680
00:31:43.925 --> 00:31:45.608
We're gonna work more deductively.

681
00:31:45.608 --> 00:31:47.907
We're gonna test out that theory.

682
00:31:47.907 --> 00:31:50.786
Or constructivist, working
with multiple realities,

683
00:31:50.786 --> 00:31:52.806
or more of a participatory

684
00:31:52.806 --> 00:31:55.509
where we wanna collaborate
with our participants.

685
00:31:56.613 --> 00:31:58.127
Or pragmatism,

686
00:31:59.473 --> 00:32:03.253
singular and multiple
realities, very practical,

687
00:32:03.253 --> 00:32:05.490
looking at what works.

688
00:32:05.490 --> 00:32:07.266
Starting with a question and asking,

689
00:32:07.266 --> 00:32:10.047
"What data can we collect to answer it?"

690
00:32:11.527 --> 00:32:13.129
So mixed-methods researchers have been

691
00:32:13.129 --> 00:32:15.776
very concerned about philosophy

692
00:32:15.776 --> 00:32:17.436
from the very beginning.

693
00:32:17.436 --> 00:32:19.015
We even went through a historical period

694
00:32:19.015 --> 00:32:20.779
they call the paradigm debate period

695
00:32:20.779 --> 00:32:22.660
of the '90s.

696
00:32:22.660 --> 00:32:25.017
What has surfaced now?

697
00:32:25.017 --> 00:32:26.677
There are a number of people that

698
00:32:26.677 --> 00:32:28.454
are talking about different worldviews

699
00:32:28.454 --> 00:32:31.959
that seem to influence
mixed-methods research,

700
00:32:31.959 --> 00:32:33.909
pragmatism.

701
00:32:33.909 --> 00:32:37.057
What works, the importance
of the question,

702
00:32:37.057 --> 00:32:40.133
looking at multiple
ways of gathering data.

703
00:32:40.133 --> 00:32:42.791
Donna Mertens down at Gallaudet

704
00:32:42.791 --> 00:32:45.280
brings in a transformative

705
00:32:45.280 --> 00:32:48.240
paradigm, where the objective of research

706
00:32:48.240 --> 00:32:50.760
is to begin to address
social justice issues

707
00:32:50.760 --> 00:32:53.221
and bring about change.

708
00:32:53.221 --> 00:32:55.588
Joe Maxwell brings in critical realism,

709
00:32:55.588 --> 00:32:57.807
which is both, that reality is both

710
00:32:57.807 --> 00:33:00.442
subjective and objective.

711
00:33:00.442 --> 00:33:02.625
The latest is Burke Johnson's work

712
00:33:02.625 --> 00:33:05.677
on dialectic pluralism, that we bring in

713
00:33:05.677 --> 00:33:07.660
multiple worldviews, and out of this

714
00:33:07.660 --> 00:33:10.419
comes some dialectic.

715
00:33:10.419 --> 00:33:12.291
That's a good social work term, isn't it?

716
00:33:12.291 --> 00:33:14.342
A dialectic development.

717
00:33:16.471 --> 00:33:18.399
Some people have said we can use

718
00:33:18.399 --> 00:33:20.128
several worldviews.

719
00:33:21.755 --> 00:33:24.476
My writing tends to flow along the lines

720
00:33:24.476 --> 00:33:28.029
of matching a worldview
to the type of design.

721
00:33:29.316 --> 00:33:32.024
For example, I'm gonna
start with a survey.

722
00:33:34.092 --> 00:33:35.874
I'm going to gather survey data.

723
00:33:35.874 --> 00:33:39.090
I'm using a postpositivist perspective.

724
00:33:39.090 --> 00:33:41.179
I followed up focus groups.

725
00:33:41.179 --> 00:33:43.548
I'm using more of a constructivist.

726
00:33:43.548 --> 00:33:45.963
I really have multiple paradigms operating

727
00:33:45.963 --> 00:33:49.085
at different phases in my design.

728
00:33:49.085 --> 00:33:50.919
If you go back to Thomas Kuhn's book,

729
00:33:50.919 --> 00:33:52.812
and I saw that it was on the bookshelf

730
00:33:52.812 --> 00:33:55.493
here at the Smith
Bookstore, so you're reading

731
00:33:55.493 --> 00:33:56.828
Thomas Kuhn.

732
00:33:56.828 --> 00:33:59.116
I went back and re-read that last year,

733
00:33:59.116 --> 00:34:01.496
but his idea is that worldview

734
00:34:01.496 --> 00:34:04.271
is really shaped by
the scholarly community

735
00:34:04.271 --> 00:34:06.092
that you're in.

736
00:34:06.092 --> 00:34:08.518
Is there a way that social workers

737
00:34:08.518 --> 00:34:10.529
think about problems?

738
00:34:10.529 --> 00:34:12.454
Are there accepted methods

739
00:34:12.454 --> 00:34:15.063
for how you go about gathering data?

740
00:34:17.097 --> 00:34:19.759
Are you socialized as graduate students

741
00:34:19.759 --> 00:34:21.949
in social work to

742
00:34:21.949 --> 00:34:24.004
buy into the community?

743
00:34:24.004 --> 00:34:27.407
That's exactly what Thomas
Kuhn was talking about.

744
00:34:30.322 --> 00:34:32.085
You can see there's multiple perspectives

745
00:34:32.085 --> 00:34:34.128
on philosophy

746
00:34:34.128 --> 00:34:35.847
within mixed-methods research.

747
00:34:35.847 --> 00:34:38.203
In terms of theory,

748
00:34:38.203 --> 00:34:40.665
we do use, in mixed-methods research,

749
00:34:40.665 --> 00:34:43.242
social science and behavioral theory

750
00:34:43.242 --> 00:34:45.285
to often inform our work.

751
00:34:45.285 --> 00:34:48.546
Sometimes we use transformative theories.

752
00:34:48.546 --> 00:34:52.309
Often we use community-based
participatory theories

753
00:34:52.309 --> 00:34:54.405
that are brought throughout the studies.

754
00:34:55.529 --> 00:34:57.477
For example, one of my doctoral students

755
00:34:57.477 --> 00:34:59.740
in 2009

756
00:34:59.740 --> 00:35:02.539
looked at the theoretical
lenses being used

757
00:35:02.539 --> 00:35:04.674
in the mixed-methods projects across

758
00:35:04.674 --> 00:35:06.740
a number of journals.

759
00:35:06.740 --> 00:35:08.249
Some of these are social work,

760
00:35:08.249 --> 00:35:11.095
"Family Relations,"
"Critical Public Health,"

761
00:35:11.095 --> 00:35:12.882
"Women and Health."

762
00:35:12.882 --> 00:35:15.261
You can see that feminist studies

763
00:35:15.261 --> 00:35:17.596
are very popular in the
mixed-methods field.

764
00:35:17.596 --> 00:35:20.974
There's a number that
address racial issues.

765
00:35:20.974 --> 00:35:23.350
Then it goes on down from there.

766
00:35:24.828 --> 00:35:26.920
Yes, these different

767
00:35:26.920 --> 00:35:29.637
theoretical lenses, both
social and advocacy,

768
00:35:29.637 --> 00:35:31.576
are being used.

769
00:35:31.576 --> 00:35:34.025
Returning to that Farrell article

770
00:35:34.025 --> 00:35:36.412
on the impact of this program.

771
00:35:37.798 --> 00:35:39.480
they do have a passage at the beginning

772
00:35:39.480 --> 00:35:42.375
where they talk about an
ecological model of housing.

773
00:35:43.578 --> 00:35:46.278
They refer to this author, and they refer

774
00:35:46.278 --> 00:35:48.902
to family-centered practices.

775
00:35:48.902 --> 00:35:51.933
So they have kind of a
conceptual orientation

776
00:35:51.933 --> 00:35:54.539
that flows into the
beginning of this project.

777
00:35:56.182 --> 00:35:59.642
Write a rationale for
mixed-methods research,

778
00:35:59.642 --> 00:36:02.765
why you're using both
quantitative and qualitative.

779
00:36:02.765 --> 00:36:04.587
It's a statement

780
00:36:04.587 --> 00:36:06.632
in a mixed-methods project
in which the author

781
00:36:06.632 --> 00:36:10.322
advances the reasons for
why, whoops, excuse me,

782
00:36:10.322 --> 00:36:12.459
the reasons for why it's important

783
00:36:12.459 --> 00:36:14.130
to collect it.

784
00:36:14.130 --> 00:36:16.326
Why are you doing mixed-methods research?

785
00:36:17.511 --> 00:36:18.701
Why is this important?

786
00:36:18.701 --> 00:36:20.370
Back when qualitative was new

787
00:36:20.370 --> 00:36:22.541
in the social sciences, you'd always find

788
00:36:22.541 --> 00:36:24.678
a reason documented as to why

789
00:36:24.678 --> 00:36:26.976
this was gonna be an ethnography.

790
00:36:26.976 --> 00:36:28.904
Still today, you have

791
00:36:28.904 --> 00:36:31.422
that pervading the qualitative.

792
00:36:31.422 --> 00:36:33.872
Mixed methods is newer yet, and so

793
00:36:33.872 --> 00:36:36.409
we need a rationale
for why we're doing it.

794
00:36:38.144 --> 00:36:40.142
You could say this is
the types of problems

795
00:36:40.142 --> 00:36:44.042
that are best suited for
a mixed-methods project.

796
00:36:44.042 --> 00:36:46.039
Think about your own
study that you're doing

797
00:36:46.039 --> 00:36:48.373
for a moment.

798
00:36:48.373 --> 00:36:50.219
One rationale for using mixed methods

799
00:36:50.219 --> 00:36:53.562
is you want different,
multiple perspectives,

800
00:36:53.562 --> 00:36:55.798
or to build a complete understanding.

801
00:36:57.858 --> 00:36:59.357
There are perspectives to be gained

802
00:36:59.357 --> 00:37:02.015
by administering a survey instrument.

803
00:37:02.015 --> 00:37:03.896
There are perspectives from focus groups

804
00:37:03.896 --> 00:37:06.392
or individual interviews.

805
00:37:06.392 --> 00:37:08.657
They'll give you different perspectives.

806
00:37:08.657 --> 00:37:10.595
One would be that in-depth perspective.

807
00:37:10.595 --> 00:37:14.023
The other would be a
more general impression.

808
00:37:15.912 --> 00:37:17.592
Sometimes we need to confirm our

809
00:37:17.592 --> 00:37:20.709
quantitative measures with
qualitative experiences.

810
00:37:22.213 --> 00:37:24.430
I was with the Massachusetts
Department of Health

811
00:37:24.430 --> 00:37:26.207
a few weeks ago.

812
00:37:26.207 --> 00:37:27.889
They were talking about one of the surveys

813
00:37:27.889 --> 00:37:29.469
that they administered across the state

814
00:37:29.469 --> 00:37:31.291
of Massachusetts.

815
00:37:31.291 --> 00:37:34.066
They were saying, "You
know, there were really

816
00:37:34.066 --> 00:37:36.144
"some interesting findings that came out

817
00:37:36.144 --> 00:37:38.129
"of that survey."

818
00:37:38.129 --> 00:37:41.268
Of course, I said, "Did you stop there?

819
00:37:42.749 --> 00:37:44.257
"Why didn't you follow up a little bit

820
00:37:44.257 --> 00:37:46.393
"with some qualitative interviews

821
00:37:46.393 --> 00:37:48.600
"or talk to some people?

822
00:37:48.600 --> 00:37:51.060
"Why didn't you confirm
your quantitative measures

823
00:37:51.060 --> 00:37:52.801
"with qualitative experiences to find out

824
00:37:52.801 --> 00:37:55.019
"what was really going on?"

825
00:37:55.019 --> 00:37:57.132
I had everyone in that room nodding.

826
00:37:57.132 --> 00:37:59.617
This would be a pretty good idea.

827
00:37:59.617 --> 00:38:02.774
I was framing it from a mixed methods.

828
00:38:02.774 --> 00:38:05.359
Needing to explain quantitative results.

829
00:38:05.359 --> 00:38:08.643
Sometimes these instruments that we have,

830
00:38:08.643 --> 00:38:10.549
we're trying to fit 'em to the people

831
00:38:10.549 --> 00:38:12.371
that we're studying.

832
00:38:12.371 --> 00:38:15.399
They're not very contextually sensitive.

833
00:38:16.565 --> 00:38:19.271
They may not fit the diversity

834
00:38:19.271 --> 00:38:21.456
of the people we're working with.

835
00:38:23.927 --> 00:38:26.848
Let's gather some qualitative data,

836
00:38:26.848 --> 00:38:28.775
and then develop some instruments

837
00:38:28.775 --> 00:38:30.773
or measures or interventions that might

838
00:38:30.773 --> 00:38:33.994
actually work with the
people we're studying.

839
00:38:33.994 --> 00:38:36.594
My friends in global
health down at Harvard

840
00:38:36.594 --> 00:38:38.858
are all using this idea.

841
00:38:38.858 --> 00:38:40.869
It's because my colleague

842
00:38:40.869 --> 00:38:42.735
that's going to Sierra Leone

843
00:38:42.735 --> 00:38:45.568
can't just drop a depression instrument

844
00:38:45.568 --> 00:38:47.252
into Sierra Leone.

845
00:38:47.252 --> 00:38:49.434
She's gotta start talking to some people

846
00:38:49.434 --> 00:38:51.431
and gathering qualitative data.

847
00:38:51.431 --> 00:38:54.937
Surprise, every one of
her mixed-methods projects

848
00:38:54.937 --> 00:38:56.969
begins with a strong qualitative part,

849
00:38:56.969 --> 00:39:00.011
and then builds into that quantitative.

850
00:39:00.011 --> 00:39:01.798
That quantitative is much better because

851
00:39:01.798 --> 00:39:04.046
of that qualitative.

852
00:39:06.210 --> 00:39:08.787
We need to enhance our experiments.

853
00:39:08.787 --> 00:39:10.993
There's been a lot of discussion about

854
00:39:10.993 --> 00:39:13.412
randomized controlled trials

855
00:39:13.412 --> 00:39:15.454
and the problems with that.

856
00:39:15.454 --> 00:39:17.846
We really don't understand the mechanisms.

857
00:39:19.263 --> 00:39:21.462
Why can't we put some qualitative data

858
00:39:21.462 --> 00:39:23.750
in, after the experiment is over,

859
00:39:23.750 --> 00:39:26.361
to understand why that treatment

860
00:39:26.361 --> 00:39:28.346
had such a positive impact,

861
00:39:28.346 --> 00:39:30.813
would explain why it had that impact.

862
00:39:32.217 --> 00:39:34.349
Or do we just stop with p-values,

863
00:39:34.349 --> 00:39:36.937
confidence intervals, effect sizes?

864
00:39:38.319 --> 00:39:41.539
Putting qualitative into an experiment.

865
00:39:41.539 --> 00:39:43.764
I'm gonna show you some models of that.

866
00:39:45.939 --> 00:39:48.517
When we're doing our community studies,

867
00:39:48.517 --> 00:39:50.421
tomorrow I'll be talking
to community researchers

868
00:39:50.421 --> 00:39:52.499
in the Boston area.

869
00:39:52.499 --> 00:39:54.495
They are gathering trend data

870
00:39:54.495 --> 00:39:56.339
and individual perspectives,

871
00:39:56.339 --> 00:39:57.595
keeping 'em separate.

872
00:39:57.595 --> 00:39:59.091
They're not thinking about how they might

873
00:39:59.091 --> 00:40:01.194
be linked to get a better understanding

874
00:40:01.194 --> 00:40:04.703
of what those community
members' needs might be.

875
00:40:04.703 --> 00:40:06.673
Sometimes we need to evaluate success

876
00:40:06.673 --> 00:40:09.934
of a program by using a needs assessment.

877
00:40:09.934 --> 00:40:12.720
Then actually do a quantitative test

878
00:40:12.720 --> 00:40:14.858
to see the success of the program.

879
00:40:14.858 --> 00:40:17.655
Again, we're putting qualitative

880
00:40:17.655 --> 00:40:19.710
together with quantitative.

881
00:40:19.710 --> 00:40:22.379
Can you see these rationales for why

882
00:40:22.379 --> 00:40:24.848
mixed methods is popular?

883
00:40:28.950 --> 00:40:31.098
I looked at the Farrell article

884
00:40:31.098 --> 00:40:33.536
on engagement in the supportive housing

885
00:40:33.536 --> 00:40:35.498
for family program.

886
00:40:35.498 --> 00:40:36.937
They had a few comments.

887
00:40:36.937 --> 00:40:39.715
"Research has begun to
include parent voices."

888
00:40:41.281 --> 00:40:43.506
Then they said, "We then
compared and contrast

889
00:40:43.506 --> 00:40:46.179
"quantitative and qualitative findings."

890
00:40:48.154 --> 00:40:50.085
Had they studied my list, maybe they

891
00:40:50.085 --> 00:40:52.628
could think about what
would be a good rationale

892
00:40:52.628 --> 00:40:55.553
for this, such as "a
complete understanding

893
00:40:55.553 --> 00:40:58.106
"of the parents'
experiences with the program

894
00:40:58.106 --> 00:41:00.464
"can be attained through the combination

895
00:41:00.464 --> 00:41:02.867
"of parent voices and
assessment of measures

896
00:41:02.867 --> 00:41:05.049
"of their engagement."

897
00:41:05.049 --> 00:41:07.023
You see how I gave a rationale for both

898
00:41:07.023 --> 00:41:09.171
quantitative and qualitative here?

899
00:41:09.171 --> 00:41:11.366
Can be assessed through the voices,

900
00:41:11.366 --> 00:41:14.290
qualitative, and the
assessment of measures,

901
00:41:14.290 --> 00:41:15.773
quantitative.

902
00:41:18.109 --> 00:41:19.620
So whenever an article would come into

903
00:41:19.620 --> 00:41:21.222
our "Journal of Mixed Methods Research,"

904
00:41:21.222 --> 00:41:23.858
we'd be looking for a rationale statement.

905
00:41:23.858 --> 00:41:25.518
It's very important.

906
00:41:25.518 --> 00:41:27.178
A lot of people will just drop

907
00:41:27.178 --> 00:41:28.897
quantitative instruments into a study,

908
00:41:28.897 --> 00:41:31.821
and a lot of different
qualitative stuff into a study.

909
00:41:33.239 --> 00:41:35.182
It just leads to extra effort

910
00:41:35.182 --> 00:41:37.787
and extra resources.

911
00:41:37.787 --> 00:41:40.411
If you think through your rationale,

912
00:41:40.411 --> 00:41:43.433
you're gonna be able to
limit your data collection.

913
00:41:43.433 --> 00:41:46.660
The fourth point, in qualitative
data, we wanna collect

914
00:41:46.660 --> 00:41:50.143
both quantitative and qualitative data,

915
00:41:50.143 --> 00:41:51.710
so I came upon this book,

916
00:41:51.710 --> 00:41:54.785
"Clinical Assessment for Social Workers,"

917
00:41:54.785 --> 00:41:57.663
which some of you are
probably reading this summer.

918
00:41:57.663 --> 00:41:59.417
I found it to be a very good book,

919
00:41:59.417 --> 00:42:01.004
very interesting.

920
00:42:02.301 --> 00:42:03.751
They talked about quantitative

921
00:42:03.751 --> 00:42:06.942
clinical assessment techniques,

922
00:42:06.942 --> 00:42:09.695
which I'm sure you're familiar with.

923
00:42:09.695 --> 00:42:13.255
Then qualitative clinical
assessment techniques,

924
00:42:13.255 --> 00:42:17.375
interviews, life stories,
individual case analysis.

925
00:42:18.537 --> 00:42:20.944
You know what I was
looking for in this book?

926
00:42:22.330 --> 00:42:24.188
Where they brought the two together.

927
00:42:24.188 --> 00:42:27.137
Towards the front, in all
fairness to these authors,

928
00:42:27.137 --> 00:42:28.554
they had what they called an

929
00:42:28.554 --> 00:42:30.818
integrated assessment checklist

930
00:42:30.818 --> 00:42:32.907
where they did have both of 'em there,

931
00:42:32.907 --> 00:42:34.845
but they never really gave us that thought

932
00:42:34.845 --> 00:42:37.916
about how you might combine

933
00:42:37.916 --> 00:42:41.038
these types of data to really feed

934
00:42:41.038 --> 00:42:44.077
into each other to gain greater insight.

935
00:42:46.217 --> 00:42:48.948
Maybe I'll write 'em and
say, "In your next edition."

936
00:42:48.948 --> 00:42:51.989
I thought it was an
excellent, very nice book.

937
00:42:51.989 --> 00:42:54.109
When we go to the Farrell study,

938
00:42:55.593 --> 00:42:57.621
they collected data

939
00:42:57.621 --> 00:43:00.262
from 41 parents.

940
00:43:02.601 --> 00:43:04.016
They administered that instrument

941
00:43:04.016 --> 00:43:05.815
to 41 parents.

942
00:43:05.815 --> 00:43:07.906
They administered a rating scale

943
00:43:07.906 --> 00:43:10.600
and they did on-on-one interviews.

944
00:43:10.600 --> 00:43:13.103
Yes, they are collecting
both forms of data

945
00:43:13.103 --> 00:43:14.978
in this project.

946
00:43:17.109 --> 00:43:19.524
The next thing is, in mixed methods,

947
00:43:19.524 --> 00:43:21.494
it needs to be rigorous.

948
00:43:23.261 --> 00:43:26.496
It needs to have some
rigor in your methods.

949
00:43:26.496 --> 00:43:29.051
I put together

950
00:43:29.051 --> 00:43:33.067
John's foolproof list of rigorous methods.

951
00:43:33.067 --> 00:43:35.413
I want something about sampling,

952
00:43:35.413 --> 00:43:37.527
recruitment, sample size,

953
00:43:37.527 --> 00:43:39.348
forms of data collection,

954
00:43:39.348 --> 00:43:41.310
methods of recording data,

955
00:43:41.310 --> 00:43:44.027
methods for transcribing,
storing the data,

956
00:43:44.027 --> 00:43:47.231
cleaning up the database,
some initial analysis,

957
00:43:47.231 --> 00:43:49.507
some advanced analysis.

958
00:43:49.507 --> 00:43:51.371
I actually put together
a checklist like this

959
00:43:51.371 --> 00:43:53.531
for my colleagues in
counseling psychology,

960
00:43:53.531 --> 00:43:56.526
which they're using as
they write articles,

961
00:43:56.526 --> 00:43:58.905
especially qualitative journal articles.

962
00:43:58.905 --> 00:44:01.263
It's not exhaustive list,

963
00:44:01.263 --> 00:44:03.769
but it's a fair list of rigor.

964
00:44:03.769 --> 00:44:05.650
If you were to put this in on both

965
00:44:05.650 --> 00:44:09.110
the quantitative and qualitative
side of your project,

966
00:44:09.110 --> 00:44:12.134
I think you'd have a great
doctoral dissertation.

967
00:44:13.510 --> 00:44:15.205
Then I looked at the Farrell article

968
00:44:15.205 --> 00:44:16.725
in terms of its rigor.

969
00:44:20.127 --> 00:44:22.918
They did have information about sampling,

970
00:44:22.918 --> 00:44:25.541
but it was non-probabilistic sampling.

971
00:44:25.541 --> 00:44:27.318
They'd had some purposeful sampling

972
00:44:27.318 --> 00:44:29.291
over on the qualitative side.

973
00:44:29.291 --> 00:44:31.358
They had some recruitment techniques

974
00:44:31.358 --> 00:44:33.297
on both sides.

975
00:44:33.297 --> 00:44:35.874
They had the N of 41.

976
00:44:35.874 --> 00:44:38.045
They had a questionnaire with 22 items

977
00:44:38.045 --> 00:44:40.019
and open-ended questions.

978
00:44:41.632 --> 00:44:42.927
They'd (mumbles) methods.

979
00:44:42.927 --> 00:44:45.888
They didn't talk about
an interview protocol.

980
00:44:45.888 --> 00:44:47.863
I would have put that in that article.

981
00:44:49.364 --> 00:44:51.938
They inputted this into SPSS.

982
00:44:51.938 --> 00:44:55.299
They recorded the interviews
and transcribed 'em.

983
00:44:56.453 --> 00:44:59.437
They talked about
cleaning up the database.

984
00:44:59.437 --> 00:45:02.804
They then reported mean engagement scores.

985
00:45:02.804 --> 00:45:05.450
They talked about Pearson correlations,

986
00:45:05.450 --> 00:45:07.630
site comparisons.

987
00:45:07.630 --> 00:45:11.412
They had a nice table on
themes, qualitative themes.

988
00:45:12.695 --> 00:45:14.668
They mentioned that they
used grounded theory,

989
00:45:14.668 --> 00:45:17.977
but I don't think they really
used much grounded theory.

990
00:45:17.977 --> 00:45:20.662
But kudos to them

991
00:45:20.662 --> 00:45:22.983
for acknowledging the fact

992
00:45:22.983 --> 00:45:25.154
that there are some larger strategies

993
00:45:25.154 --> 00:45:27.267
of qualitative research out there.

994
00:45:27.267 --> 00:45:29.972
When I looked down through this list,

995
00:45:29.972 --> 00:45:33.908
they've got some really
good, rigorous procedures.

996
00:45:33.908 --> 00:45:35.359
If you were to read that article,

997
00:45:35.359 --> 00:45:38.215
which was available out in the lobby,

998
00:45:38.215 --> 00:45:40.896
some of you may have gotten a copy of it,

999
00:45:40.896 --> 00:45:42.976
you will see, I think,

1000
00:45:42.976 --> 00:45:46.458
very good, sophisticated, rigorous methods

1001
00:45:46.458 --> 00:45:48.925
in a social work article.

1002
00:45:52.634 --> 00:45:54.913
Now, select a mixed-methods design.

1003
00:45:57.301 --> 00:45:59.530
There are a number of designs out there.

1004
00:46:01.664 --> 00:46:03.808
It can be very confusing
to read some of the books,

1005
00:46:03.808 --> 00:46:06.421
but I talk about them in terms of basic

1006
00:46:06.421 --> 00:46:08.848
and advanced designs.

1007
00:46:08.848 --> 00:46:10.298
We're gonna look for the overall intent

1008
00:46:10.298 --> 00:46:12.445
of your design.

1009
00:46:12.445 --> 00:46:14.558
We're gonna start with a basic, and add in

1010
00:46:14.558 --> 00:46:17.020
more advanced if we need that,

1011
00:46:17.020 --> 00:46:18.657
and we're gonna consider some procedures

1012
00:46:18.657 --> 00:46:20.978
for the design.

1013
00:46:20.978 --> 00:46:22.662
What has happened in
the mixed-methods field

1014
00:46:22.662 --> 00:46:24.787
is that we have these different ways

1015
00:46:24.787 --> 00:46:26.900
of doing mixed-methods research.

1016
00:46:26.900 --> 00:46:29.163
Just like in quantitative, we have

1017
00:46:29.163 --> 00:46:31.462
different ways of doing
quantitative research.

1018
00:46:31.462 --> 00:46:33.645
We can do a true experiment.

1019
00:46:33.645 --> 00:46:35.281
We can do a quasi-experiment.

1020
00:46:35.281 --> 00:46:37.940
We can do single-subject research.

1021
00:46:37.940 --> 00:46:39.647
In qualitative, we have different ways

1022
00:46:39.647 --> 00:46:42.026
of doing qualitative.

1023
00:46:42.026 --> 00:46:44.464
My book on the five approaches,

1024
00:46:44.464 --> 00:46:47.146
ethnography, narrative, case study,

1025
00:46:47.146 --> 00:46:49.364
phenomenology, et cetera,

1026
00:46:49.364 --> 00:46:50.722
different ways of doing it.

1027
00:46:50.722 --> 00:46:52.684
So it's no surprise,

1028
00:46:52.684 --> 00:46:54.901
as this methodology has evolved,

1029
00:46:54.901 --> 00:46:56.805
that we now have different designs

1030
00:46:56.805 --> 00:46:59.226
for doing mixed-methods research.

1031
00:47:02.842 --> 00:47:05.245
We've given these some names.

1032
00:47:05.245 --> 00:47:07.776
The first is convergent.

1033
00:47:07.776 --> 00:47:09.747
It's where you gather quantitative data

1034
00:47:09.747 --> 00:47:12.128
and analyze it, and qualitative data,

1035
00:47:12.128 --> 00:47:13.487
and you bring the two together

1036
00:47:13.487 --> 00:47:15.443
and just compare the results.

1037
00:47:18.327 --> 00:47:20.371
I'll have to say,

1038
00:47:20.371 --> 00:47:22.113
the social work articles I've found

1039
00:47:22.113 --> 00:47:23.784
that are mixed methods,

1040
00:47:23.784 --> 00:47:25.983
this is your most popular design,

1041
00:47:27.390 --> 00:47:30.956
but think about other possibilities.

1042
00:47:32.305 --> 00:47:34.710
We can start quantitatively, let's say

1043
00:47:34.710 --> 00:47:36.951
with a survey,

1044
00:47:36.951 --> 00:47:38.727
come up with some interesting results,

1045
00:47:38.727 --> 00:47:41.188
and then follow up
qualitatively to explain

1046
00:47:41.188 --> 00:47:43.355
those results in more detail.

1047
00:47:44.555 --> 00:47:46.692
That design right there

1048
00:47:46.692 --> 00:47:50.237
is a graduate student design, ready made.

1049
00:47:50.237 --> 00:47:52.501
In my mixed-methods class,

1050
00:47:52.501 --> 00:47:54.869
I'll have half my class choose this.

1051
00:47:54.869 --> 00:47:56.100
It's very clean.

1052
00:47:56.100 --> 00:47:58.306
You've got two phases.

1053
00:47:58.306 --> 00:48:01.208
Often people are coming,
at least in my program,

1054
00:48:01.208 --> 00:48:03.588
from more of a quantitative background,

1055
00:48:03.588 --> 00:48:05.542
so starting quantitatively.

1056
00:48:07.047 --> 00:48:10.630
Very easy, elegant design.

1057
00:48:10.630 --> 00:48:12.766
The exploratory sequential design.

1058
00:48:12.766 --> 00:48:16.238
We're gonna first explore qualitatively.

1059
00:48:16.238 --> 00:48:18.002
From that qualitative data, we're gonna

1060
00:48:18.002 --> 00:48:21.763
build something quantitatively.

1061
00:48:21.763 --> 00:48:25.014
My colleague is building

1062
00:48:25.014 --> 00:48:28.938
in Sierra Leone a new
depression instrument,

1063
00:48:28.938 --> 00:48:32.142
or putting in a new intervention,

1064
00:48:32.142 --> 00:48:34.428
or even coming up with some new variables

1065
00:48:34.428 --> 00:48:37.041
that may not be in the literature.

1066
00:48:37.041 --> 00:48:40.495
Then administering that quantitatively.

1067
00:48:43.206 --> 00:48:45.248
Mixed methods is not more complicated

1068
00:48:45.248 --> 00:48:48.267
than these three designs,
for the most part.

1069
00:48:48.267 --> 00:48:50.821
If you were to take many of the projects

1070
00:48:50.821 --> 00:48:52.505
that are being published across the

1071
00:48:52.505 --> 00:48:55.442
social, behavioral, health sciences,

1072
00:48:55.442 --> 00:48:58.760
they're gonna fit one
of these three designs.

1073
00:48:58.760 --> 00:49:01.415
We need to think about the intent here.

1074
00:49:02.500 --> 00:49:06.177
What's the intent of bringing
these two databases together?

1075
00:49:06.177 --> 00:49:10.043
We can see if that interview data

1076
00:49:10.043 --> 00:49:12.384
reports similar results
to what we found out

1077
00:49:12.384 --> 00:49:13.854
quantitatively.

1078
00:49:16.795 --> 00:49:19.089
We can confirm one with the other,

1079
00:49:19.089 --> 00:49:21.191
or even validate.

1080
00:49:21.191 --> 00:49:23.768
That may seem like you've never

1081
00:49:23.768 --> 00:49:26.543
heard of that before
or thought about that,

1082
00:49:26.543 --> 00:49:28.830
but I'm at a project, an NSF project,

1083
00:49:28.830 --> 00:49:30.815
over in Boston right now

1084
00:49:30.815 --> 00:49:33.857
where they wanted to measure school use

1085
00:49:33.857 --> 00:49:37.118
of this mathematic curriculum by teachers.

1086
00:49:37.118 --> 00:49:39.254
They searched for almost a year to try

1087
00:49:39.254 --> 00:49:42.273
to find the variables that would work.

1088
00:49:42.273 --> 00:49:45.007
They finally settled on some
that they would measure.

1089
00:49:46.280 --> 00:49:49.866
They decided to also gather
some qualitative data.

1090
00:49:49.866 --> 00:49:51.677
What they were able to do with gathering

1091
00:49:51.677 --> 00:49:53.697
qualitative data is to actually refine

1092
00:49:53.697 --> 00:49:57.157
those variables and come
up with new variables

1093
00:49:57.157 --> 00:49:59.235
that are much better.

1094
00:49:59.235 --> 00:50:01.726
They were putting these
two databases together.

1095
00:50:10.571 --> 00:50:13.671
This one, the intent of this one, really,

1096
00:50:13.671 --> 00:50:15.970
we know that survey quantitative data

1097
00:50:15.970 --> 00:50:18.512
is very important,

1098
00:50:18.512 --> 00:50:21.786
but why people checked certain scores

1099
00:50:21.786 --> 00:50:24.468
on certain items,

1100
00:50:24.468 --> 00:50:26.789
we don't know much more
than it was significant

1101
00:50:26.789 --> 00:50:29.042
or non-significant, so let's collect some

1102
00:50:29.042 --> 00:50:31.438
qualitative data.

1103
00:50:31.438 --> 00:50:33.990
Let's use the qualitative to explain

1104
00:50:33.990 --> 00:50:36.016
the quantitative results.

1105
00:50:37.972 --> 00:50:39.964
This one.

1106
00:50:39.964 --> 00:50:42.915
The intent is to first build something

1107
00:50:42.915 --> 00:50:45.143
to explore.

1108
00:50:45.143 --> 00:50:46.942
Almost all the people that come to me

1109
00:50:46.942 --> 00:50:50.263
that have a strong qualitative background,

1110
00:50:50.263 --> 00:50:52.574
such as what you've been given here

1111
00:50:52.574 --> 00:50:56.162
at Smith, they like this design.

1112
00:50:56.162 --> 00:50:59.041
You want to explore first, and then

1113
00:50:59.041 --> 00:51:01.166
build something quantitatively.

1114
00:51:01.166 --> 00:51:03.522
The intent is to first explore

1115
00:51:03.522 --> 00:51:05.647
because we may not know the variables,

1116
00:51:05.647 --> 00:51:07.539
we may not understand the population

1117
00:51:07.539 --> 00:51:09.420
we're studying, we may not even know

1118
00:51:09.420 --> 00:51:11.853
what questions to ask.

1119
00:51:15.886 --> 00:51:17.848
So what you intend to accomplish.

1120
00:51:17.848 --> 00:51:19.741
There's a certain timing,

1121
00:51:19.741 --> 00:51:22.827
your background, what's
popular in your field,

1122
00:51:24.283 --> 00:51:25.921
what would be easiest for stakeholders

1123
00:51:25.921 --> 00:51:28.022
to understand.

1124
00:51:28.022 --> 00:51:29.601
What I wanna do is I wanna take these

1125
00:51:29.601 --> 00:51:32.753
basic designs and move
to three more designs.

1126
00:51:36.148 --> 00:51:38.807
What I wanna do is I wanna frame 'em.

1127
00:51:38.807 --> 00:51:40.548
I'm gonna frame one of these designs

1128
00:51:40.548 --> 00:51:42.464
within an experiment,

1129
00:51:42.464 --> 00:51:45.506
within a social justice
perspective, and within

1130
00:51:45.506 --> 00:51:47.996
an evaluation perspective.

1131
00:51:51.368 --> 00:51:54.863
Here's an experiment running.

1132
00:51:54.863 --> 00:51:56.685
What these authors did, it's looking

1133
00:51:56.685 --> 00:51:59.889
at the management of
anti-psychotic medication.

1134
00:51:59.889 --> 00:52:02.426
They put qualitative
data in before to help

1135
00:52:02.426 --> 00:52:04.853
design the intervention,

1136
00:52:04.853 --> 00:52:07.652
and they put qualitative
in after the experiment

1137
00:52:07.652 --> 00:52:10.449
to explain the results.

1138
00:52:10.449 --> 00:52:13.502
We call these intervention
mixed-methods designs,

1139
00:52:13.502 --> 00:52:15.064
really popular.

1140
00:52:17.391 --> 00:52:19.341
Here's another one.

1141
00:52:19.341 --> 00:52:21.982
This is a study of homeless individuals

1142
00:52:21.982 --> 00:52:24.304
and how they move from a hospital

1143
00:52:24.304 --> 00:52:26.150
to a shelter,

1144
00:52:26.150 --> 00:52:28.618
and what that transition looks like.

1145
00:52:30.247 --> 00:52:33.150
These authors, Grayson, they had

1146
00:52:33.150 --> 00:52:35.152
a survey instrument that had both closed

1147
00:52:35.152 --> 00:52:37.577
and open-ended questions.

1148
00:52:37.577 --> 00:52:40.248
That's not very strong
on the qualitative side,

1149
00:52:40.248 --> 00:52:42.116
I call it mixed-methods light,

1150
00:52:42.116 --> 00:52:43.938
but it does have both qualitative

1151
00:52:43.938 --> 00:52:45.437
and quantitative data
that they were trying

1152
00:52:45.437 --> 00:52:46.911
to bring together.

1153
00:52:46.911 --> 00:52:48.850
Then what they did was they had different

1154
00:52:48.850 --> 00:52:52.709
community input points
throughout the project.

1155
00:52:52.709 --> 00:52:55.122
In other words, they
put a framework around

1156
00:52:55.122 --> 00:52:57.816
a convergent design based upon

1157
00:52:57.816 --> 00:52:59.911
community participation.

1158
00:53:04.480 --> 00:53:07.220
Another framework would
be a program evaluation

1159
00:53:07.220 --> 00:53:09.285
type of mixed-methods design.

1160
00:53:11.451 --> 00:53:13.157
This is a study of mental health needs

1161
00:53:13.157 --> 00:53:15.712
of youth in Sri Lanka.

1162
00:53:15.712 --> 00:53:18.289
They had several aspects of this project

1163
00:53:18.289 --> 00:53:20.995
going forward, from proposing the model

1164
00:53:20.995 --> 00:53:23.292
to developing a program to testing theory

1165
00:53:23.292 --> 00:53:24.767
and evaluating it.

1166
00:53:24.767 --> 00:53:26.566
They had quantitative
and qualitative going

1167
00:53:26.566 --> 00:53:28.993
throughout these different phases.

1168
00:53:28.993 --> 00:53:31.233
We call this a multi-stage design,

1169
00:53:31.233 --> 00:53:33.957
large-scale research-funded project.

1170
00:53:36.191 --> 00:53:39.255
We can draw a picture of these designs,

1171
00:53:39.255 --> 00:53:41.653
and we can add procedures in there.

1172
00:53:43.887 --> 00:53:45.291
This is the Farrell study.

1173
00:53:45.291 --> 00:53:47.904
They did not have this picture.

1174
00:53:47.904 --> 00:53:49.890
It would have been helpful.

1175
00:53:49.890 --> 00:53:53.129
The qualitative interviews,
qualitative theme analysis,

1176
00:53:53.129 --> 00:53:56.041
quantitative questionnaires analysis,

1177
00:53:56.041 --> 00:53:58.182
their comparison that went on.

1178
00:54:01.197 --> 00:54:02.884
They're integrating,
they're bringing these

1179
00:54:02.884 --> 00:54:05.021
two databases together right here

1180
00:54:05.021 --> 00:54:06.854
at the end of the article, where they're

1181
00:54:06.854 --> 00:54:09.374
comparing the qualitative themes

1182
00:54:09.374 --> 00:54:11.638
for people that have lower and higher

1183
00:54:11.638 --> 00:54:13.541
mean engagement scores.

1184
00:54:13.541 --> 00:54:15.260
Can you see how that's integrating the

1185
00:54:15.260 --> 00:54:16.547
two forms of data?

1186
00:54:17.919 --> 00:54:19.381
One of the things,

1187
00:54:21.010 --> 00:54:24.053
there are different
forms of integration that

1188
00:54:24.053 --> 00:54:25.700
detail to these different designs.

1189
00:54:25.700 --> 00:54:28.165
I'm gonna move forward here.

1190
00:54:28.165 --> 00:54:30.847
One of the complexities is understanding

1191
00:54:30.847 --> 00:54:33.387
the challenges in doing these projects.

1192
00:54:34.773 --> 00:54:37.316
By being able to picture the design

1193
00:54:37.316 --> 00:54:39.360
and look at how people are conducting 'em

1194
00:54:39.360 --> 00:54:41.518
and thinking about what the issues are,

1195
00:54:41.518 --> 00:54:44.488
we can actually come up with
challenge points in this.

1196
00:54:45.654 --> 00:54:48.160
So for example,

1197
00:54:48.160 --> 00:54:50.702
you need to start with
the similar questions

1198
00:54:50.702 --> 00:54:52.792
on both the quantitative
and qualitative side

1199
00:54:52.792 --> 00:54:54.778
in this convergent design.

1200
00:54:54.778 --> 00:54:56.357
Farrell did that,

1201
00:54:56.357 --> 00:54:57.935
if you look at their study.

1202
00:54:57.935 --> 00:54:59.420
I don't know if they did it consciously,

1203
00:54:59.420 --> 00:55:02.045
but it was a great strategy.

1204
00:55:02.045 --> 00:55:05.202
If you also notice, they
had equal sample size.

1205
00:55:05.202 --> 00:55:07.908
There's some controversy
whether you need that.

1206
00:55:09.293 --> 00:55:11.466
Some people would say just a few people,

1207
00:55:11.466 --> 00:55:12.928
qualitatively, would work.

1208
00:55:12.928 --> 00:55:14.881
We don't need to generalize.

1209
00:55:18.884 --> 00:55:22.560
They began integrating
this by bringing together,

1210
00:55:22.560 --> 00:55:24.555
in their different integration techniques

1211
00:55:24.555 --> 00:55:26.368
that go with this design,

1212
00:55:26.368 --> 00:55:28.712
a side-by-side

1213
00:55:28.712 --> 00:55:31.697
data transformation and a joint display.

1214
00:55:31.697 --> 00:55:33.971
They basically used a
joint display discussion,

1215
00:55:33.971 --> 00:55:35.550
which I'm going to show you

1216
00:55:35.550 --> 00:55:37.709
in a moment.

1217
00:55:37.709 --> 00:55:39.277
We look at these designs in terms

1218
00:55:39.277 --> 00:55:41.971
of where integration occurs,

1219
00:55:41.971 --> 00:55:43.910
then if you find differences to explain

1220
00:55:43.910 --> 00:55:46.614
those differences towards the end.

1221
00:55:46.614 --> 00:55:48.820
There's one, two, three, four

1222
00:55:48.820 --> 00:55:51.266
challenge points in this design.

1223
00:55:52.408 --> 00:55:54.128
They didn't talk about any of those,

1224
00:55:54.128 --> 00:55:56.472
but if they had known that this

1225
00:55:56.472 --> 00:55:58.865
was the design they were working with,

1226
00:55:58.865 --> 00:56:00.553
they could have talked about 'em.

1227
00:56:02.242 --> 00:56:04.159
One thing that they didn't do, too,

1228
00:56:04.159 --> 00:56:07.015
is create a joint display of the results.

1229
00:56:07.015 --> 00:56:09.255
A joint display is a table or a graph

1230
00:56:09.255 --> 00:56:11.901
in which a researcher arrays quantitative

1231
00:56:11.901 --> 00:56:14.118
and the qualitative results together

1232
00:56:14.118 --> 00:56:17.850
so that a comparison
of results can be made.

1233
00:56:17.850 --> 00:56:19.998
One thing that perplexes people

1234
00:56:19.998 --> 00:56:22.981
about this convergent design is you've got

1235
00:56:22.981 --> 00:56:25.756
numbers and you've got words.

1236
00:56:25.756 --> 00:56:27.918
How are you gonna bring the two together?

1237
00:56:29.378 --> 00:56:31.385
What they did was they took

1238
00:56:31.385 --> 00:56:33.810
the highest and lowest scores, and then

1239
00:56:33.810 --> 00:56:37.095
looked at how people talked
about them qualitatively.

1240
00:56:38.409 --> 00:56:40.687
They did this in a discussion.

1241
00:56:40.687 --> 00:56:42.685
I would have put it into
a table that's called

1242
00:56:42.685 --> 00:56:45.239
a joint-display table that we're getting

1243
00:56:45.239 --> 00:56:47.525
a lot of good examples in the

1244
00:56:47.525 --> 00:56:50.138
"Journal of Mixed Methods Research."

1245
00:56:50.138 --> 00:56:52.192
I created a table for them.

1246
00:56:53.505 --> 00:56:55.679
A low mean engagement,

1247
00:56:55.679 --> 00:56:57.687
high mean engagement,

1248
00:56:57.687 --> 00:57:00.428
and they were looking at
the case manager's job.

1249
00:57:00.428 --> 00:57:03.226
They had some significant quotes.

1250
00:57:04.365 --> 00:57:05.843
You can look down through these quotes

1251
00:57:05.843 --> 00:57:08.619
and see if people are talking about

1252
00:57:08.619 --> 00:57:10.414
engagement differently

1253
00:57:11.741 --> 00:57:13.345
because of their

1254
00:57:13.345 --> 00:57:15.145
numeric score.

1255
00:57:15.145 --> 00:57:17.838
You could fill out a table like this.

1256
00:57:17.838 --> 00:57:19.824
There have been some very creative things

1257
00:57:19.824 --> 00:57:22.225
occurring on this front.

1258
00:57:22.225 --> 00:57:23.677
For example,

1259
00:57:23.677 --> 00:57:27.183
GIS, a geographic information system,

1260
00:57:27.183 --> 00:57:29.190
geocoding.

1261
00:57:29.190 --> 00:57:31.616
You could have a geocode
map with a pullout

1262
00:57:31.616 --> 00:57:33.938
of some quotes and themes to illustrate

1263
00:57:33.938 --> 00:57:35.818
how people are talking about it.

1264
00:57:35.818 --> 00:57:38.036
That's what DataCore is doing right now

1265
00:57:38.036 --> 00:57:39.405
in the state of Delaware.

1266
00:57:39.405 --> 00:57:41.658
They're a major

1267
00:57:41.658 --> 00:57:44.004
research group looking at substance use

1268
00:57:44.004 --> 00:57:46.169
across different states.

1269
00:57:50.167 --> 00:57:52.142
My evaluation of their study.

1270
00:57:52.142 --> 00:57:55.814
I told Farrell, I checked with her.

1271
00:57:55.814 --> 00:57:57.718
I said, "I wanted to use your project,"

1272
00:57:57.718 --> 00:58:00.677
and I said, "It's really a
very, very good project."

1273
00:58:00.677 --> 00:58:02.838
It was solid on many accounts.

1274
00:58:02.838 --> 00:58:05.335
They had all three types of questions.

1275
00:58:05.335 --> 00:58:08.351
They talked about models
informing the study.

1276
00:58:08.351 --> 00:58:10.419
They stated a general rationale.

1277
00:58:10.419 --> 00:58:12.890
You saw my revision, I
would have improved it.

1278
00:58:12.890 --> 00:58:14.517
They collected and analyzed qualitative

1279
00:58:14.517 --> 00:58:16.779
and quantitative data, yes, of course.

1280
00:58:16.779 --> 00:58:19.033
Did they use rigorous procedures?

1281
00:58:19.033 --> 00:58:21.041
Yes, I went down that checklist.

1282
00:58:21.041 --> 00:58:23.944
I thought it was pretty
rigorous on both sides.

1283
00:58:23.944 --> 00:58:26.021
Did they select a mixed-method design?

1284
00:58:26.021 --> 00:58:28.506
Not really explicitly mentioned.

1285
00:58:29.721 --> 00:58:32.876
Did they draw a visual
diagram of the design?

1286
00:58:32.876 --> 00:58:35.337
Really not in the article.

1287
00:58:35.337 --> 00:58:37.915
Did they add in procedures and integration

1288
00:58:37.915 --> 00:58:39.459
and threats?

1289
00:58:39.459 --> 00:58:40.736
Not really.

1290
00:58:40.736 --> 00:58:42.875
Did they create a joint display?

1291
00:58:42.875 --> 00:58:45.452
Yes, to a certain extent

1292
00:58:45.452 --> 00:58:47.333
in their discussion section.

1293
00:58:47.333 --> 00:58:50.941
So to me, this article
scores actually very high.

1294
00:58:52.754 --> 00:58:54.453
I don't know whether they've read

1295
00:58:54.453 --> 00:58:56.287
the books mixed-methods research

1296
00:58:56.287 --> 00:58:58.968
or read much in the field.

1297
00:58:58.968 --> 00:59:00.953
My recommendations for social work,

1298
00:59:00.953 --> 00:59:02.407
my last slide.

1299
00:59:03.565 --> 00:59:05.419
Consider the full array of designs,

1300
00:59:05.419 --> 00:59:08.009
not just convergent designs.

1301
00:59:08.009 --> 00:59:09.599
How many of you before today knew

1302
00:59:09.599 --> 00:59:11.132
that you could sequence

1303
00:59:11.132 --> 00:59:13.917
these phases, qualitative
and quantitative,

1304
00:59:13.917 --> 00:59:15.508
and thought about that?

1305
00:59:15.508 --> 00:59:16.692
A few.

1306
00:59:18.875 --> 00:59:22.074
I would suggest, if you wanted to engage

1307
00:59:22.074 --> 00:59:24.616
in a mixed-methods project for your first

1308
00:59:24.616 --> 00:59:27.217
mixed-methods study as a master's,

1309
00:59:27.217 --> 00:59:29.503
doctoral student, I think

1310
00:59:29.503 --> 00:59:32.418
this explanatory sequential design,

1311
00:59:32.418 --> 00:59:35.796
starting quantitatively
and building qualitative

1312
00:59:35.796 --> 00:59:37.951
might work quite nicely for you.

1313
00:59:39.851 --> 00:59:42.310
I think you ought to think about

1314
00:59:42.310 --> 00:59:45.584
this checklist of 10 strategies,

1315
00:59:45.584 --> 00:59:48.162
and make sure they're all in your project.

1316
00:59:48.162 --> 00:59:50.741
You would have a very rigorous project.

1317
00:59:52.369 --> 00:59:53.718
I would recommend that you train

1318
00:59:53.718 --> 00:59:56.678
your graduate students in mixed methods.

1319
00:59:56.678 --> 01:00:00.173
Down at Johns Hopkins in
the School of Public Health,

1320
01:00:00.173 --> 01:00:02.808
no, excuse me, the College of Nursing,

1321
01:00:02.808 --> 01:00:05.467
they have a qualitative
course, and they've got

1322
01:00:05.467 --> 01:00:07.945
two units on mixed methods.

1323
01:00:09.345 --> 01:00:12.493
Next spring at Harvard in
the School of Public Health,

1324
01:00:12.493 --> 01:00:15.569
they will have, for their
master's-level students,

1325
01:00:17.767 --> 01:00:19.427
a completed mixed-methods course

1326
01:00:19.427 --> 01:00:20.874
that they're going to offer.

1327
01:00:22.491 --> 01:00:24.405
It could be added as an elective.

1328
01:00:26.685 --> 01:00:29.635
You might think about coming as a student

1329
01:00:29.635 --> 01:00:31.853
to one of these two-
or three-day workshops.

1330
01:00:31.853 --> 01:00:33.838
Basically, the way it works out,

1331
01:00:33.838 --> 01:00:35.805
you bring a project in, and within two

1332
01:00:35.805 --> 01:00:38.348
or three days, we go through these steps.

1333
01:00:38.348 --> 01:00:41.262
You get feedback, and
you have sketched out

1334
01:00:41.262 --> 01:00:43.489
a pretty good mixed-methods project.

1335
01:00:44.861 --> 01:00:47.117
I did it down at Chapel Hill

1336
01:00:47.117 --> 01:00:48.580
last year.

1337
01:00:48.580 --> 01:00:50.960
I'll be doing it at Johns Hopkins

1338
01:00:50.960 --> 01:00:54.001
this Thursday, Wednesday and Thursday.

1339
01:00:54.001 --> 01:00:56.553
I'll be doing it back
at Harvard next week.

1340
01:01:00.819 --> 01:01:02.247
So do you know a little bit more about

1341
01:01:02.247 --> 01:01:05.065
mixed methods than when
you came in here tonight?

1342
01:01:05.065 --> 01:01:06.621
- [Voiceover] Absolutely.

1343
01:01:06.621 --> 01:01:09.163
- How many are going to do a mixed-methods

1344
01:01:09.163 --> 01:01:11.404
project?

1345
01:01:11.404 --> 01:01:14.347
Almost all the hands
went up, Jim. (laughter)

1346
01:01:15.700 --> 01:01:16.883
Thank you very much.

1347
01:01:16.883 --> 01:01:18.162
Let me take any questions you have.

1348
01:01:18.162 --> 01:01:20.424
I know a couple of you were

1349
01:01:20.424 --> 01:01:22.004
raising your hands throughout

1350
01:01:22.004 --> 01:01:24.569
and looking perplexed.

1351
01:01:24.569 --> 01:01:25.440
Barbara.

1352
01:01:25.440 --> 01:01:26.394
- [Voiceover] This is not
the question I had earlier,

1353
01:01:26.394 --> 01:01:27.590
but that's okay.

1354
01:01:27.590 --> 01:01:30.538
You didn't address in your presentation

1355
01:01:30.538 --> 01:01:34.078
whether one researcher would do both,

1356
01:01:34.078 --> 01:01:36.970
or whether, what the
difference is when you

1357
01:01:36.970 --> 01:01:39.748
have the team

1358
01:01:39.748 --> 01:01:43.015
reading the two elements

1359
01:01:44.327 --> 01:01:46.143
or one person.

1360
01:01:46.143 --> 01:01:50.021
Also, other eyes need to be brought in

1361
01:01:50.021 --> 01:01:52.836
and the point when you
begin to merge the data?

1362
01:01:55.071 --> 01:01:57.368
That's a whole 'nother question,
and I don't know if you

1363
01:01:57.368 --> 01:01:59.690
can say anything about that.

1364
01:02:01.264 --> 01:02:04.341
- I didn't talk about academic teamwork

1365
01:02:04.341 --> 01:02:06.374
in doing mixed methods, but

1366
01:02:06.374 --> 01:02:09.420
it's a popular subject in
the mixed-methods field.

1367
01:02:10.551 --> 01:02:12.003
There are some good writings

1368
01:02:12.003 --> 01:02:14.662
on how to form a mixed-methods team.

1369
01:02:14.662 --> 01:02:15.945
For example,

1370
01:02:17.482 --> 01:02:20.549
in this NSF project down at Boston,

1371
01:02:21.585 --> 01:02:24.289
over at Boston,

1372
01:02:24.289 --> 01:02:26.170
the quantitative
researchers are on one side

1373
01:02:26.170 --> 01:02:28.006
of the table, the qualitative
are on the other side

1374
01:02:28.006 --> 01:02:29.343
(laughter) of the table.

1375
01:02:29.343 --> 01:02:30.842
You know where I sit?

1376
01:02:30.842 --> 01:02:32.257
In the middle. (laughter)

1377
01:02:32.257 --> 01:02:34.917
I'm often a translator
going back and forth.

1378
01:02:36.850 --> 01:02:38.487
It takes good leadership to put together

1379
01:02:38.487 --> 01:02:40.031
a team like that.

1380
01:02:40.031 --> 01:02:42.008
It takes a certain openness.

1381
01:02:45.290 --> 01:02:46.823
Some people have learned one way

1382
01:02:46.823 --> 01:02:49.203
of doing their research.

1383
01:02:49.203 --> 01:02:52.582
They will stay by it
the rest of their life.

1384
01:02:52.582 --> 01:02:55.020
Others

1385
01:02:55.020 --> 01:02:57.399
like to explore alleys

1386
01:02:57.399 --> 01:03:00.627
and take different routes to things.

1387
01:03:00.627 --> 01:03:02.426
If you go to the Mixed-Methods Conference,

1388
01:03:02.426 --> 01:03:06.657
it's this group of very creative artists.

1389
01:03:06.657 --> 01:03:07.971
That's the way I would say it.

1390
01:03:07.971 --> 01:03:09.385
These are people who are very open

1391
01:03:09.385 --> 01:03:11.382
to exploring possibilities.

1392
01:03:11.382 --> 01:03:14.029
Really, we have these problems

1393
01:03:14.029 --> 01:03:16.224
that we need to use what I call

1394
01:03:16.224 --> 01:03:19.568
our entire toolkit to understand 'em,

1395
01:03:19.568 --> 01:03:21.378
I think.

1396
01:03:21.378 --> 01:03:24.699
I didn't talk about academic teams.

1397
01:03:24.699 --> 01:03:26.413
In terms of one individual,

1398
01:03:27.735 --> 01:03:30.273
yes, you need some basic skills

1399
01:03:30.273 --> 01:03:33.952
in qualitative data collection analysis

1400
01:03:33.952 --> 01:03:36.146
and quantitative data collection analysis,

1401
01:03:36.146 --> 01:03:38.040
which I understand you get those

1402
01:03:38.040 --> 01:03:40.938
basic skills in your (mumbles)
research courses here.

1403
01:03:42.319 --> 01:03:44.654
It's just now a matter
of putting it together.

1404
01:03:49.327 --> 01:03:52.566
Some people may feel that that's calling

1405
01:03:52.566 --> 01:03:55.287
on a skill level that's too much.

1406
01:03:57.140 --> 01:03:59.298
I would say, in terms
of our research across

1407
01:03:59.298 --> 01:04:02.456
the country, it's becoming more and more

1408
01:04:02.456 --> 01:04:04.881
international, interdisciplinary,

1409
01:04:04.881 --> 01:04:06.623
sophisticated.

1410
01:04:06.623 --> 01:04:08.574
Mixed methods, right now, is being seen

1411
01:04:08.574 --> 01:04:11.801
as the latest methodology,

1412
01:04:11.801 --> 01:04:14.331
throughout the health
sciences, especially.

1413
01:04:14.331 --> 01:04:17.241
It's just going from
field to field to field.

1414
01:04:21.970 --> 01:04:23.833
I don't know if I answered your question.

1415
01:04:27.668 --> 01:04:29.085
What other questions do you have?

1416
01:04:29.085 --> 01:04:31.076
Comments?

1417
01:04:34.588 --> 01:04:36.131
Jim.

1418
01:04:36.131 --> 01:04:37.989
- [Jim] I noticed in one of your tables

1419
01:04:37.989 --> 01:04:40.706
that there was the capital Quan

1420
01:04:40.706 --> 01:04:42.795
and small quan thing.

1421
01:04:42.795 --> 01:04:44.873
Has that pretty much been superseded

1422
01:04:44.873 --> 01:04:47.879
by the three design types

1423
01:04:47.879 --> 01:04:50.764
as a mode of analysis?

1424
01:04:55.934 --> 01:04:57.954
- What came forward in
the mixed-methods field

1425
01:04:57.954 --> 01:04:59.939
is that if you're going to give emphasis

1426
01:04:59.939 --> 01:05:01.987
to either qual or quan,

1427
01:05:01.987 --> 01:05:04.808
you capitalize those words.

1428
01:05:04.808 --> 01:05:07.327
If you're gonna give them less emphasis,

1429
01:05:07.327 --> 01:05:09.405
one of 'em less emphasis, you put it

1430
01:05:09.405 --> 01:05:11.286
in small letters.

1431
01:05:11.286 --> 01:05:12.946
You'll notice

1432
01:05:12.946 --> 01:05:15.140
that qual and quan both had

1433
01:05:15.140 --> 01:05:17.752
four characters,

1434
01:05:17.752 --> 01:05:19.981
speaking to the issue of equality.

1435
01:05:19.981 --> 01:05:21.604
(laughter)

1436
01:05:25.460 --> 01:05:28.402
This whole area of emphasis,

1437
01:05:28.402 --> 01:05:30.868
to me, is not as strong

1438
01:05:30.868 --> 01:05:33.082
as what you intend to do.

1439
01:05:34.770 --> 01:05:36.535
It's a soft area.

1440
01:05:36.535 --> 01:05:39.681
For example, do I count how many pages

1441
01:05:39.681 --> 01:05:41.353
in your mixed-methods study are devoted

1442
01:05:41.353 --> 01:05:44.866
to qualitative versus how
many pages quantitative?

1443
01:05:46.615 --> 01:05:49.161
Do I start, look at how it begins from a

1444
01:05:49.161 --> 01:05:51.961
strong postpositivist, quantitative,

1445
01:05:51.961 --> 01:05:53.798
theoretical orientation?

1446
01:05:53.798 --> 01:05:57.097
How do I really go about assessing

1447
01:05:57.097 --> 01:05:58.977
that weight?

1448
01:05:58.977 --> 01:06:00.555
When I look at mixed-methods projects,

1449
01:06:00.555 --> 01:06:02.576
I do see a different weight,

1450
01:06:02.576 --> 01:06:05.837
but I'm not sure that it's something that

1451
01:06:07.352 --> 01:06:09.558
is as important as intent.

1452
01:06:09.558 --> 01:06:12.972
I have demoted it

1453
01:06:12.972 --> 01:06:15.079
in my thinking some,

1454
01:06:18.200 --> 01:06:20.500
the weight, the priority.

1455
01:06:21.621 --> 01:06:23.270
Another interesting one that's surfaced

1456
01:06:23.270 --> 01:06:24.846
is the timing.

1457
01:06:26.732 --> 01:06:29.041
If I do my

1458
01:06:29.041 --> 01:06:30.388
survey,

1459
01:06:31.717 --> 01:06:33.193
and then as soon as you're done

1460
01:06:33.193 --> 01:06:35.839
completing that, I have
you do an interview,

1461
01:06:35.839 --> 01:06:38.497
is that a sequential data collection,

1462
01:06:38.497 --> 01:06:41.250
or is it basically a data collection

1463
01:06:41.250 --> 01:06:43.127
at the same time?

1464
01:06:45.938 --> 01:06:48.037
In the medical field,

1465
01:06:48.037 --> 01:06:51.241
these doctors I work
with, they're busy people.

1466
01:06:51.241 --> 01:06:54.050
They do a lotta convergent design work

1467
01:06:54.050 --> 01:06:55.594
because they can get out to the field

1468
01:06:55.594 --> 01:06:57.864
once and gather data.

1469
01:06:59.890 --> 01:07:02.666
The question of timing,

1470
01:07:02.666 --> 01:07:04.630
the question of priority,

1471
01:07:04.630 --> 01:07:07.761
are soft dimensions to me now.

1472
01:07:07.761 --> 01:07:10.390
I'll ask more, what is the intent?

1473
01:07:11.645 --> 01:07:13.077
People are always writing me with these

1474
01:07:13.077 --> 01:07:15.310
complicated designs.

1475
01:07:15.310 --> 01:07:17.210
Will they have qual or quan,

1476
01:07:17.210 --> 01:07:18.870
qual, quan,

1477
01:07:18.870 --> 01:07:20.647
all these drawings.

1478
01:07:20.647 --> 01:07:23.118
All I do is say, "Okay, what are you

1479
01:07:23.118 --> 01:07:25.218
"trying to accomplish?

1480
01:07:25.218 --> 01:07:27.436
"Are you trying to get a
complete understanding?

1481
01:07:27.436 --> 01:07:28.701
"That's more of a convergent.

1482
01:07:28.701 --> 01:07:30.629
"Are you trying to explain results?

1483
01:07:30.629 --> 01:07:32.629
"What are you trying to accomplish?"

1484
01:07:32.629 --> 01:07:34.486
They might be trying to
accomplish several things.

1485
01:07:34.486 --> 01:07:36.240
Then I say something like,

1486
01:07:38.089 --> 01:07:39.866
"What's at the heart of this?"

1487
01:07:39.866 --> 01:07:42.739
That Grayson article with
the community participatory,

1488
01:07:44.533 --> 01:07:46.239
they had a lot going on in that project,

1489
01:07:46.239 --> 01:07:49.721
but at that heart of
it was trying to merge

1490
01:07:49.721 --> 01:07:51.428
the quantitative and qualitative data

1491
01:07:51.428 --> 01:07:53.311
from that survey instrument.

1492
01:07:54.933 --> 01:07:57.066
The intent was to really get a more

1493
01:07:57.066 --> 01:07:58.454
complete understanding.

1494
01:07:59.562 --> 01:08:01.712
Some of these issues, for me, yes,

1495
01:08:01.712 --> 01:08:04.196
have kind of fallen not by the wayside,

1496
01:08:04.196 --> 01:08:05.403
but into

1497
01:08:07.098 --> 01:08:09.166
a lesser importance.

1498
01:08:10.795 --> 01:08:13.147
That's a good technical question.

1499
01:08:14.336 --> 01:08:15.816
Yes, sir?

1500
01:08:15.816 --> 01:08:17.569
- [Voiceover] Yes. (clears throat)

1501
01:08:17.569 --> 01:08:19.392
You had shown one type

1502
01:08:19.392 --> 01:08:23.257
of research that might be
social justice oriented.

1503
01:08:23.257 --> 01:08:24.923
Could you give an example of that

1504
01:08:24.923 --> 01:08:28.059
or flesh out

1505
01:08:28.059 --> 01:08:30.450
what that might look like?

1506
01:08:36.439 --> 01:08:38.386
- A feminist mixed-methods study.

1507
01:08:41.000 --> 01:08:43.510
What would that look like?

1508
01:08:43.510 --> 01:08:45.230
First of all,

1509
01:08:45.230 --> 01:08:47.807
the feminist perspective.

1510
01:08:47.807 --> 01:08:49.747
We've studied this, and
I've got some good articles

1511
01:08:49.747 --> 01:08:52.010
that would illustrate this.

1512
01:08:52.010 --> 01:08:53.751
The feminist perspective would flow into

1513
01:08:53.751 --> 01:08:56.097
the study in different places.

1514
01:08:56.097 --> 01:08:58.233
It would help shape the problem

1515
01:08:58.233 --> 01:08:59.812
at the beginning.

1516
01:08:59.812 --> 01:09:02.610
It would shape the
theoretical orientation.

1517
01:09:02.610 --> 01:09:04.838
It would shape the data collection

1518
01:09:04.838 --> 01:09:08.391
where you don't want
to further marginalize

1519
01:09:08.391 --> 01:09:10.839
the women that you're studying.

1520
01:09:10.839 --> 01:09:13.841
It would shape the outcomes.

1521
01:09:15.506 --> 01:09:17.104
Call for action.

1522
01:09:18.873 --> 01:09:21.544
What we have done in
the mixed-methods field,

1523
01:09:21.544 --> 01:09:24.482
in analyzing the feminist
mixed-methods projects,

1524
01:09:24.482 --> 01:09:28.009
is to be curious about how
the feminist perspective

1525
01:09:28.009 --> 01:09:30.400
flows into different phases.

1526
01:09:31.950 --> 01:09:35.502
You could do an explanatory sequential

1527
01:09:35.502 --> 01:09:36.918
feminist study.

1528
01:09:38.342 --> 01:09:40.408
The feminist perspective
would shape the problem.

1529
01:09:40.408 --> 01:09:42.312
It would probably dictate

1530
01:09:42.312 --> 01:09:44.250
the theoretical orientation

1531
01:09:44.250 --> 01:09:47.792
that you bring in in your
quantitative first phase.

1532
01:09:47.792 --> 01:09:49.869
It would then shape, perhaps,

1533
01:09:49.869 --> 01:09:52.932
what qualitative follow-ups you would do

1534
01:09:52.932 --> 01:09:54.390
to explain

1535
01:09:55.581 --> 01:09:57.627
some of the injustices.

1536
01:09:57.627 --> 01:10:00.334
It would shape the outcomes.

1537
01:10:01.559 --> 01:10:03.995
I actually have a couple slides on this

1538
01:10:03.995 --> 01:10:05.501
where I show where

1539
01:10:07.096 --> 01:10:10.085
the feminist perspective has flown into

1540
01:10:10.085 --> 01:10:14.314
an explanatory sequential type of design.

1541
01:10:14.314 --> 01:10:17.239
But remember, I used social justice.

1542
01:10:17.239 --> 01:10:19.364
I said it's a framework that surrounds

1543
01:10:19.364 --> 01:10:21.520
one of these basic designs.

1544
01:10:21.520 --> 01:10:23.552
That's the way I look at it.

1545
01:10:23.552 --> 01:10:25.596
It helps to shape,

1546
01:10:25.596 --> 01:10:27.731
but still at the heart of that project

1547
01:10:27.731 --> 01:10:29.362
is an explanatory

1548
01:10:29.362 --> 01:10:31.940
sequential project.

1549
01:10:31.940 --> 01:10:35.143
The basic design is always in the center,

1550
01:10:35.143 --> 01:10:37.373
and then the advanced,

1551
01:10:37.373 --> 01:10:40.294
the social justice would
be a framework around it.

1552
01:10:44.454 --> 01:10:45.920
Whether it's

1553
01:10:48.058 --> 01:10:50.604
a racial study,

1554
01:10:50.604 --> 01:10:52.613
a cultural study,

1555
01:10:52.613 --> 01:10:54.482
a critical theory.

1556
01:10:54.482 --> 01:10:56.944
Critical theory could be a framework

1557
01:10:56.944 --> 01:10:59.736
surrounding one of these
basic designs as well.

1558
01:11:08.610 --> 01:11:11.210
What do you think about training in?

1559
01:11:11.210 --> 01:11:12.407
I'll take your question.

1560
01:11:12.407 --> 01:11:13.707
Then I wanna ask you what do you think

1561
01:11:13.707 --> 01:11:16.633
about training in mixed methods.

1562
01:11:16.633 --> 01:11:17.955
- [Voiceover] I was wondering if you

1563
01:11:17.955 --> 01:11:20.400
wouldn't mind telling us some about

1564
01:11:21.554 --> 01:11:24.004
what brought you to be so convinced

1565
01:11:24.004 --> 01:11:26.577
and passionate about these methods?

1566
01:11:33.501 --> 01:11:34.709
- That's a good question.

1567
01:11:34.709 --> 01:11:36.810
I started as a quantitative researcher,

1568
01:11:36.810 --> 01:11:38.482
then started teaching qualitative,

1569
01:11:38.482 --> 01:11:41.349
and then started going back and forth

1570
01:11:41.349 --> 01:11:42.406
between the two.

1571
01:11:42.406 --> 01:11:46.109
I had a doctoral proposal
development class

1572
01:11:46.109 --> 01:11:48.117
where

1573
01:11:48.117 --> 01:11:49.424
some of the students were coming

1574
01:11:49.424 --> 01:11:51.124
with quantitative projects,
and some were coming

1575
01:11:51.124 --> 01:11:52.645
with qualitative.

1576
01:11:52.645 --> 01:11:54.538
I would say, "Okay, tonight we're gonna

1577
01:11:54.538 --> 01:11:57.443
"talk about designing a hypotheses.

1578
01:11:58.573 --> 01:12:01.176
"Then following that,
we're gonna talk about

1579
01:12:01.176 --> 01:12:04.870
"a good qualitative, open-ended question."

1580
01:12:04.870 --> 01:12:07.460
That got me thinking about

1581
01:12:07.460 --> 01:12:10.034
there's some crossover going there.

1582
01:12:12.086 --> 01:12:14.290
Back in 1991,

1583
01:12:17.459 --> 01:12:20.324
I saw a paper being presented

1584
01:12:20.324 --> 01:12:22.983
at the American Educational
Research Conference,

1585
01:12:22.983 --> 01:12:24.701
AERA.

1586
01:12:24.701 --> 01:12:26.199
This was a doctoral student.

1587
01:12:26.199 --> 01:12:29.716
She had labeled it mixed-methods research.

1588
01:12:29.716 --> 01:12:31.306
I got so excited.

1589
01:12:31.306 --> 01:12:34.081
I said, "Oh, I'm gonna go to that."

1590
01:12:34.081 --> 01:12:36.461
I went to the session.

1591
01:12:36.461 --> 01:12:39.559
You know how many people were there?

1592
01:12:39.559 --> 01:12:40.651
- [Voiceover] The two of you?

1593
01:12:40.651 --> 01:12:42.032
- Two of us.

1594
01:12:42.032 --> 01:12:44.958
The student and myself. (laughter)

1595
01:12:47.767 --> 01:12:49.493
I said to her, I said, "You know,

1596
01:12:49.493 --> 01:12:51.815
"this is the methodology of the future."

1597
01:12:51.815 --> 01:12:54.254
She thought I was a crazy
professor. (laughter)

1598
01:12:54.254 --> 01:12:56.272
I was just so excited about this.

1599
01:12:57.736 --> 01:13:00.608
I like the idea of working on

1600
01:13:00.608 --> 01:13:03.788
the frontiers of a new methodology,

1601
01:13:03.788 --> 01:13:07.283
and one that would really

1602
01:13:07.283 --> 01:13:08.723
bridge

1603
01:13:08.723 --> 01:13:10.835
our community or researchers, rather than

1604
01:13:10.835 --> 01:13:12.820
the division of
quantitative and qualitative

1605
01:13:12.820 --> 01:13:14.297
we often have.

1606
01:13:14.297 --> 01:13:16.198
I see this as a bridging.

1607
01:13:16.198 --> 01:13:19.972
I see this as building a larger community

1608
01:13:19.972 --> 01:13:22.967
than either quantitative or qualitative

1609
01:13:22.967 --> 01:13:24.497
separately.

1610
01:13:29.013 --> 01:13:32.813
Then I think that 1994
book on research design

1611
01:13:32.813 --> 01:13:34.717
that had that one chapter on combining

1612
01:13:34.717 --> 01:13:36.402
qualitative and quantitative

1613
01:13:36.402 --> 01:13:40.058
really got me thinking about the potential

1614
01:13:40.058 --> 01:13:41.813
of that.

1615
01:13:41.813 --> 01:13:44.481
That book was used

1616
01:13:44.481 --> 01:13:47.088
in the undergraduate program

1617
01:13:47.088 --> 01:13:50.884
in a liberal arts college
that my son was attending.

1618
01:13:50.884 --> 01:13:52.862
People were writing to me from Harvard,

1619
01:13:52.862 --> 01:13:56.148
they were using it in their
doctoral program at Harvard.

1620
01:13:56.148 --> 01:13:59.004
I thought that there's a methodology here

1621
01:13:59.004 --> 01:14:01.721
that would transcend levels.

1622
01:14:01.721 --> 01:14:03.381
I've learned that.

1623
01:14:03.381 --> 01:14:05.947
I often get the question, is mixed methods

1624
01:14:05.947 --> 01:14:08.045
so rigorous,

1625
01:14:08.045 --> 01:14:10.007
so taxing,

1626
01:14:10.007 --> 01:14:12.921
that it's only for the best and brightest

1627
01:14:12.921 --> 01:14:14.538
of doctoral students?

1628
01:14:15.766 --> 01:14:16.924
But you know, you don't have to do

1629
01:14:16.924 --> 01:14:18.805
a full-blown quantitative and a

1630
01:14:18.805 --> 01:14:20.279
full-blow qualitative.

1631
01:14:20.279 --> 01:14:22.624
It could be a smaller part.

1632
01:14:22.624 --> 01:14:25.178
Do a survey with a few interviews.

1633
01:14:25.178 --> 01:14:27.500
I was in South Africa,
and I was called upon

1634
01:14:27.500 --> 01:14:29.845
to be a judge

1635
01:14:29.845 --> 01:14:32.201
of the best research project among

1636
01:14:32.201 --> 01:14:34.361
the undergraduate students.

1637
01:14:34.361 --> 01:14:36.962
Out of the five students
that presented that day,

1638
01:14:36.962 --> 01:14:39.063
four undergraduates were doing

1639
01:14:39.063 --> 01:14:41.060
mixed-methods research.

1640
01:14:41.060 --> 01:14:43.265
They were gathering both forms of data.

1641
01:14:44.989 --> 01:14:46.765
I think it's an approach

1642
01:14:46.765 --> 01:14:48.369
that transcends levels,

1643
01:14:49.865 --> 01:14:51.860
it transcends fields.

1644
01:14:53.666 --> 01:14:56.185
It's very practical.

1645
01:14:56.185 --> 01:14:59.470
I use the example of the Boston bombing,

1646
01:14:59.470 --> 01:15:02.791
but you could turn to CNN on any night,

1647
01:15:02.791 --> 01:15:06.100
and they are doing
mixed-methods broadcasts

1648
01:15:06.100 --> 01:15:08.317
where they report the figures about Sandy,

1649
01:15:08.317 --> 01:15:10.279
and then they're interviewing people.

1650
01:15:10.279 --> 01:15:12.699
They're doing mixed-methods broadcasts.

1651
01:15:14.598 --> 01:15:16.448
Al Gore's documentary

1652
01:15:18.218 --> 01:15:19.475
on the

1653
01:15:20.784 --> 01:15:22.792
crisis in our environment,

1654
01:15:22.792 --> 01:15:24.650
if you look at his documentary, it's a

1655
01:15:24.650 --> 01:15:27.007
mixed-methods documentary.

1656
01:15:27.007 --> 01:15:30.413
That practical part reaches out to me.

1657
01:15:31.707 --> 01:15:35.346
It's something that, I
think it's very intuitive.

1658
01:15:36.513 --> 01:15:39.554
It's not learning some
statistical procedure.

1659
01:15:39.554 --> 01:15:41.864
It's not learning

1660
01:15:41.864 --> 01:15:43.792
the details of

1661
01:15:45.285 --> 01:15:48.244
how to do a good one-on-one interview.

1662
01:15:49.545 --> 01:15:52.175
It's something that we
see in our daily life.

1663
01:15:54.282 --> 01:15:56.354
That's a long answer.

1664
01:16:00.490 --> 01:16:02.563
Other questions or comments?

1665
01:16:04.855 --> 01:16:07.160
Is this what you expected to hear tonight?

1666
01:16:08.535 --> 01:16:10.484
Just kind of an introduction

1667
01:16:10.484 --> 01:16:12.539
to mixed methods,

1668
01:16:12.539 --> 01:16:16.080
and also to make it user friendly

1669
01:16:16.080 --> 01:16:17.589
so you could see where you could do

1670
01:16:17.589 --> 01:16:19.469
a type of project.

1671
01:16:19.469 --> 01:16:20.666
Some of you were thinking about

1672
01:16:20.666 --> 01:16:22.280
your own studies, how they could be shaped

1673
01:16:22.280 --> 01:16:24.913
in that direction right now, weren't you?

1674
01:16:27.585 --> 01:16:28.193
Good.

1675
01:16:28.193 --> 01:16:30.003
- [Jim] Jack Creswell,
thank you very much for

1676
01:16:30.003 --> 01:16:31.342
filling us in (applause).

1677
01:16:31.342 --> 01:16:32.944
- Thank you.

1678
01:16:32.944 --> 01:16:33.875
Thank you.

1679
01:16:33.875 --> 01:16:36.487
(applause)

1680
01:16:36.487 --> 01:16:37.956
Thank you.