bookreviews

An experiment in reviewing books in the educational technology arena. First up, a review of a new ebook via Eric Deeson of the British Journal of Educational Technology. The review may be published in a future issue of BJET.

=**Title, abstract and metadata**=

Author: Justus J. Randolph Published: 2007 Language: [|__English__] Source: [|source of full e-book] ([|http://justus.randolph.name/methods)] (Note that this resource can also be downloaded from the publisher's website (in Finnish) at: http://www4.hamk.fi/julkaisut/julkaisu.php?id=490; however, **best avoid this route** unless you are a fluent e-worker in Finnish!) Keywords: [|__educational technology__]; [|__research__]; [|__e-learning__]; [|__research methods__]; [|__development__]; [|__instructional technology__]; Over the past thirty years, there has been much dialogue, and debate, about the conduct of educational technology research and development. In this brief volume, Justus Randolph helps clarify that dialogue by theoretically and empirically charting the research methods used in the field and provides much practical information on how to conduct educational technology research. Within this text, readers can expect to find answers to the following questions:
 * Multidisciplinary Methods in Educational Technology Research and Development** (2007)

(a) What are the methodological factors that need to be taken into consideration when designing and conducting educational technology research? (b) What types of research questions do educational technology researchers tend to ask? (c) How do educational technology researchers tend to conduct research? (d) What approaches do they use? What variables do they examine? What types of measures do they use? How do they report their research? (e) How can the state of educational technology research be improved?

In addition to answering the questions above, the author, a research methodologist, provides practical information on how to conduct educational technology research--from formulating research questions, to collecting and analysing data, to writing up the research reports--in each of the major quantitative and qualitative traditions. Unlike other books of this kind, the author addresses some of research approaches used less commonly in educational technology research, but which, nonetheless, have much potential for creating new insights about educational phenomena--approaches such as single-participant research, quantitative content analysis, ethnography, narrative research, phenomenology, and others.

Multidisciplinary Methods in Educational Technology Research and Development is an excellent text for educational technology research methods courses, a useful guide for those conducting (or supervising) research, and a rich source of empirical information on the art and science of educational technology research.

=Welcome to BJET reviewers= Visitors from the BJET reviewers' panel will know what we're looking for: clear, succinct advice as to whether or not the book is worth using in relevant contexts, and why. We appreciate the issue of the book as a freely downloadable e-text: you may wish to comment on that too. Corrie will, I hope, moderate your inputs and lead to some kind of publishable review! Huge thanks to you, Corrie!
 * EricDeeson 10 Mar**

=Ground Rules= Here are the ground rules: 1. Play nice.
 * CorrieBergeron 10 Mar** (agreed by **EricDeeson 11 Mar**, dealing quietly with only two tiny embarrassing typos!)

If that's not clear, here are some more detailed guidelines. (They're not numbered because they're *all* important.)
 * No pseudonyms - sign your work. This is a gathering of educational professionals engaged in the social creation of meaning to benefit others. (Good Lord, did *I* really say that?!?) If you won't attach your name to it, don't say it.
 * Note that the author of the book, Justus Randolph, may read your comments and reply to them. So be kind, even if finding fault.
 * Since we're offering our personal opinions and reactions to the book, not creating a definitive encyclopedia article, I'm suggesting that we do not edit another's work (except perhaps to helpfully and quietly correct an embarassing typo).
 * Feel free to add structure, however - create sections, tables of contents, etc. You're probably more experienced with wikispaces than I am.
 * In case you didn't notice, wikispaces is [|Creative Commons Attribution 3.0 licenced]so others may reuse your words without notice or compensation.
 * I reserve the right to delete anything I deem out-of bounds without notice. It's my name on the site.

Justus Randolph, //Multidisciplinary Methods in Educational Technology// // Research and Development // **__Some Comments on Chapter 1.__** This book apparently originated in an e-learning project at HAMK University in Finland. There is no doubt that a state-of the-art survey of the field of Educational Technology is a hugely useful thing to develop, and Randolph brings to the task an amiable can-do exuberance, and an impressive range of intellectual skills and competencies, without which the task would be impossible. However, potentially valuable as the book undoubtedly might be to seasoned researchers and ET practitioners, it should be accompanied by some serious health warnings.  In the //Preface//, Randolph  tells us that: ‘The target audiences for this book are primarily educational technology students, their supervisors, and educational technology researchers’. This startlingly ambitious brief may explain some of the curious features of the book. I would certainly not recommend it to undergraduates, as the text is dauntingly technical in places, and requires considerable sophistication across a broad range of disciplines. The book also takes a great deal for granted and could easily cause confusion in the minds of readers who are not already familiar with the ideas that are its currency. Whilst the “//Questions to Consider//” sections at the end of each chapter do read like prompts for undergraduate essays, much of the material in the book is far too unrefined and problematic to be treated as a standard textbook. Whilst supervisors of research students, and researchers are a more satisfactory audience, this group is unlikely to form a very large potential market. Maybe that is why students get pulled into the list of target audiences. In any area, writing a book that works as a student text, and also has novelty or challenge for supervisors and researchers would be a hard trick to pull off. Here, I don’t think it works.  Chapter 1 discusses the ‘methods-choice debate and the factors that influence methods choice in educational technology research and development’. In this contribution to the group review I am going to confine my comments to the material in Chapter 1 as it clearly constitutes the core of the book, and also exhibits a number of problematic features that inevitably affect the book as a whole.  What does Randolph  mean by “Educational Technology”? Randolph  doesn’t appear to offer an explicit definition, at least in Chapter 1, though clearly the reader develops an idea of what he intends from the way he goes about things. One reason that this is problematic, is that Randolph  seems to treat Educational Technology (henceforward, ET) as a second-order discipline. One might reasonably argue for example, that Microsoft //Powerpoint// or an Overhead Projector are instances of Educational Technology. In this case, software authoring techniques (in the case of PP) or transistorized circuit construction and physical optics (in the case of the OHP) might be the methodologies of choice in developing them. This is not what Randolph  has in mind however, for him ET consists in developing and evaluating //applications// of first-order ET in this sense. For Randolph  Educational Technology is, as it were Educational Technology-//ology.// This is more than just a debating point. Randolph <span style="color: rgb(35, 31, 32)"> is seeking to develop a textbook to be used in educating practitioners of Educational Technology in his second-order sense. But maybe this is not a desirable goal. Maybe students should be educated in the first-order disciplines of education and technology. What exactly are Educational technology//ologists// good for? <span style="color: rgb(35, 31, 32)"> There are also a number of reflexivity issues that <span style="color: rgb(35, 31, 32)">Randolph <span style="color: rgb(35, 31, 32)"> does not address. In writing his book, he has clearly chosen various research methods. Does the book then not consist in a body of knowledge that might have taken quite a different character had the methodologies employed been different? To give a concrete example, Randolph presents a classification of ET research produced by Burghar and Turns using ‘an emergent coding technique to create an initial set of research question categories from all the articles published over a two to four year time period from three major educational technology forums – the proceedings of //Frontiers in Education// (FIE)//, Educational Technology Research &// Development (ETR&D) and //Human-Computer Interaction// (HCI)’. Here he is using a form of operational definition of ET – ET consists of whatever the editors or organizers of the forums under analysis chose to include within them. But what definitions of ET were //they// using? And what about the definitions of ET used by forums not included in the survey (such as BJET). Or characterizations of ET by users of the term who don’t involve themselves with such forums at all (e.g. postmodern thinkers/philosophers)? The questions I have so far raised about Chapter 1 of Randolph’s book are pretty fundamental. These are questions which one might expect an undergraduate text to consider and provide some kind of answer to, not just sweep them under the carpet. A book about choice of methodology needs to consider why it employs the methodology it does. There are other problems with Randolph ’s book though, which whilst not so fundamental are nonetheless serious. A number of these are connected with the way in which terminology is used. Early in Chapter 1 <span style="font-family: 'Times New Roman',Times,serif">,  <span style="font-family: 'Times New Roman',Times,serif"><span style="font-family: 'Times New Roman',Times,serif">  Randolph tells us that  <span style="font-size: 9.5pt; font-family: Georgia; color: rgb(35, 31, 32)"> understanding methods choice involves :   <span style="font-family: 'Times New Roman',Times,serif"><span style="font-family: 'Times New Roman',Times,serif"> //factors that influence the formulation of the research question// and //factors that influence how a research question is answered//. The first of the factors that influence the formulation of the research question is said to be “The research problem”. Clearly if this is not just a time-wasting circularity, some distinction is being made between “the research question” and “the research problem”. Randolph obviously thinks he knows what the distinction is, but he doesn’t tell the reader. I don’t think the precision of the terminology will support the weight Randolph is placing on it. For most readers “research problem” and “research question” are synonymous expressions. I doubt if it is useful or possible to make a systematic distinction between them, If Randolph thinks it is, then at least the distinction should be articulated. One of the factors said to influence how a research question is answered is “the research act implied in the research question”. One problem here is how a “research act” can be implied when implication is properly a logical relation holding between symbolic expressions (not acts). In a note on terminology, Randolph says that “research act” ‘refers to the types of intellectual activities that are characteristic of a certain research approach. For example quantitative description is a characteristic intellectual activity of survey research’. So a research question “implies” that quantitative description or qualitative explanation must be used. This is hardly a mere factor influencing choice. Randolph is actually telling us that the research methodology is implicit in the way the research question is formulated. So we are left uncertain: does the research question determine the methodology to be employed, or is it merely one factor influencing it. I know what I think, but I have no idea what Randolph ’s opinion is. Another factor influencing how the research question is answered is ‘the degree of utility needed’. However, we have earlier been told that a factor involved in formulating the research question is how “actionable” the resulting evidence will be (some researchers may wonder how this is computed in advance of the data). ‘Methods choice involves a careful weighing of many factors to create the most actionable evidence possible’. So now we have a distinction between “actionable” (presumably meaning whether the evidence suggests some action) and “utility” (presumably meaning the value of evidence in deciding what action is appropriate). So either “actionable” means the same thing as utility (in which case why cause confusion by using the term?) or it means “suggesting action //regardless of utility//” which would mean we formulate the research question most likely to produce evidence that suggests some action, even if that action is unhelpful or dangerous. If not confused this is certainly confusing. Finally Randolph recommends that choice of research method should take into account //accuracy.// ‘In some cases, it is necessary for research to have much accuracy; that is, it must produce sound information that is (a) comprehensive, (b) technically adequate, and (c) with judgments that are logically aligned with the data collected’. These characteristics are indeed valuable features of research evidence. But why on earth is the word “accuracy” being used? “Sound information” suggests that //validity// is a more relevant term. “Comprehensive” appears to have nothing at all to do with accuracy. A dart thrown at a dartboard bullseye is accurate to the extent that it gets close to the intended target. To be comprehensive, it would presumably have to hit every other number on the board as well. Experienced researchers will find Randolph ’s book engaging, compendious, wide-ranging and provocative. It is far from being the last word or the only perspective on the issues it raises. Some important issues are ignored. And theoretical terms are sometimes used in a way more likely to cause confusion than to illuminate. Especially for students. Roger Lindsay, //Cognitive Evaluation//, Cumbria, UK.