I was challenged recently at a Learning and Teaching conference about how I assess students in Biosciences. Apparently my assessment methods are ‘not authentic’ and ‘overly test knowledge’ on the basis that almost every module on my course is in part, assessed by a formal exam. There was some mention of a dodgy Einstein quote derived from “[I do not] carry such information in my mind since it is readily available in books. The value of a college education is not the learning of many facts but the training of the mind to think.” I’m sure context is everything in this quote, and very much doubt he was talking about fundamental knowledge underpinning established theories in physics. That got me thinking: Is knowledge worth testing anymore given that everything is accessible from Google? Furthermore: Seth Godin in his TEDx talk on education said, “There is zero value in memorizing anything, ever again. Anything worth memorizing can be looked up.” Sugato Mitra in “Living in the age where knowing is obsolete” suggested that answers to Physics tests could be googled and discussed, rather than answered alone in an exam. Eric Mazur suggested that any answer to a test that can be googled is not an authentic test. There are many other examples, as outlined very nicely here (sorry, this blog by @webofsubstance has since disappeared). There are two inter-related themes going on here: Knowledge is ‘out there’ and accessible so there is little benefit to memorising it. Even if knowledge is good, testing of knowledge is bad. Related to this final point, all assessments should be ‘authentic’.
As a relative novice to ‘authentic assessment’, I was keen to find out more. In the style of the above experts (excluding Einstein), I Googled it, and came with a link via Wikipedia on the definition of authentic assessment. A brief search suggested that the article is written by an expert on the topic, so I’m confident in the author being an authority on the subject. I have now read this and will discuss this document in relation to my assessment methods, which are really quite traditional (also known as ‘bad’) In ‘authentic assessment’, the assessment tool should mimic real-life situations. So having knowledge and applying it to a situation seems OK, but not testing the knowledge directly, which I think is what Eric Mazur is getting at. However if someone cannot complete an ‘authentic assessment’, what exactly is limiting the student’s progress? Is it an inability to recall the required knowledge to apply it in context, or inability to apply their acquired knowledge in context? If Google is allowed in the examination, as proposed by some experts, then we are focussing on the application of knowledge. Sorry, did I say knowledge? It’s just that it’s not necessarily knowledge is it? It’s more of a ‘state of transient observation’ whilst the information is on screen. Either way, the student got to the answer in a real-life connected-to-the-internet sort of way, so full marks all round. Well done. I’m not convinced.
My view of assessment, and in particular ‘authentic assessment’ in science, is that it should rely upon some known facts, and students should be able to apply those facts to conclude or deduce something, like an authentic scientist. As a research scientist, I have had some ideas over the years, and none of them have come about by not knowing anything about the topic. In fact I’d go as far to say that ALL of what I’d call half-decent project ideas have come from deep understanding two or more disparate topics in great detail, and making some sort of interacting link between the two. So the question is: Am I assessing in a manner that is authentic to science? The classic inauthentic/traditional test is a MCQ test. It is not very often in real life that you have to take an educated punt at 4 possible answers, three of which are deliberately contrived to trip you up. However it can test knowledge. I find a close correlation between MCQ scores and more analytic problems, but rarely can MCQs be designed in such a way. Where does the traditional University essay-style exam fit in? Authentic or not? It is dependent on the wording and what is being tested, and yes, many exams at lower levels do not really test anything other than pure recall of the lecturers’ notes. However any scientific or research report writing, where the introduction is a descriptive knowledge base, ideally with hypothesis based on knowledge, and a discussion containing critical analysis of evidence is essentially what many of us aim to replicate in traditional essay-style exams. So an essay question along the lines of:
Discuss the role of X and Y the process of Z. In answering this question, explain the relative contributions of X (and/or Y) on Z in some relevant setting.
This is a fairly typical degree-level essay exam. Some knowledge recall, in that for good marks, you need to define X and Y before putting them in context of process Z. In the second part, the students have to make some judgements based on the relative importance based on knowledge, which is not possible from memorising a 2D flow diagram. This exam structure is used below in one of my cancer-related exams that I might ask at Final year degree or Master’s-level Molecular Pathology students. Forgive the technical detail on this, but it is really very very interesting…
Discuss the role of p53 and pRb on regulating the cell cycle. In answering this question, explain the relative contributions of inactivation of genes coding for p53 and pRb in Cervical cancer vs Colon cancer (Feel free to skip this bit, but for those who are bothered or interested):
p53 and pRb are proteins that stop cells dividing at different stages of the cell cycle (that is, time from division to next division) by inter-related and distinct mechanisms. Loss of these (by gene inactivation, or protein degradation) promotes tumour growth. In cervical cancer, genes coding pRb and p53 are very rarely inactivated because Human Papilloma virus (HPV) inactivates/degrades p53/pRb protein, so no there is no selective pressure on inactivating the genes coding for p53/pRb protein. In contrast, p53 gene is usually mutated (inactivated) in Colon cancer, and pRb is either inactive, or something controlling pRb ‘messes up’ the cell cycle control process…
Is this an authentic assessment? Well, according to many experts in education, and the expert definition of authentic assessment, no it isn’t, given that I can get the answer from Google and that it is not a ‘real-life’ situation. I can get nicely written reviews on the topic, and even Wikipedia distils this down to the basics quite nicely, so no, this is not authentic at all. Here are the 5 ‘not-at-all-trying-to-induce-a-false-dichotomy’ features of Traditional vs. Authentic assessment by Jon Mueller, as obtained via Wikipedia.
Traditional Assessment vs. Authentic Assessment:
1) Selecting a Response vs. Performing a Task
2) Contrived vs. Real-life
3) Recall/Recognition vs. Construction/Application
4) Teacher-structured vs. Student-structured
5) Indirect Evidence vs. Direct Evidence
In my essay, students are 1) performing a task which is authentic to any scientist. Yes, they are selecting a response from memory and relaying that information, but they must apply their knowledge of why mutations arise in a certain manner. It is a 2) real-life task, in that it could represent the key elements of a justification/intro for a research proposal or research paper studying these genes in either cancer type. Yes, this is all on Google, but how would you know what to search for without the appropriate knowledge-base and understanding of how to put that information in to context? Although the first part of the exam question above relies on the traditional recall. 3) Recall of knowledge “Discuss the role of p53 and pRb on regulating the cell cycle”, the question then asks students to apply this knowledge, and hopefully construct their own hypothesis of why certain events occur. They can only properly apply the knowledge if they understand two important concepts that have been taught previously, but are not specifically asked for. They must also recall and relate the causative events in Colon cancer. They must then ‘explain the relative contributions of inactivation of genes coding for p53 and pRb in Cervical cancer vs Colon cancer‘. So although both pathways are ‘messed up’ differently in each tumour type, the net effect is the same: Cells grow fast and refuse to die. That is an important point to appreciate, and this essay allows me to assess whether the students have grasped this concept. So how about 4) Teacher-structured vs. Student-structured (whatever that means): As helpfully defined by Jon Mueller, “A student’s attention will understandably be focused on and limited to what is on the test. In contrast, authentic assessments allow more student choice and construction in determining what is presented as evidence of proficiency.” Well, it’s an exam, designed and written by myself. I really don’t understand how a student-structured task makes it more authentic, other than to tick the ideologically ‘non-traditional’ box. I suppose I could leave it up to the students to write about a tumour type of their choice, or cell signalling pathway of their choice, but that might miss some conceptual nuances that I want to test. As the teacher/’expert’, I know that these pathways are relevant to ALL cancers, unlike a student’s possible choice of some random cell signalling pathway. I know that studying these two tumours highlights crucial differences in the mechanisms of how these important regulatory pathways are bypassed in cancer. Studying say, Colon vs. Pancreatic wouldn’t cover these nuances, but the students don’t know that. Finally, what about 5) Indirect evidence or direct evidence: Again, as helpfully defined by Jon Mueller “What thinking led the student to pick that answer? We really do not know. At best, we can make some inferences about what that student might know and might be able to do with that knowledge. The evidence is very indirect, particularly for claims of meaningful application in complex, real-world situations.” I know why a student has come to the final conclusion at the end of my essay question as I have tested the underlying fundamental knowledge first, and seen how they have applied their interpretation of the fundamental knowledge to a situation that requires synthesis of ideas from distinct parts of the curriculum. So yes, direct evidence is used to assess the student on application of knowledge. So the startling conclusion is that essay-style questions under test/exam conditions can be ‘quite authentic’, even if the answer can be obtained directly from Google. It also leads us to conclude that being able to Google the answer to tests is not necessarily a good way of assessing students. Finally, if a quote appears to be obviously ridiculous, it probably is ridiculous. There are so many situations where we cannot reasonably be attached to the internet. Fundamental knowledge recall will always be needed and hence, it’s probably worth ‘testing’ whether students have this prior to progression academic progression. As a scientist, here are just a few:
1) Understanding subsequent teaching sessions that rely on prior knowledge.
2) Listening to a conference paper presentation and wondering how X relates to Y, and whether it might also relate to Z, which the authors had not considered (but will form the basis of my next research project).
3) The PhD voice viva examination. A must for any independent research scientist.
4) Any voice viva examination at any level, so a must for any of my BSc or MSc graduates.
5) Being interviewed on how your recent research findings are relevant to X by the media, and how they might relate to something else (usually quite obscure…).
6) A lab meeting with the professor: “How does X relate to Y. What is Y? Why are you studying Y if you don’t know what Y is…? Get out of my office…”
7) Being interviewed for a job, where getting an interview may in part be based on your knowledge base and ‘decent grades’.
“Oh hang on, I’ll just ask Google”. Yes, that will go down well…
PS: I rarely use Wikipedia as an initial research tool, but given the Godin, Mitra and Mazur quotes, I thought I would give it a go. Sorry. I won’t do it again.
PPS: In future posts I’ll discuss how I get students to answer questions based on interpretation of data from scientific research papers under exam conditions. I’d go as far as to say that this is also ‘somewhat authentic‘.