Image by Gerd Altmann from Pixabay
In the time since I started writing about the contextual nature of research and research as a subject of study, I’ve noticed that I have a habit of using the phrases “information literacy skills” and “research skills” more or less interchangeably. But really IL and research aren’t one and the same. So I’ve started wondering lately where exactly the line is between them and wanted to spend some time thinking through this issue.
Image by StockSnap from Pixabay
As an academic librarian, I tend to think about the contextual nature of research mostly through the lens of the academic library environment. Specifically, information literacy, since that’s my specialization.
But before becoming an academic librarian, I spent some time in public libraries: three years as a clerk at a small public library in my hometown and then two years as an intern at a larger public library in the suburbs near where I went to grad school. As an intern, I spent some time at the reference desk and helping out with programming.
Some recent conversations have gotten me thinking about how all this talk about the contextual nature of research might apply not only in the academic library environment but also in public libraries. Thinking back on my own experiences working in public libraries as well as my continuing experience as a public library patron, I actually think public librarians are in many ways better primed to address the importance of context to the research process than academic librarians are.
Image by Myriam Zilles from Pixabay
I’ve mentioned it a couple of times before but I wanted to spend a little time talking about Naming What We Know: Threshold Concepts of Writing Studies, a book by Linda Adler-Kassner and Elizabeth Wardle which was the main inspiration behind the research I’ve been doing related to research as a subject of study.
The main reason I originally picked up Naming What We Know is because the ACRL Framework had recently introduced the idea of threshold concepts into thinking about information literacy and I was still trying to get my ahead around what threshold concepts even are. I’d read a bunch of stuff by Meyer and Land, the researchers who originated the idea, but a lot of the examples used in those books are from economics, biology, and other fields of study that are outside my expertise. So I was excited to find a book on threshold concepts for writing studies.
As an information literacy librarian, writing studies is considered outside of my professional realm but there are some connections there. For example, at my institution, our writing and critical inquiry program has a close relationship with our information literacy department (or, more accurately, my colleague who is the liaison to that program) because as part of those courses first year students have to write at least one research paper, which means that in addition to this being their first encounter with college-level writing, it’s also their first encounter with college-level research.
Besides that, I also have a Bachelor’s degree in English with a concentration in creative writing, so I have at least some understanding of research in that field. At least more of an understanding than I do in some of the other more technical fields where I’d seen threshold concepts discussed.
Reading through Naming What We Know is what sent me on my current research path. Here are some thoughts.
Image by Wokandapix from Pixabay
So last month when I did a webinar for the GLA Carterette Series on some of my ideas for incorporating the contextual nature of research into information literacy instruction, there were a lot of great questions at the end about assessment. In answering them, I realized that this was something of a hole in my discussion of this topic and I wanted to see if I could address some of it here.
First, it might help to know why assessment is such a blind spot for me. Basically, the culture around assessment in my current institution is a lot different from what I think the norm is for most libraries. I experienced something closer to that norm at my previous institution, where we were asked to constantly assess student learning and some part of the library’s value (not to mention our value as a reference and instruction department within the library) was directly tied to our program-level learning outcomes and how well our students met those outcomes. All of this was, in turn, very closely tied to questions of student retention and the role the library played in the institution’s retention efforts.
Where I am now, there is certainly interest in making sure that what we teach contributes toward student learning and student retention. And there are conversations about finding a way to assess our teaching in order to speak to our value both in the library and on campus. But because instruction responsibilities here are so fragmented, any assessment effort on this level would require buy-in across several departments in the library. As you can imagine, there would be some difficulty there. For now, everyone just kind of does their own thing. That’s been a big part of what’s allowed me to take more creative approaches to my teaching, which is an aspect of my job that I’m very grateful for.
But these more creative approaches aren’t exactly useful if students don’t learn anything as a result. Hence: it’s time to talk about assessment.
I’ve mentioned before that part of the reason the ACRL Standards focused on basic research skills was because those are the things we can assess. It’s much easier to assess whether a student can successfully identify a scholarly source in a library database than it is to assess a change in their way of thinking. How do you measure something like that?
Of course, this is a question we’ve all been struggling with to one degree or another since the advent of the ACRL Framework, which uses threshold concepts instead of learning outcomes. Threshold concepts are literally all about changing someone’s way of thinking.
Teaching about the contextual nature of research is in a large sense about changing the way students think about research. It’s asking them to recognize that the conventions and methods of research are going to be different depending on the context in which research is taking place. Not just disciplinary contexts, but contexts outside of academia as well.
No matter what context of research you’re working with, there are going to be skills involved. So one idea for assessing the contextual nature of research is to determine what the skills associated with the context(s) you’re teaching are and assessing students’ ability to not only perform those skills but recognize the appropriate context for those skills. For example, if a student is searching for or citing a peer-reviewed source when you’ve asked them to perform the type of research associated with a non-scholarly or non-academic context, they’re showing that they have good research skills but that they’re not applying them to the correct context.
This is something that can be captured in a number of ways. You can observe a student’s information behavior to judge whether it’s appropriate to a given task. You can have the student create a research product and judge how well they show awareness of the conventions of a particular type of research. You can create a video that explains the conventions of a particular research context and then quiz students on their understanding of what they watched.
Of course, being able to judge whether students are using skills and following conventions appropriate to a particular context requires establishing what those appropriate skills and conventions even are. Not to mention establishing what the contexts of research might be.
In my own work, I’ve suggested a few very broad categories or “genres” of research, including academic, scholarly, personal, professional, scientific, and creative research. I even outlined some of the characteristics of these genres in my article introducing these ideas. But this outline was meant to illustrate a point rather than act as a guide. Clearly, more work needs to be done here.
But that doesn’t mean you can’t teach the contextual nature of research until that work is done.
In my own classes, I have quizzes that students take after reading or listening to a lecture that I’ve written on a given topic (it’s an online class). These lectures address the contextual nature of research in mostly general terms and I test students’ understanding of this concept by including questions like the following on the associated quiz:
What type of research are scholarly, peer-reviewed articles most appropriate for?
It’s a simple question that tells me a lot about how much students understand about this concept even without a lot of specifics about the conventions of each type of research. Students who get it right have shown me what they’ve learned. Students who get it wrong—like the surprising number who try to argue that peer-reviewed sources should be considered appropriate for all types of research because their other professors have always told them that they are the “gold standard” of credibility even after I’ve told them all the reasons this isn’t actually the case—show me that there’s still a ways to go before they cross that threshold of understanding.
I also had an experience recently where I participated on a committee whose charge was to create and implement a library research award for undergraduate students. As part of that work, the committee had to come up with a way to evaluate the work we were seeing, which could come from any discipline being studied on campus. We wanted to make sure the award process was open not just to students who had completed standard research papers but also those who had done research in connection to more creative projects and we needed a rubric to reflect that.
We ended up adapting a rubric (with permission) from one that had been used by several other institutions. But where the original rubric mentioned skills appropriate to a particular discipline, we substituted the phrase “appropriate to the context.” That might seem like a small change, but not all research takes place within an academic discipline. We also wanted to make sure that students who had conducted their research in more creative contexts knew that they were eligible for the award as well. Either way, the wording is a way to capture that an excellent research project is one in which the student applies skills and conventions appropriate to the context of the research.
So there’s not as much concrete information about assessment here as I would like. Like I said, assessment tends to be a little under my radar for a variety of reasons but this is something I’m going to continue to think about and share some thoughts on in the future. If anyone else has thoughts, I’d be interested in hearing those as well.
Image by PublicDomainPictures from Pixabay
I’m off for the holidays and won’t be posting any new content but I thought I’d pin a thing here highlighting some favorite past posts in case you missed them.
Thanks for reading and see you in the new year!
In my last post, I talked about how, like most instruction librarians, when I teach students how to evaluate information, I used to rely heavily on tools like the CRAAP test to help students through the process. In some cases, particularly one-shot sessions where there may not be time to teach students the nuances of this process, I still do. If nothing else, they are a handy way to help students learn that they should be thinking about the quality of the sources they are using. And they fit well on a handout.
The CRAAP test and similar tools were (arguably) a good fit for Standards-based teaching, where the evaluation of information was an explicit learning outcome. But even before we traded the Standards for the Framework, a lot of librarians were dissatisfied with these tools because they oversimplified the evaluation process. The CRAAP test in particular seemed to mostly only apply to internet sources, giving students the false impression that these were the only types of sources that needed to be evaluated. Plus, they didn’t stop students from just choosing whatever came up first in their list of search results, regardless of whether it was clearly biased or too old.
These criticisms are particularly true when you take into account the contextual nature of research, as the Framework does. Currency, for example, is important for some research topics, like those based on technology or science, but less so for others, like history and literature. And that’s just within academic and scholarly research. There are also likely to be differences in how information is evaluated in professional, creative, and scientific contexts as well.
While I don’t have much flexibility with one-shot sessions, I wanted to start thinking about how to adapt my usual lessons on evaluating information to also get students to think about the contextual nature of research. It turns out, it didn’t need much tweaking.
Here’s how the lesson works.
I’ve mentioned before that one of the cool things about the study of research is that it’s already out there, in so many forms and in so many fields (not just library and information science!), even if that’s not what the researchers doing this work would necessarily call it. I saw a lot of examples of this at the ACRL 2019 Conference and I wanted to spend some time here taking a closer look at a few of them.
So let’s take a closer look at “Reshaping the Library Literature: Scholarship Challenges and Opportunities for Technical Services Librarians at Smaller Institutions”by Heather Getsay and Aiping Chen-Gaffey.
You may not be aware of this, but every year the ACRL Instruction Section Teaching Methods Committee puts out two lists of Selected Resources, one focused on teaching methods and instructional design and the other focused on assessment. These lists feature articles and other materials that have been published the previous year that are worthy of note. It’s a resource that doesn’t get as much use or attention as it should, so I’ve decided to assign myself the homework of making my way through each item on last year’s lists and write about it here.
Today we’re all about “Using-Practitioner-Engaged Evidence Synthesis to Teach Research and Information Literacy Skills: A Model and Case Study” by Sarah Young and Mary Maley.
Disclosure: I am currently a member of the ACRL Instruction Section Teaching Methods Committee, which selects and evaluates materials for the Selected Resources lists. I played a role in the selection process and reviewed several of the items that ended up on the final list for the current year as part of that process.
Image by Wokandapix from Pixabay
I’ve mentioned before that one of the cool things about the study of research is that it’s already out there, in so many forms and in so many fields (not just library and information science!), even if that’s not what the researchers doing this work would necessarily call it. I saw a lot of examples of this at the ACRL 2019 Conference and I wanted to spend some time here taking a closer look at a few of them.
I hope the researchers whose work I plan to talk about for this series don’t mind that I’ll be applying the “study of research” label to what they do, but in each case I’ll try to make it clear why I’m doing that.
So let’s take a closer look at “Spinning a Scholarly Story: Using Faculty Interviews to Develop a Scholarly Communications Agenda for Liaison Librarians” by Teresa Auch Schultz and Ann Medaille.
Image by Fathromi Ramdlon from Pixabay
Early on in the information literacy course I teach each semester, I introduce students to a couple of common myths about research, things students commonly believe because of their experience with academic research. This includes things like “research is about finding the right answer” and “citation sucks” (which I tell them isn’t really a myth because, well, citation does suck).
Now that I’m spending some time thinking about the role of research in creative writing, I’m finding that there’s a whole other set of myths/beliefs that keep cropping up, ones that I hadn’t thought about or that don’t apply to the type of research I usually teach.
Studying Research 