“Your literature review is sparse and full of self-citation”: Content analyses in LIS

Image by Karolina Grabowska from Pixabay

So a few months ago I submitted the follow-up to my C&RL article from last year for review. This one is an analysis of LIS literature to try to determine how prevalent the study of research is in this field. The feedback I got from reviewers was helpful and I recently submitted a heavily revised (hopefully improved) version of the article. Two pieces of feedback I received on the original, though, stuck out. Both were from the same reviewer and they gave me a lot to think about.

The first was when the reviewer highlighted a line from my abstract that read: “The study of research is prevalent in the LIS field.” The reviewer felt that this was a truism and that it should be removed. I had to look up what this means. It turns out a truism is a statement that’s so obviously true that it basically doesn’t mean anything.

The second was about my lit review, which the reviewer felt was “sparse” and “skewed to self-citation”(1).

I had to do a lot of work in my revision to try to untangle that second comment and in doing so I found myself also untangling the other.

Now, with all due respect to the reviewer, the lit review I wrote contained little to no self-citation. Which is to say, there were places in the article where I did cite myself, mostly to refer back to my previous article but none of these references (in which I redacted my name to preserve the blind peer review process) were in my lit review. It could be that, in my writing, I just wasn’t making it clear enough when I was discussing my previous work and when I was citing others. Or it could be that the reviewer saw that I was heavily citing one particular author and mistakenly thought that I was that author. Or it could be that in referring to the “lit review” the reviewer meant all of my cited sources, not just the ones cited in the actual lit review section. Either way, I wasn’t mad about it. It just seemed like a possible misunderstanding.

The second part of the comment, that my lit review was sparse, was also puzzling at first. I’d cited about 30 sources, almost all of them various content analyses that had been done of LIS literature over the years. I was pretty sure I had all of the “key” analyses that get cited over and over again plus a few more that are probably less well-known but seemed worth throwing in for the sake of completeness. The whole thing seemed pretty thorough to me.

But I’d learned my lesson in the past when it comes to being too dismissive of peer reviewer comments I felt were off-base, so I took another look at what I’d written.

The peer reviewer had a point. First, the lit review was short. It’s true that there were 30 or so sources cited but the majority of them were dispensed with in about two sentences so I could spend a couple of paragraphs focusing instead on the one that made up the backbone of my own study. The only point I really made with them was that no previous content analysis had used the research-as-subject metaconcept lens. Because the research-as-subject metaconcept didn’t exist before my previous article. At least, it hadn’t been articulated as such.

What, exactly, was I supposed to do with this?

I ended up going back and re-reading every single article in my lit review plus about 10-15 more that I found to make sure I really was covering all my bases. I created a spreadsheet(2) to keep more careful track of what LIS research topics these scholars had identified as part of their work. I read them in chronological order by publication date so that I could get a sense of how this had changed over time.

What I found was that no two content analyses which purport to identify popular research topics in the LIS field describe those topics in exactly the same way unless the purpose of one of those content analyses is to validate another’s findings. There are dozens of ways of describing LIS research topics. In a field that prides itself on its ability to describe information in a consistent fashion so that it can be more easily discovered, this is both amusingly ironic and a little maddening.

I rewrote my literature review so that it articulated in more detail what these content analyses had found. This helped me make the point that, for all the ways we have found to describe the scholarship in our field, no one has used the “study of research” lens before.

Which brings me back to that first comment the reviewer made, calling the statement that “the study of research is prevalent in the LIS field” a truism. So obvious as to be meaningless.

No one has ever described what LIS scholars do as “the study of research” before. But the idea that the study of research is prevalent in our field is a truism. How can both be true? It’s so weird.

The thing is, the research-as-subject metaconcept (that research is both an activity and a subject of study) isn’t supposed to represent some brand new way of thinking. In the spirit of Adler-Kassner and Wardle’s work in the writing studies field, it was always meant to be a way of putting a name to something we already knew. Articulating what was always there.

So the reviewer was right. That LIS scholars study research is a truism. To us. So much so that we’ve never needed to name it to ourselves.

The problem is that no one else knows this. They think research is a basic skill and that what librarians do could be done by unpaid volunteers instead of trained experts. But what we do is based on the knowledge we build about how the research process works in various contexts. And we have to keep rebuilding that knowledge because the way people interact with information is constantly changing.

I’m hoping that the changes I made to my article, based on comments from this reviewer and the others who read my work, have made it stronger. It’s certainly challenged me to think more about what, exactly, it is I’m trying to accomplish with this work and to find better ways of communicating it.

But I guess I’ll have to wait for the second round of reviews to find out. Fingers crossed.


(1) These are the actual quotes. I’m just noting that so you don’t think the use of quotation marks is sarcastic or something.

(2) The name of the spreadsheet file is “your lit review is sparse and full of self-citation” because apparently I think I’m hilarious.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s