This takes two forms. If the article is written in a readable and lively manner, it is "too slick." If you go out of your way to write in a scholarly fashion, it is "too dry, too formal." My experience has been that you take those reports, make some minor changes to the article, and then it is eventually run. I've been on the receiving end of this three times now, and each article was eventually published and then cited in other peer-reviewed journals. A friend of mine got a comment on his second submission to JASIS: "You really improved the introductory remarks in this article." He was happy for the comment, but he hadn't changed a word in that section.
All of this inspired me to check out something that Herbert White had written - he said that editorial boards in the scientific journals are normally populated by the top writers in that field - people who had made their reputations by turning out a high volume of quality research. I wrote White, and he told me that he had never formally checked to see how well this applied to the library science journals, so I did the research. It turned out to be a simple matter to verify this in a selection of the library science journals, because most of them listed the names of editorial board members on the masthead. After that, it was just a case of looking up the names in Library Literature. This study was conducted in 1995, and I will do the same searches in 2001 as a follow-up.
The following file contains the results - judge for yourself. To summarize, more than half of the editorial board members have not written for the publication that they work for. Another 15% have never published anything at all - not even a book review. We started working on a parallel project in the field of biology and found that these people Were the top writers in their field. Clearly, the people involved in scholarly publications in library science should take a good look in the mirror. Could it be that an appointment to an editorial board is a reward for committee service, rather than a way to further true scholarship?
There is a steady pool of librarians who need to have one article in a refereed journal to make tenure. The law of supply and demand being what it is, the writers have to put up with the referee's behavior, no matter how hostile and posturing, because they need to get in that journal. My own experience is that the editor (who needs to fill a journal with good material) will step and and override the referees, no matter what they suggested. If editors followed all of the referee's reports, quarterlies would become annuals. I will stop short of calling this all a sham, because I liked the long-term attention that I received for my few peer-reviewed articles.
The journals are listed by name, but I have taken out the names of the board members, a few of whom are people that I know and like. I sat on this data for several years, but as of August, 2000, it has been quietly available on my web page, and I note that dozens of librarians have taken a look: