@QueroJeremy @lemechantneolib Je ne t'ai pas demandé ta "street cred" lmao. Tu utilises des arguments de bureaucrate/brasseur de papier. "Mes directeurs n'aimeraient pas ça, ce n'est pas revu par les pairs". La revue par les pairs c'est de la merde : http
@bxjaeger @cbokhove @ceptional @renebekkers @psforscher Yes, interrater reliability of peer reviewers is low. Bornmann et al (2010): https://t.co/N1yGQMenWP Cicchetti (1991): https://t.co/eS6YEyyHXi Cole et al (1981): https://t.co/aiQZNuTIGY Jackson e
RT @KirkegaardEmil: Peer review is super important to science guys, it keeps the bad science out! Expert peer reviewers spot the bad stuff…
RT @doughertyorama: So how does peer review fare in terms of reliability and validity? Turns out it's pretty miserable. Like so miserable t…
RT @doughertyorama: So how does peer review fare in terms of reliability and validity? Turns out it's pretty miserable. Like so miserable t…
RT @doughertyorama: So how does peer review fare in terms of reliability and validity? Turns out it's pretty miserable. Like so miserable t…
So how does peer review fare in terms of reliability and validity? Turns out it's pretty miserable. Like so miserable that any paper reporting reliabilities at the same level as obtained for peer review would be soundly rejected. https://t.co/bxdsk0PYBe
@KirkegaardEmil Sean Last cites these two studies in his article on trusting academic experts https://t.co/RHCrCAY4vb - this one is probably what you're looking for https://t.co/KSYruFT634 - this is a meta-analysis on inter-rater agreement more generally
@lakens - Here's a meta-analysis of journal peer review, showing reliability is low but never really 0 - Reliability using *content* of reviews (qualitative) might be higher than quant measures https://t.co/P8iBcoWLOd
@AmandaAskell For example, low inter reviewer reliability https://t.co/ePISPCuIpA
RT @KirkegaardEmil: Peer review is super important to science guys, it keeps the bad science out! Expert peer reviewers spot the bad stuff…
Peer review is super important to science guys, it keeps the bad science out! Expert peer reviewers spot the bad stuff consistently and block publication. https://t.co/DFRibIBhQE https://t.co/G3BgneSHW0 https://t.co/JM9ZypMBvv
(2/3) First, it neglects the abysmally low reliability of reviewers. At Social Problems, it was r = .134. Other studies also find that journal reviewers agree "at levels barely beyond chance" (Kravitz et al., 2010: 1). Here is a meta-analysis: https://t.
@NoraNewcombe @tuebang There are various studies on this that I know of, none of which make me very optimistic about it. See the quoted tweed, for example, but there are more. If you know of very positive studies, let me know. https://t.co/XO29fyee10
RT @trished: In journal #peerreview low IRR btwn reviewers (https://t.co/A5bFdOXDXs) often matters < usefulness of comments. But in #grantr…
RT @trished: In journal #peerreview low IRR btwn reviewers (https://t.co/A5bFdOXDXs) often matters < usefulness of comments. But in #grantr…
RT @trished: In journal #peerreview low IRR btwn reviewers (https://t.co/A5bFdOXDXs) often matters < usefulness of comments. But in #grantr…
RT @trished: In journal #peerreview low IRR btwn reviewers (https://t.co/A5bFdOXDXs) often matters < usefulness of comments. But in #grantr…
In journal #peerreview low IRR btwn reviewers (https://t.co/A5bFdOXDXs) often matters < usefulness of comments. But in #grantreview where scores rule, low agreement surely worrying https://t.co/5vcw0Do46r
RT @Research_Tim: Ah, peer-review, the glorious gatekeeper of academic publishing! We might have a bit of a problem considering that the i…
RT @Research_Tim: Ah, peer-review, the glorious gatekeeper of academic publishing! We might have a bit of a problem considering that the i…
Inter-Rater Reliability for peer reviews is surprisingly low. https://t.co/l3trZYmrln
Ah, peer-review, the glorious gatekeeper of academic publishing! We might have a bit of a problem considering that the inter-rater reliability of peer-review is, well, worse than poor. (https://t.co/DkOwRRYv5T) https://t.co/zhGVVTAcSt
Es gibt eine saucoole Meta-Analyse von Bornmann et al zur Übereinstimmung von Peer Reviewern. https://t.co/SSEYWy3SjV
@cian_neuro but it's a pretty crappy bias ; big filter by non-professional scientist + random selection by n=2 https://t.co/NEzSmUdZzf
@maxcoltheart @EikoFried plenty of recent convergent findings, eg https://t.co/t9GQtHSOpk, little reason to think it would be better now imo
For all scientists who wonder why #PeerReview can be such a lottery sometimes https://t.co/uj5BLoqh96
Meta-analysis suggests peer-review bad at determining quality of articles. Gets better if refs rewarded. http://t.co/BXbdPkNfaa cc @Publons
Maybe this means peer review shouldn't be a judgement, but a collaborative conversation designed to make work better. http://t.co/IgeY5WyNdg
Hmmm, peer reviews of journal papers have poor inter-rater reliability http://t.co/ycYzQZQjOQ
I am increasingly being dragged, against my will, to the conclusion the peer-review is a waste of time and effort. http://t.co/eMeDhElqVU
Hmmm, peer reviews of journal papers have poor inter-rater reliability http://t.co/ycYzQZQjOQ
I am increasingly being dragged, against my will, to the conclusion the peer-review is a waste of time and effort. http://t.co/eMeDhElqVU
A Reliability-Generalization Study of Journal Peer Reviews: A Multilevel Meta-Analysis of Inter-Rater Reliability http://t.co/NExvEyVOcl
I am increasingly being dragged, against my will, to the conclusion the peer-review is a waste of time and effort. http://t.co/eMeDhElqVU
I am increasingly being dragged, against my will, to the conclusion the peer-review is a waste of time and effort. http://t.co/eMeDhElqVU
I am increasingly being dragged, against my will, to the conclusion the peer-review is a waste of time and effort. http://t.co/eMeDhElqVU
I am increasingly being dragged, against my will, to the conclusion the peer-review is a waste of time and effort. http://t.co/eMeDhElqVU
I am increasingly being dragged, against my will, to the conclusion the peer-review is a waste of time and effort. http://t.co/eMeDhElqVU
I am increasingly being dragged, against my will, to the conclusion the peer-review is a waste of time and effort. http://t.co/eMeDhElqVU
I am increasingly being dragged, against my will, to the conclusion the peer-review is a waste of time and effort. http://t.co/eMeDhElqVU
I am increasingly being dragged, against my will, to the conclusion the peer-review is a waste of time and effort. http://t.co/eMeDhElqVU
I am increasingly being dragged, against my will, to the conclusion the peer-review is a waste of time and effort. http://t.co/eMeDhElqVU
I am increasingly being dragged, against my will, to the conclusion the peer-review is a waste of time and effort. http://t.co/eMeDhElqVU
"A Reliability-Generalization Study of Journal Peer Reviews": http://t.co/tHQPHw3o
Peer review in science is broken. How can we save it or should we? http://t.co/1lhqABK