Yagmur Ozturk

Abstract:

The scientific record is dynamic, constantly growing and expected to be corrected when errors are found. However, the volume of problematic scientific publications is growing faster than formal correction mechanisms (e.g. corrections and retractions) or community-driven efforts, such as post-publication peer review (PPPR), can keep pace with. We address this problem by assisting PPPR with natural language processing (NLP), investigations of the scientific record and standardized guidelines. We pursue these fronts through three contributions. First, we investigate how problematic content enters the record in the first place. One such example of this is paper mills, businesses that sell authorship of scientific papers, often advertised through social media. Through a systematic investigation, we trace these offers to their resulting publications in conference proceedings and document recurring red flags in the literature. In doing so, we advance the knowledge on paper mill activity in conference proceedings, which is a venue that remains under-studied compared to journals. Second, we address a gap in existing PPPR tools, which largely rely on bibliographic metadata or pre-defined problematic textual content alone when re-assessing publications. We present CiteScreener, a decision support system that combines bibliographic and retraction metadata with NLP-based citation-context classification to help PPPR practitioners (regardless of their expertise level) identify problematic citations. CiteScreener's web interface has been evaluated by PPPR practitioners with varying levels of experience, who found it useful not only for surfacing problematic citations but also for identifying issues beyond its intended scope. Third, motivated by a recurring finding that practitioners need more guidance in PPPR, we present the Collection of Open Science Integrity Guides (COSIG). COSIG is an open-source project that curates and standardizes expert PPPR knowledge into an accessible source. Its aim is to reduce the barriers to entry that currently keep effective PPPR participation concentrated among a small number of experienced practitioners. Together, these contributions address the scientific record's correction problem from three complementary angles: understanding the problematic content, equipping practitioners with tools to identify such content, and curating the knowledge needed to act on it.