Opinion Paper

Avoiding the elephant in the room: The real reasons behind our research crisis

Crystal Hoole

Received: 30 July 2019; Accepted: 02 Oct. 2019; Published: 12 Dec. 2019

Copyright: © 2019. The Author(s). Licensee: AOSIS.
This is an Open Access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

Problemification: In recent years, the so-called publication crisis has reached alarming proportions, and the psychology and industrial psychology profession are not left unscathed. Efendic and van Zyl investigated the crisis and emphasised on open science practices as a strategy to address the issues.

Implications: This article argues that the problem is much deeper than a matter of replication and should be approached from a systemic and holistic perspective. The author argues that the root causes can be grouped into four main categories, namely policies, ethics, systemic or institutional and research methodology. Unless the root causes are not addressed, the crisis will deepen.

Purpose: The purpose of this article is to reflect on Efendic and van Zyl’s analysis and offer additional insights.

Recommendations: A number of additional strategies are offered to address the real problems behind the replication problem.

Keywords: Research methodology; Replication crisis; Higher education; Academic dishonesty; Ethics; Policies.

Introduction

Efendic and Van Zyl (2019) presented an extensive case for introducing open science practices and methodological improvements for addressing recent concerns about the apparent failing scientific rigour in Industrial and Organisational Psychology (IOP) science.

According to the authors, the IOP discipline has been plagued by several issues in recent years. These include deceptive research practices, inadequate and incomplete methodologies, and a general lack of adhering to good scientific research principles and obtaining sufficient ethical clearance. ‘Replication crisis’ has been used as an umbrella term to refer to several issues that place a question mark on the validity and quality of research practices. The problem is not confined to IOP and psychology in South Africa or only the related professions for that matter, and the sheer magnitude of the broader dilemma on the local front warrants a South African perspective.

Efendic and Van Zyl (2019) did a sterling job in providing a comprehensive analysis of a wide range of problems, which do not need to be repeated here. Not intending to lessen the importance of their work, this article aims to provide a big-picture perspective and a reflection on the actual reason behind the current crisis, and how Efendic and Van Zyl’s suggestions can be further promoted. Rather than responding on a point-by-point basis, the authors’ sayings are reviewed using a holistic approach. I do not believe that the issues or symptoms at hand can be adequately addressed without getting to the root of the matter. This article is, therefore, both a response and a reflection, in an attempt to contribute to the solutions. I will first address the issues and then offer potential solutions.

Real reasons behind the crisis

From a sense-making perspective, the list of issues can be broadly grouped into four main categories, namely research methodology, policies, ethics and systemic or institutional issues and research methodology. These categories overlap and cross-pollinate, which makes it almost impossible to address them individually. Understanding and correcting these are going to require a holistic and integrated approach. The overemphasis on aspects such as replicability suggests that many do not fully understand the underlying dynamics causing the crisis (Salteli & Funtowicz, 2017).

Although the majority of issues listed fall under research methodology, these are symptomatic of a much deeper problem that can be linked to the dilemma facing higher education specifically in South Africa, and also in Africa and elsewhere in the world. Granted that efforts such as the implementation of open science practices and improvement of methodologies are commendable and should be followed, the majority of efforts should focus on the root cause of the weakening scientific output. The recent rise in concerns about replicability, small sample sizes, manipulation of data and so forth are merely the scapegoats for something that has lurked under the surface for a long time and has now reached the boiling point.

Systemic and institutional issues and policies

The higher education scene has faced significant changes and challenges over the last two decades. Tertiary institutions are expected to increase their postgraduate numbers to address previous inequalities. The National Development Plan (2013) proposes a target of more than 100 doctoral graduates per million people by 2030. To move from a resource-based economy to a knowledge-based economy, research and, especially, postgraduate education are seen as the catalysts to making this happen (NDP, 2013; Sonn, 2016; Thaver, Holtman, & Julie, 2013).

Furthermore, higher education institutions are now required to align their objectives with the government’s socio-economic development goals aimed at the disadvantaged (Phakiti, 2008). Having such lofty objectives and directives is one thing, implementing these in practice is another. In reality, the current higher education sector is reeling under many aggravating circumstances that will make it very difficult to reach these goals. On the one hand, universities are required to increase their number of students, specifically postgraduate students, but, on the other hand, their resources to do so are considerably reduced. One of the most surprising and even outright contradictory factors compounding the situation is the government’s adoption of a neo-liberal ideology where social spending such as funding of public education is drastically curbed, which has far-reaching consequences for most universities (Sifuna, 2014). Recent events such as the #FeesMustFall campaign marked a shift towards making more funds available for tertiary education access and support, but do not help the current dilemma – universities are forced to cut costs and supplement their income through other avenues.

The shift towards incentivising research outputs as part of the government’s higher education funding model is one of the few available opportunities universities have to supplement their income. This practice has been criticised frequently as discussed by Efendic and Van Zyl (2019). The demand to ‘publish or perish’ is a direct result of this strategy. The pressure to publish is in fact a worldwide phenomenon, and many authors have warned about its perils (Editorial, 2018; Pfleegor, Katz, & Bowers, 2019; Van Dalen & Henkens, 2012).

Another problem related to higher education is ageing academics, with the majority of academics being middle aged and fast approaching retirement. There is also a noticeable trend of an upward trajectory of the number of junior lecturing staff and a decrease in senior lecturing staff– mostly associate professors and professors (Breetzke & Hedding, 2018). Many universities have a strict retirement policy for people aged 60 or 65 years but do not have adequate plans to retain them once they retire and utilise the institutional and research-related knowledge that these academics have. Universities should urgently devise strategies to address this threatening void.

The lower salaries and fierce market competition are factors that have made this profession unattractive for the younger generation (Breetzke & Hedding, 2018).

Another contributing factor is the growing emphasis on post-graduate education. This is increasing the load on the already over-burdened academics who are capable of supervising postgraduate students (Tintswalo, 2017). Department of Higher Education and Training (DHET) (2012) reported that only a fraction of Master’s (20%) and Doctoral students (12%) complete their course. In 2015, the DHET reported that although there was an increase in the number of Doctoral students (2797) completing their studies, it still fell short of their target of 12 000 by 2019. This indicates an alarming waste of time and resources, which we can ill afford.

There are many reasons for this tragic state of affairs. These include a failing basic education system, debilitating financial burdens, lack of institutional resources, cutting of government subsidies, increasing student numbers and growing tension between universities’ struggle for survival and devoting resources to their core functions, which are teaching and research (Boughey, 2018; Sifuna, 2014; Swartz, Ivancheva, Czerniewicz, & Morris, 2019). The two biggest reasons, however, are integrally linked to the ‘replication crisis’. Firstly, there is a relentless push for publications. Efendic and Van Zyl (2019) allude to the DHET’s strategy to incentivise research output, provided the output meets certain criteria that benefit universities and researchers alike. Many universities favour research output as one of their key performance areas, linking the number of publications to the promotion and awarding of research grants. For many academics, there is no choice but to focus their efforts on research and publication.

The ever-increasing quotas for research outputs lead to all sorts of undesirable outcomes, notably placing quantity over quality, targeting predatory journals, taking research shortcuts, Hypothesising after the Results are Known (HARKing) practices, manipulation of data and unethical behaviour such as resorting to plagiarism (Mouton & Valentine, 2017; Murphy & Aguinis, 2019; Thomas, 2019).

The increase in dishonesty and issues such as unethical behaviour and plagiarism is a more serious matter, as it lies at the heart of academic integrity. Even though a number of papers have been written to highlight these issues (Thomas, 2019), very little action has been taken. Thomas (2019) reported that many universities and journals are reluctant to deal with academic dishonesty among faculty and that many journals still do not have formal policies on plagiarism.

Research methodology

The second reason, and potentially the biggest contributing factor to our current research crisis, relates to the failure of adhering to good scientific practices and, more specifically, the lack of adequate training in research methodology and practices. As alluded to before, the problem is complex, with many contributing factors. As Efendic and Van Zyl (2019) point out, only a small percentage of researchers deliberately manipulate their data and fabricate their findings to suit publishing criteria set by journals. Many of the factors identified, such as the pressure on academics to publish and supervise and increasing number of students, inadequate exposure to research methodology at the undergraduate and postgraduate level, low throughput levels of postgraduate students and the inadequate research capacity of competent researchers, show that at a systemic level universities are not geared to achieve NDP’s research objectives.

Several papers have reported on the reasons why students struggle to complete their post-graduate research (Botha, 2018; Sonn, 2016; Thomas, 2017). Reasons range from difficulty with the identification of research problems, difficulty with academic and proposal writing, poor understanding of which research methods to apply and inability to execute and report on the statistical analyses, to poor supervision and failure of supervisors to provide the required in-depth guidance. In all these instances, training in research methodology is inherently problematic and inadequate.

Lack of skill and research knowledge is not unique to students and supervisors; it is also evident in journal reviewers. As gatekeepers to ensure scientific rigour, journals should have top researchers in the field acting as reviewers. Many journals do not apply rigorous selection criteria in selecting their reviewers and rely on volunteers, for which there is seldom adequate recognition. There is very little evidence of quality control to ensure that reviewers indeed have the required level of expertise to review a wide range of manuscripts.

Efendic and Van Zyl (2019) present a strong case for adopting open science practices, where researchers must make their full data protocol available. This may be a bit ambitious, in the sense that expecting journal editors and reviewers to scrutinise every manuscript’s metadata and syntaxes may not be feasible, especially for journals that receive a high number of submissions. It is a good practice to have this information available in cases where the manuscript is not clear and does not adhere to the normal requirements, but it should not be up to the journal’s review team alone to try and solve the crisis.

Possible solution for systemic and institutional issues and policies

The first systemic issue relates to academic output requirements. The roles, responsibilities and context of academics have changed dramatically over the last few years, yet very little has formally been done to investigate the effects of all these increasing demands on academic performance. A lot has been written about the adverse effects of the increasing demands on academic wellbeing, but mostly from a research and not an institutional and policy-making perspective. One cannot keep adding demands without including adequate resources and support. This issue cannot be solved in isolation and must be addressed at the highest level of higher education. Higher education dictates what happens in universities to a large extent, and unless drastic steps are taken for looking at the sustainability of all the added demands, more and more academics will leave the profession and lead to a further deepening of the crisis.

Regarding the problems surrounding aging academics, possible strategies such as enrolling retired academics into mentorship programmes aimed at young academics, appointment as research associates where they continue to supervise students and even amending policies to relax the retirement age for productive academics can ensure that institutional knowledge and years of experience are ploughed back into academia.

Dealing with academic dishonesty requires a more hard-line approach, both within universities and from a journal perspective. Sub-standard papers and the use of inferior journals can be addressed by implementing appropriate policies and practices, and also promoting more collaborative practices. Several authors have called for collaboration, both cross-disciplinary and cross-institutional. It is, however, imperative that current reward structures change from where individual performance is rewarded and credits for research are highest when working either alone or in small teams, to rewarding collaborative practices, as well as revisiting the credit allocation system.

From a journal perspective, in support of Thomas’s suggestions, a zero-tolerance approach should be adopted. In conjunction with the strategies many universities and journals already apply, such as the use of software to detect similarities, oversight by ethical committees and hosting ethical behaviour workshops, firm action should be taken against trespassers. Journal Editors must take a hard-line approach by making ethical clearance a compulsory requirement for submission to their journals, as well as simply not accepting articles without the opportunity to correct them, promotions must be reversed if these were based on the results of unethical practices, research subsidies must be paid back if criteria are not met, and individuals involved in unethical behaviour must undergo remedial programmes if they are first-time offenders. Repeat offenders should not be allowed to submit to the same journals again. Only when the consequences are tangible, will behaviour change accordingly.

Possible solutions for research methodology issues

As alluded to before, research methodology problems lie at the heart of the so-called replication crisis. If research is such a key priority, it should be reflected in tertiary curricula from as early as first-year education. The rigour and standard required by top journals, competition in publishing in high-impact journals and the recent emergence of open science practices require a more advanced level of research knowledge and execution. Gigerenzer and Marewski (2015) further argue that another problem that exists is that many researchers read statistics like a recipe. What is required is that these be used as a tool for decision-making, which requires understanding and critical thinking. It is impossible to learn this in just a year or two, and, without the guidance of a seasoned researcher, errors are bound to occur.

Going back to basics and adhering to good scientific principles of research will address many of the concerns raised. Sufficient training from the first year and continuous training throughout a professional’s careers will go a long way in building capacity and ensuring high-quality research. Furthermore, the current reality of many research studies in South Africa as well as globally is that the researchers are often not able to gather large enough samples to ensure statistical power (Schmidt & Oh, 2016). When the focus is on quality and not quantity, researchers will put in extra effort to meet more stringent requirements such as larger samples.

Journals can and should do more to educate their reviewers on the latest developments and perhaps offer refresher courses on the latest practices. Journals should also identify minimum criteria for reviewers, such as the number of publications they have produced.

Journals can also stipulate in their requirements that, unless all the information required to come to reasonable conclusions is submitted, the manuscript will not be reviewed. It may also be helpful for journals to review their publication requirements, such as the maximum number of words. With the advent of open science and digitisation, allowing authors a bit more space to provide the additional information will also address some of the issues.

Lastly, a potential way of building research capacity is the use of social critical theory principles to develop communities of practice. According to Wenger (1998, p. 4), as cited in Sonn (2016), capacity-building has four building blocks, namely, learning as belonging, learning as becoming, learning as experience and learning as doing. Communities of practice are where teaching, learning and research, come together, and where transference of learning and knowledge takes place. Again, incentive policies will have to be aligned to promote this ‘us’ behaviour, as opposed to individual action. One way of achieving this is through continuous professional development. Currently, the profession of psychology in South Africa is regulated by the Health Professions Act No. 56 of 1974, which requires all registered psychologists in all categories to earn 30 continuous development points per year. Six of these must be ethics-related. It will serve the profession well if a similar requirement is set for training in research methodology.

Conclusion

The current crisis of replicability is much deeper than it appears, and is affecting the very ethos of science, with higher education at its centre. Its impact is crippling and far-reaching. The solutions are complex and multifaceted, and the issue cannot be remedied by addressing factors in isolation. Policy-makers, higher education institutions, researchers, academics and journal editors alike will have to collaborate to come up with a suitable and sustainable system.

It is crucial that the DHET relook its policies related to higher education and how it should be funded. It is a much broader issue than offering monetary rewards for research outputs. The very essence of government’s role in higher education must be questioned, as well as the changing role of universities and how they must be supported. Universities in developing countries all over the world are continuously facing multiple and incompatible demands (Sifuna, 2014). Can we realistically expect universities to be both developmental and centres of excellence at the same time? History in other parts of Africa and elsewhere in the world suggests that this is not feasible (Gray, O’Regan, & Wallace, 2018; Sifuna, 2014; Van Der Walt, 2017). All indications in the current South African higher education landscape point towards similar failures. We need to learn from others and not repeat their mistakes.

Lastly, SAJIP should continue to play a pivotal role in ensuring that its practices are aligned with best practices of providing a state-of-the-art platform for the dissemination of knowledge. Open science practices should be promoted and researchers must be encouraged to add additional information such as the surveys used as an appendix. It should update its guidelines in terms of what is acceptable and what information is required for all manuscripts to be considered for review. Furthermore, SAJIP should consider offering its reviewers refresher workshops from time to time, to ensure that the quality of manuscripts is constantly elevated. This could potentially also increase the journal’s impact, as it will attract better research papers, the quality of reviews will be better and the overall governance of the review process will improve. Another potential intervention that could be considered is starting communities of practices in collaboration with tertiary institutions, journals – locally and internationally and professional bodies, through which the objectives of science, as well as the needs of society and business, are promoted.

As Efendic and Van Zyl (2019) deliberated, the problem is complex and multifaceted, as is the solution, and cannot be provided by any entity in isolation. Policy-makers, higher education institutions, researchers, academics and journal editors alike will have to work together to restore trust in our scientific work and to preserve the integrity of our profession.

Acknowledgements

The author declares that they have no financial or personal relationships which may have inappropriately influenced them in writing this article.

C.H. is the sole author of this research article.

This article followed all ethical standards for a research without direct contact with human or animal subjects.

No funding was obtained for this work.

Data sharing is not applicable to this article as no new data were created or analysed in this study.

The views and opinions expressed in this article are those of the author and do not necessarily reflect the official policy or position of any affiliated agency of the author.

References