In higher education, digital data is seen as transformative, and there is an omnipresent belief in its value. However, digital data is not inherently valuable; rather, it needs to be made so. This article investigates five challenges of datafication in the sector, addressing common misconceptions. It advocates for slow and responsible data innovation to meet the sector’s evolving needs.

Digital data is perceived to be valuable in contemporary economies and societies. In higher education, stakeholders believe that collecting, analyzing, structuring, managing, and using data and data outputs—such as analytics, dashboards, or scores—will improve the sector. Universities aim to make data useful and strive to become data-driven organizations in their strategic and operational activities. Educational technology (EdTech) companies strive to monetize the digital data they collect, i.e., make data economically valuable. Policy makers seek to base their decisions on real-time data. However, there are many misconceptions and challenges in realizing the data value imaginary in higher education.

In this article, five challenges of higher education datafication are identified investigating universities, EdTech start-up companies, and investors in EdTech in the United Kingdom. Datafication refers to quantifying social and natural worlds and representing them in machine-readable digital formats, often involving complexity reduction. The findings are likely applicable beyond the United Kingdom, as the investigated EdTech companies and investors work in many countries and global regions.

Data Is Not Inherently Valuable

There appears to be a consistent and omnipresent belief in data value across universities, EdTech companies, and investors in EdTech. However, this value is not really realized, at least not to the extent wished by stakeholders. Data value means different things to different stakeholders. On the one hand, universities are interested in using the value of data to enhance the student experience and the efficiency of administration processes, to personalize learning, and to automate processes. On the other hand, EdTech companies experiment with different strategies to monetize student and staff data that they collect. In both cases, making data useful and valuable proves to be hard and demands resources. Data is not inherently valuable; rather, it must be made so.

Good Data Practices Are Costly

Data processing demands technological, financial, and human resources. From the point of view of universities, data discourse promises efficiency and savings, but research points to an incredible amount of labor backing datafication. This includes academic and administration staff inputting and sorting data, testing and tweaking data outputs, changing working practices, and more. It also includes a need for new skills and jobs, such as data scientists, IT developers, project managers, and vendor managers, which also require organizational changes. Costs for digital infrastructure are rising, including moving to the Big Tech cloud infrastructure. Similarly, EdTech companies struggle with the cost of data practices, especially more sophisticated data analyses and outputs beyond descriptive statistics or simple comparisons. Moreover, should they wish to develop cross-institutional data insights, it takes about five years to develop big databases. They struggle to attract enough investment for high-quality data processes, as the return on investment is perceived to be lower in education than in other sectors.

Not All Data Is Useful

Not all data outputs are the same. Many university constituents feel that some data outputs released in EdTech products are not needed for teaching and learning. Moreover, some data outputs are not representative of what they claim to represent. In addition, not the same metric is needed to support teaching online or on-campus, yet it often feels that EdTech companies promote data outputs similarly. University constituents say that “simply because data can be collected and analyzed, it does not mean it should be.” Indeed, EdTech companies often experiment with data outputs in search of finding the right recipient who is willing to pay for it. Different data outputs and metrics can be developed and promoted to various parties, which could be to the detriment of other actors. For example, a metric showing how many students accessed an assigned reading and how much time they spent on it could be promoted as a measure of academic staff performance to university leaders. But whether student access to electronic text is, in fact, a good measure of staff performance is a different question.

Data Outputs Are Consequential

The most common way EdTech companies attempt to monetize user data is by “datafying products,” i.e., integrating data outputs into products with a different primary offering. For example, a video call platform could integrate analytics of call descriptors or participant engagement in the discussion on a call. As mentioned before, different data outputs and metrics can be produced for different audiences, including actors beyond universities, such as publishers (e.g., what titles are read, to what extent, and how) and governments (e.g., what skills are present or missing in a specific population). These data outputs are performative, i.e., lead to social action and have effects. They might be consequential at the institutional level (e.g., when a university decides to intervene in a student’s life based on an algorithmic score), at the commercial level (e.g., a publisher deciding which academic texts to publish based on reading statistics, which might be based on behavioral nudging), at the policy level (e.g., a government making policy decisions), and so on. Hence, it matters what kind of data outputs are constructed, by whom, with what purpose, and with what consequence.

Data Practices Are Not Democratic

Datafication of higher education is not challenging only technologically and legally, but is also a process full of contradictions and disagreements. Many individuals raise continuous concerns about data practices. Academics often see that previously established teaching and learning practices are more meaningful (e.g., close relationship between academics and students, and formative assessment) than extensive data collection and outputs (e.g., learning analytics). However, individuals raising concerns often feel accused of being against progress. If various metrics are imposed on individuals without them seeing the benefit, datafication will not reach the attempted aims. This indicates complex internal struggles and different motivations and aims regarding what kind of datafication we want, why, and how, within one institution and between institutions or systems. Without taking time for an open and democratic discussion and agreement among higher education constituents, the datafication of higher education cannot deliver on its promises.

A Way Forward

Research indicates that datafication imagined and imposed onto individual staff and students by governments, EdTech companies, or university leaders will not deliver positive outcomes. Instead, the following may bring better results. First, it is important that the business models of EdTech firms be perceived as legitimate, so that university constituents do not feel exploited. Second, there is a high appreciation of datafication to support administrative processes and efficiencies, such as saving costs for software or publishing licensing based on usage or reading trends. Promotion of data outputs or datafied products must be honest, valid, and evidenced, as well as allow variety and flexibility of use, and individual agency in usage. There must be a clear and agreed upon purpose for data outputs entering higher education systems at any level and scale. Experimentation with datafication seems useful, but it should allow rolling back if data outputs do not bring value to users. EdTech companies should work with universities respectfully and support university aims, cultures, and communities. Finally, taking time and slowing down to innovate responsibly and test products, as well as investing enough resources in good datafication, is paramount.

Janja Komljenovic is senior lecturer at the University of Edinburgh, United Kingdom. E-mail: [email protected].

The article is based on a 2024 report from the Centre for Global Higher Education, “Edtech in Higher Education: Empirical Findings from the Project ‘Universities and Unicorns: Building Digital Assets in the Higher Education Industry’”.