key: cord-0708667-34ywblps
authors: Smite, Darja; Tkalich, Anastasiia; Moe, Nils Brede; Papatheocharous, Efi; Klotins, Eriks; Buvik, Marte Pettersen
title: Changes in perceived productivity of software engineers during COVID-19 pandemic: The voice of evidence()
date: 2021-12-27
journal: J Syst Softw
DOI: 10.1016/j.jss.2021.111197
sha: 8fa723932ba79b78373f6185f34f7f8488ceee38
doc_id: 708667
cord_uid: 34ywblps

BACKGROUND: The COVID-19 pandemic triggered a natural experiment of an unprecedented scale as companies closed their offices and sent employees to work from home. Many managers were concerned that their engineers would not be able to work effectively from home, or lack the motivation to do so, and that they would lose control and not even notice when things go wrong. As many companies announced their post-COVID permanent remote-work or hybrid home/office policies, the question of what can be expected from software engineers who work from home becomes more and more relevant. AIMS: To understand the nature of home telework we analyze the evidence of perceived changes in productivity comparing office work before the pandemic with the work from home during the pandemic from thirteen empirical surveys of practitioners. METHOD: We analyzed data from six corporate surveys conducted in four Scandinavian companies combined with the results of seven published surveys studying the perceived changes in productivity in industrial settings. In addition, we sought explanations for the variation in perceived productivity among the engineers from the studied companies through the qualitative analysis of open-ended questions and interviews. RESULTS: : Combined results of 7686 data points suggest that though on average perceived productivity has not changed significantly, there are developers who report being more productive, and developers being less productive when working from home. Positively affected individuals in some surveys form large groups of respondents (up to 50%) and mention benefiting from a better organization of work, increased flexibility and focus. Yet, there are equally large groups of negatively affected respondents (up to 51%) who complain about the challenges related to remote teamwork and collaboration, as well as emotional issues, distractions and poor home office environment and equipment. Finally, positive trends are found in longitudinal surveys, i.e., developers’ productivity in the later months of the pandemic show better results than those in the earlier months. CONCLUSIONS: We conclude that behind the average “no change” lays a large variation of experiences, which means that the work from home might not be for everyone. Yet, a longitudinal analysis of the surveys is encouraging, as it shows that the more pessimistic results might be influenced by the initial experiences of an unprecedented crisis. At the end, we put forward the lessons learned during the pandemic that can inspire the new post-pandemic work policies.

As the novel coronavirus SARS-CoV-2 started to spread globally in late 2019 and early 2020, many technology companies closed their offices and sent employees to work from home (WFH), marking the turn of the history in the magnitude of experience and perception of telework. While some warn that quarantine work is not normal working from home due to the experiences of an unprecedented crisis [1] , many agree that this event will lead to significant changes in the post-pandemic workplace and is an opportunity to derive the important lessons learned regarding development productivity [2] [3] [4] [5] .

One of the first rigorous studies quantifying the changes in developer activity brought by the transition into the work-fromhome mode is made available by the GitHub research team [6] . The analysis of all GitHub projects (open source and corporate) concludes that the developer activity in terms of the number of pushes, pull requests, code reviews and commented issues remained similar or slightly increased compared to the prepandemic year. This finding is consonant with numerous other studies that conclude that software companies have nothing to worry about since working from home is per se not a significant challenge for software engineers [5, 7, 8] . In fact, earlier studies included in the state-of-the-art of telework by Fløvik et al. [35] suggest that most related studies report higher productivity and somewhat higher job satisfaction for those working from home in comparison with those working from the office [35] .

At the same time, there is little knowledge about the long-term effects of WFH, because studies related to pre-pandemic telework included only part-time voluntary teleworkers. However, some researchers focusing on the experiences from the first months of the pandemic already warn about the possible burnout caused by an increased activity during the non-core hours, such as early mornings, late evenings and weekends [4, 6] , which is also known from earlier studies of telework [35] . If developers put more time into achieving the same result, this means that their productivity has decreased. Yet, artefact-based analyses fall short to provide convincing evidence or a valid explanation, since it is difficult to know whether registered activity reflects the true work activity, i.e., whether the time between the first and the last commit or email sent does not contain longer breaks or periods of inactivity.

To complement the quantitative studies referred to above and the studies focusing solely on a short WFH time period, the current study is set out to summarize the evidence from perception-based studies of developer productivity over one year's time. This research is, thus, driven to address the following research question:

How did the perceived productivity change when software engineers worked from home during the COVID-19 pandemic?

Furthermore, as many companies announce their post-COVID permanent remote-work or hybrid home/office policies, we are motivated to better understand what hinders and enables productivity of software engineers when working from home. Thus, we pose the second research question:

The remainder of the paper is organized as follows. In the next section we outline the background related to the productivity measurement in software engineering, as well as refer to the studies of pandemic productivity. In Section 3, we describe the thirteen surveys of the changes of perceived productivity during the COVID-19 pandemic included in our study. Section 4 is dedicated to the results, which are discussed in Section 5. Finally, Section 6 concludes our paper with a short summary of the key findings.

Researchers and practitioners have been interested in productivity of software engineers for many decades. Yet, the questions of how to measure or even define developer productivity remain elusive [9] . Studying productivity in software engineering is inseparable from the discussion about the degree to which the productivity metrics reflect the actual productivity, in other words, validity of productivity as a construct [10] . Productivity generally refers to the rate of produced output per unit of time [11, 12] . Examples of metrics typically used for monitoring productivity in software engineering include a number of tasks per month [13] , deployment 1 Available online at https://www.hcp.med.harvard.edu/hpq/info.php frequency [9] , lines of code per person-month [14] , time to resolve a modification request [15] , a number of pull requests or a number of builds per engineer per hour [16] , a number of bugs found and fixed, a number of emails sent [11] , and function point productivity [17] . However, there is no metric that has not been criticized. For example, measuring productivity with lines of code can be a matter of preference among different developers [1, 10] and can depend on the choice of the programming language [17] . Similarly, function point productivity has been criticized because not all development efforts are counted as function points [17] . Such considerations discourage many researchers from studying productivity [18] , while others look at developers' productivity as a complex and nuanced phenomenon [9] , affected by technical aspects (e.g., tools), as much as social aspects (e.g., team organization and work environment) [12] . Thus, it is not surprising that subjective measures of productivity become popular instruments in the productivity research.

Perceived productivity can be viewed as an alternative self-reported indicator which allows to evaluate how productive developers feel. A typical method is asking developers to self-assess productivity on a Likert scale from low (e.g., very unsatisfied with productivity) to high (very satisfied) [11] . Just as objective metrics, self-reported productivity represents a threat for the construct validity.

Belonging to the toolbox of survey research, assessment of selfreported productivity requires rigorous design and validation of the questionnaires [19] . A more advanced and well-established tool for measuring perceived productivity (although not specific to software engineers as a profession) is the WHO's Health and Work Performance Questionnaire (HPQ) 1 .

By the time we write this paper, a number of research studies have reported the experiences with pandemic programming, including survey studies that report practitioners' perceived changes in productivity when comparing the office times with the WFH period [1, 2, 7, 20] (see our detailed description in Section 3.2 and a summary of the papers in Table 2 ). Additionally, we found three surveys of perceived productivity [4, 8, 21] that are related but do not match our research questions for different reasons, which we summarize in the following.

The first study by Miller et al. [4] reports a continuation of the work conducted by Ford et al. [3] which is a set of surveys conducted in Microsoft. The study was excluded from our analysis because the authors focused on team productivity, while we are interested in individual productivity. The study reports results from the second survey included in [3] combined with an additional teamproductivity survey, ran in July 2020. The new analysis concludes that 20% of respondents report their team productivity being positively affected by WFH, 56% remaining the same, and 23% being negatively affected, followed by a rich insight into various aspects of communication and collaboration that explain the results.

Changes in Perceived Productivity During COVID-19 Pandemic JSS, In Practice Silveira et al. [21] surveyed 279 professionals from 32 countries and concluded that the overall productivity and task completion time did not decrease during the pandemic period compared to the usual time. We do not include this study in our analysis, because the authors only examine the negative changes or the absence of such, which made it impossible to assess whether respondents have been affected positively (which is important for our analysis).

Finally, Bezerra et al. [8] assessed the state of practice of WFH in Brazilian software companies in May 2020 on a scale of unsatisfactory, regular, good and excellent, i.e., the current productivity during the pandemic and not on the changes. Hence, this study was excluded from our analysis as well. The authors found that the majority reported good or excellent productivity and that productivity was influenced by external interruptions, adaptation and emotional well-being.

In this paper, we offer the results of thirteen surveys reporting the changes in perceived productivity in WFH when compared to the pre-pandemic times, and the key reasons behind these changes. Hereby, we first describe data from six surveys that we collected (secondary data) from four software-intensive companies (see a summary in Table 1 ), and then seven related surveys from previous studies which we found in the literature with similar measures of perceived productivity (see a summary in Table 2 ). We conclude the section with the data analysis approach.

The cases included in this paper are GlobCo, Norbank, SavingsBank, and NorITpseudonyms of four companies, the identities of which are preserved for confidentiality reasons. In all cases, we relied on company-internal surveys that were designed and ran independently from our study, which qualifies our actions as secondary analysis of the secondary data [22] . In three of four cases we did not have access to the raw survey data. For this reason and due to confidentiality, we are unable to share the data openly.

Northern Europe. GlobCo develops a wide range of softwareintensive products and solutions, including generic software products offered to an open market and complex compound systems with customized versions. In this study, we examine one geographic site of GlobCo in Sweden. GlobCo was very early to start preparations for the pandemic, as the company limited the business travel in early 2020 and closed their sites in March. All employees were allowed to borrow office equipment and directed to work from home. During the summer 2020 and episodically throughout the pandemic, employees were allowed to visit the offices for important meetings, strictly limiting the number of simultaneous visits and the number of people in the meetings rooms and offices. However, in practice, many chose to remain working from home full time. In November 2020, the company announced the program for reimbursing home office equipment and furniture, with the intention to improve the WFH conditions.

We analyzed the results from an internal WFH survey of employees in one department at GlobCo. The survey was run in March 2021, and had 458 respondents, representing 75% of employees in that location (which is above the acceptable level (i.e., 60%) [22] ). 94% of respondents have been working from home full time. In the survey, respondents were asked to rate their perceived productivity on a 3-point Likert scale (less than normal, as normal, more than normal). 

Norway, which operates in the Nordic markets and offers pension, savings, insurance and banking products to both the private and the business market. While the company employs more than 2,000 people in total, this study includes software developers working in software teams from different parts of the company. Following the lockdown in March 2020, a vast majority of Norbank employees moved from working in an open office and applying agile practices and ceremonies (for over 10 years) to working from home. Those with business-critical tasks could use the office, and during the fall those not having a good home office environment could even send a request to return to the office. Further, team members could borrow whatever technical equipment they needed from the office, and each person received 500€ for buying equipment. Finally, management arranged frequent meetings to make sure everyone was informed regarding the current situation.

We analyzed the results of an all-company internal WFH survey run in June 2020. After removing non-developers, 87 responses were included in this study (80% response rate among developers, which is well above the acceptable level [22] ). In the survey, respondents were asked to rate how their productivity was affected by the COVID-19 and the WFH on a 5-point Likert scale ranging from significantly decreased (1) to significantly increased (5) .

Qualitative data: In the survey, respondents were also asked to describe the main challenges and benefits associated with WFH in a free-text form. We received 60 responses, which were used for qualitative analysis. Further, the third and the sixth author presented the results back to the company to capture their opinions and understanding of the results. 

In SavingsBank, we based our findings on the results from two waves of an internal survey of employees with 268 respondents (response rate 67%, which is acceptable [22] ) in the first wave (completed in May 2020) and 225 respondents (response rate 61%, which is acceptable [22] ) in the second wave (April 2021). In both surveys, respondents were asked to quantify their productivity in comparison with the times when they were working in the office by judging their output: "I have produced/delivered…" with the response options on the 5-point Likert scale (much more from home than from the office, more from home than from the office, equally much from home as from the office, slightly less from home as from the office, significantly less from home than from the office). The survey was designed and collected by the SavingsBank and analyzed by the three first authors.

Qualitative data: To complement the survey findings and find explanations for the results, the second and the third authors analyzed the free text responses (150 comments in the first wave and 151 comments in the second wave). We also interviewed 7 company representatives. The interviews were transcribed and used in the qualitative analysis. Finally, we presented the results back to the company to elicit their opinions.

NorIT develops products consisting of hardware and software components. The company combines traditional project management with agile development methods. Before the pandemic, the teams were either situated in an open office, in own offices or shared the office with one or two teammates. Notably, 8% worked regularly from home already prior to the pandemic due to a long commute time (~2-3 hours/day). In March 2020, 91% went into the work from home mode as the company sites closed. Many of those that needed access to a restricted network had to work from the office. Everyone was offered to take the equipment they needed from the office. During the course of the pandemic, employees were allowed back to the NorIT office when the spread of the virus decreased. This allowed the company to start testing how to organize a hybrid workplace for the future.

In NorIT, we based our findings on two surveys in two different departments conducted at two different points of time.

The first survey was ran in May 2020 and gathered responses from 395 respondents working on one specific product, among which 131 were working on software development (75% response rate among software developers, which is above the acceptable level [22] ). The second survey was performed in January 2021 and gathered 45 responses from developers working on another product, with 18 being involved in the software development (90% response rate among the software developers, which is well above the acceptable level [22] ). Most developers worked from home all days during the pandemic. Perceived productivity was assessed through an item used in both surveys: "When I work from home I am:…" with the response options: "more productive than at the office, have the same productivity home and at the office, less productive than at the office."

Qualitative data: As in other corporate surveys from Norwegian cases, respondents from NorIT were asked to describe the main challenges and benefits associated with WFH in a free-text form.

We received 71 comments in the first NorIT survey, and 13 comments in the second survey, which were all used in the qualitative analysis.

To answer our research questions, we supplemented the perceived productivity data from the empirical cases with the data and/or results available from earlier related studies that compare the prepandemic office productivity of software engineers with that during the pandemic [1, 2, 7, 20] . In the following, we describe the survey instruments employed in each of the included cases. [1] performed an extensive study of the pandemic impact on programming, including productivity in the early months of WFH. The authors also assessed how perceived productivity is affected by disaster preparedness, change in well-being and home office ergonomics. They conclude that perceived productivity has declined (admitting a marginal effect size) as a result of negatively affected wellbeing and that organizations need to accept that expecting normal productivity under the crisis circumstances is unrealistic.

gathering a large number of responses from 2078 professionals from 53 countries (a subset of those who responded to the perceived productivity question). The analysis included respondents who switched from working in the office to working from home. With respect to the profession and affiliation, the respondents are very diverse, representing organizations ranging from 0-9 employees to more than 100,000, with 80% being software developers or equivalent and 20% being other kinds of software professionals (e.g., project manager, quality assurance analyst). To elicit perceived productivity, the authors applied the WHO's Health and Work Performance Questionnaire (HPQ) and report average total scores for both before and after the transition to WFH. The participants of a single survey were asked to evaluate their own productivity for the period before the switch (HPQB) and the period since the switch (HPQS), where each period was assessed through 9 items. The total perceived productivity scores were calculated by deriving a total of several items from the HPQ separately for before and since the pandemic. The items were standardized, and two items were omitted from the traditional HPQ to increase the convergent and discriminant validity of the factor analysis. Furthermore, the authors assessed disaster preparedness using Yong et al.'s disaster preparedness scale [33] , emotional well-being using WHO´s five-item wellbeing index and ergonomics through a self-created six-item 6-point Likert scale concerning distractions, noise, lighting, temperature, chair comfort and overall ergonomics. [2] conducted a two-wave study with the first wave including Microsoft engineers from one location in the US, and the second -Microsoft engineers from all locations in the US two surveys in Microsoft. Both surveys indicated that the productivity increased among some participants and stayed the same or Changes in Perceived Productivity During COVID-19 Pandemic JSS, In Practice decreased among the others, which led the authors to conclude that the productivity during the COVID-19 pandemic was dichotomous.

Surveys 8-9: Data from the first survey was collected among 1369 developers and program managers in March 2020, whereas the second wave was run in April-May 2020 among 2265 developers, program managers and data scientists. In both surveys the respondents were asked, "Compared to working from home, how has your productivity changed?" and gathered responses on a 4point Likert scale: Less productive, About the same, More productive, and Significantly more productive. The results were summarized by waves and compared across the waves. Additionally, Survey 9 elicited experiences with the challenges and benefits when working from home, as well as their impact ("What work-related challenges have you experienced working from home and how impactful are these challenges?", "What work-related challenges have you experienced working from home and how impactful are these challenges?")

Qualitative data. Survey 8 collected qualitative insights to better understand the change in perceived productivity. The participants were asked to follow-up on their response on perceived productivity: "Please share details about your answer to the previous question on how your productivity has changed". The answers were used to design a list of challenges/benefits that the participants could choose from when answering the matching questions in Survey 9 (see above). [7] performed a two-wave longitudinal study with a diverse group of professionals, diving into the impact of over 50 psychological, social, situational, and physiological factors and their ability to predict the variance in well-being and productivity.

The study concludes with a few associations between the studied factors and perceived productivity and puts forward actionable recommendations. In particular, the authors emphasize boredom as an important factor associated with productivity, which was measured using the 8-item from the Boredom Proneness Scale [34] , including items like "It is easy for me to concentrate on my activities" and "Many things I have to do are repetitive and monotonous" assessed on a 4-point Likert scale. Another important factor found by Russo et al. was distraction at home, which was measured with two items: "I am often distracted from my work (e.g., noisy neighbors, children who need my attention)" and "I am able to focus on my work for longer time periods" (recoded) with responses given on a 5-point scale. Regarding the changes in productivity, the authors conclude that WFH does not per se present a challenge for either organizations or developers. They asked the respondents to rate their productivity by comparing it to their own normal. The change in productivity (ΔProductivity) was operationalized through three variables: P1 ("How many hours have you been working approximately in the past week?", P2 ("How many hours were you expecting to work over the past week assuming there would be no global pandemic and lockdown?" and P3 "If you rate your productivity (i.e., outcome) per hour, has it been more or less over the past week compared to a normal week?". The total ΔProductivity was calculated for each wave using the formula (P1/P2) × ((P3 + 100)/100), where the values between 0 and .99 would reflect that people were less productive than normal, and values above 1 would indicate that they were more productive than normal. [20] gathered data from two online surveys of Brazilian professionals. The authors found that perceived productivity in WFH when comparing with the office times has increased. Another important finding made regarding the changes in perceived productivity during the pandemic was that the number of positively affected respondents grew from 40% in the first wave to 60% in the second wave.

The two surveys were run in April and May 2020 and gathered responses from a diverse set of professionals, 75% being software developers or equivalent. Most of respondents work in companies with more than 100 employees (63%), others work in companies with 10-50 (18%), 51-100 employees (9%) and less than 10 employees (9%). All respondents work 100% remotely, although they started remote working at different points of time.

Perceived productivity data was gathered by inquiring, "Do you consider that your productivity in this period: Continued the same, Increased, and Decreased". Interestingly, the order of the response options in this study is different from the other studies that offered the response options from negative to neutral and then to positive options. Furthermore, the authors have made stratified analysis on different groups of respondents to understand if any of the groups is particularly affected.

To answer our first research question (RQ1: How did the perceived productivity change when software engineers worked from home during the COVID-19 pandemic?), we summarized the results of thirteen surveys after standardizing the scales across all surveys (see Figure 1 ). In the previous section, we described the different scales used in each study. Most of the studies used a Likert scale of three, four or five points. Two studies [1, 7] used continuous scales, which we have standardized according to the original operationalization. Since the actual distributions of the perceived productivity responses are not disclosed in either of the papers, we replicated the calculations of the productivity scores in these studies using the publicly available replication packages to estimate the number of participants with scores indicating decreased, same or increased productivity after the switch to WFH (frequency distributions). See the mapping of the variables in Table 3 . Proceeding in this way allowed us to transform the data in scales comparable to the Likert scales used by the majority of the studies. The frequency distributions from the study by Ralph et al. [1] were estimated by comparing the total scores before the switch (coded as HPQB in the survey) with those since the switch (coded as HPQS in the survey). Similarly, we estimated the frequency distributions from Russo et al. [7] by comparing the total Δ Productivity scores to 1 (according to the authors' guidelines). The calculations lead to reduction of the data points when division with zero values JSS, In Practice Smite et al.

occurred in the process. In Table 3 , we provide a summary of the translation of the scales.

To address our second research question (RQ2: What factors predict changes in perceived productivity?), we synthesized the findings from multiple data sources. First, we extracted the reasons for lowered and increased productivity as reported in the related studies, included in our analysis [1, 2, 7, 20] . Secondly, we coded the free-text responses in the available case surveys that explained the changes in perceived productivity, which we labeled positive and negative changes with respect to the WFH situation compared to the pre-COVID one. This included the analysis of 445 free-text comments from NorBank, SavingsBank 1 and 2 and NorIT 1 and 2. Similar coding of the reasons for lowered and increased productivity was performed for the GlobCo and SavingsBank cases using the transcripts of interviews (eleven interviews in GlobCo and seven interviews in SavingsBank). The following codes exemplifies our coding process: "More difficult to take the quick informal conversations with colleagues -especially across teams and departments" was coded as "Greater difficulty communicating with peers"; "Better concentrationno disturbances" coded as "Fewer interruptions from colleagues"; "Less stress around delivery and picking up my kids in the kindergarten" was coded as "Better work-life-balance" and "Less stress". Finally, we grouped the emerging codes into four higher level categories, including Emotional issues factors, Organization of work and time, Home office environment and equipment, Team and work processes and collaboration (see Tables 4 and 5 for the summary of all resulting codes and categories). We used the coded excerpts from the interviews and survey responses as quotes to illustrate the factors described in Section 4.2.

Our results, namely the changes in perceived productivity and the list of explaining factors, are subjects to validity threats. In the following, we describe the factors that impact the reliability and generalizability of our results.

Our study is descriptive in nature and the purpose of our summary is to illustrate the variability in the results of individual surveys ( Figure 1 ). The studies included in our analysis are performed at different points in times using different survey instruments based on samples drawn from different populations and can therefore not be used for direct comparison. The reliability of the results of each individual survey, in their turn, highly depends on the survey instrument. The fact that we used data collected through both Likert and non-Likert scale instruments, is therefore one of the validity threats in our study.

Belonging to the toolbox of survey research, assessment of selfreported productivity requires rigorous design and validation of the questionnaires (Molléri et al., 2020) . We thus start by bringing forward the differences in operationalizing the survey instruments to study the perceived productivity of software engineers as a potential validity threat in our study. All surveys assessing perceived productivity with a single Likert-scale item, as in [2] and [20] and all our own cases report the trend that productivity did not change or improved. Yet, approaching perceived productivity measurement with the Likert-scale approach is potentially less reliable comparing to applying more complex measures such as the ones in [1] and [7] . The results from Ralph et al. [1] and Russo et al. [7] are evidently most distinct (see Figure 1 ) with half of the respondents reporting being negatively impacted. Ralph et al. [1] used the modified WHO's Health at Work Performance Questionnaire that computes a perceived productivity score based on a series of questions. Interestingly, the use of the traditional version of the WHO instrument (without any further changes) has been regarded as unfit for the purpose by Russo et al. [7] who state that knowing other workers productivity (which is used for comparison with own productivity) might be problematic when working remotely. In their turn, Russo et al. [7] have measured productivity relative to the expected productivity by contrasting productivity in the past week with the participant's expected productivity level before the lockdown with response options on a bipolar slider measure. The authors reported the test-retest reliability of their instrument to be good. Although the results are not much different from those in the studies using the 3-, 4-or 5point Likert scales, the mentioned studies based on a more advanced measurement report a much lower number of respondents reporting the same perception of productivity (i.e., no changes) than found in the studies using a Likert-scale. This is understandable, because the distance between "the same" and "decreased" or "increased" on a Likert-scale might be perceived larger than the distance between 1 (the same) and 1,05 or 0.95.

The validity of the self-assessed productivity by responding to a Likert-scale item has been questioned before also by pointing out that the respondents, when asked to rate their productivity from very unsatisfied to very satisfied, most participants will typically report good (satisfied) or neutral productivity [11] . Likert-scale surveys included in our analysis might suffer from this bias, since most of them demonstrate bias towards the neutral and positive responses, except for Ford et al. [2] who found that a larger group of 38% reported being negatively affected in their second survey. Tables 4 and 5 list the factors predicting changes in perceived productivity based on the qualitative analysis of the interviews, survey comments from the industrial cases and the articles reporting the related surveys. The qualitative analysis results could be subject to researcher bias. To mitigate this threat, several researchers have been involved in each case. First, two researchers were involved in each interview. Second, two researchers were involved in the data analysisone performed the coding, and the other validated the emerging list of factors from the case. Finally, we do not claim that the resulting list of factors predicting changes in perceived productivity is exhaustive. While the factor categories identified in the cases of GlobCo, NorBank, SavingsBank and NorIT have been saturated and confirmed by all four cases (see the factors unique to our cases in Tables 4 and 5 Tables 4 and 5) can help predicting the changes in perceived productivity. However, our findings are biased by the time when the surveys were executed, by the locations of the survey respondents and by the pandemic situations/conditions in the given time and location. With respect to time, our findings are likely to be biased towards the experiences from the first three months of the pandemic. With respect to geographic locations, the surveys in our study include both single-company single-region surveys and surveys from diverse multi-country samples (41 and 53 country representatives). This is likely to increase the generalizability of our results. However, to predict the changes in perceived productivity in a particular geographic location, one needs to consider the particular lockdown measures and that people from different cultural environments might differently react to the crisis, to sharing of household and parenting duties, to connecting to the internet from home or to working in solitude. When comparing the trends in perceived productivity, we notice the datasets that we collected from the Scandinavian companies (from Norway and Sweden) appear to report the most positive experiences comparing to studies performed in the USA and Brazil. This could be attributed to the organizational culture of trust, autonomy and selfmanagement routed in extensive experiences with agile ways of working [5] , as opposed to more traditional management of control in USA and Brazil, or the relatively relaxed lockdown measures and early reopening of the society. However, more research is needed to determine the true reasons. Figure 1 summarizes the evidence from the thirteen surveys on the the state of the changes in perceived productivity of software engineers while working from home during the COVID-19 pandemic. The data behind the summary comes from the total of 7686 data points from the six corporate surveys ran in our case companies (1187 respondents in total, min 18 in NorIT 2 and max 458 in GlobCo) and the seven related survey studies [1, 2, 7, 20] (6499 respondents in total, min 182 in the second wave of [7] , and max 2265 in the second wave of [2] ). The main findings based on the analysis of the data are twofold: all surveys are dichotomous and latter surveys report more optimistic results than the surveys conducted in the early months of the pandemic.

Dichotomous trends: Among the 13 surveys we find those reporting half of respondents negatively affected [1, 7] and surveys that report predominantly no change or positive affect of WFH on perceived productivity ( [2, 20] and all our company surveys). 

Perceived productivity scale: 

No changes in productivity: The results show that relatively many employees in ten out of 13 datasets reported no changes (38-52%).

Negative changes in productivity: The groups of negatively affected developers range from 11% in the second wave of NorIT to 51% in the study by Ralph et al. [1] . The studies reporting the most pessimistic results include the surveys by Russo et al. [7] and Ralph et al. [1] , whose data indicates that around half of the respondents experience a decline in perceived productivity (50% in the first wave and 46% in the second wave in [7] and 51% in [1] ). The amount of negatively affected respondents in all six corporate case surveys was not high. Roughly every fifth respondent in SavingsBank 1 (21%), NorBank (19%) and GlobCo (17%) as well as roughly every tenth employee in NorIT 1 (12%), NorIT 2 (11%) and SavingsBank 2 (12%) reported that their productivity was lower than normal.

Positive changes in productivity: Considerably many employees have reported an increase in productivity (ranging between 24% in the first wave of Oliveira et al. [20] to 50% in the second wave of SavingsBank). Further, several studies have performed two-wave analysis of the changes in perceived productivity [2, 7, 20] , and our two cases -SavingsBank and NorIT. In all longitudinal studies, the second wave responses were more positive than those in the first wave. The positive changes over the course of the pandemic are also consonant with our findings from studying remote pair programming during WFH, in which we found that pairing from almost an extinct practice in the first months of the work from home, became increasingly practiced in early 2021 [24] .

Evidently, the corporate case company surveys emerge as a block of studies with a more optimistic results, while the variation in responses across the published surveys is larger and they do not report the same positive trend. But the results might be also attributed to the time of inquiry.

To see the perceived productivity trends over time, we sorted the surveys in Figure 1 chronologically. The time when each survey was executed is marked on the timeline in the right side of the figure. Evidently, time seems to be an important predictor, as the surveys at the bottom of the page (later months of the pandemic) tend to be more positive than those on the top (early months of the pandemic). We also find that subsequent runs of the surveys in all cases report improved perceptions of productivity, which is perhaps the most interesting finding. The positive changes over time were also emphasized by our interviewees and in the freetext survey responses. These changes varied from those relating to the changing lockdown measures to those relating to the adjustments that people and organizations have made over time. For example, in Norway schools reopened after the strict spring lockdown in 2020 and the experiences in the first few months were drastically different from the later ones. Significant improvements have been also made by many in the home office equipment. As someone from GlobCo explained the effect of the reimbursement bonus introduced in the beginning of October 2020: "Earlier I was sitting in the kitchen and, you know, a kitchen table is not the best work area for sure. That has improved." Similarly, the distributed collaboration tools and familiarity with those were said to considerably improve few months into the working from home mode. In fact, in some companies like SavingsBank approval of the new tools took a while. As an interviewee from SavingsBank explained: "We have increased our competence when it comes to digital collaboration, getting new tools also improved my productivity." Participants from NorBank also mentioned the exceptional change in the meeting frequency in the early months of the pandemic, which has returned to the norm after some time, as someone explained: "After a couple of weeks the meeting frequency increased significantly, but it calmed down after a while." Many interviewees also commented on the way teams have experimented and adjusted their ways of working, someone even referred to this painful period as a "rollercoaster." Many interviewees from GlobCo and SavingsBank mentioned improving their ways of working, restructuring the daily routines and ceremonies, and improving the meeting culture both in terms of communication and preparations. As two representatives from GlobCo explained: "We set up some action points to improve what people did not like about the current situation" and "That's all been an ongoing conversation ever since March -[…] how we collaborate to make it better. I would say we talk about that a lot. All the time." All these adjustments might explain why perceptions of productivity in the later months tends to be more optimistic than in the early months of the pandemic, especially when comparing the number of respondents reporting negative changes. Table 4 and Table 5 summarize the factors that predicted the changes in perceived productivity. The analysis is based on the associations reported in related studies and our own qualitative analysis of the case survey free-text responses that explain the changes in perceived productivity.

The authors of the studies showing the most pessimistic productivity change do not report the perceived productivity but study the variables that explain the variance in well-being and productivity. Russo et al. suggest that stress predicted negative well-being, while boredom and distractions predicted negative productivity [7] , while Ralph et al. found that the pandemic has had a negative effect on developers' wellbeing and productivity due to disaster preparedness, fear related to the pandemic and home office ergonomics [1] . Another interesting aspect of these studies is that they both operationalize the perceived productivity measure differently than the other studies. The values in both survey instruments seem to be biased towards either decline or increase and are less sensitive to those whose productivity is about the same. The explanations for why the productivity decreased in the other studies include emotional issues related to COVID-19 [20] , difficulties to focus [20] , interruptions from family members [2, 20] , connectivity problems [2, 20] and poor workspace equipment [2, 20] , not matching working hours in the team [20] , unavailability of all resources [20] and a greater difficulty communicating with people [2, 20] .

Smite et al.

Similarly, respondents from the case companies shared their thoughts about the feeling of isolation, boredom and fears related to the pandemic. Work from home was said to fail to provide the so needed energy, as an engineer from GlobCo explained: "It's a bit boring to be home … It's a different environment, of course, when you're in the office and some of the noises,.. it's like energyin the way people laugh in the corridor and offices next door and so on. That gives some energy, I guess, as well." Further, we learned that stimulus and motivation to work were also negatively impacted by the blurred work-life balance and the lack of positive peer pressure when working from home. As another engineer from GlobCo explained: "I'm working by myself now in silence here. So it's like, … you don't get the same stimulus as you usually would, and I think that's not ideal. I think that [positive peer pressure] helps a lot." With respect to work and time organization, we found that working from home is associated with distractions and interruptions from family members, especially in families with children on home schooling. In NorBank, participants also complained about increased workload combined with lower efficiency. A developer commented: "Everything is more difficult. Must work more, but complete less, but in practice same requirements, as before." Increased number of meetings and worse understanding of what is going on in the team were also reported by those who complained about the decreased productivity. As someone from SavingsBank reported -"With so many team meetings, you do not have time to produce anything." For others, it was not about the number of meetings as much as the exhaustion from running meetings virtually. Many respondents from Norway and interviewees from GlobCo revealed that employees have been challenged by technical problems, such as connectivity and bandwidth, especially in the beginning of the WFH. Negatively affected respondents from SavingsBank and NorIT further complained about a lack of information and awareness of what is going on, and a lack of socialization with colleagues. Interestingly, as much as engineers enjoyed the focused time and the lack of interruptions from the peers, we received numerous complaints about the negative impact of not overhearing the office conversations. Those suffering the most were teams that were used to highly collaborative work style. These teams required considerable time for adjustment. As a manager from GlobCo explained, "I think that now the teams have adapted to this way of working. OK… now I feel that it's not a blocker for them.

[…] that it works pretty good for many of the teams." Further, running sessions with open content, such as brainstorming sessions and problem-solving sessions, was mentioned to be more challenging and to require more time due to the lack of accustomed whiteboards, possibility to spontaneously connect to the needed people, and requiring considerably more time to prepare for. Isolation from colleagues has also been reported to loosen the ties with colleagues beyond the teammates, which means that the information is not circulating in the same way as it did in the office. Finally, an emerging vulnerable group in our study is recently onboarded individuals. As an engineer from SavingsBank explains that he is significantly less productive: "Because I cannot observe what others do and have not learned any tasks I can do independently," and another reports, "[I am] brand new to the job and need a lot of follow-ups." Similarly, a recent hire at GlobCo explains that getting help remotely requires more time: "There's just a slight barrier compared to just asking a question out in the room. Then you have to maybe ping several people... and it takes longer time to get an answer as well, compared to just when you're sitting in the same room with someone." Interesting in our study was that even people who reported increased productivity were still dissatisfied with other aspects of working from home (e.g., having distractions from kids at home or a lack of contact with colleagues). This highlights that high productivity in the home-office does not always coincide with the satisfaction with WFH. 

In this paper, we have presented our findings from summarizing six corporate surveys in four case companies and seven surveys reported by other authors diving into the changes of perceived productivity when moving from the office work to work from home during the COVID-19 pandemic. Combining the results of 7686 data points we learned that there are developers who report being more productive, developers being less productive and those maintaining their usual productivity when working from home. The decreased productivity when moving from collocated to fully distributed setup is not that surprising. Distributed work is infamous for being considerably less productive than collocated work due to impaired teamwork [26] . What is in fact surprising, is that some WFH developers perceive being more productive.

Understanding the latter is therefore important for the future of telework. Furthermore, we evidenced positive trends in the later months of the pandemic with regards to the fewer complains of decreased productivity, supported by the reports of numerous adjustments in the way remote work is carried out from home. In this section, we discuss the key implications of our findings for the future and important research topics as the companies transit to the post-COVID permanent remote-work or hybrid home/office policies. We then discuss the validity of our findings and try to understand why some surveys convey a more pessimistic view of the WFH while others convey a more optimistic view.

In Tables 4 and 5 , we summarized the factors that predicted or explained the changes in perceived productivity in the included surveys as productivity barriers and drivers. These include factors grouped into the categories of Emotional issues, Organization of work and time, Home office environment and equipment, Teamwork and work processes, and Commuting to work.

The first evident reflection is that the number of barriers (these are 22, see Table 4 ) is twice as high as the number of drivers (these are 11, see Table 5 ). Yet, combining these findings with the overall perceived productivity changes from the surveys suggests that the impact of the drivers might be much more profound that the listed challenges. Alternatively, items on both sides might have a cancelling effect, in case they are experienced by the same people.

Deeper analysis shows that productivity barriers mainly concern the emotional issues and team and work processes, while the drivers relate to the organization of work from home. The most common barriers included emotional issues and pandemic-related fears, distractions and interruptions, poor workplace equipment and ergonomics, and teamwork-related issues such as socialization decrease, poor awareness of what is going on, greater difficulty to communicate with peers, grow a contact network or run brainstorming and problem-solving sessions. Among the most common benefits of working from home were better work-life balance, fewer interruptions from colleagues, better focus time, more time to complete work and increased flexibility in planning the work hours, as well as more efficient meetings and less time spent on commuting and communicating with the management.

Interestingly, similarly to the perceived productivity results that demonstrate polarity of experiences, some of the issues included in the barriers and drivers are contradictory. This means that what one JSS, In Practice Smite et al.

considers a benefit, for another might be a barrier. For example, some complained about the difficulties to focus [7, 20] , while others reported a better focus time when working from home [2, GlobCo, NorBank, SavingsBank, NorIT] . This highlights once again the fact that the work from home mode is not for everybody.

The next important question relates to whether the identified factors can be attributed to the actual working from home or to pandemic circumstances. Our analysis shows that a few barriers can be attributed to the pandemic -Disaster preparedness and Emotional issues, pandemic-related fears (see Table 4 ), while most factors are related to working from home. Evidently, none of the drivers of perceived productivity are attributed to the pandemic. This means that in the likely post-pandemic scenario, in which people will continue working from home, there will be fewer barriers while drivers will remain the same.

We also see positive trends over timethere are fewer negatively impacted respondents in the later months of the pandemic. Many of the factors reported as barriers to productivity have been also in the focus of experimentation and improvement initiatives. This means that the impact of these barriers might have been more profound in the early months.

At the same time, it is important to note that the surveys included in our research are conducted within the first 13 months of working from home and the long-term effects of some of the barriers, such as the loosening ties which is compared to the melting iceberg of relationships [32] , might not have yet surfaced. Table 4 exemplifies the barriers associated with decreased teamwork and networking the true long-term effects of which shall be monitored, including a greater difficulty communicating with people or growing a contact network, poor awareness of what is going on, loosening social ties across the organization, and decreased socialization in the team.

Notably, some of the barriers and drivers that we found have been studied before. For example, earlier studies included in the stateof-the-art of telework by Fløvik et al. [35] suggest that the balance between work and leisure was found to be a challenge for those working from home, while job overall productivity and job satisfaction tended to benefit from the home environment. Our findings provide further details about the problems and benefits associated with working remotely from home, and perhaps more clearly demonstrate the polarity of experiences.

In our study, we identified several important research directions.

First, it is paramount to better understand the different groups of engineers or teams that either benefit or are hindered by the work from home. We suggest analyzing the productivity in different contexts. Our study pointed to the possible differences in the WFH experiences based on the geographical locations, but more research is needed to better understand the differences with respect to locations (e.g., larger cities with longer commute times versus smaller towns), and national and organizational cultural impacts. Other related surveys have highlighted the vulnerability of different gender groups, people with disabilities and groups with varying family conditions [1] . Other context factors such as the nature of tasks (routine work versus complex or innovative tasks), learning opportunities, levels of interdependence and coordination needs, leadership styles and many others could also contribute to the success or failure of for remote work. More research is needed to understand which of these contexts are favorable and which are unfavorable for WFH to inform the future remote work policies.

Second, we suggest exploring the circumstances that destine the same characteristics of the working from home to play in advantage or disadvantage under the work-from-home conditions. In particular, a better understanding of the work-life balance, focus and virtual meetings is needed.

Third, our results identified a number of factors affecting wellbeing and mental health of individuals working from home, which, in our opinion, deserve extra attention.

Fourth, we believe that it is crucial to continue monitoring the work from home experiences to better understand the long-term effects of remote work that might not have yet surfaced [32] .

Finally, in the light of different future work policies, it is important to dedicate research efforts into the understanding of the impact of hybrid work practices (working partially in the office and partially from home) on well-being, productivity, and teamwork.

The key implications of our findings are twofold. Overall, our results indicate that there is no reason to be concerned about the productivity of software engineers working from home. Many engineers benefit from working from home, individual engineers and teams have managed to adjust their home environments and ways of working to the requirements of remote work, and the number of negatively affected engineers has decreased over time.

Yet, the factors summarized in Table 4 highlight important issues such as the negative impact of WFH on mental health and relationships inside the team and across the organization. In the following, we put forward several recommendations emerging from our findings.

Individual employees seem to respond to home office situation very differently, depending on their personal characteristics, and family situation. For example, some report better work-life balance at the home office, while others report the opposite. Further, we know little about the favorable and unfavorable circumstances for remote work with respect to the nature of tasks (routine work / complex / innovative), levels of interdependence and coordination needs, and organizational culture. Therefore, there will likely be no one-fitsall future work strategy. Yet, it is fair to assume that while some employees will rush into the office when the pandemic is over, there will be employees willing to remain working fully or partially at home. Thus, we recommend forming future work policies with respect to the needs of different individuals and groups.

Notably, working from home as reported in our study is influenced by the disaster of the COVID-19 pandemic, while the future Changes in Perceived Productivity During COVID-19 Pandemic JSS, In Practice working from home is likely to be very different. One may therefore believe that in the absence of interruptions from home schooling children and pandemic fears we might evidence a further increase in productivity. Indeed, our findings show that engineers adopt to the new circumstances and report increased productivity 13 months into the pandemic when comparing with the results from the early months. At the same time, the true long-term effects of remote teamwork and loosening ties is yet to be understood [32] . Besides, the identified emotional and well-being issues that are associated with remote work are worrisome and might signal about the possible mental health problems or burnouts as also suggested in related studies [9] . Thus, we recommend monitoring the longterm effects of remote work, and experimenting and adjusting the work policies in response to the gained experience.

Finally, as the COVID-19 pandemic continues, we recommend companies to urgently address the needs of those employees who struggle when working from home. Our results show that employees will benefit from improved home office ergonomics and equipment, and internet connectivity. Such issues are relatively easy to address through the reimbursement programs, as the ones described in the case companies. Similarly, we can recommend improving the meeting culture. There are also barriers that are more challenging, such as distributed teamwork and collaboration. A related study shows that companies that had a strong collaborative culture coupled with rich tools supporting remote work had a relatively easy transition to working from home [5] . For other companies, we therefore recommend continuously supporting distributed collaboration processes and tools, especially because remote work is likely to stay even after the pandemic.

On the one hand, our findings confirm that software intensive companies have moved rather smoothly into the WFH mode during the COVID-19 pandemic and that on average, the organizational productivity has not significantly changed [5, 6] . On the other hand, and more importantly, our analysis of individual productivity clearly shows that there are sufficiently large groups of developers who report being more productive, and groups of developers reporting being less productive as also found in a commit-based productivity study [9] , meaning that on an individual level, things have changed. We also evidenced that things changed over time, as individual engineers and teams managed to adjust their home environments and ways of working to the requirements of remote work. And although "it depends" type of finding is not very useful in predicting the changes in perceived productivity in a random company, it is an important observation with important practical implications. First, companies shall find ways to support those developers who do not enjoy working from home and teams that could not reach the satisfactory remote operation, especially because the return to the normal operation appears to be much slower than expected and the periods of forced WFH might reappear periodically in the future. Second, it is fair to assume that the likely future workplace after the world returns to the normal operation, will include the elements of working-from-home, as developers who have been highly productive might want to continue working from home at least parttime. The companies, therefore, shall be prepared to facilitate the hybrid work environment in the future.

What we learned about pandemic programming by no means is complete or reflects the normal work-from-home experience. However, it gives hope that the positive factors and useful adjustments relating to the work-from-home are likely to be lasting beyond the pandemic would the companies decide to grant their employees with flexibility, while the issues of concern provide sufficient ground to plan future research. Specifically, we emphasize the importance of addressing the issues related to individual well-being and mental health as well as issues associated with remote teamwork and collaboration, whose direct impact might not have yet surfaced. ANASTASIIA TKALICH is is a researcher at SINTEF Digital, Norway, where she focuses on software process improvement, software teams, software product innovation, agile coaching and digital transformation. Tkalich holds a Master´s degree in work and organizational psychology from the Norwegian University of Science and Technology.

NILS BREDE MOE is a chief scientist at SINTEF im Norway. He works with software process improvement, intellectual capital, innovation, autonomous teams, agile and global software development, and digital transformation. He has led several nationally funded software engineering research projects covering organizational, sociotechnical, and global/distributed aspects. Moe received a dr.philos. in computer science from the Norwegian University of Science and Technology, and holds an adjunct position at the Blekinge Institute of Technology in Sweden. ERIKS KLOTINS is a post-doctoral researcher at the Blekinge Institute of Technology, in Sweden, working on the cost-benefit perspective on continuous software engineering (CI/CD). His work includes analyzing how to utilize continuous principles throughout the organization best to benefit from faster time-to-market, frequent customer input, and datadriven techniques to fine-tune the product to exact customers' needs and attain the organizational objectives. Klotins has extensive industry experience in developing software products in fast-paced, dynamic environments. He recently acquired a Ph.D. in the area of software engineering practices for start-ups.

MARTE PETTERSEN BUVIK is a senior research scientist at SINTEF in Norway. Her main research topics include psychosocial working environment, teamwork, leadership, industrial relations, change processes and organizational development. Buvik received a PhD in organization and management studies from the department of industrial economics and technology management at NTNU, with the topic of trust in project teams. She has published in several international journals and led nationally funded research projects related to autonomous teamwork in software development and working environment.

A tale of two cities: Software developers working from home during the COVID-19 pandemic

How does Working from Home Affect Developer Productivity?--A Case Study of Baidu During COVID-19 Pandemic

Software Development Teams Working From Home During COVID-19

From Forced Working-From-Home to Working-From-Anywhere: Two Revolutions in Telework

An analysis of developer productivity, work cadence, and collaboration in the early days of COVID-19. Octoverse spotlight

Predictors of Well-being and Productivity of Software Professionals during the COVID-19 Pandemic-A Longitudinal Study

How Human and Organizational Factors Influence Software Teams Productivity in COVID-19 Pandemic: A Brazilian Survey

The SPACE of Developer Productivity: There's more to it than you think

Construct validity in software engineering research and software metrics

Software developers' perceptions of productivity

A Review of Productivity Factors and Strategies on Software JSS

Proceedings of the Fifth International Conference on Software Engineering Advances

Developer fluency: Achieving true mastery in software projects

An analysis of trends in productivity and cost drivers over years

Socio-technical congruence: a framework for assessing the impact of technical and work dependencies on software development productivity

Helping our developers stay productive while working remotely

Measuring and predicting software productivity: A systematic map and review

Why we should not measure productivity

An empirically evaluated checklist for surveys in software engineering

Surveying the impacts of COVID-19 on the perceived productivity of Brazilian software developers

2020. A Deep Dive on the Impact of COVID-19 in Software Development

Real world research: A resource for social scientists and practitioner-researchers

Real world research

From Collaboration to Solitude and Back: Remote Pair Programming During COVID-19

Challenges and recommendations to publishing and using credible evidence in software engineering

An empirical study of speed and communication in globally distributed software development

Home Teleworking: A Study of its Pioneers

Research design

Collaborating during coronavirus: The impact of COVID-19 on the nature of work (No. w27612)

Exploring the Impact of COVID-19 Lockdown on Social Roles and Emotions while Working from Home

From Dusk till Dawn: Reflections on the Impact of COVID-19 on the Development Practices of a R&D Project

Loosening Ties: Permanently Virtual Teams and the Melting Iceberg of Relationship

Risk perception and disaster preparedness in immigrants and Canadian-born adults: Analysis of a national survey on similarities and differences

Boredom proneness--the development and correlates of a new scale

Arbeid hjemmefra, helse og arbeidsmiljø

Reasons for productivity increase: The qualitative explanations for why perceived productivity increases are similar across our sources. Many respondents claim to have less stress [20] , less time to commute [2, 20] , better focus time [2] , fewer interruptions [2, 20] , more time to complete work [2] , more efficient meetings [2] , better/more comfortable work environment [2, 20] , better work life balance [2] . Many people we interviewed were surprised to deliver good results despite the sudden shift from the office to the WFH setup. A developer from NorBank expressed: "Didn't expect to function so well at the home office. The team functioned from day one, and we delivered good results." The benefits reported in our case companies include an increased ability to focus, fewer distractions, increased flexibility to organize ones work hours, less time spent on commuting, as well as more efficient and shorter meetings mentioned by many respondents in all companies. A participant from NorIT commented: "I have more free time saving about two hours driving time to/from work." Another participant from NorIT was also satisfied with the efficiency and availability of the colleagues: "I think I work more effectively. Use less time