The Life Course: An interdisciplinary framework for broadening the scope of research on crowdwork

This paper reports outcomes of a systematic scoping review of methodological approaches and analytical lenses used in empirical research on crowdwork. Over the past decade a growing corpus of publications spanning Social Sciences and Computer Science/HCI have empirically examined the nature of work practices and tasks within crowdwork; surfaced key individual and environmental factors underpinning workers’ decisions to engage in this form of work; developed and implemented tools to improve and extend various aspects of crowdwork, such as the design and allocation of tasks and incentives or workflows within the platforms; and contributed new techniques and know-how on data collection within crowdwork, for example, how to conduct largescale surveys and experiments in behavioural psychology, economics or education drawing on crowdworker samples. Our initial reading of the crowdwork literature suggested that research had relied on a limited set of relatively narrow methodological approaches, mostly online experiments, surveys and interviews. Importantly, crowdwork research has tended to examine workers’ experiences as snapshots in time rather than studying these longitudinally or contextualising them historically, environmentally and developmentally. This piece-meal approach has given the research community initial descriptions and interpretations of crowdwork practices and provided an important starting point in a nascent field of study. However, the depth of research in the various areas, and the missing pieces, have yet to be systematically scoped out. Therefore, this paper systematically reviews the analytical-methodological approaches used in crowdwork research identifying gaps in these approaches. We argue that to take crowdwork research to the next level it is essential to examine crowdwork practices within the context of both individual and historicalenvironmental factors impacting it. To this end, methodological approaches that bridge sociological, psychological, individual, collective, online, offline, and temporal processes and practices of crowdwork are needed. The paper proposes the Life Course perspective as an interdisciplinary framework that can help address these gaps and advance research on crowdwork. The paper concludes by proposing a set of Life Course-inspired research questions to guide future studies of crowdwork. 44 A. Margaryan and H. Hofmeister / Human Computation (2020) 8:1


INTRODUCTION
The unfolding digitisation of society has stimulated the development of new types of digital labour, underpinned by the practice of crowdsourcing: the use of Internet-based platforms to bring together people from across the world to carry out tasks (Howe, 2009;Lehdonvirta and Ernkvist, 2011). Crowdsourcing includes heterogeneous practices ranging from paid work to contest-based tasks, citizen science initiatives, barter or volunteering (Howcroft and Bergvall-Kareborn, 2018). Some of these practices are location-independent, i.e. they occur entirely online, within digital platforms or apps. Others are location-dependent, that is they are coordinated online, but the actual delivery of services occurs offline ( Figure 1). This paper is focused on and limited to paid crowdsourced work where the delivery of service occurs entirely online; in line with the literature (Margaryan, 2019a;Schmidt, 2017), we use the term crowdwork to characterise this form of digital work (the upper right quadrant in Figure 1). Crowdwork platforms act as intermediaries between people or organisations who post tasks and workers who perform them, by managing the distribution, submission and quality control of the work tasks (Degryse, 2016).  (Margaryan, 2019a) A. Margaryan and H. Hofmeister / Human Computation (2020) 3:5 45 Literature distinguishes two main types of crowdwork: microwork and online freelancing (Kuek et al., 2015). In microwork large tasks (such as datasets to be digitalised or to be processed and labelled by human workers to train artificial intelligence algorithms) are outsourced to the platforms by clients, broken down into small units of work (micro-tasks) and posted on the platform for crowdworkers to carry out for pay. Examples of microwork are tagging images, finding or verifying information on the Web, writing content, for example short product descriptions, or carrying out basic administrative tasks such as data entry and transcription (Gadiraju et al., 2014). Examples of some of the most prominent microwork platforms are Amazon's Mechanical Turk, Aspen (previously Figure Eight), Clickworker and Microworker. In online freelancing type of crowdwork, clients also contract services to distributed workers, however, compared to microwork, online freelancing tasks tend to be larger, more complex, performed over longer periods of time and often requiring specialist skills. Examples of online freelancing tasks are graphic design; software development; business consulting or marketing services. Examples of online freelancing platforms are Upwork, People per Hour, Fiverr, among many others.
Surveys estimated that 5-9% of the EU population and the US population regularly undertake crowdwork (Huws et al., 2016;Smith, 2016). The crowdwork industry is growing fast: a 50% growth in the crowdwork industry internationally has been recorded since May 2016 (Kässi & Lehdonvirta, 2018). With the growing uptake of crowdwork across the world, the development of an empirical, holistic, and nuanced understanding of the nature of crowdwork practices becomes increasingly important. In particular, it is important to understand the nature and range of different types of tasks crowdworkers undertake, crowdworkers' reasons and motivations to engage in this form of work, how crowdworkers go about organising and managing their work, the motivations and perspectives of other key stakeholders such as platform providers and clients, and other key personal, historical, contextual and temporal factors impacting crowdwork.
Over the past decade, the nascent scholarship in crowdwork spanning Social Sciences and Computer Science/HCI has begun to address some of these issues and questions. First, a group of studies focused on empirical investigations of work practices within crowdwork and of factors underpinning workers' decisions to engage in this form of work (e.g. Berg, 2016;Difallah et al., 2015;Gupta, 2017;Gupta et al., 2014;Huws et al., 2016;Kaufmann et al., 2011;Naderi et al., 2014;Posch et al., 2017;Wood et al., 2018). Broadly, these types of studies examine what crowdwork entails, that is, what types of tasks, how it is carried out, who engages in it, and why people take up crowdwork. A second subset of empirical literature in crowdwork has focused on intervention research, in particular on developing and implementing software tools to improve and extend various aspects of crowdwork, such as the design and allocation of tasks and incentives or workflows within the platforms (e.g. Catallo & Martinenghi, 2017;Pilourdault et al., 2017;Valentine et al., 2017;Whiting et al., 2017). Finally, a third subset of the literature has focused on methodological issues. Covering a range of domains and disciplines, this literature has contributed new techniques and know-how on data collection within crowdwork, for example, how to conduct large-scale surveys and experiments in behavioural psychology, economics or education drawing on crowdworker samples, or how to ethically interact with crowdworkers during research studies(e.g. Chandler & Paolacci, 2017;Follmer et al., 2017;Paolacci et al., 2010;Rand, 2012;Vakharia & Lease, 2013).
Our initial observations of the crowdwork literature suggested that research seemed to rely on a limited set of relatively narrow methodological approaches, mostly online experiments, surveys or interviews. Also, the perspectives of key stakeholders, for instance platform providers and clients, seemed yet to be integrated or considered to a sufficient degree, for instance platform clients and platform providers are rarely included as respondents in the data collection effort. Importantly, research has tended to examine workers' experiences as snapshots in time rather studying experiences longitudinally or contextualising them historically and developmentally. This emerging, piece-meal approach has given the research community initial descriptions and interpretations of crowdwork practices and provided an important starting point in a nascent field of study. However, the depth of research in the various areas, and the missing pieces, have yet to be systematically scoped out. Therefore, we set out systematically to review empirical crowdwork literature to identify exactly what methodological approaches and analytical lenses have been used in crowdwork research to date and where the key gaps are.
We argue that to take crowdwork research to the next level of a more nuanced, holistic understanding, it is essential to examine crowdwork practices within the context of both individual and historical-environmental factors impacting it. To this end, interdisciplinary methodological approaches that bridge sociological, psychological, computational, individual, collective, online, offline, and temporal processes and practices of crowdwork are needed. One such approach that can be helpful in bringing a more holistic view to crowdwork research is the Life Course perspective, an interdisciplinary analytical framework focused on understanding human behaviour across the life span (Elder & Giele, 2009;Hofmeister, 2015;Levy, 2013). Next we briefly outline the Life Course perspective. We then describe the review method we used. Subsequently, we present and discuss our findings and evaluate them according to the Life Course analytical framework. We conclude by outlining how the Life Course perspective could be applied to help address the gaps in crowdwork studies, proposing concrete sets of life-course inspired research questions to guide future scholarship in the field.

THE LIFE COURSE PERSPECTIVE
The Life Course perspective stresses the importance of the socio-cultural environment in explaining individual behaviour and life history (Mortimer & Shanahan, 2003). Drawing on sociology, psychology, anthropology, history and biology, the Life Course perspective helps examine the interplay of the individual, his or her setting, and the dynamic processes of change individuals undergo within these settings over time. Similar to nature-versus-nurture debates in human development and psychology, the structure-versus-agency debates in social sciences seek to explain causality: why things end up the way they do, why some people have some outcomes and others have different outcomes. The structure side would say that the way society is designed, the rules and institutions, shape lives and decisions. The agency side says that individuals have choices as to how they negotiate crossroads and decisions and a set of tools and personal qualities that enable or constrain those choices. The question is not so much whether structure or agency is the best or the right explanation in all cases, but rather how and under what circumstances structure or agency determine causal outcomes. The Life Course perspective integrates the influences of structure and agency on social life by focusing on four features: agency, linked lives, context, and timing (Giele & Elder, 1998). The latter three can be seen as aspects of structure. Together, these create processes, for example, the process of working in crowdwork.
Agency refers to an individual's skills, motives and goals, and the self-initiated activities undertaken to fulfil them (Elder, 1994;Hofmeister, 2010). The individual is an active agent who can shape what happens in their life based on their own abilities, goals, and sense of self (Elder, 1995;Giele & Elder, 1998). Human capital -educational achievements, personal networks, self-regulatory skills, knowledge base -contributes to an individual's agency. Applied to crowdwork settings, the life course notion of 'agency' could be operationalized to include such aspects as workers' personal motives to take up crowdwork; their short-term performance goals and longer-term career aspirations; the self-directed learning and skill development activities they undertake to improve their performance; their approaches to planning and structuring their crowdwork activities; and/or their decisions as to what new types of crowdwork projects and tasks they could aspire for.
Linked lives denotes the interrelations between individuals, how the choices and lives of others affect the individual and vice versa (Moen & Hernandez, 2009). Linked lives asks the researcher to examine the influence of social relationships on an individual, whether these social relationships are with parents, spouses, children, co-workers, or neighbours. Social capital is one application of linked lives. Lives can be linked across generations and over historical time, so the linked lives concept also invites us to explore how the decisions and behaviours of earlier generations affect the present, and how the present generation's decisions and attitudes affect future generations. In crowdwork settings, the concept of 'linked lives' could include, for example, online and offline networks and communities such as social media discussion fora or co-working spaces workers are part of; networks with clients; or family and neighbourhood ties and friendships that may support or otherwise impact upon their crowdwork.
Attention to Context extends the research lens outward, considering conditions beyond the individual and the network of his or her influential others to include what we often think of as "structure" (Hofmeister, 2010). Context refers to the setting in which each individual acts or decides: social attitudes, institutions, laws and rules, patterns of behaviour in society, levels of crime, and the prevalence of particular patterns, as well as the built environment. Context, therefore, includes social, cultural, organisational, technological and physical settings, as well as the patterns that emerge from the interplay among these features over time (Blossfeld, 2009). In crowdwork settings, relevant aspects of 'context' include both more immediate factors such as the workers' workspace set up (physical and digital); the platform design (its affordances, interface design, task and workflow design, or the business model); as well as broader social and economic factors such as local cultural attitudes to freelancing and entrepreneurship; local and national laws and technical infrastructures; or local economic conditions such as employment, welfare and regulatory regimes.
Timing refers to the sequencing of events and pathways within an individual life (Elder, 1998;Viry et al., 2013) as well as the placement of the life in a historical context (Hofmeister, 2010). The activities individuals engage in to reach their goals happen in a temporal sequence; the principle of timing says that the sequence matters. Timing tells us we may find important explanations for behaviours and patterns if we look at the opportunity structure of an individual, including social or biological age, in interaction with specific historical events, such as a war, geopolitical transformations such as the fall of the Iron Curtain, or the opening or closing of local employment opportunities (Hofmeister, 2010). When applied to crowdwork, the concept of 'timing' could be operationalized to include factors such as workers' prior educational and work trajectories; personal events such as birth of children, retirement or disability; major local infrastructural developments such as arrival of high-speed internet; or global events such as migration, global financial crisis, or a pandemic.
Methodologically, these four Life Course components can be considered individually and in interaction to help explain observable patterns of similarity and difference, improve causal explanatory power, and understand, for example, the process, motives, and behaviours within crowdwork over time. Compared to mono-disciplinary analytical frameworks such as those from sociology or economics, an advantage of the Life Course perspective is its interdisciplinarity and its focus on an integrative analysis of the individual crowdworker, their peers and networks, their settings, and the dynamic processes of change individuals undergo within these settings. Both quantitative and qualitative methods have been applied in Life Course research (Brannen, 2005;Elder & Giele, 2009), sometimes effectively together, in mixed-method designs (e.g. Brannen, 2005;Brannen et al., 2000;Laub & Sampson, 1998;Moen et al., 2000;Moen & Hernandez, 2009;Sampson & Laub, 1993).

METHODOLOGY
To determine the methodological approaches and analytical lenses used in empirical studies of crowdwork we used the methodology of systematic scoping review (Arksey & O'Malley, 2005;Davis et al., 2009). There are many definitions of scoping studies, but typically they are seen as a form of literature review in which "key concepts and constructs underpinning a research area and the main sources and types of evidence available are mapped rapidly…especially where an area is complex or has not been reviewed comprehensively before" (Mays at al., 2001, cited in Arksey & O'Malley, 2005. A distinguishing characteristic of scoping reviews is that the quality assessment of research reported in the studies included in the review is not part of their remit, unlike in other forms of review, for example meta-review (Arksey & O'Malley, 2005). Another key feature of scoping reviews is that the breadth of coverage of literature is typically prioritised over the depth, i.e. the amount of information extracted and reported from studies. As Arksey and O'Malley (2005) point out, "the extent to which a scoping study seeks to provide in-depth coverage of available literature depends on the purpose of the review itself" (p. 6). They articulate four common situations in which a scoping review may be an appropriate methodology: (i) to examine the extent, range and nature of research activity as a way of mapping new fields where the exact range and nature of material available is difficult to visualise; (ii) to determine the value and feasibility of undertaking a full systematic review; (iii) to summarise and disseminate research findings to policymakers, practitioners or consumers; and (iv) to identify research gaps in the existing literature, drawing conclusions from existing literature regarding the state-of-the art in an area of research. In scoping reviews, the process of identifying gaps in the literature does not include determining gaps in the quality of research reported, only in the coverage of key concepts, constructs or questions motivating the review, because assessing the quality of reported research is not the focus of scoping reviews.
From these four types of scoping studies, our approach was driven by reasons (i) and (iv). First, we aimed to scope and map the range and nature of methodologies and analytical lenses used in empirical studies in the emergent field of crowdwork research. In line with the remit of scoping reviews, we did not assess the methodological quality of the studies we scoped. Having said this, all the studies we reviewed had been published in peer-reviewed journals and conference proceedings; publication in a peer-reviewed venue was a key search criterion as detailed below. Therefore, all the literature included in this review has undergone quality checks at least through the peer-review process. Second, we set out to identify gaps in methodological-analytical approaches used in this segment of crowdwork literature and to propose how an established analytical framework from cognate disciplines (the Life Course perspective) could help address these gaps advancing crowdwork research to the next level.
Whilst we did not carry out a full systematic review, we did apply systematic search and reporting techniques to scope and analyse the literature, hence why we refer to our methodology as 'systematic scoping review'. Our method comprised the following three stages (adapted from Arksey & O'Malley, 2005): Stage 1. Identifying the research question Stage 2. Conducting systematic literature search, using systematic keywords, databases and systematic inclusion/exclusion criteria to identify and select relevant studies Stage 3. Mapping and synthesising the data, including the following two sub-stages: Sub-stage 3.1. Extracting and recording key information from shortlisted literature using a uniform template (what constituted key information was determined by our research question) Sub-stage 3.2. Analysing and synthesising the data These stages are described in detail next.

Identifying the research question
Our starting point was to articulate the research question to guide our scoping review, thinking about which specific aspects of the new but burgeoning field of crowdwork research were particularly important, given our aims. Our research question was: 'What methodological approaches and analytical lenses have been used in published empirical analyses of the practice of crowdwork?' 'Crowdwork practice' was defined broadly to encompass the following key aspects: (i) the nature and the design of crowdwork tasks as perceived and experienced by the workers (for example, the types of work tasks available; workers' perceptions of the complexity (or lack of such) of these tasks; or workers' perceptions of meaningfulness of tasks and the types of meanings workers ascribe to their tasks); (ii) workers' and stakeholders' motivations for participating in crowdwork; (iii) the specific crowdwork practices (such as actions, performance, behaviours), experiences with and perspectives on crowdwork platforms by workers and other key stakeholders such as clients, platform providers, policymakers.
It is important to note that we did not examine the entirety of crowdwork literature, but only a bounded subset of the literature comprising empirical studies focused on the above key aspects. This means that, in deciding if a paper was within scope of our review, we examined whether or not the paper addressed at least one of the above three key aspects. If it did address at least one of these three key aspects, and additionally if it aligned with the inclusion criteria specified below in sub-section 'Conducting systematic literature search' then we considered the paper to be within scope.
We did not include the conceptual, non-empirical literature on crowdwork (for example, we did not include any literature reviews/meta-reviews or other conceptual work). Similarly, we did not include non-peer reviewed sources such as books, nor did we examine the literature focused on purely the technical aspects of crowdwork such as the design of algorithms and machine learning models underpinning crowdwork platforms, the design of workflows or other engineering and computational aspects of interface design. The technical literature is largely focused on machine behaviour rather than human behaviour within crowdwork hence this literature lies outside the scope of our paper.

Conducting systematic literature search
An extensive, systematic literature search sourced articles using the following general and discipline-specific databases: Google Scholar, arxiv.org, ERIC, PsycINFO, ACM digital library, IEEE Xplore, JSTOR, Web of Science, and Sociological abstracts with ProQuest full texts.
Both the UK and the US spelling variants of the following keywords and keyword combinations were used in our search: • Crowdwork, crowdwork and digital labour • Crowdwork behaviour • Crowdwork performance • Online labour markets, online labour markets and microwork/microtask, and crowdwork • Microwork • Online freelancing • Microtask, microtask and crowdsourcing • Crowdwork/crowdworker motivation • Crowdsourcing; crowdsourcing and microtask Digital labour With the help of these keywords, 7931 articles were identified across the nine databases we used. From this extended list, a shortlist was created using the following inclusion criteria: • Studies of crowdwork that address the nature of crowdwork tasks and key actors' (workers', platform owners', clients' and policymakers') perspective -behaviours and motivations -related to crowdwork • Articles about either of both two types of crowdwork only (as defined in the Introduction), that is paid, online work that is both allocated, coordinated and conducted online, covering the microtask and online freelancing platforms • Empirical studies only • Articles published in peer-reviewed journals or conference proceedings • Papers in English or German The following types of empirical studies were excluded: • Books, project reports, briefing papers, and other non-peer reviewed literature • Papers focused on other types of 'gig-economy' work where the coordination of work is carried out online, but the actual delivery of service occurs offline (e.g. taxi hailing, domestic tasks, and food delivery such Uber, Deliveroo or TaskRabbit).
• Studies focused solely on software development, technical infrastructure and machine behaviour within crowdwork platforms, unless such studies offer empirical insight into crowdworkers' and other actors' actual behaviour, motivation or perspective (as explained in the above sub-section 'Identifying the research question).
The papers were shortlisted by three researchers applying the criteria, reading the abstracts, and, in case of doubt (e.g. when the abstracts were incomplete), scanning the full texts. A total of 70 articles fitting the above criteria were shortlisted, read, and summarised.

Mapping and synthesising the data
The next stage of our study involved mapping the key information from the shortlisted articles, extracting and recording key information in a uniform template, followed by analysing and synthesising the material. The consistent approach and the template allowed us to visualise and compare studies, map these onto our framework, and identify the existing gaps in methods and analytical approaches.
We extracted and recorded the following information for each of 70 papers included in the review (see Supporting Information spreadsheet): • Identifying article number • Full reference including author(s), year, title, etc.
• Type of crowdwork addressed: microwork or online freelancing • Specific platform(s) covered • Key variables, research questions or hypothesis • Respondent types, for example crowdworkers, platform owners, clients, etc.
• Sample size • Research design (e.g. experiment, mixed-methods, ethnography) • Specific method(s) used • Analytical lens/framework adopted in the study • Life Course components addressed Next, we analysed and synthesised the data, first based on the types of crowdwork, sample population, and type of study, and then according to the Life Course framework. Based on this mapping, we developed a narrative account of our findings, a typical approach used in scoping studies (Arksey & O'Malley, 2005;Pawson, 2002).

Scoping the crowdwork literature
Several key patterns of the way crowdwork is studied emerged, in particular with regards to the types of crowdwork and respondents, the methods/research designs, and the analytical frameworks used.

Types of crowdwork and respondents studied
Microwork platforms dominated the type of crowdwork researched (77%, 54/70), including 78 percent of these using Amazon's Mechanical Turk (MTurk) or Mturk-related workers communities (42/54). Only ten out of 70 studies (14%) focused on online freelancers. A small number of studies covered both microwork and online freelancing (6%, 4/70). We hypothesize that the reason for this overproportion of microwork and the comparative neglect of online freelancing is the prevalence of survey and experimental research (as detailed later in this section), particularly the relative ease with which the large samples required for surveys and experimental studies can be drawn from microwork platforms, in a considerably shorter time and at lower cost than they can be from online freelancing platforms. A consequence of this skew in the crowdwork literature towards microwork and the Mechanical Turk platform is that microworkers' and Mturkers' practices and views may be overrepresented in the literature. Given the differences in the demographics, qualification requirements, compensation levels and the nature of tasks within microwork and online freelancing we discussed earlier, the current literature likely offers a biased and incomplete understanding of the practices and behaviours within crowdwork.
Furthermore, we observed that 78% of the studies in our sample (55/70) focused on crowdworkers only, whilst 10 of 70 studies (14%) included other stakeholders alongside workers, including platform managers, and seldom also clients, trade union representatives, career coaches, policymakers, and researchers. In sum, our review points to an under-representation in the crowdwork literature of online freelancers and of stakeholders other than workers.

Survey research.
The largest group of studies in our sample (30%) follow survey design. Most of these use cross-sectional designs and use their own scales; only two survey studies used extant psychometric scales (Posch et al., 2017; and only one was longitudinal (Difallah et al., 2018). Most survey studies in our sample examined microwork, focusing on MTurk, Figure Eight, Clickworker, Microworkers, Prolific, LeadGenius, Amara, and CrowdWorks (a Japanese platform, Majima et al., 2017); only two studies which had adopted survey design focused on online freelancing. All survey studies were conducted with crowdworkers; they did not include other stakeholders. Most crowdworker samples used in the survey studies were international in scope, although in some studies the samples were drawn specifically from crowdworkers in the US or India (Berg, 2016;Bucher & Fieseler, 2016;Fieseler et al., 2019;Jacques & Kristensson, 2019;Jiang et al., 2015;Kost et al., 2018;Newlands & Lutz, 2020), Japan (Majima et al., 2017) or Arabspeaking countries (Mubarak et al, 2016). In survey studies, most samples ranged from 460 to 12000 respondents; and one particularly large sample had 40,000 respondents collected over 28 months (Difallah et al., 2018). The smallest samples among survey studies had 110, 113 and 203 participants respectively Kost et al., 2018;Margaryan, 2019b).
Mixed-method research. The second largest group of studies in crowdwork literature is mixedmethod studies (23%). These used the following methods and method combinations (Morse, 2010; the notations used below are based on Creswell & Plano Clark, 2011): • Quantitatively driven, sequential qual->QUAN design studies were either focused on questionnaire survey of crowdworkers informed by (semi-structured) interviews with them (Zyskowski et al., 2015), or involved interviews with experts such as scholars, trade union representatives and platform providers, followed by a questionnaire survey of workers (Al-Ani & Stumpp, 2016;Feldman et al., 2017). • Quantitatively driven, sequential QUAN-> qual studies, where the major data collection effort focused on a questionnaire survey, followed by a small number of interviews (Rani & Furrer, 2020). • Quantitatively driven, concurrent qual+QUAN designs draw on the analysis of digital behaviour trace and log data as well as work artefacts collected from the platform, coupled with a questionnaire survey of crowdworkers . • Qualitatively driven, sequential QUAL-> QUAL -> quan design studies which start off with observations of crowdworkers completing tasks, followed by interviews, analysis of workers' performance on the tasks, and a questionnaire survey (Vashistha et al., 2018). • Concurrent QUAL+QUAN design studies, where both paradigms were given equal importance, draw on ethnographic observation, interviews and questionnaire surveys of crowdworkers combined with the analysis of behavioural trace data mined from the platform (Graham et al., 2017;Gray et al., 2016) or a combination of survey and interview whereby both types of datasets were given equal importance (Wood et al., 2018). • Sequential QUAN -> QUAL design studies that have both paradigms equally central to the research design, for example a longitudinal survey, followed by interviews and ethnographic observation, coupled with a geographic mapping task to determine crowdworkers' location (Kingsley et al., 2015). • Quantitatively driven, sequential intervention research using QUAN -> qual -> intervention, for example questionnaire survey and interviews with crowdworkers followed by an intervention and evaluation of it (Gadiraju et al., 2017;Khanna et al., 2010;Wang et al., 2017).
• Qualitatively driven research supplemented by document analysis, in particular interviews coupled with review of information and data publicly available on platform's website  or online fora associated with the platform (Gerber & Krzywdzinski, 2019). • Quantitatively driven research combined with an experimental study and behavioural trace data (Zhuang & Gadiraju, 2019).
About two-thirds of the mixed-method studies in our sample were conducted within microwork platforms, predominantly MTurk, with a limited number of studies focusing on platforms such as Microsoft UHRS, LeadGenius, Amara.org or Figure Eight  For most mixed-method studies, the data were drawn only from crowdworkers (10/16), while six studies included data from other stakeholders, in particular clients ; job coaches (Zyskowski et al., 2015); experts such as data scientists/data analysts (Feldman et al., 2017); platform providers, scholars, and trade union representatives (Al-Ani & Stumpp, 2016; Gerber & Krzywdzinski, 2019); and policymakers (Graham et al., 2017). In the majority of mixedmethod studies, a broad spectrum of crowdworkers was included, however some studies drew on special groups of participants such as workers from specific countries, e.g. US and/or India (Gadiraju et al., 2017;Gray et al., 2016;Khanna et al., 2010;Kingsley et al., 2015); or blind, lowincome crowdworkers (Vashistha et al., 2018). In most mixed-method studies, the sample size ranged from 100 to 502 participants; only one study drew on a significantly larger sample of 6338 participants , whilst three of the 16 mixed-method studies used significantly smaller samples of 22, 32 and 39 participants (Gerber & Krzywdzinski, 2019;Retelny et al., 2017;Vashistha et al., 2018).
Experiment. The third largest group of studies are behavioural experiments (20%). Most use experiments as the only method, but in some cases experiments are combined with (longitudinal) digital behaviour trace data or various activity logs amassed from the platforms (Dalle et al., 2017;Hata et al., 2016;Sodré & Brasileiro, 2017). We term the latter sub-group of experimental studies 'Experiment Plus.' The experimental studies in crowdwork settings are typically focused on testing hypotheses about various aspects of workers' behaviour and the impact of various aspects of the platform design on these behaviours. They typically involve presenting workers with different experimental conditions or inducing a particular affect (such as mood), then testing their responses with (psychometric) questionnaires.
All experimental studies focused on microworkers drawn predominantly from MTurk and, to a limited extent, FigureEight. No experimental studies included online freelancers. Most experimental studies were centred on workers; only in one study experiments with workers were supplemented by digital behavioural trace data on clients (Sodré & Brasileiro, 2017). Most experiments included an international sample of microworkers, except one study which focused on US workers . The samples sizes ranged from 45-250 for smaller experiments to 779-2000 for larger studies. The digital trace datasets were typically large, ranging from 2.4 million to 9 million units and include data such as annotations, number of jobs available on the platforms and so on.

Interview research.
There were eight interview studies in our sample. Most of these were semistructured interviews (in person or by telephone), and one study used written interviews (Deng & Joshi, 2016). Some studies combined interviews with document analysis, for example review of publicly available platform policies and materials (Barnes et al., 2015) or case study development (D'Cruz, 2017 (Barnes et al., 2015;Schörpf et al., 2017), as well as clients and crowdsourcing experts (Kinder et al., 2019;Schörpf et al., 2017). The interview studies typically had sample sizes of 14-24 respondents. Only one study, where written interview was used to collect narrative responses from MTurkers had a larger sample of 55 respondents (Deng & Joshi, 2016).

Digital/behavioural trace data analysis.
Five studies used digital trace data analysis as a standalone method. Digital trace data typically involved behavioural traces and other activity log data collected directly from the platforms. Examples included logs on workers actions/clicks, time spent on tasks and other interactions on the platforms, typically collected over a period of time. For example, one study collected behaviour trace data on 27 million tasks performed by 70,000 workers (Jain et al., 2017), whilst another crawled data on 80,000 workers (Hirth et al., 2011). Some of these digital trace data were longitudinal; for example, a study analysed log data on workers systematically collected over a 5-year period (Difallah et al., 2015). All the studies using digital trace data involved MTurkers only, not limited to any particular country.
Ethnography. Four out of the 70 studies in our literature review used ethnography as the key methodological approach, including online or digital ethnography, where researchers conducted (participant) observations of crowdworkers' interactions in various online discussion fora and other online environments. The ethnographic studies often used two or more qualitative methods including document analysis, auto-ethnographic observation and field notes from researchers' own experiences as workers or clients on crowdwork platforms (Lehdonvirta, 2016), interviews with workers and platform providers (Lehdonvirta, 2016), and online participant observation coupled with interviews with workers (Lehdonvirta, 2016(Lehdonvirta, , 2018Martin et al., 2014Martin et al., , 2016. All ethnographic studies were conducted within microwork settings, in particular the Mechanical Turk, MobileWorks and CloudFactory; none focused on online freelancers. Most ethnographic studies included an international sample of crowdworkers, but some targeted only US and Indian microworkers with a specific intention of comparing these national samples (Martin et al., , 2016. In ethnographic studies, sample sizes were around 32-35 participants and several hundred hours of online observation.

Social network analysis.
One outlier in our sample was a study focused on an ego-centric social network analysis mapping crowdworkers' connections to each other . This study focused on microworkers within MTurk, analysing social network data from 10,000 workers.

Document review.
Finally, another outlier study used content analysis examining the marketing and informational materials publicly available on the website of 44 different platforms (Pongratz, 2018). This study focused on the platforms' perspective only. The study was atypical among the literature we scoped in that it used document analysis as the only, stand-alone method; within crowdwork literature document analysis is typically used in combination with other methods as described earlier.

Analytical frameworks/lenses used
Most crowdwork studies (61/70) did not explicitly mention any overarching analytical lens or framework being used (Table 1). We recognize that the studies that did not mention an analytical framework may in fact have used one to guide the collection and/or interpretation of their data. However, for the purposes of this review, we can only ascertain that a particular analytical framework was applied if it is explicitly stated and described in the paper shortlisted for inclusion in the review. Therefore, Table 1 lists only those analytical lenses/frameworks that were explicitly mentioned in the papers reviewed.
The eight studies that explicitly articulated an analytical lens used different general methods of inductive analysis such as grounded theory (Zyskowski et al., 2015), ethnomethodology (Martin et al., , 2016, and hermeneutic phenomenology coupled with critical theory (D'Cruz, 2017). Some applied existing theories, typically from psychology or sociology, to structure the analysis and explanation of their findings: these were Person-Environment/Person-Job Fit Theory (Feldman et al., 2017), Self-assessment Theory (Gadiraju, 2015), Job Characteristics Theory coupled with Work Value Perspective (Deng & Joshi, 2016), Equity Theory of Motivation (d'Eon et al., 2019), Actor Network Theory (Kinder et al., 2019) an employability framework (Barnes et al., 2015). One study (Posch et al, 2017) stated that their data collection instrument was grounded in the Self-Determination Theory, but provided no further evidence of whether and how the theory was actually used to structure the analysis and the interpretation of their findings.
As Table 1 suggests, the interview studies appeared more likely to articulate an analytical lens compared to the other types of studies. Among studies using the methods of survey (with one exception as explained above), digital trace data analysis, document review and social network analysis, no analytical frameworks were articulated and/or applied. • Grounded theory (Zyskowski et al., 2015) • Person-Environment/Person-Job Fit Theory (Feldman et al., 2017) • No analytical lens (n=14)

Social Network Analysis (n=1)
No analytical lens

Document review (n=1) No analytical lens
Having summarised the overall scope of the literature, in the second round of the analysis we focused on mapping the literature on to the Life Course perspective.

Mapping the crowdwork literature on to the Life Course perspective
In mapping the crowdwork literature onto the Life Course framework, we carried out two kinds of analyses. First, we examined each paper individually to identify which of the four components of the Life Course the paper had addressed (Table 2). We found that only three studies addressed all four components of the Life Course perspective, albeit without explicitly setting out to do so (D'Cruz, 2017;Kingsley et al., 2015;Yang et al., 2018). For example, Kingsley et al. (2015) used a mixed-method design including a longitudinal survey, interview, ethnographic observation and a geographic mapping task to analyse power asymmetries in microwork settings. In particular, they examined the influence of contextual factors such as country and culture (US and India), wage structures, socioeconomic values and task design on the dynamics of power distribution between crowdworkers and clients (linked lives), analysing workers' individual perspectives and situations (agency) using longitudinal data (timing). In the second example, using the lens of hermeneutic phenomenography coupled with critical theory, D'Cruz (2017) addressed the four life course components in her interview study focused on Indian online freelancers' experiences of empowerment and disempowerment on the platforms. Her in-depth analysis of these freelancers' individual experiences, views and actions in terms of income, quality of life, career development, upward mobility and work-life balance (agency) is grounded in a wide-ranging analysis of environmental factors such as platform checks and facilities, the specificities of the local labour markets such wide-spread feudalistic work cultures, delayed remuneration and workers' sociodemographic backgrounds (context) and connections with clients in 'the West' (linked lives) as well as workers' historic trajectories leading them into online labour (timing).
Most other studies in our sample addressed at least one or two components: 33 studies covered one component, 26 studies covered two components, and seven studies covered three components, as summarised in Table 2. Overall, we found that 'context' was the most frequently addressed component: 52/70 articles studied context-related variables/factors, followed by 'agency' (43/70). Only 6 studies covered 'timing' and 18 studies examined 'linked lives.'  Yang et al. (2018) x x x x 4 Gray et al. (2016) x x x 5 Lehdonvirta (2018) x x x 6 Martin et al. (2016) x x x 7 Wang et al. (2017) x x x 8 Retelny et al. (2017) x x x 9 Fieseler et al. (2019) x x x 10 Wood et al. (2018) x x x 11 Posch et al. (2017) x x 12 Vashistha et al. (2018) x x 13 Bucher & Fieseler (2016) x  Morris et al. (2013) x 38 Schörpf et al. (2017) x 39 Dalle et al. (2017) x 40 Pongratz (2018) x x 42 Hata et al. (2016) x 43 Jain et al. (2017) x 44 Naderi (2018) x 45 Graham et al. (2017) x 46 Gadiraju et al. (2017) x 47 Hirth et al. (2011) x 48 Khanna et al. (2010) x 49 Jacques et al. (2019) x 50 Gerber & Krzywdzinski (2019) x 51 Rani & Furrer (2020) x 52 Newlands & Lutz (2020) x 53 Zyskowski et al. (2015) x 54 Gadiraju et al. (2015) x 55 Barnes et al. (2015) x x 69 Ma et al. (2018) x 70 Mubarak et al (2016) x Second, we analysed the literature to identify what specific sub-components of Agency, Linked Lives, Context and Timing were addressed (Table 3). We found that the two most frequently examined components -Agency and Context -were also the most comprehensively detailed ones, as determined by the number of sub-components for each identified in the literature, as shown in Table 3. In particular, a broad set of contextual sub-components have been covered in the crowdwork literature, ranging from micro-factors such as design of crowdwork tasks, workflows, and platform features; meso-factors such as skill requirements for specific types of crowdwork tasks, workers' broader toolsets, personal lives and contexts; to macro-level factors such as the influence of country and national contexts on crowdwork, or crowdwork policies, incentives and reward systems. Agency-specific sub-components studied in the literature included various aspects of individual behaviour and motivation to engage on platforms, for example workers' selforganisation, collaboration and tool development practices to address the various constraints imposed by the platforms; workers' perspectives and experiences with regards to potentialities and limitations of crowdwork such as their views on meaningfulness of crowdwork; their personal values impacting crowdwork and their experiences with the other key actors of online labour (platform providers, clients, policymakers); as well as analyses of experiences and requirements of particular groups of workers such as the disabled. In contrast, considerably fewer sub-components of Linked Lives and, particularly, Timing have been addressed in the literature. With regards to Linked Lives, crowdwork studies have focused on mapping workers' networks and collaborative constellations, both online and offline, examining, for example, the dyadic relationships between workers and clients or triadic relationships between workers, platform providers and clients. Subcomponents of Timing have included analyses of changes in the demographics of platform workers and clients and changes in power distribution between workers and clients over time, as well as crowdworkers' biographical trajectories, employment histories and patters. The findings are detailed in Table 3.
To supplement our analysis, in the next section we give examples of how the Life Course perspective may be operationalised for crowdwork studies. To this end, we outline a set of Life

A. Margaryan and H. Hofmeister / Human Computation (2020) 3:5 65
Course guided research questions and methods that may be used by crowdwork scholars to extend and advance the research designs developing a more holistic and nuanced understanding of crowdwork.

APPLYING THE LIFE COURSE PERSPECTIVE TO CROWDWORK: POTENTIAL QUESTIONS TO GUIDE FUTURE RESEARCH
We may want to know about the perspectives of workers, platforms, or clients and we can directly ask them particular questions as exemplified by the mixed-method, interview, survey and ethnographic studies in our sample. In addition, we can cull workers' online materials as the studies using digital trace data analysis have done, but we can also conduct background research on the region where the worker is located (none of the studies in our sample explicitly did this). We can use platform websites to get a sense of the policies and scope of the platform, as Pongratz (2018) has done. We can draw on the experiences of participant observers (researchers posing as workers, as in Lehdonvirta, 2016), we can talk to local policy makers (Graham et al., 2017), and we can inform our analysis through a comprehensive review of literature about crowdworkers. For the majority of research on crowdwork from a Life Course perspective, the experiences and background of the crowdworkers themselves will be of interest.
The two easiest ways to get Life Course-related data on workers are online surveys or biographical, life history interviews (e.g. Barbeiro & Spini, 2017). We may also get some information, for example, by coding online discussion threads, as a number of studies in our sample have done, but we may also review broader literature on the modern history for participants' countries, so that we may be in a better position to understand the geopolitical and the socioeconomic contexts of the workers. Next, we identify example questions with possible methods for collecting these data according to the four areas of life course research: agency, context, linked lives, and timing.
Research questions that open the exploration of crowdworkers' agency. Agency refers to the skills, motives and goals an individual brings to the picture. To assess levels of agency, for example, a survey could reveal the crowdworkers' self-regulatory strategies and self-efficacy beliefs (Fontana et al., 2015;Margaryan, 2019aMargaryan, , 2019b, and, when combined with interview with workers, could provide examples and details of these beliefs and their behavioural manifestations. Mixedmethod designs blending interviews, surveys, inductive analysis of crowdworkers' biographic descriptions and inductive analyses of their online interactions and activities could help us understand some key factors which, moving forward, would be central to advancing our understanding of crowd workplaces. First, key personality traits such as risk taking, tolerance for ambiguity, internal locus of control, need for achievement, tenacity that may draw individuals into online labour platforms and/or enable them to succeed in this form of work (e.g. Behrend et al., 2011). Second, individuals' (goal-directed and self-reflective) approaches to career planning in the context of the overall life course, captured in the life course concept of 'planful competence' (Clausen, 1993). Third, the broad range of diverse motivations to take up crowdwork, uncovering the variety of personal and environmental drivers shaping individuals' decisions to engage and persist in crowdwork as a number of studies within our literature review have done. Finally, personal developmental and learning goals, knowledge and skills, crowdworkers' learning strategies and activities -how workers go about setting, monitoring, modifying, achieving their learning goals and what learning activities they undertake to achieve these (Barnes et al., 2015;Margaryan, 2016).

Research questions that open the exploration of crowdworkers' linked lives.
Research on linked lives points out the way crowdworkers are not working in isolation, but rather are surrounded by other people and influences, both within the platform and in their broader communities, currently and in the past. To find out about linked lives, social network analyses (bounded and egocentric), in addition to (auto)biographical interview and survey, would be useful (e.g. Aleandri & Russo, 2015;Scott & Carrington, 2011). What are the interdependencies with other people in crowdworkers' lives, especially where these have implications for career choices (e.g. family and caring responsibilities, or dependence on loans from family members)? Who are 'passage-helpers' and 'significant others' in relation to crowdwork -mentors, friends, family, role models -in workers' current situations and from their past? We would also be interested in linked lives from the perspective of crowdworkers' broader professional networks. What are the main professional networks in and outside of the crowdwork community? What are the workers' online communities both related and unrelated to crowdwork? What are the crowdworkers' client networks? Here using (auto)biographical, narrative interview questions and various SNA methods, including the analysis of online, social media and offline, analogue networks coupled with digital trace data analysis, would be useful. Finally, we would want to ask, what is the balance of online and offline networks and how do crowdworkers straddle and traverse these to help support their work on online labour platforms? To date, limited attention has been paid to these questions in the crowdwork literature.
Research questions that open the exploration of crowdworkers' context. Context examines the circumstances within which crowdwork is taking place: the platform as a workplace but also the broader physical, geographic and socio-economic and cultural environments inhabited by the workers. Our analysis showed that of the variety of relevant contextual variables, crowdwork literature to date has focused largely on examining how the platform design and task design influence crowdwork using survey and experimental research.
Moving forward, additional methods and measures could be used to elucidate the influence of platform design features on crowdworkers' practices, including interviews with workers, platform providers and clients; review and analysis of platform design and scoping and taxonomic categorisation of crowdwork tasks as well worker-supplied artefacts. To further explore context in crowdwork settings, ethnographic designs, surveys and biographical interviews are most useful, supplemented with historical reports and existing literature about particular platforms, geographic chararegions, and policies. In particular, studies could analyse historical times the workers live in specific to their countries and regions as well as broader international historical events, including both past and present events, for example large-scale geopolitical transformations, such as revolutions and dissolution of states, and major economic crises they lived through, or their migration experiences over the life course, which may have influenced crowdworkers' personality traits, self-efficacy beliefs and employment pathways. Furthermore, crowdwork studies could examine family values on education, employment, autonomy and lifelong learning that the crowdworkers experienced growing up, alongside the analysis of their educational and training trajectories. Also, our understanding of crowdwork practices could be advanced if we examined the influences of social class, age and gender and the patterns in which these factors are implicated within crowdwork settings.
Further points of interest on context would include the key characteristics of the generation or cohort that crowdworkers belong to that may have implications for understanding their practices. We would be able to construct cohorts or generations by year of birth and significant events through information in a survey together with historical information to create cohort concepts, including gender and region or country. Another contextual dimension worth analysing is the physical environment of the crowdworker, e.g. the typical location of work, the technology set up, as well as the local infrastructure such as country-specific electrification and internet speed, technology adoption rates, local culture (e.g. attitudes to entrepreneurial activity), local economic conditions, employment and regulatory regimes, and so on. To gather this information, we may need to draw on interview questions and inductive analysis, published literature such as country-specific statistics published by the UN and other relevant agencies, interviews with policymakers, and worker-supplied artefacts such as photographs of their workplaces and the surroundings.

Research questions that open the exploration of crowdworkers' timing.
To find out about timing, that is, the sequencing of events that might have an influence on outcomes, biographical interviews and surveys are most useful, and document reviews are possible. For example, life history calendars can help answer some questions around timing (Freedman et al., 1988). Demographic data from surveys would also be helpful in elucidating relevant pathways and trajectories. For example, we could collect information on the age when someone began earning their own income, the time or age or circumstances at which retirement occurs, information about disabilities, migration histories, when the decision to freelance occurred and whether it was voluntary or not, or other events of abrupt change that may have influenced crowdwork (e.g. illness, divorce, loss of job). Longitudinal surveys and interviews, coupled with analysis of digital trace data culled from platforms, would be essential to elucidate these factors that hitherto have been addressed in a limited way within the crowdwork literature.

CONCLUSION
The paper presents findings of a systematic scoping review of methodological approaches and analytical lenses used in empirical research on crowdwork, focusing on the nature of crowdwork, the work practices and human behaviours within crowdwork. The paper identifies important gaps in the current methodological-analytical approaches. The paper contributes an interdisciplinary methodological framework -the Life Course perspective -to help address the gaps and advance the emergent but growing scholarship on crowdwork. The Life Course Perspective stresses the importance of the socio-cultural environment and life history in explaining individual behaviour. By focusing on the interplay of the individual crowdworker, their peers and networks, their settings, and the dynamic processes of change individuals undergo within these settings, the Life Course Perspective facilitates the production of nuanced and contextualised analyses of crowdwork practices that, as we demonstrate through our review, have hitherto largely been lacking in the crowdwork literature.
This review study identified eight key research designs in crowdwork studies to date, with the top three most frequently used designs being survey research, mixed-method research, and experiment. These are followed by interview studies, digital/behavioural trace data analyses, ethnographies, social network analysis and document review. Most crowdwork studies in our review did not explicitly apply any overarching analytical frameworks, that is no analytical framework was articulated in the published articles arising from those studies. The limited range of analytical approaches used in crowdwork studies included general methods of inductive analysis, such as grounded theory, ethnomethodology and hermeneutic phenomenology coupled with critical theory, as well as some theoretical frameworks in particular from (work) psychology, sociology and employability studies. The interview studies were more likely to use an analytical lens, compared to other types of studies.
In mapping the crowdwork literature onto the Life Course framework, we found that only three papers in our sample addressed all four components of the Life Course -the agency, linked lives, context and timing. The majority of the studies addressed only one or two components -typically context or agency -and only a small minority of studies covered three components of the life course. In particular, a broad set of contextual sub-components have been covered in the crowdwork literature, ranging from micro-factors such as design of crowdwork tasks, workflows, and platform features; meso-factors such as skill requirements for specific types of crowdwork tasks, workers' broader toolsets, personal lives and contexts; to macro-level factors such as the influence of country and national contexts on crowdwork, or crowdwork policies, incentives and reward systems. From this analysis and mapping, we produced a set of life-course guided research questions, with suggestions for relevant methodologies, to help advance research in crowdwork.
This paper demonstrates how the Life Course Perspective could be operationalised to help enrich, extend and methodologically strengthen crowdwork research. The life course framework and the associated research questions we outlined could help bridge the individual, social, contextual and temporal dimensions and factors that play a central role in crowdwork yet have been overlooked by crowdwork researchers. One of our key findings is the over-representation of microworkers from MTurk within crowdwork studies, therefore future research should focus on online freelancers and online freelancing platforms, as well as on microwork platforms other than the MTurk. Furthermore, we found that almost three quarters of crowdwork studies focus on crowdworkers' only; research would benefit from more triangulation of perspectives, in particular including clients', policymakers', job market intermediaries and other key stakeholders' perspectives in future research.
Central to advancing our understanding of crowdwork is the use of appropriate and holistic methodologies and analytical lenses. The Life Course Perspective is powerful analytical, interdisciplinary instrument that can usefully supplement the growing methodological repertoire within crowdwork research.