Prolific
Other Microtask
UK-based platform, founded to recruit participants for academic microtasks, especially surveys and online experiments.
Terms of Service
Rejection:negative rating Change:negative rating Warranty:negative rating Client Contact:positive rating Worker Contact:positive rating
Rejection:negative rating
Change:negative rating
Warranty:negative rating
Client Contact:positive rating
Rejection:positive rating
Platform Overview
Prolific, founded in 2014 at Oxford University as Prolific Academic, recruits workers for online studies. The site’s marketing specifically targets academic researchers and startups as potential customers. In social media, Prolific has positioned itself as an alternative to Amazon Mechanical Turk for researchers seeking to gather data online quickly and at low cost.[3]
-
Platform History
Prolific was launched as “Prolific Academic” in April 2014 by Ekaterina Damer and Phelim Bradley, graduate students at Oxford and Sheffield Universities in the UK. In September 2014 the company joined Oxford University’s startup incubator program, part of Oxford University Innovation. In March 2017, Oxford University Innovation announced that “more than 1,300 researchers from more than 300 academic institutions worldwide” had used the platform, which hosted a pool of 70,000 registered workers and growing. Researchers had collected “over one million unique responses in over 6,500 studies” via the platform since its 2014 launch, the announcement said.[4]
-
Business Model
The platform charges a fee of 12.5% of participant rewards + £0.10 per participant.
Their website states that the company “endorse[s] the principle of ‘ethical rewards'”, meaning that they require “researchers compensate participants with at least £5 (~6.50€ / $7.50) per hour.”[5] However, at present researchers appear to sometimes underestimate the time their studies will take, leading to pay of less than £5 per hour.[6]
Some workers report success contacting Prolific management regarding underpaid tasks. At least one worker reports that they once contacted management regarding an underpaid tasks; management contacted the researcher, who then paid the workers to meet the wage requirement. It’s not clear how often this happens, however.
-
Worker Classification
Workers are self-employed.
-
Jobs and Clients
Academic surveys and experiments; software testing.
Example Jobs
Active studies on 30 Apr 2017:
Example Clients
Most clients are academic researchers. According to a March 2017 press release by Oxford University Innovation, Prolific’s customers include researchers at prestigious universities including Cambridge University, the London School of Economics, Yale, and Stanford.[7]
-
Work Process
Researchers/customers post studies; workers complete them; customers review and accept or reject (i.e., pay or do not pay for) the submitted work. Workers can “cash out” after they have accumulated £5 in their account.
-
This information was collected from 25-100 verified workers on the platform in 2016 and 2017. More information
-
Introduction and Survey Notes
In contrast to other platforms on which we surveyed workers, a majority of respondents to the Prolific survey worked only a few hours a week on the platform. Only one respondent worked more than 10 hours per week, and this individual worked only twelve hours a week on the platform.
Nonetheless, many respondents were experienced crowdworkers. Three quarters of respondents had worked on Prolific for at least 6 months. In addition, almost half of respondents reported also working regularly on other online work platforms — including Clickworker, Mechanical Turk, CrowdFlower, Appen, and Upwork.
-
Pay
Hourly Wages MINIMUM: €0.47 MAXIMUM: €16.44 AVERAGE: €6.60 MEDIAN: €5.87 Payment Overview
The median rate of pay for our survey respondents was only €5.87 — lower than wages reported for both Mechanical Turk and MyLittleJob. However, according to respondents, the site does pay much better than the wages they receive on other survey-specific online sites.
Nonpayment was much less of an issue on Prolific than other platforms. Less than a third of respondents had ever experienced nonpayment, and 100% of those had only encountered it once or twice.
Hourly Wages
Our survey results suggest that pay on Prolific is not as high as Prolific promises users. One of the key features of Prolific is supposed to be its principle of “ethical rewards” — that is, a minimum wage for workers on its platform:
On Prolific, we endorse the principle of ‘ethical rewards’. That is, we stipulate that researchers compensate participants with at least £5 (~6.50€ / $7.50) per hour.[8]
Current exchange rates on the date of our survey put the prolific standard of “at least £5” per hour at €5.87 per hour – the same as the median wage for our survey respondents. Thus, nearly half of respondents made less than the promised wage.
An even smaller number of respondents — only about 28% — made at least a German minimum wage of €8.50/hour.
Hourly Wage Distribution
Percentage of survey respondents whose hourly wages fell into each category
Curiously, most workers described in comments that they were happy with the pay on Prolific — and indeed most perceived it as having relatively high pay, especially for an online survey platform. We believe this might partially be explained by the geographic diversity of our survey respondents. Many of the lowest paid workers in our survey were living in countries with minimum wages ranging from €1 to €3 per hour; most of the workers who reported a wage less than the Prolific standard wage were making close to or above their local minimum wage.
As with other platforms, workers reported spending a fair amount of time looking for tasks. On average, respondents in this survey spent a little over an hour looking for work for every hour that they actually worked on tasks on the platform. If this time were taken into account, wages on the platform would be even lower.
Nonpayment experiences
29.27% of survey respondents have experienced non-payment at least once
Only about a third of survey respondents reported ever experiencing nonpayment, and of these, not a single person reported it as something that happened more than once or twice.
Reasons for nonpayment were varied.
Three respondents reported running into an issue where they took too long filling out a survey, and thus did not receive payment.
Two respondents had their work outright rejected, in one case because of a failed “attention check” question. While the worker agreed that they got a particular question ‘wrong,’ they did feel that they had actually been paying attention and completed the survey to the best of their ability.
More troubling, however, was that three quarters of those reporting nonpayment, described a bug either in the survey setup by a requester or in the Prolific platform. These included things like surveys not properly registering a Prolific worker ID, surveys not generating completion codes, and errors in the screening process:
Prolific Academic normally pre-screens individuals for certain jobs (i.e. they must fit a certain demographic), but occasionally this screening fails and I was a couple times assigned a task I wasn’t qualified to do, and did not get paid for doing it.
While some workers were successful at working out payment with either the platform or the requester directly, others were turned away without satisfactory resolution.
Frequency of Nonpayment Experiences
Timeliness
Some workers also commented on Prolific’s responsiveness about pay, appreciating that payment through the platform was always on time.
This is probably best site for studies on internet. Prolific Academic pay always on time. It is very interesting and fun.
The platform itself was deemed reliable as a payment intermediary.
-
Communication
The communication features on Prolific are very good for workers with regards to contacting both requesters and platform management.
Workers reported very positive experiences with Prolific support. Few workers had actually contacted a requester or client, but those who commented on client communication mostly reported positive interactions.
Prolific does not provide online forums or other discussion features for workers to communicate with each other.
Communicating with management
Most respondents were very happy with communication with the Prolific team. For example, as this worker elaborated:
Again, the communications with Prolific have been excellent. Even when my tone is very harsh and frustrated, they have replied using the utmost in professionalism. Some of the frustrations might have given me cause to just say too much trouble, but the openness of the Prolific staff has given me a feeling that they are trying to make things better all the time. There have been poor experiences with researchers, albeit few, but Prolific took care of them. I have never been let down when communicating with the Prolific Academic team. Never.
Percentage of respondents who said that communication with management is...
Communicating with clients
Prolific also provides a messaging system to send email-like messages to clients on its website. Workers were very happy with this arrangement and liked that they could contact clients/requesters directly through the platform interface.
Percentage of respondents who said that communication with clients is...
Communicating with other workers
Prolific does not, to our knowledge, offer on-site forums or other communication media for workers to talk to eachother. Workers did use offsite forums and social media to communicate with eachother.
Outside of the subreddit r/ProlificAc, most respondents who reported communicating with other Prolific workers did so on forums for other platforms, especially Mechanical Turk:
The other Prolific users I know are also Mturkers, so I talk to them on Mturk-related sites.
Workers mentioned sites including mturkcrowd.com and Mturk IRC channels as well as more general forums and sites about how to earn extra money, such as the subreddit “r/beermoney” and the website “Money Saving Expert.”
In addition, several respondents said they had personal offline friends that they worked on Prolific with, and so they would just chat in person, in addition to using general social media tools like Facebook and twitter.
-
Evaluation
Prolific does not offer direct evaluation of either workers or requesters/researchers as part of its interface.
However, work products are of course evaluated in that requesters/researchers can accept or reject survey responses.
Percentage of respondents who said that evaluations of workers are...
Only a few workers had experienced rejection of their work on Prolific.
There is not really a specific review system in place. Seems that work is either rejected or approved. I have never received a rejection for poor work and do not expect to.
A few workers ran into issues like running out of time on a survey or not getting a completion code that amounted to automatic rejection of their work. However, most of these workers reported that they resolved the issue after contacting requesters — either through the online interface, or by emailing researchers directly:
I have yet to have anything rejected, but there have been timer issues. A malfunction or glitch in the system that did not produce a code. Every single time I have been paid. Allowing manual approval is an excellent tool that other platforms do not seem to use.
-
Tasks
Prolific scored very high marks from workers for having tasks that are meaningful, interesting, fun, and satisfying most of the time. Also, very few workers reported finding work on the platform that was dangerous, demeaning, or ethically questionable.
Positive features of tasks on Prolific
Beyond the questions about task quality, 18 of 41 respondents mentioned something about the surveys on prolific being “interesting” in free comment areas about their reasons for working on the platform and what they like about it:
I enjoy the studies. I find them very interesting and it is a way of making extra money.
Others left notes about the educational value of the platform, for example:
The tests are meaningful, engaging and the results will go on to contribute to real issues.
The content of the work on Prolific — surveys in service of scientific research — was a significant positive feature of the platform for many workers.
How often is the work ...
... meaningful?
... interesting?
... fun?
... satisfying?
Negative features of tasks on Prolific
Because it is primarily an academic survey platform, we would expect the content of tasks on Prolific to be fairly above board. As is clear from the graph here, very few workers experienced work that was physically or psychologically harmful on the platform.
A small number of workers did report encountering work on a regular basis that was “ethically questionable,” but it is not clear from survey comments what kinds of tasks these were or why workers viewed them as somehow compromised.
How often is the work ...
... physically dangerous or harmful?
... demeaning or psychologically harmful?
... ethically questionable?
-
Technology
In general, the Prolific website seems to function pretty well for most workers.
Respondents were especially positive about Prolific’s user interface, describing it as “clean and well designed.” The most common issue raised by survey respondents concerned the website’s speed and responsiveness.
Percentage of respondents who said the technology is...
Multiple respondents noted that it was far more user friendly than other platforms with which they were familiar.
Some platforms are better than others when it comes to the specific surveys. The Prolific Academic platform is pretty user-friendly. I might make some changes in the list layout and usability of the records, but it is functional. Better than the others I have experience with.
Speed and responsiveness could be improved — or made more consitent and reliable — according to several workers. For example,
On the large majority of days, yes, it’s a relatively speedy website. However there are still fairly frequently days where it can take a long time for pages to load and possibly even time out.
Ocassionally, workers also experienced errors with the website and survey crashes, although not all surveys are hosted directly on Prolific, so sometimes these errors may be the fault of clients posting tasks on the platform.
Slow webpage loading speeds at times. Survey crashes or errors.
-
Things Workers Like
Interesting and Meaningful Work
Many respondents work on Prolific specifically because it is an academic survey site. Many respondents liked both the content of the work and liked feeling like they were making a meaningful contribution to society. Nearly half of respondents commented that Prolific was an “interesting” place to work. Other descriptors for the surveys on the site included things like “intriguging,” “engaging,” and “enjoyable.”
it’s the best paid survey website on the internet, also it helps the researchers and science in general , which makes it more good .
A simple way to make money at home, and often the studies are very interesting and it is fun to participate in some of them.
In contrast to other microtask sites, each survey on Prolific is unique. Some workers discussed this variety in the work as something which contributed to their enjoyment of the site:
Because it’s a nice side income and the work is relatively interesting, every task is different and I really enjoy that. It’s also nice to get a glimpse of some of the research going on, and thinking about what the researchers might be attempting to investigate. It reminds me of my days as an undergraduate.
Participants familiar with other online survey sites also drew a contrast between Prolific and these other sites, related not only to pay but also about the more interesting nature of Prolific surveys:
I searched for a lot of survey-based websites, but most if not all of them were working together with companies that sell general goods, foods, games, etc. So most of the studies were just boring stuff like “Would you buy product X?” “Would you rather buy product X or Y?” “Does anyone in your family buy product X?”. But with Prolific Academic the surveys are way more interesting because they don’t just want to sell you a product, but research human psychology and other interesting aspects about how humans live, react, decide, etc. I also can’t lie about the fact that Prolific Academic pays quite a lot more than your average survey website, so that’s a positive thing as well.
Prolific’s Attempt to Make a Better Platform for Workers
In addition to liking Prolific’s focus on academic and scientific studies, many workers also felt good about working on Prolific because the platform is trying to improve working conditions for workers:
It is an enjoyable to earn some extra and needed finances, most of the opportunities are both very engaging and frequently enlightening and I believe that supporting academic research is commendable, even if I do enjoy the compensation. There are also other platforms available for academic researchers where abuse of crowdsource workers is not only allowed, but almost condoned as acceptable. The amount of compensation is often deplorable and really should be addressed by some government agency. I support Prolific Academic’s attempt to bring fairness and transparency to the acquisition of data via the Internet.
The platform encourages a more fair working environment and shows respect for workers by providing a well-organized interface, good support, and handles payments promptly:
Because the site is well organized and they have great support and ofcourse they payout regularly.
In my experience, prolific does a great job in responding to queries and handling payments.
In addition to good communication between Prolific staff and workers, the site interface also does a good job at communicating details of pay, survey length, and expectations:
The site mandates a minimum wage to strive for. There are details on pay rate and length of the surveys with clear expectations set. The pay is good and the requesters post interesting studies.
Better pay also made workers feel more respected, and related to their positive feelings about the scientific work they were contributing to:
The studies are focused on science instead of buying patterns/ads that would allow corporations to develop better marketing strategies. That gives a feeling that the work we do will actually contribute to something meaningful to society. The pay is also incomparably better than most other survey sites which throw you pennies for filling out shady surveys and make you feel like a virtual laborer.
Flexibility
Like workers on other platforms, many respondents also liked working on Prolific because it allowed them the flexibility to work from home and at times that suited them.
This was beneficial for care givers and others with domestic responsibilities:
I can work from home which suits my domestic situation. I like that I can contribute to scientific research and it pays a fair amount for the work involved which is useful as a bit of extra income.
This flexibility was also important for people who cannot work in a traditional workplace with traditional schedules because of their health:
I am currently not able to work normal hours or a normal job, my health is up and down and so I use prolific academic both as additional income as well as to give me purpose and keep my mind active during my difficult time.
Prolific was not a primary income stream for any respondents. Although money earned through Prolific was an important or necessary component of most respondents’ budgets (61%), it figured into budgets as a supplement for people who were unable to make ends meet with a traditional job, including retirees on insufficient pensions:
I was introduced to the platform by my son, I use it so earn a little extra on the side as in the UK wages are not enough and with pensions going to be so low I will struggle to survive.
These workers liked that they could earn “a little extra on the side” without the formality and constraints of a second, or full-time, job.
-
Worker Concerns
Not enough work
The most common issue raised by survey respondents was the lack of tasks available on the platform.
Only one of 39 respondents reported typically working more than 10 hours a week on the platform (and this person worked only 12 hours). While many respondents on Mechanical Turk were able to create a full time job out of their work on the platform, Prolific could only function as a supplement to other income streams because there was not enough work available.
As one respondent summarized:
There is not enough work unfortunately, I would make prolific my primary site but it just does not have the volume to make more money.
No way to rate requesters
As noted by a few respondents, there is no way on Prolific to rate requesters or view information about other workers’ interactions with a particular requester.
Because many workers’ experiences with nonpayment seemed to be related to broken studies — rather than something wrong with Prolific itself — workers wanted the ability to rate requesters/researchers to keep track of who has a good track record:
This isn’t prolific’s fault, but it’s frustrating when studies screw up, they don’t work or you can’t submit them. Especially if the error only occurs after working on the study for a fair amount of time.
It might be nice to be able to rate researchers so you can see when someone has a history of broken studies.
Low Pay
Despite many workers appreciation for the high rates of pay they received for their work on Prolific, not all workers were making the promised £5 per hour. Furthermore, at the time of the survey, this promised wage equated to only €5.87 per hour, far less than, for example, a German minimum wage of €8.50/hour.
Some underpaid workers were still happy with their wages, especially those who lived in countries or areas with very low local wages:
It pays pretty well, better than minimum wage in this country, and also there is an estimate on how long the job should take you therefore you know when you’re almost finished.
Certainly, setting fair wages on international platforms will likely remain a challenge for platforms seeking to pay well across stark global economic differences.
Other workers found surveys that paid less than the promised wage insulting. It is not always possible to judge how well a survey will pay because surveys are paid by the task, and posted survey length information is always an estimate. Surveys vary greatly in both their length and their wage:
I don’t like that the pay seems to be so varied and often unfair. Although it’s my choice to do the work, I think it should be less sort of, almost like slave labour at time, to do so much effort for so little reward, it’s almost insulting.
Another respondent with more experience on the platform reported that wages have gone down more recently as the platform has grown, compounding issues they were having with the site running out of work for them to do:
Increasing amount of surveys are paying less than the time needed to complete them. There used to be a fair amount of high paying surveys overall.
Also don’t like how i’m receiving less surveys than ever before. I can sometimes go without a survey invitation for a week.
-
Ability to Refuse Payment Rejection:negative rating
Customers can refuse payment for any reason. -
Change to Terms of Service: Change:negative rating
Terms may be changed unilaterally at any time. Registered users are notified of changes. Continued use of the service indicates acceptance of changed terms. -
Warranty Warranty:negative rating
No option to revise rejected work. -
Contact with Employers Client Contact:positive rating
No prohibition on client contact. -
Contact with Workers Rejection:positive rating
No prohibition on contact to other workers.
-
Notes
- Oxford startup Prolific takes on Amazon.
- Prolific Academic on Founder.org.
- For example in this tweet.
- Press release: Oxford startup takes on Amazon, 2 Mar 2017 [PDF].
- Pricing – What researchers pay to use Prolific. Accessed 30 Apr 2017.
- Kristy Milland of worker forum Turker Nation has documented some of these: e.g., tasks paying £ 2.31/hour, £ 3.75/hour, £ 3.89/hour, and £ 1.38/hour.
- Oxford startup takes on Amazon, 2 Mar 2017 [PDF].
- Pricing – What researchers pay to use Prolific https://www.prolific.ac/about/pricing
