Platform Review Information
The background information presented on this site was gathered by researchers working for IG Metall (the German Metalworkers’ Union), Encountering Tech, and M&L Communication Marketing GmbH between October 2016 and March 2017.
Citations to publicly available sources are provided in the text where possible.
Worker Reviews
The worker reviews presented on this site were gathered from a survey conducted by IG Metall and Encountering Tech between December 2016 and March 2017.
A total of 256 verified workers on 7 different platforms completed the full survey.
The 95-question survey covered 8 different topic areas:
- Basic demographics (e.g., age, location, gender)
- General experiences as a platform worker, including hours worked per week, and history of crowdwork
- Pay and nonpayment
- Communication with clients, platform operators, and other workers
- Reviews, ratings, and evaluation of workers and clients
- Experiences with platform technology (e.g., ease of use and reliability of app or website)
- Quality, character, and availability of tasks
- General likes and dislikes with respect to their work
Pay data was collected in the survey respondent’s choice of Euros, US Dollars, or GB Pounds. All pay information was converted into Euros using the average exchange rate for the day that the survey was taken as published on http://xe.com
Hourly wage data was calculated by dividing workers’ reported most recent week’s total income by their most recent week’s total hours worked. Wage data was then trimmed at 1% and 99% of the overall distribution to eliminate extreme outliers before further calculations or graphs were made.
Star ratings are mathematical extrapolations of workers’ responses to multiple survey questions in each rating area.
For the pay rating, a star rating was given based simply on the median hourly wage reported by surveyed workers:
| Less than €2.50 | 0.5 stars |
| €2.50 – €4.99 | 1.0 stars |
| €5.00 – €7.49 | 1.5 stars |
| €7.50 – €9.99 | 2.0 stars |
| €10.00 – €12.49 | 2.5 stars |
| €12.50 – €14.99 | 3.0 stars |
| €15.00 – €17.49 | 3.5 stars |
| €17.50 – €19.99 | 4.0 stars |
| €20.00 – €22.49 | 4.5 stars |
| €22.50 or more | 5.0 stars |
The other star rating calculations are generally a simple average of participants’ responses to multiple survey questions in each topic area. These questions were generally 5-point Likert-type questions, and ratings were calculated by giving 1 point for the most negative choice, 5 points for the most positive choice, and averaging across all participants and across all questions in that subtopic area. Averages were rounded to the nearest half-star.
The overall star rating is based on the 5 sub-ratings, derived via this formula, which intentionally weights pay more heavily than the 4 other sub-ratings:
Overall Rating = ((2 x Pay) + Communication + Evaluation + Technology + Tasks)) / 6
Again, the result was rounded to the nearest half-star.
The decision to weight pay twice as heavily as the other components of the overall rating is based on the responses to IG Metall’s 2016 Crowd Worker Survey. In the 2016 survey, over 600 German crowd workers were asked to rank the importance of various criteria for satisfactory working conditions on crowd work platforms. Out of ten criteria, fair payment was by far the most important for respondents.