Usertesting.com's main page header

Is Usertesting the next Amazon Mturk?

Recently, a Harvard fellow found in a paper published by Cornell’s Industrial & Labor Relations Review that “modern reliance by app-based companies on independent contractors is a new application of an aggressive labor relations tactic, designed to strip away worker rights.” 

Similar to how the construction industry got away without having to provide their laborers with basic benefits such as insurance or worker’s compensation, gig economy companies like Uber, Lyft, and Doordash have adopted a similar strategy to much success.

I think it’s safe to say that we can add Usertesting.com to their ranks. I believe that the upper echelon of the Usertesting team would secretly celebrate this fact.

USER EXPEIRENCE

Their platform revolves around UX, user experience. According to wikipedia, “User experience is a person’s emotions and attitudes about using a particular product, system or service. It includes the practical, experiential, affective, meaningful, and valuable aspects of human–computer interaction and product ownership.” 

There are plenty of ads and articles out there that detail how the platform can help companies that use the service improve UX for their business, and even some that are critical of the process used by these companies. There are not, on the other hand, many resources for the testers on usertesting.com to vent their frustrations over what I observed as a change in priority that left the testers holding the shit end of the stick.

With an unprecedented number of companies going out of business and a rising unemployment rate, people searching for work rely on websites like this one to provide extra cash. The little bit of extra income they provide can relieve financial pressure for individuals who’ve lost their regular incomes or seen them diminished. 

The testers are held to a rigorous standard, told that the tests take an average of 10-15 minutes and harshly penalized for anything that can be deemed as out of line with usertesting.com “Terms of Service”. 

Many individuals who use the site for extra cash can attest that since the global pandemic began, the tests offered on the site have gotten noticeably worse, day over day, month by month.

This is a troubling trend to those who depend on these types of sites for extra money to get by, but also paints a very recognizable picture of a powerful company devaluing their workers’ labor, silencing dissent, and performatively throwing their hands up in their air as if they have no ability to control what’s happening. 

There’s a case to be made that not only are usertesting.com and their clients leveraging economic disparity, but that the results they get aren’t genuine, due to the fact that the subjects aren’t volunteers. It’s possible that they’re manipulating their testers into working harder for less money, and the whole affair could be futile if improperly utilized. 

TERMS OF SERVICE

I’ve noticed longer tests that often violate the Usertesting TOS, though I haven’t gone so far as to document that with proof. I choose to maintain Usertesting’s intellectual property by not duplicating their work here, but many tests ask for personally identifying info (a TOS no-no). 

According to cursory research, poor customer service is common for testers while excellent customer service can be expected for UX designers and programmers. It’s obvious that there’s a double standard that favors the companies over testers.

I can point to the endless stream of TOS violations that could be easily policed with minimal effort if not for want of sheer profit. Sounds melodramatic, and where as being ripped off once is one thing, having a company allow you to be ripped off a million times means another thing altogether. This purposeful, looking the other way at the testers expense thing can be reigned in fairly easily too.

Testers on Userstesting.com often complain on various platforms, as evidenced herein. A majority of complaints seem to revolve around unfair practices on scoring, customer service (or lack thereof), and TOS issues.

I’ve collected a tons of negative reviews stories about Usertesting on glassdoor, reddit, twitter, facebook, et al, but I’m not using that material here or hyper linking every example to build this ridiculous interweb of pseudo-evidence that doubles for good SEO.

Instead of building a case upon a series of negative anecdotes I just wanted to lay out my understanding of the situation in simple terms. I think people are smart enough to be able to draw their own conclusions.

Is Usertesting suspect?

Add to that, on Twitter, when you search usertesting, the search results are awash with positive articles and reviews from the upper echelon of usertesting.com staff flooding the social media outlet with good news. The ensuing flood of encouragement from biased individuals seem to bury negative tweets from the testers themselves who want to be heard on social media. This could be conjecture, I admit, but I’m trying to be open-minded.

There’s a similar tactic employed by Amazon, another shady employer, like Usertesting before them, with highly questionable ethics. Some businesses thrive because they give their users a voice to air out their complaints. Others, like Amazon and the digital equivalent of skid row that is Amazon Mturk, thrive because they allow second rate companies to use desperate people as chattel.

As is, a company performing a test on this site can secretly give a tester a score of “1” as a rating without the tester being able to even see that they were scored that way, with no explanation as to why or any ability to challenge the assertion. They’re known as secret ratings.

Secret ratings effect a tester’s access to future tests, directly impacting their earning power. They have neither the knowledge they were scored poorly nor any recourse to challenge the assertion from some powerful company that prefers to remain nameless at the individual testers expense. Seems a bit of a ridiculously harsh policy, no?

One major problem I have with UX designers is that they act like testers don’t have basic knowledge of web design. I think most people know that it’s ridiculously easy to monitor how long an individual is visiting a website, what they look at, how they engage, their metadata. 

From my own experience, testers are led to believe that Usertesting itself, a UX design testing company, is incapable of easily monitoring and fixing this issue when it comes to test times and tests that go outside the TOS. Is this company so innocent, naive, and technologically inept as to be completely disabled when it comes to how long their tests run? 

 

SUGGESTED SOLUTIONS

Here’s a suggestion that provides a good solution. If a test goes over 15 minutes, the app or website company running the test should forfeit their right to be able to rate the tester and the tester should get an automatic five stars. Our time is valuable. Don’t devalue our labor. 

Another easily-implemented solution would be to make it transparent and easy for testers themselves to respond to poor reviews or challenge them. Provide a button that is as easily-accessible as any one of the menu items on the dashboard for testers, instead of having to dig through the help center for a way to write a ticket, like Indiana Jones excavating the booby-trapped ruins of a Mayan temple, would also help a great deal.

Add to that, in the completed tests section, where Usertesters are reviewed, the UX designers who create the tests can write detailed reviews of the testers, as they see it. Make it possible for the testers to respond directly to these critiques, or challenge the response and escalate to a higher authority. 

If the website was ironclad in the standards for tests, if they were all 10-15 minutes, if the screeners were appropriately descriptive in seeking out testers, and if their apps and websites were all actually ready to be properly tested, I could understand having that policy, but such is not the case. 

NOT TO BE TAKEN SERIOUSLY

These are apps and websites that are specifically being tested to work better, so assume they’re in a state of disarray. As much as you want to believe that these companies are all professional, a lot of them are really bad at testing people and should be questioned logically and reasonably about simple tasks and issues.

What do TOS really mean but only as a potential method to penalize and punish testers/workers? These tests run longer than 15 minutes, ask for personally identifiable information from testers, and some ask you to spend money on a test without recompense, all of which take place daily on usertesting.com’s platform. At times, it’s downright predatory.

It’s not as if I’m writing this with only complaints. The solutions I suggest are easily implementable and would make the platform fairer for workers, and more democratic. These are solutions which can concretize usertesting.com as not just the best looking online user testing platform, but the best behaved and most human.

I doubt this will change anything. I don’t care if it does or not, because it won’t solve my personal problems and won’t fix the planet or the economy. A powerful company gained power due to widespread financial distress, and allowed the standards for the workers to remain strict and steadfast while the working conditions themselves slowly deteriorated into the next Amazon mturk.

They’re gonna ride this train all the way down, til it rolls right off the tracks.

 

Leave a Comment

Your email address will not be published. Required fields are marked *