Showing posts with label remote usability tests. Show all posts
Showing posts with label remote usability tests. Show all posts

2/22/2020

Userlytics (Online Platform, Paid Tests)



This is a review of my experience as tester.

Userlytics is a legit company offering UX tests to testers from around the world. In fact, this is the only site that got me a test on the same day I registered. 

 POSITIVE
Unlike most usability tests companies, they don't require an entry test. It is great for testers. If they perform well, no test was needed; if they don't perform well, they'll get low ratings and won't get many paid jobs. 

They don't ask for too many personal details when you join. That is, they don't profile you as hardly as other companies do. 

You can cancel your account any time on your own if you aren't happy with the opportunities they offer. 

They have a nice simple app recorder for Android. You don't even have to login, because, when you get an invitation, there is a unique code with your details that you can use instead; this makes the login into specific tests way faster.Their desktop recorder is easy to use, very intuitive, and also works by code.  

They do send you tests and pay you for them.

Customer support is existent and relatively fast. 

NOT SO GOOD
The fact that they don't make you do an entry test tells me that their panel of testers is way inferior to any other company's panel.

For the test to be considered payable, one has to produce a proper recording. That's totally fair. However, as they provide the recording programs, and they aren't very good, any malfunctioning due to their incompetence, will result in you spending your time in producing an unusable test that won't be paid.  
  
Both their Android and Desktop recorders are sub-par, full of bugs and a clear indication of the company's caliber. If an UX company does not have a reliable system to record their tests, why would anyone want to pay for their services? My experience is that their Android app is not reliable, so you might end wasting your time for no income, and in my case also conflicted with my phone, and that's never good. The desktop recorder is bulky and interferes with testing. In my case, it started uploading my video without me touching anything and while I still was recording the test. Fortunately, I found a way to stop it. Then, the file, about 15 minutes long, took a good amount of time to get uploaded.


Customer support is really idiotic and unnerving. They are unable to provide specific instructions to fix any issue you might have with the app. "Check your settings for accessibility" is not a reply to the question, "What exactly do you want me to check for this specific problem?" You know, testers have the basics under control, that's why they are testers. It is innate. You would not bother CS for basic stuff on your phone.

You cannot update your PayPal email yourself, you have to contact CS. However, since they are quite incompetent, you will find problems to do something as simple as that. In addition, sharing personal info via email is a no-no for security reasons.  I had to exchange a considerable amount of emails for them to do that would not do it because I was writing from another email, the one I wanted to link to my new Paypal. I was so unnerved that I decided to erase my account without being able to change anything. They could not find me in the system, even though I provided them with a valid email account.  

IN SHORT
The company line is "The most advanced user testing platform". Having been a tester for them, and having experienced many other UX testing companies and platforms, the tagline is just bragging not something that reflects what I have experienced. I do tests for many online companies and this is perhaps the less advanced, the buggiest. 

They are a legit company, so I don't think you have anything to fear, if you get a test and do it, and their recorder works, you will be paid. However, there are greener pastures out there. Just saying. 
 
REPLY FROM THE COMPANY ON 9/1/24
Thank you for sharing your comprehensive feedback about your experience as a tester with Userlytics. We appreciate your positive remarks regarding the ease of entry, app recorder functionality, and prompt customer support. However, we understand your concerns and frustrations with the number of available tests, the quality of recording programs, and the challenges with customer support.

Your insights are crucial for our continuous improvement, and we're actively addressing the issues you've highlighted. We apologize for any inconvenience caused and value your continued support as we work to enhance our platform.

Best regards,
Userlytics

12/19/2019

Validately (online platform, Paid tests)




This is a review of my experience as tester.

They are a legit company and pay $10 USA per test. 

Like many other UX companies, you have to pass an entry test, not paid, to be admitted. Once you are, you fill-in some general info about your demographics and wait for the tests to be thrown at you. 

You are informed of any test by email, not on the site. That is, there is no member area or dashboard for testers on their website where to find the tests.  You just get the link if something matches your demographic. Click on the test link and follow the prompts. To do the test on the desktop you have to use Chrome browser and install their recording extension; on the phone, you have to download their app from Google Play. 

Although they do send tests often, they don't seem to have the volume of tests that other UX companies have, so testers are like hungry zombies at the smell of a healthy human. Once the email arrives, even if it takes you three seconds to open it, the page is already locked and shows a message saying that there are too many testers trying to access the test at the same time, try again. You keep trying, by reloading the page, to not avail. Then, the test is over and you got nothing. Or simply, you get there and, in the unlikely case no one is fighting for the test, you get through the screener and this one rejects you. All of these situations are very frustrating. 


The phone app is really nice and user friendly, definitely the best I've tried from different companies. However, it could be better designed. If, by mistake, you press step two without having finished step one, there is no arrow or way of going back, not even by using your phone back arrow option.That's bad design.

Once you perform the test and it's uploaded onto the cloud, you'll receive an email saying that they've got it, that they'll  review it within seven days and, if accepted, they'll pay you in the following seven days via PayPal.

It takes about 7 days for the test to be reviewed, and if the rating is acceptable they immediately pay you by PayPal, which is really nice.

Unless you fit their preferred demographics, you won't earn much there. That's at least my experience. So there are greener pastures out there; this is a sort of small shed you keep just in case it rains something, if you know what I mean.

Still, they are reliable and have a good deal of business.

12/06/2019

User Testing (Online Platform, Paid Tests)




I've tried many user testing companies and this has been the best for me so far. I've made some bucks on this site, and they have been paid, as promised, within a week of performing the tests. This company is the only one that, at least for my demographic, provides me with tests mostly every day. 

Some major business and companies use this site, and not the others, and I think they are already established and well organised, so one can see how things flow here and not elsewhere, at least for me.
 
HOW TO BECOME A TESTER

Like most user testing companies, you have to perform an entry test, which isn't paid, to see if you qualify. What they ask you to do is similar to what you would be doing as remote website tester. 

Once you are accepted, you will be given access to your member dashboard and see a list of tests offered to you, from which you can choose from. There is a constant stream of tests, and it is up to you if you try any or all of them.  

HOW IT WORKS

The first paid test you do is examined by the UT team. They will give you the initial rating. It takes them between 6 and 9 days to do that, but they do it; not only that, they even comment on your performance, and things needing of improvement in the future. Getting a high rating from them is very important because, the higher your rating is, the more in demand you are, and the more tests you are offered to take part in.  You will be rated for each of your tests, but after the first one, the ratings will come from the companies that paid for them.
 
Like similar sites, customers are companies with specific requirements. They won't tell you what they want directly,  but will screen you to see if you are what they are looking for. They do so by asking you a series of questions with answers to choose from. You need to answer honestly or might find yourself in trouble. If you qualify you can take the test. The screening takes just a few seconds to perform.

Before you enter the test, your task is mentioned. The system gives you 15 minutes to start the test. That's great if you have your kids around, you are cooking and need to put things aside, or have to go to the loo. Also great in case the recording program is misbehaving and you have to reinstall it. You have to allow their desktop app (actually a Chrome extension) to run, that is, to record your micro sound and share your screen. Once you do that, you press enter and the test starts. You have to read the instructions out loud, and verbalise your thoughts on the tasks you are doing, and do that continuously.

In the member area settings you can decide whether you want to receive emails for incoming tests, or just hear the ding when you are logged in. I think the latter is best, because, as I've said, there are plenty of tests coming in at times and you don't want to flood your inbox with emails. To hear the beep, needless to say, you have to have a tab open, logged into your account and the dashboard "Take Tests' open. 

DOWNSIDES
>> The ratings you get from the companies aren't always fair. They might not like your accent, that you took a bit less or more time from what they wanted you to, or simply that you did your job as tester and told them, politely, that their website sucks.  
 >> The extension/program they use on the Chrome desktop is not always reliable. I lost plenty of tests for which had qualified because the recorder was malfunctioning. They always tell you to check this and that, but most testers have already done that when complaining to CS; they are testers after all! When it works, it is great, but when it does not, it makes testers lose money and UT customers get annoying silent recordings and waste their time. Just fix the recording or just provide with an alternative.  
>> You might have problems with your microphone or the app during the session, and if the recording cannot be heard or has poor quality, your test will be rejected and you won't be paid. Or they will give you the lowest rating, which directly damages your prospectives of getting good tests offered to you. I mean, why not allowing the tester to repeat the test for free? 
>> You don't qualify for 90% of the tests coming at you, as they are targeted to specific industries and consumers. They might want a person who is bilingual in Korean. A person who uses a certain airline, bank, supermarket or store. A person who doesn't know anything about a certain product, or a person who has kids, just to mention a few examples. This varies from test to test. You have to take the screener and be honest because, otherwise, you will be in trouble and won't be paid. 

>> I have the strong impression that many of those screeners are archived and added to your profile, so they profile you to the core and quite accurately.  I am never happy with that, especially because you aren't really working for the company, you are working for rewards not for a salary. Yet, they will have so many details about you, besides your voice. If something goes wrong, the site is hacked for example, your whole life info could be used for fraudulent purposes. I am not sure if their security system is really good, either.

>> Some tests have between 12 or 60 steps and, although enjoyable, they are time consuming; you might spend 30 to 60 minutes on a test and still be paid 10 bucks. That is unacceptable. I dont blame the site for allowing that to happen. I also blame those customers from big companies and corporations that run on the cheap when it comes to paying, and expect a tester to go through their demands for alms. Dear UX designer if you test is 50+ steps is because you don't know how to design an US remote usability test, as simple as that.

>>  Unless you live in a country where 10$ USA is a lot of money, that's a very low per-hour fee. At the moment, the duration of the test is not mentioned, so it could take you 10 minutes, which is the best case scenario, or 60+.

>> They mostly work on Chrome, so you can't perform the tests on any other browser unless the test is specific for that browser.

THEIR PHONE APP
>> I installed their phone app from Google Play, but it required me to have my phone on developer settings, which I allowed it to. Then my phone itself started to request me to switch this mode off or the phone being at risk of crashing or being hacked.
>> In addition, the app didn't allow me to log in, rejecting my login email as incorrect, which is totally bananas! 
>> I removed the app and I don't do tests on the phone, which is a pity, because there are tons of phone tests coming in.  
>> There are simple apps out there that capture sound and screen and aren't invasive at all. I actually tested one of them for another company, and it was fantastic. Why not using that instead? 
 
CUSTOMER SERVICE
CS is useless. It takes ages for you to get a reply, and although they reply, eventually, it is first with a Zen desk robotic message that shows that the officer hasn't really read your query and acted on an impulse by reading the subject. When you complain about some of the companies testing requiring a huge amount of time for little money and for not explicitly saying that at the beginning, they blame you for staying in the test; but for some people this income is necessary and they put up with bad behaviour out of desperately needing extra cash. I had CS officers who needed four of my messages being as blunt as it gets for them to actually read my emails and respond to what I was asking not what they wanted to reply to. It is not acceptable that a company that trades in the stock market has such poor service for those who make possible the site, which are the testers. The worst part is that customer service has grown worse at the same pace as the company is growing up and cashing in.

 

12/03/2019

Enrollapp (Online Platform, Paid tests)


They were previously called Zurb. This is a legit company, it is just that, while they wait for you to receive a test, if any, they will profile very accurately, with your help. Genius! 

Easy to join. Very fun and likeable website, with that lovely cow walking while you reply to the site queres. 

They start profiling you, hard core, from the beginning, in ways that I don't think I would like to be profiled or I am comfortable with. Of course, they might come and say, it is all voluntarily, you do it because you want (no, I need money, mind you, that's why I am here). Supposedly, they are getting your info to provide you with the right test or survey. Questions are very specific about you and your finances. 

I suspect that they are data collectors, a la Google, just done in a more fun way and having you replying directly to the questions, no machine involved. I know for sure that they don't need of all that info for you to take a test. Their privacy policy also seems legit, but if you read between the lines, you will see that very personal information is collected through the tests themselves, and also that they can share your data, just removing your name and personal details. It smells fishy. My impression.

They might provide legit work, but that sort of profiling shouldn't be allowed or legal!  So, needless to say, I left soon after joining.  I might be totally mistaken, for sure, but the questions about my bank account are too precise to be random or innocent.

 Avoid.

TrymyUI (Online Platform, Paid Tests)


To be accepted you have to perform a free test, which will be examined. If you are considered fit, you will be part of their panel of remote testers. You need to download a program that will allow them to record your screen and voice and tests will be done in real time. Tests are paid well and you won't spend much time on each. 

They are the most didactic of the bunch. Before you start your acceptance test, they explain to you how the system work. You have to download a mini program, very easy to use, that will do the recording and uploading, so they take the time to explain to you how it works. None of the other companies bothers, so that's really cool and considerate.

These is a legit company, but if you are remote tester, might not find any source of income here.

They do email tests according to your demographic, but there are so many people waiting for them, that you'll be lucky if you get any.

It happens to me, all the time: I takes me barely three seconds to open the browser with the emailed link and I get the message that someone is already taking it, or the test is no longer available. So I guess the other person received the mail before I did.

It might be my demographic, but the only test I did, for free, was the entry test. I've been there for some weeks and I was able to open a test that wanted a question answered, and after doing so, it said that it didn't fit the profile they are looking for.

So, money earned, zero.

They work this way: There are a bunch of hungry dogs in a pit separated by fences. The bosses throw a steak and decide that the white dogs are the ones will better benefit from it, so they do so, and the bunch of white dogs fight for the steak. The one who gets it, gets fed, the rest of white dogs, black dogs and brown dogs will get nothing. Would you want to be there or move to a place where there are more steaks and the dogs are not in a pit? Your choice.

It might work for you, if you are student, or have children, or who knows what is the most popular demographic. Otherwise, you will be receiving emails that never lead to anything. 

I quit.