What still works in Qualtrics free accounts (3/20/2016)

In the last few months Qualtrics has begun handicapping its free accounts. For a couple of years they were the best deal in online research: the Qualtrics system, very few restrictions, and a price tag of $0.  Of course I should have known that the days of wine and data wouldn’t last forever. They appear to be over.

So, to save other researchers a little time (assuming this post shows up in Google), here is what my students and I have currently experienced. Of course, our experiences might not be representative, and things might change overnight (as they seem to have done recently).

Restrictions on Qualtrics “free accounts” (or “trial accounts”) AFAIK as of March 20, 2016:

  • No Downloading Data. This is the absolutely most important thing: newly-created accounts, starting a relatively short time ago (a couple of months?) are do  not allow download. You can look at your data only through Qualtrics’ “reports” or summary stats on a per-variable basis. Or that’s what I think the situation is. But no downloading means no correlations, no t-tests, no regression analysis, no ANOVA, no trend analysis, etc.
  • No Importing Survey Files. So if you have a survey file from a colleague, or a previous account, you can’t import it into Qualtrics, unless you have a paid account. That kinda bites.
  • No Viewing, Creating, or Managing Panels Linked to Collaborated Surveys. I am a “collaborator” (with my free account) on a multi-survey project housed in a colleague’s full account. The survey uses panels and validation. I apparently have no access to that.
  • Only one survey can be active at a time. But that was always a free account limitation; no change.
  • No Panel Data. Again, this was always a free account limitation, AFAICR.

As of now, I don’t know of any limitations on number of responses, number of surveys you can create in your account (as long as only one is active), etc.

It’s a little painful to me to see Qualtrics going in this direction, though it’s predictable and probably very good for their bottom line. Nearly a decade ago I was evangelizing for Qualtrics at my former university, singing the company’s praises, including the reasonable pricing for academics. Now, the “special plan for academics” is 1000 responses for $500. That probably doesn’t even register on some people’s radar. For me, however, doing research at a university with fairly little research support, it’s painful. We usually get about $600 per year for professional conferences–which, of course, covers about 1/3 of the costs of a national conference. Universities all over the US are in varying stages of budget crisis, so this situation is pretty common.

I think Qualtrics has realized it’s the dominant product right now, and has decided to charge what it can, because it can. Sadly, this leaves some academics out in the cold.

Graphs with multiple variables

I’ve been trying to get some graphs prepared from the Knowledge & Attitudes data from Fall 2012 (K&A 2012). One of our chunks of data was a series of questions given to survey respondents about their views of the immorality and probable harm caused by various sexual situations (I call these the scenarios): each scenario specified an initiator, a recipient (for lack of a better term), and the age (15 or 21) and sex of each. The initiator was described as “starting a sexual relationship” with the recipient. They always engaged in the same activities: kissing and touching each other’s genitals. There are a lot of variables. Here’s one way to look at them:

  • Initiator sex
  • Initiator age (2 levels: 15 or 21 years old)
  • Recipient sex
  • Recipient age (15 or 21)
  • Immorality rating (DV)
  • Harm rating (DV)

To make things slightly more complex, I simplified the survey forms by cutting out some of the potential crossings of the variables: initiator and perpetrator sex are fully crossed (M/M, M/F, F/M, and F/F conditions) but the ages aren’t–15-year-old initiators are always paired with 21-year-old recipients, and vice-versa. For analysis (but possibly not graphing) purposes it’s also important to know that some comparisons happened between subjects (there were two more-or-less randomly-assigned forms) and others within subjects, with the order of presentation (sadly) fixed. In the future I may do this with more rigorous crossing of all the variables.

I could present subsets of variables in graphs, but I’m really interested in getting as many (independent) variables as possible represented in a single chart, not least because I expect higher-order interactions, and only showing a few variables might obscure those or even mislead the viewer. Here’s an initial stab with only some of the variables:

s.h.01 Continue reading Graphs with multiple variables

other people’s research (opr)

International researchers have found negative, long-lasting effects of early TV viewing on childhood development. I suppose I’ll have to find actual babysitters or something. On the plus side, I now have one more excuse for all my own personality flaws.

Along those lines, the May issue of the Archives of Pediatric and Adolescent Medicine has several research reports explaining how all sorts of factors during the first five years of life can have important long-term effects on development.

Shockingly, clinical trials can be biased in favor of the pharmaceutical company that is so generously paying for the study. That seems crazy, I know. Next, they’ll tell us that receiving money from lobbyists influences congressional votes, or that getting a paycheck influences whether we go to work.

And, finally, the Supreme Court is going to hear a case about violent video games. The research has been quite contentious, and there’s a lot of money at stake. I’m not as concerned about the outcome; I’m more interested to see what the Court does with the science. Courts have a history of mangling that stuff beyond belief.