[LINK] Online multi-tasking test

[Roger Clarke]

At 17:38 +1000 15/8/11, Greg Taylor wrote:
>How good are you at multi-tasking?

How good are they at research design?

This kind of research is *really* hard to do in a worthwhile manner.

By 'a worthwhile manner', I mean other than as an exercise in 
gathering numbers to crunch, making statements that are limited to 
the form 'the people who participated said this, and did this, and 
the correspondence between what they did and what they said was 
this', and publishing papers for the sake of it.  (In academic 
jargon, the external validity is low to zero).

It isn't clear what the population is.  It isn't clear what the 
sampling frame is.  It appears to be a convenience sample, i.e. they 
have no pre-controls over who participates.  They can dream up 
post-controls (e.g. exclude some of the participants from the 
analysis, or pretend that they designed the research to address a 
particular sub-set of people and/or questions);  but that doesn't 
really help much.

And it appears to lack environmental controls, e.g. what's the 
context that the participant is in (office, lounge, bus-stop; 
peaceful, noisy, interruption-prone);  and - quite crucially - what's 
the extent to which the participant is also multi-tasking *outside 
the experimental set*?!  Their sole control appears to be to make the 
last few tests *really* hard to handle (presumably relative to 
existing data about human cognition and performance under stress). 
[Declaration:  I haven't done the test, so that remark is based on 
Greg's comment to that effect]

There's nothing about separation of the data from the identifier 
(viz. the participant's email-address, first name, gender, age in 
years and postcode).  There's no apparent need to keep the name and 
email-address with the data once the test has been completed;  and 
I'd have very much expected Ethics Committee processes to have 
required that they be separated at that stage.  (If not, then I'd be 
very leery about the UofQ Ethics Committee).

There are a couple of other turnoffs for the privacy-wary, which may 
reduce participation - and which would therefore create 
uncontrollable non-response bias:

-   the term "strictly confidential" is about as vacuous as you can 
get.  Is the activity subject to the Information Privacy Act 2009 
(Qld)?  Is it subject to a a specific Privacy Code?

-   there's no link on the sign-up page that enables the intending 
participant to make enquiries about the short, 
fixed-and-non-negotiable Terms

Added to that, 'Have Your Say' requires login.

Negotiating yet another identity would take more time than it's 
likely to be worth;  so I haven't provided these comments on the 
site.  They're open.

__________________________________________________________________________

>As part of National Science Week (which appears to last for a 
>month), a group of Australian scientists is conducting an
>online multi-tasking test, the results of which will eventually be published.
>
>It needs your full attention for about 25mins. So turn off the 
>phones, shut the office door, and give it a go. It's open
>to all ages.
>
>http://www.multitaskingtest.net.au/the-science/about
>The Multi-tasking Test is a Citizen Science & Research Project 
>undertaken by ABC Science in conjunction with The
>University of Queensland's School of Psychology, Queensland Brain 
>Institute, and Science of Learning Centre. It aims to
>find out whether there are any underlying cognitive factors that 
>make someone good at multi-tasking....
>
>The test is rather full-on in the later stages. Make sure you are 
>sharp and alert before starting it.
>
>
>Greg
>
>_______________________________________________
>Link mailing list
>Link at mailman.anu.edu.au
>http://mailman.anu.edu.au/mailman/listinfo/link

-- 
Roger Clarke                                 http://www.rogerclarke.com/
			            
Xamax Consultancy Pty Ltd      78 Sidaway St, Chapman ACT 2611 AUSTRALIA
                    Tel: +61 2 6288 1472, and 6288 6916
mailto:Roger.Clarke at xamax.com.au                http://www.xamax.com.au/

Visiting Professor in the Cyberspace Law & Policy Centre      Uni of NSW
Visiting Professor in Computer Science    Australian National University