Acceptability,usefulness,and satisfaction with a web-based video-tailored physical activity intervention:The TaylorActive randomized controlled trial

Stphni Schopp *,Mitch J.Duncn ,Ronl C.Plotniko ,W.Krry Mummry ,Amn Rbr Stphni Ally Quyn To Cmill E.Short ,Cornl Vnlnott

a Central Queensl and University,School of Health,Medical and Applied Sciences,Appleton Institute,Physical Activity Research Group,Rockhampton,QLD 4702,Australia

b The University of Newcastle,College of Health,Medicine,and Wellbeing;School of Medicine&Public Health,Newcastle,NSW 2308,Australia

c The University of Newcastle,Priority Research Centre for Physical Activity and Nutrition,Newcastle,NSW 2308,Australia

d The University of Newcastle,Priority Research Centre for Physical Activity and Nutrition,College of Human and Social Futures,Newcastle,NSW 2308,Australia

e The University of Alberta,Faculty of Kinesiology,Sport and Recreation,Edmonton,AB T6G 2R3,Canada

f The University of Melbourne,Faculty of Medicine,Dentistry and Health Science,Melbourne School of Psychological Sciences,Melbourne,VIC 3010,Australia

Abstract

Keywords:Online;Physical activity;Process evaluation;Program;Video-based

1.Introduction

In adults,150 min of moderate intensity physical activity(PA)per week can significantly reduce the risks of developing cardiovascular disease,type 2 diabetes,cancer,depression,or dementia,and increase life expectancy.1-3Despite these significant health benefits,over half of the Australian adult population(55%)is not meeting national PA guidelines.4,5To reduce the burden of chronic disease and health care costs,effective and scalable populationbased interventions that can reach large numbers of people are needed.6

Web-based interventions that provide personally tailored PA advice(also known as computer-tailored interventions)have demonstrated effectiveness for increasing PA levels compared to interventions offering generic or targeted information.7The personally relevant PA advice is generated through the use of brief online surveys,a message database containing all possible response options,and IF-THEN algorithms that determine what advice is provided based on the participants’answers to online surveys.8Compared with generic messages,tailored messages are more likely to be read,remembered,saved,and discussed with others.9Content in web-based,computer-tailored interventions is typically delivered as text-based information.Yet,eye-tracking studies have shown that Internet-based reading predominantly happens by means of browsing,scanning,keyword spotting,and non-linear reading,with less time spent on in-depth and concentrated reading.10,11This means that text-based information may not be effectively disseminated using the Internet.12,13Consequently,computer-tailored intervention content may not be as effective as it could be when the personalized advice is not read,processed,and actioned as intended.

This obstacle may be overcome by delivering web-based PA advice through personally tailored videos.Watching web-based video content is increasingly popular,and videos may be more effective as they reduce the cognitive effort needed to process information,which can lead to better attention and comprehension.14Furthermore,computer-tailored interventions using personally tailored videos in other fields(i.e.,diet,weight gain prevention,smoking cessation)have demonstrated effectiveness.15-17This was further supported by our own research in the development of personalized videos,18which demonstrated that they are feasible to use and implement,18that they lead to more attention and recall(using eye tracking),11and that they are effective in increasing PA.19

To examine the effectiveness of a web-based video-tailored PA intervention in adults,we also conducted a large,fully powered,3-group randomized controlled trial(i.e.,the Taylor-Active trial).20The trial examined the long-term efficacy of a web-based PA intervention that provided personally tailored videos and compared it to a traditional personally tailored text condition and a control condition receiving generic information.20However,the findings,assessed using ActiGraph accelerometers(Version 3.2.1;ActiGraph GT3X+,Pensacola,FL,USA),showed the web-based,video-tailored TaylorActive intervention was ineffective at increasing PA behavior.21Given the encouraging findings from the formative research11,18and pilot study,19this outcome was surprising.

An important next step is to conduct an in-depth process evaluation to better understand why the TaylorActive intervention was ineffective at increasing PA and inform the development of future video-based computer-tailored interventions.To advance the field,the publication of research examining interventions that are ineffective is equally as important as the publication of research examining interventions that are effective.Associated with this,and equally important,is finding out why some interventions work or and others do not.Therefore,the aim of this study was to examine the usage,acceptability,usability,perceived usefulness,and satisfaction with a webbased video-tailored PA intervention for Australian adults(i.e.,the TaylorActive intervention).

2.Methods

2.1.Study design

The TaylorActive intervention was a 3-group randomized controlled trial conducted at Central Queensland University in Rockhampton,Australia.Participants were randomized into 3 groups:video-tailored intervention,text-tailored intervention,and control group.Ethical approval for the study was received from the Central Queensland University Human Ethics Committee in August 2014(H14/07-163).Written informed participant consent was obtained online.Trial assessments were conducted at baseline,3 months,and 9 months.However,data collected in relation to the process evaluation presented in this paper were collected at the 3-month time point.

2.2.Participants

Eligible participants were aged 18+years,had broadband Internet access,could speak and read English,were living in Australia,were insufficiently physically active(i.e.,≤150 min of moderate-to-vigorous PA(MVPA)per week),answered no to all questions on the Physical Activity Readiness Questionnaire22or obtained medical clearance,were not pregnant,had a body mass index over 17.5,and reported no impairments that could prevent them from becoming more active.Participants were recruited through social media advertisements(i.e.,Facebook),traditional media(e.g.,radio,The Conversation),email(e.g.,Central Queensland University staff),and third-party databases(i.e.,www.trialfacts.com).Interested individuals were directed to a recruitment webpage that contained detailed study information and asked them to complete a screening survey to determine their study eligibility.

2.3.The TaylorActive intervention

The TaylorActive intervention was described in detail elsewhere.20,21Briefly,TaylorActive was a web-based,personally tailored PA intervention.The intervention aimed to increase various domains of PA(e.g.,leisure,active travel,house/garden work,occupation).A text-based library,available to all groups,contained 19 articles about different aspects of PA(e.g.,“Why be active”,“Get started walking”,“Make time to be active”).The text-and video-tailored intervention groups received 8 sessions of tailored PA advice,which were generated in response to questions about participants’current PA,as well as questions relating to evidence-based individual,social,and environmental determinants of PA.IF-THEN algorithms were applied to select personally relevant advice from a comprehensive message database.The intervention content was informed by established health behavior theories including Self-Determination Theory,23Social Cognitive Theory,24and Theory of Planned Behavior.25The main focus was on behavior change through enhancement of self-efficacy,intentions,social support,knowledge,outcome expectancies,attitudes,facilitators,risk perception,intrinsic and extrinsic motivation,need for relatedness,peripheral and central cues,and habits.Furthermore,the following evidence-based behavior change techniques26were applied to change theoretical constructs:providing feedback,self-monitoring,goal setting,habit formation,instruction,problem solving,and action planning.PA advice and goals were tailored to participants’main motivation to increase activity levels as expressed in the baseline survey:(1)improve health,(2)increase fitness,(3)increase strength,(4)lose weight,and(5)reduce stress.The 8 sessions with tailored PA advice were delivered in a set order at a set time.New sessions could only be accessed when previous sessions were completed.Regarding the intervention sessions,it should be noted that the TaylorActive website was designed to be used infrequently(i.e.,only when a new intervention session was available,which was weekly for the first 4 weeks and then every 2 weeks for the next 2 months).Up to 3 email reminders were sent when participants did not access new sessions.A website feature for creating PA action plans was also available for both intervention groups.27At the end of each session(except for the first and last)participants were asked to set long-,medium-,and short-term PA goals and to create an action plan for how they would meet their PA goals.20For example,participants were asked very specific questions about how they planned to meet their PA goals:What PA they will do,where they will do it,when they will do it,how often they will do it,how long will each activity session be,and with whom they will do it.20When participants had completed all the questions,the TaylorActive website provided an action plan that could be printed on a single page.

2.3.1.Video-tailored vs.text-tailored interventions

The difference between the 2 intervention groups was how the personally tailored feedback was delivered.In the text-tailored intervention group,feedback was displayed as plain text on the TaylorActive website and supplemented with graphs indicating progress where relevant.In the video-tailored intervention group,feedback was presented as a video for each session with a male or female actor(generically named“Taylor”in either case)that could be selected by the participants.The content of text-and video-tailoring was identical,addressed participants by their first name and was kept brief,as the formative research for this study showed that the videos should be short(range:4-7 min)to prevent disengagement.18Information that could be easily provided in the text-tailored feedback(e.g.,name,body mass index,minutes of MVPA,graphs showing progress over time)but could not be pre-recorded into the videos was presented as text layered on top of the video(i.e.,an“overlay”)to make the videos as tailored as possible.

2.3.2.Control group

The control group received access to a limited version of the TaylorActive website that only included the text-based library with generic PA information.The control group had no access to other website components and was regarded as“usual care”condition,as generic PA information is freely available on the Internet.28

2.4.Procedures

Research officers verified participant eligibility and contact details following participants’completion of the screening survey.Subsequently,participants completed a 30-min baseline survey administered through Computer-Assisted Telephone Interviewing(CATI),a telephone surveying technique in which the interviewer follows a script provided by a software application.The CATI surveys were conducted by trained and blinded interviewers from Central Queensland University’s Population Research Laboratory,Rockhampton,Australia.The CATI surveys were repeated at 3-and 9-month assessments.Once baseline data were obtained,participants were randomized using a randomly generated sequence via www.randomization.com.There was no face-to-face contact with participants at any time during the randomized controlled trial.

2.5.Measures

Intervention process data were assessed using the post intervention(3 months)CATI survey measures and Google Analytics.This paper presents intervention process data on intervention usage,acceptability,usability,perceived usefulness,and satisfaction.Socio-demographic characteristics of participants measured in the baseline survey included sex,age,education(in years),relationship status(in a relationship or not in a relationship),urbanization(major city,regional city,or remote or very remote area),occupational category(professional,white collar,or blue collar),29employment status(fulltime;part-time/casual;or other:unemployed,home duties,student,retired),ethnicity(Caucasian or Indigenous/African/Asian/other),and household income per year(＜AUD65,000,AUD65,000-130,000,or＞AUD130,000).

2.5.1.Usage

Usage of the TaylorActive website was measured using objective Google Analytics data.Google Analytics is a free tool commonly used in web and app-based interventions to gather usage statistics.30,31Mean minutes spent on the Taylor-Active website over the 12-week intervention period was measured using Google Analytics,as was the mean number of sessions and action plans participants completed over the 12-week intervention period.Furthermore,self-reported website usage was measured in the 3-month CATI survey using questions similar to those applied in previous computer-tailored PA interventions.32,33Participants in all groups were asked“How often did you use the TaylorActive website?”;their answers were recorded using a 7-point Likert scale(1=I didn’t use it,2=less than once every two weeks,3=once every two weeks,4=once a week,5=2-4 times a week,6=every day,7=more than once a day).Based on data distribution,the response options were grouped into≥2-4 times per week,≤1 time per week,and≤once every 2 weeks.Participants in all groups were also asked“How would you prefer to receive personally relevant PA information via the Internet?”Response options included“Video to watch online”and“Text to read and print”.Intervention group participants were also asked:(1)“Overall,how much of the personalized feedback did you read/watch of the sessions that you completed?”Responses were recorded using an 8-point Likert scale(1=all of it,2=most of it,3=more than half,4=about half,5=less than half,6=not much of it,7=nearly nothing,8=nothing),based on data distribution,the response options were grouped into all/most,about half,and less than half;(2)“Did you watch/read the PA information more than once?”;(3)“Did you request new PA information for the same session more than once by completing the online survey again?”Both questions(2 and 3)were rated on a 5-point Likert scale(1=never,2=rarely,3=sometimes,4=often,5=always).Based on data distribution,the response options were collapsed into often/always,sometimes,and never/rarely;(4)“Did you share/show your PA information with anybody else?”and(5)“Did you discuss the content of the PA information with someone else?”had response options of yes/no.

2.5.2.Acceptability

Acceptability of the TaylorActive website was measured using questions applied in previous computer-tailored PA interventions.34,35Website acceptability was assessed in all groups using 14 statements rated on a 5-point Likert scale(from 1(strongly disagree)to 5(strongly agree)).For illustrative purposes the response options 4(agree)and 5(strongly agree)were grouped into a percentage of participants agreeing.Rated statements included,for example,“It was easy to answer the questions prior to receiving personalized feedback”,“The physical activity information was easy to understand”,“The physical activity information was personally relevant”,and“Too much physical activity information was provided per session”.In addition,acceptability questions specific to the intervention condition(video-tailored vs.text-tailored)were also asked.Statements relevant to the video-tailored condition included,for example,“I found the people presenting the information in the videos just right”,“I found the production(technical quality)of the videos poor”,and“I could easily understand everything that was said in the videos(sound quality)”.Statements relevant to the text-tailored condition included“I found the images and illustrations in the text matching well with the content”and“I would have preferred to receive the advice as a personalized video over text”.

2.5.3.Usability

Usability of the TaylorActive website was measured using the standardized Systems Usability Scale(SUS).36Website usability was assessed in all groups using 10 statements rated on a 5-point Likert scale(from 0(strongly disagree)to 4(strongly agree)).For illustrative purposes the response options agree and strongly agree were collapsed into a percentage of participants agreeing.Statements included,for example,“I thought the TaylorActive website was easy to use”,“I would like to continue to use the TaylorActive website frequently in the future”,and“I found the TaylorActive website very cumbersome to use”.Furthermore,for comparability with similar studies,37the overall SUS score was calculated using its established scoring protocol.38The score contribution for each of the 10 items ranged from 0 to 4.For Items 1,3,5,7,and 9(the positively worded items),the score contribution was the scale position minus 1.For Items 2,4,6,8,and 10(the negatively worded items),the contribution was 5 minus the scale position.The sum of the scores was multiplied by 2.5 to obtain the overall SUS score,which ranged from 0 to 100.The SUS has shown high reliability (α=0.91)when assessed through Cronbach’sα39and acceptable construct validity(r=0.70)when measured through Spearman rank correlation.40

2.5.4.Perceived usefulness

Perceived usefulness of the TaylorActive website was assessed using questions derived from a previous online computer-tailored PA intervention.19Perceived usefulness was measured across the 3 groups using 5 questions rated on a 5-point Likert scale(from 1(not at all useful)to 5(very useful)).Based on data distribution,response options were grouped into very/quite useful,somewhat useful,and not/a little useful.Participants were asked“How useful was the website in helping you to(1)increase your confidence to engage in regular PA over the past 3 months?(2)overcome barriers to participating in PA over the past 3 months?(3)increase the support you receive for participating in PA over the past 3 months?(4)plan for PA over the past 3 months?and(5)stay motivated to participate in PA over the past 3 months?”

2.5.5.Satisfaction

Satisfaction with the TaylorActive website was assessed through 2 open-ended questions:“What did you like about using the website?”and“What did you not like about using the website?”Responses were completed in open text format.

2.6.Sample size

The randomized controlled trial sample size(n=501)was calculated based on the primary study outcome(i.e.,Actigraph measured MVPA(min/week),which is reported elsewhere).20,21Briefly,to achieve 80%power using anαlevel of 0.05 and accounting for an estimated 30%attrition,167 participants per group(video-tailored,text-tailored,and control group)were recruited at baseline.At post intervention(3 months),72 participants in the video-tailored,83 in the text-tailored,and 104 in the control group completed the CATI survey.Only the post intervention sample(n=259)was used in the process evaluation presented in this paper.

2.7.Data analyses

For the quantitative analyses,categorical data were presented using frequencies and percentages,and continuous data were presented using means and standard deviations.χ2tests and one-way analysis of variance were used to assess differences in intervention usage,acceptability,usability,and perceived usefulness by group.All analyses were performed in IBM SPSS Statistics(Version 26.0;IBMCorp.,Armonk,NY,USA)with the significance level set at 0.05.Thematic content analysis was used to identify emergent themes in the qualitative analyses of the open text responses to the questions on intervention satisfaction.41A pen profile,which is an increasingly utilized technique to present analyzed text data in a diagram,42was constructed based on themes identified in relation to“Likes”and“Dislikes”of the TaylorActive intervention.Example verbatim quotations were then extracted directly from the text responses to illustrate a theme.To provide an indication of the prevalence of the themes,the number of times a specific theme was mentioned was also presented.

3.Results

3.1.Baseline data

The socio-demographic characteristics of the full trial participants(n=501)are reported elsewhere.21Table 1 presents the socio-demographic characteristics of participants(n=259)who completed the post intervention survey at 3 months and are included in the analyses presented in this paper.Participants’mean age was 44.0±13.1 years(mean±SD).Most participants were females(70.7%),had 14 or more years of education(81.4%),lived in a major or regional city(92.3%),and were Caucasian(91.1%).Retention rate was 52%,with 259 participants completing the CATI survey at the 3-month assessment(post intervention).There were no significant differences in socio-demographic characteristics between participants retained(n=259)and those lost to follow-up at post intervention(n=242)(all p＞0.05).

Table 1 Socio-demographic characteristics of participants(n(%)).a

3.2.Usage

Usage of the TaylorActive website in the video-tailored,text-tailored,and control group is presented in Table 2.Google Analytics data showed that participants’overall time spent on the TaylorActive website during the 12-week intervention period was on average 90.1±71.8 min in the video-tailored group and 77.0±80.0 min in the text-tailored group,compared to 8.5±16.1 min in the control group(p＜0.001).The number of times participants visited the TaylorActive website was on average 7.6±7.2 in the video-tailored group and 7.3±5.4 in the text-tailored group,compared to 1.4±1.9 in the control group(p＜0.001).Out of the 8 intervention sessions,participants in the video-tailored group completed on average 4.8±2.8 sessions,and participants in the text-tailored group completed 5.0±2.8 sessions(p=0.487).Out of 6 action plans to be completed on the website,participants in the video-tailored group completed on average 2.7±2.1 action plans;similarly,those in the text-tailored group completed 2.7±2.2 action plans(p=0.940).The post intervention survey data showed that most participants used the TaylorActive website less than once every 2 weeks,with significant differences being observed between intervention and control groups(66.7%video-tailored,62.7%text-tailored,87.5%control;p＜0.001).Most intervention participants watched/read the tailored PA advice(77.5%video-tailored,72.2%text-tailored;p=0.516).However,the majority of them never or rarelywatched/read the tailored PA advice more than once(69.6%video-tailored,63.3% text-tailored;p=0.699).Moreover,nearly all intervention participants(98.6% video-tailored,97.5% text-tailored;p=0.641)never or rarely requested updated tailored PA advice for the same session by completing the survey again.Few participants in either intervention group(21.7%video-tailored,29.1%text-tailored;p=0.306)shared their PA advice with someone else,and less than half of the intervention participants(43.5%video-tailored,45.6%texttailored;p=0.798)discussed the content of the PA advice with someone else.

Table 2 Usage of the TaylorActive website in the video-tailored,text-tailored,and control group.

3.3.Acceptability and usability

Acceptability and usability of the TaylorActive intervention are shown in Table 3.Regarding acceptability,most intervention participants found the PA information interesting(81.2%video-tailored,74.4%text-tailored,50.0%control;p＜0.001),credible(94.2% video-tailored,93.9% text-tailored,91.3% control;p=0.805),easy to understand(95.7%video-tailored,96.3%texttailored;p=0.771),and personally relevant(70.6%video-tailored,67.1%text-tailored,51.1%control;p=0.080).More than half of intervention participants(61.8%video-tailored,63.4%text-tailored)and significantly fewer control participants(14.3%;p＜0.001)said they used the PA information to become more active.However,few intervention participants(32.4%video-tailored,22.0%text-tailored)and significantly fewer control participants(8.3%;p=0.009)felt that the PA information was developed for them personally.Furthermore,few participants across all groups(33.8% video-tailored,27.2% text-tailored,29.7% control;p=0.675)changed their opinion about PA because of this study.Regarding usability,most participants across all groups thought the TaylorActive website was easy to use(95.5%video-tailored,93.6%text-tailored,91.0%control;p=0.567)and found the various website features well-integrated(87.5%video-tailored,72.7%text-tailored,and 74.6%control;p=0.080).Most participants across all groups thought that people could learn to use the Taylor-Active website very quickly(95.5%video-tailored,96.2%text-tailored,87.8%control;p=0.086),and nearly all participants felt very confident using the TaylorActive website(98.5%video-tailored,92.3%text-tailored,90.5%control;p=0.136).However,fewer participants across all groups(52.9%video-tailored,42.2%text-tailored,38.5%control;p=0.174)would like to continue to use the TaylorActive website frequently in the future.The mean SUS scores by group were 77.3(video-tailored),75.7(text-tailored),and 74.1(control),with no significant between-group differences(p=0.274).

Acceptability of the specific video-tailored and text-tailored components in the TaylorActive website is shown in Table 4.In the video-tailored group,all participants could easilyunderstand what was said in the videos(i.e.,sound quality).Very few participants found the production(technical quality)of the videos poor(5.8%)or the downloading/buffering of the videos slow(7.2%).Few participants(17.1%)would have preferred to receive the PA advice as personalized text over a video.In the text-tailored group,most participants(77.9%)found that the images and illustrations in the text matched well with the content.Some participants(23.8%)would have preferred to receive the advice as a personalized video over text.Across both the video-and text-tailored groups,most participants(73.0%)found the length of the tailored PA advice just right in most sessions.Further,the majority of participants(78.9%)found the graphs displayed in the video/text just right,and most participants(85.7%)thought the tone of the videos was just right(factual and convincing).However,some participants(23.8%)found the personalized text/video boring,and some(20.3%)would have preferred the video/text to be more light-hearted and humorous.A third of intervention participants(33.6%)would have liked to receive printable transcripts alongside the text/videos.

Table 3 Acceptability and usability of the TaylorActive intervention in the video-tailored,text-tailored,and control groups(n(%)agreeing)a.

3.4.Perceived usefulness

Perceived usefulness of the TaylorActive intervention is presented in Table 5.In each group,only a small proportion of participants found the TaylorActive website very/quite useful for increasing their confidence to engage in regular PA(23.1%video-tailored,22.4%text-tailored,13.0%control;p＜0.001).More than half of intervention participants(52.3%video-tailored,57.3%text-tailored)and significantly more control participants(81.1%;p=0.003)thought the website was not/a little useful for overcoming barriers to engage in PA.Further,most participants across all groups(70.3%video-tailored,66.7%text-tailored,80.0%control;p=0.085)found the TaylorActive website not/a little useful for increasing the support they receive for engaging in PA.Less than half of the intervention participants(43.1%video-tailored,40.8%text-tailored)and significantly fewer control participants(12.2%;p＜0.001)found the website very/quite useful to plan their PA.Similarly,few intervention participants(33.8%video-tailored,44.7%text-tailored)and significantly fewer control participants(15.8%;p＜0.001)thought the TaylorActive website was very/quite useful for helping to stay motivated to engage in PA.

3.5.Satisfaction

Satisfaction with the TaylorActive website in the video-tailored and text-tailored group was reported qualitatively through text responses to open-ended questions.Satisfaction was expressed as“Likes”and“Dislikes”,which were then themed and illustrated in a pen profile(Fig.1).However,given that“Likes”and“Dislikes”with the intervention were assessed through 2 optional open-ended questions,there were limited text responses available.Hence,the frequency withwhich themes recurred was low(n=5-21 video-tailored,n=1-24 text-tailored).Regarding “Likes”,participants mostly liked(n=21 video-tailored,n=24 text-tailored)that the TaylorActive was easy to use:

Table 4 Acceptability of the specific video-tailored and text-tailored components of the TaylorActive website.

Easy to navigate and found it simple to use.(male,33 years)

Participants liked the personalized feedback(n=12 video-tailored,n=9 text-tailored)in that it was tailored to the individual information participants had provided,tracked participants progress,and gave them advice.Participants also liked the information provided on the TaylorActive website(n=5 video-tailored,n=7 text-tailored),such as tips for becoming more active,and the information accessible through the library:

It was a really informative,fantastic library.It was friendly and approachable.(female,40 years)

In the video-tailored group,participants(n=5)liked that the TaylorActive website was motivational:

It reminded me to stay with the project—prompted me to keep going.(female,67 years)

Some participants(n=5)also liked the videos and goal setting:

Watching the videos after the questionnaire,and it suits what you’ve just talked about.(female,30 years)

The fact that I did set goals and times to be active.(female,58 years)

In the text group,participants(n=6)liked the action plan:It made me make a plan,think about how I would achieve my goals.(female,42 years)

Table 5 Perceived usefulness of the TaylorActive intervention in the video-tailored,text-tailored,and control groups.

Fig.1.Satisfaction with the TaylorActive website in the video-tailored and text-tailored groups(“Likes”and“Dislikes”).

Regarding“Dislikes”,participants mostly disliked(n=9 video-tailored,n=18 text-tailored)the repetitive and tedious aspects of the TaylorActive website.For example,they did not want to fill out the action plan and restate their goals at every session but preferred instead to access their action plan from the previous week and modify it,if necessary:

I had to fill out the action plan every week.Just wanted to put in a plan for the whole time period and work with it.(male,31 years)

Participants(n=11 video-tailored,n=3 text-tailored)also disliked that they had to answer questions every week,and they found the response options too specific,leaving them with too few choices:

I found the optional answers too specific.It forced to make a choice that was not quite right.(female,60 years)

Technical limitations were also noted in both intervention groups(n=3 video-tailored,n=4 text-tailored),such as problems with opening the video,accessing the PA advice from the previous week,or not being able to save the PA advice in a file.Some participants(n=3 video-tailored,n=1 text-tailored)disliked the personal trainer:

Images were not realistic.Everyone was young,beautiful,and fit.(female,56 years)

In the text-tailored group,some participants(n=5)noted that there was too much to read,and some(n=3)found the TaylorActive website too time consuming.

4.Discussion

This study examined the usage,acceptability,usability,perceived usefulness,and satisfaction with a web-based,video-tailored PA intervention for Australian adults(Taylor-Active).Examination of process data at 3 months revealed that participants’usage of the TaylorActive website(i.e.,website visits,intervention sessions,and action plans completed)was modest in both the video-tailored and text-tailored groups.In both intervention groups,acceptability and usability of the TaylorActive intervention was mostly rated high.However,perceived usefulness of the TaylorActive website for increasing PA was rated low across the board.Satisfaction with the TaylorActive website was mixed in that participants in both intervention groups liked its ease of use,personalized feedback,and tracking of progress;however,they also found completing action plans and survey questions each week to be repetitive and tedious,and they thought the survey questions were too specific.

The post intervention survey showed that most intervention participants used the TaylorActive website only once or less in 2 weeks.Further,participants in both intervention groups completed just over half of the intervention sessions and less than half of the weekly action plans offered during the 12-week intervention.The TaylorActive website was designed to be used infrequently(i.e.,only when a new intervention session was available,which was weekly for the first 4 weeks and then every 2 weeks for the next 2 months).This explains the modest usage findings to some extent.Furthermore,low or declining website usage is a common issue in web-based PA interventions.30,43-45Another explanation for the modest website usage in this study may be that providing personally tailored PA advice(either video or text)alone was insufficient for participants to more regularly engage with the website.Perhaps,usage would have been higher if the TaylorActive intervention included additional features,such as continuous self-monitoring using activity trackers,social interaction(e.g.,online social networking features),and gamification(e.g.,rewards),all of which are recognized motivators of PA engagement.46Indeed,a subsequent study using the TaylorActive website showed that personally tailored PA advice combined with a Fitbit activity tracker for continuous self-monitoring positively impacted intervention usage,with twice as many participants in the Fitbit group completing all intervention sessions.47

Compared to the control group,acceptability of the Taylor-Active website was rated relatively high(i.e.,interesting,easy to understand,credible)by most intervention participants.However,while most intervention participants thought that the personally tailored PA advice was indeed“personally relevant”,only a third of participants felt that it was“personally developed for them”.This finding is interesting but difficult to explain.Perhaps intervention participants thought that the personally tailored PA advice was generally correct.For example,when advised they are not meeting the PA recommendations,they would agree with that.Nonetheless,participants may have thought that the tailored PA advice did not achieve the appropriate level of personalization and hence may have perceived it as impersonal(i.e.,because it did not incorporate contextual factors well enough or because they could tell the advice was produced by a computer).If so,interventions that can more dynamically interact with participants through dialogue(e.g.,via an artificially intelligent virtual coach)and use ongoing real-time data to tell whether or not they are on track(instead of a single interaction every week or 2 weeks)may be more appealing.With technological advances in machine learning and artificial intelligence,the delivery of personally tailored PA interventions via an artificially intelligent virtual health coach(i.e.,chatbot)has shown promise.48

The majority of participants rated the usability of the TaylorActive website high(i.e.,easy to use,features well integrated),with no significant differences between the 3 groups.The usability score was also good(i.e.,＞68,which is the threshold for good usability)38and higher than what has been reported in other web-based PA interventions.37However,only half of the participants in the video-tailored group and even fewer participants in the text-tailored group stated they would continue to use the TaylorActive website frequently in the future.This is likely due to how the website was organized.For example,for participants who completed all the sessions,there would be little reason to continue to use the website.Providing new or updated content is likely necessary for people to continue using the TaylorActive website.

Perceived usefulness of the TaylorActive website(e.g,for increasing the confidence to be active,planning PA,overcoming barriers,staying motivated)was mostly rated low in both intervention groups,though higher compared to the control group.One explanation may be that the TaylorActive website had very few features beyond the tailoring of PA advice(e.g.,no social networking or gamification).Most electronic and mobile health interventions promoting PA incorporate a plethora of activity tracker-,web-,and app-based features as intervention components.6However,this makes it impossible to identify which intervention components are effective.6The TaylorActive study was designed with a clear objective to isolate the effectiveness of computer-tailoring(video or text)from any other intervention components that may mask its contribution.20It appears to be the case that implementing too few intervention components may fail to sufficiently motivate people to become more active.Nevertheless,studies that evaluate intervention components in isolation are needed,as this knowledge will help make future interventions more effective overall.

Participants in both intervention groups found the completion of action plans and survey questions for each session to be repetitive and tedious.This finding is consistent with those from previous computer-tailored PA interventions showing that having to complete many questions prior to receiving personalized PA advice is discouraging for participants.30,34This obstacle could be overcome by using automatically integrated activity tracker data instead of repeated online surveys to generate the personally tailored PA advice.This approach was successfully implemented in a subsequent intervention study using the TaylorActive website.47

This process evaluation identified factors that may partially explain why the TaylorActive intervention was ineffective,21and it provides ideas for improving the web-based video-tailored PA intervention.The inclusion of a comprehensive set of previously used measures and the use of both self-reports and objective data as part of a large-scale,rigorously conducted,randomized controlled trial are strengths of this process evaluation.However,a significant limitation of this study was the attrition of participants(48%at the 3-month survey),which corresponds to the modest usage and low perceived usefulness ratings revealed in the evaluation.Possible reasons for attrition were discussed in detail elsewhere.21A brief interview or survey with participants who dropped out(i.e.,an exit survey)might have added more in-depth information to the process evaluation.Another limitation was that the full trial sample(n=501)was not entirely representative of the target population.For example,the TaylorActive trial targeted adults who did not meet the Australian PA guidelines(i.e.,≤150 min of MVPA per week).Despite screening participants for this during recruitment,more intervention participants(24%videotailored,22%text-tailored)than intended met the PA guidelines at baseline when measured using acceleromtery.21Moreover,the full trial sample included a high proportion of females(72%)and highly educated(80%)participants.21

5.Conclusion

The process evaluation of the TaylorActive intervention revealed that providing personally tailored PA advice on its own,either through video or text,is insufficient to promote usage,usefulness,and satisfaction with a web-based PA intervention.Strategies to address these issues may include the incorporation of additional components in web-based interventions.For example,using automatically integrated data from activity trackers(e.g.,Fitbit,Garmin)instead of repeated online surveys to generate the personally tailored PA advice may improve participant satisfaction and retention.Furthermore,the use of different technologies,such as artificial intelligence and machine learning,may allow for a more personalized dialogue with participants thereby increasing the appeal of personally tailored interventions.The incorporation of social media may also stimulate intervention usage by providing participants with greater opportunities for social interaction.Finally,the availability of a user-friendly mobile version(i.e.,smartphone app)in addition to an intervention website may increase the convenience and perceived usefulness of the intervention.

Acknowledgments

The TaylorActive trial was funded by the National Health and Medical Research Council(1049369).CV(100427),MJD(100029),and SS(101240)were,and SA(102609)is currently,supported by a research fellowship from the National Heart Foundation of Australia.CES (1090517),RCP(1100138),and AR(1105926)were,and MJD(1141606)and SS(1125586)are currently,supported by a research fellowship from the National Health and Medical Research Council.A special thanks goes to the TaylorActive project officers Cindy Hooker,Doreen Canoy,and Deborah Power,as well as to senior research fellow Dr.Camille Short,for their contributions to intervention development and evaluation.We also thank Nathaniel Fitzgerald-Hood for developing and maintaining the software platform that housed the TaylorActive intervention.Furthermore,we thank the creative agency Headjam based at Newcastle(Australia),which was responsible for the website design and video production.

Authors’contributions

CV,CES,RCP,WKM,and MJD conceived the project and procured the project funding.CV led the coordination of the trial.CV,CES,RCP,AR,SA,SS,WKM,and MJD assisted with the protocol design.CV and CES developed intervention content for the trial,and MJD performed the sample size calculations.SS conducted the analysis,and CV,RCP,AR,SA,WKM,MJD,and QT interpreted the data for this paper.SS and CV drafted the manuscript.All authors have read and approved the final version of the manuscript,and agree with the order of presentation of the authors.

Competing interests

The authors declare that they have no competing interests.