How PhD students benefit from Scholarcy
How do PhD students benefit from using Scholarcy in their workflow?
We recently set out to measure the quantifiable benefits that PhD students get from using Scholarcy as part of their reading and research workflow.
The aim of the study was to determine if more advanced users, undertaking research projects, would experience tangible time-saving and productivity benefits from using Scholarcy within their existing workflow.
We also wanted to determine if Scholarcy would have an overall positive effect on PhD students’ experience of screening, assimilating, and applying new literature.
Finally, we wanted to understand if Scholarcy makes it easier for advanced users to retain information from academic texts, and if the library system would help them better organise and retrieve information.
How the study was conducted.
PhD students were selected to participate in the study based on their familiarity with sourcing and reading complex academic texts. We wanted to learn how this advanced user group could benefit from Scholarcy, particularly in relation to processing and applying information from large collections of articles.
Each participant was sent an initial survey to complete before being given free access to Scholarcy. After they had used Scholarcy for 6 weeks participants were sent a follow-up survey to complete.
Responses to each question in the surveys were recorded on a scale of 1 to 10. The difference in results was used to determine if Scholarcy had an overall positive, negative, or neutral effect on the ease with which participants' read, understood, analysed, and applied literature. The responses were averaged and compared for the before and after scenarios to show Scholarcy’s impact on their knowledge and productivity.
How were the participants selected?
23 PhD students were selected. They were recruited for the study via X (formerly Twitter), word of mouth, and in-person networking. The participants had to be enrolled in a PhD program at the time of the study, with no prior experience of, or exposure to Scholarcy.
The participants were asked to answer each question on a scale of 1 to 10 so that their experience of using Scholarcy could be quantifiably measured.
Participants were given an annual Scholarcy subscription, and £50 for taking part in the study. An annual subscription to Scholarcy was provided upon completion of the first survey, and the Amazon voucher was sent to the participant after the second survey was completed.
Part-1 survey questions:
1) How easy do you find approaching new literature? (1=Very difficult, 10=Very easy)
2) How easy do you find retaining new information extracted from academic text? (1=Very difficult, 10=Very easy)
3) Do you feel that performing a literature review takes you a long time? (1=Very short, 10=Very long)
4) How easy do you find organising and sorting papers you wish to reference? (1=Very difficult, 10=Very easy)
5) How difficult do you find comparing research outcomes from multiple academic texts? (1=Very difficult, 10=Very easy)
Time between surveys
Participants were given 6 weeks between surveys to use Scholarcy as part of their regular reading and research work.
After 6 weeks, the second survey was emailed to them for completion.
Part-2 survey questions:
1) How easy do you find approaching new literature using Scholarcy? (1=very difficult, 10=Very easy)
2) How easy do you find retaining new information extracted from academic texts using Scholarcy? (1=very difficult, 10=Very easy)
3) How long do you feel that performing a literature review with Scholarcy took? (1=Very short, 10=Very long)
4) How easy do you find organising and sorting papers with Scholarcy that you wish to reference? (1=Very difficult, 10=Very easy)
5) How difficult do you find comparing research outcomes from multiple academic texts with Scholarcy? (1=Very difficult, 10=Very easy)
6) How likely would you be to recommend Scholarcy to a peer? (1=Unlikely, 10=Extremely likely)
7) How much would you miss Scholarcy if you stopped having access to it? (1=Not at all, 10= A lot)
Participants were also asked if they had any feedback, and if there were any additional features they would find useful in Scholarcy.
What did we find?
23 PhD students found that with Scholarcy…
- It was easier to approach new literature
- New information was easier to retain with Scholarcy’s flashcards
- Reviewing literature took less time
- Organising their collection of literature was easier
- Comparing research outcomes became more effective
The figures above show the difference between the results for each question in the second survey. The results of each survey were averaged. The part 2 result has been labelled with the difference between results from each survey.
The students taking part in the study were also very likely to recommend Scholarcy to a peer(average rating 8.7, 11 rating 9 or higher).
The students stated that they would miss Scholarcy if they stopped having access to the service (7.8 point average, 9 rating 9 or higher)
How well was Scholarcy received
.png)
Average feedback scores after users were asked how much they would miss Scholarcy, and how likely they would be to recommend Scholarcy to a peer. The net promoter score was calculated to be 53, showing how well PhD students received Scholarcy.
What does this mean?
By surveying students actively engaged in a PhD programme, we were able to quantify the positive effect Scholarcy had across all categories investigated.
The students surveyed consistently found that the time spent reading articles decreased, whilst they were able to retain more information, and more easily organise their collection.
As PhD students are already very familiar with the structure of academic articles, as well as the techniques required to find relevant papers, they typically already have an established workflow for conducting literature searches, and screening articles. The results gained from this study show the impact Scholarcy had on even more advanced users in terms of efficiency and productivity.