SORT is a self-administered ‘research readiness’ tool designed to be used at an organisational level. It assesses the readiness of a healthcare organisation to support nurses to undertake research related activity through available structures and processes within it.
SORT is for all nurses working in health and social care interested in developing research capacity and capability within their organisation. It has been developed in partnership with colleagues in academia, the third sector and those who support the development of research. SORT supports the implementation of the CNO Strategic Plan for Research and provides a useful resource for other professions and groups across health and care. For more on how SORT was developed, see Appendix C.
Undertaking a SORT assessment can be the start of a process of improvement planning and making visible the type and level of support needed to develop research capacity among nurses and gain the benefits of research related activity in your organisation.
The findings of the SORT process can:
provide structured information for chief executives, chief nurses, boards and managers that will assist in planning
encourage self-reflection, prioritisation, and planning. As a result of completing SORT, strengths can be celebrated and monitored for impact, and gaps can be identified for improvement planning
The 44 statements in SORT provide a framework to assess both level of maturity and scale of research support provided. SORT is divided into 5 sections which are based on underlying themes or concepts:
Releasing potential – spotting talent, enthusiasm and resilience, ‘learning by doing’, training and research skill use in practice. It also covers understanding the importance of an enabling environment, facilitating and supporting research careers from the start.
Embedding research – reducing barriers to research related activities by providing time and resources. It covers making research legitimate in the organisation, recognising the impact of research and nurses’ contribution to research.
Linkages and leadership – activities related to forming research links outside the organisation, promoting nurse research leadership to influence the wider research agenda.
Inclusive research delivery – activities related to supporting public and patient involvement in research. It also covers engaging the wider nursing workforce in delivering portfolio research, creating more opportunities to deliver research and making the contribution of nurses visible.
Digitally enabled research – activities related to research leadership and skills in digital technologies and data science including the skills needed to undertake research and service developments.
The SORT tool
A checklist for the SORT process
Sources of evidence template
Examples of evidence
Improvement plan template
The use of SORT should be determined by your own local context and support available. Organisational structures and processes may differ across the health and care system. SORT can be used locally, regionally or nationally – it is up to you to decide.
SORT was developed with flexibility in mind and therefore it can be used as a framework for collective reflection or as a validated measure (see Appendix B).
You should use SORT in a way that ensures equality of access for all nurses to undertake research related activity, particularly those from Black, Asian and other ethnic minority backgrounds who have been marginalised, discriminated against and excluded. Aligning SORT to equality, diversity and inclusion strategies within organisations is paramount to increasing accessibility for all, to ensure all individuals in the communities we serve have equitable access to health and care research opportunities. Building capacity and capability in this way will help to achieve this and supports the vision of the NHS Long Term Plan for research and innovation to drive future outcomes.
The SORT approach can be initiated by anyone who has responsibility to develop a research culture within the organisation. We strongly advise that SORT should be undertaken with support at the executive level with senior nurse leadership to ensure action, legitimacy, consistency of assessment and ownership in the process. If you are using SORT as a framework, think carefully about who should be in the assessment group, and if you are using it as a validated measure, think carefully about who should be in the planning group.
A maturity matrix is a scale used to assess the extent to which an organisation has developed or implemented an initiative. In this instance, the initiative is supporting research capacity development among nurses. Unlike more traditional ratings or scales, a maturity matrix enables organisations to contextualise developmental steps as a positive, rather than negative option.
The matrix levels within SORT are:
Score 0 – Not yet planned
Not under consideration. There is little awareness and no planned activity in this area.
Score 1 – Planned
Some awareness of need. Planning is underway. There is some evidence of early activity based on plans.
Score 2 – Early progress
Activity is occurring in small pockets of the organisation.
Score 3 – Substantial progress
Activity is evident and expanding. Widely seen and adopted in the organisation not just in certain areas.
Score 4 – Established
Activity is embedded in the organisation and widely understood to be available. Monitoring and review of progress and continuous improvement is undertaken.
The assessment or planning group should spend some time thinking about where to find evidence before embarking on making judgements. An example collection template is provided in the key resources list. Some examples can be found below; this is not an exhaustive list and sources of information will vary by organisation.
Releasing potential:
mentorship programme documentation
annual appraisal information
research support systems
internal and external communications
finance departments
research and innovation department
learning and development team
research and nursing leadership documents
nursing audit and review
library and information resources
human resources department
Embedding research:
job descriptions
nursing and central communication
strategic documentation
annual reports from research and innovation department
Linkages and leadership:
leadership forums
research and innovation department
communities of practice
research strategy and leadership
Inclusive research delivery:
research and innovation department
patient and public involvement and engagement activities
involvement in commercial and non-commercial research
authorship on research outputs
acknowledgement on research outputs
nurses working as principal investigators (PIs) for portfolio research
Digitally enabled research:
training to work in the digitally enabled environment
training using and interpreting data
named digital nurse leader
sharing best practice strategies
support systems
Some examples of evidence were gathered during the development of SORT and can be found in the key resources list to guide you. Please note that the same statement is answered by different organisations.
How you present SORT findings will depend on how you use SORT.
If you use SORT as a framework, we suggest that you produce 1 document with assessment and evidence for each statement. This will include a completed SORT tool for your organisation that you have agreed, along with an improvement plan (a template for this can be found in the key resources list). You will be able to reflect together on the findings of SORT tool based on your consensus.
If you use SORT as a validated measure, we suggest that you produce a report that includes the mean (average score for each statement) and the spread of each answer described as the standard deviation. You may wish to undertake a subgroup analysis by grade, profession or department depending upon what will provide the most insight to produce an improvement plan.
The planning group should review the findings and identify priorities and actions accordingly. Once you are at the improvement planning stage of the project, re-assess skills and assets within the group to ensure that it has the skills and authority to plan and ensure change can happen. Complete an improvement plan based on this work.
The report may include:
a sampling frame and rationale for this
response rates
findings based on mean scores for each statement.
subgroup analysis, if undertaken
conclusions – the improvement plan should follow on from the conclusions
Remember to celebrate successes and identify where you consider a change is needed, what should happen next and who needs to act and by when.
Use SORT to make successes visible and share these. Use the improvement plan (see key resources for a template) to build a strong case for additional support, resources and planning. Use the improvement plan to engage others to contribute to supporting research capacity development in nurses, for example the library and knowledge services, learning and development or external partners like universities and other higher education institutions.
It is up to you how and where you share your results. Work with the senior nurse leaders in your organisation to decide who should know about the results. Use findings to instigate discussion about capacity building and plan to share the priorities for action with people that can help you to make improvements.
You may want to share your findings with other similar organisations and learn from one another, or work across organisations to support one another when undertaking SORT. These are your decisions and many people consulted during the development of SORT thought that working with similar organisations would be helpful.
It will be useful to repeat the SORT process in your organisation over time, and we encourage you to do this. Use the findings of the SORT to develop appropriate, objective and measurable improvement plans. The time between review should be based on your improvement plan, timescale and measurement of its objectives.
SORT is designed to be a self-administered assessment tool to be used within an organisation itself. You can use the SORT to show progress over time by using the same methods undertaken at baseline. It might be useful to work with other organisations to learn from one another. SORT is not designed as a benchmarking tool for comparison between different organisations, however, it can be used locally, regionally or nationally to learn together.
This is okay and we all must start from somewhere. Many of the SORT statements are intentionally aspirational. By this, we mean if such activity was taking place in the organisation, then it would achieve success in supporting nurses to undertake the full spectrum of research as common practice.
Many organisations are starting at an early phase in their organisational and professional development to accomplish this vision. Undertaking the SORT assessment is the first step towards positive change and achievement.
This term includes the full spectrum of activity linked to research that could be undertaken by nurses. It includes:
research use in the form of evidence-based practice and service development, for example
research delivery – how nurses can provide practical support to clinical studies both in the NHS and across health and social care. This research is often led by others outside the organisation but also requires recruitment and participation from patients, the public and staff members. Such studies can be National Institute for Health and Care Research (NIHR) portfolio or non-portfolio studies
collaboration in research – where nurses are active members of the research team and contribute to the development of research questions, methods, data collection, analysis or conclusions of the research
leadership – where a nurse leads the research project by developing research questions and design, and completes the project as an individual or by leading a team
There is a growing body of evidence that suggests there are many benefits for undertaking research related activity in healthcare organisations. These include:
organisations that engage in research have high levels of patient satisfaction, reduced staff turnover and increased staff satisfaction (Harding, et al., 2016. Organisational benefits of a strong research culture in a health service: a systematic review)
there is an association between research engagement of practitioners and improvements in the process of care (Boaz, et al., 2015. Does the engagement of clinicians and organisations in research improve healthcare performance)
embedded research practitioner posts can enhance clinical skill development, increase research activity, promote service changes, increase collaborations, and enhance research culture (Wenke, et al., 2017. Allied health research positions). Having embedded researchers in an organisation also increases the use of evidence in policy (Wolfenden, et al., 2017. Embedding researchers in health service organizations improves research translation and health service performance)
treating patients within clinical trials is associated with cost savings for healthcare institutions involved (Liniker, et al., 2013. Treatment costs associated with interventional cancer clinical trials conducted at a single UK institution over 2 years (2009-2010)) and improved mortality rates (Harding, et al., 2016; Ozdemir et al., 2015. Research activity and the association with mortality)
organisations that deliver clinical trial activity are associated with better Care Quality Commission outcomes (Jonker, et al., 2018. The correlation between National Health Service trusts’ clinical trial activity and both mortality rates and care quality commission ratings)
Importantly, having more nurses undertaking and leading research will create a stronger evidence base for the profession, the wider health system and improved care for patients and their families.
Collective reflection is when a group of people within an organisation work together to reflect on the organisation’s position in relation to research capacity in nursing, using the statements to assess the level of readiness. The people within the assessment group should work together to agree the position of the organisation’s maturity against each statement and provide evidence to illustrate this judgement. If SORT is used in this way, consider involving different sections/departments of your organisation in this process, for example research and innovation (R&I) departments, library staff, communication teams and learning and development departments. Also include a broad cross-section of nurses in the assessment process.
SORT may be used to plan research capacity initiatives for midwives and allied health professionals (AHPs). Although SORT was commissioned for the nursing workforce, we have consulted with midwives and AHPs during its development, and it is also considered relevant to them. So, if you are undertaking a joint strategy with this wider group of professionals within your organisation, it would be appropriate to include them within the assessment team. This will support conversations about research capacity with other professions and provide collective judgement about an organisation’s current position or maturity level. Working together across departments may be a first step to support change and can illustrate different perspectives. It may also be an opportunity to harness support to promote improvement planning. Ensure that the assessment team includes those with authority to instigate identified actions arising from the completion of SORT.
SORT can be used as a questionnaire to measure organisational readiness from a sample of people in your organisation. This can include practitioners at all grades and managers.
Decisions on who you collect this information from, and how you interpret the resulting data should be undertaken by a planning group. Think carefully about who will be in this group, not only for deciding a sampling frame (that is, who you send the questionnaire out to), but also for the action planning phase of the project. You will also need statistical know-how and data management software to do this. Using SORT as a validated measure means you collect views from a larger range of individuals and collect their perspectives on the organisation’s level of maturity in supporting research capacity.
A summary of steps to use for each approach can be found in the checklist in the key resources list.
The statements in SORT were developed in a systematic way. Initial statements were generated from the research and theoretical evidence and refined using a consensus technique with a large panel of experts. These experts were asked to agree on the most relevant statements for supporting research capacity development at an organisational level within the health and care system.
The statements in SORT therefore show good face validity, that is, the statements appear useful and relevant for practice. We then tested the SORT prototype with approximately 200 nurses, midwives and AHPs, and undertook a statistical analysis to find the most reliable statements to use. SORT can therefore be considered as a validated measure, against which an organisation can assess their progress over time.