Our Impact
By maintaining high professional standards and delivering high-quality learning experiences, we do our best to ensure that our partners and stakeholders receive value from our work.
Introduction
The Scottish Digital Academy has been working to develop an approach to ensure that we can maintain our standards and assure the quality of our offer and service. Our approach is outlined here, to illustrate how we intend to measure the impact and value of our work over the next few years.
International standards
A well-conceived measurement and reporting strategy is necessary to ensure organisational and individual development processes are managed efficiently and effectively to produce the desired outcomes. We therefore align our approach to measuring learning to those set out in ISO/TS 30437:2023 Human resource management: Learning and development metrics.
Annual monitoring and reporting
We report annually on our progress through monitoring. This may take the form of an in-depth report or an update to the previous report, published on our website. Our last in-depth report was published in October 2022, with the next one expected in 2026.
Our new standards were introduced in January 2024 and have therefore not informed any previous reporting, although there will be some similarities compared to previously reported data. We expect that they will begin to inform a better understanding of impact from this date.
Crucially, our standards will determine whether an intervention is achieving what it set out to do and whether it is making a positive difference, and assess the systematic and regular collection of information related to professional learning.
The expected reporting timetable is illustrated in the table below. This is subject to change.
| October 2022 | Annual Monitoring Report |
| January 2024 | Annual Monitoring update and introduction of the standards |
| January 2025 | Annual Monitoring update and benchmarking the standards |
| January 2026 | Annual Monitoring Report |
Annual Monitoring 2023
We’ve taken a lighter approach to monitoring this year, as our strategic priorities have changed. When we last reported in 2022, our focus was on leadership, professional learning, innovation, partnership and community. Although these principles remain at the heart of what we do, through the Scottish Government’s Digital Programme, we’ve refreshed our focus to align to wider public service reform and to better support the public sector now, and in the years to come.
Annual Monitoring Report 2023Quality improvement
In addition, so that we may improve the quality of our offer, we will draw on best practice from the wider field of professional learning. While there is no set way to evaluate professional learning in education, the work by Guskey suggests effective evaluation of professional learning requires consideration of five critical levels of information:
- Participant reaction – this includes the reaction to the experience in terms of usefulness, value, content, materials and information. It also includes questions about the organisation, setting and facilities offered at the venue.
- Participant learning – measuring the new knowledge, skills and attitudes/ dispositions that participants gain. To successfully do this it is important to outline indicators of successful learning before activities begin.
- Organisational support and change – focus on organisational dimensions that may be vital to the success of the professional learning experience. Questions focus on the organisational characteristics and attributes necessary for success.
- Participants use of knowledge and skills – did the new knowledge and skills that participants learned make a difference in their professional practice? Important here is the specification of clear indicators of both the degree and quality of implementation.
- Learning outcomes – what was the impact on learners? Did the professional learning benefit them in any way? The particular student learning outcomes will depend on the goals of the specific professional learning.
Guskey, T. R. (2000). Evaluating professional development. Thousand Oaks, CA: Corwin.
Our standards
Standard One: Digital, Data and Technology (DDaT) Profession
The Digital, Data and Technology (DDaT) profession is able to access opportunities to upskill or reskill throughout the year.
Measures of success will include:
- The number of registrations completed by DDaT Professionals.
- The proportion of participants involved in professional learning opportunities reporting increased awareness, understanding, knowledge and skills.
- The proportion of participants involved in professional learning opportunities reporting they are likely to implement agile methodologies and user centred design into their practice.
- Inclusion of data from the most recent skills survey.
Standard Two: Public and Third Sector Employees
The public and third sector is able to access opportunities to upskill or reskill throughout the year.
Measures of success will include:
- The number of registrations completed by participants across the public and third sector.
- The proportion of participants involved in professional learning opportunities reporting increased awareness, understanding, knowledge and skills.
- The proportion of participants involved in professional learning opportunities reporting they are likely to implement agile methodologies and user centred design into their practice.
Standard Three: Organisations
Organisations are able to access opportunities to upskill or reskill throughout the year.
Measures of success will include:
- The number of registrations completed
- The proportional representation of public and third sector organisations
- The rate of satisfaction upon engagement with the Academy
Standard Four: Agile Coaching Service
The agile coaching service has scaled and increased its level of intervention to build digital capability across the public and third sectors.
Measures of success will include:
- The number of referrals made via the Digital Scotland Service Manual.
- The number of business leads.
- The number of agile coaching interventions.
Standard Five: Value
We offer professional learning at value to the public and third sector.
Measures of success will include:
- The procurement and/or operational cost to provide a course or programme.
- A comparative cost for similar course or programme offered by another supplier.
- Use and uptake of the Dynamic Purchasing System – Lot 3 Digital Training.
Standard Six: Partnership and Collaboration
We collaborate with our partners and stakeholders to support the public and third sectors and to develop digital capability.
Measures of success will include:
- Delivery of work aligned to the Digital Programme.
- Delivery of work aligned to the Digital Scotland Service Manual.
- Programmes and projects jointly delivered.
Standard Seven: Leadership
Leaders from across the public and third sectors are able to access professional learning opportunities.
Measures of success will include:
- Participants engaging in informal leadership development opportunities.
- Participants engaging in formal leadership development opportunities.
Standard Eight: Business and Operations
Those engaging with us receive a high quality service and a positive experience.
Measures of success will include:
- Service level agreements and statements of work.
- Response times.
- Level of satisfaction with our service.
Standard Nine: Diversity and Inclusion
We promote diversity and inclusion.
Measures of success will include:
- Delivery of professional learning to diverse groups of people.
- Accessibility of our learning content and delivery.
Standard Ten: Responding to Change
We will respond to change and complex environments by being proactive as well as reactive.
Measures of success will includes:
- Design, delivery and evaluation of professional learning related to technological advancement.
- Design, delivery and evaluation of professional learning related to system changes and the needs of the public and third sectors.
- Emerging trends determines by annual monitoring and quality improvement.
Standard Eleven: Community
We nurture a thriving community that develops digital capability.
Measures of success will include:
- Engagement and activity of Digital, Data and Technology (DDaT) communities of practice.
- Growth and engagement of our Alumni community.