Organic Approaches to Learning Analytics in Higher Education: Co-Reflective Insights from the Sector's Practical Experiences

Dr Raghda Zahran reflects on the 2023 Learning Analytics Virtual Conference.

Background

Indeed, higher education professionals have been anticipating the potential of students' data and analytics to inform learning and teaching decision-making. However, it is becoming evident that the sector is still lagging in fully realising the potentials of analytics to improve learning.

When I started my journey exploring the use of analytics in 2020, I found myself two decades behind and in need of a deep understanding of the current state of the sector, particularly with the surge in online learning during the COVID-19 pandemic. This involved engaging with relevant literature and colleagues within and outside the UK through direct conversations, conferences, and forums.

Through this process, I realised that despite eagerness to utilise analytics, institutions naturally fell as the Innovation Diffusion Theory suggests into categories of innovators, early adopters, and laggards. I quickly realised that there is a limited means of understanding the sector's current state beyond knowledge exchange initiatives.

About the Conference

Fortuitously, the 2023 Learning Analytics Conference organised by the HE Professional offered a valuable opportunity to gain insights into analytics practices across multiple institutions. This online forum gathered a specialist group of analytics enthusiasts, including researchers, students, and colleagues in academic and professional support roles. Those actively engaged in open discussions and networking sessions. Six education and technology experts who shared their reflections on their experiences and visions of analytics in the sector made up the agenda. Their notions and focuses have the potential to contribute to an emerging framework and conversations for further development.

An Emerging Framework

Stakeholders and Broader Landscape - Clare Walsh, Head of Education at the Institute of Learning Analytics, and Professor Bart Renties, President of SoLAR, emphasised the role of contextual influences on analytics acceptance, adoption, and deployment. Walsh highlighted the importance of interoperability within higher education and the need for tested data quality and governance to support users' autonomy. Professor Renties emphasised the engagement of senior leaders in facilitating change through analytics. Drawing on his experience since the early adoption of analytics at Open University's, Professor Renties stressed the significance of a balanced equation involving all stakeholders in adoption and sense-making. Renties also prompted discussions on key beneficiaries’ identification and challenges in implementation. Both Walsh and Renties emphasised governance and the role of stakeholders’ engagement and autonomy to enable change and realise its value.

Practical Experiences– Both Marinella Vowles, Head of Teaching, Academic and Research Technologies and Ed Foster, Head of Student Engagement and Analytics at Nottingham Trent University presented their institutions implementation of analytics. Vowles provided an overview of 3-phase scaffolded and structured implementation over three years. The establishment of a dedicated ‘Student Success’ team working along academics to deepen understanding of analytics, agree on responses seems to help in realising the benefit of data. Foster, reflected on a Nottingham Trent’s two-phase journey implementing engagement dashboard, traffic lights and responses also through a dedicated ‘contact team’ to support students. This seems to be perceived positively by students and tutors. Foster’s student-centered support spectrum suggests a structured but varied approach to analytics utilisation.

A Vision for Ongoing Improvements - Andy Ramsden, Director of Teaching and Learning, and Matt Laidler, Learning Enhancement and Technology Advisor, offered a forward-thinking perspective that juxtaposed the traditional standardisation of analytics. Ramsden's, with a more generative and self-service approach, emphasises the need to deepen insights gained from data. While Laidler shed light on the progressive use of analytics to support planning and learning design. Laidler reflected on a scaffolded journey highlighting a vision to benefit from artificial intelligence to deepen the understanding. Both perspectives support continuous improvement of learning and teaching and offer visionary view of data democracy and advocate for the inclusion of students as primary stakeholders. This links tightly with Walsh’s notions of users’ autonomy to realise the value data, also aligning with the growing power of interoperability that deepen the use of data, analytics, and artificial intelligence.

These organic approaches, reflections, and discussions propose opportunities and challenges within the frame of:

• Conceptual and Theoretical Foundation

• Broader Landscape of Learning Analytics within the Higher Education Sector

• Stakeholders Engagement

• Real-World Practical Experiences

• Practical Experiences and Organic Approaches

• A Vision for Improving Education and Empowering the Users' Autonomy

The speaker's institutions exhibit a range of diversity. Open University and The University of Law are actively engaged and eagerly anticipate ongoing improvements. Surrey and Nottingham Trent have successfully completed their delivery and are currently examining the impact of their contributions. Newcastle University has employed a scaffolded delivery approach embarking on value-driven implementation.

Recommendations

At the end of this session and using an online collaborative board, I posed a question to speakers onto their key recommendations based on their experiences. I also asked the audience about their key takeaways from the discussions.

Participants Takeaway

• Analytics is not the solution.

• What is more critical:

  • data governance

  • stakeholder engagements

  • closing the loop of data utilisation

• Need more discussions around:

  • institutional governance

  • scoping data

  • analytics for curriculum development

  • analytics for strategic insights

  • impact measurement

Speakers’ Recommendation

• Start with:

  • data governance

  • data quality

  • refine objectives

  • scope and requirements

  • data, integration, and testing right

  • engage with stakeholders in workshops, conversations, and case studies

• Be prepared for:

  • data anomalies at any point of your implementation

  • schools having their bespoke management of their student data

  • to have subjective and deep insights into your students learning journey

Chair’s Reflections

Overall, this event was an excellent wrap up for analytics in 2023. It provoked discussions around approaches to scoping, requirements, implementation, and impact measurement for both the users, wider stakeholders, and institutions. These imperative demand ongoing conversations and open reflections on both challenges and best practices.

However, the emerging frame for analytics from this event suggests that effective and growing implementation is not limited to analytics systems although these could be merely aimed at dynamic data presentation, user friendly dashboards, reasonable systems architecture. Such elements are critical to afford seamless interoperability, and appropriate tools and techniques that enable deep data connections and meaning making.

The discussions suggest that a supportive context with a tested governance structure, engaged stakeholder, evolving deployment, and practical reflections could contribute to the change towards a culture of data-informed practices that supports the learners and colleagues’ journeys within higher education. Despite using structured approaches to implement analytics, the insights from participants and speakers’ experiences suggest an organic approach which emphasises flexibility, adaptability, and evolutionary growth in analytics process development.

Dr Raghda Zahran is Programme and Projects Manager at Newcastle University.