We design and optimise retrieval systems through benchmarks and standardised measurements. But are these measurements truly reliable? Do they accurately model the human needs they are meant to represent? There is a deep literature examining how best to make such measurements in information retrieval, but it is based on the assumption that our test collections are representative and that measured scores are a good approximation of human behaviour. In this talk, I argue that neglect of these issues has consequences in practice. Blind pursuit of performance gains based on optimisation of scores, simplistic use of test collections, and analysis based solely on aggregated measurements can lead to misleading or even meaningless outcomes.

Artificial intelligence (AI) is becoming increasingly common in higher education, with students using AI tools to support activities such as research, writing, and revision. Although students and educators generally view AI positively, its integration into teaching and learning remains uneven. In many courses, AI is still used as an additional tool rather than being fully incorporated into teaching practices and learning activities. A gap also remains between AI-related training and its application in practice. While some lecturers have received AI literacy training, students may not always feel that sufficient guidance is available on how to use AI appropriately and effectively. In addition, the rapid adoption of AI has raised concerns related to academic integrity, data privacy, ethical use, and equitable access. This presentation discusses recent global developments in AI in education together with Malaysia’s emerging policy landscape, including national guidelines and selected initiatives at Universiti Malaya. It considers how AI may be used not only to improve efficiency but also to support critical thinking, meaningful learning, and responsible decision-making. The presentation proposes several areas for further development, including AI-supported teaching, AI literacy, authentic assessment, the changing role of educators, and responsible data governance. It also highlights practical approaches to assessment design that encourage students to think, verify information, apply knowledge, and explain their work. Overall, the presentation emphasises the importance of using AI in ways that support students’ actual knowledge, skills, and learning development.

Generative AI is transforming higher education by enabling highly personalized learning experiences. This talk presents a GenAI-driven framework featuring an AI-powered WebDev Learning Assistant integrated into undergraduate curricula. Utilizing multimodal large language models, Canvas LMS integration, and real-time analytics, the platform provides adaptive learning paths and intelligent instructional support. Crucially, this system augments rather than replaces educators. By offering scalable academic support, continuous formative assessments, and data-driven insights, it empowers instructors to make timely interventions. Aligning with the conference theme, "Beyond Retrieval," we explore how AI transitions educational systems from traditional search methods to intelligent knowledge ecosystems that analyze, explain, and customize information for individual learner needs. The presentation emphasizes accessibility-by-design and the ethical, transparent use of AI, outlining future directions for holistic, real-time AI learning frameworks that will support the next era of advanced education.