Título: Assessing the Reliability of Explainable AI for Deep Neural Networks
Palestrante: Prof. Marco Zulich – Faculty of Technology, Policy, and Management – TU Delft, Delft, the Netherlands
Data e hora: Quarta-feira, 08/07, às 11h
Local: Sala 215 do IC/UFF
Abstract: Explainable AI (XAI) is often represented as a technology to enhance transparency on artificial neural networks (ANNs). However, the nature of XAI is often approximate, given its human-centric nature: explanations have to be simple enough to be understood by human stakeholders, depending on their level of expertise on the problem at hand. The compression of the predictive logic of arbitrarily complex models, such as ANNs, can cause explanations to be fundamentally unreliable; for instance, they can lack alignment with the model predictive dynamics or they can be highly sensitive to input and model perturbations. The functional assessment of explanations is an underexplored topic within XAI: indeed, these tools are often treated as oracles that can provide insights into the functioning of black-box models. But XAI, especially in socio-technical settings, should always be used with an eye on the stakeholders’ needs and regulatory frameworks, and attention should always be given to the assessment of explanations. Low-quality explanations can cause AI model failures (e.g., bias or cases of bad generalization) to fly under the radar, instilling a false sense of security in the stakeholders. Within this seminar, I will be giving a brief overview of the current state of the art for feature attribution, i.e., the scoring of the input features based on their importance in determining a model prediction; I will then proceed to describe the main framework for assessing feature attributions. I will conclude my seminar with an overview of my exploration into XAI evaluation and an outline of the main needs that different stakeholders of an AI system might have from XAI.
Bio:
Marco Zullich is an Assistant Professor in Trustworthy AI Systems at TU Delft (Netherlands), where he started working in 2026. He is currently teaching in programming- and AI-related courses and is the cluster coordinator for education in the Data & Digitalization domain.
His main field of research is Explainable AI, specifically the evaluation of explanations.
He conducts additional research and educational activities in other areas of Trustworthy AI, such as Uncertainty Quantification and Sustainability.
Prior to his employment in Delft, he was a lecturer at the AI Department at the University of Groningen (Netherlands) for three years.
He obtained a PhD in Information Engineering at the University of Trieste (Italy) in 2023 with a thesis on the topic of Neural Networks Compression.
Espero que todos e todas aproveitem a palestra!
Graduado em Ciências Atuariais pela Universidade Federal Fluminense (UFF) e Mestrando em IA no Instituto de Computação da UFF (nota máxima no CAPES). Palestrante e Professor de Inteligência Artificial e Linguagem de Programação; autor de livros, artigos e aplicativos.
Professor do Grupo de Trabalho em Inteligência Artificial da UFF (GT-IA/UFF) e do Laboratório de Inovação, Tecnologia e Sustentabilidade (LITS/UFF), entre outros projetos.
Proprietário dos projetos:
entre outros.
💫 Apaixonado pela vida, pelas amizades, pelas viagens, pelos sorrisos, pela praia, pelas baladas, pela natureza, pelo jazz e pela tecnologia.
