Optimizing Data Delivery and Scalable HI Profile Classification for the SKA Era: Infrastructure and Science Challenges at the Spanish SRC

Add to your list(s) Download to your calendar using vCal

• Dr. Manu Parra-Royón (Astrophysics Institute of Andalucia - Spanish National Research Council)
• Tuesday 08 July 2025, 11:15-12:00
• Coffee area, Battcock Centre.

If you have a question about this talk, please contact Charles Walker.

This talk presents ongoing work at the Spanish SKA Regional Centre (esSRC) in the context of the SRC Net 0.1. The first part focuses on the development of efficient data delivery techniques from the distributed Rucio-based storage system to the SRC infrastructure and, ultimately, to user workspaces. Several approaches have been evaluated to support science-ready access, yet current solutions often involve unnecessary data duplication in user areas, resulting in increased usage of storage and computational resources. To address this, we have prototyped mechanisms based on file linking, caching, and data reuse, enabling more efficient access paths for users. While these methods show promising improvements in terms of performance and resource usage, challenges remain, particularly in terms of orchestration, scalability, and compatibility with existing workload managers. The second part presents advances in the automated classification of neutral hydrogen (HI) profiles using machine learning methods, building on previous work [Parra et al., 2024, arXiv:2501.11657]. We outline a roadmap for extending these techniques to handle the data volumes expected from the SKA Observatory. This includes developing scalable pipelines capable of ingesting and processing large spectral datasets in a reproducible and efficient manner, and adapting the classification models to cope with the diversity and complexity of the SKA data products.

This talk is part of the Hills Coffee Talks series.

This talk is included in these lists:

• All Cavendish Laboratory Seminars
• All Talks (aka the CURE list)
• Cambridge Astronomy Talks
• Cavendish Astrophysics Seminars
• Centre for Health Leadership and Enterprise
• Coffee area, Battcock Centre
• Combined External Astrophysics Talks DAMTP
• Cosmology, Astrophysics and General Relativity
• Featured lists
• Hills Coffee Talks
• ME Seminar
• Neurons, Fake News, DNA and your iPhone: The Mathematics of Information
• School of Physical Sciences
• Thin Film Magnetic Talks

Note that ex-directory lists are not shown.