![]() |
COOKIES: By using this website you agree that we can place Google Analytics Cookies on your device for performance monitoring. | ![]() |
University of Cambridge > Talks.cam > Hills Coffee Talks > Optimizing Data Delivery and Scalable HI Profile Classification for the SKA Era: Infrastructure and Science Challenges at the Spanish SRC
![]() Optimizing Data Delivery and Scalable HI Profile Classification for the SKA Era: Infrastructure and Science Challenges at the Spanish SRCAdd to your list(s) Download to your calendar using vCal
If you have a question about this talk, please contact Charles Walker. This talk presents ongoing work at the Spanish SKA Regional Centre (esSRC) in the context of the SRC Net 0.1. The first part focuses on the development of efficient data delivery techniques from the distributed Rucio-based storage system to the SRC infrastructure and, ultimately, to user workspaces. Several approaches have been evaluated to support science-ready access, yet current solutions often involve unnecessary data duplication in user areas, resulting in increased usage of storage and computational resources. To address this, we have prototyped mechanisms based on file linking, caching, and data reuse, enabling more efficient access paths for users. While these methods show promising improvements in terms of performance and resource usage, challenges remain, particularly in terms of orchestration, scalability, and compatibility with existing workload managers. The second part presents advances in the automated classification of neutral hydrogen (HI) profiles using machine learning methods, building on previous work [Parra et al., 2024, arXiv:2501.11657]. We outline a roadmap for extending these techniques to handle the data volumes expected from the SKA Observatory. This includes developing scalable pipelines capable of ingesting and processing large spectral datasets in a reproducible and efficient manner, and adapting the classification models to cope with the diversity and complexity of the SKA data products. This talk is part of the Hills Coffee Talks series. This talk is included in these lists:
Note that ex-directory lists are not shown. |
Other listsImagine2027 Multi-Agent Cooperation Sustainability Leadership LaboratoriesOther talksTheory of Phase Behaviour and Fluctuations in Polydisperse Systems: From Carbon-fiber Composites to High-performance Membranes Chalk talk Cambridge RNA Club - IN PERSON Barycenters and coycles on the Furstenberg boundary Lunch at Churchill College Diffusion modelling for amortised inference |