Learn from, and leverage, industry’s investment of time and capital to revolutionise eResearch capabilities.
Aotearoa’s eResearch community has a unique opportunity to leapfrog legacy approaches by adopting proven industry practices and tools in modern platforms. This session explores how implementing modern cloud-native approaches can transform research capabilities whilst embedding best practices for data and model governance and drive researcher engagement with self-service tools.
Drawing from industry experience in data engineering, data science, and platform engineering, across hybrid cloud-native environments, we outline practical steps for modernisation through four key areas:
- Platform Modernisation: Combining Kubernetes with HPC-specific schedulers like YuniKorn and high-performance storage solutions enables research teams to maintain HPC performance while gaining cloud-native flexibility.
- Automation: Implementing standardised patterns to data engineering, model deployment, monitoring, and lifecycle management ensures reproducibility and reliability. Modern tools enable streamlined version control and automated deployment processes in a cloud-native fashion.
- Workforce Development: Building cloud-native critical skills enables researchers to leverage industry best practices in platform engineering, data engineering, data science, and includes them into the wider cloud-native ecosystem and worldwide community. This enhances both individual career mobility between research and industry while strengthening research institutions’ capability for engagements with industry.
- Stewardship: Modern platforms enable granular control over location, access, and governance, ensuring sovereignty of choice and practices.
Practical examples will demonstrate:
- Converting traditional workloads and workflows to cloud-native approaches
- Implementing platforms with sovereignty considerations
- Establishing automated workflows leveraging proven industry patterns
- Optimising performance in hybrid environments
- Modern tools for researchers interested in utilising AI and machine learning
By adopting industry-hardened practices, Aotearoa’s research sector can build future-ready platforms and people that support innovation and ensure sovereignty of world-class research outcomes. This transformation creates an inclusive, collaborative research environment aligned with best practices, building resilience for our digital future.
Thanks to industry leaders, mentors, and the Aotearoa eResearch community for their commitment to advancement and collaboration. Special acknowledgement to those working to ensure research infrastructure supports inclusive and equitable access, and sovereign practices. Key references will include reference architectures, best practices, design patterns, and frameworks.
Citation: Admin, eRNZ (2025). Learning from Industry data science platforms: A leapfrog opportunity for eResearch in Aotearoa. eResearch NZ. Presentation. https://doi.org/10.6084/m9.figshare.28536623.v1
ABOUT THE AUTHOR
Adrian Torrie, of Ngāti Porou descent, is a skilled solutions architect with broad experience building and maintaining data systems from the ground up. He has broad skills across all IT domains, with specific depth in automating data analytics/machine learning/artificial intelligence solutions in hybrid environments, along with the necessary skills for training others in this specialty. Adrian’s last role had a primary focus of enabling self-service, via Platform Engineering, for Data Scientists and Data Engineers, allowing rapid iteration of models and their release. He excels in delivering secure solutions for large scale, high performance computational requirements using modern automation approaches.
https://www.linkedin.com/in/adriantorrie/
For more information about the eResearch NZ/eRangahau Aotearoa conference, visit:
https://eresearchnz.co.nz/
