TERRACES Environment Simulations and Research

Dr Jed O. Kaplan leads the TERRACES Research Group in the Department of Earth Sciences at The University of Hong Kong, where he and his team study the role of land cover influencing climate change. Specifically, Dr Kaplan and his team examine how climate change affects land use and land cover, and how these in turn affect hydrological cycles, soil, atmosphere, and the built environment including interactions and feedbacks among these components. Dr Kaplan’s group tackle some of the major unsolved problems in understanding global environmental change in the past, present, and future, including the role of climate change on wildfires, the potential of tropical forests to draw down CO₂ from the atmosphere, and how deforestation affects the frequency of drought. For more than 25 years, Dr Kaplan has worked on the development and application numerical simulation models of the global land surface that are used to address these questions.

Dr Kaplan’s research work is highly compute intensive, involving multiple datasets and large scale simulations. Typical projects can run to 500 or more simulations, with subtle variations in assumptions and variables. Each simulation results in 3-10 GB of data across climate and surface vegetation models, with entire project results being 5TB or more. To analyse these very large data sets, the team uses advanced statistical analyses, Geographic Information Systems and 3D visualisations, and a range of custom software to understand the key drivers and consequences of global change.

In recent decades, computer simulation modelling of global environmental change has only become more complex, with more variables and higher spatial and temporal resolution, leading to larger datasets to analyze and archive. With Dr Kaplan’s research team growing rapidly, his group could be processing up to 50TB of active, work in progress data. This could easily double in the coming years as simulations grow in size and complexity.

New HPC Cluster

When Dr Kaplan arrived at The University of Hong Kong in 2019, this work on environmental modelling and simulations was a new research field for the University. While an existing High Performance Computing cluster was in place at the HKU, it was in high demand across the institution. Fortunately, the University supported Dr Kaplan and the TERRACES team with an investment in a new HPC cluster for the team.

From previous experience, Dr Kaplan knew exactly what he needed, as he explained the team required “a small/medium high performance computing cluster – with the best compute capability for the budget, and a requirement for enough high performance data storage capacity to handle the simulations”. Functionally, the HPC cluster had to be able to run model simulations, perform statistical analysis of the results, and prepare visualisations for a team of up to 10-12 researchers. Dr Kaplan was also looking for a modular design that would allow for easy addition of more compute or more storage as required in the future.

As The University of Hong Kong went to market for the HPC cluster, Dr Kaplan was looking for an IT partner to provide the latest advice and experience in HPC technology and design. In the market, Dr Kaplan found that his HPC cluster project “was too small for the global vendors, and too large for the local resellers”. XENON, partnering with local reseller ModernTech, responded to the tender and Dr Kaplan liked their approach, noting that the XENON team was “really able to listen to our requirements and help us get to a solution.”

CLUSTER DESIGN

Scalable Software Stack

Dr Kaplan’s new cluster runs XENON Cluster Stack (XCS) – XENON’s own open source, containerised HPC management system. This containerised architecture utilises open source tools such as Linux, Docker and Singularity to build up a complete cluster operating and management system. Because all components of XCS are open source, this is a very cost effective HPC management system, and ensures longevity of the solution with an easy to update, easy to expand, hardware-agnostic software layer.

The XENON Cluster Stack also provides a high degree of security through design, while also allowing shared access to be contained and controlled. This has been important for Dr Kaplan’s team as they collaborate with colleagues abroad and distribute large datasets to the scientific community. XENON  implemented shared access and publishing tools including WordPress on XCS, nextcloud, and a git LFS server to facilitate sharing information and hosting large datasets.

Seamless Delivery

Download the HKU Case Study

A unique aspect of the project was that the teams across HKU, XENON and ModernTech worked remotely. This commenced even before COVID made remote collaboration the “new normal”. XENON was able to work with Dr Kaplan to progressively architect the solution that fully met his team’s requirements, as he said, “zeroing in on a solution that really worked.”

The delivery of the solution was a seamless positive experience as well. The XENON team stood up the cluster in the XENON production facility in Melbourne, fully testing all components, configuring the XENON Cluster Stack, and installing the specific application software required by the TERRACES team. The HKU technical team was able to remotely access the cluster to test the operation and performance. Once it was fully tested the cluster was packed up and shipped to Hong Kong.

In Hong Kong, ModernTech took over and installed the cluster in the HKU data centre. Within a week following delivery, the cluster was up and operational and has had zero downtime in the more than six months it has been in operation.

These results have exceeded Dr Kaplan’s expectations. “We are super pleased with the process and the product. XENON managed to do it all remotely and ModernTech delivered on the ground here in Hong Kong. This cluster is a key tool in the exciting science that the TERRACES team is undertaking. We look forward to years of use of the cluster and the ability to expand it as the team and research demands grow”, Kaplan noted.

The TERRACES Research Group in the Department of Earth Sciences at the University of Hong Kong studies the role of land cover influencing climate change. Learn more about the TERRACES group and Dr Kaplan’s work at http://web.hku.hk/~jkaplan/

Get a QuoteTalk to a Solutions Architect