This paper is directly based on an EU Horizon research proposal we had submitted in 2022 for compute continuum platform that employs Kubernetes to operate a combined platform consisting of Cloud, HPC and Edge environments.
Achieving this, according to the design we presented in the paper, involves employing the cloud environment to coordinate the others.
While there are projects available to integrate IoT devices with Kubernetes operating on edge servers, HPC opens the question whether to operate these through Kubernetes as well or whether the existing batch scheduling systems should be integrated with Kubernetes.
The second case is preferable as it enables extending the cloud continuum into HPC environments without disrupting said environments such that they can be employed as normal in addition to being part of the cloud continuum.
However, depending on the control required and the nature of workloads to be run on the HPC nodes, this may prove difficult.

For instance, if given the ability to submit workloads to an HPC environment from the Kubernetes-based cloud environment, this does not mean that there exists an understanding of the current load on the HPC environment in the Kubernetes environment.
Additionally, by default, HPC environments are not container-based and cannot directly execute Kubernetes workloads without converting them or setting up a Kubernetes platform on top of the HPC environment.

These are among the challenges that this project faces and must solve.
For more information see the respective webpage or the paper here: https://doi.org/10.1145/3587135.3592179

Next Post Previous Post