Resilient model training on Red Hat OpenShift AI with Kubeflow Trainer