Hi, I was wondering how people set up their HPC when they work with HAIL. So far, I’ve always requested a whole node (128 CPUs with 2TB of RAM) when I work with hail. I am wondering how I can set up HAIL to work with multiple nodes. I’ve looked into magpie, which integrates SPARK and SLURM, but it seems I need to use nodes as a whole (--exclusive). I don’t want to block a whole node just for the SPARK master. Does anybody wants to share their insight if they have a similar setup?
Thanks!