Tell Hail to make different use of Spark processing


I would like to know if there is any option to tell Spark to run just on disk or use a different ratio of in-memory processing. Any way to easily perform benchmarking ?


Hi Pedro,
Spark configuration is pretty tough, and we’re not especially good at it (we’re also not sure who is). You can change the memory settings here:

Regarding benchmarking, we’re building a benchmarking suite but it’s not at a point where it’s especially useful beyond our development work.