This article lists the memory limits for jobs launched through the Slurm job manager, and suggests how to request memory resources for different types of jobs.
There are two ways of specifying memory requirements in Slurm: demanding memory per core or demanding total memory per node.
Total memory per node is demanded with option --mem=size[unit], where unit is one of M for megabytes, G for gigabytes or T for terabytes.
sbatch --mem=48G [other parameters] your_script.slm # Request a total memory of 48 GB per node sbatch --mem=1T [other parameters] your_script.slm # Request a total memory of 1 TB per node |
If no unit is specified, the value is interpreted as a number of megabytes.
The upper limit for this value is the total memory of the node (see table above). Note that if more than one node is requested, it is not possible to demand different memory sizes for different nodes – the same memory size will be requested in each node.
Memory per core is demanded with option --mem-per-cpu=size[unit], where again unit is one of M for megabytes, G for gigabytes or T for terabytes.
sbatch --mem-per-cpu=6G [other parameters] # Request a total memory of 6 GB per core |
If the value requested for this option is above the per-core memory limit of the partition (see table above), --cpus-per-task is automatically defined so that enough cores are assigned to each task to satisfy the memory requirement: for instance, if a job sent to a thin node of the std partition (48 cores, max. 4 GB per core) specifies -n 4 and --mem-per-cpu=8G, it will be interpreted as a job with 4 tasks, each running on 2 cores with 4GB per core.
As a final note, the Slurm implementation of memory limits relies on Resident Set Size (RSS); if you need to fine-tune your program's memory requirements, you might be interested in reading more about how that metric is computed.
Remember that jobs sent to the gpu partition are always assigned whole 24-core sockets, and jobs sent to the knl partition are always allotted whole nodes. Jobs sent to the std, std-fat and mem partitions, on the other hand, are free to specify any number of cores, and allow for greater flexibility. |
Related articles appear here based on the labels you select. Click to edit the macro and add or change labels.
|