The Discover cluster provides several different types of file systems: home, nobackup, and temporary/scratch. See the showquota documentation for information on how to monitor your storage usage.
| File System | Type | Variable on Discover cluster | Default Quota | Backup Cycles |
|---|---|---|---|---|
| Home Directory | IBM GPFS | $HOME | 1GB | Daily |
| Scratch | IBM GPFS | $NOBACKUP | 5Tb/300k inodes | No Backups |
| Scratch | local | $LOCAL_TMPDIR | node specific | No backups, data is deleted at the end of the job or session |
| Scratch | GPFS | $TMPDIR | Same as NOBACKUP | No backups, data is deleted at the end of the job or session |
| Fast Access Scratch | GPFS | $TSE_TMPDIR | 250GB, 200k inodes | No backups, data is deleted at the end of the job or session |
Best Practices
- Scratch filesystems $LOCAL_TMPDIR and $TMPDIR are node specific file systems, users may want to be careful using them as they may store files/data on them and lose it when they log onto the Discover cluster, but on a different node.
- Data in $TSE_TMPDIR is automatically deleted when your job ends because storage on this resource is extremely limited.
- ALWAYS use the symlinks, e.g, $HOME, $NOBACKUP, /home, or /discover/nobackup, in your scripts to specify paths. NEVER use the real path, /gpfsm/dnbxx, because it could be changed due to disk augmentations or system events.
- Adding packages to a Python or Conda environment in $HOME tends to use a lot of temporary data in the .conda, .cache, and .local directories which can quickly exceed the user $HOME quota. To resolve this, move these directories to $NOBACKUP and then create a soft link to them from $HOME. Example:
mv $HOME/.conda $NOBACKUP
ln -s $NOBACKUP/.conda $HOME/.condaRepeat these commands for the .cache and .local directories as well.
- For users of the GEOS software, please read the GMAO’s guidance on how to control GEOS output, both to learn how to reduce the number of data collections saved and how to create a custom data collection to eliminate the need to write unneeded data to disk or tape.
Accessing CSS through Slurm
CSS read-only access on Discover is provided to a subset of Discover’s Slurm-managed compute nodes.