How do I retrieve a list of files with wget?

To download multiple URLs, create a text file with the URLs, one per line, with no trailing spaces or line feeds. The URLs can be copied from the NetCDF Subset Service page and edited. For instructions on how to generate URLs, see the section titled “Where do I find the dataset and variable URLs?” Spreadsheets are handy for formatting and exporting large numbers of URLs.

Below are three example URLs that can be saved as a list, used here as an example file files.txt. Lists should not have trailing spaces or line feeds:

http://dataserver3.nccs.nasa.gov/thredds/ncss/bypass/NEX-GDDP/bcsd/historical/r1i1p1/tasmin/inmcm4.ncml?var=tasmin&horizStride=1&time_start=1950-01-01T12%3A00%3A00Z&time_end=1950-01-31T12%3A00%3A00Z&timeStride=1

http://dataserver3.nccs.nasa.gov/thredds/ncss/bypass/NEX-GDDP/bcsd/historical/r1i1p1/tasmin/inmcm4.ncml?var=tasmin&horizStride=1&time_start=1950-02-01T12%3A00%3A00Z&time_end=1950-02-28T12%3A00%3A00Z&timeStride=1

http://dataserver3.nccs.nasa.gov/thredds/ncss/bypass/NEX-GDDP/bcsd/historical/r1i1p1/tasmin/inmcm4.ncml?var=tasmin&horizStride=1&time_start=1950-03-01T12%3A00%3A00Z&time_end=1950-03-31T12%3A00%3A00Z&timeStride=1

The command to use a file of URLs, for example files.txt, is:

wget --content-disposition -i files.txt

Using –content-disposition writes an output file name in the form inmcm4.ncml.nc.

To use the -O option within a list to rename output files, use wget with a line reading utility, example:

while read -r line; do wget $line; done

Here is an example of the content of files.txt used with the -O option:

-O NEXbcsd-tasmin-195001.nc http://dataserver3.nccs.nasa.gov/thredds/ncss/bypass/NEX-GDDP/bcsd/historical/r1i1p1/tasmin/inmcm4.ncml?var=tasmin&horizStride=1&time_start=1950-01-01T12%3A00%3A00Z&time_end=1950-01-31T12%3A00%3A00Z&timeStride=1

-O NEXbcsd-tasmin-195002.nc http://dataserver3.nccs.nasa.gov/thredds/ncss/bypass/NEX-GDDP/bcsd/historical/r1i1p1/tasmin/inmcm4.ncml?var=tasmin&horizStride=1&time_start=1950-02-01T12%3A00%3A00Z&time_end=1950-02-28T12%3A00%3A00Z&timeStride=1

-O NEXbcsd-tasmin-195003.nc http://dataserver3.nccs.nasa.gov/thredds/ncss/bypass/NEX-GDDP/bcsd/historical/r1i1p1/tasmin/inmcm4.ncml?var=tasmin&horizStride=1&time_start=1950-03-01T12%3A00%3A00Z&time_end=1950-03-31T12%3A00%3A00Z&timeStride=1