Transferring data: Difference between revisions

The standard tool for downloading data from websites is [https://en.wikipedia.org/wiki/Wget wget]. Another often used is [https://curl.haxx.se/ curl]. Their similarities and differences are compared in several places such as this StackExchange [https://unix.stackexchange.com/questions/47434/what-is-the-difference-between-curl-and-wget article] or [https://draculaservers.com/tutorials/wget-and-curl-for-files/ here].  While the focus here is transferring date on alliance linux systems this [https://www.techtarget.com/searchnetworking/tutorial/Use-cURL-and-Wget-to-download-network-files-from-CLI tutorial] also addresses Mac and Windows machines.  Both [https://www.thegeekstuff.com/2009/09/the-ultimate-wget-download-guide-with-15-awesome-examples/ wget] and [https://www.thegeekstuff.com/2012/04/curl-examples/ curl] can resume interrupted downloads by rerunning them with the [https://www.cyberciti.biz/tips/wget-resume-broken-download.html -c] and [https://www.cyberciti.biz/faq/curl-command-resume-broken-download/ -C -] command line options respectively. When getting data from various cloud services such as Google cloud storage, Google Drive and Google Photos, consider using the [https://rclone.org/ rclone] tool instead. All of these tools (wget, curl, rclone) are available on the alliance clusters by default (without loading a module).  A detailed listing of command line options can be obtained from their man pages or by running each command with <code>--help</code> or more simply <code>-h</code>.