Transferring data: Difference between revisions

The standard tool for downloading data from websites is [https://en.wikipedia.org/wiki/Wget wget]. Another tool commonly used is [https://curl.haxx.se/ curl]. The usage differences of these two tools is compared in many places such as this [https://unix.stackexchange.com/questions/47434/what-is-the-difference-between-curl-and-wget StackExchange article] or this [https://draculaservers.com/tutorials/wget-and-curl-for-files/ complete guide].  Although the focus here is usage on alliance linux systems [https://www.techtarget.com/searchnetworking/tutorial/Use-cURL-and-Wget-to-download-network-files-from-CLI this article] also addresses Mac and Windows users.  An important aspect is both [https://www.thegeekstuff.com/2009/09/the-ultimate-wget-download-guide-with-15-awesome-examples/ wget] and [https://www.thegeekstuff.com/2012/04/curl-examples/ curl] can continue/resume interrupted downloads of files using their respective [https://www.cyberciti.biz/tips/wget-resume-broken-download.html -c] or [https://www.cyberciti.biz/faq/curl-command-resume-broken-download/ -C -] command line options. For getting data from various cloud services such as Google cloud storage, Google Drive and Google Photos, consider using the [https://rclone.org/ rclone] tool instead. All tools (wget, curl, rclone) are available on our clusters without loading a module.  A detailed listing of command line options can be found by checking their man pages or running each command with the <code>--help</code> argument.