cc_staff
65
edits
General directives for migration: Difference between revisions
Jump to navigation
Jump to search
General directives for migration (view source)
Revision as of 20:46, 24 November 2023
, 5 months agoupdate globus settings (file integrity is enabled by default)
No edit summary |
(update globus settings (file integrity is enabled by default)) |
||
| Line 38: | Line 38: | ||
== What to do during the migration process == <!--T:12--> | == What to do during the migration process == <!--T:12--> | ||
If it is supported at your source site, use [[Globus|Globus | If it is supported at your source site, use [[Globus|Globus]] to set up your file transfer. It is the most user-friendly and efficient tool we know for this task. Globus is designed to recover from network interruptions automatically. We recommend you enable the setting to "preserve source file modification times" in the "Transfer & Timer Options". | ||
<!--T:13--> | <!--T:13--> | ||
If Globus is not supported at your source site, then | If Globus is not supported at your source site, then compressing data and avoiding duplication is even more important. If you are using [https://en.wikipedia.org/wiki/Secure_copy scp], [https://en.wikipedia.org/wiki/SSH_File_Transfer_Protocol sftp], or [https://en.wikipedia.org/wiki/Rsync rsync], then: | ||
* Make a schedule to migrate your data in blocks of a few hundred GBs at a time. If the transfer stops for some reason, you will be able to try again starting from the incomplete file, but you will not have to re-transfer files that are already complete. An organized list of files will help here. | * Make a schedule to migrate your data in blocks of a few hundred GBs at a time. If the transfer stops for some reason, you will be able to try again starting from the incomplete file, but you will not have to re-transfer files that are already complete. An organized list of files will help here. | ||
* Check regularly to see that the transfer process has not stopped. File size is a good indicator of progress. If no files have changed size for several minutes, then something may have gone wrong. If restarting the transfer does not work, contact our [[Technical support]]. | * Check regularly to see that the transfer process has not stopped. File size is a good indicator of progress. If no files have changed size for several minutes, then something may have gone wrong. If restarting the transfer does not work, contact our [[Technical support]]. | ||
<!--T:14--> | <!--T:14--> | ||
Be patient. Even with Globus, transferring large volumes of data can be time consuming. Specific transfer speeds will vary | Be patient. Even with Globus, transferring large volumes of data can be time consuming. Specific transfer speeds will vary, but expect hundreds of gigabytes to take hours and terabytes to take days. | ||
== What to do after migration == <!--T:15--> | == What to do after migration == <!--T:15--> | ||