The HPC team is updating this page. Check back for new information.
Introduction
Globus is an easy-to-use, high-performance data transfer tool developed by the Computation Institute, which is a partnership between The University of Chicago and Argonne National Laboratory. UConn has deployed Globus as one mechanism to facilitate data transfer to and from the Storrs HPC cluster. You can also use Globus to transfer data between external institutions and UConn, including to and from your personal machine. Data can be transferred using a convenient web interface or a command line interface.
After you initiate a data transfer through the Globus interface, UConn's Globus node takes over and does the work on your behalf. You can navigate away from the Globus web page, close the browser window, and even logout. Globus will optimize the transfer for performance, monitor the transfer for completion and correctness, and recover from network errors and endpoint resource downtime. The Globus service is highly reliable, providing nearly uninterrupted oversight of data transfers. If a problem is encountered part-way through a transfer, Globus resumes from the point of failure and does not retransmit all of the data specified in the original request. Globus can handle extremely large data transfers, even those that do not complete within the authentication expiration period of an endpoint. If your credentials expire before a transfer completes, Globus will notify you to re-authenticate on the endpoint so that it can continue the transfer. These wide-ranging capabilities make data transfer with Globus truly "fire and forget".
| Table of Contents | ||||
|---|---|---|---|---|
|
Get Started
Globus Web App is accessible through any web browser. In order to access Globus Web App, you must first create an account through the Globus Auth service. Globus Auth is integrated with UConn NetID, so the process is quick and convenient.
...
The instructions below demonstrate how to move data from the /shared file system to the /archive file system. The same instructions could be used to transfer data between any two file systems which are permanently mounted on the cluster. UConn's Globus node has access to the following file systems:
Path | Description |
|---|---|
| User home directories |
| Group shares |
| Long term data storage (deprecated) |
| High-performance for temporary files |
Go to the Globus Web App. Select University of Connecticut from the Organization dropdown, then . Then authenticate using your NetID and password.
On the left, click the File Manager link.
Inside the Collection box, search for and select
dtn2.storrs.hpc.uconn.edu, which is the identifier for UConn's Storrs HPC Globus server.If you haven't accessed Globus in a while, you will be asked to login with your UConn username/password. Under Please authenticate to access this endpoint enter your NetID next to Username, and then your NetID password:.
In the left pane, next to Path, enter the absolute path of the folder containing the files you wish to transfer to another location. For example,
/shared/mylab.On the right pane, click inside the Search box. Search for and select the endpoint you wish to transfer files to.
In the right pane, next to Path, enter the absolute path of the folder to which you would like to transfer the data selected in the left pane. For example,
/archive/mylab.In the left pane, select the files to transfer and click Start to move the data from the location on the left, to the location on the right.
You can click on the Activity tab at the top of the window to monitor the progress of your data migration. You will also receive an email alert when the migration is complete.
...
To transfer To the dtn2.storrs.hpc.uconn.edu endpoint, reverse the source and destination in the above steps.
Transfer Data To or From Your Personal Computer
...
Credentials expire after 24 hours by default. You should be able to extend your credentials to a maximum of 7 days. To do so, follow these steps after logging onto the Globus GUI:
Click the "Endpoints
...
link at the top of the page
Click "Search All"
Search for "uconn"
Select "dtn2.storrs.hpc.uconn.edu" from the resulting list
Go to the "Extend Activation" tab
Click "Reactivate Endpoint" (or "Activate Endpoint if you haven't already logged onto it)
Enter your credentials and then click the "Advanced" link below that . In the "Credential Lifetime (hours)" field, enter 168
Click Authenticate
Your credentials should now be active for 7 days. This is the maximum time that Globus will allow credentials to remain active.
Command Line Interface
Globus also has a command line interface that you can use with SSH keys which allows transfers to be scripted.
...
Login to the Globus Web App using your Globus ID. To do so, visit this page: https://www.globus.org/app/transfer . Click on the link Then use Globus ID to sign in.
Enter your Globus ID username and password and click Log in:.
Click on the Account link at the top right of the Globus Web App:.
Click Add Linked Identity:.
In the search box, enter University of Connecticut and click Continue:.
Finally, you will be redirected to UConn's NetID login web page, as shown in Figure 13. Enter your NetID and password, and then click Login. Your UConn NetID will now be linked to your Globus ID.
Setting up a Lab Machine/Instrument or Shared Computer
| Warning |
|---|
Sharing NetID passwords raises significant security concerns for the user and their data. |
If a machine or instrument that is shared between many users need a globus endpoint set up for transfer to Storrs HPC or elsewhere, it is better to create a separate email and globus account to create the local end point.
Example:
Create a new gmail for lab machine:
labmachine3@gmail.com
Log in to globus web with lab machine email
Install (or resintall) globus personal connect and log in with lab machine email
name the local endpoint “UConn Lab Machine 3”
All users of the lab machine should now have access to the local globus endpoint and transfer files to where it is needed.
If they wanted to transfer to Storrs HPC, they can find the http://dtn2.storrs.hpc.uconn.edu/ and login with their Netid to proceed.