Handson with Gadi

Overview

This workshop provides an introduction to basic command-line operations and guidelines for working efficiently on Gadi, the high-performance computing (HPC) machine at the National Computational Infrastructure (NCI).

Learning Outcomes

By the end of this course, participants will be able to:

1. Navigate Gadi’s Architecture: Identify the key components of the Gadi supercomputer and select the appropriate file system (scratch, home, or project) for different stages of their project.

2. Operate in a Remote HPC Environment: Confidently use the Australian Research Environment (ARE) to access Gadi via web-based interfaces, including Jupyter Notebooks and Virtual Desktops.

3. Use the Command Line Interface: Perform essential remote file management and text editing using Linux commands tailored for a high-performance computing context.

4. Manage Computational Jobs: Write and submit batch scripts to the job scheduler, and differentiate between interactive and batch modes to optimize resource usage.

5. Utilise Specialised Hardware: Configure ARE sessions to request and utilise GPU resources for accelerated computing tasks in Jupyter and desktop environments.

6. Apply HPC Best Practices: Navigate NCI’s specific queues and resource limits to ensure efficient job execution and collaboration within the Australian research ecosystem.

Prerequisites

1. Know how to use a computer.

2. Familiar with Linux command line interface is recommended, but we will cover the basics in the workshop.

Suggested Workshop Agenda

Topics	Duration
Introduction to HPC, NCI and Gadi	20 minutes
Logging in to Gadi	10 minutes
Navigating Gadi File System	30 minutes
Modules	10 minutes
Using Python on Gadi	10 minutes
Requesting Resources	30 minutes
Run Jupyter Notebooks on Gadi	20 minutes
Virtual Desktop in ARE	10 minutes
Total	140 minutes

Contents

Before the Workshop

• Access and Setup

Tutorials

• 1. Introduction to HPC, NCI and Gadi
  • 1.1. Why HPC?
  • 1.2. What is HPC?
  • 1.3. Differences from Desktop Computing
  • 1.4. Gadi
• 2. Logging in to Gadi
  • 2.1. By Australian Research Environment (ARE)
  • 2.2. Practice: Login to ARE and open a terminal
  • 2.3. By Terminal
• 3. Navigating Gadi File System
  • 3.1. Gadi File System
  • 3.2. Understanding your location (pwd)
  • 3.3. Listing files (ls)
  • 3.4. Practice: Find your bearings
  • 3.5. Changing directory (cd)
  • 3.6. Practice: Explore shared locations
  • 3.7. Managing files and directories
  • 3.8. Viewing and editing text
  • 3.9. Practice: Create, inspect, tidy up
  • 3.10. Copying files to and from Gadi (scp)
  • 3.11. Practice: Prepare the code for your first job on Gadi
• 4. Modules
  • 4.1. Module Commands
  • 4.2. Practice: Load the openmpi module
• 5. Using Python on Gadi
  • 5.1. Create a Python Virtual Environment on Gadi
• 6. Requesting Resources
  • 6.1. Planning Jobs
  • 6.2. Running Batch Jobs
  • 6.3. Practice: Write a PBS job script
  • 6.4. Submit and Monitor Jobs
  • 6.5. Practice: Submit and Monitor Your First Job
  • 6.6. Interactive Jobs
• 7. Run Jupyter Notebooks on Gadi
  • 7.1. Cluster Access
  • 7.2. Launch JupyterLab Session on ARE
  • 7.3. ARE FAQs
  • 7.4. JupyterLab Usage
  • 7.5. JupyterLab FAQs
• 8. Virtual Desktop in ARE
  • 8.1. Requesting GPUs

References

• Reference