Modules section

book/modules-software.md

@@ -3,7 +3,8 @@
 In this session we will learn about software on Aire, and how to access software via the module system. We will also discuss some alternatives to install software yourself on the system.
 
 `````{tab-set}
-````{tab-item} Introduction to Modules
+````{tab-item} 1. Intro
+
 - **What are Modules?**
   - Modules are a way to manage different software environments on HPC systems.
   - They allow users to load and unload software packages dynamically.
@@ -25,7 +26,7 @@ module avail
 ::::
 ````
 
-````{tab-item} Using Modules
+````{tab-item} 2. Loading modules
 - **Loading a Module:**
   - To use a software package, load its module.
   - Example:
@@ -46,6 +47,10 @@ module avail
     module list
     ```
 
+````
+
+````{tab-item} 2.1 Ex.
+
 :::: exercise
 **Exercise 2:** Load the `gcc` module and verify it's loaded.
 ::::
@@ -55,13 +60,60 @@ module avail
 
 :::: solution
 ```bash
+#!/bin/bash
+
 module load gcc
 module list
+
 ```
 ::::
 ````
 
-````{tab-item} Requesting New Modules
+````{tab-item} 3. Using modules
+
+In the previous exercise, we saw how to write a bash script that loads a particular module.
+
+Say we have a Python file called `hello_world.py`:
+
+```python
+print("hello world!")
+```
+
+How would we write a bash script that loads the Python module and runs the Python script?
+
+````
+
+````{tab-item} 3.1 Soln.
+
+We want to write a bash script that runs the following Python script (`hello_world.py`):
+
+```python
+print("hello world!")
+```
+
+A simple script could look like this:
+
+```bash
+#!/bin/bash
+
+module load python/3.13.0
+python hello_world.py
+
+```
+
+After creating this file, called `python_test.sh` you will need to add executable permissions to it: `chmod +x python_test.sh`. Use `ls -F` to check that it is executable.
+
+To run the script:
+
+```bash
+./python_test.sh
+```
+
+Note: when running Python jobs, we would recommend that instead of using the basic Python install, to instead use the Miniforge module to create a conda environment. [Read our documentation on how to set this up](https://arcdocs.leeds.ac.uk/aire/usage/dependency_management.html)
+
+````
+
+````{tab-item} 4. Requesting New Modules
 - **Centralized Management:**
   - Popular software is centrally installed by the Research Computing team.
   - Ensures optimized performance and avoids conflicts.
@@ -71,15 +123,15 @@ module list
   - Submit a Research Computing Query with details about the software.
 
 :::: exercise
-**Exercise 3:** Identify a software you need that's not available and draft a request. Can you find the Research Computing Query form via the IT website?
+**After the course:** Identify a software you need that's not available and draft a request. Can you find the Research Computing Query form via the IT website? Try to locate it through the IT website (*Home > Research IT > Research IT Query*), but if you're stuck, [here's a link to the request form](https://bit.ly/arc-help)
 ::::
 
 :::: solution
 Provide the software name, version, and a brief justification for its use in your research.
 ::::
 ````
 
-````{tab-item} Alternative Software Management
+````{tab-item} 5. Alternative Software Management
 
 You also can manage your own software on Aire via a number of different routes. Many users will not need to do this; but it may be neccessary if you want fine-grained control and need to use older versions of software than are available via the module system.
 
@@ -101,7 +153,7 @@ You also can manage your own software on Aire via a number of different routes.
   - Ensures portability and consistency across systems.
 
 :::: exercise
-**Exercise 4:** Choose a simple software package and outline steps to install it using Spack.
+**After this course:** Choose a simple software package and outline steps to install it using Spack.
 ::::
 
 :::: solution

book/storage.md

@@ -13,7 +13,9 @@ In this session, we will work to understand the main file areas on Aire (home, s
 - Symlinks exist: `/scratch` → `/mnt/scratch`, and `/flash` → `/mnt/flash`.  
 - Home is backed up and not automatically purged; scratch and flash are not backed up and flash is deleted after each job.  
 - **Always** copy important results from temporary storage (flash or node-local) back to your home directory or another permanent area **before** the job ends.
+````
 
+````{tab-item} 1.1 Ex.
 
 ::: important  
 **Exercise:** On the Aire login node, display the values of `$HOME` and `$SCRATCH` using `echo`.  
@@ -53,7 +55,9 @@ It's important for you to be able to check how much of your allocated quota you
   - Add the flag `-s` to summarise the results (and not recursively provide results for subdirectories).
   - `du -hs *` will provide you with the space taken up by the data in each directory. This can be a little slow!
 
+````
 
+````{tab-item} 3.3 Ex.
 
 ::: important  
 **Exercise:** Check your current disk usage and quotas. *(Hint: use the `quota` command.)*  
@@ -80,27 +84,62 @@ Disk quotas for user yourusername (uid 12345):
 - Remove unneeded files with `rm` to free space. After cleanup, re-check your usage with `quota -s` or `du` as needed.  
 ````
 
+````{tab-item} 4.4 Ex.
+
+::: important  
+**Exercise:** Navigate around the file system using `cd directory_path`, and `cd` to return home.
+
+- Try to use the environment variables (like `$SCRATCH`)
+- Use `pwd` to check where you are in the directory structure, and `ls` to see what other files are present
+
+:::
+
+````
+
 ````{tab-item} 5. Transferring Files
 
 - **scp** (secure copy) to/from the login node. For example, from **your local machine** to Aire:  
   ```bash
   scp myfile.txt <username>@target-system:$SCRATCH/
   ```  
-  And to copy a file **from** Aire to local:  
+  And to copy a file **from** Aire to local (current directory):  
   ```bash
-  scp <username>@target-system:/users/yourname/results.txt .
+  scp <username>@target-system:/path/results.txt .
   ```  
 - **rsync** for efficient transfers (especially many files). Example:  
   ```bash
-  rsync -avh data/ <username>@target-system:$HOME/data/
-  ```  
+  rsync -avh data/ <username>@target-system:path/data/
+  ``` 
+
 - **wget/curl** on the login node to download from a URL. For instance:  
   ```bash
   wget https://example.com/data.zip
   ```  
 - **GUI/SFTP clients:** Use FileZilla, WinSCP or Cyberduck to connect to Aire via SFTP.  
 - **Best practice:** Transfer all needed input data to `$SCRATCH` *before* running jobs, and copy results out of `$SCRATCH` after jobs finish.
 
+````
+
+````{tab-item} 5.1 Transferring Files
+
+- From off campus, you will have to use the usual tunnelling/jumphost commands.
+  - rsync:
+    ```bash
+    rsync -r --info=progress2 -e 'ssh -J e<username>@jump-host' file_to_copy.txt <username>@target-system:path/
+    ```
+    - `--info=progress2` gives you a progress bar for the upload/download
+    - `-e ssh` allows you to set up a remote ssh shell (to use the jumphost)
+  - scp:
+    ```bash
+    scp -rq -J <username>@jump-host <username>@target-system:path/file-to-copy.txt local-folder-path/
+    ```
+    ```bash
+    scp -rq -J <username>@jump-host local-folder-path/file-to-copy.txt <username>@target-system:path/
+    ```
+````
+
+````{tab-item} 5.2 Ex.
+
 ::: important  
 **Exercise:** On your **local machine**, write the `scp` command to download `results.txt` from your home directory on Aire into your current local directory.  
 :::