Initialization Commands#

When running COAWST, you can specify init_commands, a set of commands executed before compilation. These are useful for copying necessary files from the COAWST directory to your working directory.

init_commands = [
    # Copy LANDUSE.TBL for the simulation to ".".
    # Note that "." points to /workdir/output/artifacts/
    "cp /workdir/output/artifacts/__COAWST/LANDUSE.TBL ."
]

# Run simulation
task = coawst.run(
    input_dir="/Path/to/input_files",
    sim_config_filename="sim_config.in",
    build_coawst_script="build_coawst.sh",
    init_commands=init_commands,  # Pre-compilation commands
    n_vcpus=360,
    use_hwthread=True,
    on=cloud_machine
)

Objective#

In this simulation, we will run the JOE_TC/DiffGrid case using COAWST, where ROMS and SWAN share the same grid, while WRF runs on a different grid. This setup demonstrates how to configure and execute models with different grid resolutions within COAWST.

The goal is to show you how you need to configure your build script in order to run your simulation with us.

Prerequisites#

To get started, download the JOE_TC/DiffGrid project from the Official COAWST GitHub Repository.

You will also need the standard build_coawst.sh script which you can download from this link.

Lastly, download the three files (namelist.input, wrfbdy_d01 and wrfinput_d01) present in the JOE_TC folder.

Once downloaded, place all files inside a folder named JOE_TC_DiffGrid.

Your folder structure should now look like this:

-rwxr-xr-x@ 1 paulobarbosa  staff      3085 Feb 27 10:47 INPUT_JOE_TC_COARSE
-rwxr-xr-x@ 1 paulobarbosa  staff     15528 Feb 27 10:56 build_coawst.sh
-rwxr-xr-x@ 1 paulobarbosa  staff      7266 Feb 28 07:12 coupling_joe_tc.in
-rwxr-xr-x@ 1 paulobarbosa  staff      2978 Nov 11 20:45 joe_tc.h
-rwxr-xr-x@ 1 paulobarbosa  staff    116775 Nov 11 20:45 joe_tc_coarse_bathy.bot
-rwxr-xr-x@ 1 paulobarbosa  staff   1671068 Nov 11 20:45 joe_tc_coarse_grd.nc
-rwxr-xr-x@ 1 paulobarbosa  staff    195000 Nov 11 20:45 joe_tc_coarse_grid_coord.grd
-rwxr-xr-x@ 1 paulobarbosa  staff   4763160 Nov 11 20:45 joe_tc_coarse_ocean_init.nc
-rwxr-xr-x@ 1 paulobarbosa  staff      5132 Feb 28 07:19 namelist.input
-rwxr-xr-x@ 1 paulobarbosa  staff    167380 Feb 28 07:13 ocean_joe_tc_coarse.in
-rwxr-xr-x@ 1 paulobarbosa  staff  25403104 Nov 11 20:45 scrip_joe_tc_diffgrid.nc
-rwxr-xr-x@ 1 paulobarbosa  staff  46541404 Nov 11 20:45 wrfbdy_d01
-rwxr-xr-x@ 1 paulobarbosa  staff  70658632 Nov 11 20:45 wrfinput_d01

Updating Your Input Files#

In this section, we will update the input files (build_coawst.sh and .in files) to ensure they point to the correct paths.

Running Your Simulation#

Now that you’ve made all the necessary changes to the input files, it’s time to run your simulation. Below is the Python code you need:

"""COAWST Simulation."""
import inductiva

# Instantiate cloud machine
cloud_machine = inductiva.resources.MachineGroup(
    provider="GCP",
    machine_type="c2-standard-4"
)

# Initialize the simulator
coawst = inductiva.simulators.COAWST(
    version="3.8")

# Run simulation
task = coawst.run(
    input_dir="/Path/to/JOE_TC_DiffGrid",
    sim_config_filename="coupling_joe_tc.in",
    build_coawst_script="build_coawst.sh",
    n_vcpus=3,
    on=cloud_machine
)

task.wait()
cloud_machine.terminate()

task.download_outputs()
task.print_summary()

In this example, we’re using a relatively small cloud machine (c2-standard-4), which has 4 virtual CPUs. COAWST requires a strict core allocation for its simulations, meaning the number of CPUs must exactly match the simulation’s configuration. Here, the input files specify that the simulation must run on 3 cores, no more, no less (n_vcpus=3).

This is defined in the coupling_joe_tc.in file:

! Number of parallel nodes assigned to each model in the coupled system.
! Their sum must be equal to the total number of processors.

   NnodesATM =  1                    ! atmospheric model
   NnodesWAV =  1                    ! wave model
   NnodesOCN =  1                    ! ocean model
   NnodesHYD =  0                    ! hydrology model

Each component of the simulation is assigned a specific number of cores. We can increase this number later, but for now, we’ll keep it as is.

Once the simulation is complete, we terminate the machine, download the results and print a summary of the simulation as shown below.

inductiva tasks info 6wt3dp49uhy45y708x848eu2y
Task status: Success

Timeline:
	Waiting for Input         at 27/02, 19:32:00      7.963 s
	In Queue                  at 27/02, 19:32:08      15.545 s
	Preparing to Compute      at 27/02, 19:32:24      11.374 s
	In Progress               at 27/02, 19:32:35      36630.294 s
		├> 12.082 s        cp -r /opt/COAWST /workdir/output/artifacts/__COAWST
		├> 1.063 s         create_all_sim_links
		├> 1335.318 s      bash build_coawst.sh
		├> 35278.638 s     /opt/openmpi/4.1.6/bin/mpirun --use-hwthread-cpus --np 3 coawstM coupling_joe_tc.in
		├> 1.234 s         rm -r __COAWST
    └> 1.065 s         clean_all_sim_links
	Finalizing                at 28/02, 05:43:05      132.651 s
	Success                   at 28/02, 05:45:18      

Data:
	Size of zipped output:    4.45 GB
	Size of unzipped output:  4.79 GB
	Number of output files:   43

Estimated computation cost (US$): 0.71 US$

Go to https://console.inductiva.ai/tasks/6wt3dp49uhy45y708x848eu2y for more details.

The simulation details may seem overwhelming at first, but let’s focus on the In Progress stage, as this is the part unique to your simulation. All other steps are common to every simulation run on Inductiva.

Understanding the In Progress steps#

The “In Progress” section details the commands executed during your simulation, along with their durations. Below is a breakdown of the key steps:

• cp -r /opt/COAWST /workdir/output/artifacts/__COAWST

  • Copies the COAWST directory into the input files directory for you to compile and have access to all files related to COAWST.

• create_all_sim_links

  • Creates symbolic links (shortcuts to other files) in the working directory for all COAWST files. We do this because some simulations require specific files (e.g., CAMtr_volume_mixing_ratio, LANDUSE.TBL) to be present in the working directory. By creating this symbolic links we avoid having to send all those files with the input.

  • If a file with the same name already exists, the link is skipped, allowing you to provide your own version of the file. For example, if you include your own LANDUSE.TBL file in the input, it will be used for the simulation instead of the default version from the COAWST folder.

• bash build_coawst.sh

  • Runs your provided script to build COAWST.

  • This script is executed inside /workdir/output/artifacts, and COAWST is located in /workdir/output/artifacts/__COAWST.

• coawstM coupling_joe_tc.in

  • Runs the simulation in parallel.

• rm -r __COAWST

  • Cleans up the simulation directory by removing the COAWST folder, keeping the output size minimal.

• clean_all_sim_links

  • Cleans all symbolic links created with the create_all_sim_links commands.

With these steps, your COAWST simulation is successfully executed and managed in the cloud. 🚀

Scaling Up Your Simulation#

Looking at the execution times, we can see that the compilation took 1,335 seconds (around 22 minutes), while the simulation itself ran for 35,278 seconds (approximately 9 hours and 47 minutes).

It’s important to note that compilation time varies depending on the COAWST configuration you choose. However, the simulation runtime is primarily influenced by the computational resources allocated—including the number of virtual CPUs.

In this section, we’ll explore strategies to scale up your simulation, in order to reduce the simulation time.