Manage and Retrieve Results#
The output results of any simulation are the focal point of the entire simulation process. It is, therefore, important to adequately manage and retrieve these outputs in a way that is both efficient and organized. In this section, we will show how to configure where the simulation outputs are saved in the remote storage space and how to retrieve them to the user’s local machine. In this document, we will cover the following topics:
Saving simulation outputs: Learn how to configure where the outputs of a simulation are stored in the user’s remote storage;
Downloading outputs: Learn how to download the outputs of a task and control where the files are downloaded to.
Most of the topics that follow rely on the Task
class. Please refer to the
section about Tasks on our tutorials
for more information about it.
Saving simulation outputs#
By default, the outputs generated by a simulation are saved to a directory named
after the task id. In the following snippet, the outputs of task with id
i4ir3kvv62odsfrhko4y8w2an
are saved to a directory named
i4ir3kvv62odsfrhko4y8w2an
:
import inductiva
# Instantiate machine group
machine_group = inductiva.resources.MachineGroup("c2-standard-4")
machine_group.start()
simulator = inductiva.simulators.REEF3D()
task = simulator.run(input_dir="path-to-directory-with-input-files-for-reef3d",
on=machine_group)
print(task.id) # will print i4ir3kvv62odsfrhko4y8w2an
# Terminate the machine group
machine_group.terminate()
Using the command line interface (CLI), the user can list the contents of his remote
storage space to confirm the presence of the directory i4ir3kvv62odsfrhko4y8w2an
:
$ inductiva storage list --max-results 2 --order-by creation_time --sort-order desc
NAME SIZE CREATION TIME
i4ir3kvv62odsfrhko4y8w2an/ 21.67 MB 07 Feb, 13:47:03
If the user wishes to save the outputs of the simulation to a different directory,
he can do so by setting the storage_dir
argument in the run
method of the
simulator:
task = simulator.run(input_dir="path-to-directory-with-input-files-for-reef3d"
storage_dir="my_reef3d_simulation")
Again, using the CLI, the user can now confirm that the directory
my_reef3d_simulation
was indeed created in his remote storage space:
$ inductiva storage list --max-results 2 --order-by creation_time --sort-order desc
NAME SIZE CREATION TIME
my_reef3d_simulation/ 21.67 MB 07 Feb, 14:57:11
i4ir3kvv62odsfrhko4y8w2an/ 21.67 MB 07 Feb, 13:47:03
Listing of the contents of the user’s remote storage space can also be
accomplished programmatically using the inductiva.storage.listdir
function:
>>> import inductiva
inductiva.storage.listdir(max_results=10, order_by="creation_time", sort_order="desc")
Refer to the Storage how-to documentation for more information on how to manage the user’s remote storage space.
Downloading outputs#
Downloading the outputs of a task is a common operation. In the following snippets, we show how to download the outputs of a finished task and how to control which files and where those files are downloaded to.
# Download all the files produced by the task
>>> output_dir = task.download_outputs()
Downloading simulation outputs to inductiva_output/i4ir3kvv62odsfrhko4y8w2an/output.zip.
100%|██████████████████████████████████████| 5.04M/5.04M [00:00<00:00, 13.3MB/s]
Uncompressing the outputs to inductiva_output/i4ir3kvv62odsfrhko4y8w2an/
>>> print(ouptut_dir)
PosixPath('inductiva_output/i4ir3kvv62odsfrhko4y8w2an')
The downloaded files can be inspected using the ls
command:
$ ls -1 inductiva_output/i4ir3kvv62odsfrhko4y8w2an
log
stderr.txt
stdout.txt
vtk
By default, all output files are downloaded to a directory named after the task
into a parent directory named inductiva_output
. For example, the outputs
of the task with id abc123
would be downloaded to inductiva_output/abc123/
.
The name of the parent directory is a package-level variable that applies to all
downloads with a session. It prevents the user from cluttering the current
working directory with a large number of downloaded files. However, the user can
change this behavior by setting the parent directory name using the
inductiva.set_output_dir
function.
>>> import inductiva
# set the parent directory name
>>> inductiva.get_output_dir()
'inductiva_output'
>>> inductiva.set_output_dir("my_vault")
>>> inductiva.get_output_dir()
'my_vault'
# download the outputs again
>>> output_dir = task.download_outputs()
Downloading simulation outputs to my_vault/i4ir3kvv62odsfrhko4y8w2an/output.zip.
100%|██████████████████████████████████████| 5.04M/5.04M [00:00<00:00, 13.3MB/s]
Uncompressing the outputs to my_vault/i4ir3kvv62odsfrhko4y8w2an/
>>> print(ouptut_dir)
PosixPath('my_vault/i4ir3kvv62odsfrhko4y8w2an')
# Unset the parent directory name so that all downloads are done to the current
# working directory
>>> inductiva.set_output_dir(None)
>>> output_dir = task.download_outputs()
Downloading simulation outputs to i4ir3kvv62odsfrhko4y8w2an/output.zip.
100%|██████████████████████████████████████| 5.04M/5.04M [00:00<00:00, 13.3MB/s]
Uncompressing the outputs to i4ir3kvv62odsfrhko4y8w2an/
>>> print(ouptut_dir)
PosixPath('i4ir3kvv62odsfrhko4y8w2an')
The download_outputs
method of the Task
class has the option to specify the
name of the folder where the outputs are downloaded inside the parent directory.
# reset the parent directory name to the default
>>> inductiva.set_output_dir("inductiva_outputs")
>>> output_dir = task.download_outputs(output_dir="my_outputs")
Downloading simulation outputs to inductiva_output/my_outputs/output.zip.
100%|██████████████████████████████████████| 5.04M/5.04M [00:00<00:00, 13.3MB/s]
Uncompressing the outputs to inductiva_output/my_outputs/
>>> print(ouptut_dir)
PosixPath('inductiva_output/my_outputs')
In addition to controlling how downloaded files are organized, the user can also configure which files to download. By default, all files produced by the task are downloaded. However, the user can select to retrieve only a subset of those files by specifying the filenames of interest:
# Download only the files of interest
>>> output_dir = task.download_outputs(filenames=["stdout.txt", "stderr.txt"]
output_dir="my_outputs")
Downloading simulation outputs to inductiva_output/my_outputs/output.zip.
100%|██████████████████████████████████████| 1.04k/1.04k [00:00<00:00, 671kB/s]
Uncompressing the outputs to inductiva_output/my_outputs/
In this case, we can confirm that only the files stdout.txt
and stderr.txt
were indeed downloaded:
$ ls -1 inductiva_output/my_outputs
stderr.txt
stdout.txt