Experiment

#include "navground/sim/experiment.h"

struct Experiment

An experiment supervises the execution of several runs of simulation.

It initializes simulations sharing the same Scenario and run them while collecting the desired data through ExperimentalRun.

• use run_once to perform a single run.

• use run to perform all runs, optionally saving the data to a HDF5 dataset.

• use start, stop, start_run, stop_run, update_run to record data without launching a simulation, for instance if you are using a different run-loop.

Public Types

using RunCallback = std::function<void(ExperimentalRun*)>

The type of callbacks called during an experiment

using ScenarioInitCallback = std::function<void(Scenario*, unsigned run_seed)>

The type of callbacks called before initializing a world with a scenario

Public Functions

inline explicit Experiment(ng_float_t time_step = 0.1, unsigned steps = 1000)

Constructs a new instance.

Parameters:

• time_step – [in] The default simulation time step

• steps – [in] The default number of simulation steps

inline const std::map<unsigned, ExperimentalRun> &get_runs() const

Gets the map of recorded runs.

Runs are associated to the seed used to initialize them.

Returns:

The recorded runs.

inline bool has_finished() const

Determines if the experiment has finished.

Returns:

True if finished, False otherwise.

inline bool is_running() const

Determines if the experiment is running.

Returns:

True if running, False otherwise.

ExperimentalRun &run_once(unsigned seed)

Perform a single run.

1. it initializes a world from its scenario by calling Scenario::init_world

2. it runs the simulation, step by step, collecting data in a ExperimentalRun by calling ExperimentalRun::run

Parameters:

seed – [in] The index (and random seed) of the run

Returns:

The recorded run.

void run(bool keep = true, unsigned number_of_threads = 1, std::optional<unsigned> start_index = std::nullopt, std::optional<unsigned> number_of_runs = std::nullopt, std::optional<std::filesystem::path> data_path = std::nullopt)

Perform several runs and optionally record the data in a HFD5 file.

The number of runs is specified by the default number_of_runs if not specified.

Runs will be indexed sequentially and their index used as a random seed.

If save_directory not empty but points to an existing directory, it creates a HDF5 file <name>_<hash>_<timestamp>/data.h5 with attributes

• begin_time [string], ISO 8601 formatted string of the time when the experiment is run, see get_begin_time;

• duration_ns [unsigned], total duration in nanoseconds, see get_duration.

• experiment [string], YAML serialization of the experiment;

Moreover, at the end of each run, it saves a group run_<index> with attributes:

• duration_ns [unsigned], total duration in nanoseconds, see get_duration;

• seed [unsigned];

• steps [unsigned], actual number of steps performed;

• maximal_steps [unsigned], maximal number of steps that could have been performed;

• final_sim_time [float], the simulated time at the end of the run;

• world [string], YAML serialization of the world at the begin of the experiment.

datasets:

• times [float] (if RecordConfig::time is set);

• poses [float] (if RecordConfig::pose is set);

• twists [float] (if RecordConfig::twist is set);

• cmds [float] (if RecordConfig::cmd is set);

• actuated_cmds [float] (if RecordConfig::actuated_cmd is set);

• targets [float] (if RecordConfig::target is set);

• collisions [unsigned] (if RecordConfig::collisions is set);

• deadlocks [float] (if RecordConfig::deadlocks is set);

• efficacy [float] (if RecordConfig::efficacy is set);

• safety_violations [float] (if RecordConfig::safety_violation is set);

and groups:

• task_events (if RecordConfig::task_events is set), where each agents logs, in dataset task_events/<id> [float], the events emitted by their task.

Apart from saving data to the HDF5 file, each run is performed similarly to run_once.

Parameters:

• keep – [in] Whether to keep runs in memory

• number_of_threads – [in] How many threads to use. When more than one, runs will be distributed in parallel over the threads.

• start_index – [in] The index/seed of the first run. If unspecified, it will use the experiment’s run_index

• number_of_runs – [in] The number of runs. If unspecified, it will use the experiment’s number_of_runs

• data_path – [in] A path to set optionally as get_path. If set, it will save an HDF5 file, but no YAML, to this path. If not set, get_path will be automatically set to <save_directory>/data.h5 if save_directory is set.

inline virtual void remove_all_runs()

Clear the recording.

inline virtual void remove_run(unsigned seed)

Removes a recorded run.

Parameters:

seed – [in] The seed/index of the run

void start(std::optional<std::filesystem::path> path = std::nullopt)

Signal to start an experiment.

Note that this won’t neither execute any simulation nor record data. It will just record time stamp and change the state to running.

This is only needed when using an external run-loop for the simulation. In this case:

1. call start

2. For each run

   i. Call init_run and start_run ii. from your run-loop, call update_run to record the current state of the simulated world iii. Call stop_run

3. Call stop

Use run instead to perform the simulations and record them all at once.

Calling start is only effective once per experiment.

Parameters:

path – [in] A path to set optionally as get_path. If set, it will save an HDF5 file, but no YAML, to this path. If not set, get_path will be automatically set to <save_directory>/data.h5 if save_directory is set.

void stop(bool save_runs = false)

Signal to stop an experiment.

See start. This is only need when using an external run-loop to simulate.

Parameters:

save_runs – [in] Whether to save the runs before closing

virtual ExperimentalRun &init_run(int seed, std::shared_ptr<World> world = nullptr)

Initializes a run.

See start. This is only need when using an external run-loop to simulate or when manually calling ExperimentalRun::run later. Else use run_once (or run for all runs) to initialize and run at once.

Parameters:

• seed – [in] The random seed

• world – [in] The world to simulate. If null, it will initialize a new world.

void start_run(ExperimentalRun &run)

Start recording a run.

See start. This is only need when using an external run-loop to simulate.

Call start_run only once per run.

Parameters:

run – The run being recorded

void update_run(ExperimentalRun &run)

{ function_description }

See start. This is only need when using an external run-loop to simulate.

Call update_run every time you need to record the current state of the world, typically after each simulation step.

Parameters:

run – The run being recorded

void stop_run(ExperimentalRun &run)

Stop the run recording.

See start_run. This is only need to use an external run-loop.

Parameters:

run – The run being recorded

inline void add_run_callback(const RunCallback &value, bool at_init = false)

Adds a callback to be executed before/after each run.

Parameters:

• value – [in] The callback

• at_init – [in] Whether the callback should be called when initializing the run. If not set, it will be called at the completion of a run.

inline const std::map<bool, std::vector<RunCallback>> &get_run_callbacks() const

Gets the run callbacks.

Returns:

The callbacks

inline void set_run_callbacks(const std::map<bool, std::vector<RunCallback>> &value)

Sets the run callbacks.

Parameters:

value – [in] The desired callbacks

inline void clear_run_callbacks()

Remove all run callbacks.

inline std::optional<ScenarioInitCallback> get_scenario_init_callback() const

Gets the callback called before the scenario initializes a world.

Returns:

The callback or null if none is set

inline void set_scenario_init_callback(const std::optional<ScenarioInitCallback> &value)

Sets the run callbacks.

Parameters:

value – [in] The desired callbacks

inline std::optional<std::filesystem::path> get_path() const

Gets the path where the experimental data has been saved.

Returns:

The path or none if the experiment has not been run yet.

inline std::chrono::nanoseconds get_duration() const

Gets the duration required to perform the whole experiment.

Returns:

The duration in ns or 0 if the experiment has not finished.

inline std::chrono::time_point<std::chrono::system_clock> get_begin_time() const

Gets the system time when the experiment began.

Returns:

The time

inline bool has_file() const

Determines if it is recording data to a file or it has done it.

Returns:

True if file, False otherwise.

inline ng_float_t get_time_step() const

Gets the default time step used for simulation during each run.

Returns:

The time step.

inline void set_time_step(ng_float_t value)

Sets the default time step used for simulation during each run.

Parameters:

value – [in] The desired value

inline unsigned get_steps() const

Gets the default maximal number of steps to simulate during each run.

Returns:

The maximal number of steps.

inline void set_steps(unsigned value)

Sets the default maximal number of steps to simulate during each run.

Parameters:

value – [in] The desired value

inline bool get_terminate_when_all_idle_or_stuck() const

Gets whether to terminate when all agents are idle or stuck.

Returns:

Whether to terminate when all agents are idle or stuck.

inline void set_terminate_when_all_idle_or_stuck(bool value)

Sets whether to terminate when all agents are idle or stuck.

Parameters:

value – [in] The desired value

inline void add_probe(const std::function<std::shared_ptr<Probe>()> &factory)

Register a probe to be added to all runs.

Parameters:

factory – [in] A function that generate the probe.

template<typename T>
inline void add_record_probe(const std::string &key)

Register a probe to record data to during all runs.

Parameters:

key – [in] The name associated to the record

Template Parameters:

T – The class of the probe. Must be subclass of sim::RecordProbe

template<typename T>
inline void add_group_record_probe(const std::string &key)

Register a probe to record a group of data to during all runs.

Parameters:

key – [in] The name associated to the group

Template Parameters:

T – The class of the probe. Must be subclass of sim::GroupRecordProbe

void save(std::optional<std::filesystem::path> directory = std::nullopt, std::optional<std::filesystem::path> path = std::nullopt)

Save all recorded runs.

Parameters:

• directory – [in] A path to set optionally as save_directory where to save YAML and HDF5 file.

• path – [in] A path to set optionally as get_path. If set, it will save an HDF5 file, but no YAML, to this path. If not set, get_path will be automatically set to <save_directory>/data.h5 if save_directory is set.

Public Members

RecordConfig record_config

The record config on which data to record

RunConfig run_config

How to perform runs;

unsigned number_of_runs

Default number of runs to perform

std::filesystem::path save_directory

Where to save the results

std::map<unsigned, ExperimentalRun> runs

All runs, ordered by seed

std::string name

The name of the experiment

std::shared_ptr<Scenario> scenario

The scenario used to initialize a world

unsigned run_index

The seed/index of the next run

bool reset_uids

Whether to reset the Entities UID to zero before each run

bool record_scenario_properties

Whether to record (sampled) scenario properties as world attributes