Experiment

Experiment#

#include "navground/sim/experiment.h"

struct Experiment#

An experiment supervises the execution of several runs of simulation.

It initializes simulations sharing the same Scenario and run them while collecting the desired data through ExperimentalRun.

use run_once to perform a single run.
use run to perform all runs, optionally saving the data to a HDF5 dataset.
use start, stop, start_run, stop_run, update_run to record data without launching a simulation, for instance if you are using a different run-loop.

Public Types

using RunCallback = std::function<void(ExperimentalRun*)>#: The type of callbacks called during an experiment

using ScenarioInitCallback = std::function<void(Scenario*, unsigned run_seed)>#: The type of callbacks called before initializing a world with a scenario

Public Functions

inline explicit Experiment(ng_float_t time_step = 0.1, unsigned steps = 1000)#

Constructs a new instance.

Parameters:

time_step – [in] The default simulation time step
steps – [in] The default number of simulation steps

inline const std::map<unsigned, ExperimentalRun> &get_runs() const#

Gets the map of recorded runs.

Runs are associated to the seed used to initialize them.

Returns:: The recorded runs.

inline bool has_finished() const#

Determines if the experiment has finished.

Returns:: True if finished, False otherwise.

inline bool is_running() const#

Determines if the experiment is running.

Returns:: True if running, False otherwise.

ExperimentalRun &run_once(unsigned seed)#

Perform a single run.

it initializes a world from its scenario by calling Scenario::init_world
it runs the simulation, step by step, collecting data in a ExperimentalRun by calling ExperimentalRun::run

Parameters:: seed – [in] The index (and random seed) of the run
Returns:: The recorded run.

void run(bool keep = true, unsigned number_of_threads = 1, std::optional<unsigned> start_index = std::nullopt, std::optional<unsigned> number_of_runs = std::nullopt, std::optional<std::filesystem::path> data_path = std::nullopt)#

Perform several runs and optionally record the data in a HFD5 file.

The number of runs is specified by the default number_of_runs if not specified.

Runs will be indexed sequentially and their index used as a random seed.

If save_directory not empty but points to an existing directory, it creates a HDF5 file <name>_<hash>_<timestamp>/data.h5 with attributes

begin_time [string], ISO 8601 formatted string of the time when the experiment is run, see get_begin_time;
duration_ns [unsigned], total duration in nanoseconds, see get_duration.
experiment [string], YAML serialization of the experiment;

Moreover, at the end of each run, it saves a group run_<index> with attributes:

duration_ns [unsigned], total duration in nanoseconds, see get_duration;
seed [unsigned];
steps [unsigned], actual number of steps performed;
maximal_steps [unsigned], maximal number of steps that could have been performed;
final_sim_time [float], the simulated time at the end of the run;
world [string], YAML serialization of the world at the begin of the experiment.

datasets:

times [float] (if RecordConfig::time is set);
poses [float] (if RecordConfig::pose is set);
twists [float] (if RecordConfig::twist is set);
cmds [float] (if RecordConfig::cmd is set);
actuated_cmds [float] (if RecordConfig::actuated_cmd is set);
targets [float] (if RecordConfig::target is set);
collisions [unsigned] (if RecordConfig::collisions is set);
deadlocks [float] (if RecordConfig::deadlocks is set);
efficacy [float] (if RecordConfig::efficacy is set);
safety_violations [float] (if RecordConfig::safety_violation is set);

and groups:

task_events (if RecordConfig::task_events is set), where each agents logs, in dataset task_events/<id> [float], the events emitted by their task.

Apart from saving data to the HDF5 file, each run is performed similarly to run_once.

Parameters:

keep – [in] Whether to keep runs in memory
number_of_threads – [in] How many threads to use. When more than one, runs will be distributed in parallel over the threads.
start_index – [in] The index/seed of the first run. If unspecified, it will use the experiment’s run_index
number_of_runs – [in] The number of runs. If unspecified, it will use the experiment’s number_of_runs
data_path – [in] A path to set optionally as get_path. If set, it will save an HDF5 file, but no YAML, to this path. If not set, get_path will be automatically set to <save_directory>/data.h5 if save_directory is set.

inline virtual void remove_all_runs()#: Clear the recording.

inline virtual void remove_run(unsigned seed)#

Removes a recorded run.

Parameters:: seed – [in] The seed/index of the run

void start(std::optional<std::filesystem::path> path = std::nullopt)#

Signal to start an experiment.

Note that this won’t neither execute any simulation nor record data. It will just record time stamp and change the state to running.

This is only needed when using an external run-loop for the simulation. In this case:

call start
For each run

i. Call init_run and start_run ii. from your run-loop, call update_run to record the current state of the simulated world iii. Call stop_run
Call stop

Use run instead to perform the simulations and record them all at once.

Calling start is only effective once per experiment.

Parameters:: path – [in] A path to set optionally as get_path. If set, it will save an HDF5 file, but no YAML, to this path. If not set, get_path will be automatically set to <save_directory>/data.h5 if save_directory is set.

void stop(bool save_runs = false)#

Signal to stop an experiment.

See start. This is only need when using an external run-loop to simulate.

Parameters:: save_runs – [in] Whether to save the runs before closing

virtual ExperimentalRun &init_run(int seed, std::shared_ptr<World> world = nullptr)#

Initializes a run.

See start. This is only need when using an external run-loop to simulate or when manually calling ExperimentalRun::run later. Else use run_once (or run for all runs) to initialize and run at once.

Parameters:

seed – [in] The random seed
world – [in] The world to simulate. If null, it will initialize a new world.

void start_run(ExperimentalRun &run)#

Start recording a run.

See start. This is only need when using an external run-loop to simulate.

Call start_run only once per run.

Parameters:: run – The run being recorded

void update_run(ExperimentalRun &run)#

{ function_description }

See start. This is only need when using an external run-loop to simulate.

Call update_run every time you need to record the current state of the world, typically after each simulation step.

Parameters:: run – The run being recorded

void stop_run(ExperimentalRun &run)#

Stop the run recording.

See start_run. This is only need to use an external run-loop.

Parameters:: run – The run being recorded

inline void add_run_callback(const RunCallback &value, bool at_init = false)#

Adds a callback to be executed before/after each run.

Parameters:

value – [in] The callback
at_init – [in] Whether the callback should be called when initializing the run. If not set, it will be called at the completion of a run.

inline const std::map<bool, std::vector<RunCallback>> &get_run_callbacks() const#

Gets the run callbacks.

Returns:: The callbacks

inline void set_run_callbacks(const std::map<bool, std::vector<RunCallback>> &value)#

Sets the run callbacks.

Parameters:: value – [in] The desired callbacks

inline void clear_run_callbacks()#: Remove all run callbacks.

inline std::optional<ScenarioInitCallback> get_scenario_init_callback() const#

Gets the callback called before the scenario initializes a world.

Returns:: The callback or null if none is set

inline void set_scenario_init_callback(const std::optional<ScenarioInitCallback> &value)#

Sets the run callbacks.

Parameters:: value – [in] The desired callbacks

inline std::optional<std::filesystem::path> get_path() const#

Gets the path where the experimental data has been saved.

Returns:: The path or none if the experiment has not been run yet.

inline std::chrono::nanoseconds get_duration() const#

Gets the duration required to perform the whole experiment.

Returns:: The duration in ns or 0 if the experiment has not finished.

inline std::chrono::time_point<std::chrono::system_clock> get_begin_time() const#

Gets the system time when the experiment began.

Returns:: The time

inline bool has_file() const#

Determines if it is recording data to a file or it has done it.

Returns:: True if file, False otherwise.

inline ng_float_t get_time_step() const#

Gets the default time step used for simulation during each run.

Returns:: The time step.

inline void set_time_step(ng_float_t value)#

Sets the default time step used for simulation during each run.

Parameters:: value – [in] The desired value

inline unsigned get_steps() const#

Gets the default maximal number of steps to simulate during each run.

Returns:: The maximal number of steps.

inline void set_steps(unsigned value)#

Sets the default maximal number of steps to simulate during each run.

Parameters:: value – [in] The desired value

inline bool get_terminate_when_all_idle_or_stuck() const#

Gets whether to terminate when all agents are idle or stuck.

Returns:: Whether to terminate when all agents are idle or stuck.

inline void set_terminate_when_all_idle_or_stuck(bool value)#

Sets whether to terminate when all agents are idle or stuck.

Parameters:: value – [in] The desired value

inline void add_probe(const std::function<std::shared_ptr<Probe>()> &factory)#

Register a probe to be added to all runs.

Parameters:: factory – [in] A function that generate the probe.

template<typename T> inline void add_record_probe(const std::string &key)#

Register a probe to record data to during all runs.

Parameters:: key – [in] The name associated to the record
Template Parameters:: T – The class of the probe. Must be subclass of sim::RecordProbe

template<typename T> inline void add_group_record_probe(const std::string &key)#

Register a probe to record a group of data to during all runs.

Parameters:: key – [in] The name associated to the group
Template Parameters:: T – The class of the probe. Must be subclass of sim::GroupRecordProbe

void save(std::optional<std::filesystem::path> directory = std::nullopt, std::optional<std::filesystem::path> path = std::nullopt)#

Save all recorded runs.

Parameters:

directory – [in] A path to set optionally as save_directory where to save YAML and HDF5 file.
path – [in] A path to set optionally as get_path. If set, it will save an HDF5 file, but no YAML, to this path. If not set, get_path will be automatically set to <save_directory>/data.h5 if save_directory is set.

Public Members

RecordConfig record_config#: The record config on which data to record

RunConfig run_config#: How to perform runs;

unsigned number_of_runs#: Default number of runs to perform

std::filesystem::path save_directory#: Where to save the results

std::map<unsigned, ExperimentalRun> runs#: All runs, ordered by seed

std::string name#: The name of the experiment

std::shared_ptr<Scenario> scenario#: The scenario used to initialize a world

unsigned run_index#: The seed/index of the next run

bool reset_uids#: Whether to reset the Entities UID to zero before each run

Experiment

Contents

Experiment#