DataNode.hpp

DataNode objects track a specific type of data over the course of a run.

Collection: New data can be pushed or pulled. Add(VAL… v) pushes data to a node AddDatum(VAL v) pushes just one datum, but can be used as an action for a signal.

Process: What should happen on Reset() ?

Trigger an action to process the prior update’s data stored.
Clear all data.
Send data to a stream (or stats automatically have a stream that, if non-null data is sent to?)

Todo:: : The Archive data node should have Log as a requisite and then copy the current vals into the archive on reset. This change will also make it so that the size of the archive correctly reflects the number of resets.

Typedefs

template<typename T, data... MODS> using DataMonitor = DataNode<T, data::Current, data::Info, data::Range, data::Stats, MODS...>: A node that stores data about the most recent value it received, as well as the distribution (min, max, count, total, and mean) of values it has received since the last reset. It also allows you to give it a name, description, and keyword.

template<typename T, data... MODS> using DataLog = DataNode<T, data::Current, data::Info, data::Log, MODS...>: A node that stores data about the most recent value it received, as well as all values it has received since the last reset. It also allows you to give it a name, description, and keyword.

template<typename T, data... MODS> using DataArchive = DataNode<T, data::Info, data::Archive, data::FullRange, MODS...>: A node that stores all data it receives in an archive (vector of vectors). The inner vectors are groups of data that were received between resets. This node also keeps a record of the min, max, count, and total of each vector, so you don’t have to recalculate it later. Additionally, it allows you to give it a name, description, and keyword.

Enums

enum class data

A set of modifiers are available do describe DataNode.

Values:

enumerator Current: Track most recent value.

enumerator Info: Include information (name, keyword, description) for each instance.

enumerator Log: Track all values since last Reset()

enumerator Archive: Track Log + ALL values over time (with purge options)

enumerator Range: Track min, max, mean, total.

enumerator FullRange: Track Range data over time.

enumerator Histogram: Keep a full histogram.

enumerator Stats: Track Range + variance, standard deviation, skew, kurtosis.

enumerator Pull

Enable data collection on request.

Various signals are possible:

enumerator SignalReset: Include a signal that triggers BEFORE Reset() to process data.

enumerator SignalData: Include a signal when new data is added (as a group)

enumerator SignalDatum: Include a signal when each datum is added.

enumerator SignalRange: Include a signal for data in a range.

enumerator SignalLimits: Include a signal for data OUTSIDE a range.

enumerator UNKNOWN: Unknown modifier; will trigger error.

template<typename VAL_TYPE> class DataNodeModule<VAL_TYPE>

#include <DataNode.hpp>

Base form of DataNodeModule (available in ALL data nodes.)

Public Types

using value_t = VAL_TYPE 

Public Functions

inline DataNodeModule()

inline size_t GetCount() const: Return the number of values that have been added to this node since the last reset.

inline size_t GetResetCount() const: If reset count not tracked, always return 0. If any modifier causes history to be saved, it will override this function and indicate how much history is stored.

inline double GetTotal() const

inline double GetMean() const

inline double GetMin() const

inline double GetMax() const

inline double GetVariance() const

inline double GetStandardDeviation() const

inline double GetSkew() const

inline double GetKurtosis() const

inline double GetMedian() const: Calculate the median of observed values.

inline double GetPercentile(const double) const: Calculate a percentile of observed values.

inline const std::string &GetName() const

inline const std::string &GetDescription() const

inline const std::string &GetKeyword() const

inline void SetName(const std::string&)

inline void SetDescription(const std::string&)

inline void SetKeyword(const std::string&)

inline void SetInfo(const std::string&, const std::string &_d = "", const std::string &_k = "")

inline void AddDatum(const VAL_TYPE&)

inline void Reset()

inline void PrintDebug(std::ostream &os = std::cout): Print debug information (useful for figuring out which modifiers you included)

Protected Functions

inline void PullData_impl()

Protected Attributes

size_t val_count: How many values have been loaded?

vector<VAL_TYPE> in_vals: What values are waiting to be included?

class CurrentModule

#include <DataNode.hpp>

== data::Current == This module lets you track the current (i.e. most recently added) value To use this class, add data::Current to the template arguments on your DataNode. Do not use the CurrentModule class directly - it is a simplification for documentation purposes and does not actually exist.

Public Functions

inline DataNodeModule()

inline const VAL_TYPE &GetCurrent() const: Return the current (most recently added) value.

inline void AddDatum(const VAL_TYPE &val)

Add.

Parameters:: val – to this DataNode

inline void PrintDebug(std::ostream &os = std::cout): Print debug information (useful for figuring out which modifiers you included)

Protected Types

using this_t = DataNodeModule<VAL_TYPE, data::Current, MODS...>

using parent_t = DataNodeModule<VAL_TYPE, MODS...>

using base_t = DataNodeModule<VAL_TYPE>

Protected Attributes

VAL_TYPE cur_val: Most recent value passed to this node.

class InfoModule

#include <DataNode.hpp>

== data::Info == This module adds information such as a name, description, and keyword for this node. To use this class, add data::Info to the template arguments on your DataNode. Do not use the InfoModule class directly - it is a simplification for documentation purposes and does not actually exist.

Public Functions

inline DataNodeModule()

inline const std::string &GetName() const: Get this DataNode’s name.

inline const std::string &GetDescription() const: Get this DataNode’s description.

inline const std::string &GetKeyword() const: Get this DataNode’s keyword.

inline void SetName(const std::string &_in): Set this DataNode’s name to specified value.

inline void SetDescription(const std::string &_in): Set this DataNode’s description to specified value.

inline void SetKeyword(const std::string &_in): Set this DataNode’s keyword to specified value.

inline void SetInfo(const std::string &_n, const std::string &_d = "", const std::string &_k = "")

Set this DataNode’s information

Parameters:

_n – value to set name to
_d – value to set description to
_k – value to set keyword to

inline void PrintDebug(std::ostream &os = std::cout): Print debug information (useful for figuring out which modifiers you included)

Protected Types

using parent_t = DataNodeModule<VAL_TYPE, MODS...>

Protected Attributes

std::string name: Name of this data category.

std::string desc: Description of this type of data.

std::string keyword: Short keyword.

class LogModule

#include <DataNode.hpp>

== data::Log == This module lets you log all of the values that have been added since the last re-set To use this class, add data::Log to the template arguments on your DataNode. Do not use the LogModule class directly - it is a simplification for documentation purposes and does not actually exist.

Public Functions

inline DataNodeModule()

inline const vector<VAL_TYPE> &GetData() const: Get a vector of all data added since the last reset.

inline double GetMedian() const: Calculate the median of observed values.

inline double GetPercentile(const double pct) const: Calculate a percentile of observed values.

inline void AddDatum(const VAL_TYPE &val)

Add.

Parameters:: val – to this DataNode

inline void Reset(): Reset this DataNode (clear the current log of data)

inline void PrintDebug(std::ostream &os = std::cout): Print debug information (useful for figuring out which modifiers you included)

Protected Types

using this_t = DataNodeModule<VAL_TYPE, data::Log, MODS...>

using parent_t = DataNodeModule<VAL_TYPE, MODS...>

using base_t = DataNodeModule<VAL_TYPE>

Protected Attributes

vector<VAL_TYPE> val_set: All values saved since last reset.

size_t val_count: How many values have been loaded?

class ArchiveModule

#include <DataNode.hpp>

== data::Archive == This module keeps track of historical values in addition to those added since the last re-set. Every time Reset() is called, all values that have been added since the previous time Reset() are stored in a vector in the archive. To use this class, add data::Archive to the template arguments on your DataNode. Do not use the ArchiveModule class directly - it is a simplification for documentation purposes and does not actually exist.

Public Functions

inline DataNodeModule()

inline const auto &GetArchive() const: Get all data ever added to this DataNode. Returns a vector of vectors; each vector contains all data from a single time point (interval between resets)

inline const vector<VAL_TYPE> &GetData(size_t update) const

Get a vector of all data that was added during the.

Parameters:: update – ‘th interval between resets.

inline const vector<VAL_TYPE> &GetData() const: Get a vector of all data that has been added since the last reset.

inline size_t GetResetCount() const: Get the number of time intervals recorded in this DataNode. Note that this is one more than the number of times it has been reset

inline void Reset(): Reset this DataNode, starting a new grouping of values in the archive. Resetting is useful for tracking data from different time points, such as per update or generation.

inline void PrintDebug(std::ostream &os = std::cout): Print debug information (useful for figuring out which modifiers you included)

Protected Types

using this_t = DataNodeModule<VAL_TYPE, data::Archive, MODS...>

using parent_t = DataNodeModule<VAL_TYPE, MODS...>

using base_t = DataNodeModule<VAL_TYPE>

Protected Attributes

vector<vector<VAL_TYPE>> archive: Data archived from before most recent reset.

size_t val_count: How many values have been loaded?

class RangeModule

#include <DataNode.hpp>

== data::Range == This module allows this DataNode to store information (min, max, mean, count, and total) about the distribution of the values that have been added since the last call to Reset(). To use this class, add data::Range to the template arguments on your DataNode. Do not use the RangeModule class directly - it is a simplification for documentation purposes and does not actually exist.

Public Functions

inline DataNodeModule()

inline double GetTotal() const: Get the sum of all values added to this DataNode since the last reset.

inline double GetMean() const: Get the mean of all values added to this DataNode since the last reset.

inline double GetMin() const: Get the min of all values added to this DataNode since the last reset.

inline double GetMax() const: Get the max of all values added to this DataNode since the last reset.

inline void AddDatum(const VAL_TYPE &val)

Add.

Parameters:: val – to this DataNode

inline void Reset(): Reset DataNode, setting the running calculations of total, min, mean, and max to 0.

inline void PrintDebug(std::ostream &os = std::cout): Print debug information (useful for figuring out which modifiers you included)

Protected Types

using this_t = DataNodeModule<VAL_TYPE, data::Range, MODS...>

using parent_t = DataNodeModule<VAL_TYPE, MODS...>

using base_t = DataNodeModule<VAL_TYPE>

Protected Attributes

double total: Total of all data since last reset.

double min: Smallest value passed in since last reset.

double max: Largest value passed in since last reset.

size_t val_count: How many values have been loaded?

class FullRangeModule

#include <DataNode.hpp>

== data::FullRange == This module makes the DataNode store a history of distributional information measured by data::Range between calls to Reset(). Series of historical values are stored in vectors (except mean, which is calculated from total and count). To use this class, add data::Histogram to the template arguments on your DataNode. Do not use the FullRangeModule class directly - it is a simplification for documentation purposes and does not actually exist.

Public Functions

inline DataNodeModule()

inline double GetTotal() const: Get the sum of all values added to this DataNode since the last reset.

inline double GetMean() const: Get the mean of all values added to this DataNode since the last reset.

inline double GetMin() const: Get the minimum of all values added to this DataNode since the last reset.

inline double GetMax() const: Get the maximum of all values added to this DataNode since the last reset.

inline double GetTotal(size_t update) const

Get the sum of all values added to this DataNode during the.

Parameters:: update – specified.

inline double GetMean(size_t update) const

Get the mean of all values added to this DataNode during the.

Parameters:: update – specified.

inline double GetMin(size_t update) const

Get the minimum of all values added to this DataNode during the.

Parameters:: update – specified.

inline double GetMax(size_t update) const

Get the maximum of all values added to this DataNode during the.

Parameters:: update – specified.

inline size_t GetResetCount() const: Get the number of time intervals recorded in this DataNode. Note that this is one more than the number of times it has been reset

inline void Reset(): Store the current range statistics in the archive and reset for a new interval.

inline void PrintDebug(std::ostream &os = std::cout): Print debug information (useful for figuring out which modifiers you included)

Protected Types

using this_t = DataNodeModule<VAL_TYPE, data::FullRange, MODS...>

using parent_t = DataNodeModule<VAL_TYPE, MODS...>

using base_t = DataNodeModule<VAL_TYPE>

Protected Attributes

vector<double> total_vals: Totals from previous resets.

vector<size_t> num_vals: Value counts from previous resets.

vector<double> min_vals: Minimums from previous resets.

vector<double> max_vals: Maximums from previous resets.

size_t val_count: How many values have been loaded?

class StatsModule

#include <DataNode.hpp>

== data::Stats == Note: These statistics are calculated with the assumption that the data this node has received is the entire population of measurements we’re interested in, not a sample.

Note 2: Kurtosis is calculated using Snedecor and Cochran (1967)’s formula. A perfect normal distribution has a kurtosis of 0. To use this class, add data::Stats to the template arguments on your DataNode. Do not use the StatsModule class directly - it is a simplification for documentation purposes and does not actually exist.

Public Functions

inline DataNodeModule()

inline double GetVariance() const: Get the variance (squared deviation from the mean) of values added since the last reset.

inline double GetStandardDeviation() const: Get the standard deviation of values added since the last reset.

inline double GetSkew() const: Get the skewness of values added since the last reset. This measurement tells you about the shape of the distribution. For a unimodal distribution, negative skew means that the distribution has a longer/thicker tail to the left. Positive skew means that ths distribution has a longer/thicker tail to the right.

inline double GetKurtosis() const: Get the kurtosis of the values added since the last reset. This is another measurement that describes the shape of the distribution. High kurtosis means that there is more data in the tails of the distribution (i.e. the tails are “heavier”), whereas low kurtosis means that there is less data in the tails. We use Snedecor and Cochran (1967)’s formula to calculate kurtosis. Under this formula, a normal distribution has kurtosis of 0.

inline void AddDatum(const VAL_TYPE &val)

Add.

Parameters:: val – to this DataNode

inline void Reset(): Reset this node (resets current stats to 0)

inline void PrintDebug(std::ostream &os = std::cout): Print debug information (useful for figuring out which modifiers you included)

Protected Types

using this_t = DataNodeModule<VAL_TYPE, data::Stats, MODS...>

using parent_t = DataNodeModule<VAL_TYPE, MODS...>

using base_t = DataNodeModule<VAL_TYPE>

Protected Attributes

double M2: The second moment of the distribution.

double M3: The third moment of the distribution.

double M4: The fourth moment of the distribution.

size_t val_count: How many values have been loaded?

class HistogramModule

#include <DataNode.hpp>

== data::Histogram == Make the DataNode track a histogram of values observed since the last reset. To use this class, add data::Histogram to the template arguments on your DataNode. Do not use the HistogramModule class directly - it is a simplification for documentation purposes and does not actually exist.

Public Functions

inline DataNodeModule()

inline VAL_TYPE GetHistMin() const: Returns the minimum value this histogram is capable of containing (i.e. the minimum value for the first bin)

inline VAL_TYPE GetHistMax() const: Returns the maximum value this histogram is capable of containing (i.e. the maximum value for the last bin)

inline size_t GetHistCount(size_t bin_id) const

Return the count of items in the.

Parameters:: bin_id – ‘th bin of the histogram

inline double GetHistWidth(size_t bin_id) const

Return the width of the.

Parameters:: bin_id – ‘th bin of the histogram

inline const vector<size_t> &GetHistCounts() const: Return a vector containing the count of items in each bin of the histogram.

inline int GetOverflow() const: Return the count of numbers added to this histogram that were above the upper bound on the histogram

inline int GetUnderflow() const: Return the count of numbers added to this histogram that were below the allowed lower bound

inline vector<double> GetBinMins() const: Return a vector containing the lowest value allowed in each bin.

inline void SetupBins(VAL_TYPE _min, VAL_TYPE _max, size_t num_bins)

Sets up the ranges of values that go in each bin of the histogram.

Parameters:

_min – - the lowest value allowed in the histogram
_max – - the largest value allowed in the histogram
num_bins – - The number of bins the histogram should have. The distance between min and max will be easily divided among this many bins.

inline void AddDatum(const VAL_TYPE &val)

Add.

Parameters:: val – to the DataNode

inline void Reset(): Reset the DataNode (empties the histogram)

inline void PrintDebug(std::ostream &os = std::cout): Print debug information (useful for figuring out which modifiers you included)

Protected Types

using this_t = DataNodeModule<VAL_TYPE, data::Histogram, MODS...>

using parent_t = DataNodeModule<VAL_TYPE, MODS...>

using base_t = DataNodeModule<VAL_TYPE>

Protected Attributes

VAL_TYPE offset: Min value in first bin; others are offset by this much.

VAL_TYPE width: How wide is the overall histogram?

IndexMap bins: Map of values to which bin they fall in.

vector<size_t> counts: Counts in each bin.

size_t overflow

size_t underflow

size_t val_count: How many values have been loaded?

class PullModule

#include <DataNode.hpp>

== data::Pull == This module makes it possible to give the DataNode a function that it can call to calculate new values or sets of values that it will then track. These functions are called every time the PullData method is called on this node, and the values they return are measured as specified by the other modules in this node. To use this class, add data::Pull to the template arguments on your DataNode. Do not use the PullModule class directly - it is a simplification for documentation purposes and does not actually exist.

Public Functions

inline DataNodeModule()

inline void AddPull(const std::function<VAL_TYPE()> &fun)

inline void AddPullSet(const std::function<vector<VAL_TYPE>()> &fun)

inline void PrintDebug(std::ostream &os = std::cout)

template<typename VAL_TYPE, data... MODS> class DataNode : public DataNode_Interface<VAL_TYPE, FormatDataMods<MODS...>::sorted>

#include <DataNode.hpp>

Public Functions

inline void Add()

template<typename ...Ts> inline void Add(const VAL_TYPE &val, const Ts&... extras): Methods to provide new data.

inline void PullData(): Method to retrieve new data.

inline void Reset(): Methods to reset data.

inline void PrintCurrent(std::ostream &os = std::cout) const: Methods to Print the templated values that a DataNode can produce.

inline void PrintLog(std::ostream &os = std::cout, const std::string &spacer = ", ", const std::string &eol = "\n") const

inline void PrintDebug(std::ostream &os = std::cout): Print debug information (useful for figuring out which modifiers you included)

Private Types

using parent_t = DataNode_Interface<VAL_TYPE, typename FormatDataMods<MODS...>::sorted>