protomotions.agents.evaluators.config module#

Configuration classes for evaluators.

class protomotions.agents.evaluators.config.EvaluatorConfig( _target_='protomotions.agents.evaluators.base_evaluator.BaseEvaluator', metric_components=<factory>, failure_components=<factory>, score_key=None, max_eval_steps=600, eval_metrics_every=200, )[source]#

Bases: object

Configuration for base evaluator.

Attributes:: metric_components: Per-environment numeric metrics; log mean/min/max over scored frames. failure_components: A TerminationRule (any-frame failure) or ThresholdFailure(rule, reduction=…) over sum, mean, min, max or last value per trial. score_key: ‘eval/success_rate’ for checkpoint selection, None for metrics only. max_eval_steps: Maximum steps per evaluation episode. eval_metrics_every: Evaluate metrics every N epochs. None = disabled.

metric_components: Dict[str, MdpComponent]#

failure_components: Dict[str, TerminationRule | ThresholdFailure]#

score_key: str | None = None#

max_eval_steps: int = 600#

eval_metrics_every: int | None = 200#

validate()[source]#

__init__( _target_='protomotions.agents.evaluators.base_evaluator.BaseEvaluator', metric_components=<factory>, failure_components=<factory>, score_key=None, max_eval_steps=600, eval_metrics_every=200, )#

class protomotions.agents.evaluators.config.MotionWeightsRulesConfig( motion_weights_update_success_discount=0.999, motion_weights_update_failure_discount=0.999, min_motion_weight='1/num_motions', )[source]#

Bases: object

Configuration for motion weights update rule.

Attributes:: motion_weights_update_success_discount: Discount factor for successful motion weights. motion_weights_update_failure_discount: Discount for failed motions. 0 = set weight straight to 1. min_motion_weight: Minimum weight for any motion. ‘1/num_motions’ or float value.

motion_weights_update_success_discount: float = 0.999#

motion_weights_update_failure_discount: float = 0.999#

min_motion_weight: float | str = '1/num_motions'#

__init__( motion_weights_update_success_discount=0.999, motion_weights_update_failure_discount=0.999, min_motion_weight='1/num_motions', )#

class protomotions.agents.evaluators.config.MimicEvaluatorConfig( _target_='protomotions.agents.evaluators.mimic_evaluator.MimicEvaluator', metric_components=<factory>, failure_components=<factory>, score_key=None, max_eval_steps=600, eval_metrics_every=200, save_predicted_motion_lib_every=3, motion_weights_rules=<factory>, )[source]#

Bases: EvaluatorConfig

Configuration for Mimic evaluator.

Attributes:: metric_components: Per-environment numeric metrics; log mean/min/max over scored frames. failure_components: A TerminationRule (any-frame failure) or ThresholdFailure(rule, reduction=…) over sum, mean, min, max or last value per trial. score_key: ‘eval/success_rate’ for checkpoint selection, None for metrics only. max_eval_steps: Maximum steps per evaluation episode. eval_metrics_every: Evaluate metrics every N epochs. None = disabled. save_predicted_motion_lib_every: Save pred_motion_lib every M evals. None = disabled. motion_weights_rules: Rules for updating motion sampling weights.

save_predicted_motion_lib_every: int | None = 3#

motion_weights_rules: MotionWeightsRulesConfig#

__init__( _target_='protomotions.agents.evaluators.mimic_evaluator.MimicEvaluator', metric_components=<factory>, failure_components=<factory>, score_key=None, max_eval_steps=600, eval_metrics_every=200, save_predicted_motion_lib_every=3, motion_weights_rules=<factory>, )#

class protomotions.agents.evaluators.config.MaskedMimicActionSource(value)[source]#

Bases: Enum

Student action path used for MaskedMimic evaluation.

POSTERIOR = 'posterior'#

PRIOR = 'prior'#

class protomotions.agents.evaluators.config.MaskedMimicEvaluatorConfig( _target_='protomotions.agents.evaluators.masked_mimic_evaluator.MaskedMimicEvaluator', metric_components=<factory>, failure_components=<factory>, score_key=None, max_eval_steps=600, eval_metrics_every=200, save_predicted_motion_lib_every=3, motion_weights_rules=<factory>, action_source=MaskedMimicActionSource.POSTERIOR, )[source]#

Bases: MimicEvaluatorConfig

Mimic coverage with posterior training evaluation and prior inference.

Attributes:: metric_components: Per-environment numeric metrics; log mean/min/max over scored frames. failure_components: A TerminationRule (any-frame failure) or ThresholdFailure(rule, reduction=…) over sum, mean, min, max or last value per trial. score_key: ‘eval/success_rate’ for checkpoint selection, None for metrics only. max_eval_steps: Maximum steps per evaluation episode. eval_metrics_every: Evaluate metrics every N epochs. None = disabled. save_predicted_motion_lib_every: Save pred_motion_lib every M evals. None = disabled. motion_weights_rules: Rules for updating motion sampling weights.

action_source: MaskedMimicActionSource = 'posterior'#

validate()[source]#

__init__( _target_='protomotions.agents.evaluators.masked_mimic_evaluator.MaskedMimicEvaluator', metric_components=<factory>, failure_components=<factory>, score_key=None, max_eval_steps=600, eval_metrics_every=200, save_predicted_motion_lib_every=3, motion_weights_rules=<factory>, action_source=MaskedMimicActionSource.POSTERIOR, )#