protomotions.agents.evaluators.mimic_evaluator module

class protomotions.agents.evaluators.mimic_evaluator.MimicEpisodeContext(motion_ids, frame_limits)

Bases: object

Per-episode-batch state for mimic evaluation.

motion_ids: MockTensor

frame_limits: MockTensor

__init__(motion_ids, frame_limits)

class protomotions.agents.evaluators.mimic_evaluator.MimicEvaluator(agent, fabric, config)

Bases: BaseEvaluator

Evaluator for Mimic agent’s motion tracking performance.

__init__(agent, fabric, config)

Initialize the evaluator.

Parameters:

• agent (Any) – The agent to evaluate

• fabric (Any) – Lightning Fabric instance for distributed training

property motion_lib: MotionLib

Motion library (from agent).

property motion_manager: MimicMotionManager

Motion manager (from env).

initialize_eval()

Initialize evaluation tracking and cache the state eval will overwrite.

Saves the env state, the model’s rollout context, and the motion manager’s clip assignment, all restored by cleanup_after_evaluation.

evaluate_episode(env_ids, max_steps)

Run a single episode batch with the policy’s own actions.

Evaluation is an ordinary rollout in which only env_ids are scored. Every other env plays its background clip (see _background_motion_ids) and, like a training env, restarts it whenever it falls or the clip ends. No env is parked or left idle: all envs share one physics scene, so the unscored ones should load it the way training does. Parking them below the floor instead cost ~12.8 points of SAMP eval success at 8192 envs (75.96% vs 88.80%). Scoring stays on the deduped env_ids subset – widening it would corrupt the motion-id-indexed buffers (duplicate indices drop writes).

Every env reset here, scored or not, also gets fresh model rollout context (for example MaskedMimic’s per-episode latent noise), as training does after an env finishes. env.reset already clears the env’s own history.

run_evaluation()

Run evaluation across multiple motions.

process_eval_results()

Process results and update motion sampling weights.

cleanup_after_evaluation()

Restore the env, model rollout context, and motion manager state.

env.restore_state first clears the simulator’s contact memory (see Simulator.flush_contact_cache), then writes every env’s saved state back, so training continues the episodes it was running before eval.