protomotions.agents.fine_tuning.agent module

PPO base class for agents that fine-tune frozen pretrained modules.

class protomotions.agents.fine_tuning.agent.FineTuningAgent(fabric, env, config, root_dir=None)

Bases: PretrainedModulesMixin, PPO

PPO base for models that capture frozen pretrained modules during setup.

pretrained is populated before create_model(), consumed by the concrete model, then dropped after model reset. Surviving references live through the model that captured them. Subclasses should put adapter setup or other initialization that depends on those captured modules in _post_create_model_hook.

abstractmethod create_model()

Build the trainable model. self.pretrained is already populated.