GLOVES

Flow-based Policy Adaptation without Policy Updates

A goal-agnostic expert flow that corrects the action chunks of a frozen agent — a VLA or a code-as-policy LLM — and steps in only when an action looks out of distribution.

Luzhe Sun*1 Jingtian Ji*1 Haoran Chen1 Jiawei Zhou2 Matthew R. Walter1
1Toyota Technological Institute at Chicago  ·  2Stony Brook University  ·  *equal contribution
📄 Paper arXiv Poster Code (soon)
45→100%real charger insertion, VLA + GRIP
33→77%Slalom, VLA + LIFT
64→98%Slalom, code-as-policy + LIFT
0.36 msOOD gate per chunk
The problem

Good intent, unreliable execution

Pretrained agents know what to do, but their actions drift under domain shift — and the agent itself often can't be retrained.

Agents propose actions

VLAFLOWER · language goal
Code-as-PolicyLLM writes the code
IL / RLany frozen policy

…but actions fail locally

Misalignment
Collision
Jitter
Misgrounding
Can we fix the action without updating the agent?

Why not finetune?

  • Black-box API no access to weights
  • Human operator a person can't be retrained
  • Few expert demos too few to finetune safely
  • Cost & forgetting may erode pretrained skills
Setup
\[ \underbrace{\mathrm{Agent}(I,{\color{#d9962b}g})}_{\text{frozen}} \mapsto {\color{#d9962b}a_u},\qquad \tilde a \sim \mathcal{A}_\theta(\,\cdot \mid {\color{#d9962b}a_u},\, I\,) \]
GLOVES never sees the goal. The proposed chunk \(a_u\) is the intent cue; the skill comes from an expert prior over \((I,a)\).
One expert flow, two jobs
Train a conditional flow \(v_\theta(x\mid t, I)\) on goal-free expert pairs \(\mathcal D_e=\{(I_i,a^e_i)\}\). Run it backward to decide when to help, and forward (guided) to decide how.
no policy updatesno goal labelsno separate detector
Method

When to adapt, and how

Each proposed chunk is scored by the reverse flow. In-distribution chunks are executed as is; flagged chunks are edited by GRIP or LIFT.

GLOVES overview: OOD gate, then GRIP or LIFT editing of flagged chunks.
When

Flow-based OOD gate

score\( \hat z = a_u - v_\theta(a_u\mid 1, I),\ \ s=\|\hat z\|_2^2 \)
calibrate\( p=\frac{1+\sum_i \mathbf 1\{s_i\ge s\}}{n+1} \)
gate\( \text{edit only if}\ \ p\le\alpha \)
One midpoint reverse step · 0.36 ms per sample · calibrated on held-out expert data.
How ①

GRIP

add noise\( x_t = t\,{\color{#e8892b}a_u} + (1-t)\,{\color{#9e9e9e}\epsilon} \)
estimate\( {\color{#e8705f}a_e=\hat x_1} = x_t + (1-t)\,{\color{#2166ac}v_\theta(x_t\mid t, I)} \)
regularize\( E = \tfrac12\|{\color{#e8705f}a_e}-{\color{#e8892b}a_u}\|_2^2 \)
guided step\( x_{t-1} = x_t + \Delta t\,({\color{#2166ac}v_\theta}{\color{#e8892b}-\lambda_{\mathrm{align}}\nabla_{x_t}E}) \)
Guided Refinement with an Intent-Preserving energy. Needs only the expert flow; \(\lambda_{\mathrm{align}}=1\) everywhere.
How ②

LIFT

agent side\( {\color{#1e90e8}q_k} = (1-t_k){\color{#9e9e9e}\epsilon_k} + t_k{\color{#d9962b}a_u} \)
shift\( {\color{#e8705f}p_k} = {\color{#1e90e8}q_k} + ({\color{#c9a06b}a^{(k)}} - a_u) \)
transport\( v_p = v_u(p_k)+\omega[{\color{#e8705f}v_e(p_k)}-{\color{#1e90e8}v_u(p_k)}] \)
update\( a^{(k+1)} = a^{(k)}{\color{#7b61c9}+\eta\Delta t_k(v_p-v_q)}{\color{#2f8a3a}+\lambda(\hat a_u-\hat a_e)} \)
Learned Inversion-Free Transport. Adds an agent flow \(v_u\) from rollouts; no noising depth to tune.

Try it: GRIP in 1-D

Expert actions live in two modes. The agent proposes \(a_u\) slightly off the upper mode. Exact flow for this toy, computed live.
samples keep the intended mode
mean edit size |ã − au|
Results

Better agents, same weights

Success rate (%) of two very different frozen agents before and after correction. + = with OOD gate.

agent alone prior shared-autonomy baselines (DDPM, CSA) GRIP LIFT

Real robot · charger insertion — precise local correction

VLA
VLA + GRIP
VLA + LIFT

Real robot · cup serving — sustained trajectory refinement

VLA
VLA + GRIP
VLA + LIFT
Rollouts

Simulation, side by side

Same VLA, same scenes — only the action-level correction changes.

Analysis

Does the gate catch bad chunks?

Rollouts have no per-action labels, so we inject known perturbations into a near-perfect imitation agent and measure detection and recovery.

- - clean chunk   — perturbed chunk   · AUROC: mean over 3 tasks
.975precision
.758recall
.938AUROC
.003FP rate
3 tasks × 4 perturbations · α = 0.005 · 1 reverse step · 0.36 ms
Reverse-flow score densities of clean vs. perturbed Keypad chunks.
Recovery across all four perturbations (composite success, %)
Equal-weight mean over Noised / Laggy / Slow / Shift; triangles show the change from Unassisted in percentage points; ⁺ = OOD gate. Residual is trained on these very perturbations; GLOVES is not. Laggy is the hardest to detect: a repeated chunk can still fit the scene — detecting the injection ≠ detecting harm.
Cite

BibTeX

@article{sun2026flow,
  title={Flow-based Policy Adaptation without Policy Updates},
  author={Sun, Luzhe and Ji, Jingtian and Chen, Haoran and Zhou, Jiawei and Walter, Matthew R},
  journal={arXiv preprint arXiv:2606.06461},
  year={2026}
}