He pulls back and so do they, worried that they'd slid over a boundary or poked him in the eye or something--and then they see where he's looking.

Frisk visibly struggles with a flash of irritation. He doesn't need that right now, it's not his fault he doesn't know and is worried about it. "Yeah. 'S old. Don't worry." They tug their scarf back into place.