He... isn't sure what he said wrong. All he knows is Frisk is still and quiet, suddenly much less comfortable than they were even when they began this conversation. He wants to apologize again, but it seems Frisk doesn't really enjoy that.

So... empathy may help.

"I understand how you feel."

Not being sure whether your family actually misses you, he means. He has a feeling he has doubts about it for very different reasons than Frisk, but at least if he shows they have some similarities, maybe they won't feel so bad.