It's genuinely heartbreaking because Fredo is so repentant, and because it seems clear (to me at least) that he didn't know his actions would lead to an attempted assassination of Michael. Of course, even more bone chilling is Michael's pretend forgiveness later on, before he has him killed. Ice cold.