I think there has to be some point at which the joke-teller bears a degree of responsibility for causing additional emotional trauma. For example, what if you tell a rape joke knowing that the recipient bears emotional scars for being raped? I think the vast majority of individual would say that the joke-teller is responsible for triggering that trauma, even if he was not the rapist.just because I have not made the choice to guarantee that nothing I say will ever cause someone to remember being raped, does not make me responsible for the harm of them remembering the rape. That harm was caused by the rapist.
Conversely, if joke-teller believed (mistakenly) with 100% confidence that the recipient was not raped, the vast majority would say that he is not responsible.
The problem is that those two scenarios are very rare, and the vast majority of common scenarios are far murkier, and the larger the audience, the murkier it gets and the more likely that we're going to trigger someone. I think based on that we can assign some degree of responsibility to the joke-teller.