Rain is how the manchineel gets you: water running off its leaves picks up milky sap and drips onto bare arms and shoulders, raising blisters. No touch required.
The fruit is worse. In a BMJ account from 2000, radiologist Nicola Strickland tasted a sweet beach apple on Tobago, then struggled to swallow for hours.
That is why people paint warnings on the trees along Caribbean and South Florida coasts. The manchineel will never give you one.