how will you be able to fit in the hydration pack?
seriously though, I've never heard of any pack that wouldn't be able to withstand water. I've used various models of camelbaks in heavy rain, and the only downside is that they can be a bit heavy when soaking wet if there are padded parts of the pack, I guess that might interfere with swimming as well (but perhaps not more than your clothes).
if you don't mind looking like a hunchback Camelbak used to have a base layer shirt with a built in holder for a hydration bladder. that will probably minimize the effect of soaking it.