You can use Elliptic Curve Diffie-Hellman, assuming you have the public keys, and not just addresses. Bitcoin addresses are derived from public keys, not public keys themselves, so you'd be required to share the raw public keys.

Here's another resource, which links to a python example creating a shared symmetric key with the secp256k1 curve used by Bitcoin.