その時一緒にいなくてよかったね 、と英語で言いたい

It would have been better if we were not together that time. だと思います。