As I understand it, gzipping opens up a security vulnerability (BREACH/CRIME) if I use it with SSL/HTTPS.
What if I only use it on my CSS and JS files, is it still a sec
From what I understand, the answer is no - it's not a security vulnerability. CRIME/BEAST attack injects chosen plaintext to uncover original plaintext; in your case this would CSS and JavaScript, which carry no security value. (Presumably, you serve them over HTTPS to avoid mixed content warnings on the browser).
The attack cannot uncover your per-session symmetric key, so it cannot affect your sensitive content assuming it does not use gzip/deflate. Of course, if you wish to be 100% sure, you can also consider chunked encoding in addition to gzip, as per this article: https://community.qualys.com/blogs/securitylabs/2013/08/07/defending-against-the-breach-attack