yep, with some lowriders I reckon it’ll handle better esp. up climbs, keeps the front wheel down.
All bikes differ slightly in how they cope with a front load but I found my CX bike worked best as a tourer with low riders, half frame bag and big saddle bag, no rear panniers or handlebar bag.
I think whilst convenient the handle bar bag had the biggest influence on affecting the handling in a negative way. I started just stuffing a musette in my frame bag and then chucking my wallet, camera etc in it when I stopped.
You might not have the luxury of just 2 panniers but a front load (experiment with the light bulky stuff up front or the heavy small stuff) will help even things out.