Eh, since last month I've really cleared the sluggishness issues. Its been very responsive for me. Only times it takes longer than 1-2 seconds for a page to load is at the top of the hour (sphinx rotation uses a lot of load for about 2 minutes, then php rotation 5 minutes later that lasts for 15 seconds). I haven't witnessed any load related 502's either that aren't during sphinx rotation.
I'm going to go blame your isp's peering to our datacenter. I bet if you used a proxy on a well peered network, you'll get the same loading times as us.
Good way for you to test if it's the servers fault or not, run this forum on webpagetest.org
, Once done click on the "Waterfall" option. My averages are 0.5 to 1.5 seconds for european test locations, 1 to 2.5 seconds for the americas.