Eh, since last month I've really cleared the sluggishness issues. Its been very responsive for me. Only times it takes longer than 1-2 seconds for a page to load is at the top of the hour (sphinx rotation uses a lot of load for about 2 minutes, then php rotation 5 minutes later that lasts for 15 seconds). I haven't witnessed any load related 502's either that aren't during sphinx rotation.
I'm going to go blame your isp's peering to our datacenter. I bet if you used a proxy on a well peered network, you'll get the same loading times as us.
Good way for you to test if it's the servers fault or not, run this forum on
webpagetest.org, Once done click on the "Waterfall" option. My averages are 0.5 to 1.5 seconds for european test locations, 1 to 2.5 seconds for the americas.