Are there server handicaps?

There are a few low population servers that are further along than some high population servers in the Sunwell Isle progression. This has lead to a few questions on the forums. Blizzard had the following to say...

Each phase, and the subsequent "add ons" for each phase (anvil, alchemy table, etc.) require a set number of turn ins of the specific quests that fund them. There's no mechanic that increases that number. So no, a realm with a higher population is not handicapped.

There is however a mechanic that will auto-complete each phase after a set amount of time for those realms that aren't able to complete them within a "decent" amount of time.

What you may be seeing is a general cool down of people taking part in those quests? I can't really say why your realm may be seeing a slower progression aside from them just not completing as many quests as another realm.