I've done some searches and I've found other threads with the same issue, but there hasn't been any answer as to whether there's a solution. See threads here: http://community.vsl.co.at/forums/p/29888/192076.aspx - http://community.vsl.co.at/forums/p/31191/198885.aspx
For me it definitely is related to the number of VE instances in my sequencer. With no instances connected, CPU usage in activity monitor on my slave is normal. For each instance I connect, it jumps quite a bit. With 6 instances connected (one each for strings, winds, brass, perc, pads, etc), VE's idle CPU usage is around 50%. I'm using Pro Tools 10.3.9 and an Mbox Pro 3 but as far as I can tell people have experienced this with all kinds of DAWs. And I'd be okay with cutting down to one mega-instance but Pro Tools only allows 16 midi ports per instance, so I would need at least 3 anyway.
Here's a screenshot of the CPU before and after connecting to Pro Tools: http://i.imgur.com/Fky4ZEO.png