VMWare and VE3...hmmmm interesting idea? Or technology blackmail?

[composer22]

I think I am going to proof of concept out VMWare to see how it might work as a solution for a 64 bit server running VE3. I dont see anyone on the threads here trying this in the past. The only reference is this:

http://community.vsl.co.at/forums/p/11127/83039.aspx#83039

which is fairly old and doesn't utilize the newer VE3 nor multiprocessors in Leopard etc .

Before I get into this, there are some technical gotchas I can see coming out of it. For number one, USB port restrictions running VMware. So I have some questions up front:

1) Does the master side (DAW calling on the services of a 64bit-VE3 Server over ethernet) need to have a VSL USB Key, or is the key only needed for each VE3 server instance you need to run. For example, I run a DAW and use plugin to communicate to VE3 running remotely on machine 2 - Do I need a key on the DAW for the plug and a key on machine 2?

2) If it truns out you need a key, I believe you can pick and choose USB devices you want to use within VMWare. That would mean two VSL keys - one for DAW side and one for the VE3 side under VMWare? My experience is under parallels, so if anyone has experience with VMWare, some consideration in thoughts?

[cm]

a license for VE3 (and so a key) is only needed where you want to open an interface (*the GUI*) and load samples

hth, christian

[composer22]

if this be true, then I have confidence it really might work

I downloaded VMWare demo license and installed a home edition so I could look at the options of a Virtual Machine, and also looked at the specification online for Parallels and VMWare. VMWare is the only logical choice as it is the only VM Desktop version right now that supports 64bit and that also allows you to partition your processor cores to be used within the Virtual Machines you create. Parallels, which I have, doesn't allow for these options. Parallels does allow you to choose which USB devices to map to VM or to OSX.

The USB may or may not be a problem under VMware. When I loaded a VSL stick into the USB drive, it asked for a driver. So, not sure how this might work under a 64 bit OS version and whether it would be able to read it - especially with VE3? Perhaps it may need some driver somewhere?

Then the real pondering question is how useful would a VM hosted VE3 be? If I use Sibelius, I have to set up separate devices for each MIDI port I want to use. For VE2, it would create a separate instance for each one controlled by VE Service. That single service hosted all the VE2's in a single memory space. I'm not sure how it would work with a VE3 remote instance, and whether it will still want to create a new remote instance, or be able to utilize a single instance. The ideal would be to use all devices in Sibelius that are using different MIDI ports be able to communicate to that single VE3 remote instance. In such case, you could load many samples into that one instance and save it as a single project. I perhaps need to read the VE3 documentation some more to see how sequencer examples might relate to Sibelius.

If multiple instances of VE3 are launched to cover each device and MIDI port in the Sibelius setup, then I don't really see any benefit in having this abstract layer of software and overhead with a VM. Multiple instances are launched and loaded in both scenarios. Its just a matter of speed and convenience. e.g. The user just has to launch and load individual VE2 instances on OSX to cover the same needs. Sure, its a manual method and needs MIDI and audio routing, but its workable.

Now, if I was working on a DAW, that might be a different story. But, I'm curious if anyone has VE3 working remotely with Sibelius and whether there are any peculiarities about the need for multiple instances running within its device space?

[cm]

you would need to install the LCC in your guest system (this would bring the drivers for the *security device* ... possibly it asks for a manual installation of protect executor then ... let the hardware assistant do the work for you ... 64bit is not an issue here.

 

my only concern would be the latency added by the virtual machine  ... audio goes across the virtual network device, for MIDI maybe use MoL since its cross-platform

 

someone told me already about testing VE3 in VMware, just can't remember who ...

christian

[composer22]

Running on the same machine with a Core 8 I believe would handle it. I can see a case for it if you had one machine and only wanted to utilize it for a sequencer - out of convenience. Or, you had a OSX machine already setup that you wanted to utilize for a stand alone server and did not wish to use Bootcamp. Otherwise, not sure its really useful (for me at least).

The VM can be set higher priority than OSX if there is a processor slice allocation issue. You can also tell it specifically to use n-cores or a particular processor out of the two you have installed, which would allow you to distribute the processor according to need. VM can be set to take control of a device such as a particular network card etc. A little tuning could make it efficient by balancing both priority and cores to sequencer/score vs VM/VSL and specific hardware. Not sure latency would be that great a problem.

It's just a question on application and need, which I think is very narrow - convienience

One benefit is that you could theoretically clone VM machines and drop them easily on OSX servers. Another one is "freeze" the VM, samples loaded in full, and then next time you want the server up launching the VM to have everything preloaded for playback where you left it last. Sort of like sleep to disk.

[composer22]

Thinking some more on where I should go next in this experiment. What I might want to try and do is install a VM w/ XP 64 or Vista Premium Home on the Mac Pro 8 and see how Sibelius communicates to it via Mac Pro. For one, I don't have to uninstall VE2 from OSX MP8 just yet. I'd also like to see how VMWARE might work with freezing a runtime of VE3 to see if it can be restored. Why would you want to do this? Well, it might be faster to load. And...maybe the validation check against the Vienna Key, since it has already been performed, doenst need to be performed again, meaning theoretically you would not run out of trial licenses. :)

Any preferences on XP over Vista for a MP8 with 16g?

BTW, confirmed last evening - on a MP8 with 16g I could only load 4 instances of VE2 before screen problems started to occur with a 5th instance. My memory loads between all 4 instances was about 5G. Theoretically, based on your comment about 2.5G each, 10G would be the max memory load one could to with 4 instances.

[DG]

I would try with XP64 over Vista64. However, maybe sp1 has improved Vista from when I first tried it. You can also get a free, time limited trial of XP64, so it won't cost you anything. It also has a smaller footprint, so you may be able to load more samples.

Edit. I think that there is a problem with more than 4 standalones, but can't quite remember as it doesn't affect me. Maybe do a forum search.

DG

[DG]

Another thought. Why are you only loading 5GB with 4 instances? I can load that in 4 instances of the plugin version, so the standalone should be able to load much more.

DG

[composer22]

@DG said:

Another thought. Why are you only loading 5GB with 4 instances? I can load that in 4 instances of the plugin version, so the standalone should be able to load much more.

 

DG

DG sorry for the confusion. I only had 5 G of samples allocated. I could probably have loaded a lot more. I had them broken down into 4 groups - ww/brass + Percussion/Keys + Harp/SoloStr + Strings. The 5th instance I was loading was an attempt to feed the other four into a cheap mixer. Thought the overhead of using a sequencer overkill for just mixing the monitor.

As for XP 64 bit, unfortunately MIcrosoft doesnt allow evaluation downloads now. I picked up an OEM copy today and just started the process of downloading a evaluation copy of VMWare and configuring this thing on this machine. I'll let you know how it goes. It will be a while here as I troubleshoot - and specifically the use ethernet communication to and form teh VM.

[composer22]

I can report back this evening successful results in getting VMware to run XP Pro 64 and VE3 on the same machine with OSX.

I basically installed VMWare + XP 64 + VE3. VMWare can access specific USB ports you wish to make viewable to it, so it can be set to read the VSL Key but ignore other USB ports such as my iLok key which I use for other things in OSX. You may need to format a DOS drive (I didnt test an apple drive) for your vienna samples and point a shared directory to that drive so Directory manager in VSL can see it. VMWare / XP can allocate around 8 gig with XP 64. I don't know why there is a restriction here. Perhaps Vista Home Premium 64 bit is a better choice because it can access 16g? or maybe VMware has this throttle built in?

The other setting to be aware of is in networking. This should be set to "Allow Only Networking with the Host". This will safely keep it from attaching to a network router and set it to communicate directly to your OSX host.

You can also set the number of "Virtual Processors" to use. I took the default of 1 processor, but I don't know what this setting "really" does in terms of performance or emulation or partitioning of threads.

The configuration setup is a bit different than XP Home edition. In Home edition install I was able to tell home may cores it could utilize.

In testing this configuration, I used Sibelius and was able to load my complete working set of 5 - 6 gigs from within Sibelius and SAVE the document and complete configuration. When I loaded up the Sibelius doc again, all the instances were instantiated remotely and projects loaded in the VM just like Sibelius has been designed to do.

In terms of performance or latency, I didn't have time to really measure this, but I did not see any significant spikes when playing a long chord with 23-24 voices. It didn't appear to me to be an issue.

XP64 feels very stable here and didn't hiccup at all. The only cons I see are the limited about of memory - 8 Gigs, and that AU plugins are not available within the 64x environment. Using 4 instances of VE2 would allow an additional 2G of memory for samples under OSX. Of course, it would not be as streamlined and a bit clunky if not unstable.

This is a MP 8 with 16gig of RAM I used for testing.

Tomorrow sometime I will test out trying to set the VM to look on Apple formatted drives, such as my RAID 0 sample drive which I have Vienna on as well. I'll also see what 2 Virtual processors do to performance and memory partitioning.

If anyone wants to also try this, VMWare Fusion can be downloaded for a 30 day trial. XP 64 must be purchased. VE3 can be downloaded for a limited trial and will use a evaluation counter on your VSL Key. I would like to see what other people can do in terms of pushing this to a limit on a Mac Pro 8 (early 2008) with large memory space in the larger fast scores.

NOTE: from the VMWARE site: "VMware Fusion is compatible with a wider range of operating systems than any competing product and offers exclusive support for 64-bit operating systems and virtual machines with up to 8 GB of memory."

[cm]

thanks for reporting back ... have you installed XP64 with or without servicepacks?

 

the 8 GB limit might have its origin in VMware resp. the applied settings - we have a machine here accessing 32 GB RAM under XP (well, in fact almost 30 GB loaded instruments), so there is no limitation from the guest OS (edit: now i read your note ... seems to apply)

 

my suggestion would be to keep the setting to 1 (virtual) processor if possible (it should be sufficient regarding performance) so avoiding additional load for virtualization with multi-threading

christian

[composer22]

1) I installed XP64 with service packs right up to the minute.

2) Yes VMWare Fusion has a max limit of around 8Gigs for any OS 64 bit. However, you can run multiple VMs on one machine...of course you need VSL Licenses for each of those VM's - and I mean VSL Library licenses $$$ =) This is still better than Parallels, which doesnt allow x64 and limits you to around 2gig.

Are you saying you are running VMWare? It wasn't clear...

3) 1 Virtual processor - I tried 2 today and it seemed to distribute the load better. However, If I have problems when I start working, I will revert back to one processor. The other thing to set the general VMWare seting to have VMWare have a higher priority than OSX. So between processor settings in the VM OS, and the general preference setting to let VMWare have a higher priority should be some levers to move to tune it up. You can also disable other devices, but generally they dont seem to need to be touched.

Another thing I did today was have the VE3 in the VM point to Apple RAID drives for the Vienna Libraries through a shared folder. Normally, I believe XP could not read them. However, in the VM XP could read them just fine and stream. This might save a problem when you want to move from OSX to XP on different machines. For example, I can use the same sounds on my MBPro without having to reformat between DOS and Apple.

I played test chord again with full orchestra. In Sibelius it had a bit of a problem starting. However, after 2 false starts it worked just fine. I am thinking the buffers needed to be filled or something.

So, I am sold on this technology - at least for all my needs. I licensed the VE3 and VMWare today. It seems to be working for me and appears to be stable. But I will need to work with it for a few weeks or months to really become intimate with it.

VMware Fusion has a 2.0 beta right now. I wonder what is in there? Also, I'll bet parallels will make an announcement at the developers conference this coming month of June.

If you havent tried this out, you should. Its seems to be a good bridged solution until you come up with a native OSX 64x version - at least for us folks on Mac Pro 8's with loads of memory. I'd like to get some other peoples opinions and have others try it.

PS: One thing: when in Ve3, and loading a matrix...if I open up the folder/tree on the right display, one of the cores goes really high in processing. I'm wondering if something is throttling in your code at that point...may want to have a look. if I go back to Main display, it stops back down to more idle. .

[timkiel]

Well I too took the plunge to see if I could get VMware working, although I've only a quad core with 8gb RAM. I created a 5GB VMware XP64bit image and have managed to fill it up, have 32 channels working quite happily on the same machine.

I've been using the Suspend option in Vmware which basically copies the RAM down to a disk image. Problem is of course it takes a few minutes to load the RAM backup again! So I'm not convinced of this approach compared to multiple slaves. With the 3 mini's I can at least load 6gb in parallel! I think therefore the next big challenge is around startup times and also a memory maintaining VE - so when you switch projects you don't need to reload everything again.

Regards

Tim

[composer22]

timkiel:

Thats really cool! I was going to test the freeze feature, but I wasn't sure how it could be reconnected to Sibelius again after closing down the applications, and I got distracted with some other toys.

The basic benefit here for me is that I don't have to shell out additional cash for slave machines + peripherals when I have a perfectly expensive MP 8 sitting here hardly being utilized with passion. Hence, this holdover solution cost me something like $300 or so dollars until we see a VE3 64 come along. And then I'll be able to access more memory space. Yeah, still will load slow, but I'll live with it.

Q: I wonder how putting the computer to sleep would affect VMWare and affect reload performance? Maybe you can try this with your rig?

EDIT: I was able to load about 6 gig of samples spread over 5 instances of VE3 in my VM tonight before the UI on VE3 got to be a bit frozen. It was still operational and usable. Tomorrow, I'll try and load a 6th and see if it also can be used - pushing my luck.

[timkiel]

Just tried putting the Mac to sleep and coming back again - instant on! Up and running with all samples on resume.  This of course is expeceted as all the memory is still "live" and powered in sleep.  If you did a safe sleep (which you need to hack on a MacPro) then you'd face the same issue of loading 6GB or more back into RAM before starting...

Tim

[composer22]

Im impressed its working as such. At least its a solution for energy savings. Wow.

I need to see if there are any XP memory tweaks for 64 bit I can perform on this. I know there are some for 32bit in my Giga days. Perhaps that might help me with the interface problems I am experiencing on a 5th instance. I'll also try loading less samples in that last instance.

[timkiel]

Some things you can do - turn off the paging file, turn off background wallpaper and use the smallest size screen that you find usable (e.g. 1024x768). However I have noticed that the CPU spikes to 60% sometimes when redrawing the VE screens, no idea why.... Regards Tim

[cm]

in my experience especially the giga-tweaks are a bad idea, i noticed they actually might lower available GUI ressources .. but who knows ..

i'd rather encourage the developers to optimize further than to start again working with tweaks which may cause side effects ...

christian

[cm]

CPU spikes related to redrawing screen can sometimes be fixed by lowering hardware accelleration in your grafic driver ...

christian

[composer22]

Fixed my system cache OFF Im not sure this is correct. Under 32 windows this needed to be 2.5x your memory size regardless of physical memory. So a 2 gig machine needed to allocate a 5-6 gig fixed swap file. Had to do with some pointer allocation or something...

Background desktop off
Smaller screen size
16 bit colour
No graphics acceleration
Turned off System Restore
Visual FX - Optimize for BEST Performance

Questions:

Which should get priority on these settings?

Processor Scheduling - Programs vs Background Services?

Memory Usage - Programs vs. System Cache?

Still having screen issues...
EDIT: Just upped Virtual Memory to around 11gig, and it solved screen problems. Going to play around some more tomorrow evening with it. Any additional tips from above settings woudl be helpful. Thanks.