At least it should be a principle, but sadly I have seen otherwise (Ivory). This leads to the effect that one needs much more RAM than strictly necessary to prevent the machine from paging, or turn off the pagefile.

This requirements are quite modest, so I might have referred to them ;-)

64 samples is acceptable, 128 is a bit too much for my taste, it starts to feel sluggish. It is good to know that you have seen setups with 32 samples, even if it is not the common case.

This sounds like you are locking down the pages in memory, which is a good thing. This means once it is loaded, it will never get paged out. So I can assume that if it loads it will run.

May I ask again at which latency you are able to run it on the recommended machine without clicks/pops?

Low cost for Harddisk space, yes. But the cost for distributing such a large amount of data as DVDs might be substantial. And juggling this amount of DVDs is horrible. One could ship HDDs instead or offer downloads (see above...)

CEUS?  :-P

Regarding the Demo: really nice dynamics :) Is there soft pedal involved?

I have some questions now that the Imperial finalizes:

• What latency can I expect with a decent Computer? What is the lowest latency you observed without clicks and pops? My soundcard can go as low as 48 samples @96kHz, and Ivory is able to handle it flawlessly. Can I expect likewise?
  
• Will the damper pedal be continous, i.e. is there something like 'half pedal' or 'quarter pedal'?
• And one more philosophical question: Is the MIDI-velocity really sufficient to define the sound? Is there really no way to control the tonal quality through the way this specific velocity is being achieved? Now that you have the technical possibilities to capture these nuances, it would be really interesting to bring this to digital pianos. Of course MIDI would not be able anymore to transport this and a new generation of keyboards would be necessary. I'm really interested what the future will bring :-)
  

The reason why the Vienna Imperial is possible is, iirc, that there exists a piano that can be actuated by a computer with very high precision, otherwise it would not be possible to record 100 distinguished layers. To do this with a violin, you have to find a way to achieve something similar. I never heard of a computer actuated violin...

Are you offering me a preview for the Imperial? Go for it :D

Why not? I wouldn't mind downloading any size if that's make it a bit cheaper. E.g. my Kabel Deutschland modem has 32MBit/s, that's about the speed with which my DVD drive can copy data. It would even save me the torture of DJ-ing all those DVDs!

So, even if it's 250GB, a download version could be an option.

 

This is a nice idea, but hard to implement on device level. Normally the device doesn't know anything about files, it only knows about sectors. Of course you can teach your device how FAT32, NTFS, UFS, ZFS, but you can imagine how error prone that is. Also, if the user builds a RAID with these device, each device only sees partial filesystems.

A different approach would be to add a 'learn' button. As long as this button is held down, every read access is mirrored to flash. Otherwise the content is untouched.

This would lead to the following usage szenario: the user installs his libs to our drive. Then he starts his VST host and loads one instrument after the other, always pressing the button while the application reads the sample headers. This will give the drive exactly the information it needs, regardless if the data is organized in single or monolithic files.

Cheers,

Arne

 

Of course this wouldn't be a global option but a per-instrument option, so the user will be able to add tracks in the usual low-latency way.

...350ms to finish the first HDD access...

You should ask your developers if they thought about this. If I hadn't already have a good job I'd build a business case around these two ideas... 

I'd like to go back to the motivation, that led to my initial proposal. My goal was to save Steffen from selling his car to buy RAM, because without a car he cannot give me piano lessons anymore.

If I understand him correctly he edits his arrangement in Cubase and plays it back to review the results. He does not play any instrument live, so latency does not matter to him. To save RAM you currently have two options:

a) bounce some tracks to disk so they have not to be rendered on every playback

b) only preload the notes/velocities he really needs.

Both approaches are a bit cumbersome and time-consuming.

Why not add a 'high latency mode', where nothing is preloaded? One could delay playback for about 300ms and use the time to load the samples needed. This way you need not chose which instruments/notes/velocities you need in advance, you'd just have your complete library at your disposal, without the need for a single GB of RAM.

AFAIK Cubase is able to compensate for the latency, so it should be possible to even play some Instruments live in the conventional way. The only drawback I can see is that you have to wait 300ms after hitting 'play' before playback starts.

Just a thought.

Arne 

I see you're still not completely enthusiastic, so please give me one last shot ;-)
I made the mistake to name some numbers without knowing the real ones. So please take my '1, 10, 100ms' only as an example and replace them by the numbers you really use. My intend was not to increase the IO/s.
I'll try again: currently,
 - you preload 64k for every possible stream. This gives you 350ms for the first HDD access
 -  the HDD is accessed in portions of 32k every 175ms
change it to the following:
 - you preload 8k for every possible stream. This gives you 45ms for the first _SDD_ access. You consented reducing the preload buffer to 1/8 in case of SDD should be possible
 - read 2x32k from SDD to fill your buffer. This give you 300ms (given 40ms access time for the 32k and 10ms for the second) for the first HDD access
 - beginning from the third access, read from the HDD in 32k portions every 175ms as usual
So I don't want to add any accesses, just direct the first two to the SDD instead of HDD. There will be no additional I/O or CPU load.
Also you don't need much bandwidth to the SDD, because you need it for the first 64k of each sample only. Given a bandwidth of 50MB/s stated in an earlier post this would give you 780 events/samples per second, independant of the polyphony. For a massive polyphony you only need fast harddrives. Experimenting with the buffers might show that it is sufficient to read only 48k instead of 64k, yielding 1000 samples/s.
As a further optimization, as mentioned in my last post, you can start the HDD access right away and not only after the data from SDD arrived.
Cheers,

Arne

ok, let me try to explain :-)

I have to admit that I haven't used  VSL yet, I only have experience with Synthogy's Ivory, but I guess the way it basically works is the same.

My understand how it currently works is as follows: When the engine initially starts, it reads the first part of each sample (of each sampleset the user has selected) into RAM, let's say the first 10ms. This amounts to quite some GBs. Additionally, the engine allocates a buffer for each polyphony level. These buffers can hold a much longer period, let's say 100ms. These buffers are cheap, because you only need very few of them compared to the amount of buffers you need for the first 10ms of each sample.

The moment a (MIDI) event arrives, the engine can start playing the sample right away, because it has 10ms buffered in RAM. It starts playing from this buffer. Simultaneously it allocates one of the 100ms buffers and directs the HDD driver to fill it with the sample data from 10ms-110ms. After the first 10ms have been played, hopefully enough data has arrived from HDD to continue from the larger buffer. This buffer will constantly be refilled from HDD until the sample ends or playback has been terminated. Afterwards the 100ms buffer will be released. If the sample is being played again, the data starting from 10ms will have to be fetched from HDD again. The 10ms are chosen in a way the HDD has enough time to respond.

Now the same in my 3 layer model:

At installation time, we copy the first 10ms of each installed sample to SSD. This data will stay there until the sampleset will be deinstalled.

At engine startup, the engine reads the first 1ms of each sample into RAM. Additionally it allocates the 100ms buffers as above.

The moment a (MIDI) event arrives, the engine can start playing the
sample right away, because it has 1ms buffered in RAM. It starts
playing from this buffer. Simultaneously it allocates one of the 100ms
buffers and directs the SDD driver to fill the first 9ms of the buffer with the sample data from 1ms to 10ms. Also, it direct the HDD driver to fill it with the sample data from
10ms-101ms. After the first ms have been played, hopefully enough
data has arrived from SDD to continue from the larger buffer. After 10ms the data from HDD will have arrived so that there will be no disruption in playback. After that, this
buffer will constantly be refilled from HDD in the same way as above until the sample ends or
playback has been terminated. Afterwards the 100ms buffer will be
released. If the sample is being played again, the data will have to be fetched from SSD and HDD again. The 1ms is chosen in a way the SSD has enough time to respond, the 1+9=10ms are chosen in a way the HDD has enough time to respond.

The model described here bases on how I would implement sample streaming on an first impulse, because I have never done it. Please correct me if some (or all ;) of my basic assumptions are wrong. 

 

let's say "interesting" ;-)

Sample streaming has, but latency has not. Clearly there are well behaved systems and systems that are not. From my experience Windows is  nightmare regarding latency.

Hi,

 

I feel you have mistaken my proposal a bit: I do not want to write the data from harddisk *through* the flash. The idea is to organize the data in a way that let's say the first 64k of each sample reside on flash disk, the rest solely on HDD. There won't be any writes to the flash disk.

Also because you only read a small amount of each sample from flash, you don't need that much bandwidth. The overwhelming amount of data still come directly from HDD.

There are no additional accesses and no additional buffers. At the moment you have 2 sources for the data, first RAM (very low latency), second HDD (high latency). Just add a third source, SSD (low latency).

Regarding latency: I cannot imagine that the access time to SSD is > 0.1ms on a well tuned system as long as the drive is not overloaded, but I don't have experience with windows there, I'm a SUN/Solaris guy ;)

I don't think an experiment with _all_ data on SSD does prove anything in regard to my proposal because in this case you'll run quickly into bandwidth limitation which you won't if you follow the procedure outlined above. 

cheers,

Arne