Vienna Symphonic Library Forum
Forum Statistics

180,783 users have contributed to 42,141 threads and 254,364 posts.

In the past 24 hours, we have 1 new thread(s), 11 new post(s) and 66 new user(s).

  • Another hard drive question

    is buffer size the same as cache and is there a noticeable difference in performance between 2mb, 8mb and 16mb to justify the price differences?

  • (How frustrating to do not get any response...)

    Buffers are the size of the smallest part of a file on a drive, and you can tweak it.

    Cash is the physical memory built in a drive, and can’t be changed. It’s a bottleneck for performances. And really justify the price...


    Regards.

    Gabriel Plalame

  • last edited
    last edited

    @Another User said:


    Cash is the physical memory built in a drive, and can’t be changed. It’s a bottleneck for performances. And really justify the price...


    Cache, like its real-world counterpart, exists to speed up access to frequently or recently used information by keeping a copy of it close to where it was last used. Much of your PC's memory serves as a cache at any one time, keeping copies of things that have been recently written to, or read from, disk. Since memory is at least thousands of times faster to access than a hard disk, this is very beneficial when done right.

    The cache in your hard disk is a smaller example of the same thing, and stores copies of recently read or updated blocks on that disk, in case you want them again. Cache on a hard disk is never a bottleneck, since it always runs faster than either the communications link to the computer or the disk mechanism itself.

    However, whether you can benefit from that cache depends on how you use the data on the disk. By its nature, a cache cannot give you what it doesn't yet have a copy of, so it can only speed up the second and subsequent accesses to information, not the first one, since that has to come from the disk or the PC in order to get into the cache. Moreover, if you retrieve more data than the cache can hold before asking for the second copy of something, it will have been flushed from the cache anyway, and won't be available.

    So, if your computer spends a lot of time reading and writing data in the same general part of the disk, then the cache will be beneficial. But, if you spend a lot of time just streaming large amounts of data from the disk, cache will be of no use to you at all, since you're not asking for something twice before all of cache is over-written with new information.

    Note that operating systems and disks often retrieve information co-located with the information you actually ask for, to speed up common patterns of disk access. 'Read-ahead' is one such type of opportunistic retrieval, where the next few chunks of information on the disk get retrieved at the same time as the thing you asked for, on the off-chance you ask for the next piece shortly. So you might not have to ask for exactly the same thing again to benefit from the cache, as long as you ask for something close to it on the disk first.

  • last edited
    last edited
    Hi, FCW.

    About blocks and buffers, you are right, I made a confusion, I didn’t know that the right term was block. (Thanks to clarify my poor English...).

    About cache, I just meant a 2MB is a 'bottleneck' compared to a 16MB (if it’s in use as you explain it very clearly).

    @Another User said:


    Note that operating systems and disks often retrieve information co-located with the information you actually ask for, to speed up common patterns of disk access.

    Interesting point.

    Is this an OS and disks’ driver privilege only, or can a software manage this itself ?

    Kernel level (Gigastudio) allows that ?



    Gabriel Plalame

  • crystal, there is a complex hierarchy of caches for hardware and operating system so thereis no simple answer.
    i'd just like to comment on the bottleneck (2 vs. 16 MB). in fact a bigger cache behaving *badly* can be contraproductive. for *normal* file access it is a good idea to *read ahead* because higly probable the application will call for the *next data*.
    for streaming the application just calls for tiny parts of a file (finally just to fill the buffer of the soundcard for samples that are played), this is between 16 and 128 KB . if the operating system or even the disk itself now fills its cache with the whole file (which is say 1 MB) just to drop it a few ms later, this will not increase performance - rather the opposite.

    a good example is the apple raid, which has to be configured to not cache if it shall work best with logic. especially raid controllers try to chache as much as possible behaving the *common* way (including eg. video-streaming) and not for sampling
    christian

    and remember: only a CRAY can run an endless loop in just three seconds.
  • Hi Chistian.

    Thanks for those precisions. (This thread looks more and more instructive.)

    So, it means that softwares can’t handle themselves the way disks and OS manage caches, I suppose ?

  • Christian, this is great information. Can you give some "ideal" settings for a hard drive or file server? Block size, cache settings, disable journaling, etc.?

  • synthetic, especially during the last months and related to the Vienna Instruments (which call for much harddisk space) several times the question came up how *good* a certain raid-system would be and i've often warned to check out its behaviour before buyingfor for exactly this reason. unfortunately it is almost never mentioned in the specs if and how chaching behaviour can be configured - i for myself have been pointed to this issue years ago when 2 MB used to be a *big cache* for a harddrive and promise onboard controllers made problems when they should not.

    for single disks and some raid controllers in many cases this can be set in the device manager (expand harddrives and select properties of a certain disk - go to the tab policies, uncheck read cache if available), for others there are special tools available on the vendor's website to disable/enable read-/write-cache, for apple's Xraid you need to use terminal, some raid-controllers are not configurable at all.

    IMO read caching is not neccessary at all for sample streaming, but an old technology from SCSI-drives is great: native resp. tagged command queuing (NCQ, TCQ) which can be found with many sATA-I and all sATA-II drives, what means that data requests to a harddrive can be re-ordered in a way that the heads of the disk has to move as little as possible (of course also the sATA controller where the drive is connected to has to support this feature)

    for formatting i'd recommend a sector-size of 4096 (default would be 512 for NTFS) - resulting in 1/8 overhead from the filesystem. select more on your own risk, because windows cannot repair a corrupted filesystem with larger sector size with built-in tools. (btw: FAT32 uses 4096 by default which was probably the reason why it has been often percepted as more performant than NTFS, but i don't recommend FAT32 for other reasons)

    HFS+ is a complete other story, this filesystem is using B-trees and not only blocks (512 B) but also block-groups which offers a good performance in most cases by default (i don't find the B-tree concept and the use of extended instead of primary partitions a good idea, but this is another discussion).
    journaling (HFSJ) is always a bad idea for drives where data will be replaced, overwritten or deleted, because you are loosing the space for the deleted/overwritten files.
    (please note journaling is usually a very good idea for system volumes and for protools it is recommended for other reasons)
    for the same reason under XP system recovery is contraproductive and should be switched off for drives holding samples.

    AFAIK the caching behaviour of the operating system cannot be configured at all - possibly there are registry values which could be changed but this is always risky because you never know how in detail it will effect other behaviour too.
    christian

    and remember: only a CRAY can run an endless loop in just three seconds.