Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error

Forum Jump  
Apple RAID
Last post Thu, Mar 31 2005 by stupid8track, 30 replies.
Options
Go to last post
2 Pages12>
Posted on Thu, Mar 03 2005 19:12
by garylionelli
Joined on Wed, Jul 09 2003, Los Angeles, Posts 387
My quest to shoe-horn my entire studio into a single G5 Dual running DP under DAE (with Pro Tools HD2 hardware) is just about there. I’ve gone from 3 G5s Duals (each with 2 FW800 drives and 2 SATA drives) down to 2 G5s, and am trying to get to 1. The TDM plugs handle all the heavy lifting for FX, EQ, etc., and there are 6 LaCie FW800 250GB drives attached for all VIs, including the complete Kontakt Horizon series. Problem is, as I’ve found out, you cannot daisy-chain 6 FW800 drives off the built-in FW port on the G5 because you will flood the FW bus. Hook up that many drives and simple tasks such as copying data between any 2 takes forever, even if only 2 of the drives are actually working (bandwidth is reduced by just hooking up that many drives, even when idle). After talking to LaCie tech, and with one open slot in my G5, I installed their FW800 PCI card for 3 of the drives, but found that the FW bus is still slightly overloaded as they recommend a max of 2 drives per card (unless they’re just being used for storage). So, my setup now is 2 FW drives on the built-in FW port and 2 on the FW card, and this is working for now. However, with all the libraries accessing so much data, I’ll need more drives.

I’m aware that RAID setups have proven to not help performance with Kontakt/VSL, etc., but I think this is largely because most drives these days are fast enough for software samplers without being configured in a RAID setup. However, overloading the bus is where the problem lies, and that is why I’m considering the Apple RAID system because, according to Apple’s description “each 7200 RPM hard drive module connects to a dedicated Ultra ATA drive channel to eliminate bottlenecks and maximize the 400MB/s Fibre Channel host connection. Throughput is assured, which means bandwidth remains constant, even as more drives are added in a fabric configuration.”

So the reason for my post on this is to try to find more info using Apple’s RAID with software samplers, et al. Has anyone tried one, or can anyone point to a source of info on the subject as it relates to music applications. I know Apple privately displayed a super Logic Pro system at NAMM that included 2 of their RAIDs, but can’t find any more info about who set it up or how/if it worked.

(BTW, I’ve found that with the entire Horizon series now in Kontakt on my DP system, I’m relying on my 2nd G5 that runs Logic/VSL Pro Edition very little, if at all, for most projects.)
Pro Tools Ultimate, Avid MTRX, Mac Pro 3.5 Ghz 6-Core (2014) 128 GB RAM, Mac OS Big Sur, Samsung SSDs
Posted on Fri, Mar 04 2005 16:22
by stupid8track
Joined on Thu, Mar 13 2003, Posts 230
gary

i can't speak for RAID with DP/Kontakt/DAE - but i personally feel that RAID isn't designed/intended as a solution for sample streaming. RAID is all about throughput and sample streaming is basically all about seek time.

so what you have right now - multiple drives - is the best solution for sample streaming (though, with logic, multiple drives don't improve performance - i wouldn't be surprised if kontakt/DP/DAE actually benefited from multiple drives). your current problem is merely a 'bus' limitation.

are you thinking about the apple RAID so that you can recycle your current ATA drives that are inside the firewire cases? or are you buying it straight up, with new drives and all? can the apple RAID not be used as a RAID - but as multiple drives? if so, that could be a great solution.

but if not, perhaps you should consider buying multiple SATA drives, an external SATA drive enclosure, and an SATA PCI card, like this http://macgurus.com/prod...pages/sata/tsata-x44.php"="" target="_blank" title="http://macgurus.com/productpages/sata/tsata-x44.php">http://macgurus.com/productpages/sata/tsata-x44.php">http://macgurus.com/productpages/sata/tsata-x44.php. 300+ MB/sec bandwidth, up to 8 external ports, one PCI slot.

that would give you the bus bandwidth you need and preserve independent seek times of all the drives (all drives wouldn't have to be involved with delivering all samples).

i've found the macgurus (the company linked to above) to be very knowledgeable and helpful - and, if you want, they will build a system to suit.

unfortunately, i'm not aware of any (non firewire) multi-drive ATA enclosures.

if the Apple Raid can be run as independent, unraided drives - then that is probably the simplest solution (you can then see if the RAID improves/degrades/doesn't matter for performance vs. multiple drives) and keeps the troubleshooting to a minimum (call apple).

but if it can't, consider emailing the macgurus. also check out xlr8yourmac.com for reports on any SATA PCI cards.

if you decide to go RAID, you could also consider contacting the people at hugesystems.com and asking if you can buy one of their RAID enclosures. they have at least one which uses ATA drives. so you could recycle your drives and just add a scsi card to the G5. but they are hardware RAID, so you'd have to use the drives striped.

anyway, hope this helps. whatever you choose, please report back with real world experience.

the DP/kontakt option is a compelling one.

cheers
Posted on Fri, Mar 04 2005 18:46
by garylionelli
Joined on Wed, Jul 09 2003, Los Angeles, Posts 387
S8T,
Thanks so much for your input. I just spoke with Rick at MacGurus -- great guy, great resource! Everything you always wanted to know about hard drives...he confirmed what you said. With SATA, by default, you get drive channel throughput independence, unlike Firewire 800 (which is only good for 1 or 2 drive setups). He went on to explain that the Apple RAID would most likely work despite the fact that's it's a RAID, but that a simple SATA 6 or 8 multi-drive non-RAID setup would work just as well, if not better (and a lot cheaper). He also explained why he thinks a non-RAID setup does work better for sample-streaming (has to do with with RAIDs retrieving data sequentially), but I don't feel qualified to re-explain it. He also mentioned that I could very easliy recycle/convert my LaCie FW drives to SATA. He didn't know if the Apple can be configured as non-RAID.

Looks like the new Sonnet SATA card with 6 drives is the way to go. I had thought about getting this card before I went with the LaCie FW card, to avoid having to switch drive types, but I'll have to go with what works. I'll let you know how this all plays out -- thanks again.
Gary
Pro Tools Ultimate, Avid MTRX, Mac Pro 3.5 Ghz 6-Core (2014) 128 GB RAM, Mac OS Big Sur, Samsung SSDs
Posted on Fri, Mar 04 2005 20:31
by stupid8track
Joined on Thu, Mar 13 2003, Posts 230
gary

i agree that they are a great resource - far more knowledgeable than i regarding drives, computers, etc. and they do first rate work, imo.

i am very very curious to hear how your set-up performs. it would be great to see a set up which can finally make use of all the resources available to us - without limitations.

congrats, good luck, and enjoy
Posted on Sat, Mar 05 2005 05:31
by evan evans
Joined on Tue, Jun 17 2003, Hollywood, CA, Posts 2058
Gary,

I have a RAID5 setup. I can tell you that when you are running a system of, as I am, 6 drives, that a drive fails about once a month. If you do RAID0 your entire RAID Volume will be wiped out the moment that happens (and it will). If you do RAID1 you will need twice as many drives as hard disk space needed. RAID5 from NetCell is what I am using and I am very happy. I have rebuilt the Volume already 3 times, avoiding any lost data.

Evan Evans
Evan Evans
Film Scoring Academy
http://filmscoring.academy
Posted on Sat, Mar 05 2005 06:58
by garylionelli
Joined on Wed, Jul 09 2003, Los Angeles, Posts 387
Evan,

Thanks a lot for that info. Right, RAID 0 is scary. Just checked out the Netcell site -- their stuff looks great. I take it that you have SATA drives? And you probably have all manner of libraries installed, VSL, etc., and are getting great performance.
Also (I'm a RAID novice) when rebuilding a RAID volume -- that means reconfiguring the RAID setup and doesn't entail re-installing any libraries, is that right? Thanks again.

Gary
Pro Tools Ultimate, Avid MTRX, Mac Pro 3.5 Ghz 6-Core (2014) 128 GB RAM, Mac OS Big Sur, Samsung SSDs
Posted on Sat, Mar 05 2005 23:41
by evan evans
Joined on Tue, Jun 17 2003, Hollywood, CA, Posts 2058
If one drive fails you need to get right on top of it, and click REBUILD, before a 2nd drive fails and you have total data loss. You may still use your computer and the Volume while it wipes the bad drive and fills it with the missing data over the next 24 hours (although I try not to).

I put mine in a RAID Scorpio case with a ATA Power Tester Adapter, and also bough tsom elong SHIELDED Sata cables and a PCI 8 Cable Clamper Port Bracket.

Evan Evans
Evan Evans
Film Scoring Academy
http://filmscoring.academy
Posted on Mon, Mar 07 2005 01:28
by nick@madhatter
Joined on Fri, Jul 30 2004, Los Angeles, Posts 36
Dear 8track (and all others),
I'm similarly looking to improve performance with a better drive solution. I hear that you've done all manner of testing along this line and I would value greatly any further input you can give on how one might go about getting the best hard drive performance in working with EXS.

I've presently got my load split between FW drives and SCSI drives, and it doesn't cut it. With the streaming capabillities of EXS, one should (in theory) be able to operate almost entirely on sample streaming, rather than glutting the RAM with samples (which is a problem when using many enormous VSL patches at once...or trying to), but I've never yet been able to optimize my streaming capabilities to do this.

I've heard from various sources that a RAID setup will not improve sample-streaming, but would you clarifiy if a RAID setup will DECREASE streaming performance? If so, is there anyone out there using a JABOD drive setup? I've been looking into StorCase products to see if something they make might improve matters. If you have any info, please let me know.

My speculation is that, although far pricier than FW drives, a (StorCase) SATA drive case with Fibre Channel connection could theoretically be the answer to many prayers. Even better, and pricier, a case with FibreChannel drives and connection would really be blazing fast. Using Logic and working with really large orchestral VSL sessions, I don't know how else to do this. I'm thinking this sort of solution would be the best alternative to a multiple-computer, Giga setup. Input anyone?

Best,
Nick

P.S. Evans, if you're crashing that many drives, you might reinspect your modus operandi...what kind of drives are you crashing (make/model)?
Posted on Mon, Mar 07 2005 17:40
by Nick Batzdorf
Joined on Tue, Apr 29 2003, Los Angeles, Posts 2546
What I don't quite understand is that Tascam speaks fondly about RAID for Giga, in fact the reports of 500+ voices are from machines with RAIDs.

So is it the Mac implementation that's not a help or simply that RAID isn't a help?

And if it's the latter, why does Evan Evans, master of running 50 billion voices and loading 38 billion programs, use it?
Mac Pro 5,1 12-core 3.46 GHz, 64MB RAM, latest macOS available. Metric Halo 2882 interface.

VisionDAW Windows 7 Pro i7 950 3.07 4-core, 24GB RAM. Has an RME Hammerfall HDSP9632, but I just use VE Pro. Also several ancient P4 XP slaves, rarely used.
Posted on Mon, Mar 07 2005 20:15
by evan evans
Joined on Tue, Jun 17 2003, Hollywood, CA, Posts 2058
Nick Batzdorf wrote:
And if it's the latter, why does Evan Evans, master of running 50 billion voices and loading 38 billion programs, use it?
LOL! This just made my day!

Wink
Evan Evans
Film Scoring Academy
http://filmscoring.academy
Posted on Mon, Mar 07 2005 20:17
by evan evans
Joined on Tue, Jun 17 2003, Hollywood, CA, Posts 2058
wrote:
P.S. Evans, if you're crashing that many drives, you might reinspect your modus operandi...what kind of drives are you crashing (make/model)?
It can happen for various reasons. I got 6 drives running round the clock. And my reporting software is very good about finding errors on the drive, NOT like the Apple drivers for internal drives, etc. So it NEEDS to rebuild about once a month, to protect from more critical circumstances.

Evan Evans
Evan Evans
Film Scoring Academy
http://filmscoring.academy
Posted on Tue, Mar 08 2005 19:17
by Nigel Watson
Joined on Fri, Dec 20 2002, Cologne, Germany, Posts 365
So far I'm not aware of any external S-ATA enclosures - does anybody know of them?

I have 2 spare 250GB S-ATA drives which I would love to hook up.

Nigel
Posted on Wed, Mar 09 2005 01:07
by Glenn Morrissette
Joined on Tue, Feb 11 2003, Burbank, CA, Posts 80
Nigel,

I just intalled a pair of external 250GB SATA drives using 2 of these enclosures:

http://dealsonic.com/kisssashseat.html

Pretty basic, but so far so good.
Posted on Wed, Mar 09 2005 10:54
by evan evans
Joined on Tue, Jun 17 2003, Hollywood, CA, Posts 2058
I use a RAIDMAX Scorpio. Holds like 12 Drives or some shit.

Evan Evans
Evan Evans
Film Scoring Academy
http://filmscoring.academy
Posted on Wed, Mar 09 2005 18:53
by stupid8track
Joined on Thu, Mar 13 2003, Posts 230
wrote:
Dear 8track (and all others),
I've presently got my load split between FW drives and SCSI drives, and it doesn't cut it. With the streaming capabillities of EXS, one should (in theory) be able to operate almost entirely on sample streaming...

I've heard from various sources that a RAID setup will not improve sample-streaming, but would you clarifiy if a RAID setup will DECREASE streaming performance?

My speculation is that, although far pricier than FW drives, a (StorCase) SATA drive case with Fibre Channel connection could theoretically be the answer to many prayers.

Best,
Nick


Nick

sorry it took me so long to reply - i was away in the real world.

i agree with you that we should be able to operate entirely via streaming with Logic/EXS. the problem with your current set-up just reflects the problem with LOGIC at the moment - Logic can't use multiple drives to any positive effect, either for sample streaming or for audio playback.

yes, you can stream from multiple drives with EXS/Logic - but you do not increase the point at which data is dropped by doing so. same with audio playback with multiple drives in logic - you can playback from multiple drives, but you don't increase the number of simultaneous tracks you can playback.

in my experience, RAID performs similarly to a single drive from the RAID (8 x 15K scsi drive RAID performs the same as a single 15K scsi drive) - for both streaming and track count. i certainly haven't tried every RAID out there, but i've tried both hardware and software RAIDs and NONE of them have improved performance with Logic. they don't perform worse than a single drive, but not better either.

this is why i am so curious about Gary's coming experience with DP and Kontakt. because the 'single drive performance' limitation in Logic is due to Logic (PT isn't limited to 1 drive for audio playback - adding a second drive there increases performance significantly). it might very well be that DP with Kontakt can actually use multiple drives for streaming.

personally, i can't wait for the day that Logic can use multiple drives successfully. hopefully, the coming update with PDC will involve some sort of audio engine rewrite which allows for this. because, at the moment, RAID only really gets you MUCH faster load times. at least in my experience.

so, IMO, you shouldn't invest in anything else yet (for logic/exs) because you really won't see any improvement for your investment. wait to see how logic looks after the PDC update. and wait to see how Gary does with Kontakt and DP and multiple drives.

or give giga a look. it seems that it doesn't have trouble with multiple drives - but it's been years since i've used giga (back when it was still gigasampler).

hope this helps.
Posted on Sat, Mar 12 2005 00:20
by PolarBear
Joined on Sun, Jul 20 2003, Germany, Posts 1206
So actually you're saying Logic's audio engine can't handle more than a certain amount of streaming data? Because a RAID setup is seen still as one drive by the system... Interesting...

PolarBear
A zero can decuple an existing problem.
Posted on Sat, Mar 12 2005 01:40
by stupid8track
Joined on Thu, Mar 13 2003, Posts 230
i don't actually know what is going on with logic and RAID - but in my experience, RAID (hardware or software) has never performed better than an equivalent single drive from that RAID.

and i don't think it has anything to do with any 'limit' on streaming data as a hardware ATA RAID had worse performance than a 15K scsi drive (or even a 10K scsi drive, for that matter).

wish i knew why this was, but i don't. just reporting my experience.

cheers
Posted on Mon, Mar 14 2005 06:58
by garylionelli
Joined on Wed, Jul 09 2003, Los Angeles, Posts 387
S8T,
Just encountered a roadblock that preempts the RAID project. I tried loading 16 VSL Kontakt Perf. instruments into a single instance of Kontakt (standalone or as a DP plug-in) and it's a no-go. At about the time I tried to load the 15th instrument, I got this message:

Kontakt ERROR: could not load sample (no big enough memory block available) Continue loading?

I've got 4 GB RAM in my G5. I can load a many more Perf. instrumenrts into my other G5 Logic setup (also has 4 GB RAM). So, I'm disappointed -- looks like I can't even load in anywhere near the number of Perf. Instruments into Kontakt as can in EXS24, not to mention trying out a RAID experiment.

Gary[/i]
Pro Tools Ultimate, Avid MTRX, Mac Pro 3.5 Ghz 6-Core (2014) 128 GB RAM, Mac OS Big Sur, Samsung SSDs
Posted on Mon, Mar 14 2005 07:24
by stupid8track
Joined on Thu, Mar 13 2003, Posts 230
gary

well, that is too bad.

strange time we live in. the power we have at our fingertips is amazing.

but at the same time, we are so close to having so much more.

ah well - twas always thus, i guess.

maybe the coming PDC update for logic will include an audio engine rewrite which will allow EXS to use multiple disks to benefit.

i ain't holding my breath, though.

sorry to hear that kontakt is throwing up roadblocks. please post back with any headway.

cheers
Posted on Mon, Mar 14 2005 19:31
by nick@madhatter
Joined on Fri, Jul 30 2004, Los Angeles, Posts 36
Dear 8-Track,

Thanks for all your input. I don't quite understand your statement about Logic's Drive limitations, however. Can you expand on that for me? As of yet, I've been using Logic with many drives, so I'm not sure what you mean exactly.

Also, what is the "PDC Update"? I haven't heard about this. I'm interested to know.

Best,
Nick
2 Pages12>
You cannot post new threads in this forum.
You cannot reply to threads in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.