plowman

good to be around again.  especially as logic/EXS has finally improved in a real world way for RAM hogs.  :)

your experience with AM is mirroring mine -- logic only reports around 2 or so GB of RAM with EXS streaming on.  the RAM that the EXS samples are using doesn't appear in any process on my system, only in the "pie chart" at the bottom of AM.  i don't know why that is, but that's why you're seeing 5.7 GB used in your system even though only 1.94 GB is being used by logic.

i've documented some testing of the difference between EXS in L7 and L8 with some screengrabs and posted it in this thread over on the apple forums:

http://discussions.apple.com/thread.jspa?messageID=5382308#5382308

also, i'm not sure why you're getting "running out of memory" error when switching from "fast" to "slow".  i did all my testing with "slow" and "extensive" selected and never got "running out of memory" -- i could keep loading EXS even after i was out of free RAM and into swap space.  of course, i wouldn't expect that to be very usable.

anyway, i wish i could explain what, why, and how this is all happening, but i really have no idea.  i'm just excited about it.

and i do hope you're right about leopard making this moot.  it's my dream to one day be able to use all my system RAM in logic for any plugs.  and not have to have VM on in EXS to exceed 3.5 GB.

still, this is a very nice first step.

cheers

 EDIT - sorry but i don't know how to insert the link to the apple forum in a way that you can just click on it, so you'll have to cut and paste it.  but i did start a new thread over there for this post as that other thread was very long and the way apple threads their posts, i really had no idea where to put the info.

plowman

i'm normally stupid8track but this forum update has reset my forum ID for now, so i have to post as this maxregistrations fellow until i can get it sorted.

i just read your post here and saw to my horror that it had a typo that i'm sure only added to the confusion over there -- the paranthetical in the first part of that is supposed to say "without VM active in EXS". big whoops on my part -- will have to go correct it over there but unfortunately apple doesn't allow for edits after 15 minutes.

anyway, it does seem on my system that the RAM ceiling is now lifted with EXS in Logic 8 but only, as you all have noted here, if Virtual Memory is active in EXS options. it remains to be seen how well these "extra" EXS instances will stream. hopefully they'll play just like normal. i have no reason to believe they won't, but i haven't used them "for real" yet.

and yes, activity monitor is not showing me a process that the EXS RAM is using, but it is showing that the RAM is used.

of course, with VM off, the same old 3.5 GB limit exists for Logic.

now if i can just get my old forum ID back here, all will be fine.

cheers

ps -- the preview makes it appear that this post will not have any breaks for paragraphs. which is not how it was composed. i'm sure everyone is aware, but just wanted to mention it.

Musicmaster

i saw that post too and i'm curious about it because i'm finding the exact same RAM limits in Logic 8 that I had in Logic 7 -- 3.5 GB of Virtual Memory (as reported by Activity Monitor).

i've got a dual 2 G5 with 8GB of RAM/10.4.10 and Logic 8 locks up on me as soon as i load up 3.55 GB of VM (which equals about 3.2 GB of samples).

i'm curious to see:

1. if the user on the apple forums was mistaken and was actually streaming the samples using EXS Virtual Memory

2. or if the user on the apple forums really was loading 13 GB of samples, in which case the question becomes -- why can he load over 3.5 GB and I can't?

perhaps that user is using L8 with the leopard beta? i don't know.

all i can say is that my experience with Logic 8 is that i can't load more samples into EXS than i could in logic 7.

hopefully that will change with leopard. but as of today, the RAM limit remains for me.

cheers

nick

i don't know if there is any way to 'close the gap' between VM figure and RM figure upon launch. i can't remember one off the top of my head, but i do remember in testing a long time ago (i believe it was the evan evan's workaround thread - or at least around that time) that i did have 'less' of a gap between VM and RM. but i can't remember if that was something i did or just a function of logic 6.x being less hungry.

also - do you have a program called menumeters? versiontracker has it and it can show you if your system is hitting 'swap' space by creating swap files - perhaps that is part of why you were experiencing such slowness when you got close to 3.5 VM limit?

i wish i could tell you something that is more substantitive, but i'm really not sure how to control memory assignment with logic.

if i get a chance, i'll try to figure out what i might have done to 'reduce' the initial VM memory demand.

the worst part about playing it close to the edge (around 3 GB RM used) is that it is difficult to 'calculate' how much memory a given VSL instrument will use - and that leads to most of my crashes (i think i've got enough room left to load an instrument and i watch Activity Monitor in fear as the VM number goes higher and higher and higher and then... bam.)

thus i agree that a 'safe' figure is somewhere south of 3 GB. or, if you hit 3GB - don't try to load any more instruments - and keep an eye on plugins. some (altiverb sticks in my mind as one, but don't hold me to it) claim 'new' VM space for logic whenever they are inserted. again - sadly i don't have the time to look into it tonight.

i really just wanted to say that, in my experience, 3.5 GB of Virtual Memory space is the 'ceiling' for logic.

hope this helps.

IME, the only real, consistent memory limit for logic at the moment is 3.5 GB of Virtual Memory space - once you hit that, logic crashes. unfortunately, there is no way to 'predict' what that will mean in terms of Real Memory. you can make an 'educated guess', though.

when you first open logic, take a look at the difference between real memory and virtual memory for Logic as reported in Activity Monitor and you'll see your max real memory figure (3.5 minus the difference between VM and RM figures as reported in AM).

keep in mind that more Virtual Memory is requested by some plugs, so you always want to leave some headroom. if you're into playing it close, keep AM open and keep an eye on logic's VM allocation.

depending on routings/plugs - you should be able to get around 3 GB. maybe not, but 2.5 seems a little low to me. again - it depends on the requirements of the song and the instruments you want to load.

hope this helps

Nick

personally, i think RAID is fool's gold for audio - it isn't a technology designed for such small files. and i think you have demonstrated that again - that the limited improvement you got is not worth the cost.

i think that the big key to improving life with Logic/EXS/VSL will be if/when apple finally releases a version of logic which allows the program to truly use multiple hard drives.

if/when that happens - then the ballgame changes.

hopefully, that will happen with the coming update with delay compensation for busses. i have no idea, but it is my hope that this will involve an audio engine re-write and that one of the benefits of that re-write will be true support for multiple audio/sample drives.

but i've been hoping for this for years and still waiting.

thanks for posting the results of your testing. they are interesting and certainly confirm my experience with RAIDs for logic.

PolarBear

i have created/tested a RAM disk for high sample rate audio track playback, but not for sample streaming (the results of audio track playback fairly well mirror those of sample streaming, in my experience).

anyway - with 1 15K scsi drive i can playback around 30 stereo 96K tracks simultaneously.

with 8 x 15K scsi drives, i can playback around 30 stereo 96K tracks - no improvement

with a RAM disk, i could playback 60+ stereo 192K tracks simultaneously (and this wasn't the ceiling - i just stopped because i'm never going to actually use a RAM disk for recording)

anyway - i know this isn't a 'streaming' test, per se, but it does show the current nature of using multiple disks with logic.

my best guess is that it has something to do with the audio engine. because using protools, multiple disks does help - you can actually see a definite improvement in track count. with logic - there is no improvment (even though you can playback/stream from multiple disks - strange)

that's why i'm holding out a little hope that the PDC update will involve some kind of audio engine rewrite which will enable the functional use of multiple disks with Logic.

we shall see.

the PDC update is the update which will add plug-in delay compensation for plug ins on busses in Logic. no mention of when this will be released - just that it is coming.

as for Logic's drive limitations - logic CAN use multiple drives for audio playback and sample streaming. but doing so does NOT improve performance over a single drive.

in the most extreme example - 8 x 15K scsi drives (4 drives on each 'half' of an atto UL4d card in the 133 MHz slot of a G5) perform at the SAME level as 1 15K scsi drive. both for audio track counts and for sample streaming (i measure sample streaming by the point at which figures appear in the 'could not get data in time' window in the Virtual Memory window of the EXS)

for example - if i can stream 100 voices from 1 15K scsi drive (before data gets dropped), i can only stream 100 voices from 8 15K scsi drives (NOT 800 total. 100 total). if i can playback 30 stereo 96K tracks from 1 15K scsi drive, i can playback 30 stereo 96K tracks from 8 scsi drives (again - 30 total. not per drive)

i know it seems to be beyond any sense that this is so - but i've seen it in every test i've done. there is NO combination of drives which improves performance (adding a firewire 400 drive to a 15K scsi drive for example).

bizarre, but true. and it has been this way since 6.x through 7.01.

hopefully soon it will improve

gary

well, that is too bad.

strange time we live in. the power we have at our fingertips is amazing.

but at the same time, we are so close to having so much more.

ah well - twas always thus, i guess.

maybe the coming PDC update for logic will include an audio engine rewrite which will allow EXS to use multiple disks to benefit.

i ain't holding my breath, though.

sorry to hear that kontakt is throwing up roadblocks. please post back with any headway.

cheers

i don't actually know what is going on with logic and RAID - but in my experience, RAID (hardware or software) has never performed better than an equivalent single drive from that RAID.

and i don't think it has anything to do with any 'limit' on streaming data as a hardware ATA RAID had worse performance than a 15K scsi drive (or even a 10K scsi drive, for that matter).

wish i knew why this was, but i don't. just reporting my experience.

cheers

Nick

sorry it took me so long to reply - i was away in the real world.

i agree with you that we should be able to operate entirely via streaming with Logic/EXS. the problem with your current set-up just reflects the problem with LOGIC at the moment - Logic can't use multiple drives to any positive effect, either for sample streaming or for audio playback.

yes, you can stream from multiple drives with EXS/Logic - but you do not increase the point at which data is dropped by doing so. same with audio playback with multiple drives in logic - you can playback from multiple drives, but you don't increase the number of simultaneous tracks you can playback.

in my experience, RAID performs similarly to a single drive from the RAID (8 x 15K scsi drive RAID performs the same as a single 15K scsi drive) - for both streaming and track count. i certainly haven't tried every RAID out there, but i've tried both hardware and software RAIDs and NONE of them have improved performance with Logic. they don't perform worse than a single drive, but not better either.

this is why i am so curious about Gary's coming experience with DP and Kontakt. because the 'single drive performance' limitation in Logic is due to Logic (PT isn't limited to 1 drive for audio playback - adding a second drive there increases performance significantly). it might very well be that DP with Kontakt can actually use multiple drives for streaming.

personally, i can't wait for the day that Logic can use multiple drives successfully. hopefully, the coming update with PDC will involve some sort of audio engine rewrite which allows for this. because, at the moment, RAID only really gets you MUCH faster load times. at least in my experience.

so, IMO, you shouldn't invest in anything else yet (for logic/exs) because you really won't see any improvement for your investment. wait to see how logic looks after the PDC update. and wait to see how Gary does with Kontakt and DP and multiple drives.

or give giga a look. it seems that it doesn't have trouble with multiple drives - but it's been years since i've used giga (back when it was still gigasampler).

hope this helps.

gary

i agree that they are a great resource - far more knowledgeable than i regarding drives, computers, etc. and they do first rate work, imo.

i am very very curious to hear how your set-up performs. it would be great to see a set up which can finally make use of all the resources available to us - without limitations.

congrats, good luck, and enjoy

gary

i can't speak for RAID with DP/Kontakt/DAE - but i personally feel that RAID isn't designed/intended as a solution for sample streaming. RAID is all about throughput and sample streaming is basically all about seek time.

so what you have right now - multiple drives - is the best solution for sample streaming (though, with logic, multiple drives don't improve performance - i wouldn't be surprised if kontakt/DP/DAE actually benefited from multiple drives). your current problem is merely a 'bus' limitation.

are you thinking about the apple RAID so that you can recycle your current ATA drives that are inside the firewire cases? or are you buying it straight up, with new drives and all? can the apple RAID not be used as a RAID - but as multiple drives? if so, that could be a great solution.

but if not, perhaps you should consider buying multiple SATA drives, an external SATA drive enclosure, and an SATA PCI card, like this http://macgurus.com/prod...pages/sata/tsata-x44.php"="" target="_blank" title="http://macgurus.com/productpages/sata/tsata-x44.php">http://macgurus.com/productpages/sata/tsata-x44.php">http://macgurus.com/productpages/sata/tsata-x44.php. 300+ MB/sec bandwidth, up to 8 external ports, one PCI slot.

that would give you the bus bandwidth you need and preserve independent seek times of all the drives (all drives wouldn't have to be involved with delivering all samples).

i've found the macgurus (the company linked to above) to be very knowledgeable and helpful - and, if you want, they will build a system to suit.

unfortunately, i'm not aware of any (non firewire) multi-drive ATA enclosures.

if the Apple Raid can be run as independent, unraided drives - then that is probably the simplest solution (you can then see if the RAID improves/degrades/doesn't matter for performance vs. multiple drives) and keeps the troubleshooting to a minimum (call apple).

but if it can't, consider emailing the macgurus. also check out xlr8yourmac.com for reports on any SATA PCI cards.

if you decide to go RAID, you could also consider contacting the people at hugesystems.com and asking if you can buy one of their RAID enclosures. they have at least one which uses ATA drives. so you could recycle your drives and just add a scsi card to the G5. but they are hardware RAID, so you'd have to use the drives striped.

anyway, hope this helps. whatever you choose, please report back with real world experience.

the DP/kontakt option is a compelling one.

cheers

Musicmaster

your 'solution' is a variation on an old workaround that evan evans (a fellow VSL user here) came up with last spring, only his 'solution' involved opening multiple Logic programs (something which is no longer possible with Logic 7 - i can only open 1 instance of logic 7 at a time, though i can still open multiple instances of 6.x with 7) rather than other VST hosts.

give a look through this forum for his 'groundbreaking workaround' thread. it is fascinating and informative.

last i recall, there is at least 1 user here running multiple logics regularly - but maybe that has changed. i was tempted by the power for a while, but found the workaround to be more cost than benefit for me. but my set-up and needs are different from yours and others.

so give evan's workaround a look, if you want to be able to 'get around' the 3.5 GB VM Logic limit and still use the performance tool.

as with anything, there are pros and cons - and only you can decide what works best for you. but it is a very interesting option. if it is something which appeals to you, be prepared to move up to 8 GB of RAM.

cheers

JWL

congratulations on a great system - you are going to love it.

and using two drives certainly won't hurt performance. if you've already got em, use em.

congrats again

JWL

sorry for the delay -- i was away for a couple days. to the questions:

1. i haven't noticed any performance differences with parallel vs. serial - but i haven't tested it a great deal either, and not in a good amount of time. but, again, you'll start dropping data before you'll flood the bus, so i wouldn't worry about it too much.

2. not exactly sure what you are asking by "timed the performance of one drive as opposed to two or more". if you are asking if i have actually compared voice count performance of one vs. multiple drives, then the answer is yes, many times. regarding load times of EXS - i don't recall noticing any 'improvement' in speed with multiple drives, but confess i wasn't testing for it, either. i will say that Redmatica's EXS Manager software is a godsend for speed EXS loading (redmatica.com)

3. regarding logic 7 - there have been reports of improvements in EXS load times, but since i already use EXS Manager, i haven't noticed. A G5 allows you to use more RAM with Logic. with a G5, you can use 3+ GB of RAM for Logic, vs. around 1.7 GB with a G4. currently OS X and programs are limited to 4 GB of RAM space (in total). but the G4 is limited to 2 GB of RAM. so you will be able to load almost double the sample heads into RAM with a G5. beyond that, there isn't much VSL improvement with a G5, since VSL/EXS doesn't use a lot of CPU (plugins do - you can use many more effects on a G5)

4. i'm really the wrong guy to ask regarding kontakt/halion. i've got kontakt but haven't thought about using it with VSL, just because i use EXS. i believe they have a kontakt forum here - probably best to ask around there. it could be that there are tremendous improvements, or nothing but headaches - i just don't know.

hope this helps

JWL

sadly, in my experience, the failure of multiple drives to improve the streaming voice limit remains.

i tried recently some testing again with logic 7 with multiple firewire, firewire + SATA, and multiple scsi drives - in every case, multiple drives performed at the same level as a single drive (meaning data got dropped at the same voice count).

i understand your dealer's misunderstanding - because it makes no sense that multiple drives wouldn't improve performance. apple certainly doesn't announce that multiple drives don't help. it is further confused because you can playback/stream from multiple drives - so you would think it would be improving performance.

but, in my experience, that is still not the case - dual 2 G5/OS 10.3.5 - with either streaming or basic audio track playback. IMO, it is something in how logic is coded. protools can make use of multiple drives (even LE), logic can't (unless running DAE/TDM - then i believe it can - but don't quote me on it as i haven't had a PT/TDM rig in years, so i can't confirm). but i believe PT and logic 'see' audio in different ways. i can't explain it any more than that, but that is my understanding.

anyway, i do believe your G4 is limited internally to ATA drives which are 128 GB or less due to something with the ATA/system interface (you can put a larger drive in, but it will only be recognized as 12. i don't believe that your firewire drive will be limited in any way regarding size because i don't believe there is any 'limit' in the firewire port itself. i could be wrong because i haven't been on an older G4 in a while, but that is my guess. sorry i can't give you a guarantee.

i've got some old firewire 400 cases which don't allow for more than 128 GB to be recognized - but that is a function of the hardware in the firewire case, not the mac.

anyway, it is my opinion that multiple drives won't help you with voice count and that you should be fine with a 200 GB firewire drive. but please make sure you can return the drive in the event i'm wrong.

hope this helps and enjoy the new sounds.

cheers

zgogor

actually, i don't think you'll see any improvement in voice count (before dropped data) using a HUGE RAID vs. using a firewire 400 drive because they use the same drive - a 7200 rpm ATA drive.

yes, the HUGE RAID will have 10 of them. but i've never found multiple drives (either in RAID or separate) to improve voice count.

i've specifically tested our HUGE RAID for voice count and it performed the same as a single one of our firewire 400 drives (at least the ones with the Oxford 911 chipset). both dropped data at right about the same time, even though the single firewire drive benchmarks at around 32 MB/sec transfer rate and the HUGE RAID at over 200 MB/sec.

if you want a HUGE RAID for storage and backup and redundancy - fine. if you want it for video work - great.

but if you are buying it hoping for improved VSL voice count performance - be careful. make sure you can return it if performance doesn't improve - because i have a strong feeling it might not.

it is a lot of money to spend to end up with the same performance as a firewire drive.

hope this helps

zgogor

ah, yes, i get it. i too have lost many more firewire drives than any other kind (4 to date). knock on wood, i've never lost a scsi drive to date (owned 30+). for data security, i find firewire to be really poor. i too had to go through the whole 'VSL reinstall' when 2 firewire drives died. so now i've got the full VSL installed on 3 seperate firewire drives - 1 attached to my system, 2 as backup.

if by HUGE system, you mean a RAID from hugesystems.com - i own one of their older RAIDs. AFAIK, they are built with ATA drives (maybe SATA - haven't checked in a while) not scsi. and mine (dual media vault 1800) only streams at the same level as a single ATA drive.

personally, i think you'd probably be better served in cost/benefit by going with 3 firewire drives (1 for use, 2 for backup). the odds of all 3 drives failing at once is remote - but they will be much cheaper and perform virtually the same, IME.

that said, if you get a dual HUGE system, just leave the two halves of it as 'separate' drives and you should be good for security (especially if you get one with drives you can swap out - no good if you have to send them the unit if a drive fails because then your data is 'lost' for that period of time, too). no point RAIDing the two sides together because it won't improve performance.

also consider a company called RAID Incorporated (raidinc.com) - i've had very good luck with them with scsi raids. i know they also make a new SATA raid, though i have no idea what the price point is.

or you could get the MacGurus (macgurus.com) to build you a 2-4 drive scsi mini-tower and you could install the data across all four drives (or RAID them) and get the best performance and stability available for EXS. i'd still recommend firewire as backup, understanding that the 'backup' will most likely fail first.

hope that helps and best of luck with it

zgogor

i've personally never found RAIDs to be of any use with Logic - either for audio track playback or for streaming samples. i certainly haven't tested a hardware firewire RAID, but i have tested other RAIDs (software scsi, hardware ATA, software firewire) and the results have always been the same - the RAID always performed at almost the same level as any single drive which makes up the RAID.

thus, i would expect that the firewire RAID you are considering will behave similarly to a single firewire drive.

i could be wrong because i haven't tested that unit, but if you decide to try the firewire RAID, make sure you can return it if it doesn't improve performance.

hope that helps and good luck

Glenn and Laurent

yes, i tested the internal SATA on a G5 - both 7200 rpm and 10000 rpm - and from what i found, there wasn't a great difference in real world performance between SATA 7200 and firewire (ATA) 7200.

let me put it like this - in my test of simultaneous stereo EXS voices which could be played before data was dropped, i got the following numbers for various drives:

firewire 400 (non Oxford 911/922 chipset) - 35 voices
firewire 400 (Oxford 911 chipset) - 58 voices
firewire 800 (Oxford 922 chipset) - 58 voices
SATA - 64 voices
10K SATA - 82 voices
15K scsi - 92 voices

the relative power of the drives roughly mirrors the numbers i got for the number of 96K stereo files i could playback simultaneously in a real world test (with plugs) for these drives (i didn't test the non Oxford fw400 drive for this because i didn't yet own it):

firewire 400 (Oxford 911) - 14
firewire 800 (Oxford 922) - 14
SATA - 16
10K SATA - 25
15K scsi - 28

anyway, i personally don't think the 'increase' in performance of 7200 SATA over firewire justifies the cost. if someone else does, more power to them.

i hope this info is of some help to Glenn and anyone else considering a drive purchase for VSL.

Fred:

these forums have given me so much that it is nice to be able to give back where i can. i'm glad the post was appreciated.

best to all

edit: just to be clear - i only tested SATA internally on the G5. i am not aware of any firewire cases for SATA drives. but, even so, they didn't perform much betterh than fw400 - at least at 7200 rpm

whatever he has up his sleeve, i can't wait. he's a blessing.

certainly some kind of editor improvement is needed.

all i can say is i really really wish autosampler had been around back when i had a studio full of synths. sadly, i went all software a year and a half too early - god knows i miss a lot of the patches i had.

cheers to andrea

Glenn

in my experience, you won't see much difference between 7200 rpm SATA and 7200 rpm ATA drive in a firewire case - just make sure you buy a firewire drive/case with the oxford 9xx chipset in it - that can make a sizeable difference in performance.

10K SATA is a step up from either, but is currently limited to 76 GB.

also, fw800 is really irrelevant for you as it is merely an interface - it is the drive inside which matters which, for now, will be the same in either fw400 or fw800. and seeing as a 7200 rpm drive isn't going to come close to flooding a fw400 bus anyway, no point in fw800. the interface doesn't make the drive faster - so it doesn't improve end performance.

also, in my experience, multiple drives don't improve performance. you can use them, but you won't be able to run more simultaneous instruments without dropped data. don't know why (it doesn't make sense) but that is how logic behaves with audio track data, too. same thing with RAIDs.

so, IMO, the best thing to do is get a large 7200 rpm firewire drive with oxford 911/922 chipset and get to work. no real point to buying firewire 800 at the moment, especially if it costs any more - because it won't improve anything. though many cases offer both 400 and 800 ports - which is convenient only because it gives you more options for connection. if it doesn't cost anything, it can't hurt to have both.

i've tested firewire drives with the oxford chipset and without - and the ones with the oxford chipset performed at 160+% of the ones without it (performance measured as the point at which data started being dropped).

hope this helps and congrats on the new setup

dot

again - thank you for taking the time to write such a clear, helpful email. i've got a cousin who is a unix guy and will be seeing him soon, so i'll hold off on this until then - but this post will definitely help tremendously.

and i understand completely that you can't make any guarantees as to the outcome/result/stability/function of my system if i choose to do this. no worries. i accept full responsibility for any and all tampering i do to my system (again - the forum name ain't an accident)

that said - i really appreciate your time and patience in explaining this. i've got 2 GB RAM in my powerbook, 8 in my G5 so i'd like to give it a run. i am sufficiently backed up at most times, but will be sure to back up again just before attempting anything.

and, unfortunately, half of what i know about logic wouldn't heat a turnover. but you're welcome to it.

cheers again

DOT

thanks again for another terrific post. really, my forum name is no accident but you've made things clear enough that they even make sense to me. i really appreciate it.

one thing i would love to do is to disable dynamice paging on my system. i've maxed the RAM in both my powerbook and my G5, so i'd love to disable the dynamic paging.

if it isn't too much trouble, would you mind running through that quickly. i don't know how to do this:



do you mind stepping through how to do that?

also, does this have to be done after each reboot, or is it a do it once and you're done type of thing.

again, thanks for taking the time to explain this to an absolute layman. i really do appreciate it.

cheers