HP Smart Array Controllers and basic RAID performance factors

Technologybrief

Abstract..............................................................................................................................................2Introduction.........................................................................................................................................2HPSmartArraycontrollersandperformance...........................................................................................2SmartArrayprocessingengine..........................................................................................................2SmartArraycache...........................................................................................................................3Readcache.................................................................................................................................3WriteCache................................................................................................................................4Cachewidth................................................................................................................................4Batterybackedandflashbackedwritecache..................................................................................4ZeroMemoryRAID.......................................................................................................................4OveralleffectofcacheonSmartArrayperformance........................................................................5SmartArraydevicedriver.................................................................................................................5SASlinks,diskdrivesandarrayperformance.........................................................................................6DiskStripingandPerformance..............................................................................................................6RAIDlevels,drivecountandreadperformance.......................................................................................7Randomreadperformance................................................................................................................7Sequentialreadperformance............................................................................................................8RAIDlevels,drivecountandwriteperformance.......................................................................................9WriteperformanceforRAID0...........................................................................................................9WriteoperationsforRAID1andRAID10(1+0).................................................................................9WriteoperationsforRAID5andRAID6levels..................................................................................10Writecache,theSmartArrayprocessor,andRAIDwriteperformance.................................................11Randomwriteperformance.............................................................................................................12Sequentialwriteperformance..........................................................................................................13AdditionalRAIDperformancecharacteristics.........................................................................................14Queuedepth.................................................................................................................................14Throughputversuslatency...............................................................................................................15Formoreinformation..........................................................................................................................16Calltoaction....................................................................................................................................16

Abstract

RAIDstoragetechnologyhasbeenusedinindustry-standardserversforalmost20years.Overthattime,significantadvancesindiskdrives,storageinterfaces,RAIDcontrollertechnology,and

processingpowerhavecontinuedtochangethestoragelandscape.ThistechnologybriefprovidesanoverviewofthebasicfactorsdrivingRAIDperformancetoday,includingRAIDlevelsthemselvesaswellascontrolleranddrivetechnologies.

Introduction

Diskarraysaredesignedtoaddressseveralbasicissueswithdiskdrive-basedstorage:Allowingthecreationoflargestoragevolumesusingmultiplesmallerdiskdrives.

IncreasingtheI/Ocapabilitiesandmaximumthroughputofthestoragesubsystemoverthatofindividualdiskdrives.

Increasingthereliabilityofdatastoragebyusingredundancytechniquestoensurethatthefailureofoneormorephysicaldrivesdoesnotresultinapermanentlossofdata.

TherearemanyvariablesthatinfluencetheoverallperformanceofRAIDarrays,soitcouldbeinstructivetothinkabouttheprimaryfactorsthatinfluenceRAIDperformance:

RAIDlevels.EachRAIDlevelinfluencesoverallperformancebasedonthenumberoflow-levelread/writeoperationsandtheamountofprocessingoverheadneededtoperformtheassociatedhigh-levelreadsandwrites.

TheRAIDcontroller.ThisincludestheprocessorandmemoryrequiredtomanageandexecutetheRAIDoperationsaswellasthereadandwritecacheusedtooptimizeread/writeperformance.Thenumberofphysicaldrivesinthelogicaldrivearray.HavingmoredrivesinanarrayallowstheSmartArraycontrollertoexecutemorereadandwriteoperationsinparallel,increasingoverallperformance.

Driveperformance,includingdrivethroughputcapability(MB/s)anddriveperformancewhenperformingrandomreadsandwrites(I/O’spersecondorIOPS).

Storageinterfaceperformance,includingtheprotocols(SASvs.SATA)andthespeedofthephysicallinksbetweenthedrivesandthecontroller(3Gb/sor6Gb/s).

EachofthesevariablesnotonlyinfluencesRAIDperformancebutcanalso,dependingonthetypeofstorageoperationsbeingperformed,becomethefactorthatdeterminestheupperlimitofthedrivearray’sperformanceinaparticularapplicationenvironment.

HPSmartArraycontrollersandperformance

ThenewgenerationHPSmartArraycontrollersaredesignedtoimproveRAIDperformance.RAIDperformanceisdependentonmanydifferentfactors.Twoofthemoreimportantcontributorsto

performancearetheSmartArrayprocessorandtheread/writecachethatarepartoftheSmartArraycontroller.

SmartArrayprocessingengine

TheprocessingengineintheSmartArraycontrollerisresponsibleformanagingtheRAIDsystemandforperformingtheoperationsrequiredtotransformthehigh-levelreadorwriterequestsfromanapplicationintothecomplexseriesofindividualinstructionsrequiredtoexecutethisfortheRAID

2

array.ThecurrentgenerationSmartArrayP410,P411andP212controllersuseanembeddedRAID-on-Chip(RoC)processorrunningat600MHz(Figure1).WhileitisnotadirectmeasureofoverallRAIDperformance,thenewprocessoriscapableofsupportingupto60,0004KBrandomIOPScomparedto35,000forthepreviousgenerationengines.

Figure1.HPSmartArraycontrollerarchitecture

TheprocessingengineintheSmartArraycontrollerisresponsibleforprocessingalloperations,butitscapabilitiesareparticularlycriticaltocomplexRAIDoperationssuchaswriteoperationsfor

redundantRAIDmodes.BothRAID5andRAID6usemathematicalXOR(Exclusiveor)operationstocalculatetheparitydatathatiswrittentothedrivearrayinordertoprovidedatarecoverycapabilityinthecaseofaphysicaldrivefailure.Thismakestheprocessingengine’sperformanceakey

contributortoarrayperformance-particularlywriteperformance-fordiskarraysthatusetheseRAIDlevels.PerformanceimprovementsassociatedwiththenewerSmartArraycontrollersarealsomostapparentinarrayswithlargerdrivecounts.Withsmallerdrivecounts,logicaldrivearray

performancetendstobeconstrainedbytheaggregateI/OofthedrivesandnotthebandwidthoftheSmartArrayprocessingengine.

SmartArraycache

SmartArraycontrollersusetheiroptionalcachemodulestoimprovetheoverallperformanceofdiskarraysforbothreadandwriteoperations.Thepercentagesofthecachebeingusedforwritecachingandreadcachingcanbeconfiguredusingthearrayconfigurationutility(ACU).ThecurrentSmartArraycontrollerssupport256MB,512MBand1GBcachesizeoptions.Readcache

OnaSmartArraycontroller,readcacheisusedinapredictivecapacitytopre-fetchdata.The

controller’soperatingprogramidentifiesthepatternofthereadcommandsandreadsaheadonthedrives,placingthisdataintothecachewhereitcanbemorequicklyaccessediftheupcomingreadcommandscallforit.Readcacheisreallyonlyeffectiveinincreasingtheperformanceonsequentialreadworkloads.TheSmartArraycontrollerissophisticatedenoughtodifferentiatebetween

sequentialandrandomworkloads,usingreadcachepre-fetchonlywhensequentialworkloadsaredetected.Additionally,readcachedoesnotgreatlyimprovearrayreadperformancesincetherawperformanceofadrivearrayonreadsisalreadyrelativelyfast.ThesearetheprimaryreasonswhythedefaultconfigurationonSmartArraycontrollersassignsonly25%ofthecacheforreadcache.

3

WriteCache

SmartArraycontrollersusethewritecacheasanoutputbufferthatallowsthehostapplicationstopostwritecommandstothecontrollerandcontinuewithoutwaitingforthewriteoperationto

completetothedisk.Theapplicationseesthewriteascompletedinamatterofmicrosecondsratherthanmilliseconds,andthearraycontrollerwillcompletetheactualwritetodisklaterasitworks

throughthelistofwritecommandsthathavebeenpostedtothecache.Thistechniqueisoftenreferredtoaspostedwritesorwrite-backcaching.

Inhighworkloadenvironments,thewritecachewilltypicallyfillupandremainfullmostofthetime.Thecontrollerthenusesthisopportunitytoanalyzethependingwritecommandsinthecacheanddeterminemoreefficientwaystoexecutethem.Thecontrollercancombinesmallwritestoadjacentlogicalblocksintoasinglelargerwritethatcanbeexecutedmorequickly.Thistechniqueiscalledwritecoalescing.Thecontrollercanalsorearrangetheexecutionorderofthewritesinthecacheinsuchawaythattheoveralldisklatencyisreduced.Thistechniqueisoftenreferredtoascommandreordering.Withlargeramountsofwritecachememory,theSmartArraycontrollercanstoreandanalyzealargernumberofpendingwritecommands,increasingtheopportunitiesforwritecoalescingandcommandreorderingwhiledeliveringbetteroverallperformance.

Writecachingstorespendingwritesinthecacheforlatercompletion.Sincetheapplicationsseethewritesasalreadycompleted,thesecachedwritesmustbekeptbytheSmartArrayuntiltheyarecompletedtodisk.Ifnot,datacorruptionwilloccur.SmartArraycontrollersaddressthisissueby

usingbatteriesorflashmemorytomaintainwritecacheintegrityevenintheeventofaservercrashorpowerfailure.OnthoseSmartArraycontrollerswherebattery-backedorflash-backedcacheisanoption,thedefaultisfornoneofthecachetobeusedaswritecacheiftheoptionisnotinstalled.Thiscanbeoverridden,butdoingsoopensawindowforpossibledataloss.Cachewidth

ThenewgenerationSmartArraycontrollerssupport256MB,512MB,and1GBcachemodules.Inadditiontoprovidingsignificantlymorecacheforreadandwriteoperations,the512MBand1GBmodulesalsousea72-bitwide(bitsdata+8bitsparity)cacheinsteadofthe40-bitwide(32bitsdata+8bitsparity)cacheusedinthe256MBmodules.Thisdoublesthebandwidthformovingcachedatatoandfromthestoragesystem,contributingfurthertooverallincreasesinarrayperformance.

Batterybackedandflashbackedwritecache

Atanypointintime,thewritecacheintheSmartArraycontrollercontainsdatathattheOSandapplicationsconsidertohavebeenwrittentodiskbutwhichis,infact,stillinmemoryonthecontroller.Toavoidpossibledatacorruptionissuesintheeventofapowerloss,allSmartArray

controllersmaintainthewritecache.Batterybackedwritescacheusesanattachedbatterytomaintainthecontentsofcachememoryifpowerislost.Thebatteriesarecapableofmaintainingcachedataforupto72hours.Thenewflashbackedcachemodulesuseonboardpowerfromacapacitortowritethecacheddatatonon-volatileflashmemorywhereitcanremainalmostindefinitely.

ItisimportanttonotethatalthoughtheSmartArraycachemodulecanbeusedwithoutbatteryback-up,theSmartArraycontrollerwillnotuseanyofthecachememoryforwritecachingifthebatteryback-upisnotpresent.Thiswillsignificantlyimpactwriteperformance,particularlyinRAID5andRAID6modesandtheirderivatives.ZeroMemoryRAID

Althoughitisreferredtoasread/writecache,theSmartArraycontrolleractuallyusesfrom32toMBofthememoryinthecachemoduletosupporttheexecutionofadvancedRAIDfunctions,includingtheXORoperationsrequiredtocalculateparityforRAID5andRAID6logicaldrives.SeveraloftheSmartArraycontrollersshipwithoutcachemodulesaspartoftheirstandard

4

configuration.Nothavingthismemoryavailableaffectsmorethanjustperformance.Italsolimitsthefunctionalitythatthecontrollercansupport.

ForthenewgenerationofSmartArraycontrollers,thelimitedmodesofoperationavailablewithouttheuseofacachemoduleareknownasZeroMemoryRAID.ZeroMemoryRAIDprovidesanentry-levelRAIDfunctionalitywithRAID0andRAID1only,andsupportsonlyalimitednumberofphysicaldrivesinanarray.

OveralleffectofcacheonSmartArrayperformance

UsingcacheprovidessignificantstorageperformanceimprovementsforSmartArraycontrollers,especiallyforwrite-intensiveoperations.Whilereadcachemayprovidemodestperformancegainsforreadoperations,writecacheiscrucialtoimprovingthewriteperformanceofdrivearrays.ThisisbecauseadvancedRAIDlevelsmayrequireuptosixindividualreadandwriteoperationstophysicaldrivesinordertocompleteasinglearray-level“write”toalogicaldrive.Figure2showstherelativeperformanceofa10-driveRAID5arrayusingtheP411SmartArraycontrollerwithvaryinglevelsofcache.

Figure2.EffectofSmartArraycacheonarraywriteperformance

HPSmartArrayCacheandWritePerformance

14001200IO'spersecond10008006004002000

256MBBBWC512MBBBWC1GBFBWC

8KRandomWriteKRandomWrite

Configuration:P410andP411SmartArraycontrollers;8-DriveRAID5logicaldrive;256KBstripsize;queuedepth;ProLiantDL380G6

SmartArraydevicedriver

InadditiontotheSmartArraycontrollerandprocessingengine,theSmartArraydevicedriverfortheWindowsServeroperatingsystemimprovesstorageperformancewhenpossible.ThedriveranalyzesthependingdriveI/Oqueueattheoperatingsystemandlogicaldrivelevel,abovetheSmartArraycontrollerlevel.Undertheappropriateconditions,thedrivercoalescesthesependingrequests,reducingthetotalnumberofI/OcommandssenttotheSmartArraycontrollerinordertoincreaseperformance.

5

TheSmartArraydevicedriver’scoalescingcapabilityimprovesperformanceforsmallrequest-sizedsequentialtransactionstreamsinenvironmentsthatcreatelargependingI/Oqueuesattheoperatingsystemlevel.I/OcoalescingisnotperformedwhentheI/Oqueueissmall,sincedoingsomightactuallyloweroverallperformanceoftheSmartArraystoragesystem.

SASlinks,diskdrivesandarrayperformance

ThenewSmartArraycontrollersconnecttothediskdrivesinadrivearrayusinguptoeightprimarySAS-2physicallinks.Eachofthesephysicallinksiscapableofsupportingamaximumbandwidthofupto6Gb/s,or600MB/s,dependingonthetypeofdrivesattached.SAS-2linkswillonlyoperateat6Gb/sif6Gb/sSASdrivesareattachedtothem.SmartArraycontrollerssupportSATAdrivesoperatingatamaximumchannelbandwidthof3Gb/s(300MB/s).

SASbandwidthisneverreallyanoverallperformancelimiterinapplicationenvironmentsthatrelyheavilyonrandomreadandwriteoperations.Thefastestcurrentdiskdrivesarecapableofdeliveringabout470randomIOPSusing4KBreadsandwrites.Thistranslatestoathroughputof1.8MB/s,orlessthan1percentofthebandwidthofSAS-2physicallink.EveninalargerRAIDconfigurationusingaSASexpandertoplace6drivesbehindasingleSASchannel,theaggregatethroughputwouldbelessthan15MB/s,farlessthantheSASbandwidth.DifferentdiskdrivesinfluencetherandomreadandwriteperformanceoflogicaldrivearraysbyvirtueofthenumberofrandomIOPSthattheycansustain,asshowninTable1.

Table1.MaximumsustainedthroughputandrandomIOPScapabilitiesforHPdiskdrives

DriveRPM

FormFactor&Interface

Max.Throughput(KBseq.readatqueuedepth>4)

200MB/s155MB/s150MB/s130MB/s95MB/s

TypicalIOPS(4KBrandomreadatqueuedepthof16)

335375270140128

15,00015,00010,0007,2007,200

LFF6Gb/sSASSFF6Gb/sSASSFF6Gb/sSASLFF3Gb/sSATALFF3Gb/sSATA

Withsequentialoperations,particularlysequentialreads,SASchannelbandwidthcanbecomeafactorinoverallArrayperformance.AsTable1shows,nosinglediskdrivecansustainathroughputthatiscapableofsaturatinga3Gb/sSASchannel.

LargerdrivearrayscanhavemultipledrivessharingthebandwidthofaSASchannel.Whenmorethantwodiskdrivesshareasingle3Gb/sSASchannel,theperformanceforsequentialoperationswillstarttobelimitedbythebandwidthoftheSASchannel.With6Gb/sdrivesattachedto6Gb/sSAS-2channelsonthenewerSmartArraycontrollers,sequentialperformanceshouldcontinuetoscaleuntilmorethanthreedrivesaresharingeachchannel.

DiskStripingandPerformance

MostRAIDlevelsaredesignedtoprovideincreasedreadperformancebydistributing,orstriping,dataacrossthesetofphysicaldrivesthathavebeenconfiguredasasinglelogicaldrive.Withstriping,eachXnumberofbytesofdataofthelogicaldiskisplacedonadifferentphysicaldiskinthearrayonarotatingbasis.Inindustryterms,eachsetofXbytesiscalledastrip.Astripeisonecompleterowofdatastripsacrossallofthedrivesinanarray.HPconfigurationtoolshaveusedthe

6

termstripesizetorefertowhatmostoftheindustryreferstoasthestripsize,althoughthisisbeingchangedin2010.

Thestripsizeforanarrayisconfigurable,andcanbesetfrom16KBupto512KB.Ingeneral,usingalargerstrip(HPstripe)sizedelivershigherperformanceforaRAIDarray.TheArrayConfigurationUtility(ACU)determinesthelargeststripsizethatcanbesetforagivenlogicalarraybasedontheRAIDlevelofthearrayandthenumberofphysicaldrivesthatitcontains.

RAIDlevels,drivecountandreadperformance

Oneofthegoalsforusingdrivearraysistoincreasethereadperformanceofstoragesubsystemsoverthatofsinglephysicaldiskdrives.Ingeneral,thisisaccomplishedbyusingmultiplediskdrivesanddistributingthedataacrossthemusingstriping.Asaresult,thereadoperationsrequiredto

accessdatacanbedistributedacrossthemultipledrivesandexecutedinparallelbytheSmartArraycontroller.Ingeneral,readperformanceforSmartArraydrivearraysistypicallydeterminedmorebytheperformancecharacteristicsofthedrivesthemselvesandisnotboundbythespeedoftheSmartArrayprocessororthecachesize.

Randomreadperformance

Drivearrayreadperformance,particularlyrandomreadperformance,isgreatlyinfluencedbytheuseofdatastripingandbythenumberofdrivespresentinanarray.Datastripingdistributesdataevenlyacrossallthedrivesinanarray,allowingtheSmartArraycontrollertoachieveincreasedperformancesincereadrequestscanbeexecutedinparallelacrossallofthedisks.

RAID0,RAID5,andRAID6usedatastriping,resultinginsimilarreadperformance.Randomreadperformanceistypicallymeasuredinthenumberofsmall(4KBto8KB)randomreadoperationsthatcanbeperformedpersecond(oftenreferredtosimplyasIOPS).FortheseRAIDlevels,randomreadperformancescalesalmostdirectlywithdrivecount,asshowninFigure3.Withallotherfactorsbeingequal,a12-drivearraycandeliverapproximatelyfourtimesthenumberofrandomIOPS-per-secondasanarraywithonly3drives.

7

Figure3.Scalingof8KBrandomreadIOPS.RAID0,RAID5,RAID6andRAID10(1+0)

ScalingofRandomReadPerformance

HPSmartArrayP411

3500IO'spersecond30002500200015001000

4Drives6Drives

8Drives

10Drives

RAID1+0

RAID0

RAID0RAID5RAID6RAID1+0

Configuration:SmartArrayP411with512MBcache.256KBstripsize.Queuedepthof.DL380G6

RAID1+0alsousesstriping,anditsperformancescaleslinearlywiththedrivecount.Becauseitismirroredaswellasstriped,RAID1+0requirestwophysicaldiskstoachievethesamenetincreaseindatastoragecapacityasasingleadditionaldiskdoesforRAIDs0,5,or6.

Sequentialreadperformance

WithDriveArrays,sequentialreadperformancealsoincreasesasthenumberofdrivesinthearrayisincreased.Theupperlimitonsequentialperformanceofanyonedriveisdeterminedbythemaximumthroughputcapabilityofthedrive(Table1).WithSmartArraycontrollers,thesequentialread

performanceofanarrayalsotendstoscaledirectlywiththenumberofdrivesinthearray(Figure4).RAID1+0performancescalesmoreslowlysincethestripeddataisdistributedacrossfewerphysicaldrivesasaresultofmirroring.Withlargerdrivearrays,theultimatelimitersofsequentialread

performanceareeithertheaggregatebandwidthoftheSASlinksthemselvesorthePCIebandwidth,whicheverissmaller.

8

Figure4.SequentialreadperformanceforRAID0,RAID5,RAID6andRAID10(1+0)asfunctionofnumberofdrives.

ScalingofKBSequentialReadPerformance

HPSmartArrayP411

16001400TransferRateMB/s120010008006004002000

4Drives

6Drives

8Drives

10Drives

RAID6RAID0

RAID0RAID5RAID6RAID1+0

Configuration:SmartArraywith512MBcache.256KBstripsize.Queuedepthof

RAIDlevels,drivecountandwriteperformance

Writeoperationsaremuchmorecomplexthanreadsformostdrivearrayconfigurations.Thiscomplexityalsohasasignificanteffectonoverallwriteperformance.Withdrivearrays,allRAIDlevelsotherthanRAID0providesomelevelofdataredundancyandrecovery.ThisredundancyisessentialtotheSmartArraycontroller’sabilitytorebuildalogicaldriveandrecoverthedatawhenoneormorephysicaldrivesinthearrayfails.Thiscapabilitycomesataprice,whichisanincreaseinthenumberlow-levelreads,writes,andcalculationsthattheSmartArraycontrollermustexecutewhenperformingahigh-level“write”toalogicaldriveinanyoftheredundantRAIDlevels.

WriteperformanceforRAID0

RAID0istheonlyRAIDlevelthatdoesnotsupportanydataredundancy.Asaresult,noextralow-levelcommandsarerequiredtoexecutea“write”toalogicaldrive.Becausestripingdistributesthedataacrossthephysicaldrives,thelow-levelreadsandwritescanbeexecutedpartiallyinparallel.ForRAID0,bothsequentialandrandomwriteperformanceshouldscaleasthenumberofphysicaldrivesincreases.RAID0providesausefulbasisforcomparisonwhenevaluatinghigherRAIDlevelperformance.

WriteoperationsforRAID1andRAID10(1+0)

RAID1isthesimplestexampleoftheadditionalwriteoverheadassociatedwithredundantRAIDlevels.InRAID1,dataissimplymirroredacrossasetofdrives(Figure5).Thismeansthatforevery“write”ofablockofdatatoalogicaldrive,theSmartArraycontrollermustexecute2low-levelwrites,onetoeachofthemirroreddrives.Inasimplenon-cachedexample,thiswouldmeanthatintheworst-casescenario,writeperformancecouldbeone-halfthatofwritingtoanon-arrayedphysicaldrive.WithRAID1thereisnostriping.Thisreducesthearraycontroller’sabilitytoexecutereadsandwritesinparallelacrossmultiplephysicaldrives,whichresultsinlowerperformancethanRAID0.

9

WithRAID10(RAID1+0),dataisstillmirrored;however,itisalsostripedacrossthemirroreddrivesetstoevenlydistributethedataacrossthedrivesandprovidebetterreadandwriteperformance.RAID10requiresexecutingtwolow-leveldiskwritesforeachhigh-levelwritetothelogicaldrive.

Figure5.RAID1andRAID1+0drivearrays

WriteoperationsforRAID5andRAID6levels

RAID5providesdataprotectionbycreatinga“paritystrip”thatismathematicallycalculatedbasedonthevaluesstoredinthecorrespondingdatastripsthatcompriseanentiredatastripeacrossthearrayofphysicaldrives.RAID5requirestheequivalentofonephysicaldriveforstoringtheparityinformationforalogicaldrivearray.AsshowninFigure6,thepositionoftheparitystripisactuallyrotatedwitheachstripeinordertobalanceoverallperformance.WithRAID5,anarrayofNnumberofdrivescanstoreN-1drives’worthofdata.Anysingledrivecanfailandthedataitcontainedcanbemathematicallyreconstructedfromtheotherdrives.

Figure6.ConfigurationofaRAID5drivearray

10

WithRAID5,eachhigh-levelwriteoperationtothelogicaldrivetakesseverallowerleveloperationstoaccomplish.AsTable2shows,eachRAID5writetakesfourlowleveldiskoperationsandaparitycalculation.Intheworstcase,RAID5randomwriteperformancecouldbeonlyone-quarterthatofasingleRAID0drive.

Table2.BreakdownofaRAID5high-levelwriteoperation

Low-leveloperation

ReaddatadriveReadparitydriveComputenewparityWritedatadriveWriteparitydrive

RetrievecurrentdataRetrievecurrentparityinfo.

BasedoncurrentdataandparityplusnewdataWritenewdatavaluestothedatadriveWritenewparityvaluestoparitydrive

Purpose

RAID6,alsoknownasAdvancedDataGuarding(ADG),calculatestwoindependentformsofparitycheckdata,creatingtwoparitystripsaspartofeachdatastripeacrossthephysicaldrivesinthearray(Figure7).WithRAID6,anarrayofNdrivescanstoreN–2drivesofdata.Anytwodrivescanfailandthedatainthearraycanstillbemathematicallyreconstructed.

Figure7.ConfigurationofaRAID6drivearray

WithRAID6,thewritepenaltyisevengreaterthanwithRAID5becauseeachhigh-levelwrite

operationtothelogicaldrivepotentiallyrequiresexecutingsixlow-leveldiskread/writeoperationsand2separateparitycalculations.Intheworstcase,randomwriteperformanceforaRAID6logicaldrivewouldbeone-sixththatofanequivalentRAID0logicaldrive.

Writecache,theSmartArrayprocessor,andRAIDwriteperformance.

WritecachingandtheadvancedalgorithmsusedbytheSmartArrayprocessortomanagethewriteprocessareessentialtodeliveringacceptablewriteperformancefordrivearrayswhenusinganyoftheredundantRAIDlevels.ThesignificantwriteperformancepenaltyincurredwithoutwritecachingisoneofthereasonsthattheZeroMemoryversionsoftheSmartArraycontrollersonlysupportRAID0andRAID1.

WritecacheallowstheSmartArraycontrollertostorependingwritecommandsissuedbytheserver’soperatingsystem.TheSmartArrayprocessorthenanalyzesthependingqueueofwritecommandsanddeterminesiftherearemoreefficientwaystoexecutethemtoimproveperformance.Itdoesthis

11

byemployingthewritecoalescingandcommandreorderingtechniquesdiscussedinthesectiononSmartArraywritecache.

TheSmartArraycontrolleralsotakesadvantageofatechniqueknownasfullstripewrites.Ifthecontrollerdeterminesthatafullstripeofdataischanging—possiblyasaresultofwritecoalescing—thenonRAID5andRAID6operationsitnolongerneedstoperformtheadditionalreadoperationstoretrievethecurrentdataandparityinformation.Alloftheinformationrequiredisalreadyinthecontrollercache.Itsimplycalculatesthenewparityvaluesandthenwritesoutthenewstripe,includingtheparitystrip(s).

Usingalargerstripsizeforanarraydecreasesthenumberoffullstripewritesthatthecontrollerwillaccumulate,andthereforemaynegativelyaffectwriteperformancetoacertaindegree.Thisisbecauselargerstripswillnaturallyresultinlargerstripesandthuslowertheprobabilitythatwritecoalescingwillaccumulateafullstripeofdatainthecontrollercache.Largerstripsizesdotendtoimprovereadperformance.

Randomwriteperformance

Figure8comparestherandomwriteperformanceofRAID0,RAID5,RAID6andRAID1+0arrays(configuredasonelogicaldrive)asthenumberofphysicaldrivesisincreased.Aspredicted,thewriteperformanceofRAID5andRAID6arraysissignificantlylowerthanthatofRAID0becauseoftheoverheadinvolvedwitheachhighlevelwriteoperation.Performancedoesscaleasthenumberofdrivesincreases,althoughnotatquitetherateforRAID6asforRAID0.

Forthesamenumberdrives,RAID1+0randomwriteperformanceisaboutonehalfthatofRAID0andabouttwicethatofRAID5orRAID6.ThisisconsistentwiththefactthatRAID1+0requirestwolowleveldiskwritesforeachhighlevelarraywrite,butdoesnotrequireanyextrareadsorparitycalculationsonthepartoftheSmartArraycontroller.

12

Figure8.Scalingof8KBrandomwriteperformanceforRAID0,RAID5,RAID6,andRAID1+0.

Scalingof8KBRandomWritePerformance

HPSmartArrayP411

60005000IO'spersecond4000300020001000

0

4Drives

6Drives

8Drives

10Drives

RAID0

RAID0RAID5RAID6RAID1+0

Configuration:P411controller.512MBcache.256KBstripsize.Queuedepthof

ItisimportanttonotethatwhiletherelativerandomwriteperformanceismuchmoresignificantlyimpactedbyRAIDlevelsthanrandomreadperformance,thewritecachedoeshelpincreaserandomwriteperformanceoverall.ThisisbestexemplifiedbyRAID0,whichhasnowritepenalty.AtendriveRAID0logicaldiskperforms5015randomwritespersecondwhileachievingonly2936randomreadspersecond.Thisdifferenceisprimarilyattributabletothebenefitsofthewritecache.

Sequentialwriteperformance

Figure9comparesthewriteperformanceofthedifferentRAIDlevelswhenexecutingKBsequentialwrites.Comparedtorandomwrites,therearetwonoticeabledifferencesinthe

performancecurves.Withsequentialwrites,thedifferenceinperformancebetweenRAID0andRAID5orRAID6isnotnearlyasgreatasitwasforrandomwrites.Thiscanbeattributedtothewrite

cache,andmoreparticularlytowritecoalescing.SequentialwritesallowtheSmartArraycontrollertocoalescethemintofullstripewrites.ForRAID5andRAID6,thiseliminatestheadditionalread

operationsnormallyrequiredandthereforeincreasestheirperformancerelativetoRAID0.Secondly,sequentialwriteperformancedoesnottendtoscaleasthenumberofphysicaldrivesinthelogicalarrayincreasespastacertainpoint.WithRAID5andRAID6,thisplateauoccurswhenthecontrollerprocessingenginereachesthelimitsofitsabilitytoperformtherequiredXORcomputations.ForRAID0,performanceplateauswhenthemaximumthroughputthatthedrivescanmaintainisreached.InthetestshowninFigure9,increasesintotalthroughputtendtodiminishoncethedrivecountexceedseight.

13

Figure9.ScalingofsequentialwriteperformanceforRAID0,RAID5,RAID6andRAID1+0

ScalingofKBSequentialWritePerformance

HPSmartArrayP411

1400TransferRate-MB/s120010008006004002000

4Drives

6Drives

8Drives

10Drives

12Drives

RAID6RAID0

RAID0RAID5RAID6RAID1+0

Configuration:ProLiantDL360G6.SmartArrayP411controller.512MBCache.15K6GbSAS.256KBstripesize.Queuedepthof)

AdditionalRAIDperformancecharacteristics

ManydifferenttermsandmetricsareusedincharacterizingtheperformanceofSmartArrayRAIDlogicaldrives.Queuedepth,throughputandlatencyareoftenreferredtoinRAIDbenchmarkingtestsandneedtobeunderstoodinrelationshiptoeachother.

Queuedepth

Arrayperformancebenchmarksareoftenrunatvaryingqueuedepths.Itisimportanttounderstandthatinnormaluse,queuedepthisnotaconfigurableparameter.RAIDbenchmarkingtestscan

artificiallycontrolthequeuedepthinordertosimulatetheeffectsofcontrollerqueuedepthsgrowingorshrinkingunderanapplicationloadasshowninFigure10,atypicalRAIDbenchmarksuite.Inactualoperatingenvironments,thequeuedepth,atanygivenmoment,representsthenumberofpendingdiskcommandsthattheSmartArraycontrollerhasacceptedfromtheoperatingsystembuthasnotyetcompletedtodisk.ThecontrollercananalyzethecommandsinthequeuetofindmoreefficientwaystoexecutethemandincreaseoverallthroughputfortheSmartArraycontroller.

14

Figure10.Typicalarraybenchmarksuiterunatvaryingqueuedepthsandarraysizes

P411,2.00,RAID-5

MSA60,146GBLFF15K3GbSAS,DF0146B8052

DL380-G5

IOMETER07.27.2006

180016001400

DATARATE(MB/s)120010008006004002000

KBSEQREADThroughputversuslatency

TheSmartArraycontrollerusesvarioustechniquestoincreasedatathroughputasqueuedepthincreases.However,increasingqueuedepthsareanindicationthattheSmartArraycontrollerisfallingbehindinprocessingthediskcommandsfromtheoperatingsystemandapplications.Asqueuedepthsincrease,latency—thetimetheOSorapplicationseesittaketocompleteadisk

request—tendstoincrease.ThissituationcansometimesbeinfluencedbytheSmartArraycontroller,itself.Theverytoolsthatthecontrollerusestomaximizedatathroughput—commandcoalescingandreordering—canincreasetheoverallvariabilityoflatency.Applicationsrequiringlowerand/orconsistentlatenciesneedenvironmentswherequeuedepthsremainlow.Ingeneral,largequeuedepthsagainsttheSmartArraycontrollercanindicateapotentialcontrolleranddiskIObottleneck,whichcanpossiblybeaddressedbyaddingmoredrivestothearrayedlogicaldisk.

3216842132168421128KBSEQREAD32168421256KBSEQREAD32168421512KBSEQREAD1MBSEQREAD8+8-Drv,P411,fw:2.00,512MB,R5(256KB),2-MSA70,2LD8+8-Drv,P411,fw:2.00,512MB,R5(128KB),2-MSA70,2LD8+8-Drv,P411,fw:2.00,512MB,R5(KB),2-MSA70,2LD3216842132168421KBSEQWRITE32168421128KBSEQWRITE32168421256KBSEQWRITE32168421512KBSEQWRITE15

Formoreinformation

Foradditionalinformation,refertotheresourceslistedbelow.

Resourcedescription

HPSmartArrayControllertechnology–TechnologyBrief

PerformancefactorsforHPProLiantSerialAttachedStorage(SAS)–TechnologyBriefRAID6withHPAdvancedDataGuardingtechnology–TechnologyBrief

Webaddress

http://h20000.www2.hp.com/bc/docs/support/SupportManual/c00687518/c00687518.pdfhttp://h20000.www2.hp.com/bc/docs/support/SupportManual/c01460725/c01460725.pdfhttp://h20000.www2.hp.com/bc/docs/support/SupportManual/c00386950/c00386950.pdfCalltoaction

SendcommentsaboutthispapertoTechCom@HP.com©Copyright2010Hewlett-PackardDevelopmentCompany,L.P.Theinformation

containedhereinissubjecttochangewithoutnotice.TheonlywarrantiesforHPproductsandservicesaresetforthintheexpresswarrantystatementsaccompanyingsuchproductsandservices.Nothinghereinshouldbeconstruedasconstitutinganadditionalwarranty.HPshallnotbeliablefortechnicaloreditorialerrorsoromissionscontainedherein.MIPSisatrademarkofMIPSTechnologies,Inc.intheUnitedStatesandothercountries.TC100501TB,June2010