SoftwareReliability
Measurement
Measurement
“Measurementistheprocessbywhichnumbersorsymbolsareassignedtoentitiesintherealworldinsuchawayastodescribethemaccordingtoclearlydefinedrules”
Softwaremeasurement
• Canmeasureoneofthreeentities:-
processes-anysoftwarerelatedactivity;thesenormallyhaveatimefactor.Forinstance,design,testing,configurationmanagement,projectplanning,…
products-anyartefacts,deliverablesordocumentswhichariseoutoftheprocesses.Examples-designdocuments,testplans,sourcecode,executablecode,usermanuals,…
resources-itemswhichareinputstoprocesses,e.g.people,materials,money,…
Productmeasurement
QualityMetrics:-defectmetrics,designchanges,errorsincodeinspections,programtests,codechangesrequired
ReliabilityMetrics:-hazardrate(i.e.probabilityoferrorinintervaldgiventhatithasnotfailedintimecostofrepairingfaults,probabilityofsoftwarefailure
MaintainabilityMetrics:-relatedtocomplexitymeasures,readabilityofsourcecode
Measuringreliability
Canmeasure:-
• MTTF-MeanTimeToFailure
• MTTR-MeanTimeToRepair
• MTBF-MeanTimeBetweenFailures= MTTF+MTTR
Variousapproachesexist:-
• Userandominputsandmeasuredefects
• Carryoutcompleteobservation
• Lookatindependentsetsoftests
Reliabilitygrowth
• Identifyandfixfaultsduringtestingandcollectdataconcerning,forexample,thetimebetweenfailuresofthesoftware
• Usethisdatatopredicttimebetweenfailuresinthefuture
• ReliabilityGrowthoccursasfaultsareremovedinthesoftwareanditsreliabilityincreases.Thetimebetweenfailuresgrowslonger.
Reliabilitymeasures
• Complexitymeasuresmayinfersomethingaboutthereliabilityofsoftwarebuttheycannotbeusedtoassurereliability.
• Similarly,faultsordefectsinsoftwarecannotbeusedtopredictreliabilityastheymayormaynotleadtofailures.Somestudiesshowthatalargeproportionoffaultsneverleadtofailures.
Reliabilitymeasures(ctd)
• Thefrequencyofactualfailuresinthedevelopedsoftwareistheonlywaytopredictfuturereliablebehaviour.
• Measurementstypicallyinvolverecordingthetimesbetweensuccessivefailuresofthesoftwarewhenitisexecutinginasimulatedoroperationalenvironment.
Timeinmeasurement
Measurementscanbetakenintermsof:-
• ExecutionTime -theactualprocessingtimefortheexecutionoftheprogram
• CalendarTime-thetimeinfamiliartermsofseconds,minutes,andhours.
• ClockTime-thetimeacomputerisrunningwhileexecutingtheprogram.Otherprogramsmaybeexecutingonthesamemachineatthesametime.
Trendanalysis
• Requestforchange(RFC)-arequestputinbydevelopers,testers,qualityassurance,etc.tomodifythesoftware
• DefectRate(DR)-thenumberofdefectsdetectedduringthetestingphase
Analyzingtrends
• Defectrateincreasesasmoremodificationstakeplace-thissuggeststhatreliabilitygrowthisnotoccurring
• Asrequestsforchangestabilise,defectratedecreases-thissuggeststhatreliabilitygrowthisoccurring
Reliabilitymodelling
• Topredictreliability,currentfailuredataiscollectedandusedtoinferfuturebehaviour.
• Examplesoftheuseofsuchpredictionsinclude:-
– todetermineatwhatpointintimea particularlevelofreliabilitywillbe reached.
– todeterminewhatlevelofreliabilitywill havebeenreachedbyacertainpointin time.
Modelselection
• Severalmodelshavebeendeveloped.However,thesehavetendedtogiveoftenlargelydifferentresultsforthesamedatasets.
• Anappropriatemodelmustbeselectedaccordingtothecircumstancesandcontextandacertainamountofjudgementmustbeexercisedtodeterminewhichmodelisgivingreliableresults.
Reliabilitycurve
Hardwarereliability
• HardwareandSoftwarereliabilityarebasedonsimilartheory,theintentionbeingthatreliabilitiescanbecombinedtogiveameasureofoverallsystemreliability.
• Hardwarefailuresaretypicallyduetodeteriorationofmaterialsratherthanflawsindesign.Softwarefailuresaretypicallyduetodesignfaults.
Comparinghardwareandsoftwarefailure
• Hardwaredesignisseenasbeinglesscomplexthansoftwaredesign.
• Forhardwareatraditionalbath-shapedcurveisappropriate
• Softwarecanbereplicatedperfectlywhilethemanufacturingprocessforhardwaremayintroduceflaws.
Onsoftwarereliability
• Withsoftware,thereliabilitydoesnotalterwithtime.Assumingthestatedconditionsremainunaltered,thensoftwareisnotsubjecttorusting,corrosion,orwhateverandthatusuallysimplifiescalculationsconsiderably.Withhardwarethesituationisdifferent.
• Softwarereliabilitymayfluctuateasitismodifiedandduringtestinganddebugging.Hardwareusuallyfollowsabath-tubshapedcurvewithaconstantreliabilityformostofitsoperationallife.
Traditionalbath-shapedcurve:thehardwarepicture
Commentsoncurve
Thiscurveexhibitsthreedistinctphases:-
• aninitialperiodduringwhichtimethesystemis’running-in’;duringthisperiodreliabilityoughttoincreaserapidlyandgraduallystabilise.
• periodduringwhichthereisaconstantbutstablefailureratewhichiswellunderstood,expectedandcapableofbeinghandled.
• awear-outperiodduringwhichthefailureratesincreasedramaticallyandexhibitscharacteristicswhichsuggestreplacement.
Commentoninitialphase
• Ofthesedifferentphases,theinitialrunning-inperiodcanusuallybereducedorevenremovedbyincreasingattentiontoqualitycontrol,tototalqualitymanagement,andtorelatedactivities.
Commentsonhardwarehazards
• Theanalysisoffailuresisimportantinordertounderstandthecausesandhelppreventfuturedisasters.
• Overtheyears,surveyshavebeencarriedoutonthecausesofmetalfailure.Onesurveyrevealedthat:-
– 55%offailureswerecausedbycorrosion.– 45%offailureswerecausedby mechanicalfailure
Hazardsfrommovingmachinery
• Anadditionalsetofhazardsarisewhenthereismovingmachinerypresentinasystem.
• Thepresenceofsawsorrotatingdevicespresentaparticularlyacutesetofconcernsandleadtotheneedforguardsandsensorswhichcanautomaticallyswitchamachineoffifusedwrongly,andsoon.
