criterion performance measurements
overview
want to understand this report?
Data.Vector/1000000x1/FoldZip
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.16549642282361676 | 0.19893669365853311 | 0.21178281850708194 |
| Standard deviation | 2.115380644184678e-3 | 2.3320118712805288e-2 | 2.9666854557412876e-2 |
Outlying measurements have moderate (0.3636136676628221%) effect on estimated standard deviation.
Data.Vector/1000000x1/RecurseZip
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.1728427803399503 | 0.20457105392546132 | 0.22292190655157434 |
| Standard deviation | 1.0651533399346345e-2 | 3.133721240799802e-2 | 4.237947881418199e-2 |
Outlying measurements have moderate (0.3786922473379177%) effect on estimated standard deviation.
Data.Vector/1000000x1/RecurseZipWithN
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 7.34930453875753e-2 | 9.141201285336593e-2 | 9.888952367074126e-2 |
| Standard deviation | 4.977098221413557e-3 | 1.5980394594770523e-2 | 2.3820672986245114e-2 |
Outlying measurements have severe (0.547579172675292%) effect on estimated standard deviation.
Data.Vector/1000000x1/FoldZipWith6
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 6.795149003473754e-2 | 8.830016002408626e-2 | 9.680801328346815e-2 |
| Standard deviation | 4.237712934420799e-3 | 1.7743227009493438e-2 | 2.6398182283254266e-2 |
Outlying measurements have severe (0.5533774722702536%) effect on estimated standard deviation.
Data.Vector/1000000x1/UncheckedStFromBack
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.16003968963202625 | 0.18260184219243686 | 0.1941997987935042 |
| Standard deviation | 1.267521283448328e-3 | 2.1334900732907704e-2 | 2.6784333462703386e-2 |
Outlying measurements have moderate (0.36337319295184667%) effect on estimated standard deviation.
Data.Vector/1000000x1/UncheckedStFromBackModify
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.15973079646504784 | 0.18281653318810567 | 0.1962021293186687 |
| Standard deviation | 8.587495264547815e-3 | 2.196033744912416e-2 | 2.807564512098994e-2 |
Outlying measurements have moderate (0.3653484294247735%) effect on estimated standard deviation.
Data.Vector/1000000x1/UncheckedStFromBackBailEmpty
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.15257869132428825 | 0.1815634554835644 | 0.1923474187300235 |
| Standard deviation | 3.4992685586188316e-3 | 2.034697962920153e-2 | 2.645975403164893e-2 |
Outlying measurements have moderate (0.19004670235273774%) effect on estimated standard deviation.
Data.Vector/1000000x1/UncheckedStFromFront
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.1405110933245851 | 0.17653252356690474 | 0.19054228365140718 |
| Standard deviation | 1.190162442374694e-3 | 2.4687608296789065e-2 | 3.235415502427821e-2 |
Outlying measurements have moderate (0.37443407989931604%) effect on estimated standard deviation.
Data.Vector/1000000x1/CheckedStFromBack
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.1561516338259935 | 0.18466978591300404 | 0.19667643105959998 |
| Standard deviation | 7.639166860746019e-3 | 2.0292398224507686e-2 | 2.772815922980589e-2 |
Outlying measurements have moderate (0.18964774522945752%) effect on estimated standard deviation.
Data.Vector/1000000x1/CheckedStFromFront
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.15312230942261068 | 0.18019946948334337 | 0.19554199606774547 |
| Standard deviation | 7.215786216142994e-3 | 2.6677183310817785e-2 | 3.543204741421415e-2 |
Outlying measurements have moderate (0.37718622918249406%) effect on estimated standard deviation.
Data.Vector.Storable/1000000x1/FoldZip
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.1305886822184466e-2 | 1.2756255407896494e-2 | 1.3944688594523266e-2 |
| Standard deviation | 1.8058166426746527e-3 | 2.5027899470971447e-3 | 3.656068090623129e-3 |
Outlying measurements have severe (0.7507435627462671%) effect on estimated standard deviation.
Data.Vector.Storable/1000000x1/RecurseZip
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.305851709987909e-2 | 1.552530104317286e-2 | 1.7733995587738346e-2 |
| Standard deviation | 3.447500733292064e-3 | 4.349089011748706e-3 | 5.596461852953038e-3 |
Outlying measurements have severe (0.8566961785325582%) effect on estimated standard deviation.
Data.Vector.Storable/1000000x1/RecurseZipWithN
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.9565630873070033e-2 | 2.1837068952542695e-2 | 2.381440573876331e-2 |
| Standard deviation | 2.523805013720313e-3 | 3.976743298593864e-3 | 5.389048679392295e-3 |
Outlying measurements have severe (0.6971157459581565%) effect on estimated standard deviation.
Data.Vector.Storable/1000000x1/FoldZipWith6
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.915854053255188e-2 | 2.244390426258668e-2 | 2.4487691431281322e-2 |
| Standard deviation | 2.6153105103139595e-3 | 4.390036810819107e-3 | 6.239067439473215e-3 |
Outlying measurements have severe (0.6994025111179417%) effect on estimated standard deviation.
Data.Vector.Storable/1000000x1/UncheckedStFromBack
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 5.032804169506564e-3 | 5.8042970146874985e-3 | 6.465703554863135e-3 |
| Standard deviation | 1.0325993212038334e-3 | 1.3588243269297068e-3 | 1.8154992905416727e-3 |
Outlying measurements have severe (0.8471686179032263%) effect on estimated standard deviation.
Data.Vector.Storable/1000000x1/UncheckedStFromBackModify
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 4.937904749158965e-3 | 5.740587192405258e-3 | 6.355454723110735e-3 |
| Standard deviation | 9.749142879850184e-4 | 1.3455224095831548e-3 | 1.8320921253350525e-3 |
Outlying measurements have severe (0.8471925172309203%) effect on estimated standard deviation.
Data.Vector.Storable/1000000x1/UncheckedStFromBackBailEmpty
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 4.516933248967999e-3 | 5.278777878997258e-3 | 5.8807230794462526e-3 |
| Standard deviation | 9.947748086458646e-4 | 1.2653949054065756e-3 | 1.8918068435716585e-3 |
Outlying measurements have severe (0.8544175762609563%) effect on estimated standard deviation.
Data.Vector.Storable/1000000x1/UncheckedStFromFront
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 5.211079378463812e-3 | 5.90420411394627e-3 | 6.568863263569526e-3 |
| Standard deviation | 9.238985637663373e-4 | 1.2652233044156886e-3 | 1.7300876792805196e-3 |
Outlying measurements have severe (0.7989056720152113%) effect on estimated standard deviation.
Data.Vector.Storable/1000000x1/CheckedStFromBack
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 7.52214698256326e-3 | 8.838021611396553e-3 | 1.001391597276228e-2 |
| Standard deviation | 1.6580627642254159e-3 | 2.3607419949200435e-3 | 3.2304909377626847e-3 |
Outlying measurements have severe (0.8855096378807674%) effect on estimated standard deviation.
Data.Vector.Storable/1000000x1/CheckedStFromFront
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 7.323217987848546e-3 | 8.985774764190968e-3 | 1.0126345013605343e-2 |
| Standard deviation | 1.9129414668163627e-3 | 2.4680498954738964e-3 | 3.2023467670486013e-3 |
Outlying measurements have severe (0.8860099370306959%) effect on estimated standard deviation.
Data.Vector.Unboxed/1000000x1/FoldZip
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 9.585109459774015e-3 | 1.2151391402897326e-2 | 1.4029259091294216e-2 |
| Standard deviation | 3.035651120657935e-3 | 3.6178355369695502e-3 | 4.638453780128814e-3 |
Outlying measurements have severe (0.86656528275324%) effect on estimated standard deviation.
Data.Vector.Unboxed/1000000x1/RecurseZip
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 9.13802385682306e-3 | 1.0603908892449703e-2 | 1.2060466807270551e-2 |
| Standard deviation | 2.067498499297798e-3 | 2.72834737226564e-3 | 3.654102540212455e-3 |
Outlying measurements have severe (0.8242275423601867%) effect on estimated standard deviation.
Data.Vector.Unboxed/1000000x1/RecurseZipWithN
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.8470214854684766e-2 | 2.2143291667621107e-2 | 2.4711991748161487e-2 |
| Standard deviation | 2.889487187486072e-3 | 5.57122290421146e-3 | 7.33261472758904e-3 |
Outlying measurements have severe (0.7579957644805995%) effect on estimated standard deviation.
Data.Vector.Unboxed/1000000x1/FoldZipWith6
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 2.352347688634132e-2 | 2.685232257475761e-2 | 3.00144997829522e-2 |
| Standard deviation | 5.120831130971468e-3 | 6.540380197708889e-3 | 8.770135759366257e-3 |
Outlying measurements have severe (0.7572427428339529%) effect on estimated standard deviation.
Data.Vector.Unboxed/1000000x1/UncheckedStFromBack
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 3.665096232622929e-3 | 4.0980993154566955e-3 | 4.360625148439912e-3 |
| Standard deviation | 4.2005535963182135e-4 | 5.7507412883319e-4 | 8.364946173738271e-4 |
Outlying measurements have severe (0.6375003246056855%) effect on estimated standard deviation.
Data.Vector.Unboxed/1000000x1/UncheckedStFromBackModify
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 3.713008454307389e-3 | 4.270180949859371e-3 | 4.722657708357531e-3 |
| Standard deviation | 6.457436476380451e-4 | 9.474821455142261e-4 | 1.2878763815623634e-3 |
Outlying measurements have severe (0.8166703546797808%) effect on estimated standard deviation.
Data.Vector.Unboxed/1000000x1/UncheckedStFromBackBailEmpty
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 3.33962218225561e-3 | 3.802364075862133e-3 | 4.271234865509904e-3 |
| Standard deviation | 6.546415322202469e-4 | 8.344366768565041e-4 | 1.084092996072306e-3 |
Outlying measurements have severe (0.8239454665356054%) effect on estimated standard deviation.
Data.Vector.Unboxed/1000000x1/UncheckedStFromFront
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 3.550098656907925e-3 | 4.109376803628614e-3 | 4.708447709295017e-3 |
| Standard deviation | 7.357295695605377e-4 | 9.871572254648154e-4 | 1.2811522000945067e-3 |
Outlying measurements have severe (0.8663621797309772%) effect on estimated standard deviation.
Data.Vector.Unboxed/1000000x1/CheckedStFromBack
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 4.298078629347941e-3 | 4.88679637731293e-3 | 5.480621784045071e-3 |
| Standard deviation | 8.183953104013638e-4 | 1.1377149687287955e-3 | 1.596305807086866e-3 |
Outlying measurements have severe (0.8538630749376258%) effect on estimated standard deviation.
Data.Vector.Unboxed/1000000x1/CheckedStFromFront
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 4.758799892953123e-3 | 5.3455439034740876e-3 | 5.8925463100326335e-3 |
| Standard deviation | 7.652797705338606e-4 | 1.0310904341797186e-3 | 1.3834821148159323e-3 |
Outlying measurements have severe (0.7613289719637313%) effect on estimated standard deviation.
understanding this report
In this report, each function benchmarked by criterion is assigned a section of its own. The charts in each section are active; if you hover your mouse over data points and annotations, you will see more details.
- The chart on the left is a kernel density estimate (also known as a KDE) of time measurements. This graphs the probability of any given time measurement occurring. A spike indicates that a measurement of a particular time occurred; its height indicates how often that measurement was repeated.
- The chart on the right is the raw data from which the kernel density estimate is built. The x axis indicates the number of loop iterations, while the y axis shows measured execution time for the given number of loop iterations. The line behind the values is the linear regression prediction of execution time for a given number of iterations. Ideally, all measurements will be on (or very near) this line.
Under the charts is a small table. The first two rows are the results of a linear regression run on the measurements displayed in the right-hand chart.
- OLS regression indicates the time estimated for a single loop iteration using an ordinary least-squares regression model. This number is more accurate than the mean estimate below it, as it more effectively eliminates measurement overhead and other constant factors.
- R² goodness-of-fit is a measure of how accurately the linear regression model fits the observed measurements. If the measurements are not too noisy, R² should lie between 0.99 and 1, indicating an excellent fit. If the number is below 0.99, something is confounding the accuracy of the linear model.
- Mean execution time and standard deviation are statistics calculated from execution time divided by number of iterations.
We use a statistical technique called the bootstrap to provide confidence intervals on our estimates. The bootstrap-derived upper and lower bounds on estimates let you see how accurate we believe those estimates to be. (Hover the mouse over the table headers to see the confidence levels.)
A noisy benchmarking environment can cause some or many measurements to fall far from the mean. These outlying measurements can have a significant inflationary effect on the estimate of the standard deviation. We calculate and display an estimate of the extent to which the standard deviation has been inflated by outliers.