In April 2000, Polyteam benchmarked two caching proxy offered by Stratacache. The tests were executed in PolyLab using the PolyMix-2 workload.

We have followed the second bake-off rules and methodology to ease comparison of the results. This short report does not describe the workload or the caveats of interpreting the results. Familiarity with the official bake-off report is a must!

We also note that while the workload for this test is identical to what we used during the second bake-off, it is somewhat unfair to compare competitors results at the bake-off with the results achieved in a lab: Bake-off participants must meet the bake-off deadlines while a lab test is scheduled according to vendor preferences. Also, other vendors may have improved the performance of their products since the bake-off but did not officially test their new products yet.

We suggest that you use these results primarily as an indication of the progress done at Stratacache, rather than for inter-vendor comparison with bake-off figures.

As always, we do not give purchasing advise. We provide performance measurements and expect the reader to use this information, along with other important factors, in their decision making process.

Table of Contents

1. Where to find more information
2. Executive Summary
    2.1 Performance Changes since January
3. Performance Details
4. Product Configurations
5. Vendor Comments

Only major performance measurements are highlighted in this document. For more information, consider these sources:

• Auto-generated individual reports with raw measurements and trace data.
• Auto-generated tables and charts comparing the two products.
• Detailed configuration information including bake-off tuning options.
• Complete Polygraph logs.
• The official bake-off report where January Stratacache results were borrowed from.
• Web Polygraph site with documentation, sources, independent test results, and other benchmarking info.

The ``Executive Summary'' table below summarizes the performance results.

Links in column headers point to bar charts comparing the corresponding measurement. Product links point to a auto-generated individual reports for the entries.

All performance tests finished with negligible number of errors.

January and April results for the Flyer-100 product can be used to measure the progress achieved at Stratacache since the bake-off. Here is a summary of major changes in Flyer-100 performance since January.

• Positive changes:

  1. Document Hit ratio went up from 32% to 46%.
  2. Mean response time went down from 1.9 sec to 1.5 sec.
  3. Time until first miss went down from 4.0 min to 2.7 min.
  4. Cache age grew from 1.2 hr to 2.7 hr.

• Negative changes:

  1. Peak throughput went down from 400 req/sec to 376 req/sec.
  2. Hit response time went up from 0.04 sec to 0.11 sec.

A similar comparison for the Express-55 cache is not possible because Express-55 was not tested at the bake-off. The April results show close-to-linear scalability of the tested Stratacache products: Express-55 normalized performance and price/performance are very similar to that of Flyer-100. The only down side of the smaller box is that it failed a downtime test; manual intervention (a push of the power button) was required to boot the box.

Here is product performance in graphical form. Once again, please refer to the official second bake-off report for the discussion about these measurements.

Here are the configuration details for all tested products. Note the change in ICS software from 1.2.69 to 1.2.94.

All numbers are totals for the tested configuration unless noted otherwise. See individual product pages for more details about box configurations.

It is a Polyteam tradition to give vendors a chance to comment on the results after they have seen the review draft. The comments below are verbatim vendor submission. Polyteam has not verified any of the claims, promises, or speculations that these comments may contain.

Stratacache
http://www.stratacache.com/

The Stratacache product team wishes to thank Polyteam for the opportunity to use their testing facility to demonstrate the ongoing enhancements and improvements to our caching appliance product line.

In an effort to constantly increase the performance value of our products, we continuously focus on enhancing our products to provide better speed, more efficient caching and additional management features to the maximum benefit of our customers.

The testing results on the Flyer F-100 Series appliance show a number of these enhancements, including a 43% increase in document hit ratio (from 32% to 46%), a 21% reduction in Mean Response time as well as a 32% reduction in Time Until First Miss, and a 225% increase in cache age (from 1.2 to 2.7 hours) at 376TPS. In testing, we lowered our target rate from 400TPS to 376TPS to tune the appliance to this optimal mix of TPS vs Hit vs Response Time.

Our Express E-55 Series appliance performed exceptionally well, providing 200TPS with a 45% hit ratio. This cache is targeted specifically at the sub-100 concurrent user network, and at a $ 3,395 (list price), this product provides a new level of high performance/low cost caching for the small network environment.

Over the coming months and at the next Bakeoff, you will continue to see the results of further product enhancement and technology improvements to all of our caching appliance products that will further build on our world class caching and content acceleration product strategy.

For any questions surrounding our testing results or our products in general, please contact us at info@stratacache.com.