Comments on blog.aarondelp.com: Why VMWare's VMmark Scores Have Become Useless

CEDB, I agree you can't standardize a companie...

2010-02-20T11:07:41.877-05:00

CEDB, I agree you can't standardize a companies negotiating prowess, but there needs to be some method to ensure you're comparing apples to apples. Cisco used $200,000 worth of solid state disks to gain a 2% advantage in their 8 core VMmark. That fact is lost on the casual observer. But if a $/VMmark was added to the scoring, it will create a disincentive for this type of gamesmanship,

We are in the process of evaluating new blade serv...

2010-02-13T12:16:51.971-05:00

We are in the process of evaluating new blade servers platforms mainly for our vmware platform...
The benchmark dilenma does hit us pretty heavily as we try to be "agnostic" in every ways (hardware vendors as well as processors...).
Not convinced that specs (cint, cfp) is th right answer we investigat vmmark...
The benchmark is certainly a valid one thought limited in

Todd, I think you were in the right track to add a...

2010-02-09T16:18:21.256-05:00

Todd, I think you were in the right track to add a cost component to make VMmark more useful. The article stated that there wasn't a problem with using SSDs to gain an advantage. But those SSDs add $220,000 to the cost for hardware to support the benchmark. I believe VMware should add a $/VMmark score to the benchmark as Todd suggested. Then the score will become relevant.

Hey Todd - Thank you very much for the comment. I...

2010-02-07T16:41:14.690-05:00

Hey Todd - Thank you very much for the comment. I agree with you about comparing platforms. I was recently asked how an IBM HS22 Blade would perform against an IBM 3850 M2 4 socket box. That is a perfect example of a situation where the VMmark score would be very helpful!

My beef isn't with VMware, it is with the vendors. Too many times vendors (all of them, not picking on

I agree with you that using VMmark to make a decis...

2010-02-07T14:31:54.805-05:00

I agree with you that using VMmark to make a decision between two very similar servers isn't very useful. But VMmark scores can be very useful in lots of other ways. Like showing the difference between generations of servers or between 2-socket and 4-socket or Intel and AMD.

I think that most of your comments are really more generic about benchmarks in general if you think

Aaron: I have to admit, its nice to be able to cl...

2010-02-07T12:24:03.383-05:00

Aaron:

I have to admit, its nice to be able to claim top spot, but I also know tomorrow, or next week, or next month, things will change again.

While synthetic testing can provide an interesting data point, that may or may not be useful and relevant, I agree with you that it certainly should not be the sole buying criteria.

Regards,

Omar Sultan

Aaron, I couldn't agree with you more about V...

2010-02-07T12:08:28.835-05:00

Aaron,

I couldn't agree with you more about VMmark scores being largely irrelevant. If you make a server decision based on a 1-2% difference in a VMmark officially have your head buried in the sand. Most customers want to buy a data center architecture, not just a server.

As for Fujistu, they are using the Xenon 5590 which is not supported by Intel for server use.