[ExI] ibm takes on the commies

[David Lubkin]

Eugen wrote:

>The yardstick typically used is LINPACK http://www.top500.org/project/linpack
>Not terribly meaningful, but it meets the way people tend to solve problems,
>so it's not completely useless. Obviously, the 
>only way to measure the performance
>is to run your own problem.

Back in my [ LLNL, Apollo, HP ] days, it was 
common for hardware manufacturers to "study for 
the test." Since customers and press paid 
attention to benchmarks like LINPACK or the 
Livermore Loops, engineering resources were 
focused on doing well at what the tests measured, 
at the expense of other facets. For instance, 
superb vector operation (e.g., the Cray's ability 
to perform the same operation on 64 sets of 
floating-point operands in one instruction) was 
often coupled with mediocre performance for integer scalars.

This wasn't just at the hardware level. My boss 
for a time at Livermore was one of the top 
compiler guys anywhere, and people used his 
LRLTRAN over the Fortran that came from Cray 
(CFT) because he generated higher-performance 
machine language than Cray knew how to.

Similarly, the compiler groups at computer 
vendors focused on making the benchmarks run fast.

Nothing inherently wrong with that except that, 
as Eugen noted, you need to see if the 
(computer+compiler) runs fast on what *you* would use it for.

*However*, there were vendors caught in cheating. 
They wrote compilers that detected when a 
standard benchmark was being compiled and 
generated better code than they ordinarily could.


-- David.

Easy to find on: LinkedIn • Facebook • Twitter • Quora • Orkut