|
Discussion Forums
|
Thread: CPU performance not the same as advertised
|
|
|
Replies:
8
-
Pages:
1
-
Last Post:
Oct 13, 2007 12:54 PM
by: bfoster89
|
|
|
Posts:
12
Registered:
2/8/06
|
|
|
Posts:
4
Registered:
9/30/07
|
|
|
|
Re: CPU performance not the same as advertised
Posted:
Oct 10, 2007 1:38 PM PDT
in response to: Walter Purvis
|
|
|
"... but they seem to have decided to change their advertising (eventually,some day) rather than change EC2 to provide acceptable CPU resources ..."
lol
I assume you don't think they can just turn that little "CPU" dial on the outside of the servers to up the CPU performance??? Not to mention the one on the hard drives to up their RPM and cache .... and maybe string some really fat wires to up the network performance.
Seriously - Clearly this is a work in progress ("BETA"). Yet, here we are, PAYING for it. Good for Amazon.
Anyhow, changing the advertising would be the thing to do. Granted, they oughta do it NOW .....
Where is this advertising (the bit you're referring to? I can find my own but I'd rather hear your source from you)? I'd like to keep tabs on it.
Lovely Day
--Carl
|
|
Posts:
5,320
Registered:
3/19/07
|
|
|
|
Re: CPU performance not the same as advertised
Posted:
Oct 10, 2007 1:51 PM PDT
in response to: Carl Schuyler
|
|
|
> Where is this advertising (the bit you're referring to?
Go to
http://aws.amazon.com/ and click on "Amazon Elastic Compute Cloud (Beta)" on the lower left navigation pane
|
|
Posts:
12
Registered:
2/8/06
|
|
|
|
Re: CPU performance not the same as advertised
Posted:
Oct 11, 2007 1:38 AM PDT
in response to: Carl Schuyler
|
|
|
I assume you don't think they can just turn that little "CPU" dial on the outside of the servers to up the CPU performance??? Not to mention the one on the hard drives to up their RPM and cache .... and maybe string some really fat wires to up the network performance.
Anyhow, changing the advertising would be the thing to do. Granted, they oughta do it NOW .....
Yes, I realize there's no CPU dial (if there was, I'd want mine to go all the way to 11, naturally). I'm not concerned with the bandwidth, which is world-class, or the memory or the disk allotment/performance, both of which are generous enough to suit me.
In fact, I
might
be okay with the weak CPU, but it's important to know how much CPU resources we can expect to get with EC2 instances. The fact that the actual number appears to be only about 50% of what is advertised and that this discrepancy was pointed out and acknowledged almost a full year ago, and yet nothing has been done to resolve the discrepancy, is more than a little disconcerting.
EC2 requires you to take a leap of faith on scaling out instead of scaling up. Personally, I'd like to be able to scale up
and
scale out (e.g., I sure would like to have the option of 8 GB of RAM in one or more instances), and even though I can probably live with EC2's current RAM and disk allotments, the CPU seems rather pathetic. If I'm going to need to run 10 instances instead of 5, that's something I'll have to consider.
The point is, I want (and deserve) accurate figures and a definitive answer as to whether the current discrepancy is a bug that's going to be fixed (when?) or a permanent state of affairs, so I can make an informed decision on whether EC2 has enough "up" to go along with its "out."
|
|
Posts:
159
Registered:
8/24/06
|
|
|
|
Re: CPU performance not the same as advertised
Posted:
Oct 11, 2007 9:16 AM PDT
in response to: Walter Purvis
|
|
|
I am with Walter 100% if his claims are true. Amazon should be advertising the facts. Can anyone from Amazon comment on this?
|
|
Posts:
80
Registered:
3/7/07
|
|
|
|
Re: CPU performance not the same as advertised
Posted:
Oct 11, 2007 5:00 PM PDT
in response to: T. Reeder
|
|
|
Most of the people answering questions on this form are engineers (like me), but benchmarking is usually a fuzzy apples-to-oranges area that isn't well defined, which is probably why you don't see much comment on these threads.
There certainly are people thinking about how to best represent the capabilities of virtual machines. If you look at people selling real machines, there are all sorts of ways to represent the CPU capability, from Ghz, effective Ghz, etc. But there is no one rating that (accurately) captures everything about a machine. I'd expect to find an inverse relationship between the conciseness of a benchmark and the accuracy it provides.
Out of curiosity, I just ran some benchmarks on an instance vs my desktop, and the instance outperformed my 3.2Ghz P-4 desktop. But I'm not going to tell you what benchmark that was because it doesn't matter. The benchmark that matters is your own application. Even if/when we find a better way to represent the system capabilities, no single number is ever going to tell you how every application will run on a system. Like all CPU benchmarks, I see it as a starting point and treat it as such.
|
|
Posts:
12
Registered:
9/30/07
|
|
|
|
Re: CPU performance not the same as advertised
Posted:
Oct 12, 2007 3:54 AM PDT
in response to: Valient@AWS
|
|
|
Most of the people answering questions on this form are engineers (like me), but benchmarking is usually a fuzzy apples-to-oranges area that isn't well defined, which is probably why you don't see much comment on these threads.
Well, you could certainly make a more apples to apples comparison if someone had an Opteron 270 machine to run comparison benchmarks against. I have a dual processor Operteron 248 server (Not dual core) that I have benchmarked against I find that I agree with other posters on this forum that a single instance provides about half a core of CPU utilization.
There certainly are people thinking about how to best represent the capabilities of virtual machines. If you look at people selling real machines, there are all sorts of ways to represent the CPU capability, from Ghz, effective Ghz, etc. But there is no one rating that (accurately) captures everything about a machine.
True enough, but there are plenty of third parties that benchmark the crap out of new processors, motherboards, full servers etc. While just a single benchmark may not say much, several benchmarks will begin to provide a decent prediction of performance, and in what situations. I think we are seeing that here.
Out of curiosity, I just ran some benchmarks on an instance vs my desktop, and the instance outperformed my 3.2Ghz P-4 desktop. But I'm not going to tell you what benchmark that was because it doesn't matter.
That's too bad, It would be interesting to share this benchmark and figure out why it is doing so well, and would certainly contribute to the discussion.
On a side note I'm interested why the the stock AWS fedora images use 32 bit binaries?
|
|
Posts:
148
Registered:
8/15/06
|
|
|
|
Re: CPU performance not the same as advertised
Posted:
Oct 12, 2007 1:35 PM PDT
in response to: Valient@AWS
|
|
|
"But I'm not going to tell you what benchmark that was because it doesn't matter."
You do little for your or your employer's credibility when you make such statements.
It matters very very much as it would provide another valuable datapoint as to what works well on EC2 and what does not.
|
|
Posts:
12
Registered:
9/30/07
|
|
|
|
Re: CPU performance not the same as advertised
Posted:
Oct 13, 2007 12:54 PM PDT
in response to: Valient@AWS
|
|
|
Out of curiosity, I just ran some benchmarks on an instance vs my desktop, and the instance outperformed my 3.2Ghz P-4 desktop. But I'm not going to tell you what benchmark that was because it doesn't matter.
Based on this post:
http://developer.amazonwebservices.com/connect/message.jspa?messageID=56531#56531
I'm going to assume that you are referring to 'openssl speed rsa4096'. The funny thing is that this test will erroneously report results that are twice as good as actual. This is because openssl calculates the result using cpu time, not true transpired time. cpu time is off by half because of the virtualization and it's the same reason why the cpu maxes out at 50%.
When I run the test I get the following:
------------------------------------------------------------------------------------------------------
>> time openssl speed rsa4096
Doing 4096 bit private rsa's for 10s: 59 4096 bit private RSA's in 5.55s
Doing 4096 bit public rsa's for 10s: 3908 4096 bit public RSA's in 5.29s
OpenSSL 0.9.8b 04 May 2006
... skipped some unimportant stuf ....
timing function used: times
sign verify sign/s verify/s
rsa 4096 bits 0.094068s 0.001354s 10.6 738.8
openssl speed rsa4096 10.94s user 0.00s system 53% cpu 20.321 total
--------------------------------------------------------------------------------------------------------
So, for example sign/s is reported as 10.6 calculated with 59/5.55s. However, 5.55s is cpu time, but the actual time was 10s as reported, and verified with the time utility reporting that the total time of the test was 20 seconds (Given two 10s tests). sign/s is actually 59/10s or 5.9 nearly half what this benchmark indicates.
|
|
|
|
Sign in to the AWS Management Console
Create an AWS Account
