Seems to apply to other services like SQS and SimpleDB also.
I wonder if it has anything to do with the EC2 Nat migration scheduled for today?

Hopefully we will soon get som answers from Amazon...

We're investigating.

Thanks,
Kathrin

Hi, what is the dead line to fix this inssue, because i have many clients using the <span style="color: #ff3300">S3</span> service.

Thanks

We are seeing the same thing with our website, which hosts all videos on Amazon S3.  What is the ETA for getting this back up?  Please keep us posted on the forums, since this is mission critical for some.

Thanks.

<span>We can confirm the high error rate you’re experiencing. While we don’t have an ETA at this point, we’re working as quickly as possible to restore performance.<span>  </span>We’ll provide updates as soon as we have them.</span>

Thanks,
Kathrin

This is still a problem - since 7:30am Eastern Time, we've had these messages:

<Error><Code>InternalError</Code><Message>We encountered an internal error. Please try again.</Message><RequestId>2433D8D5AE03885A</RequestId><HostId>Zz1uFUDV5qGTp6lnvBDh2MZBBJx/N1ds5QSffwfSto5MStXJ9axNxsEtuClJNwOW</HostId></Error>

And this is why you have to setup a fail-safe. My new sites hosts over 25,000 images on Amazon and I wake up to notice major issues this morning. I switched over to using my local server and everything is back up...I really need to set something up so it does this automatically.

The s3 service is great but this just proves you can't rely on it, this is a major issue especially since it's been down for so long. Way to go Amazon. <img class="emoticon" src="http://developer.amazonwebservices.com/connect/images/emoticons/silly.gif" border="0" alt="" />

Unfortunately I think this is a 100% error rate for us - S3 is just inaccessible.

Please keep us posted - when you have an ETA (or figured out the problem/fix) please relay it to us.  Given the SLA, etc. we are using S3 in a production environment.

I have to add a major ME TOO here. My business is effectively closed right now because Amazon did something wrong. I'll have to reconsider using the service now.

Hello,

it might be also useful to know that for about an hour I couldn't sign in to this forum because clicking the Login link generated a 404 error.

now luckily I'm logged in and can post again.

this is really a widespred error error. I can't see manage any of my S3 accounts, all public objects can't be reached (500 Internal Server Error).

I have been told though that from Australia when switching from HTTP to HTTPS the objects can be retrieved though...
<span style="font-size: x-small">

<span style="font-size: x-small">
having an idea of the ETA to resume 'normal' service is essential.

best regards
andrea</span>

</span>

Whine Whine Whine... could we be a bit more dramatic? This is the first outage I have experienced since I joined the service nearly a year ago. Yes it sucks, yes I hope they get it fixed very soon... but, the sky is not in fact falling at the moment.

If you rely on a single point of storage for mission critical data, well then, you've got bigger problems.

We have the same issue here @ 14:16 GMT

Unable to login via transmit and problems requesting public images. Getting the following error:
Http/1.1 Service Unavailable

yea this is extremely widespread,

all my s3 properties are experiencing it. Even twitter is too.

it's a 100% error rate not just high. S3 service has stopped working about 2 hours ago. this is really a severe blow to confidence in trusting AWS services.

regards,
andrea

Echoing the previous comments:
- outage started around 7:25 EST
- 404 when accessing login page for forum
- initially I could not connect to EC2 or S3, at all (100% failure)
- EC2 started working again just before 9:00 EST
- S3 is still 100% failure

An ETA would be helpful, but it's understandable that it could take a little bit to determine the problem first before providing one.  Please just keep us posted once you have new info.

Also, emergency contact information needs to be provided.  If these forums are our only way of contacting support, and they stop working due to a failure (like this morning), how can we notify you?  Errors happen, but there MUST be a fail-safe way of reporting them.

Dears,

I´m really afraid to use S3 service now. My company choose to work with Amazon because of it´s reliability. We host more than 30,000 images from the number 2 TV station in Brazil. Now we are having several problems because of this S3 issues.

We should send this error to blogs and newspapers. Everyone should know what´s going on with Amazon´s Web Services.

Hope we have news soon!

Gustavo

The whole point of S3 is that it is distributed, redundant, and backed by an SLA.  Yes it would be a good idea to have a backup solution, but that's like saying I need to back up my data with three different storage providers.  With uptime it's all about figuring out (a) what the expected uptime of a service is and (b) what redundancy you have to have to make your uptime as high as you need it given that.

That being said, I also have had only minimal issues with S3 until now - which is why we made that "expected uptime" so high.  We'll have to re-evaluate that.

The errors are alternating between:
Http/1.1 Service Unavailable

and a full

<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>InternalError</Code><Message>We encountered an internal error. Please try again.</Message><RequestId>4DA9AB18343E90DA</RequestId><HostId>mik6GeTv+NChd4rFfpawsaaASfTZOdXN2FlKw5kcoJk61NnIKgpG+RaJwqUBDrUh</HostId></Error>

Don't know if that helps, but yes, this issue is really filling up our support queue with emails and complaints

Hi,
we are experiecing the same issue ...
for example, trying to access
the key: 1936.jpg
in the bucket: buddies.koinup.com
we get this error:

Error>
<Code>InternalError</Code>
<Message> We encountered an internal error. Please try again.
</Message>
<RequestId>41786F29ED914C93</RequestId>
<HostId> vWM+6yPsvhu+7ppNGiDqVNGmm2v8mxg6yoIh7rIyJ21VjWMrka9rS8xMi76a6KDj
</HostId>
</Error>

the same happens for  every key in the bucket.....
storage.koinup.com
buddies.koinup.com

some minutes ago the previous error changed in:
Http/1.1 Service Unavailable


postscript: for about 45 minutes i wasn't able to login into the forum
so i have an other delay with the communication
now is more than on hour with the issue

BTW, we faced other different issues in the last weeks.....as other users are saying
if you have Amazon, you should have also an other service for back up

this is a problem....

hope you fix this issue quickly

Message was edited by: techup

While I'm surprised this kind of error is possible, a big thanks to Amazon for getting onto this so quickly.

I've had 100% success rate on GET requests for about 20 mins (although all other requests still seem to be failing).

While S3 Europe is happily trotting along, indeed S3 US is completely down.
Which also means our site is completely down.

The second mayor outage in about a month, although last time it was a DNS issue right outside the S3 infrastructure.

I won't go as far as to say I'll stop using S3 as it's proven very reliable in the last 4 months and has allowed us to handle peak volume a lot better than in the past...
Nevertheless, it's clear that emergency scenarios need to be investigated.

How about AWS developing "replicating buckets" between EU and US?

A bit of load-balancing + error-correcting DNS on top and we've got a world class solution... and honestly, I wouldn't mind at all paying for the bandwidth usage between EU and US to replicate.

Heck if AWS adds loadbalancing DNS to S3, I'd be happy to do my own replication.

Same issue here, its going now for the 3rd hour completely without service.

Sadly, that is not the first time we see those kind of failures, although past downtimes where less than 10 minutes.

On a positive note, 30min ago I could not even post to this thread so it looks like work is being done. Hope it does not take too much longer.

lucky you...

3 hours into the disruption and the 'ERROR 500: Internal Server Error.' has now become 'ERROR 503: Service Unavailable.'