I see this also. One thought is whether the NIC is up and running when the s3cmd script runs.

Yes, that is a good idea. However, I'm positive the network is up already because I have the following snippet (props to Eric Hammond) at the top of the script:

# Wait until networking is up on the EC2 instance.
perl -MIO::Socket::INET -e '
 until(new IO::Socket::INET("169.254.169.254:80")){print"Waiting for network...\n";sleep 1}
' | $logger

And I read the user-data via curl, which succeeds.

So the network is up.

I suspect it's something specific to s3cmd, but Pyhon is not my strong suit.

It's probably related to some difference in your environment when you are logged in as opposed to when the script is running as part of the startup sequence.  I have run into similar problems with cron jobs. 

Is there any output at all in syslog?

Mitch

Oddly, my svn update also doesn't work correctly during init 4.
I have it staged at S98. A simple shell script that performs an s3cmd sync and svn update. I forward those outputs to a log and they come up blank, but log prints before and after those commands appear in the >> log.

its as if they get skipped without running.
I run the same init script after boot and it performs correctly, with correct log output.

Got me. But this needs to work.

Darren

Mitch, your comment helped me realize what's different about the startup sequence:
the operative user is root. When I log in, I'm the "ubuntu" user.

s3cmd looks in the current user's ~/.s3cfg - which didn't exist in /root/.s3cfg, only in /home/ubuntu/.s3cfg.

Luckily s3cmd allow you to specify the config file's location with --config /home/ubuntu/.s3cfg .

I realized this by redirecting stderr to stdout and piping the output to the logger:

s3cmd blah blah 2>&1 | logger

This resulted in the following output in /var/log/messages:
Dec 27 13:41:56 ubuntu S99_my_startup_script.sh: ERROR: Can't find a config file. Please use --config option.

From there it was straight sailing.

Now if I can only figure out why my startup script seems to be called twice...

That seems to be the problem.

Has anyone successfully fixed this issue?
Im having trouble running s3cmd on cron
here is my script
#! /bin/bash
env >/tmp/cronjob.log
source /home/user/.s3cfg
--config /home/user/.s3cfg
/usr/local/bin/s3cmd put file s3://bucket

the logger is saying that access_key: command not found, etc...
what should I remove or add to my script?

Thanks

Why are you using

source

? I'm not sure how that would work. Why not

s3cmd --config /path/to/config put file s3://bucket/file

Thanks for the reply...=)
I think there's something wrong with "put" command.
So here's what I did.
script:
#!/bin/bash
s3cmd sync /directory/backup-scripts s3://bucket
That works!

But using this script
#! /bin/bash
s3cmd --config /home/.s3cfg put file s3://bucket/file
Wont work..So for now I stick with s3cmd sync.
PS: also tried
s3cmd --config /home/.s3cfg put file s3://bucket
s3cmd -c /home/.s3cfg put file s3://bucket/file
still wont work..
thanks.

Hi fixed this by doing the follow

make a copy of the .s3cfg

cd /root/

cp .s3cfg /usr/local/bin/

then u can use

/usr/bin/s3cmd --config /usr/local/bin/.s3cfg sync bucket etc ...

I tried your response and issued the following in a PHP script

$upload="/usr/bin/s3cmd --config /usr/local/bin/.s3cfg put ".$tarname.$as3Bucket.$tarname;
shell_exec($upload);

but I am still unable to upload my tarballs to S3.

Any suggestions?

From the command line of course it works perfectly

Edited by: Charles R. Feilding on Feb 12, 2013 5:20 PM