Upload Files to S3 with s3cmd in Shell Script

For uploading a defined set of files to S3 using s3cmd, you can use the below shell script.

In our case, there’s two additional options (you might find useful as well)

• -P sets the permissions to publicly readable
• –add-header=Cache-Control:max-age=604800 sets the Cache-Control metadata (i.e. the file will be cached for 1 week), but you can use other metadata as well

Source code

for f in *.png
do
        s3cmd -P --add-header=Cache-Control:max-age=604800 put $f s3://yourbucket/yourpath/$f
done

For syncing an entire directory, there’s the sync option

Source code

s3cmd sync yourfolder s3://yourbucket/yourpath

To see if the files are were uploaded, respectively get the total number/count of files in a bucket

Source code

s3cmd ls s3://yourbucket/yourpath/
s3cmd ls s3://yourbucket/yourpath/ | wc -l