AWS Linux

Install s3fs on Ubuntu from SVN

s3fs mounts an Amazon AWS S3 bucket into your files system as a directory. This is fantastic for backup folders – backups are automagically off-site. Here is a better way of installing s3FS using SVN. Install Pre-requisites apt-get install build-essential libfuse-dev fuse-utils libcurl4-openssl-dev libxml2-dev mime-support Install s3fs svn checkout http://s3fs.googlecode.com/svn/trunk/ s3fsbr>cd s3fs/ autoreconf –install ./configure –prefix=/usr make make install Syntax is s3fs bucket-name mount/point You must either setup two environment variables AWSACCESSKEYID and AWSSECRETACCESSKEY or…

Continue reading

AWS

Set the region for AWS SDK for PHP

If you are using the PHP SDK for AWS, by default, all operations are done on their Virginia data center. If you want to use a different data center, you must define it with the set_hostname command $sdb = new AmazonSDB(); $sdb -> set_hostname(‘sdb.ap-northeast-1.amazonaws.com’); There are two parts to this, The Service (eg SDB, EC2, etc..) The Data Center (eg Tokyo) Valid Data Centers are Location region Position Code Tokyo Asia Pacific North East ap-northeast-1.amazonaws.com…

Continue reading

AWS Drupal

Using AWS SDK for PHP

So I want to divide the raw data being collected in my current project from the administrative data. The situation is there will be thousands of small devices gathering data and sending that back to a data repository. A series of polling servers will request (“pull”) data from the dumber of the devices and place them in this database. The smarter devices will “push” straight to an “API” server that then does the same. There…

Continue reading

AWS

Installing s3sync

s3sync is a ruby based software that synchronises files from a local computer to/from Amazons S3 service. s3sync requires ruby. yum install ruby The easy solution now is to just install the gem gem install s3sync However, this sometimes does not work and you need to do it the long way… Run these commands wget http://s3.amazonaws.com/ServEdge_pub/s3sync/s3sync.tar.gz tar xvzf s3sync.tar.gz rm s3sync.tar.gz cd s3sync wget http://mirbsd.mirsolutions.de/cvs.cgi/~checkout~/src/etc/ssl.certs.shar sh ssl.certs.shar cd .. mkdir s3backup You can get your…

Continue reading

AWS

Make an Entire S3 Bucket Readable

By default, a new Amazon AWS S3 bucket is non-readable and all objects placed inside it cannot be accessed publically. You can change each object individually and make it public, or you can set a rule for the entire bucket. Log into the AWS Console and select the S3 tab and then the bucket you want to make public. Lets assume it is called “xxxxxx”. Click on the “Add bucket policy” button on the Permissions…

Continue reading

AWS MySQL

Backup all MySQL Databases to S3

Here is a simple script that backs up all accessible databases and sends them to an S3 bucket with the date as a prefix. username=”xxx” password=”xxx” backuppath=”/home/backup/” s3bucket=”xxx” s3prefix=\`date “+%Y/%m/%d”\` s3path=”xxx” # Amazon Keys export AWS\_ACCESS\_KEY_ID=xxx export AWS\_SECRET\_ACCESS_KEY=xxx # Make the backup path if it is not there if [ ! -d $backuppath ] then mkdir -p $backuppath fi # Get a list of all databases databases=$(mysql -N -u$username -p$password

Continue reading

AWS cPanel

Create a Post-Backup script for WHM to S3

This technique has been superseded by the new cPanel/WHM backup tool. The process is still interesting in that it shows you how to create hooks, but is no longer needed since it function is now built in. If you have a WHM/cPanel based server, then you probably already have a daily backup running that keeps daily, weekly and monthly backups in another folder. But this does not help if the server disappears – you also…

Continue reading