Gimme Soda

The problem with most small photo hosts tends to be speed. Once they actually start to get popular they find themselves with the trouble of having to handle a large amount of connections and bandwidth. For those running off a VPS or even a dedicated server these problems can manifest themselves VERY quickly. Unless you have a clustered setup running on multiple network connections you will run into problems. I could never imagine many small free photo hosts running on such a network, until now.

Enter Amazon S3 in all its brilliance.

Amazon S3 is very adequately named, Amazon Simple Storage Service; the system is extremely simple to interact with (atleast on a basic level) and, well, it’s a storage service. So what makes it so brilliant? Well I think Amazon puts it best, by saying:

“Amazon S3 is based on the idea that quality Internet-based storage should be taken for granted. It helps free developers from worrying about where they are going to store data, whether it will be safe and secure, the costs associated with server maintenance, or whether they have enough storage available. The functionality is simple and robust: Store any amount of data inexpensively and securely, while ensuring that the data will always be available when you need it. Amazon S3 enables developers to focus on innovating with data, rather than figuring out how to store it.

Amazon S3 was built to fulfill the following design requirements:

• Scalable: Amazon S3 can scale in terms of storage, request rate, and users to support an unlimited number of web-scale applications. It uses scale as an advantage: Adding nodes to the system increases, not decreases, its availability, speed, throughput, capacity, and robustness.
• Reliable: Store data durably, with 99.99% availability. There can be no single points of failure. All failures must be tolerated or repaired by the system without any downtime.
• Fast: Amazon S3 must be fast enough to support high-performance applications. Server-side latency must be insignificant relative to Internet latency. Any performance bottlenecks can be fixed by simply adding nodes to the system.
• Inexpensive: Amazon S3 is built from inexpensive commodity hardware components. As a result, frequent node failure is the norm and must not affect the overall system. It must be hardware-agnostic, so that savings can be captured as Amazon continues to drive down infrastructure costs.
• Simple: Building highly scalable, reliable, fast, and inexpensive storage is difficult. Doing so in a way that makes it easy to use for any application anywhere is more difficult. Amazon S3 must do both.

A forcing-function for the design was that a single Amazon S3 distributed system must support the needs of both internal Amazon applications and external developers of any application. This means that it must be fast and reliable enough to run Amazon.com’s websites, while flexible enough that any developer can use it for any data storage need.”

So basically, S3 is brilliant.

Running a photo host on top of S3 may not be the worst idea either. Consider this, a photo host must constantly expand their storage network in order to keep files on the network available to their users. S3 is built so you don’t have to worry about that. Whether you are storing a few megabytes, or a terabyte, you don’t have to worry about expanding the storage plan. On top of that you will only end up paying for what you are using. With a normal storage plan you pay for your full storage capability, even if you aren’t using it to the fullest potential.

The Needs of a Photo Host

Reliability, Speed, and Storage. These are the techincal needs of a photo host and are mostly handled by S3. I say mostly handled because the server which runs your website is also of extreme importance. The speed your website can be accessed and how quickly it can accept uploads from your users can be a major factor in regards to keeping your users. Still, the fact that your server should never be serving the bulk of your users images should mean your main server will be able to handle much more. 

Anyhow as a company whose business is to host the photographs of its users, you can now focus on adding features which will, I would hope, make your photo service the one which will fit the needs of your users.

The First Step

Go to http://aws.amazon.com/ and create an Amazon S3 account. Remember you will be changed for all types of usage. Don’t worry though, thus far my heaviest section of testing involved querying over 60,000 alexa records and I have never had to pay over $10 worth of charges in any single month, this may be different for you.

I have an account now what?

Take a look at the technical specifications from the Amazon S3 resource center. The main documentation from Amazon is absolutely essential in the development of anything regarding their API’s. If you are not computer savvy, not a programmer, or simply don’t feel like doing it yourself; find yourself a decent programmer preferably one who has dealt with the API’s before and have the programmer build the site for you.

Also, other than the documentation, two other downloads I would recommend are:

Amazon S3 PHP Class by Geoffrey P. Gaudreault found @:
http://www.neurofuzzy.net/2006/08/26/amazon-s3-php-class-update/

Class.Upload.PHP by Colin Verot found @:
http://www.verot.net/php_class_upload.htm

Both of these classes will help make your life much easier when programming a photo hosting site from scratch.

Not bad, I’ve got a small start. Now what?

Start programming and let me know if you come up with anything interesting! Good luck as always. I will be publishing my own results with the use of these classes and my own custom built, object oriented PHP5 framework. You should be able to see the current results of the work at http://beta.blackfalconsolutions.com/ it will eventually be moved to the photo subdomain. BTW I was able to build the beta photo site using the above classes in about 4 hours. Also, my version is capable of using a threading like method to upload many files to S3 simultaneously. Most photo uploads should hit the S3 network quickly (well under a minute) and email you the URL for accessing the image.

Currently you can get some free photo hosting through my service at: http://beta.blackfalconsolutions.com/, I will be expanding the features of this service in time.

Trackback URL