Apache on the DreamHost configuration will (by default) use FastCGI for PHP, this will spawn processes as needed (and end them over time) which can lead to large spikes in memory usage. Predicting memory usage in this setup is difficult, to say the least.

Memory Usage of DH's Apache Configuration

Nginx on the DreamHost configuration is a bit easier on the eyes which you can see from the graph below. It should be noted that I run several more sites on my Nginx configuration then I do on the Apache configuration and because of this I run a much larger pool of PHP processes which increases the overall memory. If both servers were running the same set of sites with the only difference in configuration and applications being the selection of Nginx instead of Apache, memory usage would be lower and still predictable.

Memory Usage of DH's Nginx Configuration

1) Process list dumped to log on resize (date, used memory, suggested memory, and cache memory are all listed as well)
2) Ability to send email on resize request
3) Ability to tweet (uses oauth) on resize request

Check out the readme file for information on installation, upgrades, and configuration.

As always please don’t hesitate to contact me if you need help or have any questions!

What does that mean to you?

Chances are you’re using this memory manager to help figure out where your memory is going and keep your services up and running. Having a snapshot of your processes, taken when the script sees the need for a resize, will tell you where your memory is actually being used. Very soon you’ll be able to review your logs and click on the “process log” link. I’ll be posting an update to the news you see within the memory manager when the newest version is ready.

More screenshots after the jump:

Is this not awesome?

Stay tuned, I’m hoping to get this pushed out as soon as the design is implemented on the remaining pages, which may take some time but should be out within the coming week.

https://github.com/jgalvez/JJs-DH-VPS-Memory-Manager/zipball/master

The design to the UI will be getting upgraded when I get the chance to work it out, but the entire application is functional, if you have questions or run into any problems please let me know. Here is the readme so you can see how it is installed and configured:

IMPORTANT NOTE: While I work for DreamHost this application is NOT supported by DreamHost and any questions should be coming to me NOT DH Support!

This application was built to be a VPS Memory Manager focused on uptime rather then cost savings. You can see one of my initial post on the subject here: http://www.gimmesoda.com/thoughts-on-vps-memory-management/

Installation

1. Grab the files using git or download them from github: https://github.com/jgalvez/JJs-DH-VPS-Memory-Manager
2. Upload the files.
3. Create an API key for use with the memory manager from here: https://panel.dreamhost.com/index.cgi?tree=home.api&

The key must have access to the following two functions:

dreamhost_ps-set_size services-progress

1. Visit the URL you uploaded the memory manager, the install script will start.
2. Fill out the form that comes up, save the settings.

Configuration

Once installed you should fine tune the application configuration. Let’s look at the important values in the config.php file:

define(‘MIN_MEMORY’, ’300′);
define(‘MAX_MEMORY’, ’4000′);

MIN_MEMORY should be the minimum memory your VPS uses. 300 is default only because that is the lowest amount of memory you are allowed to use, change this to something more realistic.

MAX_MEMORY should be a maximum amount of memory you are willing to pay for, remember that the memory manager will only resize to this level if the memory is actually needed. You are not likely to be at this level of memory usage for long periods of time, if you are go buy a dedicated instead.

Restart the Daemon

Anytime you make a change to the configuration file you should restart the daemon (using the restart link within the application).
That’s it! Feel free to contact me at jj@gimmesoda.com with any questions.

1. Creates a .pearrc file in your home directory.
2. Sets the download_dir, cache_dir, and temp_dir to folders within your /home/user/pear directory
3. Sets the PHP_PEAR_PHP_BIN variables by adding “export PHP_PEAR_PHP_BIN=/usr/local/php5/bin/php” to your .bash_profile
4. Modifies your PATH variable by adding “export PATH=/home/user/pear:/usr/local/php5/bin:$PATH” to your .bash_profile. This change also sets the PHP5 binary as the default for your shell.

Next time you call pear you’ll be using your custom settings and when you attempt to install any modules it will be done within your home directory. Now in order to use these PEAR modules in your application you will need to add the following to your app:

$pear_user_config = '/home/user/.pearrc';
set_include_path('.' . PATH_SEPERATOR . '/home/user/pear/php' . PATH_SEPERATOR . get_include_path());

Make sure to update “user” in the path to the actual user you are using and you’ll be set.

Download and Execute the Script

Just execute the following in a shell session:

wget http://files.gimmesoda.com/dreamhost/install_pear.sh; chmod 0744 install_pear.sh; ./install_pear.sh;

You can download and review the shell script from here, just open it up in a text editor of your choice.

Goal One: Save Money

Many people seem to have a belief that just because the site/s are on a VPS, even with memory set as low as possible (300MB of RAM assigned to a machine in the case of a DreamHost VPS), they will have perfect or near perfect uptime. Sadly, this is not the case and often times this misunderstanding/confusion can lead to serious downtime and frustration. Ideally, for people who are trying to have the lowest amount of memory usage possible, a memory management application would realize that there is a need for more memory, calculate an estimated amount of memory needed to handle the increased usage, resize the VPS as closely as possible to that need, then downsize as soon as the memory is no longer needed.

There is one major issue with this scenario:

Creating estimates regarding the amount of memory needed in an environment where usage is constantly changing, is a massive challenge. To be more specific; since the ability to resize a VPS tends to take some time (anywhere from 1-5 minutes), it is possible for memory requirements to have increased during the time that the need was detected and the memory estimate was created. This could create a resize on a VPS with too little memory, causing problems for the website/s on the VPS, as the machine is given too little memory to handle the increased need.

Goal Two: Keep Services Running

If you are on a VPS, chances are you’ve outgrown shared hosting or are looking for more stability and control over the environment your applications run on. If you’ve got a decent understanding of servers, applications, and websites, you may also understand that you need to have enough memory to run all the processes and handle the requests to your sites. Often times simply setting your service to the lowest amount of memory possible is going to be asking for failed script executions and the dreaded 500 error being displayed on your site.

So with this understanding, some people go to the extreme and max out their VPS resources. At DreamHost that means $200 a month for services (4GB worth of RAM) which may not be fully utilized (and most likely are not). So the question here becomes how to balance cost and the need for memory, in order to keep services running without paying the maximum amount at all times.

A Hybrid Memory Manager Focused on Uptime

With these two potential goals in mind, I had decided to mainly focus my attentions on goal two. While I do understand the need to keep price as low as possible, it is my personal belief, gleaned from experience, that a downed website is more damaging to the bottom line than a slightly larger payment for hosting services. So with this in mind, the memory manager was created to focus on uptime, while still saving time and money for those who manually resize throughout the day, and potentially over-allocate memory for their site/s.

All this being said, I am quite excited to be releasing this software soon. The status of this release is currently closed beta. However, if you are interested in testing this on your DreamHost VPS, please contact me. I am looking to increase the pool of Beta testers before the official release.

This post is password protected. To view it please enter your password below:

Password: