The Blog of a Programmer
Programming
1 year ago
Mostly for my personal notes, but this is useful to anyone who works with raw apache logs.
awk '{A[$1] += $10} END {for (i in A) print i ": " A[i]/1024/1024 " Mb"}' access.log | sort -k2 -n
A quick little one-liner to figure out what IP is sucking away your bandwidth.
PHP-FPM ondemand!
01 year ago
PHP 5.3.9 came out on Jan 10th and reading over the changelog I geeked out.
http://php.net/ChangeLog-5.php#5.3.9
Specifically because of this line item:
Implemented FR #52569 (Add the “ondemand” process-manager to allow zero children).
Go check out that bug and the related patch. It’s going to be amazing to see shared hosting services take advantage of PHP-FPM with the ondemand process manager option. In the past, when you started PHP w/FPM, you had two options dynamic or static, both of which created a pool of php processes at startup. Using ondemand allows for the php children processes to get started when there is a request that needs to be handled.
I just switched my server over to a custom php install using this feature, here was my memory usage before enabling it:
free -m
total used free shared buffers cached
Mem: 1358 1242 115 0 0 342
-/+ buffers/cache: 900 457
Swap: 0 0 0
Immediately after implementation this is my memory usage:
free -m
total used free shared buffers cached
Mem: 1708 518 1189 0 0 491
-/+ buffers/cache: 26 1681
Swap: 0 0 0
I should note that my memory manager increased the available memory on my machine while it was compiling PHP.
Amazing.
JJ’s VPS Memory Manager v1.1 – Released
101 year ago
Check out the download links on the side bar to the right. You’ll now see that v1.1 of my memory manager has been released. Since I’ve got to start my shift soon I’m just going to give a quick summary of the new features, starting with the most important:
1) Process list dumped to log on resize (date, used memory, suggested memory, and cache memory are all listed as well)
2) Ability to send email on resize request
3) Ability to tweet (uses oauth) on resize request
Check out the readme file for information on installation, upgrades, and configuration.
As always please don’t hesitate to contact me if you need help or have any questions!
JJ’s Memory Manager – Update Coming Soon
01 year ago
I’ve said this many times before, “I’m not a UI designer.” It’s true, I’ve got a 0 on the scale of design sense. However, I’m extremely thankful to say that there are many people out there who do, and I’m fortunate enough to have some great people around who are willing to help improve my projects. Due to wonderful people donating their time and efforts my memory manager is going to be getting a facelift (I’m really excited about this) and here’s a sample of what’s to come.
Is this not awesome?
Stay tuned, I’m hoping to get this pushed out as soon as the design is implemented on the remaining pages, which may take some time but should be out within the coming week.
Configure Cron using PHP
11 year ago
Once again my insomnia proved to be useful. Have you ever wanted to automate the addition of a cron from PHP instead of having to manually configure from your host’s control panel? Check out the class I created for use with my memory manager:
http://code.gimmesoda.com/memorymanager/libs/Linux_Cron/cron.class.php
In my case I implemented the class by doing the following:
$this->start_daemon_command = '/usr/local/php5/bin/php ' . $system_path . '/daemon.php >> ' . $this->error_file . ' 2>&1 &';
$this->cron_command = '*/5 * * * * ' . $this->start_daemon_command;
function add_cron() {
$crontab = new Gimme_Cron;
$crontab->add_line($this->cron_command);
}
function remove_cron() {
$crontab = new Gimme_Cron;
$crontab->remove_line($this->cron_command);
}
Enjoy!
Another Magento Installer Update!
02 years ago
It took me long enough but I finally updated the Magento installer to handle the newest stable version of Magento. Check out the updated post here.
Simple Setup of PEAR on a DreamHost Account
02 years ago
While DH installs a basic set of PEAR modules it’s a common situation need others, when that situation comes up you’ll often be referred to the PEAR article on the DH wiki. I’ve taken the instructions found there and created a shell script you can download and execute, it’ll do the following:
- Creates a .pearrc file in your home directory.
- Sets the download_dir, cache_dir, and temp_dir to folders within your /home/user/pear directory
- Sets the PHP_PEAR_PHP_BIN variables by adding “export PHP_PEAR_PHP_BIN=/usr/local/php5/bin/php” to your .bash_profile
- Modifies your PATH variable by adding “export PATH=/home/user/pear:/usr/local/php5/bin:$PATH” to your .bash_profile. This change also sets the PHP5 binary as the default for your shell.
Next time you call pear you’ll be using your custom settings and when you attempt to install any modules it will be done within your home directory. Now in order to use these PEAR modules in your application you will need to add the following to your app:
$pear_user_config = '/home/user/.pearrc';
set_include_path('.' . PATH_SEPERATOR . '/home/user/pear/php' . PATH_SEPERATOR . get_include_path());
Make sure to update “user” in the path to the actual user you are using and you’ll be set.
Download and Execute the Script
Just execute the following in a shell session:
wget http://files.gimmesoda.com/dreamhost/install_pear.sh; chmod 0744 install_pear.sh; ./install_pear.sh;
You can download and review the shell script from here, just open it up in a text editor of your choice.
Thoughts on VPS Memory Management
02 years ago
Recently, I have been spending a significant amount of time working on an application which manages the amount of memory assigned to a DreamHost Web VPS. During the time I’ve spent developing this several choices had to be made in what the real goal of the application had to be. When it comes to memory management on a server there are really two goals which don’t always play well together.
Goal One: Save Money
Many people seem to have a belief that just because the site/s are on a VPS, even with memory set as low as possible (300MB of RAM assigned to a machine in the case of a DreamHost VPS), they will have perfect or near perfect uptime. Sadly, this is not the case and often times this misunderstanding/confusion can lead to serious downtime and frustration. Ideally, for people who are trying to have the lowest amount of memory usage possible, a memory management application would realize that there is a need for more memory, calculate an estimated amount of memory needed to handle the increased usage, resize the VPS as closely as possible to that need, then downsize as soon as the memory is no longer needed.
There is one major issue with this scenario:
Creating estimates regarding the amount of memory needed in an environment where usage is constantly changing, is a massive challenge. To be more specific; since the ability to resize a VPS tends to take some time (anywhere from 1-5 minutes), it is possible for memory requirements to have increased during the time that the need was detected and the memory estimate was created. This could create a resize on a VPS with too little memory, causing problems for the website/s on the VPS, as the machine is given too little memory to handle the increased need.
Goal Two: Keep Services Running
If you are on a VPS, chances are you’ve outgrown shared hosting or are looking for more stability and control over the environment your applications run on. If you’ve got a decent understanding of servers, applications, and websites, you may also understand that you need to have enough memory to run all the processes and handle the requests to your sites. Often times simply setting your service to the lowest amount of memory possible is going to be asking for failed script executions and the dreaded 500 error being displayed on your site.
So with this understanding, some people go to the extreme and max out their VPS resources. At DreamHost that means $200 a month for services (4GB worth of RAM) which may not be fully utilized (and most likely are not). So the question here becomes how to balance cost and the need for memory, in order to keep services running without paying the maximum amount at all times.
A Hybrid Memory Manager Focused on Uptime
With these two potential goals in mind, I had decided to mainly focus my attentions on goal two. While I do understand the need to keep price as low as possible, it is my personal belief, gleaned from experience, that a downed website is more damaging to the bottom line than a slightly larger payment for hosting services. So with this in mind, the memory manager was created to focus on uptime, while still saving time and money for those who manually resize throughout the day, and potentially over-allocate memory for their site/s.
All this being said, I am quite excited to be releasing this software soon. The status of this release is currently closed beta. However, if you are interested in testing this on your DreamHost VPS, please contact me. I am looking to increase the pool of Beta testers before the official release.
A DreamHost VPS Memory Manager – Coming Soon
02 years ago
I’ve been working on this for awhile now and it’s almost ready! I’m just working out a better set of rules for figuring out what amount to resize to and I’ll be releasing the first beta!
Top Commenters