When using Apache VirtualHosts you will be serving up sites from different directories, which also means you’ll be dumping logs in to different locations.

Rotating these logs can be a bit clumsy, however if you structure your DocumentRoot‘s in a logical fashion you can setup logrotate to do this with one config.

Usually logrotate config would look like this:

/var/www/domain1.com/logs/*.log {
        daily
        missingok
        rotate 30
        compress
        delaycompress
        ...
}
/home/domain-owner/domain2.com/logs/*.log {
        daily
        missingok
        rotate 30
        compress
        delaycompress
        ...
}
/var/www/some-other-dir/domain3.com/logs/*.log {
        daily
        missingok
        rotate 30
        compress
        delaycompress
        ...
}
...

Yet the path accepts a wildcard for the DIR, so why not set up your VirtualHost directories like this:

/var/www/vhosts
            domain1.com
                   logs
                   httpdocs
            domain2.com
                   logs
                   httpdocs
            domain3.com
                   logs
                   httpdocs

Now you can rotate logs in a one-liner with something like this:

/var/www/vhosts/*/logs/*.log {
        daily
        missingok
        rotate 30
        compress
        delaycompress
        notifempty
        create 640 root adm
        sharedscripts
        postrotate
                /etc/init.d/apache2 reload > /dev/null
        endscript
        prerotate
                if [ -d /etc/logrotate.d/httpd-prerotate ]; then \
                        run-parts /etc/logrotate.d/httpd-prerotate; \
                fi; \
        endscript
}

Are you still sending yourself emails when you run test cases? I was.

Being relatively new to TDD this took a while to get my head around, but using “mocks” has proven to be really useful.

I’ll focus on the CakeEmail class in CakePHP 2.1.x as this proved most useful when writing test cases for Copify (we send _alot_ of emails!)

Here’s what one of our model methods looked like originally (before we had a test case).

When we started writing tests for other areas of the application which used this method, it fired an email.

To get round this, we had to mock the CakeEmail component. This also means you can test it is doing what it is supposed to be.

So to test this, we first need to make a change to the method itself:

Now, we can mock the CakeEmail class and test to ensure each part of our model method does what it should:

While drunk at a football match…

Earlier this year, a fellow Morecambe FC supporter bet me I couldn’t knit a football scarf in the team colours.

Not being one to turn down a challenge, I sacrificed my remaining #manpoints and bought some knitting clobber.

The goods

I took a wild guess, and it turns out the needles I acquired are a decent size for a knitting novice like me, recognisable by simply “5mm” on the head of each needle.

They’re actually quite sharp too, would make a great weapon to attack defend from an intruder, bear, cyclist etc.

Big balls

After a consultation with an established knitting expert (my Granny) I went for four large balls of wool, two red and two white.

I had zero knowledge of purchasing wool to knit a football scarf, so I used the tried and tested  “browsing the photos on eBay for something that looked like a ball of wool” method.

In my eyes, the colour of the red had to be red, proper red. So I was careful to inspect any potential balls carefully for a vibrant look about it.

Turns out, my balls were 100g of 100% Premium acrylic. This means nothing too me. However, it’s similar to cotton wool where as in if you try to chew it, it squeaks and will make you shudder worse than finger nails on a blackboard.

The design

When you’re stood drinking hot tea on a cold Winters day, watching a football team playing questionable football – you can’t fault “the classic” red a white striped scarf…with little frilly bits at each end (do these have a name? Winkles? Frillies? answers on a post card)

Mine won’t have the silly badge on though.

I’d like to clarify, I had no idea how to knit at this point. However, YouTube provided ample videos of youghurt-weaving hipsters explaining how it’s done…

Progress

It begins....

So far so good

Changing colour like a boss

Only another 500 hours to go! Yeah!

I hated the scarf by this stage. Hated. It.

3 Months later

…and apparently the frilly bits are called “ruffles” in case you were wondering.

Today I found myself having to add a screen shot to a pull request on Github.

What is the quickest way of doing this?

There are a few ways to do this fast, but the quickest I have found is as follows.

Change the default path for screenshots

By default on Mac OSX screen shots just get dumped on your desktop. Not much use to me, as usually the whole point of a screen shot is to send it to someone. Who takes a screen shot for themselves to look at? Duh.

Instead, I want all my screen shots with ⌘⇧F4 to go in my public Dropbox folder.

To change this use the following commands at your terminal;

$ defaults write com.apple.screencapture location ~/Dropbox/Public/
$ killall SystemUIServer

(the second command just reloads your profile so the changes take effect)

Now you can simply right click on the image, select “Copy public link” and paste in to Github

Copy public URL

Using markdown, you can add an embedded image as follow:

![alt text](http://url)

Adding image with markdown

The image embedded in the pull request

The problem

So you’ve got your nice form, and want to allow users to attach a file. If the user tries to upload a file which exceeds the ‘post_max_size’ value in php.ini then WHAMMY…the entire $_POST variable will be empty.

This is a massive headache in terms of UX, your user has just spent 5 minutes entering information and simply because they tried to attach a 4Tb file then all this is lost. Validating the size of the file they attach is not the issue, the issue is that if the upload is part of a form with other text fields then their input will be lost.

The solution

First off, lets get the current value for ‘post_max_size’ and ‘upload_max_filesize’ from php.ini.

upload_max_filesize

post_max_size

As you can see, my post_max_size is 20Mb and my upload_max_filesize is 10Mb.

Why are they different? Well, anything over 10Mb will upload but will return a decent error you can catch, but if a user tries to upload a file over 20Mb then the internet explodes and PHP simply dumps the whole lot, $_POST will be empty, as will $_FILES, $_REQUEST etc.

How this would look to the user uploading 15mb file

How this would look to the user uploading 15mb file

How this would look to the user uploading 25mb file

How this would look to the user uploading 25mb file

As you can see, if post_max_size is exceeded (the second screenshot), all the $_POST data is lost and the user has to start again.

One solution to this would be to have 2 separate forms, or split up the steps in the process so the file upload is on a separate page.

However there is a better solution: Ajax.

This approach uses two separate forms but we do it on the same page in such a way the UX is seamless.

1. When the user clicks “Save and Continue” we submit form 1 (just the text/select fields) via ajax as a serialized array
2. Validate it on the server and return a JSON array of info (such as: did it validate? if there are errors what are they? etc.)
3. If it does validate, save it in a session
4. Check the response client side, and if ok we submit form 2 (just the files) the usual way

Here’s how it looks.

The HTML for the forms

The JS (I’m using jQuery)

My code on the server validates the data from the text fields, saves it to a session, and returns a JSON object. If the validation fails it returns something like this.

We can use the response to show errors BEFORE the form with the files is submitted (the normal way).

Validation errors, used a modal because i'm hip

If the response in the JSON is ‘success’ then we simply submit the second form as normal. If the file/s still exceed the post_max_size then at least the previous text input is on the session and can be used to populate the fields again.

How this would look to the user uploading 50mb file

Huge upload, $_POST is lost but we have the text fields in the session

1. In my head, I know what I want a method to do before I even write it, so I write a test that will pass if the code (that I write in step 2) works. In this case it is a basic method in one of my Models.

My fixture has 5 jobs already, I expect the next job to have an ID of 6 and a cost of 18.00

2. To make the test pass, here is my REAL method that I add to my model….

This just saves the job, the passed arg is an array like the one in the test (or at least I hope it is!)

3. Now lets run the test…

My test passes - my code does what I had in my head (saves a job)

4. Some other guy on the team changes some code…

Someone else on the team makes some weird change, probably high on meow

5. Looking at the change, it won’t work. The passed arg IS an array, so it will throw an exception. But my original test will catch the fact that this method does not work anymore, otherwise this bug could have gone unnoticed.

Before going live with these changes we would run all the tests, in this case, the app doesn't work!

From the failed test, you can even see which line of code is failing.

Obviously if you have not written a test for something it can not fail, so being thorough and making sure any bit of functionality has been tested its easy to catch problems and fix bugs.

And if you’re using Git, you can even see who broke the project. Then take it in turns to beat the developer around the head with a blunt instrument.

The worlds most basic summary

Ask yourself “The code I am about to write, what should it do/not do” – then write a test that covers any conceivable eventualities FIRST. Then write the real code to make the test pass.

Info-graphics are all the rage. So I did very complicated one which demonstrates how the average web project  progresses over time.

...

Great presentation from Paul Tuckfiled, database admin at YouTube.

Its from the MySQL conference from a few years back so a bit dated, but much of the info is still relevant (and his presentation style is pretty funny!)

I have been playing with Rackspace “cloud” servers for a few weeks now. They’re easy to fire up and shut down, and you only pay for the time the servers are online, so they are great for spinning up a development server for a few hours.

I’ve put together the following as my own selfish reference to setting up a working CentOS LAMP server.

This post assumes you have knowledge of using SSH and have some idea of what you are trying to achieve which in this case is a working LAMP web server.

But it also involves managing your own stuff. Updates, checking logs, making sure your websites are up and running, fixing issues etc. will now be your responsibility.

If the sound of all that makes you dizzy, then you might want to consider a fully managed web hosting service from a reputable hosting company instead who will do all that boring stuff for you.

Still with me?

Good. Lets fire up a server. Remember to give your server a cool name, this will help gain you the respect of more seasoned server admins. If you own a retro Transformers T-shirt, put it on now.

Create a server

When you create a server on rackspace, you will be issued with the important basics. The unique IP address, and your root username and password. So fire up a terminal and get your eyes dirty. For the purpose of this post, I will be using the server “clapton” (cool, huh?) and the IP address is 46.38.165.175, I’m logged in with a clean install of CentOS  5.5.

[root@clapton ~]#

Yum is your friend

You know that cool retro Tansformers T-shirt you’re wearing? Well Yum is even cooler.

First up, lets update any existing packages that have been included in your CentOS install by using the “update” option

yum update

The installed packages are now up to date, now lets move on to the stuff we want. I want to install the following;

• Apache
• MySQL
• PHP
• Open SSL
• Sendmail
• Git
• VIM

I’m not going to bother with FTP (real men use Git) and I want to keep things lightweight and locked down as much as possible. Having FTP running just means faffing with IP lists in an attempt to stop 7 million Chinese teenagers pwning your FTP login.

Open SSL comes included with CentOS, we just need the apache mod_ssl. I will write another post regarding setting up SSL certs, as this was my one stumbling block when I first started. I’d previously been spoiled by admin panels like Plesk when it came to things like that.

Core settings

There isn’t really much house work to take care of, other than checking the servers HOSTS file, located at /etc/hosts;

127.0.0.1    localhost    localhost.localdomain
46.38.165.175    clapton    46.38.165.175.static.cloud-ips.co.uk

In the Rackspace control panel, if you view the DNS for the server you will see a xxxx.cloud.co.uk record, add this (along with the server name if not already there) to the IP in your HOSTS file.

I also add a localhost.localdomain entry, this will make life easy for things like sendmail.

VIM

The rest of the setup may require editing files, be nice to your eyes and use VIM.

yum install vim-enhanced

Apache

Lets get Apache up and running. We want SSL support too so we’ll also install the apache mod now (OpenSSL should be installed by default as mentioned above, we just need the mod itself)

yum install httpd mod_ssl

Fire up VIM and edit the apache conf file, we need to tell apache to “listen” on port 80, its around line 135 of httpd.conf;

vim /etc/httpd/conf/httpd.conf

httpd conf listen port 80

Now Apache is listening to port 80, we now need to open this port as CentOS by default will have every port locked down for obvious reasons.

To open port 80 (http) and 443 (https) we add a rule for each to the iptables config;

iptables -I INPUT 1 -p tcp --dport 80 -j ACCEPT
iptables -I INPUT 1 -p tcp --dport 443 -j ACCEPT

and then commit these new rules;

/etc/init.d/iptables save

That’s it, fire up apache;

service httpd start

and go to your IP address in a browser;

apache default page

PHP

Use Yum to install PHP, the MySQL extension, and MySQL itself in one fell swoop;

yum install php php-mysql mysql-server

As of writing this, the latest PHP version that Yum will use is 5.1.x, which lets be fair, is pretty outdated.

5.2.3+ is fine for me, so I’ll go with that. If you want to go big and have 5.3+ (but have never installed manually before) then I would suggest get going with 5.2.x to start with, then upgrade to 5.3 later on once you have a basic web server up and running.

To update to 5.2+ we need to add a .repo file which instructs Yum to do some kung foo for us when finding which packages we want.

We then just update PHP using Yum.

So first, add the .repo file as follows;

touch /etc/yum.repos.d/CentOS-Testing.repo

Then open the empty file;

vim /etc/yum.repos.d/CentOS-Testing.repo

Then add the following config to the repo file you just created;

# CentOS-Testing:
# !!!! CAUTION !!!!
# This repository is a proving grounds for packages on their way to CentOSPlus and CentOS Extras.
# They may or may not replace core CentOS packages, and are not guaranteed to function properly.
# These packages build and install, but are waiting for feedback from testers as to
# functionality and stability. Packages in this repository will come and go during the
# development period, so it should not be left enabled or used on production systems without due
# consideration.
[c5-testing]
name=CentOS-5 Testing
baseurl=http://dev.centos.org/centos/$releasever/testing/$basearch/
enabled=1
gpgcheck=1
gpgkey=http://dev.centos.org/centos/RPM-GPG-KEY-CentOS-testing
includepkgs=php*

Now update PHP as normal with Yum;

yum update php

Restart apache

service httpd restart

MySQL users !IMPORTANT

MySQL is now installed, but comes with a default user….with NO PASSWORD. So we need to start MySQL, drop the users with no passwords, and create a user with a password;

service mysqld start

Log in as the default password-less root user;

mysql -u root

Set a password for your root user;

SET PASSWORD FOR root@localhost = PASSWORD('newpassword');

and

SET PASSWORD FOR root@127.0.0.1 = PASSWORD('newpassword');

Drop the ‘any’ user too

DROP USER ''@localhost;

All done. Exit MySQL;

exit;

Email (Sendmail)

yum install sendmail sendmail-cf

service sendmail start

Done

Services

At this stage we have 3 main services installed; Apache, MySQL and Sendmail.

But these are currently manually started, which means if you had to reboot your server, you would also have to also manually start each service again, which is pants.

To set all 3 to start by default use the chkconfig command;

chkconfig httpd on

chkconfig mysqld on

chkconfig sendmail on

You can also use the –list parameter to see the changes and view any other default services and their run levels.

chkconfig --list

Git

Git is cooler than both Yum and your Transformers T-shirt combined.

To install Git we can employ the EPEL (Extra Packages for Linux) RPM;

rpm -Uvh http://download.fedora.redhat.com/pub/epel/5/i386/
epel-release-5-4.noarch.rpm

We can now Yum install;

yum install git

git --version

Should display;

git version 1.5.5.x

Great Success

That’s it. You now have a LAMP stack on Rackspace. A great tool in the Rackspace Panel is “My Server Images”. Now would be a good time to create an image of what you have done so far, then you can roll this out on other servers really quickly in future.

But don’t be a dingbat and loose all your shit, image or no image. Remote file storage is seriously cheap, see this post on Remote Backups to Amazon S3 and save yourself a huge headache one day

Reddit is a big site (in terms of traffic and data storage)  built by some guys in the US, one of their former developers talks openly about some of the issues they have had to overcome.

Lessons Learned while at Reddit from Carsonified on Vimeo.