Interesting Bug in Django

I am working on a project that involves Django and ran into an interesting issue. Django creates a small database to keep track of various bits of data one of which is user session information in a table called django_session as follows:

CREATE TABLE django_session (

  session_key varchar(40) NOT NULL,

  session_data longtext NOT NULL,

  expire_date datetime NOT NULL,

  PRIMARY KEY (session_key)

) ENGINE=InnoDB DEFAULT CHARSET=utf8;

This is all well and good but there is an issue. InnoDB orders the rows in primary key order (primary keys are SHA1 hex digest). The problem is that these keys are effectively random so a new session row be be inserted anywhere in the table causing data to move around with every insert. While this might work when the table is small, it does not work so well when you have 500,000+ rows in it (which is another issue that I will get to).

A better schema for the table is as follows:

CREATE TABLE django_session (

  id int(11) NOT NULL AUTO_INCREMENT,

  session_key varchar(40) NOT NULL,

  session_data longtext NOT NULL,

  expire_date datetime NOT NULL,

  PRIMARY KEY (id),  

  UNIQUE KEY session_key (session_key)

) ENGINE=InnoDB DEFAULT CHARSET=utf8;

This will ensure that rows are inserted consecutively which will ensure better performance as the table grows.

Two things to note:

I am not sure whether Django specifies the ENGINE to use when creating these tables, but MySQL 5.5 uses InnoDB rather than MyISAM, and I don’t think this will be an issue with the latter.

The other thing is that Django does not seem to clear out sessions past their expiry date, so one needs to do that regularly with the following statement:

DELETE FROM django_session WHERE expire_date <= NOW()

One more thing, I think that is the case too with database backed caches too.

 

How to speed up an aging MacBook with a solid state drive

About 3 months ago I upgraded the disk drive in my MacBook Pro with an SSD, the interesting thing was how easy it was to do the upgrade with the help of iFixit. Ars does the same to a MacBook:

How to speed up an aging MacBook with a solid state drive: “When we recently detailed how to boost the storage space in a MacBook Air with a replacement solid state drive module, some readers asked what it would be like to swap the hard drive in an older MacBook with a similarly speedy SSD. We decided to investigate, and as it turns out, thanks to a common 2.5″ drive size and widely available external enclosures, the swap is quicker, easier, and cheaper than the one for a MacBook Air.”

The upgrade was a real shot in the arm of my MacBook Pro in terms of performance, even though the previous disk drive was a 7,200rpm model.

 

BBEdit 10.0

BBEdit 10.0 came out last week, I have been a long time user of BBEdit (pretty much since it came out). This release comes with a lot of changes, a lot, and has required me to change some of my work processes. There are a number of rough edges too so it might be an idea to wait for the .1 release. And you will want to make sure not to upgrade unless you have some downtime to work through all the changes.

 

Mac OS X Lion

I upgraded to Mac OS X Lion over the weekend, things went mostly well, backup, install, repeat on the other machine.

There are a number of good reviews out there, notably John Siracusa’s at ArsTechnica, and Robert Mohns’ at MacInTouch.

Some things that are driving me crazy:

• The way the new scroll bars appear and disappear, I just make them permanent. I tried to make them automatic but they would not always appear and you lose the ability to see what is and is not scrollable.
• The Address Book and iCal are a train wreck.
• For some reason my folder/document locations keep getting reset.

That being said, I really like the new Mail, Mission Control is nice (though they have made it more difficult to see minimized windows).

Overall this version is optimized to run on laptops, not desktop machines, but you can turn off all the ‘laptopy’ things.

Lastly I found that John Siracura has a podcast called Hypercritical. Worth listening to.

 

 

 

Whale Tail

This is the tail end of a baby Humpback Whale who cruised past us, when I say cruised I mean mobbed us, three people go hit (there was no damage). I caught a picture of its’ tail which was about 5 feet wide.

You can see the front part of the whale here.

Digital Copy

Last week I got a Blu-Ray/DVD/Digital copy combo disc for a popular animated movie. I am (was) a fan of digital copies that I could transfer to my computer and watch on my Apple TV. Was a fan because the copy was really bad, all sorts of compression artifacts and blotchy colors. Turns out it is a 640×480 version of the movie and the file was over 1GB in size suggesting the compression treatment on the file was terrible. Given how bad it was I am not sure why they include digital copies. Hard to see how it could look good on any device.

Note to the movie distributors (Disney), if you are going to include a digital copy you should make sure it looks good, what I saw was pretty insulting the people who created the movie.

IOGEAR USB Bluetooth adaptor

Just got my IOGEAR USB Bluetooth adaptor today, plugged it into my keyboard USB port as close to my Apple Magic Mouse as possible and it work GREAT! Finally bluetooth works on my MacPro!

 

Django and MySQL

It can be a little challenging to get Django to talk to MySQL especially if you have a non-standard setup (like I seem to have, every time…)

To access MySQL from Django you need to install MySQL-python, and this is usually where the trouble begins. MySQL-python will run ‘mysql_config’ to determine what the default MySQL settings are, the one to pay attention to is the ‘–socket’ one. This is the socket that MySQL-python will use to access MySQL if the DATABASE/HOST setting in the Django settings.py file is left empty or set to ‘localhost’. MySQL-python appears to disregard any config setting in /etc/my.cnf, so if you set the ‘socket’ setting in /etc/my.cnf to something else (such as ‘/var/lib/mysql/mysql.sock’) then Django will NOT be able to access the MySQL server.

There are two solutions to this. Either you set the ‘socket’ setting in the /etc/my.cnf file to match the setting reported by ‘mysql_config’. Or you set HOST in the Django settings.py file to the host name of the machine running MySQL.

One thing that I have noticed is that MySQL installations that are made through apt-get or yum have the socket default set to ’/tmp/mysql.sock’ whereas MySQL installations that are made from the MySQL download have the socket default set to ‘/var/lib/mysql/mysql.sock’.

Other issues you may run into is installing MySQL in a non-standard directory, for example ‘/usr/local/mysql’, while MySQL-python will probable install correctly, it may not be able to pull in the MySQL libraries when running under the Apache server.

What I generally do is let MySQL-python install however it wants, and set the HOST to the hostname of the machine where MySQL is running.

Apple Magic Mouse

Finally got sick of erratic mouse movement with my Apple Magic Mouse. I have a MacPro and the bluetooth reception on that machine is notorious in a bad way. A stretch of humid weather here only made that worse. The mouse movement was either epileptic or really slow (so felt like I was dragging a huge weight across the screen), and lets not go into dropped clicks.

So came across this post on Cult of Mac, which pointed me to this discussion on the Apple Support Forums, and decided to just buy an IOGEAR USB Bluetooth adaptor, which I will put into the USB port closest to the mouse which, I am hoping, will take care of the problem.

I think they called it Magic Mouse because you never knew what it was going to do next!

Eye Contact

I had the opportunity to get on a boat going to the Silver Banks which are located north of the Dominican Republic. There was a group cancellation and I jumped on (there is usually a two year waiting list for this trip.)

Humpback whales migrate from the North Atlantic down to the Silver Banks to raise their young for three months at the start of the year. So there is an opportunity to get in the water and swim with them.

This is a young calf (still 12 feet long) who came right up to me to check me out.