Locally Caching API Backed Rails Models

So I was working on a project that had some fairly high coupling between our system and the data we got from an API. Then someone wanted a dashboard to summarize a bunch of stuff and instead of building some “cache models” in some database I decided to just cache the attributes the API sent us using standard Rails cache. What caching engine you use is up to you, but this will be faster if held in a memory based cache (duh).

Since this was at least 3 models being fed by the API I figured lets use a Concern! Check it out:

Since we had to define the class level find for our API Backed models anyway, we decided to couple that find method with the write_local_cache method

So to grab a model from the cache I simply had to do

ModelName.local id

in the controllers feeding this dashboard. Alternatively you can do this in relationship methods on a model.

So this ended up taking us from a 65 second page load that result in 400 API requests to about a 1.5 second page load with 0 API requests. And the business was fine with this data being anywhere from 1-12 hours out of date. And since this cache is so long lived I wrote a rake task to preheat the cache that I run every day at 6am before the app starts getting used heavily:

namespace :app_name do
  desc "Pre-heats the cache based off current work assignments"
  task preheat_cache: :environment do
    Assignment.not_worked.find_each do |assignment|
      customer = Customer.find assignment.customer_id
      customer.total_due # this preheats a relationship to another API backed model
    end
  end
end

Setting up Mercurial on CentOS x86_64

Mercurial is a badass distributed version control system. Most likely, you know what mercurial is if you’re reading this, so I won’t bother explaining it. If you don’t know what it is, go to hginit.com.

Step 1. Get the RPM of Mercurial for Your Version
I got my package from packages.sw.be/mercurial/. Make sure to choose the right version. I chose mercurial-1.8.2-1.e15.rf.x86_64.rpm, but there may be more up to date versions or more appropriate versions for you.

[root@two twohlix.com]# cd /home
[root@two home]# wget http://packages.sw.be/mercurial/mercurial-1.8.2-1.el5.rf.x86_64.rpm
--2011-05-11 22:17:43--  http://packages.sw.be/mercurial/mercurial-1.8.2-1.el5.rf.x86_64.rpm
Resolving packages.sw.be... 85.13.226.40
Connecting to packages.sw.be|85.13.226.40|:80... connected.
...

Step 2. Install the RPM

[root@two home]# rpm -Uvh mercurial-1.8.2-1.el5.rf.x86_64.rpm
warning: mercurial-1.8.2-1.el5.rf.x86_64.rpm: Header V3 DSA signature: NOKEY, key ID 6b8d79e6
Preparing...                ########################################### [100%]
   1:mercurial              ########################################### [100%]

I used a few options in the install: ‘-U’ upgrades or installs (whichever it has to) the rpm. ‘-v’ makes it verbose so you can see it do things. ‘-h’ adds the #(h for hash) marks so you can see the progress over time.
If you didn’t use your root account for this be sure to use sudo:

[totally_not_root@two home]# sudo rpm -Uvh mercurial-1.8.2-1.el5.rf.x86_64.rpm

Step 3. Check on Mercurial

[root@two home]# hg

that command should result in:

Mercurial Distributed SCM

basic commands:

 add        add the specified files on the next commit
 annotate   show changeset information by line for each file
...

If that happened, you’re all done. You may need to install python if it broke.

Setting up Apache Virtual Hosts on CentOS

So you’ve got a new VPS host from HostGator. You want to host yours and your 5,000 closest friends’ sites too: but in order to do that you have to setup your apache server’s virtual hosts.

Step 1. Make a Directory to contain your Virtual Host files

[root@two twohlix.com]#
[root@two twohlix.com]# cd /etc/httpd/
[root@two httpd]# mkdir sites-available
[root@two httpd]# mkdir sites-enabled

Now that you’ve created ‘sites-enabled’ and ‘sites-available’, we’re going to make Apache check any of the conf files you have in ‘sites-enabled’.

Step 2. Tell Apache to Look for your Conf Files

[root@two httpd]# vim conf/httpd.conf

Add these lines to the end of your ‘httpd.conf’ file:

NameVirtualHost *:80
Include /etc/httpd/sites-enabled/

Step 3. Create Some Configuration Files for your Sites

[root@two httpd]# cd sites-available/
[root@two sites-available]# cat > twohlix.com.vhost.conf
#
#   twohlix.com (/etc/httpd/sites-available/twohlix.com)
#
<VirtualHost *:80>
        ServerAdmin admin@twohlix.com
        ServerName  twohlix.com
        ServerAlias www.twohlix.com *.twohlix.com

        #Indexes + Directory Root
        DocumentRoot /home/csmith/www/twohlix.com/htdocs/
        <Directory "/home/csmith/www/twohlix.com/htdocs">
                Options Indexes FollowSymLinks MultiViews
                AllowOverride All
                Order allow,deny
                Allow from all
        </Directory>

        #LOG FILES
        ErrorLog  /home/csmith/www/twohlix.com/logs/error.log
        CustomLog /home/csmith/www/twohlix.com/logs/access.log combined
</VirtualHost>

Change the DocumentRoot to wherever you want to host your website for example ‘/opt/development/username/www.yoursite.com/html/’ and make sure that directory exists (user mkdir).
Make sure your log’s directory exists also.

Step 4. Symlink the Conf Files To ‘sites-enabled’

[root@two sites-available]# ln -s /etc/httpd/sites-available/twohlix.com.vhost.conf /etc/httpd/sites-enabled/twohlix.com.vhost.conf

I symlink my sites into ‘sites-enabled’ because that way its easy to turn on and off sites (for development) without having to make/remake VHost files. Just ‘rm’ the file from sites-enabled if you want apache to stop serving it.

Step 5. Restart Apache

[root@two sites-available]# /etc/init.d/httpd restart

OR

[root@two sites-available]# /etc/init.d/httpd graceful

That should work. Just place all your servable documents (or symlink things) into the DocumentRoot you specified. Voila…i hope.
I’ll throw a script to make this easy up on Github.