Errors while installing unicorn gem on OS X

I was updating my bundle for a rails 3.1 app, when I run into the following errors.

Installing unicorn (4.2.0) with native extensions
Gem::Installer::ExtensionBuildError: ERROR: Failed to build gem native extension.

/opt/local/bin/ruby1.9 extconf.rb
checking for SIZEOF_OFF_T in ruby.h... *** extconf.rb failed ***
Could not create Makefile due to some reason, probably lack of
necessary libraries and/or headers. Check the mkmf.log file for more
details. You may need configuration options.

Provided configuration options:
--with-opt-dir
--with-opt-include
--without-opt-include=${opt-dir}/include
--with-opt-lib
--without-opt-lib=${opt-dir}/lib
--with-make-prog
--without-make-prog
--srcdir=.
--curdir
--ruby=/opt/local/bin/ruby1.9
/opt/local/lib/ruby1.9/1.9.1/mkmf.rb:381:in `try_do': The compiler failed to generate an executable file. (RuntimeError)
You have to install development tools first.
from /opt/local/lib/ruby1.9/1.9.1/mkmf.rb:491:in `block in try_compile'
from /opt/local/lib/ruby1.9/1.9.1/mkmf.rb:443:in `with_werror'
from /opt/local/lib/ruby1.9/1.9.1/mkmf.rb:491:in `try_compile'
from /opt/local/lib/ruby1.9/1.9.1/mkmf.rb:686:in `macro_defined?'
from /opt/local/lib/ruby1.9/1.9.1/mkmf.rb:822:in `block in have_macro'
from /opt/local/lib/ruby1.9/1.9.1/mkmf.rb:790:in `block in checking_for'
from /opt/local/lib/ruby1.9/1.9.1/mkmf.rb:284:in `block (2 levels) in postpone'
from /opt/local/lib/ruby1.9/1.9.1/mkmf.rb:254:in `open'
from /opt/local/lib/ruby1.9/1.9.1/mkmf.rb:284:in `block in postpone'
from /opt/local/lib/ruby1.9/1.9.1/mkmf.rb:254:in `open'
from /opt/local/lib/ruby1.9/1.9.1/mkmf.rb:280:in `postpone'
from /opt/local/lib/ruby1.9/1.9.1/mkmf.rb:789:in `checking_for'
from /opt/local/lib/ruby1.9/1.9.1/mkmf.rb:821:in `have_macro'
from extconf.rb:4:in `

'

Gem files will remain installed in /Users/tin/.bundler/tmp/34826/gems/unicorn-4.2.0 for inspection.
Results logged to /Users/tin/.bundler/tmp/34826/gems/unicorn-4.2.0/ext/unicorn_http/gem_make.out
An error occured while installing unicorn (4.2.0), and Bundler cannot continue.
Make sure that `gem install unicorn -v '4.2.0'` succeeds before bundling.

And the mkmf.log contain this, which was the clue I needed.

"/usr/bin/gcc-4.2 -o conftest -I/opt/local/include/ruby-1.9.1/x86_64-darwin10 -I/opt/local/include/ruby-1.9.1/ruby/backward -I/opt/local/include/ruby-1.9.1 -I. -I/opt/local/include -D_XOPEN_SOURCE -D_DARWIN_C_SOURCE -I/opt/local/include -pipe -O2 -arch x86_64 -fno-common conftest.c -L. -L/opt/local/lib -L/opt/local/lib -L. -L/opt/local/lib -arch x86_64 -L/usr/local/lib -lruby.1.9.1 -lpthread -ldl -lobjc "
i686-apple-darwin10-gcc-4.2.1: vfork: Operation timed out
checked program was:
/* begin */
1: #include "ruby.h"
2:
3: int main() {return 0;}
/* end */

Notice the vfork: Operation timed out line? It turns out that I was running out of system resources (VM) and gcc can’t fork. Ah, my poor 8GB i7 laptop…. 🙂

Fix was simple, I shutdown Firefox and Chrome w/their gadzillion windows and tabs opened. Then retried “bundle update” again, and voila!

Documenting it here in case other people run into similar problem.

Migrating from Bamboo to Cedar (calendon stack) on Heroku

We just migrated from Bamboo (bamboo-ree-1.8.7) to Cedar (calendon rails 3.1 stack) on Heroku. Here are the steps I took to make it work. Note that I didn’t do the code migration, my dev handle that, I do the server and infrastructure part.

Basically this mean moving from Rails 2.x (and ruby 1.8.7) to Rails 3.1 (ruby 1.9.2). Some of the changes require upgrading gem packages. Things such as images, css and js has to be put into an asset bundle.

On the site itself, I left the old site running, e.g. oldsite.heroku.com, and created a new site at newsite.herokuapp.com.

We created a new git dev branch and pushed to newsite, e.g.

git push heroku dev:master

Since we use SSL, I have to make sure custom_domains addon is there. But…

heroku addons:add custom_domains:basic custom_domains:wildcard

I can’t add the ssl addon until I am ready, because Heroku requires that I defined the domains for the app first! Chicken and egg, as it mean I have to take down the currently running production site.

So, make sure everything is running on new site first. Because the next steps mean production site will be down during the changes.

0. Make sure your SSL cert is up-to-date and you have both part, domain.cert and private.key. And most importantly, your key is passphrase-less!

1. Make sure your DNS records are updated and have shortest possible TTL. You are changing them.

2. Wait till DNS changes (TTL) have settle, could be a few hours. Then update your production site CNAME from *.yourdomain.com to newsite.herokuapp.com.

3. Clear out the domains in your old Heroku site:

heroku domains:clear --app oldsite

4. Add domains to new site:

heroku domains:add '*.yourdomain.com' --app newsite
heroku domains:add 'www.yourdomain.com' --app newsite

5. Upload SSL cert to your new app:

heroku ssl:add yourdomain.crt private.key

6. Now you can add the ssl addon.

heroku addons:add ssl:hostname

7. You should get an email from Heroku that has the hostname for your SSL DNS record to point your DNS to. The name should be something like this:

appid1234567herokucom-1234567890.us-east-1.elb.amazonaws.com

Update your DNS CNAME for www.yourdomain.com to point to this name.

8. Test, test, and test again your spanking new site. Add the New Relic addon if you haven’t (life saver!) and monitor traffic to your new site.

9. Monitor the log of your new site:

heroku logs -t --app newsite

10. Check the processes:

heroku ps --app newsite

11. Wait a week, if everything look good, you can take down the oldsite.

airpush spamming my android phone

I’ve started getting these spammy notifications to my notification bar (where else? :)). They appeared after I installed and updated several apps, but for the life of me, I could not remember which ones, since the notifications didn’t appear till a day later.

It turned out to be spams from a company called xapush.com. Googling found lots of complaints, and a possible solution. There is an app in the Market, called xapush detector. So I installed that app, and it told me there are two apps that is using xapush.

1. Overclock Widget – org.freecoder.widgets.overclock
2. First Aid – com.usa.health.ifitness.firstaid

Hmm, I don’t mind ads supporting free code, but this is over stepping the boundary. Ahhh, it turns out there is a new option in Advanced Settings for Overclock Widget, where you can turn off airpush. Yay!

Using any NAS as Time Machine

Yes, there are many posts on the web about this.  All the one my google-fu found has manual steps to set one up.  As an engineer, I like to automate things, why do the same steps over and over when you can write a script to package it all up for you?

So here is a shell script to setup sparsebundle ready for you to copy to your NAS and begin using as your Time Machine backup.  Save it to a file, I called mine: create-TimeMachine-bundle.sh.

Then run it with the size that you want to create for your backup.

E.g.  create-TimeMachine-bundle.sh 500g

will create a sparsebundle with size of 500GB.


#!/bin/bash

###################################################################################
# NOTE: even if you connect over WLAN, you MUST use en0 (ethernet) MAC address for
# TimeMachine sparsebundle!!!!

INTERFACE=en0
MACHINENAME=`uname -n | cut -d '.' -f 1,1`

if [ $# -lt 1 ]
then
echo "Usage: $0 SIZE"
echo " Where SIZE is total size (k|m|g|t) allowed for TimeMachine backup usage on NAS."
exit 1
fi
SIZE=$1

MAC_ADDRESS=`ifconfig ${INTERFACE}|grep 'ether '|awk '{print $2;}'|sed 's/://g'`

hdiutil create -verbose -type SPARSEBUNDLE -size $SIZE -fs "Case-sensitive Journaled HFS+" -nospotlight -imagekey sparse-band-size=262144 -volname "${MACHINENAME} Backups" ${MACHINENAME}_${MAC_ADDRESS}

defaults write com.apple.systempreferences TMShowUnsupportedNetworkVolumes 1

ReadyNAS PSU replacement

I had a heart stopping moment today. My SO told me that she “smelled” something funny in my office. I went in there and sure enough, smelled like burnt plastic…. I thought no problem, I have backups…. on my NAS’es (two of them).

Heh! Murphy’s Law and all that. One of them (ReadyNAS NV+) PSU was the source of the burnt plastic smell, ah fresh odor of melted circuits. The other one had (thankfully) shutdown because of the heat.

The ReadyNAS is my main NAS (4x2TB), the other one (4x1TB) is my media server, stuffs that I don’t mind losing because they are copied from the ReadyNAS.   Yes, in another word, bad backup scheme.  There is no backup for the ReadyNAS.  I’ll fix that soon enough now.

Anyway, firing up my laptop, manually set IP, DNS, etc. since my server NFS mounted most everything on the ReadyNAS…  google, google and ah ha!  Reports of failed PSU in ReadyNAS NV and NV+.  I am hoping that it’s just the PSU and nothing the main board.

http://www.readynas.com/forum/viewtopic.php?t=13492

Took my ReadyNAS apart, sniff test (smell around the various pieces) to isolate where it is coming from.   Seem to be the PSU.  Got dressed, run to Central Computer near by.  It’s closed, crap!  Head to Frys in Santa Clara…. it’s already 8:20pm on a Tuesday eve…. In luck!  Frys is still open.  Run to the parts aisle, lots of options, big PSU (500W, 650W, 1000W, 1400w….) I don’t need monster PSU, I just need something small and around 220W as that is the original NV+ PSU.

Saw a microATX PSU from Coolmax (300W) CM-300.  Look small, hope it’ll fit in the case.  Only 26 bucks (29.22 w/tax).  Grabbed it, run home.  Followed the wiki above to get the two yellow cables onto the connector.  Lucky for me, the molex connector already has two empty holes where the two yellow cables has to be inserted.  So I just had to find the yellow cables from one of the other connectors.  After some fiddling, got it.   Eventhough its a microATX PSU, it is still too big for the ReadyNAS, won’t fit in the bottom tray (arg!).  Oh well, thread the power cable through the original 3 prong power hole, plug the molex onto the board.

Leaving everything open, I plug the power cord in, then into the AC receptacle and press the power button on the ReadyNAS….. and IT TURNED ON!  Wheeee!  The LCD say “Booting up…. please wait.”   Pulled the plug, now I can put everything back in, put the screws on.  Taped the hole in the bottom tray since the new PSU is going to be sitting outside and I want things airtight for cooling.

My ReadyNAS is now going through its fsck, going to be a loooong time w/6.5TB fs.   But at least I am now back in business.

P.S. Actually, the dead PSU did cause screwed up fs.  I had to manually ssh into my ReadyNAS (thank god I had set up a root ssh login) and fix the MBR on drive 4 (/dev/hdi).  Turns out the simplest way was:

# dd if=/dev/hdg of=/dev/hdi bs=512 count=1