Repurposing netbook into gateway, firewall, AP

March 20th, 2013 No comments

The older netbooks, such as ASUS EEE 900 series, have gone down in price. I bought a refurbed EEEPC 900A w/1GB RAM, 4GB SSD and turned it into my gateway, firewall and local AP.  Upgraded memory to 2GB RAM, and added an external 8GB SD flash.

I loaded Fedora Core 17 Security Distro on it.  Now I have a nice, cheap gateway, firewall and AP (although only 802.11G speed).

I’ll add more to this as I have time…..

Unfortunately the netbook only have one wired 100M enet.  I need at least GigE interfaces.  I am currently looking at the

OEM Production 2550L2D-MxPC

which is an Atom D2550 based system with 2 Broadcom GigE, small, lower power and up to 8GB of RAM.  This should last me a long time as a router/gw/fw.

More to come….

 

File transfer speed over ssh

March 5th, 2013 No comments

I’ve known that ssh encryption has an effect on the speed of file xfers. So doing thing such as rsync (which will use ssh) or even plain scp can be pretty darn slow, especially on large files and on system with old/slow CPU.

I also know about the recommendation to use different type of encryption when transferring files. Some people recommend blowfish, others arcfour. So I thought I’d do a little bit of testing in a controlled environment.

I have two recent vintage HP servers with the following specs.

HP ProLiant DL360p Gen8
Dual quad core Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz (8 core, 16 threads total)
64G RAM
4 x 3TB, mdadm RAID10, formatted as XFS, mounted noatime,logbufs=8
Tigon ethernet NIC, connected as GigE, full duplex to HP ProCurve 2848 switch
(both servers connected to same switch)

The test file is:
3921247501 Mar 4 08:22 bigdata.tar.bz2

I am using OpenSSH_5.3p1, OpenSSL 1.0.0-fips 29 Mar 2010
Kernel is 3.8.1-1.el6.elrepo.x86_64 #1 SMP Thu Feb 28 19:15:22 EST 2013 x86_64 x86_64 x86_64 GNU/Linux

I am going to copy this file from hp1 to hp2, using scp, rsync and ftp. With scp, I’ll try different encryption, no compression to see how the different encryption affect the transfers. For comparison purposes, I also timed using plain ole FTP transfer, which mean no encryption and very little system processing; and the timing proves that.  Also tested with plain rsync protocol (direct to rsyncd).

I run this 3 times. Without specifying encryption, ssh/scp will use the default, which depends on the version of OpenSSH (for this version, the default is aes128-ctr).

run Xfer type real user system
1 scp -o Compression=no 0m52.175s 0m12.709s 0m6.504s
2 scp -o Compression=no 0m47.872s 0m12.603s 0m6.806s
3 scp -o Compression=no 0m49.317s 0m12.748s 0m6.710s
1 scp -c arcfour -o Compression=no 0m49.536s 0m14.161s 0m6.903s
2 scp -c arcfour -o Compression=no 0m49.088s 0m14.045s 0m6.921s
3 scp -c arcfour -o Compression=no 0m50.698s 0m14.162s 0m6.728s
1 scp -c blowfish-cbc -o Compression=no 0m58.673s 0m44.295s 0m13.495s
2 scp -c blowfish-cbc -o Compression=no 0m56.399s 0m43.860s 0m9.036s
3 scp -c blowfish-cbc -o Compression=no 0m54.869s 0m43.949s 0m10.673s
1 scp -c aes128-cbc -o Compression=no 0m49.776s 0m14.641s 0m7.083s
2 scp -c aes128-cbc -o Compression=no 0m48.527s 0m15.154s 0m7.068s
3 scp -c aes128-cbc -o Compression=no 0m50.554s 0m15.334s 0m6.983s
1 ncftpput -m -u ftptest -p ‘XXXXXX’ hp2 /data/ /data/bigdata.tar.bz2 0m34.306s 0m0.141s 0m4.062s
2 ncftpput -m -u ftptest -p ‘XXXXXX’ hp2 /data/ /data/bigdata.tar.bz2 0m33.351s 0m0.160s 0m3.863s
3 ncftpput -m -u ftptest -p ‘XXXXXX’ hp2 /data/ /data/bigdata.tar.bz2 0m33.839s 0m0.154s 0m3.732s
1 rsync –stats -a /data/bigdata.tar.bz2 hp2::data/bigdata.tar.bz2.1 0m33.485s 0m10.221s 0m6.692s
2 rsync –stats -a /data/bigdata.tar.bz2 hp2::data/bigdata.tar.bz2.2 0m33.490s 0m10.234s 0m6.703s
3 rsync –stats -a /data/bigdata.tar.bz2 hp2::data/bigdata.tar.bz2.3 0m33.497s 0m10.163s 0m6.545s

In terms of speed, we have:

Average over 3 runs

RSYNC:         real=33.491  user=10.206  sys=6.6467
FTP:           real=33.832  user=0.1517  sys=3.8857
AES128-CBC:    real=49.619  user=15.043  sys=7.0447
ARCFOUR:       real=49.774  user=14.1226 sys=6.8507
AES128-CTR:    real=49.788  user=12.687  sys=6.6734
BLOWFISH-CBC:  real=56.647  user=44.0347 sys=11.068

So it look like in modern OpenSSH, using AES, it’s a wash which cipher/encryption method you want to use.

Note that rsync protocol itself is pretty darn efficient, slightly faster than FTP.

3/6/13 Update

AES in SSH.  I’ve tested again from an old Dell using Pentium 4 to the fast HP, with no AES support in hardware and the default AES128-CTR is much slower.  However, good news is that AES128-CBC is still faster than BLOWFISH, but slightly slower than ARCFOUR.  As for FTP and RSYNC, they are neck-and-neck in speed, no clear winner.

So my conclusion is that whether using AES with hardware support (in new Intel CPUs) or software, using the CBC (block mode) variant of AES is usually good enough.

 

 

iSCSI initiator (netbsd-iscsi-initiator) for OS X Mountain Lion (10.8.2)

January 13th, 2013 4 comments

I am playing around with iSCSI for my macbook pro. Looked around and the used to be free SNS globalSAN iSCSI is no longer free. ATO is too expensive to play with. Saw that macport has a netbsd-iscsi, so went that route.


$ sudo port install netbsd-iscsi-initiator
---> Computing dependencies for netbsd-iscsi-initiator
---> Dependencies to be installed: netbsd-iscsi-lib
---> Building netbsd-iscsi-lib
Error: org.macports.build for port netbsd-iscsi-lib returned: command execution failed
Error: Failed to install netbsd-iscsi-lib
Please see the log file for port netbsd-iscsi-lib for details:
/opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_devel_netbsd-iscsi-lib/netbsd-iscsi-lib/main.log
Error: The following dependencies were not installed: netbsd-iscsi-lib
To report a bug, follow the instructions in the guide:

http://guide.macports.org/#project.tickets

Error: Processing of port netbsd-iscsi-initiator failed

Poking in main.log show the error at compiling disk.c.


:info:build /bin/sh ../../libtool --tag=CC --mode=compile /usr/bin/clang -DHAVE_CONFIG_H -I. -I../../include -I../../include -I/opt/local/include -pipe -O2 -arch x86_64 -MT libiscsi_la-disk.lo -MD -MP -MF .deps/libiscsi_la-disk.Tpo -c -o libiscsi_la-disk.lo `test -f 'disk.c' || echo './'`disk.c
:info:build /usr/bin/clang -DHAVE_CONFIG_H -I. -I../../include -I../../include -I/opt/local/include -pipe -O2 -arch x86_64 -MT libiscsi_la-disk.lo -MD -MP -MF .deps/libiscsi_la-disk.Tpo -c disk.c -fno-common -DPIC -o .libs/libiscsi_la-disk.o
:info:build disk.c:811:40: error: assignment to cast is illegal, lvalue casts are not supported
:info:build *((uint64_t *) ((void *)data + 8)) = (uint64_t) ISCSI_HTONLL(key);
:info:build ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
:info:build 1 error generated.
:info:build make: *** [libiscsi_la-disk.lo] Error 1

So I patched that line.


- *((uint64_t *) (void *)data + 8) = (uint64_t) ISCSI_HTONLL(key);
+ *((uint64_t *) ((void *)data + 8)) = (uint64_t) (ISCSI_HTONLL(key));

and now it builds. I just got to play with this to see if it works. More to report later.

Categories: iSCSI, NAS, Tech Tags: ,

Single DHCP server for multiple subnets (VLANs) one single interface

January 9th, 2013 1 comment

Surprisingly this was an extremely hard to find piece of information on the topic. At least one that fit my need. There were lots of questions in various online posts, but no completely working answers with all the relevant details in one place.

I am going to document it here.

Aggregation router is a pair of Cisco 6506E in VSS mode, active-active. They have ip helper pointing to my DHCP server.

Multiple VLANs and subnets

There was a pretty useful post about single DHCP server, multiple subnets on one interface here. But this does not work for my situation. He’s using a fairly simple network, and the DHCP server run on the gateway, whereas my network is different.

I have a gateway/router that aggregates multiple VLANs, one of which is a management VLAN that my DHCP server sits on. All the other VLANs has the DHCP relay helper address pointing to my DHCP server (see graph above).

Using the “shared-network” statement in dhcpd.conf does not work as that will pool all of the subnet declaration inside that. This is why the blog post uses the classes along with “match if” statements to put DHCP client requests into the correct subnets. I have anywhere from 200 to 300+ servers in each VLAN, and they are a mix of gears/vendors. So there is no way that I can use hardware (MAC) address, without it getting very complicated.

ISC DHCP actually supports what I wanted out of the box. The trick was to make all the subnet declaration, but don’t use the “shared-network” statement. Make sure the DHCP relay are setup correctly, and when client make DHCP requests, they will arrive at the DHCP with the relay address in it as the GIADDR (gateway IP address). The DHCP will see that and know which subnet it should provide addresses from.

Here is the dhcpd.conf portion of the working config. I also do PXE and kickstart boot from this dhcpd server.


authoritative;

# this is the most important line. It specifies the method
# to use to connect to the DNS server and update it.
ddns-update-style none;
ddns-domainname "example.com";
ignore client-updates;
option host-name = config-option server.ddns-hostname;

include "/etc/rndc.key";

option domain-name              "example.com";
option domain-name-servers      10.1.14.10,10.1.14.11,10.1.14.12;
option time-offset              -18000; # Pacific Standard Time
option ntp-servers              10.1.14.11;
one-lease-per-client off;
default-lease-time 86400;
max-lease-time 604800;
option ip-forwarding off;

# PXE
next-server install;
filename "/linux-install/pxelinux.0";

# Subnet for internal hosts
    subnet 10.1.0.0 netmask 255.255.254.0 {
        range 10.1.1.200 10.1.1.253;
        option routers                  10.1.0.1;
        option subnet-mask              255.255.254.0;
        #failover peer "dhcp";
    }

    subnet 10.1.2.0 netmask 255.255.254.0 {
        range 10.1.3.200 10.1.3.253;
        option routers                  10.1.2.1;
        option subnet-mask              255.255.254.0;
        #failover peer "dhcp";
    }

    subnet 10.1.4.0 netmask 255.255.254.0 {
        range 10.1.5.200 10.1.5.253;
        option routers                  10.1.4.1;
        option subnet-mask              255.255.254.0;
        #failover peer "dhcp";
    }

....and so on....

Categories: Networking, Tech Tags:

Using ldap and kerberos with ajaxplorer

November 16th, 2012 No comments

12/18/12 Update: not all is peachy keen. Login and autocreate account works, but logout can be an issue. I need to clear the session cookie when someone logout. Have not gotten around to coding that yet.

After a bit of fiddling around, I finally got ajaxplorer working with (ldap) kerberos5 as the backend authentication/access.

We are using ldap for users directory and kerberos5 for password. It’s a little bit different than what I am used to.

Anyway, I needed to get ajaxplorer working on a large filer for users to be able to access — locally and remotely — essentially our private ‘dropbox’. But getting ajaxplorer working with kerberos was a bitch! At first, I tried using ldap, got that working…. except ldap does not have our password, that’s where kerberos comes in. I thought about writing my own plugin, but damn it, I don’t have time for this.

After lots of googling, experimenting, etc. I found mod_auth_pam, which uses pam for basic HTTP auth. And since we are already using pam_krb5 for logins on our boxes, it’s a perfect solution.

Here is the section in my bootstrap_plugins.php:

$PLUGINS = array(
        "CONF_DRIVER" => array(
                "NAME"          => "serial",
                "OPTIONS"       => array(
                        "REPOSITORIES_FILEPATH" => "AJXP_DATA_PATH/plugins/conf.serial/repo.ser",
                        "ROLES_FILEPATH"        => "AJXP_DATA_PATH/plugins/auth.serial/roles.ser",
                        "USERS_DIRPATH"         => "AJXP_DATA_PATH/plugins/auth.serial",
                        "FAST_CHECKS"           => false,
                        "CUSTOM_DATA"           => array(
                                        "email" => "Email",
                                        "country" => "Country"
                                )
                        )
        ),
        "AUTH_DRIVER" => array(
                "NAME"          => "basic_http",
                "OPTIONS"       => array(
                        "USERS_FILEPATH" => "AJXP_DATA_PATH/plugins/auth.pam/users.ser",
                        "AUTOCREATE_AJXPUSER"   => true,
                        "TRANSMIT_CLEAR_PASS"   => false
                )
        ),
        array(
                "NAME"          => "serial",
                "OPTIONS"       => array(
                        "LOGIN_REDIRECT"        => false,
                        "USERS_FILEPATH"        => "AJXP_DATA_PATH/plugins/auth.serial/users.ser",
                        "AUTOCREATE_AJXPUSER"   => false,
                        "FAST_CHECKS"           => false,
                        "TRANSMIT_CLEAR_PASS"   => false
                )
        ),
        "LOG_DRIVER" => array(
                "NAME" => "text",
                "OPTIONS" => array(
                        "LOG_PATH" => (defined("AJXP_FORCE_LOGPATH")?AJXP_FORCE_LOGPATH:"AJXP_INSTALL_PATH/data/logs/"),
                        "LOG_FILE_NAME" => 'log_' . date('m-d-y') . '.txt',
                        "LOG_CHMOD" => 0770
                )
        )
);

And the section in my /etc/httpd/conf.d/ajaxplorer.conf file:

   < Directory "/usr/share/ajaxplorer">
        Options FollowSymLinks
        AllowOverride Limit FileInfo
        Order allow,deny
        Allow from all
        AuthName "Ajaxplorer Access"
        AuthType Basic
        AuthPAM_Enabled on
        Require valid-user
  	php_value error_reporting 2
   < /Directory>

The trick is these two lines for the “basic_http” auth_driver:


"USERS_FILEPATH" => "AJXP_DATA_PATH/plugins/auth.pam/users.ser",
"AUTOCREATE_AJXPUSER" => true,

That then allow my users to login, and on first time, they auth via mod_auth_pam, and ajaxplorer create their account in “AJXP_DATA_PATH/plugins/auth.pam/users.ser”.

NOTE I have to manually create the directory plugins/auth.pam and create an empty users.ser file.

But after that, everything work perfectly.

Categories: SW Dev, Tech, web Tags: , ,

MongoDB and Riak

September 28th, 2012 No comments

12/18/12 UPDATE

Since I am the only DevOps working on this, and there are tons of other things requiring my attention, I had to drop riak. The engineers only know mongodb anyway, and they are reluctant to learn a new nosql (riak). Crap! So this project had been killed. Too bad.

I have some python scripts that I wrote to copy mongodb collections over to riak, if I have time, I’ll open source them.

======================

I’ve been working with MongoDB at current $WORK and previous jobs. It (used-to-be) is the nice, shiny toy that everyone rushed to. I’ve run into numerous limitation in trying to scale it up. Operationally, it can be a nightmare if the architecture was not setup correctly at the beginning.

Mongo is also a PITA to scale. There are major sites that have Mongos in the thousands, but at that point it become throwing hw and money at the problem. That just seem stupid for startups.

So at current place of $WORK, they are currently testing MongoDB, but I wanted to look for an alternative solution before we become fully committed to yet another operational nightmare.

After a lot of googling, testing, experimenting, etc. I decide on trying Riak from Basho.

Googling shows a number of companies migrated from MongoDB to Riak. Their experiences was useful, but I was looking more for concrete HOWTO to move large MongoDB over to Riak.

First, of course was to get hands-on experiences with Riak. Installed, play with it, etc. Then I used the riak-python-client lib to start migrating some data over. I wrote a script to work through all collections in a Mongo DB, for each collection, create a Riak bucket and add the Mongo doc to bucket using the Mongo _id as the key.

Right away, I run into some issues with Riak. I have a 3 nodes Riak (on 3 physical CentOS 5.8 servers). The MongoDB I was copying over was large, about 2GB on disk file size and over a million records. Partway through the conversion, 2 Riak nodes crashed and died…. WTF! No matter what I do, they wouldn’t start back up (Riak log shows some kind of erlang errors, but I don’t know erlang). So I stopped the only node running, ‘rm -rf /var/lib/riak/*’, ‘killall epmd’, restarted all 3 nodes and they came back up.

I don’t have time to debug this problem, so restarted conversion with a smaller subset of Mongo data. But this crash worries me. The erl_crash.dump shows Riak run into resource issues, unable to allocate heap memory. Hmmm.

More on my adventure in evaluating Riak vs MongoDB in the future.

Categories: NoSQL, Tech Tags: , ,

Toyota Rav4 recall

August 9th, 2012 No comments

11/16/2012 Update: Yep, I got letter from Toyota that my 2010 RAV4 is part of the recall. Crap!

My brother recently alerted me to this. Evidently the Rav4 rear suspension arm might come loose if the nuts holding it together are not tightened correctly.

Here is link to Toyota’s announcement about it.

Check VIN here to see if your Rav4 is part of recall.

August 01, 2012
Toyota Announces Recall of Certain RAV4 and Lexus HS 250h Vehicles

TORRANCE, Calif., August 1, 2012 — Toyota Motor Sales (TMS), U.S.A., Inc., today announced that it will conduct a voluntary safety recall involving 2006 to early 2011 model year RAV4 and 2010 model year Lexus HS 250h vehicles sold in the U.S. No other Toyota or Lexus vehicles are involved.

Approximately 760,000 Toyota RAV4
Approximately 18,000 Lexus HS 250h
Toyota has determined that if the nuts on the rear suspension arm are not tightened following the proper procedure and torque specification during a rear wheel alignment service, excessive play may occur at the threaded portion of the arm, followed by rust formation. If this were to occur, the threaded portion of the rear suspension arm may wear and cause the arm to separate.

Toyota is currently developing a remedy. Once the remedy is available, we will send a notification letter by first class mail advising owners to make an appointment with an authorized Toyota or Lexus dealer to have the remedy performed at no charge.

Detailed information and answers to questions are available to customers at www.toyota.com/recall and the Toyota Customer Experience Center (1 800-331-4331) or www.lexus.com/recall and Lexus Customer Satisfaction (1 800-255-3987).

Media Contacts:
Vehicle Safety and Quality Communications
Brian R. Lyons (310) 468-2552
John Hanson (310) 468-4718
Katy Soto (310) 468-8068
Toyota Motor Sales, USA, Inc. Media Line (310) 468-5297

Categories: Uncategorized Tags: , ,

dynamic robots.txt file in Rails 3.x

April 27th, 2012 No comments

We have a need for dynamic handling of robots.txt file as we have different requirements for production, staging, dev, test, etc.

Google-fu shows various way to do this, some for Rails 2.x, some for Rails 3.x. Here is my version.

First is to edit config/routes.rb and add this line:


match '/robots.txt' => RobotsGenerator

Then add the following to app_root/lib/classes/robots_generator.rb.

NOTE: We have an old domain, foo.com, that redirects to our newfoo.com. We don’t want foo.com to get indexed, so I have special treatment for that in production

class RobotsGenerator
  # Use the config/robots.txt in production.
  # Disallow everything for all other environments.
  def self.call(env)
    req = ActionDispatch::Request.new(env)
    headers = {}
    body = if Rails.env.production?
      if req.host.downcase =~ /foo.com$/
        headers = { 'X-Robots-Tag' => "noindex,nofollow" }
        "User-agent: *\nDisallow: /"
      else
        File.read Rails.root.join('config', 'robots.txt')
      end
    else
        "User-agent: *\nDisallow: /"
    end

    [200, headers, [body]]
  rescue Errno::ENOENT
    [404, {}, "User-agent: *\nDisallow: /"]
  end
end

Finally, you want to move public/robots.txt to config/robots.txt.

I want to give credits to the people that inspired my version.

Categories: Rails, Ruby, Tech Tags: , ,

My Latest brites on Britely site

April 25th, 2012 No comments

Did you know that you can create your very own Brites on Britely website?


This should always contain the most up-to-date list of all the brites I created and published on Britely.

Read "Funny Signs" by Tin Le at Britely!






Read "Dots Life" by Tin Le at Britely!



Categories: Humor, Tech Tags: , , ,

Bingham High School 30th reunion

April 4th, 2012 No comments

Showing my age….

Bingham High school 30 year reunion is coming up on Aug 4th, 2012 (that’s this year folks). It will be from 6pm-9pm MST at Gardner Village, West Jordan, UT.

There is also a Facebook group for this.

Categories: Uncategorized Tags: ,