prerequisite concepts: prelude, basic configuration

This post is as much about customizing the root shell as it is about SSH environment variables, but I’m adding this to my OpenSSH collection because it’s applicable to any user.

I occasionally work on servers where, for a variety of reasons, I share an account with one or more other users; this is almost always suboptimal, but it does happen nonetheless. Over time I’ve grown somewhat partial to zShell, so one method I’ve used is to log in to a default shell, usually bash, then run zsh.

Even on server’s where I am the sole administrator, I usually don’t change the default shell– not so much because of days gone by when doing such a thing would break boot scripts & such, but because I try to practice the good habit of logging in as a normal user and using sudo for escalated privileges.

Eventually I struck upon the idea to have some code in the shell init script (e.g. $HOME/.bash_profile) switch to the shell of my choosing automatically as I log in. What I came up with looks something like this:

if [ -n "$CDFX_SHELL" ]; then
    tty -s && exec $CDFX_SHELL
fi

Briefly this code says, “if $CDFX_SHELL isn’t empty and the tty program says we’re connected to a terminal (on STDIN), then replace this shell by running the command in $CDFX_SHELL without creating a new process.”

Readers familiar with shell initialization may recognize potentially unnecessary checks in this example but this avoids having to delve into the differences between shell sessions which are interactive, login, both, or neither, as well as how this relates to scp and rsync. Also noteworthy are the checks which should be in the code before it’s used on a production server, such as verifying that $CDFX_SHELL specifies a valid shell. This isn’t intended to be cut-n-paste code.

Two steps are necessary for this to work: obviously $CDFX_SHELL must be set in the local environment for SSH to have anything to pass to the server, but less obviously the server must be configured to allow this variable to be set. This can be configured in the sshd config file (e.g. /etc/ssh/sshd_config) by adding this line:

AcceptEnv CDFX_SHELL

I prefer this method over those which require enabling PermitUserEnvironment because it’s less prone to unintended side-effects, as noted in the man page. In addition to (or in lieu of) using exec to switch shells, this method could also be used to set custom environments in the same shell, for different users or the same user, anytime it’s useful to have login behavior change based upon variables set in the SSH client.

Having discovered the advantages of á la carte VoIP pricing, I pondered how to extrapolate my experience for general discussion while avoiding the pitfalls of interpolation and abridgement. The Reference Book of Rates, Price Indices, and Household Expenditures for Telephone Service published by the FCC’s Wireline Competition Bureau provides a rough estimate of wireline telephone expenses averaging $45 per month in 2007, based on market research by TNS Telecoms. This isn’t too far from my own experience with residential VoIP plans which have tended to average about $35 monthly, including additional fees and charges, which can be significant: on BroadVoice’s “Unlimited World” plan, for example, “Taxes & Surcharges” account for about 35% of the monthly total. Based on these data, I use an estimated $35-$45 for generic comparison of monthly residential phone bills, or an average average of $40. As I designed our current, á la carte plan, I surmised that after discounting business use, the residential remainder was unlikely to ever exceed $30 in a single month. As the plan took shape, however, I realized that intelligent planning could lower that even further; somewhere in the neighborhood of a $20 monthly average would certainly exemplify what custom VoIP plans can offer, and half the average isn’t a bad talking point.

Though less obvious, another great feature of á la carte or “on demand” plans is scalability, if I suddenly find myself needing to host frequent call-in conference calls between a customer, their overseas manufacturing division, regional sales reps, and myself, the only change I’ll see on my invoices will be in usage. I am not aware of any “unlimited” residential plans which offer unlimited channels (simultaneous callers). With currently just three phone numbers, my setup is small enough, and with just enough complexity to provide a good example.

I use one number for my consulting, which has separate extensions, voice mail, etc.; I have a fax number for the luddite crowd, and a home number associated with a family voice mail, options for the caller to forward the call to my wife’s or my mobile phone, and a ring group which includes a line in my office. I’ll use an even usage split for comparison; for although Codefix Consulting has its own phone number, those who know me well tend to call my home number rather than risk my having a life outside of work.

My primary VoIP provider is VoiceMeUp.com and I have two DIDs (phone numbers) ($4.95 ea) and a prepaid, on-demand plan which bills 30/6 at $0.0185/min. My backup provider is CallWithUs.org who bill $0.0125 in whole minutes; while I hadn’t originally intended to acquire a DID through CallWithUs.org, I found one for $6/mo which includes 3000 free inbound minutes and couldn’t pass it up. My base VoIP price is therefore 4.95 * 2 + 6 = $15.90 plus usage, or $7.95 on an even split. Theoretically this leaves me with just over 650 minutes before exceeding my $20 target, but this doesn’t account for incremental billing, free VoIP to VoIP calls, and other variables which impinge cost.

It’s now more than a month since I dumped Broadvoice, ergo September’s charges and complete usage data are available for a real world comparison against a $40 average, a $35 example, and a $20 target. As it turned out we made no outbound calls on the (secondary) CallWithUs.com trunk and didn’t exceed the 3000 inbound minutes, so all billable usage was on the VoiceMeUp.com trunk which makes accounting easier. September’s total was 9.95 + 6 + 23.38 = $39.28 or $19.64 per split which helps validate my “less than $20 phone bill” theory. Our total usage was 36h 7m 33s (2167.55 min) or nearly 1,100 “home” minutes and more than 2,000 unused inbound minutes– how much do you talk?

AppArmor, introduced to Ubuntu with Gutsy, is yet another security tool unleashed upon the infosphere. In part, AppArmor is intended as an alternative to SELinux, which can easily be seen as daunting to configure; unfortunately, many such projects are daunting for those admins forced to walk the plank of unfamiliarity above a sea of expectations. Despite a troubled history, the project seems to be here to stay so it is likely only a matter of time before audit messages crop up in one’s kernel log. For those who find AppArmor unnecessary, unpalatable, or just untimely, herein lies a quick-and-dirty guide for telling AppArmor where to stick its audit complaints.

Ubuntu’s community documentation has some basic commands for starting, stopping, disabling, and enabling AppArmor, but if we want to know as little as possible about AppArmor then it’s not unlikely that we’re just trying to dispense with a plethora of audit complaints in our kernel log. The first time this happened to me, it was on a box which had some MySQL data in a nonstandard location, resulting in a flood of log entries similar to:

Sep 23 11:22:17 bluebeard kernel: [4960023.353512]
audit(1222183337.704:68500): type=1502
operation="inode_permission" requested_mask="r::"
denied_mask="r::" name="/u1/mysql/"
pid=1573 profile="/usr/sbin/mysqld"
namespace="default"

To allow mysqld to do its thing in /u1/mysql without sending AppArmor into a conniption fit, just add permissions to its profile, located by default at /etc/apparmor.d/usr.sbin.mysqld on Ubuntu Hardy systems.

# custom permissions
/u1/mysql/ r,
/u1/mysql/** rwk,

The first line is a comment which makes clear that the lines which follow are not default permissions. The second line grants read access to /u1/mysql/, which AppArmor expects to be a directory due to the trailing slash. The final line uses the /** syntax to specify a group of files and subdirectories, to which are granted access to read, write, and perform locking operations. Note that these rules were simply adapted from the default rules for /var/lib/mysql; such “cut-and-paste” adaptation helps avoid typos and lessens the need for a detailed understanding of the configuration syntax. Note further that this is an observation rather than a recommendation.

Once all profiles have been edited to perfection, reload AppArmor and we’re off to the races.

sudo /etc/init.d/apparmor reload

When this information proves inadequate, complete documentation and additional resources can be found at OpenSuse.

Like many others, when I set up my first Linux PBX I knew little about VoIP providers and with few sources of reliable, current information, I made a decision based on name recognition, perceived value, and minimal research. Like many others, I looked for companies who advertised a BYOD plan under the false assumption that said companies would have a clue regarding said devices, despite the cautionary warnings which politely explained that BYO, as used here, means “unsupported”. Like many others, I signed up with BroadVoice believing I had a pretty good deal; in fact, among similar plans offered by cable companies and Vonage, BroadVoice compares rather well.

By the time I started to suspect BroadVoice of stockpiling probiscus laden mammals and bleach, I had already paid setup fees and number transfer fees, and chagrined the thought of early termination fees, more number transfer fees, and a potential three to seven week transfer period. Rather than add to the copious corpus of BroadVoice complaints, I thought I’d focus on what to avoid when choosing a VoIP provider.

Customer (Dis)service
In a word, BroadVoice’s customer service is atrocious. If I want to feel marginalized and ignored I can go to the Dept. of Motor– no, on second thought, the DMV has really become much better in this area. BroadVoice’s operating principle seem to be protecting it’s own interests, even at the expense the customer’s. Read the terms and conditions carefully.

One of my more dissatisfying experiences with BroadVoice wasn’t actually covered in the terms and conditions I read: to order a second trunk their system required me to create a second account which I found bizarre but acceptable. The trouble I had was when they informed me that in addition to the credit card information I had already submitted on their encrypted website, I was now required (i.e. only for the secand account) to submit copies of my credit card and driver’s license via unencrypted e-mail or fax– in other words, a ready made identity theft kit. When I admonished them to provide some means of secure transfer, I received a response which inexplicably stated:

Please rest assure your information is protected, and locked away. You are in no way at risk of idenity theft as you may think.

Right. Naturally. Sure. (Does your multivitamin include enough irony?)

Number (De)porting
BroadVoice claims that as a service provider they are not governed by FCC local number portability (LNP) rules, which is why their terms & conditions state:

1.10 Number Transfer on Service Termination
BroadVoice may, solely at the Company’s discretion, release any telephone number that was ported in to BroadVoice by you and used in connection with your Service to your new service provider, if such new service provider is able to accept such number, upon your termination of the Service, and provided (i) your account has been terminated; (ii) your BroadVoice account is completely current including payment for all charges and disconnect fees; and (iii) you request the transfer upon terminating your account. BroadVoice will not transfer or release telephone numbers that it has assigned for use in conjunction with your Service.

Lies, Damn Lies, and Statistics
The 800lb Guerilla at the VoIP party is the unlimited residential plan. To use a more topical phrase, advertising a VoIP plan as having unlimited minutes is a bit like putting lipstick on a pig; someone has clearly tried to make an attractive offer, but it isn’t what we are supposed to think it is. Somewhere in the terms & conditions is a section which redefines “unlimited usage” as “normal usage” or some similar distinction, along with the actual number of included minutes as well as overage penalties. In the case of BroadVoice, the overage penalties on an unlimited plan can be quite steep.

In the end, I can give credit to BroadVoice on two significant points; they may be ill prepared, but their recognition of the tech-driven evolution of business models yields a significant advantage over a company like Vonage. While putting on a friendly face with flashy ads, retail deals, and quality customer service, Vonage hamstrung itself with a business model that prohibits technical innovations such as Asterisk by depending on closed systems and customer lock-in. Even if Vonage weren’t at the wrong end of a double-barrelled patent infringement lawsuit, the mobile VoIP market would still be a likely refuge for the misfortunate company.

The other laurel is for BroadVoice’s cancellation procedure; not the termination fees or the porting issues, but the actual account closure was as simple as a written request via e-maill. Each account was closed within 24 hours of my request and final charges were applied not long after (2 or 3 days as I recall). I consider myself fortunate in not having to suffer through any more customer (dis)service simply to put BroadVoice behind me.

Shortly after I last upgraded my mail server, one user reported that his mail client was failing to connect with the message:

"Unable to connect to your IMAP server. You may have exceeded the maximum number of connections to this server..."

He was the only one known to be having this issue, so after a cursory check of the server with no obvious problems, I suggested that this might be an error on his end, such as connecting to the secure IMAP port without using SSL/TLS. Occam’s Razor suggests that a server error is more likely than a client error which just happens to coincide with a server upgrade, so I eventually decided to dig up some infrequently used commands and perform a thorough analysis.

Testing proved that there was indeed a server problem with certain SSL connections, while others worked every time. Deft Googling revealed that the imapd-ssl config file shipping with Gutsy Gibbon had TLS_PROTOCOL=SSL3, whereas it should be TLS_PROTOCOL=SSL23. The user who first reported the error is tech-savvy, so I sent him the commands I used to diagnose the problem and promised to eventually write this post. Below are the commands which are now my first step in diagnosing mail connection problems.

# Test secure SMTP
openssl s_client -connect example.com:25 -starttls smtp -showcerts
#
# Test secure IMAP
openssl s_client -ssl2 -connect example.com:993 -showcerts
openssl s_client -ssl3 -connect example.com:993 -showcerts
openssl s_client -tls1 -connect example.com:993 -showcerts

To test secure POP, simply substitute 995 for 993 in the above commands, likewise if you run secure SMTP on a port other than 25, you will need to alter the first command.

I have resisted the urge to display caller id on my MythTV as somewhat obvious. I’m always looking for ways to demonstrate the freedom which comes from using open source software, but I prefer the zesty freshness of an original idea rather than anything that’s been done, redone, and done again. My wife, however, thought that Myth caller id sounded like a great idea and asked me to set it up. What follows is how I did this with the least possible effort.

I already have a couple Custom-Apps which handle direct inward dialing on my inbound routes, so sending caller id to my MythTV box was as simple as adding one line to each Custom-App in extensions_custom.conf:

exten => s,4,System(/bin/echo "Caller: ${CALLERID(name)} ${CALLERID(number)}" | /usr/bin/nc -w 3 myth.example.com 1234)

On the MythTV side I decided to use xosd rather than mythosd because the latter will only work when you are watching live TV or recordings. I created a simple wrapper script called osd.sh:

#!/bin/sh
DISPLAY=:0.0
FONT="-xos4-terminus-*-*-*-*-32-*-*-*-*-*-*-*"
osd_cat --font=$FONT --align=centre --shadow=2 --color=SeaGreen –delay=15


At this point I have the PBX sending caller id to MythTV and osd_cat to display the data on screen, but I need a simple way to catch the data on port 1234 and feed it to my osd.sh script. I did this with one command:

micro_inetd 1234 ./osd.sh &

Micro_inetd is a nice replacement for inetd, xinetd, etc. if, as here, only one service needs to be managed. I set this to always start with xfce4-autostart-editor which naturally only works with the Xfce 4 desktop environment. Gnome, KDE, and other desktops have their own session managers.

Of course, now that we have caller id on the TV, I’ll need to create a caller id lookup to compensate for the wretched state of telco support; fortunately I can put that off until next year

Update: change osd.sh to format phone numbers

#!/bin/sh
DISPLAY=:0.0
FONT="-xos4-terminus-*-*-*-*-32-*-*-*-*-*-*-*"
cid=`cat | perl -pe 's/(1?\d{3})(\d{3})(\d{4})/$1.$2.$3/'`
echo "$cid" | osd_cat --font=$FONT --align=centre --shadow=2 \
--color=SeaGreen --delay=15


I have a strange issue on my Asterisk box. If I call BroadVoice tech support using one of their trunks, I connect normally and hear the initial IVR, I press “1″ and hear “Your call is being transferred.” Then the weirdness starts: I remain connected, but I hear my own hold music. As near as I can figure, while I’m on hold in their call queue, Asterisk has dumped me in to hold and I can’t get out. If I stay on long enough for a tech to pick up, they either hear nothing or my hold music and hang up. Free beer goes to anyone who can identify and solve this issue, meanwhile I have developed a workaround.

The above problem fails to occur if I use a non BV trunk or, even better, if instead of calling a ten digit support number I use 611. Unfortunately 611 is also the dial code for a weather-by-airport feature in trixbox, so when I first tried dialing I only heard a busy signal. Fixing this problem is really easy, I only needed to edit extensions_trixbox.conf and comment out the lines with “exten => 611″. Another way to resolve number conflicts is to use the Misc Destinations module to dial one number via another.

While I was cooking last night’s dinner, I made the mistake of leaving my laptop running, open, and unattended. Because ours is primarily a Linux household (my wife is a Mac user), I normally don’t worry much about the computers. The servers, devices, and desktops tend to chug along without needing anything more than an occasional `aptitude update && aptitude upgrade`. Laptops, however, are an entirely different story. As you can see in the photo, we live with a creature that is essentially a heat seeking missle bent on killing laptop computers. Sure it was funny the first couple times, but amusement quickly turned to horror when I saw that she can actually crash Linux. All my base are belong to her.

I finally built a red box, not the phone phreak device that generates coin tones for pay phones, but rather a Linux PBX which gives me the power and flexibility of a commercial grade phone system at a fraction of the cost. I call it a red box because the primary VoIP number I chose suggests ^[1]June 20, 1963– the day the “red telephone” went live between Washington and Moscow. Once I painted the side panels a nice, shiny red, I decided that in keeping with the metallic network naming I use (cobalt, tungsten, strontium, etc.) the best name for my new PBX would be ‘copper’.

After I explained what a PBX is, my neighbor asked me, “Why do you need a $50,000 phone system?” A big factor is that it doesn’t cost $50,000 with Open Source software: I purchased $200 in computer parts, a $50 Grandstream GS-386 analog adapter, and a $275 Aastra 480i CT. My total hardware cost for a working PBX was about $525. Add in the cost of new VoIP accounts, transferring numbers, etc. and I spent about 10% of the cost of even a low end commercial PBX.

Fortunately, low end features are not what the Linux PBX offers. The possibilities of Asterisk, perhaps the prime component of a Linux PBX, are limitless because the Open Source license allows any developer the freedom to modify the software as needed. Practically speaking, this means that mature Open Source projects tend to do all the things the users want.

For my PBX, I wanted all the features I grew accustomed to as a Vonage user– call waiting, forwarding, ring groups, voice mail via phone, web, & e-mail, etc.– but I also wanted one high end feature not normally available without a costly PBX: Direct Inward Dialing.

Direct Inward Dialing (DID) is a feature which allows virtual phone numbers to be routed directly to extensions while using shared trunks (phone lines). DID has clear advantages when one considers that VoIP lines usually cost $15-$30 per month while auxiliary numbers are only $2-$5 per month. Dedicated fax numbers, home/work numbers, listed/unlisted numbers, even toll free numbers all become reasonable options.

By far the best part of a Linux PBX is the ability to work around problems. I’m using Broadvoice as a primary VoIP provider, in conjunction with CallWithUs for overflow, backup, and cheap rates. The inbound calls come through Broadvoice; unfortunately all calls are routed through the same DID, which is to say that all inbound calls appear to be placed to the same phone number. With a bit of research and experimentation, I was able to use the distinctive ring feature to create my own DID and route the calls as I needed. Open Source to the rescue!

1. I dumped the number when I got smart;
   it was 206-1963 (20.6.1963). ^

Some time ago I enabled recipient delimiters (e.g. user+foo@host.tld) as a convenient way to know if shady web forms are contributing to my spam folder. The idea is that when House Depot requires me to have an account before I can see if they have loose screws in stock locally, I can sign up with garrison+housedepot@codefix.net instead of my usual e-mail. With recipient delimiters enabled, postfix will try to deliver any incoming mail to garrison+housedepot but when it finds no such user, it will try garrison and I get my mail. The problem arises when I discover that House Depot’s broken web form rejects any e-mail addresses with “+” in the user name as invalid. I’m already using garrison+foo style addresses elsewhere so I don’t want to change the recipient delimiter, but neither do I trust my real address to a company that can’t even create a proper web form.

Postfix allows only one recipient delimiter, but address rewriting can mimic multiple recipient delimiters; I chose to use regular expression tables in my aliases database. I’m most comfortable with Perl, so I chose to install the Perl compatible regular expression package, on Debian/Ubuntu this is done with:

sudo aptitude install postfix-pcre

Next I created a file named aliases-pcre with the following content:

/^garrison\./ garrison

Finally I updated main.conf with these lines:

alias_database = hash:/etc/postfix/aliases
alias_maps = hash:/etc/postfix/aliases, pcre:/etc/postfix/aliases-pcre

Now I can enjoy e-mail addresses like garrison+housedepot@codefix.net or garrison.housedepot@codefix.net without needing to keep track of each alias. If I get spam addressed to that address, I can easily block it and complain loudly to House Depot’s customer service in India.