Useful GMail filter tip

As anyone who works at Google will tell you e-mail is a major part of life and we get A LOT of it. For example I received over 200 e-mails just yesterday, and that number is only going to go up over time as I become further assimilated. Since we use GMail internally for everything I've naturally begun to play around with its filtering capabilities in earnest.

One major concern that I have (and I'm sure many others share) is now that I have agents that are automatically managing my e-mail for me how do I trust that they're doing the right thing? Obviously for this system to work to my advantage I need to trust that it's doing the right thing and trust it to the point that I don't feel like I need to check up on it frequently.

A nice feature of GMail is that when you define a filter it optionally asks you if you'd like to run the filter on the mail you've already received. This provides a convenient way to test things out on the mail you already have in order to make sure that the filters work. This is nice because you don't have to wait for new emails to come in to know that your filter is broken. This alone isn't enough to ensure that your filters work, for that you really need to test them on new mail.

When testing filters on newly incoming messages I've found that it's useful to put into place a safety net to ensure that I don't miss anything that's important. For me this safety net will tag every incoming message with an additional label (I call it "new"). This way I can check up on my filters by looking at only the new messages that have been received since I changed the filters. I can then very quickly look at the mail tagged with the "new" label and know whether or not something was missing a label or incorrectly labeled. To setup the filter which applies the "new" label I use the "Has the words:" field with an advanced search operator of "to:(me OR -me)". Here's a screenshot:



I use this for a few days after changing my filters just to make sure I didn't mess anything up. So far has been working pretty well for me.

Transcoding video

This is mostly for myself to find later, but they may also be useful to others. This command will transcode video into a format that the AppleTV can understand:

ffmpeg -y -i video.avi -pass 1 -f mp4 -vcodec h264 -r 23.976 -sameq -acodec aac -ab 128k -ar 14400 -ac 2 video.mp4


Cable fishing complete

Last week I completed fishing cables for my video distribution system. Still no idea what the mystery blockage in the middle of my wall was, but after a lot of random wiggling and the use of a fish tape in conjunction with the fish stix I managed to get a cable run. Once that was complete, it was quite easy to terminate the cat5e thanks to some tool less keystone jacks. I highly recommend these, very easy to use!

So in total I did four of the six runs that I was originally hoping to do. I decided not to do two of the runs because they were to bedrooms that I never use, and they were pretty a pretty deep and hard to reach portion of the attic.

Now that I have cat5e jacks in my major TV viewing areas I'm in the market for a video player of some sort. The AppleTV may be a possibility although I can't help but feel underwhelmed by it. What I really want is something that's open (or easily hackable and somewhat supported) that will allow me to run 3rd party apps. I have dreams of a 3rd party app sitting on an AppleTV that will allow me to search for and schedule torrent downloads that I'll later be able to view on the TV. First I need a huge NAS that can run a torrent client though.

Cable fishing sucks

I have plans for a gigabit ethernet network that is used for video distribution. So because I didn't have the foresight to have my builder run CAT5e along with coax I've recently been trying to fish some wires down some interior walls in my house. To be perfectly honest my friends did tell me to have the CAT5e run, but I was stubborn and stupid and didn't want to pay my builder's exorbitant fees for each extra run of cable. I figured I could run it myself later and that it would be easy since I had attic access to all of the areas I needed to run cable to.

So up to this point I've spent several hours in my attic, purchased a set of fish stix, spade drill bits, and several sheets of plywood (for a makeshift attic floor). I've also borrowed some high tech cable finding gadgets from a friend.

Even after all of this I still don't have a single wire run. I've managed to get the cable up into the attic -- turns out this is the easy part with the help of the fish stix. I've also found the location in my attic where I need to drop the CAT5e wire down and drilled a hole through the header for the wire. This is where the real problems began -- I've used the fish stix in the hole I drilled, but for some reason there's a mystery blockage somewhere in the wall. Using a cable tester I can verify that the cable is in the right spot in the wall and about 6-12" above the location I need it to come out at. Unfortunately at this point it seems like it's blocked and I have no idea what it's blocked on. The stud finder doesn't see anything, and the AC line finder doesn't see anything either. To add to the mystery if I use a fish wire and push it up from the box with coax in it I can get about 3-4' into the wall, well above where the blockage seems to be on the way down. What's up with that?!?!

So I'm running out of ideas at this point. One friend suggested a fiberscope, but they're a bit too expensive. So I'm stuck.

Hybrid Language Applications

I've recently been working on a small side project for my own personal use and have been considering using a mix of languages for the implementation -- specifically Java and Python but also possibly some Ruby. The reason I'm considering this is that there are unique advantages to each language that I'd like to leverage and I feel like I could implement things faster and more easily they were all available to me.

Specifically for the project I'm working on there's a component which needs to be able to parse RSS feeds. There happens to be an amazing python library called feedparser which seems to be by far the best RSS parsing library out there. So I plan to implement a simple program which knows how to retrieve a set of RSS feeds in python in order to take advantage of this library.

What I'm mostly unsure about is how to glue together my little programs. Right now I'm envisioning the python program producing output to standard output in a line based record format (probably CSV). We all know that a line based format is the "right thing to do" because ESR told us so. My big concern is whether or not there is a better way? It seems like with the addition of a Javascript interpreter in Java 6 (not necessarily required, but might make it even easier), JSON might be on the table.

What do others thing? How do you solve this problem?

Useful learning resources

I'm a big fan of video training, screencasting, and basically any type of online media that attempts to teach you something. I feel like there is so much more to be learned by watching someone work and hearing what they have to say about it than trying to read a prepared tutorial.

Recently I've discovered that Google has posted a ton of technical talks on video.google.com. There are talks there ranging from a dry run of Guido's Python 3000 OSCON talk, a demo of all of the interesting thing's Apple's Automator tool can do, to discussions about nanotechnology. Not all of the talks are great, but a lot of them are!

I strongly encourage you to have a look, you'll no doubt find something there that interests you. Try searching for "Tech Talk" or "engEDU".

Please mock me

Lately I've been working on a new line of code at work and have had the opportunity to introduce a few new technologies. Most recently I've been playing around with mock objects and the jMock library. So far I'm quite impressed with how flexible the library is. If you haven't already you owe it to yourself to take some time out to investigate mock objects.

For me, I'm most intrigued with how much more concise my tests feel. I feel like I'm writing more tests in a shorter amount of time while at the same time testing my code better than ever before. I attribute all of this to a change in test philosophy that jMock has given me.

If you've never used mock objects before the principle is simple, you use the mock object library to specify the behavior you want certain objects to emulate. You then use these mock objects in your test case to interact with the code you're trying to test. When your test case is done, the mock object library revisits the expectations you had of the interactions between the mock object and the code under test. If the expectations you defined weren't met, then the test fails. If they were, then the test passes. It's as simple as that.

For example, suppose we had a Model that we've defined that allows users to register a series of listeners that are notified when changes to the underlying model occur. The listener interface might look something like:

public interface ModelListener
{
  void onAdd(String name, int value);
  void onChange(String name, int oldValue, int newValue);
  void onRemove(String name, int oldValue);
}

A test case which uses this code may then look something like this:

Mock mock = mock(ModelListener.class);
mock.expects(once()).method("onAdd").with(eq("a"), eq(1));
mock.expects(once()).method("onChange").with(eq("a"), eq(1), eq(2));
mock.expects(once()).method("onRemove").with(eq("a"), eq(2));

Model model = new Model((ModelListener) mock.proxy());
model.add("a", 1);
model.change("a", 2);
model.remove("a");

Pretty neat.

Java Brainteaser #1

I was reading some blogs tonight and came across some interesting Java brainteasers. Here's one of them.

In certain circumstances the following Java code will compile and run. What are those circumstances?

if(this) {
  System.out.println("this is, inded, true!");
}

I'll give you a hint, they require a Java 5 compiler.

UPDATE: Answer in the comments.

Interesting technologies...

A couple of interesting technologies (in no particular order) that I haven't gotten a chance to check out yet but have been meaning to...

JXPath - Allows you to access JavaBeans via XPath

XMLBeans - XML to JavaBean binding library, it's supposed to have some XPath/XQuery support in it

Subversion - Seems to be the most popular revision control system nowadays

Buildix - Subversion, Trac, and CruiseControl all bundled together into a single LiveCD

Genshi - As the author says, "A toolkit for stream-based generation of markup for the web." It seems like a pretty decent and modern templating tool for python, appears to be XML only though.

Janino - An embedded Java compiler. Doesn't even seem to need source code to reside on the disk.

What other interesting technologies are out there that people are interested in but haven't yet gotten a chance to explore?

Mono running on OS X

The other day I installed Mono on OS X 10.4.7. The install was painless, I downloaded a .dmg, opened it up, and double clicked the install package. 20 seconds later I had Mono installed and was writing my first "Hello, World!" program. That's how installing all programming environments ought to be.

Now all I need is an idea of a test program to try it out on. :-)

Python Plug-ins

Years ago I was writing an IRC bot in python and had the need to create a plug-in architecture so that I could both abstract specific pieces of functionality away from the core of the bot as well as dynamically load and unload bot behaviors at runtime. At that point in time the ability to do this type of thing was present in python but there wasn't a standardized paradigm for how to do it. Since then things have become a bit more standardized with the addition of standardized import hooks and my code is surely outdated.

There was however one particular aspect of my implementation that proved to be nice I wanted to share -- the ability to reload and reinitialize a plug-in at runtime. This ability was accomplished mostly as a natural side-effect of using the imp standard library module. Here's how my code looked:

class PluginManager:
  def __init__(self):
    self.plugins = {}

  def load_plugin(self, name, config):
    from imp import find_module, load_module
    fh, fn, desc = find_module(name, ['./plugins'])
    self.plugins[name] = load_module(name, fh, fn, desc)

    if hasattr(plugin, '__loadplugin__'):
      plugin.__loadplugin__(self, config)

  def unload_plugin(self, name):
    try:
      plugin = self.plugins[name]

      if hasattr(plugin, '__unloadplugin__'):
        plugin.__unloadplugin__(self)
           
      del self.plugins[name]
    except KeyError:
      raise PluginException("Plugin %s not loaded." % name)

Because I manage the set of loaded plug-ins myself in the plugins member variable I can control the lifetime of a plug-in. This is also coupled with the fact that imp.load_module() always reloads a module, even if it had been previously loaded. Also note the use of the __loadplugin__ and __unloadplugin__ module level methods. If a plug-in module defined either one of these then it would be called during (de)initialization. In essence it's a very simple plug-in protocol that's being defined here.

Hopefully this will be of some use to someone in the future. By the way I don't advocate writing your own IRC bot nowadays. There are plenty of high quality, extensible, already built options out there. My personal favorite is Supybot.

UPDATE: In browsing around I found a library that does something very similar to what I described here, Sprinkles. At first glance it doesn't seem to have equivalents for __loadplugin__ or __unloadplugin__ but they may be hidden in there somewhere.

Powerbook Freezes

I've had a 15" high resolution (the one with the 1440x960 resolution) PowerBook G4 for about 10 months now, and up until a few days ago I've been dealing with hard freezes every couple of days or so. Very annoying to say the least. Well, recently I came across a useful thread on the Apple discussion forums that appears to have completely solved my problem.

The fix goes something like this:

1. Download and install Apple's Computer Hardware Understanding Development Tools (link)
2. Restart the PowerBook
3. Go into System Preferences and select the new Processor preference pane
4. Uncheck the "Allow Nap" option

It's important to note that this fix is only temporary. The next time your PowerBook is rebooted this setting will be lost. For a permanent fix just make your /etc/rc.local contain:

#!/bin/sh
# Disable CPU NAP
/usr/bin/hwprefs cpu_nap=0

There is a drawback to this however. The CPU NAP setting is what allows your processor to automatically adjust its speed downward when it's not being used heavily. This has the advantage of conserving power and lessening the amount of heat generated. So with the CPU NAP disabled, you're looking at more heat and a shorter battery life. Let's all hope for a better solution from Apple (not likely to come now that they're focused on Intel).

Hopefully this will save someone the months of agony that it caused me.

UPDATE: 27 days and counting (as of September 6th) without a crash!!! Everything seems to be fixed, although things are much hotter and my battery life is significantly shorter. It's better than crashing though.

[~]$ uptime
10:38  up 27 days, 12:57, 3 users, load averages: 1.57 1.20 0.95

Hello World

Hello everyone,

Welcome to my blog. I'm still not entirely sure what I plan on discussing here, but it will most likely be a mixture of posts about programming, technology, and anything else I find interesting that I think would be worthwhile sharing with the world. :-)