Live Ruby process inspection

Posted on September 23, 2006

OK, this is totally insane. First Jamis Buck hit us with Inspecting a live Ruby process article where he demonstrated a number of very interesting technics you can use in order to poke around your live Ruby process.

But then Mauricio Fernandez showed us how masters do their katas ( 型 ) in "Inspecting a live Ruby process", easier if you cheat.

I didn't know that gdb can be that powerful! I need to read more about it.

On a side note, I've just finished reading Scalable Internet Architectures by Theo Schlossnagle and I very enjoyed it. If you followed this interesting exchange of opinions, you should definitely read this book. The author make a clear point that the raw speed of your language of choice does not matter in creating scalable solutions. Even if your application is powered by C, you still can reach a point when your single server is not able to serve all incoming requests and you need to scale your application horizontally. This book explains how.

Next on my "to read" list is Building Scalable Web Sites by Cal Henderson.

Ruby interface to libtransmission

Posted on September 15, 2006

One of my projects required a bittorrent integration. I've tried two available in Ruby options:

  1. rubytorrent is a pure Ruby bittorrent client.
  2. libtorrent-ruby is an inteface for libtorrent library.

Unfortunately, the first was not that stable and the latter I wasn't able to build at all. I am pretty sure that I was doing something wrong, but the time was pretty tight and I had to come up with a stable solution.

I remember there is a pretty decent open source application called Transmission. It didn't take me more than several hours to create a binding for Ruby. So here we go:

Transmission is a simple native bittorrent client for Ruby based on libtransmission library.

Installation

The only prerequisite for this library is that your Ruby must be compiled with pthread support enabled:

    $ ./configure --enable-pthread
    $ make
    $ sudo make install

Having done that, download gem and you should be able to install this gem without problems:

    $ sudo gem install transmission.gem

Synopsis

Assuming that you have progressbar gem installed:

    $ sudo gem install progressbar

You can use this script to download with a torrent file:

    require "rubygems"
    require "transmission"
    require "progressbar"

    # create a new session
    session = Transmission.new

    # open a torrent file
    torrent = session.open(ARGV[0])

    # set destination directory
    torrent.download_to File.expand_path('~/tmp')

    # starts a new native thread in background
    torrent.start

    progress = ProgressBar.new(" Downloading", 1.0)
    trap("INT") do
      torrent.stop
      progress.finish
      exit
    end

    # display progress bar
    until torrent.just_finished?
      progress.set(torrent.stat.progress)
      sleep 0.5
    end
    progress.finish

Known issues

  1. I have tested it only on Linux and Mac OSX. It definitely is not going to work on Windows.
  2. API is quite limited. For example, you can't create a new torrent file with it.

Enjoy.

Debugging Rails application

Posted on September 14, 2006

In this article I am going to describe several ways you can debug your Rails application with ruby-debug starting from simplest and proceeding with more sophisticated setup.

Simplest of all

The simplest and most obvious way is to start your application with rdebug script:

    $ rdebug ./script/server webrick

This method doesn't work with lighttpd since the latter starts several child Rails processes.

When you execute this command you end up with debugger prompt where you can set up your breakpoints.

Debugging without rdebug script

First you need to activate the debugger in your development.rb file:

    ./config/environments/development.rb:

    ...
    require 'ruby-debug'
    Debugger.start

Now you can use Kernel#debugger method to activate debugger:

    def my_action
      debugger
      ...
    end

Remote debugging.

If for some reason you want to connect remotely to debugger, you can start it with remote debugging enabled:

    $ rdebug -s ./script/server webrick

and then connect to it with

    $ rdebug -c
    Connected.
    /usr/local/lib/ruby/1.8/webrick/server.rb:91: if svrs = IO.select(@listeners, nil, nil, 2.0)
    (rdb:1)

Once again, at this point you can add breakpoints and proceed.

Adding breakpoints directly in your code

What if you don't want to setup breakpoints all the time from the debugger's command prompt and you want to use Kernel#debugger method instead. In this case when you start a debugger, you should notify it not stop when a remote client is connected with -n option:

    $ rdebug -sn ./script/server webrick

Remote debugging without rdebug script

As always activate your debugger in development.rb file:

    ./config/environments/development.rb:

    ...
    require 'ruby-debug'
    Debugger.start_remote

Here you can control it with two options:

    ./config/environments/development.rb:

    ...
    require 'ruby-debug'
    Debugger.wait_connection = true
    Debugger.stop_on_connect = true
    Debugger.start_remote
  • Debugger.wait_connection options makes debugger wait until you connect to it remotely.

  • Debugger.stop_on_connect drop you to the debugger prompt as soon as you connect to it.

The method I use most of the time

All described so far methods suffer from one thing: the debugger is always activated. It can slow down your application significantly. What if I want to activate it only in a particular place in my code? As always there are two ways to do that. But first you should only require debugger in development.rb file without starting it:

    ./config/environments/development.rb:

    ...
    require 'ruby-debug'
  • Debugger#start takes a block:

    def my_action
      ...
      # at this point the debugger is still disabled
      Debugger.start do
        # here debugger is enabled
        # below goes the code you want to debug
        ...
      end
      # at this point the debugger is disabled
      ...
    end
    
  • Use Module#debug_method (available since version 0.4.2):

    class MyController < ApplicationController
      def my_action
        ...
      end
      debug_method :my_action
    end
    

    Now whenever you reach my_action method, the debugger is activated.

Fun with Ferret

Posted on September 05, 2006

I've been using Lucene for many years on many different projects. Thanks to Dave Balmain, this library is ported to Ruby and it seems to be even faster than original. It's called ferret. I've just started playing with it and so far it looks very promising.

Sometime ago, Zenspider posted a neat trick that allows searching ri database. Since Ruby version 1.8.5, this database is expanded significantly in size, not to mention that every installed gem extends it with its own documentation. It might take some time if you just go through all these yaml files and grep for an interesting information.

This particular task nicely fits with what ferret can offer.

Install ferret

    $ sudo gem install ferret

Build index using the following script (ri_indexer):

    $ cat ri_indexer
    #!/usr/bin/env ruby

    require "rdoc/ri/ri_driver"
    require "rubygems"
    require "ferret"
    require "find"
    require "yaml"
    include Ferret

    INDEX_FILE = File.expand_path('~/.ri_index')

    fis = Index::FieldInfos.new
    fis.add_field :name, :term_vector => :no
    fis.add_field :content, :store => :no
    fis.create_index(INDEX_FILE)

    index = I.new(:path => INDEX_FILE, :create => true)

    dirs = RI::Paths::PATH
    dirs.each do |dir|
      Find.find(dir) do |fn|
        next unless File.file?(fn)
        doc = YAML.load(File.read(fn))
        next unless doc.respond_to?(:comment)
        next unless doc.comment
        index << {
          :name => doc.full_name, 
          :content => doc.comment.map{|f|f.body if f.respond_to?(:body)}.join("\n")
        }
      end
    end
    index.optimize
    index.close
    $ ./ri_indexer

Now you can use this script for searching (ri_search):

    $ cat ri_search
    #!/usr/bin/env ruby

    require "rubygems"
    require "ferret"
    require "find"
    require "rdoc/ri/ri_driver"
    include Ferret

    INDEX_FILE = File.expand_path('~/.ri_index')

    query = ARGV.join(' ')
    ARGV.clear

    RI::Options.instance.use_stdout = true
    ri = RiDriver.new
    index = I.new(:path => INDEX_FILE)
    index.search_each(query) do |id, score|
      puts
      begin
        ri.get_info_for(index[id][:name])
      rescue Exception
        puts $!.message
      end
    end
    $ ./ri_search kill

This is way more faster than the original script. Also this script accepts quite sophisticated query expressions. For example,

    $ ri_search rescue AND public
    $ ri_search +split -String

Refer to ferret's trac web site, where you can find more information about this wonderful library.