LarryOnLine

I really like Markaby for HTML templates. I think it’s especially good if Ruby programmers are creating and updating the views. If you have to get a designer involved (i.e. non Rubyist) maybe ERB or Haml would be better, but, otherwise it’s great, because everything is Ruby.

So, I started out trying to get Markaby working with Rails 2.1.0. With some hours of confusion as to why it didn’t work and digging into Rails and Markaby code, I came up with the following two hacks which patch up Markaby to work with changes in the Rails rendering code.

In vendor/plugins/markaby/init.rb, change

ActionView::Base::register_template_handler 'mab',  Markaby::Rails::ActionViewTemplateHandler

to

ActionView::Template::register_template_handler 'mab',  Markaby::Rails::ActionViewTemplateHandler

Create config/initializers/markaby_fixup.rb:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

# see http://onemanswalk.com/work/2007/12/13/using-markaby-w-rails-2-0-1/
ActionView::Base
class ActionView::Base
  def string_path(string)
    string
  end
end

Markaby::Rails::ActionViewTemplateHandler
class Markaby::Rails::ActionViewTemplateHandler
  def compilable?; false; end

  def render(template)
    assigns = @action_view.instance_vars.to_h.merge(template.locals)
    builder = Markaby::Rails::Builder.new(assigns, @action_view)
    eval_args = [template.source, template.filename].compact
    builder.capture { instance_eval(*eval_args) }
  end
end

So this was working fine until I updated to Rails 2.1.1, at which time I got several errors like the following:

undefined method `/people/1320/edit_path' for #<ActionView::Base:0x3d0a090>
On line #26 of people/show.html.mab

    23:   text h(@person.maiden_name)
    24: end
    25: 
    26: link_to 'Edit', edit_person_path(@person)
    27: link_to 'Back', people_path

    vendor/plugins/markaby/lib/markaby/builder.rb:166:in `send'
    config/initializers/markaby_fixup.rb:26:in `method_missing'
    app/views/addresses/new.html.mab:30:in `render'
    vendor/plugins/markaby/lib/markaby/builder.rb:105:in `instance_eval'
    vendor/plugins/markaby/lib/markaby/builder.rb:105:in `capture'
    config/initializers/markaby_fixup.rb:17:in `render'
    spec/views/addresses/new.html.mab_spec.rb:18

I’ll keep digging to figure out how to fix my fix so I can continue to use Markaby, but really what I’m wondering is — am I the only one trying to us Markaby with recent versions of Rails? I found a relevant bug report or two, but no apparent action to fix them. I’m a little worried about the health of Markaby in relation to Rails.

Anyway, I’ll keep plugging and report back here if I work out anything.

I’ve ported my old blog entries to the new blog and filed them under ‘archive’. Hopefully moving them around won’t have confused anybody.

If you’re wondering, I didn’t use the Mephisto supplied Typo->Mephisto conversion script (see article). I just dumped the previous database, copied the Markdown text, pasted it into new articles and then back-dated them in Mephisto.

I thought, for the first entry on my new blog, I’d explain how I got it all up and running. Partly so I will have a reference in the future and partly hoping other people might find it useful.

hosting – ovh

I chose to use a dedicated server from ovh for hosting. I wanted something independent that I had control over and with some reasonable performance. Here’s the configuration I chose:

• ovh config: rsp2
• os distro: linux ubuntu server 8.04.1
• processor: Intel Atom Dual, 2x 1.6 GHz, L2: 2x 512Ko, FSB: 533MHz, Dual-Core
• RAM: 1GB
• storage: 10GB (network attached via. NFS)
• bandwidth: 100Mbps
• throughput limit: none
• 1+1 IP addresses

For me the only real compromise is the network storage instead of dedicated attached disks, but the price was really attractive (€15/month) so I thought I’d see how things go. I’ve noticed that from the command line, disk operations can be kind of slow, but the main purpose of the machine is network services (web, (light) backup, admin, etc.), so I’m hoping that the remote storage won’t be a problem.

I ordered the server on-line on the 7th and it was up and running yesterday (the 10th). So, about a 4 day turn-around. Everything was pretty easy except for one minor hitch – the login ID was wrong on the email they sent to let me know the server was ready. It was admin but should have been root. I guess the other thing that annoyed me slightly was how bear the distro was. I ended up having to install quite a few packages to get a really basic set-up running, but I guess I can’t really disagree with the decision to keep the dristro lean, otherwise you’d potentially end up with a bunch of stuff you don’t need cluttering things up.

blog engine – mephisto

I do a lot of work with Ruby on Rails so I wanted to use, if possible, a blog engine based on Rails. I’ve tried Typo (quite a while ago), and figured I’d play with the new kid on the block and see if he’s any more fun.

Rails deployment

First I had to set up some way to serve up Ruby on Rails. I chose to use Passenger (mod_rails) with Apache’s httpd server. I’ll just create a big list of steps here to give you an idea what I had to do. If you’ve got questions let me know:

apache

1
2

sudo apt-get install apache2
sudo apt-get install apache2-threaded-dev

ruby

1
2
3
4
5
6
7

sudo apt-get ruby-full
sudo apt-get ruby-pkg-tools
# ruby gems -- I chose to install from source to allow ruby gems to update itself
cd ~/src
wget 'http://rubyforge.org/frs/?group_id=126'
cd ~/src/rubygems-1.3.0`
sudo ruby setup.rb # see http://www.rubygems.org/read/chapter/3

rails

sudo gem install rails`

passenger

1
2
3

sudo gem install passenger
sudo passenger-install-apache2-module
sudo vi /etc/apache2/mods-available/passenger.conf

1 2 3

LoadModule passenger_module /usr/lib/ruby/gems/1.8/gems/passenger-2.0.3/ext/apache2/mod_passenger.so PassengerRoot /usr/lib/ruby/gems/1.8/gems/passenger-2.0.3 PassengerRuby /usr/bin/ruby1.8

4
5

ln -s /etc/apache/mods-available/passenger.conf /etc/apache/mods-enabled
sudo vi /etc/apache/sites-available/default

1 2 3 4 5 6 7 8 9

. . . DocumentRoot /var/www/mephisto/current/public/ <Directory /var/www/mephisto/current/public/> Options Indexes FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> . . .

mephisto

Setting up mephisto was a little more work than I would have liked. Two things made it complicated:

• the TZ environment variable needs to be set and that’s not strait forward for mod_rails. There’s a hack that works, but it’s a bit messy.

• the stable version (at the time of writing this) for mephisto is 0.8 which requires rails version 2.0.2 (i.e. mephisto won’t work with newer versions), so I had to freeze mephisto to the 2.0.2 version of rails.

Here’s detailed list of steps I took to get mephisto setup.

1
2
3
4
5
6
7
8
9
10
11
12

sudo gem install tzinfo
wget http://github.com/technoweenie/mephisto/tarball/master.tar.gz
cd /var/www
mkdir mephesto
cd methisto
tar xzf ~/src/technoweenie-mephisto-22dab17c881f9db136194243844d8ace37536099.tar.gz
mv technoweenie-mephisto-22dab17c881f9db136194243844d8ace37536099 v0.8
ln -s v0.8 current
mkdir shared ; mkdir shared/log; mkdir shared/db
cd current
sudo su www-data -c 'rake rails:freeze:edge RELEASE=2.0.2'
vi config/database.yml

1 2 3

production: adapter: sqlite3 dbfile: /var/www/mephisto/shared/db/production.sqlite3

13
14

rm log; ln -s ../shared/log .
rake db:bootstrap

hack to set TZ environment variable that mephisto needs

1
2

sudo mkdir /etc/apache2/passenger
sudo vi /etc/apache2/passenger/envronment

TZ=UTC

sudo vi /etc/apache2/passenger/ruby_with_env

1 2 3

#!/bin/bash` . $(dirname $0)/environment` /usr/bin/ruby $*

sudo vi /etc/apache2/mods-available/passenger.conf

1 2 3 4

LoadModule passenger_module /usr/lib/ruby/gems/1.8/gems/passenger-2.0.3/ext/apache2/mod_passenger.so PassengerRoot /usr/lib/ruby/gems/1.8/gems/passenger-2.0.3 #PassengerRuby /usr/bin/ruby1.8 PassengerRuby /etc/apache2/passenger/ruby_with_env

So, the web site for the opensource wiki project my budy, David, and I have been working on, Informl, is now being crawled by Google. Looking at the logs, it was pretty silly to see that Googlebot is crawling, indexing and caching the page edit forms (all the same content as the wiki pages, but with annoying HTML form textareas for looking at the content).

So, we wanted to keep Googlebot from returning the page edit forms for search queries. We looked at using robots.txt to exclude these pages. For Google this can work, but only because of special extended behavior for Google in the robots.txt.

The URLs we wanted to exclude look like:

http://informl.folklogic.net/pages/1234;edit

It’s the part on the end – ;edit – that indicates that the URL is for editing the page. Normally, robots.txt only pays attention to the initial part of the URL to make exclusion decisions, but Googlebot allows wild cards, so we put the following in the file:

User-Agent: Googlebot
Disallow: /*;edit

We used the Google Webmaster Tools to see how Googlebot would interpret the URLs in relation to the above rule, which appears to do the trick.

Now, we started thinking about other search engines (Yahoo, Microsoft, etc.) and realized that they wouldn’t generally handle the wildcards that Googlebot will. So, searching for another solution we found the robot meta tags.

So we wanted to add the following to our edit HTML pages:

<meta name="robots" content="noindex,nofollow">

between the start and end head tags.

Now, pretty much all the HTML content on project uses the same Rails layout, and the layout contains the head tags and we wanted to add content to that part of the HTML, but only on the edit pages.

We came up with the following to do this. First, in the layout we added:

<%= @extra_header_content %>

This is pretty safe, because if @extra_header_content is undefined, the line will simply be blank.

Next, we added the following to the edit view:

<%- @extra_header_content = """
  <meta name="robots" content="noindex,nofollow">    
""" -%>

The important thing to note here, is that the view is evaluated before the layout is applied, so setting an instance variable can be used effectively in the layout.

So, that’s how we got our edit URLs to be ignored by (most) webcrawlers using robots.txt and the <name="robots" content="noindex,nofollow"> tag and how we got the meta tag into the HTML header using rails views and layouts.

I wanted to take a hash with a bunch of arrays, strings and other hashes and convert all the “word” like stings in all of them to Ruby symbols. I started to write a gory method that took an object and checked its class, but then I thought better and extended the base classes and used the #collect() method pull everything together.

So, at first I created a method like the following:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

def symify(obj)
  case obj
  when Hash
    o2 = {}
    obj.each {|k,v| o2[symify(k)] = symify(v)}
    obj
  when Array
    o2 = []
    obj.each {|v| o2.push(symify(v))}
    o2
  when String
    obj =~ /\A\w+\z/ ? obj.to_sym : obj
  else obj
  end
end

I was actually happy to find that the Ruby case statement handled the class comparison “out of the box”, but, really, this code is pretty ugly, since there is object class specific behaviour of different classed collected together.

One way to deal with this is to sub-class String, Array and Hash, but there are a few problems with that:

• since these are pretty basic types, you can’t use natural syntax for the data you want to create (lots of stuff like my_string = MyString.new("mystring") instead of my_string = "mystring"); and
• there’s no really clean way of dealing with the default cases for all the other object classes.

Ruby’s ability to extend existing classes – even the basics like String and Array – can simplify this kind of thing a lot. You can even extend the base class Object to deal with the default case.

So, here’s my final code for my #symify() method (now implemented on individual objects):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

#!/usr/bin/env ruby

class Object
  def symify; self; end
end

class String
      def symify; match(/\A\w+\z/) ? to_sym : self; end
end

class Hash
  def symify
    hash_array = inject({}) do |obj, p| k, v = p
      obj[k.symify] = v.symify
      obj
    end
  end
end

class Array
  def symify; map { |v| v.symify }; end
end

Here’s some sample input and output:

1
2
3
4
5
6
7
8
9
10

"word".symify                   # -> :word
    
['a', 'b', 'c', 'x x'].symify   # -> [:a, :b, :c, 'x x']
      
{'a' => 'x', 'b' => 'y', 'c' => 'a b', 'x x' => 'c'}.symify
                                # -> {:a => :x, :b => :y, :c => 'a b', 'x x' => :c}
{'a' => 'x', 'b' => [ 'c', 'd', 'x x']}.symify
                                # ->{:a => :x, :b => [ :c, :d, 'x x']}
['a' , { 'b' => 'c', 'd' => 'x x'}] 
                                # -> [:a , { :b => :c, :d => 'x x'}]

I had to fiddle quite a bit with the Hash#inject() method to figure out how it worked. It wasn’t immediately obviously to me that the key/value pair would be supplied as an array. In any case, the code here works and demonstrates how the inject works with a Hash.

So, I ran into a situation in Ruby where I needed to expand tabs in a string and replace them with spaces. I rolled my own, but found that I really just needed to improve my googling skills and to RTFM.

Now I didn’t want to simply replace each tab character with 4 or 8 spaces, but instead to replace tabs with enough spaces to put the content following it on the next tab-stop column.

I googled around and found some stuff on the web ^1,2, but it didn’t really do what I wanted. So I wrote my own:

class String
  def expand(width=4)

    tr = self
    tc = ""
    while 0 != tr.length
      case tr
      when /\A([^\t\n]{#{width}})/
        tc += $1
        tr = $'
      when /\A([^\t\n]{0,#{width-1}})\t/
        tc += $1 + " "*(width-$1.length)
        tr = $'
      when /\A([^\t\n]{0,#{width-1}}\n)/
        tc += $1
        tr = $'
      else
        tc += tr
        tr = ""
      end
    end

    tc
  end
end

You’ll notice that I wrote this as an extension of the String class; so, you can do stuff like:

no_tab_string = "x\txx".expand      # -> "x   xx"
# ^^^^^^^0^^^^^^^^^0^^^^^^^^^0^^^^^^^^^0^^^^^^^^^0
#        1         2         3         4         5

I was pretty satisfied with this, but I did some additional googling for better regular expression methods, and low and behold, I found the Ruby FAQ, which as the following suggestions for tab expansion:

  1 while a.sub!(/(^[^\t]*)\t(\t*)/){$1+' '*(8-$1.size%8+8*$2.size)}
#or
  1 while a.sub!(/\t(\t*)/){' '*(8-$~.begin(0)%8+8*$1.size)}
#or
  a.gsub!(/([^\t]{8})|([^\t]*)\t/n){[$+].pack("A8")}

Well, I guess I should have RTFM. All those are much better than what I came up with.

I particularly liked the 3rd one. I did have to make a minor change to it to make it work the way I wanted – I needed a way to handle tabs across multiple lines:

class String
  def expand(width=4)
    gsub(/(\n?)(?:([^\t\n]{#(width}})|([^\t\n]*)\t)/n) \
      {$1+[$+].pack("A#{width}")}
  end
end

So, through this exercise I learned a little about Ruby, but more about learning from others…

I’ve been working on a wiki parsing engine in Ruby for a while, and the heart of the engine uses a set of cascading regular expressions (RE) to parse the input and transform it to HTML. I’ve been mulling over better ways to handle managing the overall parsing task based on the expressions and have been envying the regular expression power of Perl et. al. I was guessing that other Ruby hackers were in the same boat, so I started to look around for what was available beyond the base regex options in Ruby 1.8. Well, there are several, but using them for a published project wouldn’t be the easiest thing – at least not for a while.

The Ruby 1.8 regex based REs are pretty good for what they do, and they handle most cases you generally run into in “casual” use, but they really pale in comparison to what’s available in Perl, Python and PHP. Perl has its own RE implementation (which has changed quite a bit between v5 and v6), which has been the driver for much of the RE feature advances in other languages and libraries. PHP and Python, for instance, use the pcre library, which provides Perl compatible REs (PCRE stands for “Perl [5] Compatible Regular Expressions”).

So, what are the alternatives for Ruby? Well, the ultimate target is the use of the Oniguruma library, which handles the generally desired Perl style features (named captures, lookbehind, etc. ), and it will be available in Ruby 2.0. (It’s available now in 1.9, but that’s not a “real” release FWICT.) If you don’t want to wait for 2.0 here are some other options:

• PCRE module
• Pure Ruby regular expression module

If you need the features, one of these looks like the right way to go. I’ve currently coded around the need for the more advanced RE features, but I will probably want them in the near future in order to make my parser work efficiently.

So, I’ll try and hold out until some reasonable release of Ruby 2.0 comes along, and monitor to see if a reasonable 1.8 release comes out with Oniguruma, but otherwise I’ll probably use one of these two options.

A while a go my PowerBook started spacing out for minutes at a time while the rainbow cursor spun and I was left staring and feeling my blood pressure rise. After a while I realized that the problem only happened when I had iCal running. So for a while I just tried to avoid running iCal, but that wasn’t really working since I have my whole life in there. So, I’d start it – wait – check a date – wait – add an event – wait – switch to Thunderbird – wait – switch back to iCal – wait – quit iCal – wait – you get the idea.

In the mean time, I did notice that in the /var/log/system.log I had the following enigmatic error message:

    doublewide kernel[0]: disk0s3: I/O error.

That scared me a bit. I started to worry that there was something wrong with my disk, which initiated a round of Disk Utility checks, but nothing turned up. Once I got over the initial fright of a fried hard-disk, I started to get very frustrated with the lack of information associated with the error. After all, it really gives no help at all in figuring out what the source of the problem is. What process/application caused it? What file was being accessed when the problem occurred?

So, finally I figured “I’m a programmer and I should be able to figure this out”, so I broke down and pulled out the big guns with ktrace.

I used the following:

    % ps auxww | grep '[i]Cal ' | awk '{print $2}'
    2592
    % ktrace -p 2592
    % kdump -l

or some variation there of. I’d see lots of file errors, but there didn’t seem to be much correlation of the errors with I/O error entries in the system.log file nor with the stalls. Thrash, thrash, thrash, pull out hair, thrash. A t some point I looked at the man page for ktrace and found the -i flag which shows the activity of child processes as well. Once I did that, I stared to see the SyncServer process show up. Thrash, grep, thrash, scan voluminous kdump output, grep, thrash….

I finally started to see a pattern emerge. Most of the time, but not all the ti me, when the stalls occurred and the I/O error was reported in the system log, I ‘d see that there was a an error in the kdump like: 2311 SyncServer RET read -1 errno 5 Input/output error

It took me a while to figure out what SyncServer was doing when this happened, b ut scanning backward in the trace file, I saw the file descriptor of the file be ing accessed: 2311 SyncServer CALL read(0xa,0x197fc30,0x1000) and then I scanned back and found the name of the file associated with the fd, and it was:

    2311 SyncServer NAMI  "/Users/larry/Library/Application Support/SyncServices/Local/.dat0907.004"
    2311 SyncServer RET   open 10/0xa

When I looked into the offending file, it appeared to be a database file, so I s imply moved it aside so SyncServer wouldn’t find it and rebooted.When I did that all was good with the world. It was like having a new Mac.

I can now use iCal with out cringing and it seems like the overall performance of my laptop is better (but I suspect that is just an over generalisation of my e lation of finding the problem).

So, that’s how I saved myself from computicide and switched back to the love side of the love/hate relationship with my PowerBook.

Now if I could just fix the one broken pixel on my 17” display…