2009/01/30

Rest with Merb: CouchDB resources



I've been studying couchdb. There are several ways to use couchDB with merb. CouchRest, dm-couchdb-adapter, relaxdb. At the beginning, I've focussed on dm-couchdb-adapter. Because there is a good example, the muddle. It works on my local machine too. I looked into the source codes, and it was not easy to understand them. My head was full of relational database concepts, and it prevented me to go further. I decided to spend more time on couchDB. I asked @merbist via twitter that which would be the proper way to use couchDB. And he answered that "Couchrest for sure, the 2 other ones don't use couch properly"

These are the resources to start.

There is a ruby interface written by jchris, Couchrest. He has a plan to launch Couchrest::Model seperately [link].

And Couchview has been replaced by couchapp [link](I've followed Peepcode screencast, and spent some time to figure out why "couchview" command is not working.)

$ couchapp
Usage: /opt/local/bin/couchapp [options] (push|generate)
-q, —quiet Omit extra debug info
-h, —help Display detailed help and exit
-v, —version Display version number

Here is what jchris said about couchapp at couchrest google group[link]
====================
Couchapp is equivalent to couchview, except for a few details about
how it expects to see documents stored on the filesystem.

If I recall correctly, couchview takes something like this

views/test-map.js
views/test-reduce.js

and pushes it to the views member of a design doc, which you'd specify
on the command line, with a default of falling back to a directory name. Couchview does that job just fine, but it is no generalized to handle the other functions that can be stored in a design doc.

the CouchApp script takes a filesystem and pushes it transparently to a design doc, so the on-disk layout it expects is a little different:

views/test/map.js
views/test/reduce.js
validate_doc_update.js
shows/post.js
lists/index.js
_attachments/images/example.png
_attachments/foo/bar.html

you are free to leave any of that out (so if you only care about views then you should be able to use it just fine.)

currently the python version of couchapp also has the ability to "clone" from a url. I'm working on getting the ruby version back up to speed, but for now you might be happiest installing the python version.


Things move so fast around couchDB and couchrest that one article is not proper to summarize them. I will post time to time about it.

2009/01/27

merb pagination

written by billturner #merb freenode irc channel.

Q. "@current_page = (params[:page] || 1).to_i " what does that refers to?
A. billturner> when you use the pagination, and the <%= paginate(..) %> stuff, it gives you links like /posts/index?page=2
A. billturner> so, if there's no "page=X" then it will pull the first page of results


2009/01/23

502 Bad Gateway



Recently, I've changed my server from mod-passenger to nginx because of the comment at merb google groups[link]. Passenger was fit for me. It makes deployment more easier than before. However, merb's way of handling processes seems complicated and advanced as Ezra said[link]. And it is not fully compatible with passenger now.

So I came back to nginx. I thought it would be easy. But it took some time to run my small app. I 've seen a lot of "502 Bad Gateway" page. I guessed that compiling was wrong at first time. It wasn't nginx but my logic in the codes. So the best way to check what's wrong, just type "merb" on the server. I haven't installed necessary gems and libraries. Yes, it is very basic stuff but easy to miss.

2009/01/16

Twitter on Merb : Merb flat app Walkthrough





Description


Motivation

I’m a newbie in merb community. Since “the merger”, I needed correct and reliable reference and so was other newbies in the community. I started to translate merb-book in Korean. But it was not enough for us .I thought that learning merb would be easier if I build an app what I really need such as tracking recent changes. It was easy since there are good tutorials and tools. And scraping nightly version of README file could be helpful too. Because merb core team members are too busy to write guides and tutorials.

I laid aside scraping the README files for now. I just aggregated merb commits rss and send them to twitter. I use TwitterFox as twitter client on my desktop. Whenever a new message comes, It shows the message in a brief moment. It's really handy.



Generating a flat app




This is what I did at first time. As you see, It is really small and shows basic structure how merb works. If you want to learn more about internals of merb, visit here(http://github.com/michaelklishin/merb-internals-handbook)


Using Model in a flat app

I edited config/framework.rb file. Because I could not use twitter_search gem to verify the message is unique or not. This is same as twitter timeline api. It seems that they use cache. Therefore I had to add a model to check the message I'm sending is unique or not.



First, I added one line :model => Merb.root / "models" in config/frameworks.rb file. And then, wrote dependencies in init.rb file.

$merb-gen model tweet



$rake db:automigrate


Scraping with nokogiri

Since merb flat app has a single controller file, It was easy to start. But the twitter api and nokogiri took some time to use. I was not familiar with nokogiri, I googled and found this tutorial. (http://www.robertsosinski.com/2008/12/08/scraping-pages-made-easy-with-ruby-and-nokogiri/).
And I better use a well designed web site template. I picked one of the recommended templates from samshmagazine.(http://www.smashingmagazine.com/2008/12/01/100-free-high-quality-xhtmlcss-templates/)

There were 3 or 4 ruby libraries for twitter. And twitter4r has more documents than the others. It took much time to discover that timeline api and search api use cache. I just forgot about the cache. Anyway I installed necessary gems.

$sudo gem install twitter4r twitter nokogiri shorturl

And then I eddited index.html.erb file. In the controller, it only needs render method. I just wanted to use feedtools and feedupdater gem. But there were not enough document to reference and as merb-book says, I better not use codes that I don't understand. Scraping pages with nokogiri is easy but my app would be fragile.



I know the codes above is not that beautiful. But I hope this code help another rubist who want a scraping example through nokogiri. As you see above, k.content which was return by nokogiri's search method was Hash. This is why i used to_s method.


Scheduling background works



And posting a message to twitter was tricky for me. Because of increasing popularity of twitter, posting throuth api was too fast for twitter sometimes. Posting messages takes much time and that code block needs to sleep. In the initial codes, I wrapped the block with run_later. But I found that I need to schedule the tasks. There would be several way to acheive scheduling the daemons. In version 0.0.1.4(Twitter-on-Merb), I used crontab and rake. But the codes were dirty and it did not work as I expected. I talked about this with lakteck via tiwtter, and I remembered that I've seen rufus-scheduler at igvita.com[link]. It was really easy to implement scheduler with rufus-scheduler gem. The scheduler scrapes web sites on every 10 minutes. And I made a starling consumer daemon. I referenced "Advanced Rails Recipes #42"

confing/init.rb


lib/daemons/starling_consumer.rb file


lib/daemons/starling_consumer_ctl.rb file


config/consumer.god file. Daemons are started after "cap:deploy".


Cache

My flat app was too slow, and it was time to implement cache. I referenced,
Here's quick introduction [original gist document link]

== Quick intro
With fragment caching, you can mix dynamic and static content.

With action caching, the whole template is cached
but the before filters are still processed.

With page caching, the whole template is put in html files in a special
directory in order to be handled directly without triggering Merb.

by Dan Kubb on July 22, 2008




But my init file was wrong at that time and throws erros whenever I run "rake db:automigrate" since I set up the cache(http://gist.github.com/44702). I really didn't know how to fix this. At first, I guessed that datamapper version could be the reason why causes it. After updating the gems, error were still there. Fortunately I have known Guillaume Maury(giom) recently via twitter. He helped to get out of this. This code was not familiar with me. I thought I knew the way to setup the cache.
The codes were already in his article (http://gom-jabbar.org/articles/2008/12/14/example-nginx-configuration-for-merb-with-page-caching-using-the-file-store). I run my apps on passenger so I simply skipped it. :) I highly recommend his blog if you are in Merb2 or Rails3. Here is the snippet written by him. (The initial flat app with plain dependency line did not work.)


And I referenced here.
http://blog.evanweaver.com/files/doc/fauna/memcached/files/README.html

$ wget http://download.tangent.org/libmemcached-0.25.tar.gz
$ tar -xzvf libmemcached-0.25.tar.gz
$ cd libmemcached-0.25.tar.gz
$ ./configure
$ make && sudo make install
$ sudo gem install memcached
$ irb
> require 'rubygems'
> require 'memcached'


Deploy

$cap deploy:setup
$scp -r database.yml to server
$scp -r twitter.yml to server

$rake db:create
$rake db:automigrate
$cap deploy

this is my deploy.rb file.

tip

And if you have problems after your deployment, check if config.ru(must be 'config.ru', not 'Config.ru') file exists and the permissions are correct(chmod 755 -R).

2009/01/02

Free merb screen casts

















#1 MerbCamp - Yes, all merbists have seen it.
#2 Merbunity - It's maintained by hassox!
#3 Atlanta ruby user - I found this site recently. I think it's worth to see.
#4 Merb Dependencies and Bundler - by Carl Lerche
#5 peepcode (well, not free. but it's 9$) - The screencast was recorded on merb 0.9 but it still is a good one.

And there are good tutorials.