Tuesday, August 12, 2008
Recently I've started to build a new, rich user interface for
OtherInbox using
SproutCore, which has been very enjoyable. One thing that was not enjoyable was properly configuring my development environment.
In production, you'll usually deploy your SproutCore app as a static file, so all you have to do is arrange for your users to hit that URL (which out of the box is configured as /static, but could be anything).
In development mode, though, you want to be regenerating your client on the fly by serving it dynamically from sc-server. To use your app, you talk to http://localhost:4020, and if you want your client to communicate with a backend server, you configure the "proxy" setting in sc-config. Thus when the Sproutcore server gets a request for "/gadgets", it proxies that request to your local development server.
For some kinds of apps, this works well. For OtherInbox though, everything the Sproutcore app does requires you to be signed-in and have an active session with the Rails application server. This caused all kinds of cookie problems, probably because of same origin policy (e.g. my Rails app running at otherinbox.dev was issuing cookies that were somehow getting mangled by the proxying process).
<VirtualHost *:80>
ProxyPass /app http://localhost:4020/other_inbox/
ProxyPassReverse /app http://localhost:4020/other_inbox
ProxyPass /static http://localhost:4020/static/
ProxyPassReverse /static http://localhost:4020/static
ProxyPass / http://localhost:3000/
ProxyPassReverse / http://localhost:3000
ProxyPreserveHost on
</VirtualHost>
See how that works? When I load http://otherinbox.dev/app in the browser, Apache proxies that request to sc-server, which dynamically generates my Sproutcore client app.
When components of that app make requests for other parts of Sproutcore, using the /static URL, Apache also proxies those back to sc-server.
When the app makes requests for anything else, those requests get proxied by Apache to the Mongrel I have running the Rails code. Because my Sproutcore app is making REST calls to the backend, this ensures that anything the Sproutcore app asks for from my server gets proxied properly, in this case to localhost:3000.
As soon as I did this, all of the cookie issues were done. You'll also have to add some application-specific code about how you want to force logins if the user is not already signed-in. In our code, I just check for a logged-in cookie, and if it's not there, we open up the URL for a signin window.
Labels: rails, sproutcore
Wednesday, July 30, 2008
My good friend
Paul Barry and I have been helping our friends at
Medialets out with a
neat little Rails micro-app that culls iPhone App Store data from Apple's endless array of plists, and makes a
chart, a dynamically-generated
Gruff graph, and a bunch of RSS feeds.
My favorite thing about this app is the
New Apps RSS feed which lets me keep up with the newest time-wasters/productivity enhancers for the iPhone. Let me take this opportunity to say that the world does not need any more iPhone tip calculators or fortune-telling games.
Labels: gruff, micro-apps, rails
Wednesday, July 2, 2008
At
OtherInbox we love open source and are looking for ways to share some of our labors with the community. Today I came across a great opportunity to contribute something to
Ruby-on-Rails core development. I'm posting it here so everyone can see how easy it is to contribute.
I was building a JSON API to enable some new awesome features we're working on. Following the
JSON request specification, I had the client setting its MIME type to "application/jsonrequest". But this was not causing Rails to recognize the request as JSON and thus the request body was not properly parsed. After doing some digging, I realized that Rails only looks for MIME type "application/json".
Fortunately, MIME type processing is implemented really humanely in Rails, so I whipped up a little patch that adds "application/jsonrequest" as a synonym for the JSON MIME type. First I wrote a test to prove that this was a problem. Once I had a failing test, I added the MIME type, and got my test passing. I followed the
git patch instructions on lighthouse, then jumped into IRC #rails-contrib to garner support for it.
I happened to see that
Rick Olson, the author of the existing JSON parsing code, was in the chat, so I pinged him with the lighthouse ticket. He tested it and applied it, and now
our one line of code is a part of Rails!
Hopefully this will save some future JSON implementer a bit of pain.
Labels: git, rails
Monday, June 2, 2008
I wrote up a few articles on the
OtherInbox blog about my experiences at RailsConf 2008:
The best blow-by-blow coverage so far is from
Drew Blas. Of the ones I attended or heard the best feedback about, I most strongly recommend looking at the slides for these:
Labels: otherinbox, rails, railsconf
Wednesday, May 28, 2008
I'll be leading two Birds of a Feather sessions at
RailsConf 2008 that I hope everyone will attend (or flock to):
I was also invited to sign copies of the book
Advanced Rails Recipes (to which I contributed a couple of recipes) at
Powell's books in Portland on May 30th at 12:30 pm. Hope to see you there!
I'll be there with the full
OtherInbox contingent, so if you're looking for an awesome startup to join, come and track us down.
Labels: rails
Sunday, May 25, 2008
Reading
The Ruby Programming Language was a great experience — like revisiting a country I thought I knew intimately, but with expert tour guides who showed me whole new landscapes. It's also a good primer on what's changing in Ruby 1.9.
One of my favorite discoveries was Ruby's
autoload method. Using autoload, you can associate a constant with a filename to be loaded the first time that constant is referenced, like so:
autoload :BCrypt, 'bcrypt'
autoload :Digest, 'digest/sha2'
The first time the interpreter encounters the constant BCrypt, it will load the file 'bcrypt' from Ruby's current load path, which it assumes will contain the definition of that constant. (Note that autoload takes the name of the constant, in symbol form, not the constant itself).
Here's an example of how useful it can be.
OtherInbox uses
beanstalkd in a few places where we haven't yet migrated to
SQS. I was loading the
beanstalk client with a Rails initializer, 'config/initializers/beanstalk.rb':
require 'beanstalk-client'
BEANSTALK = Beanstalk::Pool.new(['localhost:11300'])
Making this initial connection on our production server takes five seconds or more each time I restarted the app or dropped into the console. That doesn't sound like much but when you're doing that a few times a day, it starts to add up. So I moved the beanstalk code out of the initializer and into 'lib/etc/load_beanstalk.rb'. I placed all of my autoloads in a single initializer, 'config/initializers/autoload.rb'. For beanstalk, the statement is:
autoload :BEANSTALK, 'etc/load_beanstalk'
Now, the app starts more quickly, and even better, this library doesn't get loaded into memory by parts of the app that don't need it.
Labels: rails, ruby
Tuesday, May 6, 2008
I contributed two recipes to the newly-released Advanced Rails Recipes which I highly recommend. It's got 84 very eye-opening solutions to problems faced by a lot of Rails programmers.
I feel lucky to be working with cool, open technology that's yielded an opportunity to be part of a project like this.
Labels: rails
Thursday, May 1, 2008
For the past seven months I've been building a cool new consumer web app with some folks in Austin, TX,
otherinbox.com/, and we're looking for experienced Ruby On Rails developers. We're a small agile team led by Steven Smith, founder of
FiveRuns.
The whole site is Rails-based and makes extensive use of Amazon Web Services (EC2, S3, and SQS) and there's a new awesome challenge every day. More info on our
jobs page.
Labels: otherinbox, rails
Sunday, March 23, 2008
I thought Mark Bush did a great job of explaining Rails' auto-loading behavior on the Rails-talk mailing list. I'm posting it here to help others find it, and so I remember where to find it next time I'm struggling with it. I've known that Rails automatically requires and loads classes on the fly using standard conventions for module names -> file names, but I had a hard time grasping how it worked. Mark explains it very concisely.
The Rails list has a low signal-to-noise ratio but it's still worth paying attention to for gems like this.
Labels: rails
Monday, February 18, 2008
I had a strange error occur in one of my rspec model unit tests today, and I wanted to document it here because my solution (which is a bit of a hack) is the opposite of what worked for other people.
I have a bunch of tests that check to make sure I'm validating various properties of a model. All of a sudden, I started having tests fail because the validations seemed to be adding the same error twice.
'User creation should require domain names to be unique' FAILED
expected 1 error on :domain, got 2
This error only occured when my full suite of tests ran. If I ran the unit test by itself (or even if I just ran only the model tests by themselves), it didn't happen. Unfortunately I didn't notice whatever it was I did to introduce this error, so I couldn't reverse it.
Several other people have encountered this problem, so I know it's because my tests are leaking state in some way. Somehow I am doing something during my tests that rspec isn't able to clean up. Usually it's because the tests are doing something weird with extra require or load statements, which causes multiple copies of the class to be loaded. Removing these statements usually works.
I had no such statements and so spent a while trying to sort this out. On a whim, I added a require statement to the top of the failing User model spec, and that fixed it:
require 'user'
I wish I had more time to investigate, because it makes me thing I don't know enough about Rails autoloading behavior, or Ruby's loading behavior.
If you're running into this same problem, I found these mailing lists threads to be useful:
Labels: rails, rspec, testing
Tuesday, February 12, 2008
I've been working on a really cool new project, to be announced soon, where I've built a Rails-based web app that has two interfaces: one for humans to interact with inside of a browser, and a RESTful API for browser plugins to interact with via GETs and POSTs. We want people to be able to interact with our site while visiting other sites.
I had seen other Firefox plugins that were click-to-install, but I had a hard time figuring out how to make it work for our plugin. Firefox users were having to "Save Link As.." and open the downloaded .xpi file manually. Very old-fashioned. So here's a quick note to help future Mozilla or Firefox developers who need to create a click-to-install plugin:
1) It's all done through Javascript, so anyone without Javascript will have to install your plugin the old-fashioned way. The Mozilla site documents the API call you need to make.
2) I'm a huge proponent of unobtrusive javascript (UJS), which I learned by using Dan Webb's excellent LowPro framework. Thus I wanted to make sure that the click-to-install javascript was offered as a progressive enhancement to the normal HTML links we provided. That way, everyone could have a link to the plugin file itself for manual installation, but people with Javascript could enjoy click-to-install.
In this part of the site, we weren't using any other Javascript libraries, so it seemed like overkill to include Prototype and LowPro just for this one effect. So it was a great chance to learn how to roll my own UJS without library support. I whipped up a quick UJS click-to-install technique following inspiration from this presentation. Here's what I came up with:
<script defer="defer" type="text/javascript">
//<![CDATA[
function doXPITrigger() {
if (!document.getElementsByTagName) return false;
var links = document.getElementsByTagName("a");
for (var i=0; i < links.length; i++) {
if (links[i].className.match("firefox")) {
links[i].onclick = function() {
xpi={'Awesome New Project Toolbar':'/downloads/awesome_project.xpi'};
InstallTrigger.install(xpi);
return false;
}
}
}
}
window.onload = doXPITrigger
//]]>
3) I've seen other advice recommending you configure your web server to recognize the .xpi mimetype appropriately. I did this but it didn't make much difference in my case. Still, it's probably worth doing. I added this line to our Apache config:
AddType application/x-xpinstall .xpi
Labels: firefox, Javascript, rails
Wednesday, January 16, 2008
I followed a tip from
David Chelimsky's blog and began running autotest with the -v flag for verbosity. When you first run it, you get a bunch of lines like this:
Dunno! spec/other_inbox_spec_helpers.rb
Dunno! app/views/layouts/main/_footer.erb
Which show all the files for which autotest doesn't have a mapping. With
ZenTest 3.8.0 out, it's easy to add mappings (which tell autotest which tests to run when a matching file changes) and exceptions (which tell autotest which files to ignore). You can also set up .autotest files for particular projects, or for your whole development machine (~/.autotest).
Here's an example of a per-project .autotest I use, sitting in the Rails root directory. I've got a custom spec helper and I want to rerun all my tests if this file ever changes:
Autotest.add_hook :initialize do |at|
%w{ domain_regexp perfdata coverage reports }.each { |exception| at.add_exception(exception) }
at.add_mapping(/spec\/app_spec_helper.rb/) do |_, m|
at.files_matching %r%^spec/(controllers|helpers|lib|models|views)/.*\.rb$%
end
end
And here's part of my site-wide .autotest file, mostly cribbed from David's blog, where I'm ignoring other kinds of cruft that pile up in projects. Also note the mapping for spec/defaults.rb, a file I commonly setup in my specs containing default parameters for different models.
Autotest.add_hook :initialize do |at|
%w{.hg .git .svn stories tmtags Rakefile Capfile README spec/spec.opts spec/rcov.opts vendor/gems autotest svn-commit .DS_Store }.each {|exception|at.add_exception(exception)}
at.add_mapping(/spec\/defaults.rb/) do |f, _|
at.files_matching %r%^spec/(controllers|helpers|lib|models|views)/.*\.rb$%
end
end
Labels: autotest, rails, rspec, testing
Tuesday, January 8, 2008
A few weeks back I gave a talk at a
Bmore on Rails meeting all about Presenter classes, which I learned about from
Jay Fields. I can't claim any authorship of this idea, but it's been very helpful to me, so I thought I'd share some of my examples here plus a bit of extra code I wrote to make Presenter integration with ActiveRecord a bit more fun.
Presenters allow you to extract the logic needed for complex views (especially views that require the use of more than one model) into a separate, easily testable class. This helps you write clean code and
skinny controllers, among other benefits.
1) The key background material is here:
2) I extended Jay Fields' code by adding methods to combine error messages from different models:
3) An example Presenter, combining a User object and an Account object into a Preference presenter, is here:
4) An example controller, using the Preference presenter, is here:
Labels: presenters, rails
Thursday, January 3, 2008
hello everyone,
Pragmatic Programmers just released an updated beta version of
Advanced Rails Recipes which contains another recipe that I contributed, this one about dynamic content caching. I've built a few sites that had a lot of static content, with only bit of dynamic content on each page (usually a signout button, or an admin link). In these cases, I was able to use page caching with a little bit of Javascript that looks for a cookie in the client browser and alters the page accordingly. The book is shaping up great and I'm learning a lot by reading other recipes, so definitely check it out!
Labels: caching, rails
Monday, December 10, 2007
I contributed an acts_as_ferret recipe to a forthcoming Rails book,
Advanced Rails Recipes, which is now available as an online Beta. Check out my publishing quasi-debut!
Labels: rails
Wednesday, November 28, 2007
I am building a Rails app that requires some portions of the site to use HTTPS, so naturally I'm using the
SSL requirement plugin. The plugin works great, but if you're using Mongrel or WEBrick running out of script/server in your development environment, you now won't be able to talk to those parts of your site (since these servers do not include SSL encryption).
The solution is pretty easy, but it's not something I found written up elsewhere, so I thought I'd document it here. All you have to do is install your own local Apache server and have it proxy requests to the Mongrel or WEBrick instance, similar to how you would set up your production environment. For simplicity I didn't use a cluster of mongrels, or mod_balance, or anything like that, just a straight-through proxy. See
"A Simple Single Mongrel Configuration" on the Mongrel site for details.
But there's a bit more you need to do in order to make things work with Rails and the SSL requirement plugin. Below is a subset of my httpd.conf file; for clarity, I cut out all the default settings and just left what I added or changed.
Listen 80 # included in the default config
Listen 443 # Apache needs to know you want to accept connections over HTTPS
SSLCertificateFile /usr/local/apache/conf/newcert.pem
SSLCertificateKeyFile /usr/local/apache/conf/newkey.pem
# Below is optional, but was helpful to me in debugging this setup
CustomLog logs/ssl_request_log "%t %h %{SSL_PROTOCOL}x %{SSL_CIPHER}x \"%r\" %b"
<VirtualHost *:80>
ServerName localhost
ServerAlias 127.0.0.1
ProxyPass / http://localhost:3000/
ProxyPassReverse / http://localhost:3000
ProxyPreserveHost on
</VirtualHost>
<VirtualHost *:443>
SSLEngine On
ServerName localhost
ServerAlias 127.0.0.1
ProxyPass / http://localhost:3000/
ProxyPassReverse / http://localhost:3000
ProxyPreserveHost on
RequestHeader set X_FORWARDED_PROTO 'https' # don't forget this line!
</VirtualHost>
<IfModule ssl_module>
SSLRandomSeed startup builtin
SSLRandomSeed connect builtin
</IfModule>
What you're doing is setting up two different proxies through Apache to your Mongrel server; one via port 80, unencrypted, and one via port 443, encrypted with SSL. If you don't include the X_FORWARDED_PROTO line in your 443 virtual host, Rails won't know that it's using SSL and the SSL requirement filter will fail.
The two SSLcertificate directives refer to the cryptographic data needed to encrypt the traffic. I just made some self-signed certificates (which are free). There are a million tutorials out there; I used the one on
Apple's site. You'll need to pick a passphrase to protect your private key. Pick something short (since this isn't the production site) because you'll need to enter it every time you want to reboot your server.
Start things up and you're ready to go!
wymanpark~ $ sudo httpd -k start
Apache/2.2.6 mod_ssl/2.2.6 (Pass Phrase Dialog)
Some of your private key files are encrypted for security reasons.
In order to read them you have to provide the pass phrases.
Server localhost:443 (RSA)
Enter pass phrase:
OK: Pass Phrase Dialog successful.
wymanpark~ $
Labels: apache, rails, ssl
Sunday, October 28, 2007
Capistrano comes with a nice little tasked called deploy:web:disable that puts up a maintenance page on your site. Assuming your webserver detects the presence of this page and rewrites any requests to display that page, this effectively disables your site. Very handy.
I found that the common configuration available on the Internet didn't work for me. Specifically, the Apache Rewrite rule needs to reference DOCUMENT_ROOT. Here are my rewrite rules; remember that for this to work, they have to be first:
RewriteCond %{DOCUMENT_ROOT}system/maintenance.html -f
RewriteCond %{SCRIPT_FILENAME} !maintenance.html
RewriteRule ^.*$ %{DOCUMENT_ROOT}system/maintenance.html [L]Labels: capistrano, rails
Thursday, October 25, 2007
DeploymentI've recently deployed
acts_as_solr and the
Apache Solr server for full text searching in a new app I am building. The plugin works great, is easy to setup, but can cause some havoc with your deployment strategy if you don't pay attention.
For one thing, out of the box the plugin stores volatile information (tmp files, the indices used for searching, and log files) within the plugin itself, because it comes with its own version of Solr pre-installed. That makes it easy to get started and play with Solr but is not acceptable for a production environment (because every time you redeploy your code you'll be destroying and re-creating those directories within the plugin).
The Java UnderworldUnfortunately this means you'll have to descend from the mighty, glistening tower of Ruby into the Java underworld to setup a Tomcat servlet running Solr.
This tutorial does a great job of explaining the process which is pretty easy.
Starting and Stopping SolrI also have added some Capistrano tasks to start and stop the Solr server during deployments to lessen the chance that the index will be corrupted:
namespace :solr do
task :start, :roles => :app do
run "cd #{latest_release} && #{rake} solr:start RAILS_ENV=production 2>/dev/null"
end
task :stop, :roles => :app do
run "cd #{latest_release} && #{rake} solr:stop RAILS_ENV=production 2>/dev/null"
end
task :restart, :roles => :app do
solr.stop
solr.start
end
end
For Capistrano to be able to run the above tasks, I also had to patch the start:solr rake task that comes with acts_as_solr to use backticks instead of exec:
`java -Dsolr.data.dir=solr/data/#{ENV['RAILS_ENV']} -Djetty.port=#{SOLR_PORT} -jar start.jar`
Testing
acts_as_solr will greatly slow down any tests that use fixtures (because the indices get updated every time the fixture data gets added or removed). Ideally you would mock the Solr server in unit and functional testing (unless you're directly testing your app's interaction with Solr), but I haven't found a way to mock Solr
before your tests run, when fixtures are being added. One way to go might be to create something like
mailtrap for Solr requests. The other thing that might help is installing a modification of Solr called
background-solr which defers all the index save and destroy requests and performs them in batches -- the idea being that during testing, maybe you would just avoid calling that batch update method for most of your tests, except where you are testing interaction with Solr (as in an integration test).
Building the IndexThis is really great rake task to help build the Solr index. It's much faster and more effective that doing it manually for each of your models.
Labels: rails, searching, solr
Thursday, September 20, 2007
I just released my first Ruby gem. I have a library of functions that I use for generating realistic data so I can have more meaningful examples to work with during development. So I used the
newgem generator and
hoe to package it all into a gem called
random_data.
It provides a Random singleton class with a series of methods for generating random test data including names, mailing addresses, dates, phone numbers, e-mail addresses, and text. This lets you quickly mock up realistic looking data for informal testing.
Instead of:
>> foo.name = "John Doe"
You get:
>> foo
.name
= "#{Random.firstname} #{Random.initial} #{Random.lastname}">> foo.name
=> "Miriam R. Xichuan"
The gem also includes code for phone numbers, e-mail addresses, physical addresses, and (primitive) text generation.
You can install it via:
sudo gem install random_data
For more details and full documentation, visit the
rubyforge site.
Labels: rails, ruby
Sunday, August 26, 2007
I use page caching in a lot of my projects, but could never figure out how to test them properly. There's a pretty good
plugin that does this but I could never get it to work right. Until today, when I noticed that caching is turned off in testing mode! No wonder I couldn't test it!
So, it seems obvious now, but it never occurred to me to change this line in my test.rb file:
config.action_controller.perform_caching = true
Note that you will have to make sure your pages get swept properly after running your tests. I have a rake task that deletes all of the files, I think I got it from the Peepcode caching episode:
desc "Delete all cached files"
task :sweep_cache => "tmp:cache:clear" do
%w(index.html shows* about.html audio.html contact.html photos.html photos/*.html).each do |pattern|
rm_rf(Dir[File.join(RAILS_ROOT, "public", pattern)])
end
end
A little clunky, but it works. I might someday come up with a way to invoke all of my sweepers instead so I don't have to manually add all of those filenames. I think call this task from an autotest hook:
[:red, :green].each do |hook|
Autotest.add_hook hook do |at|
`rake sweep_cache` if File.exist?("#{FileUtils.pwd}/lib/tasks/cache.rake")
end
end
Because the plugin doesn't have many examples, here's a couple of my tests:
caching_story_test.rb
class CachingStoryTest < ActionController::IntegrationTest
fixtures :all
def test_page_caching
assert_cache_pages(about_path, contact_path)
end
def test_storyteller_caching
storyteller = storytellers(:storyteller_show_1_id_1)
assert_cache_pages(show_storyteller_path(:id => storyteller.id, :show_id => storyteller.show.id))
end
end
sweeping_story_test.rb
class SweepingStoryTest < ActionController::IntegrationTest
fixtures :all
include LoginLogoutDSL
include StoopDefaults
def test_should_sweep_news_items
logs_in
news_item = news_items(:double_whammy)
assert_expire_pages(news_items_path, news_item_path(news_item), index_path) do |*urls|
put news_item_path(:id => news_item.id), :news_item => @@news_item_default_values
end
assert_expire_pages(news_items_path, index_path) do |*urls|
post news_items_path, :news_item => @@news_item_default_values
end
assert_expire_pages(news_items_path, news_item_path(news_item), index_path) do |*urls|
delete news_item_path(:id => news_item.id)
end
end
end
(I'm sure there's a way to DRY up that last test, but I found it much harder to figure out which test was failing when I did that.
Last note: the expiration test only work with path routes, not url routes. So you call "delete news_item_path(...)" vs "delete news_item_url(...)".
Labels: rails, testing
Tuesday, August 21, 2007
I recently launched a new Rails app that powers
Stoop Storytelling which was quite a labor of love. One thing I discovered was that if you take the time to design your models carefully, it's easy to add powerful capabilities later. Since The Stoop has a large trove of mp3 audio files containing all of the stories told during the series, I thought it would be fun to build a
podcast. It turned out to be fairly easy although there are a few gotchas.
ShoutoutI was inspired by the incomparable Topfunky's podcast application:
http://svn.topfunky.com/podcast/I've added code to manage the iTunes metadata that extends the original RSS standard.
Ingredients
You need to use the RSS library that comes with that standard Ruby library, but if you want to add iTunes metadata, you'll need to update the standard library like so:
cd /usr/local/src/
wget http://www.cozmixng.org/~kou/download/rss-0.1.9.tar.gz
tar zxvf rss-0.1.9.tar.gz
cd rss-0.1.9
ruby setup.rb
sudo ruby setup.rb
Then you need to require the right modules in a Rails initializer file. Mine is config/initializers/require.rb:
require 'rss/2.0'
require 'rss/itunes'
ModelLike Topfunky, I found it easier to include the bulk of the function in the model. You could also build this up using a Builder template, which might be easier to test, since you can use
assert_xml_select to test it. But my philosophy was generate it in the model, test it as best I could with a unit test, then test the output completely in my controller test.
def self.rss
author = "Overall Podcast Author/Artist"
rss = RSS::Rss.new("2.0")
channel = RSS::Rss::Channel.new
category = RSS::ITunesChannelModel::ITunesCategory.new("Arts")
category.itunes_categories << \
RSS::ITunesChannelModel::ITunesCategory.new("Literature")
channel.itunes_categories << category
channel.title = "Podcast Title"
channel.description = "Podcast description, can be a paragraph"
channel.link = "http://www.example.com/"
channel.language = "en-us"
channel.copyright = "Copyright #{Date.today.year} I Own This"
channel.lastBuildDate = Audio.last_modified.updated_at
# the above uses a method I built on the Audio model that finds
# the last modified file and makes that the build date for the
# whole podcast channel
# below is your "album art"
channel.image = RSS::Rss::Channel::Image.new
channel.image.url = "http://www.example.com/images/app_rss_logo.jpg"
channel.image.title = "Same as podcast title"
channel.image.link = "Should be same as link for whole channel"
channel.itunes_author = author
channel.itunes_owner = RSS::ITunesChannelModel::ITunesOwner.new
channel.itunes_owner.itunes_name=author
channel.itunes_owner.itunes_email='info@example.com'
channel.itunes_keywords = %w(Common Misspellings of Key Words)
channel.itunes_subtitle = "This appears in the description column of iTunes"
channel.itunes_summary = "This appears when you click the 'circle I' button in iTunes"
# below is what iTunes uses for your "album art", different from RSS standard
channel.itunes_image = RSS::ITunesChannelModel::ITunesImage.new("/path/to/logo.png")
channel.itunes_explicit = "No"
# above could also be "Yes" or "Clean"
Audio.find(:all).each do |audio|
item = RSS::Rss::Channel::Item.new
item.title = audio.title
link = "http://www.example.com/#{audio.public_filename}"
item.link = link
item.itunes_keywords = %w(Keywords For This Particular Audio Clip)
item.guid = RSS::Rss::Channel::Item::Guid.new
item.guid.content = link
item.guid.isPermaLink = true
item.pubDate = audio.updated_at
description = "Long description of this particular audio file, appears in circle I section of
iTunes"
item.description = description
item.itunes_summary = description
item.itunes_subtitle = audio.nice_title
item.itunes_explicit = "No"
item.itunes_author = author
# TODO can add duration once we can compute that somehow
item.enclosure = \
RSS::Rss::Channel::Item::Enclosure.new(item.link, audio.size, 'audio/mpeg')
channel.items << item
end
rss.channel = channel
return rss.to_s
end
Easy, right? What you're doing is building up an RSS feed that complies with the
RSS 2.0 standard and with
Apple's iTunes extension to the base standard. The category code is a little tricky and there may be a better way to do it. I found the RSS library docs a bit hard to understand.
ControllerAll of the heavy lifting is done for us, so the controller is easy.
class AudioController < ApplicationController
def index
respond_to do |format|
format.html { @audio = Audio.find(:all) }
format.xml { render :xml => Audio.rss }
end
end
end
RouteBecause of my inflection rules (where audio is singular and plural) I can't use the baked-in REST route to handle this situation, so for this to work I needed to manually wire the following route:
map.formatted_audio 'audio.:format', :controller => 'audio', :action => 'index'
Unit Test
I haven't written enough tests for this, but here's a start so I at least know the right number of audio files are in there. In audio_test.rb:
def test_should_generate_audio_rss
assert_equal Audio.count, Audio.rss.scan(/- /).size
end
Functional TestAs mentioned earlier, testing the method here gives you the added benefit of assert_select, but you need to apply Jamis Buck's
wizardry to get it to work with XML files by creating an assert_xml_select method.
I'm testing almost the exact same thing as the unit test, but at least I know the file gets served up the way I expect it, and now I'm also testing whether the items have been nested properly within the channel.
def test_should_get_audio_rss
get :index, :format => 'xml'
assert_response :success
assert_xml_select 'channel item', :count => Audio.count
end
I want to improve this by incorporating XML validation into my tests but haven't figured that out yet. For now, I'm validating the feed itself.
Feed Validation
The last step before submitting to iTunes is to make sure this feed works and is valid XML. I used FeedValidator.
iTunes Submission
Pretty straightforward. Just follow Apple's directions. Getting the category right was a little tough, and there's also an issue with iTunes not recognizing the feed image properly. Doesn't look like they've fixed it yet.
Submitting Elsewhere
I added the following to the head section of my app's layout to help other aggregators find the feed:
%link{:rel=>"alternate", :type=>"application/rss+xml", :title=>"Stoop Storytelling Podcast", :href => formatted_audio_path(:format => 'xml', :only_path=>false)}
This reference was helpful in constructing the above tag.
Advertising the Link
Finally, I posted links to the iTunes store on the site itself. If people have iTunes or another podcast client installed, clicking on the store link will cause them to subscribe and help boost the rating of the podcast (thus, you don't want to give people the link to the audio.xml file itself so direct subscriptions won't get counted by Apple).
Final Product
You can listen to the podcasts here:
http://www.itunes.com/podcast?id=262444919References
Beyond the references already cited, I relied heavily on the
RSS library tutorial and
reference.
Future
Is there interest in a plugin to help streamline this process further?
Labels: rails
Saturday, August 18, 2007
Problem: Your Rails app is working beautifully in development and test mode, but for some reason, on the production server, users intermittently have trouble POSTing data. It seems like Apache is turning POST requests into GET requests. Looking at the Apache log, you see the POST request come in. Looking at the Rails production log, you see it has been rewritten as a GET request, triggering a "show" or "index" action in your RESTful controller. This may have started when you implemented page caching, but you're not sure. What gives?
Solution: This is a clash between page caching and the Apache rewrite rules. Apache sees that there is a directory in your RAILS_ROOT/public/ directory that matches the name of the URL you are POSTing to, and tries to satisfy the request by serving up content from that directory. You need to change your rewrite rules so that only GET requests for URLs are satisfied with a static file, and all POST requests are delivered to your rails app (to dispatch.fcgi, your mongrel cluster, etc.)
Solution 2: Even if you're not using caching, this problem could also occur if you have a directory in your public/ directory that matches a route in your app (so if you have a route for /assets/, and there's a RAILS_ROOT/public/assets/ directory, you may have the same problem.
Details:
1) Change this following rule in your .htaccess or your Directory directive:
RewriteCond %{REQUEST_FILENAME} !-f
to this:
RewriteCond %{REQUEST_FILENAME} !-f [OR]
RewriteCond %{REQUEST_METHOD} !^GET*
2) I'm not 100% sure this next step is necessary, but I couldn't get it to work otherwise. Add the following directive,
IF the caveats in the
mod_dir documentation don't apply:
DirectorySlash Off
These two steps will tweak the way Apache views incoming URLs and delivers them to your page_cached, RESTful Rails app.
Other Complications
This bug is hard to track down because it's intermittent, happening only when there are cached pages or a cached directory. On the site where I had this problem, I had a Capistrano task that swept my cached pages whenever I redeployed the app. So you can imagine my agony when I'd make a change, redeploy, the app would work great, then an hour later when pages had been cached, everything stopped working. Argh!
ReferencesThis is about 20 hours of my life I'll never get back, so I hope this saves you some pain. The following references were extremely helpful in devising this solution:
Rails mailing list post #1
Rails mailing list post #2Mephisto wikiLabels: rails
Wednesday, May 30, 2007
I found a lot of
good tutorials on how to setup attachment_fu (which lets you easily store images or other binary data in a Rails app, either in the filesystem or in the database itself), but none of them explain how to write proper functional tests. Since a lot of other people have been posting this question, I thought I would post what I just got working:
def test_should_create_new_attachment
fdata = fixture_file_upload('/files/photo1.jpg', 'image/jpeg')
login_as :bob
assert_difference Photo, :count, 2 do
post :create, :photo => { :uploaded_data => fdata }, :html => { :multipart => true }
end
assert_redirected_to user_url(users(:bob))
assert_valid assigns(:photo)
end
The above assumes you create a 'files' directory inside of your fixtures directory. Also note that if you have thumbnailing enabled, each file you upload will create multiple attachment objects (in this case, one to represent the original image and one to represent a thumbnail).
If you're curious, here's how I have the plugin configured:
class Photo < ActiveRecord::Base
has_attachment :content_type => :image,
:storage => :file_system,
:max_size => 500.kilobytes,
:resize_to => '320x200>',
:thumbnails => { :thumb => '100x100>' },
:processor => 'ImageScience'
validates_as_attachment
end
Labels: rails, testing
Sunday, May 20, 2007
I've recently started using the rails_rcov plugin to test how well I am testing the code I am writing. It's been a great help, but I was getting very strange results until I read
this post about how to configure rcov properly to concentrate its testing. (Basically you have to force it to run your unit and functional tests in isolation which seems counterintuitive). It's easy enough to add parameters to your rakefile that will straighten it out.
Now I am going to try to get
Heckle working for even better testing of my tests.
Labels: rails, testing
Tuesday, May 15, 2007
So far in my career I've only developed web applications for internal use -- intranet sites and the like. Recently I gave myself the challenge to create a simple public web app over the course of the weekend. Writing the code took only the weekend, but getting the deployment configuration right, and optimizing to withstand just the small trickle of traffic I was seeing took two more weeks.
I thought I knew Rails and Apache and MySQL and other technologies inside and out, but once I had to really cope with real-world obstacles I was knocked on my ass. My original configuration was not fast enough, and I had to implement caching to make things more responsive. I set up a number of monitoring systems to automatically restart my dispatchers if something went wrong. I found an awesome
SSH client for my Treo which I had to use when my server started acting up and I was away from my home computer.
I also learned that the Internet is international -- I unconsciously assumed that most of my users would be based in the United States, but that's exactly wrong. I've got users from
Växjö to
Santiago de Compostela.
The site itself isn't much to look at -- I'm no graphic designer -- but it represents a major lesson for me in how to deploy a public-facing site. I'm greatly influenced by it while pursuing my other projects -- for instance, I now bake caching considerations directly into my code. I am not relying on flash[:notice] to pass messages back to the user. I'm either creating separate notification pages that can be cached, or I'm using Ajax to just update small parts of the page with status messages.
Labels: rails, web development
Sunday, April 22, 2007
I had a lot of trouble writing a test to check whether all elements of an array were showing up in a table properly; I needed more fine-grained testing than
assert_select was allowing me. I needed to guarantee that every element in an array could be found as a child of a particular table.
Fortunately, I noticed that Rails also has a
css_select method which returns an array of selected elements without running tests on them. This lets you run your own tests using the results of the selection.
This isn't the prettiest code in the world (I'm also testing a formatting helper, hence the literals '$200.00' and so on), so I'll probably refactor it later. There might also be a way to do this more elegantly with assert_select. But I think this does show the utility of css_select.
elements = css_select('table#investors tr td')
[[investors(:investor_8),'$200.00'],
[investors(:investor_9),'$343.40']].each do |i,amount|
assert_not_nil elements.detect \
{ |e| e.to_s =~ /#{i.name}/ }
assert_not_nil elements.detect \
{ |e| e.to_s =~ /#{Regexp.escape(amount)}/ }
end
Things to watch out for:
- You need to force the conversion of the element to a String for the regular expression comparison (with to_s)
- In this particular example, since I am using string literals with special characters, I had to use Regexp.escape to make $ and . match properly
Labels: rails, testing
Wednesday, March 14, 2007
I don't know how general this case is, but I've found that if you want to use acts_as_ferret in your production Rails app, you need to run your
Ferret server as a separate DrbServer process.
Backgroundacts_as_ferret gives you a powerful search capability, and it's much easier to implement than
MySQL full text querying.
Numerous tutorials (
here and
here) describe how to do this. In development mode, it's as simple as adding something like this to your model:
acts_as_ferret :fields => [ :city, :state ]
and this to your controller:
@users = User.find_by_contents(params[:keywords])
ProblemI started getting all kinds of corrupted index errors when I put this into production, because acts_as_ferret can't handle multiple separate processes access the index at once. This caused a lot of scrambling as I was starting to get a lot of traffic.
SolutionFortunately I discovered that acts_as_ferret comes with its own built-in DrbServer. As soon as I set that up, everything worked great.
The developer does a
great job explaining what to do. You add :remote => true to your acts_as_ferret declaration, setup a ferret_server.yaml config file, then run:
RAILS_ENV=production script/runner vendor/plugins/acts_as_ferret/script/ferret_server
...and you're golden.
UPDATE: To make the acts_as_ferret unit tests run correctly, add the following line to your test/environment.rb file:
AAF_REMOTE = true
Labels:
