MountainWest Ruby Conf 2010 Notes
I just finished attending MWRC2010, which I thoroughly enjoyed. The presentations were all excellent. Below are the notes I took during the presentations, really for my own reference…
Day One
Archeopteryx
@gilesgoatboy
Ruby library for making music
uses midi
gem install midilib — for files
gem install midiator — for live midi
google giles midi for github URL; use midi_files branch
foo.bar = lambda { return “bar”}
foo.bar[] #(square brackets can execute — not just .call())
book:
probabilistic reasoning in intelligent systems (judea pearl)
“reason” — the application with synths, drum machines, samplers
probability matrices for determining when different drums sound, to create cool random beats
Cooking with Chef
Recipes for sysadmin tasks
Can auto-restart services on changes to their config files
ERB
Managing Ruby Projects with RVM
command line Ruby Version Manager written in bash
install:
rvm 1.8.6,1.9.1,jruby install
use:
rvm use 1.9.1
gemset:
rvm gemset use rails3
(even different shells can use different gemsets)
tests:
rvm 1.8.6,1.9.1,jruby tests
(runs your tests, logs to log files)
monitoring:
rvm 1.8.6,1.9.1,jruby monitor
(watches your test directory, runs changing tests — like autotest)
benchmark:
rvm 1.8.6,1.9.1,jruby benchmark my_code.rb
raking:
rvm 1.8.6,1.9.1,jruby rake the:rakes
git is recommended for installing rvm
rvm update –head # get this often
use a .rvmrc file
rvm –create –rvmrc use 1.9.1%projecta
rvm info
rvm gemset export projecta.gems
can copy this gem file into another project and do rvm gemset import to use the same gems in another project
#rvm on freenode
Rack
github.com/mjijackson/mwrc2010
mjijackson.com
TODO: Read the spec:
http://rack.rubyforge.org/doc/SPEC.html
an app just needs a call method to run on rack
middleware has an initialize(app) method and a call(env) method
# using ruby
require ‘rubygems’
require ‘rack’
app = Rack::Builder.new {
use Rack::ContentType
use Rack::ContentLength
run lambda {|env| [200, {}, ["Hello world!"]] }
}.to_app
Rack::Handler::WEBrick.run app, 
ort => 9292
# using rackup
# just a few lines in config.ru
# using shotgun
# auto-reloading version of rackup
TODO: see examples on github. check out twitter cache app — makes sure you don’t hit twitter too often
debugging libraries
Rack::Lint
Rack::ShowExceptions
racksh
Rack::Bug
TODO: Check out Rack::Bug
useful middleware
rack/lib — streaming, chunked http responses, etag headers, sessions, cookie handling
rack-contrib
search for “rack” on GitHub
Testing libraries
Rack::MockRequest and Rack::MockResponse
rack-test
Modularity
Yehuda Katz
more in-depth information on blog
“Agile” –
basic plan
build app
requirements
build app
requirements
x n
build app
MODULARITY (figure it out later)
alias_method_chain (starts to fall down in the face of reality)
Reduce Assumptions
so that we can
Increase Reuse
(modularity should only be added in order to increase reuse)
Constants are essentially Global Variables
Rails 2.3: include ActionController::Routing::Routes
Rails 3.0: include application.routes.route_helpers
Rule #1 Eliminate things that are global
Tests are the canary in the coal mine
Be suspicious of any use of Constant.foo, including Constant.new
rails 2.3 — @@perform caching (did not allow changing per request)
rails 3.0 — class_attribute perform caching (now can do controller.perform_caching per request)
Rule #2 Use Objects
Stop worrying about allocations. Object allocation is essentialy free
Don’t hardcode the specific class
Objects present opportunities to memoize
Again, tests are the canary in the coal mine
Rails 2.3 — Template.new — takes a path (hard-coded to file system. hard to store templates in the database).
All the way up the stack takes a path
Rails 3.0 — Entire system is storage agnostic. Template.new takes “Source”, which could be from a database, file, it doesn’t care.
Rule #3 Use Modules (instead of alias_method_chain)
BAD:
class Person
def speak(word)
puts word
end
end
class Yeller
alias_method_chain :speak, 
ld_speak
def speak(word)
old_speak(word.upcase)
end
end
GOOD:
module Speaker
def speak(word)
puts word
end
end
module Yeller
def speak(word)
super word.upcase
end
end
class Person
include Speaker
include Yeller
end
Use ActiveSupport::Concern — mechanism for making sure module includes happen in the right order
How HTTP Solved all your performance problems 10 years ago
Paul Sadauskas
TODO: Read HTTP spec
github.com/paul/mwrc2010
TODO: See last slide with Links — Caching Tutorial, Doing HTTP Caching Right, RFC2616 Section 13
Performance Features in HTTP/1.1
Persistent Connections (Pipelining, Keep-Alive)
built-in to Apache (Keep-alive on) and nginx
Caching
http headers —
Expires and Cache-Control: max-age
(google js libraries are set to a year from now)
(you can also set it in rails)
Validation: Last-Modified, Etag, Vary
Last-Modified — User Agent can use If-Modified-Since to make a request conditional
Etag — If-None-Match
fresh_when(:etag => @sombrero…)
if stale?(:last_modified => @obj.last_modified) #render else return :304 end
Vary Header — keep the cache from server private stuff to the wrong person
Cache-Control public, private
Manipulate: Cache-Control
Hierarchical Proxies
load-balancing proxy (such as haproxy)
caching proxy — provide previous responses without hitting the origin server (squid, varnish)
REST – written by one of the authors of the HTTP spec at the same time he was writing the spec
HATEOAS – Hypertext as the engine of application state (rather than using the session)
Resource – Anything identified by a URI; can have an HTTP verb performed on it; can have multiple representations (HTML, JSON, etc)
Proper Cache treats each resource separately.
POST/PUT/DELETE to a resource invalidates a cached response.
Misusing session makes things uncacheable (for example, flash messages)
if you’re using cookies for auth, need to Vary on the Cookie
if session is in the cookie, the cache is less effective
if page content is generating from session, cache will be invalid
Rails has defaults that support the worst-case behavior.
Tweak defaults when you know you need to be caching
Ruport
Gregory Brown
# No notes for this – it was more of a code walkthrough of how
# you can take a fairly simple concept and make a horribly
# complicated mess out of it 
Documentation
YARD — documentation tool
Documentation is important
* it makes you actually think about your API
Rule #1 – Consistent
Rule #2 – Correct
* Documentation should be audited/tested just like code
Rule #3 – Coherent
* Documentation should be reviewed just like code
YARD
Consistent
Meta-data @tags
Correct
Roll your own auditing tools — for example github.com/dkubb/yardstick
Coherent
Extensible
gem install yard-rspec — your specs in your docs
gem install yard-sinatra — sinatra routes
gem install yard-pygments — source highlighting for diff langs
gem install yard
host docs at yardoc.org
Code of Art 2 – ruby-processing
github.com/jcasimir
creating images with ruby-processing
beanstalkd — simple, fast task processing
port install beanstalkd
gem install beanstalk-client
Textorize
book – Learning Processing
examples in ruby-processing on github
Lightning Talks
Ship code every single day
– Have Clear Rules
– Lower your Standards
– Get the skill then stop
sequel-mongo, nosql.rb compare to SQLite – sequel-mongo can do all the same thing, gets the same results as sqlite
ci.pivotallabs.com — open sourced on github.com/pivotal/pulse
– ci.rubyonrails.com or something — shows state of ror on different envs
– sad dog pic that the tests don’t pass
memprof.com/demo
– web-based heap visualizer & leak analyzer
sorenson – easy video capture
– squish – captures video, compresses, uploads video
– developer.sorensonmedia.com — free test account
– mtnwest.integrumdemo.com — demo
– mflathers@sorensonmedia.com – email to ask for addl capabilities
– supports flash and mp4 output
HTML5 “Offline” – Reduce Download (for mobile)
– GET request, Accept Application/JSON, respond_with @articles
– update page in js
– text/cache-manifest
– “Local Storage”
– have something that shows up right away, and pull content off local storage
github.com/remi/presentations
– Rack::Payment (on github) — easy ecommerce
– gem rack-paymentd
– “kinda alpha”, but super easy to use
– creates the default payment form and everything for you
managing projects with gnu-screen
– terminal multiplexer
– screen manager
– starts things up for you
– github.com/timcharper/screenmanager
SimpleGeo
– heroku for geo stuff
– restful interface
– geotagged twitter firehose
– stickybits uses it
– austin.vicarious.ly
– open source
– something about openstreetmap in the future
RailsGen
– http://railsgen.com
– RailsBoost++
– more social
ButterflyNet
– www.butterflynet.org
– github – quartzmo
– irb creates tests
– quartzmo@gmail.com
– help requested
CoffeeScript
– compiles down to js
– kind of ruby-like
Google app engine – easy deployment
– sudo gem install google-appengine
– appcfg.rb generate_app mwrc_emo
– cd mwrc_demo
– dev_appserver.rb
– appcfg.rb update .
– now it’s published and running…
– curl command to generate rails app
Day Two
Ruby 124C41+
Matz
from Ralph 124C41+ book — Hugo Gernsback
ruby started in 1993
took 6 months for “hello world”
released publicly Dec 21, 1995 (same age as java)
future -
faster
* multiple vm
* gc improvement
* JIT?
more powerful
* distributed programming
* faster IPC (messagepack)
* multi-core aware
broader
* small -> embedding
* large -> HPC
more modular
* selector namespace – override things only in certain scope, in order to not break other things
* traits – operational combination of features (methods) — module operation +, -
* map etc do not work for infinite enumerable. mapx (assumed name) would return enumerator — lazy list
ruby-core mailing list
From Matz’s screen –
“Not only is the options hash not documented, it is not working either.”
Macros in Ruby
evaluation at parse time
macro add(a,b)
^a + ^b )
end
p add(1,2) => p 1 + 2
“print ‘hello’”
print ‘hello’)
proc{ print ‘hello’ }
RedParse
redparse -e ‘a(1)’
redparse -e ‘if a: b end’
redparse -e ‘def of; bar baz end’
slow
Ruby Techniques by Example
Jeremy Evans
1. extending code
class Person
def self.all
end
def name
end
end
hard to override
method lookup – singleton class, methods in class, methods in modules included in class, suerclass
module PersonPlugin
module ClassMethods
end
module InstanceMethods
end
end
use plugin method to extend/include
class Person
plugin PersonPlugin
end
2.
class variables are shared throughout the object hierarchy
3. DSL
ruby 1.9 — BasicObject fix (BasicObject does not have access to constants from Object)
def self.const_missing(name)
::Object.const_get(name)
end
add file and line position to block to trace error
opts.has_key? (instead of logical check)
rescue => e
ensure
blah if e
blahblah
end
Hubris
James Britt
haskell
* purely functional
* type inferencing
* referential transparency – variables don’t vary (like erlang)
this guy’s slides are awesome (illustrator)
immutable data, no state, higher order functions. based on lambda calculus
hubris – ruby – haskell bridge. call haskell code from your ruby code
why? – speed up code, learn haskell, powerful abstractions, parallelism
mandelbrot example – 23 sec in ruby; 3 sec in haskell
TODO: www.lisperati.com/haskell
hubris is on github. repo includes examples.
require ‘hubris’
class Target
hubris :inline => “triple::Int=>Int; triple n = 3*n”
end
t = Target.new
puts t.triple(10)
inline – you don’t have control over visibility of your methods
instead can use external code
GHC 6.12
MRI 1.8.x or 1.9
(see github wiki for instructions on building order)
right now – does not like 64 bit (won’t run on snow leopard)
James runs it on a vm
hubris
* grabs haskell
* compiles it into a shared object
* exposes object via ruby shared lib
2 parts: ruby hubris + haskell hubris
quirks:
functions exposed via hubris can only take one argument (they are working on this)
ruby string objects are mapped to haskell bytestrings
Cross-Platform Mobile Applications with Ruby
Sarah Allen
@ultrasaurus
different platforms, different languages as a mobile developer
CDD “Conference Driven Development” github.com/blazingcloud/rhodes_rubyconf
demo of same app on iphone, nexus 1, blackberry
mobile phones have extra features that you should use:
geolocation, camera, connected, everyone you know
mobileruby.heroku.com
rhomobile platform
MVC, but it doesn’t go out to the web — this is all local
request => RhoController -> Rhom -> Local DB
response < -- html erb files <-- render <-- RhoController
rho/rhoapplication
tabs defined in application. each item maps to a controller
MapView - crossplatform rhodes api
phonegap - html + javascript: free. fast-changing. not great with blackberry platform
rhodes - gpl - you have to open source your code. $500 for client-side $5000 for sync server (most applications don't need this)
Person.find -- similar to active resource. if keeping data locally, you don't need the sync server
Mongo DB Rules
document based database, open source, C++
Kyle Banker
@hwaet
mongodb.org
google group — mongodb-user
1. get to know the real mongodb (don’t jump straight to the ORM that hides the details)
TODO: http://try.mongodb.org — play with an instance
or using ruby driver
mongo id – timestamp/machine id/process id/counter
BSON – bsonspec.org — binary format
when we call save:
* add an _id
* serialize to BSON – format the data is stored in
* socket.send()
so we don’t have to wait for the response to get the ID
2. use object id
MongoDB standard
3. use rich documents
one document can contain all your data — “human oriented” — doesn’t require 30 joins to different tables
as a traditional database would
dynamic queries
@orders.find(:user_id => 3)
@orders.create_index(’line_items.sku’) <-- indexing an inner object (line items is an array))
@orders.find({'line_items.sku' => ‘rbi-re2′})
query operators – $ne $in $nin, etc
4. array keys rule, too
simplify tiny relations
link = { :title => ‘Hacker News’, :url => ‘blah’, :tag => {’time water’, ‘blah} }
query all links with certain tag can be indexed
many-to-many
5. Use atomic operators
findAndModify Comman
$inc, $set, $push, $pop
6. map-reduce is for aggregation. results in a new collection
map = <<-MAP
function() {
emit(this['shipping_address']['zip], {total: this.total})
}
MAP
js -- mongo understands js functions
7. indexes are indexes
collections support up to 40 indexes
similar to relational indexes
don’t build too many
build compound indexes
8. GridFS
specification for storing binary data in mongodb
simple api to read in binary file
9. Replicate and backup
10. Performance notes
ruby driver does a lot of work
embedded documents
— db doesn’t do joins
queries should use indexes
keep indexes in RAM
11. auto-shard, to keep it in RAM
wow, data can be automatically sharded.
mongos – lightweight client, redundant config servers, shards of data
12. Two production cases
sourceforge.net – store all their product pages. each page is in a single document.
github.com — backend analytics
harmonyapp.com — originally mysql, switched to mongo. CMS – supports dynamic data needed for a CMS
vs. couch
couchdb — http, mongo – binary protocol over socket
couch – indexes via map-reduce language. mongo – queries ad-hoc, build indexes more like mysql
mongo – supports atomic updates
trivial relationships should be in its own document
can normalize a little bit
everything UTF-8
have an explain for optimization
roadmap
2d geo indexing
sharding beta – 100% automatic failover
your system’s binary interface
joe damato
http://timetobleed.com
only do this stuff in a vm
it’s very easy to leak references in your ruby code
–a lot of memory
–gc scans each object, wasting time
–how to track them down?
ABI – application binary interface
* alignment
* calling conventions
* object file and library formats
* syscalls
evil devices
nm – dump symbol table
objdump – disassemble objects
readelf (linux)
dwarfdump – debugging info
nm /usr/bin/ruby
objdump -D /usr/bin/ruby
readelf -a /usr/binruby
dwarfdump -a /usr/bin/ruby
registers
%rax – results = 64 bits, 1quadword
%eax – lower 32 bits of %rax
%ax – lower 16 of eax
%ah
%al
ELF objects
* headers
* memprof uses libelf extracting info
.text – code lives here
plt – procedure linkage table
ruby calls rb_newobj – to allocate object, add_freelist – deallocate
hook rb_newobj objdump output – find rb_newobj in memory, and override it
only works for ruby build with –disable-shared (no library.so)
require ‘memprof’
Memprof.start
require “stringio”
SringIO.new
Memprof.stats
tells you exactly which file, line number, class the objects are coming from
can track a block
or dump entire heap as JSON
require ‘memprof/middleware’
- watches your rails stuff
memprof.com
memprof gem – free
github.com/ice799/memprof
don’t gem install memprof (out of date)
#memprof on irc.freenode.net
memprof.com – separate — reads the output
someone recommended reading
Programming From the Ground Up
A Tale of Two Codebases
Pat Maddox
in ruby, happiness matters
business case for happy developers
happy developers are more productive
attracts other good programmers
agile, fast development is good
but it’s important to maintain a sustainable pace
demand excellence
– there is always time to refactor
– there is always time to get the tests working
– push back on business requirements to make time
themed releases — focus on one area at a time
– don’t bite off more than you can chew
– it’s draining to try and improve lots of little things at the same time
– pick items in a similar area from backlog
– gives the opportunity to make all the internal changes you need
technical debt is a myth
– the best time to solve a problem is right now, when you know what it is
– technical debt is often used as more of an excuse
– improve things any time you have the chance
don’t lose sight of passion for ruby & code
Recommended Reading:
smalltalk best practice patterns
design patterns
refactoring
EventMachine
scaling non-blocking i/o in ruby
Aman Gupta
github.com/tmm1
@tmm1
slides on timetobleed.com
EventMachine – implementation of the Reactor pattern
heavy production use
* I/O
* mysql query responses
* http responses
* memcache results
most web apps are I/O bound
basic idea – instead of waiting, use that time for other stuff
–scaling I/O heavy applications
event machine takes blocks of code to execute asynchronously
queue.on_push = proc{ use(queue.pop)}
gem install eventmachine
require ‘eventmachine’
EM.run{
puts “reactor started!”
} # this is blocking. nothing else will run until the reactor is stopped
EM.next_tick – queue a proc to be executed on the next iteration of the reactor loop
— splitting up work across multiple patters
— common pattern – “recursive” calls to next_tick
— tickloop
— not used too often
EM::Iterator
by default, EM does not use threads. EM.defer can set up threads
use EM::Timer instead of sleep (b/c sleep would block the reactor)
EM::Callback – specify event handlers
EM::Deferrable (!= EM.defer)
even if event already fired, the block gets automatically executed
your callback always gets called
EM::Queue – returns async queue
pop takes a block
workers pop items off the queue
items can be processed in parallel
EM::Channel – similar to queue, but publish/subscribe model where multiple things can subscribe
EM.system – run external commands without blocking. block receives stdout and exitstatus
EM Handlers – it is recommended to use these
EM::Connection – networking
EM::Protocols — send_object, receive_object — handles marshalling, unmarshalling of objects
can watch files or processes with the reactor
could use an EM enabled webserver — Thin or Rainbows!
async_sinatra for streaming and delayed responses — good for long polling
here’s the code:
http://gist.github.com/329682
Lightning Talks
Testing Queries
– @ginnyhendry
– Never do an unindexed read
– This test will make sure you always have a good index on your query
– gist.github.com/328822 — mongo specific
git instaweb
– git instaweb –httpd webrick
– nice github like interface without needing internet
– already installed on your system!
ruby-fs-stack gem
– tap into the world’s geneology
– FamilySearch
– new.familysearch.org — collaborative family tree
– REST-like Web Service
- ruby gem: http://github.com/jimmyz/ruby-fs-stack
– Family Search Developers Conference in April
RubyMine
– IntelliJ Ruby IDE — about $100
– nice documentation, code completion, etc
Geo Tweeter
– geo-tweeter.heroku.com
– set or fake out the geolocation on tweets
– sinatra-twitter-oauth
Command Event DTO Architecture
– speaker – Alistair Cockburn
– Hexagonal architecture – insulate app from outside world
– the app never knows anything about the external world
– user
– database
– @gregyoung
– Command Event DTO Architecture
– greg put another hexagon around the database, and app and db talk to each other through message queue
Bash Deployment & Server Manager (BDSM)
– Wayne E. Seguin (RVM guy)
– Service Management via Extensions, and Deployments
– modular — any executable file, template system, config, etc
– deploy, rollback, update
Gitty
– Unobtrusively extending Git
– install hooks from common repository & share hooks with team
– example of hook — don’t allow commits with trailing whitespace
– github.com/timcharper/gitty
Agile – Back to the Future
– Agile is developer-geared
– Wall of Confusion
– agileroots.com
Donkey
– Howard Yeh (hayeah)
– Concurrency == Reliability == Scalability
– Learn from Erlang — use message-passing Actors
– based on EventMachine and AMQP
