Important Update
The Guide Feature will be discontinued after December 15th, 2023. Until then, you can continue to access and refer to the existing guides.
Author avatar

Adam Fortuna

Response Streams with Rails 4 and Redis

Adam Fortuna

  • Jan 10, 2019
  • 5 Min read
  • 8,589 Views
  • Jan 10, 2019
  • 5 Min read
  • 8,589 Views
Ruby
Ruby on Rails

Introduction

If you're on the fence about updating an older application to use Rails 4, the addition of ActionController::Live might be helpful in making your decision a little easier. It enables keeping a connection open to your server, which can then respond with partial updates with ease. This bridges one of the bigger gaps that causes people to choose node.js over Rails for projects.

A Basic Redis Connection

Aaron Patterson wrote a great post about Live Streaming in Rails, over a year ago, but the interface is mostly the same today. That post is still a good starting point for ActionController::Live.

I first ran into the subject when working on Code School's Rails 4: Zombie Outlaws course. The last level is all about streaming, with a mention towards the end about using streaming with Redis. If you connected to this endpoint in a browser, the page would load forever and occasionally send back responses to the browser.

1class ActivitiesController < ApplicationController
2  include ActionController::Live
3
4  def index
5    response.headers["Content-Type"] = "text/event-stream"
6    redis = Redis.new
7
8    redis.psubscribe("user-#{current_user.id}:*") do |on|
9      on.pmessage do |subscription, event, data|
10        response.stream.write "data: #{data}
11
12"
13      end
14    end
15  rescue IOError
16    # Client disconnected
17  ensure
18    redis.quit
19    response.stream.close
20  end
21end
ruby

Here's a quick recap of what's going on:

  • We're using Puma which allows for concurrent connections to the server.
  • We create a new connection to Redis. This is important because when we call psubscribe, that connection is locked and can't do anything else.
  • Use psubscribe to subscribe to all messages for this user by using an expression. Elsewhere in the application, we are publishing messages to this same channel.
  • When a message is received, it's passed down to the client. In this case we're passing down JSON.
  • ensure that the redis connection is quit and the response is ended.

The Problem

If you wrote the above code and opened up that action in a browser, it would actually work fine -- until you tried to load the page again. At that point, there would be two connections open from the servers standpoint, but only one active. This is due to the fact that the server doesn't know that the client disconnected.

That IOError error isn't triggered when the client disconnects, as you might expect, but instead when the server attempts to write to the response.stream only to find that it is no longer active. Turns out this is a well discussed problem That leaves us with a few options on how to test if the client has disconnected:

  • Have the server connection timeout every minute or so (if you're on Heroku, my guess is this will automatically happen).
  • Ping the client every few seconds to see if they are still there.

A Working Solution

I ran into a StackOverflow post on this exact topic, which led to a working solution for this. This solution follows the "ping" method.

1class ActivitiesController < ApplicationController
2  include ActionController::Live
3
4  def index
5    response.headers["Content-Type"] = "text/event-stream"
6    redis = Redis.new
7
8    ticker = Thread.new { loop { sse.write 0; sleep 5 } }
9    sender = Thread.new do
10      redis.psubscribe("user-#{current_user.id}:*") do |on|
11        on.pmessage do |subscription, event, data|
12          response.stream.write "data: #{data}
13
14"
15        end
16      end
17    end
18    ticker.join
19    sender.join
20  rescue IOError
21    # Client disconnected
22  ensure
23    Thread.kill(ticker) if ticker
24    Thread.kill(sender) if sender
25    redis.quit
26    response.stream.close
27  end
28end
ruby

This solution is based on the idea that the server will know the client has disconnected when it attempts to write to it only to find it's not there. In this case we open up two threads -- one that does our Redis subscription and another that handles making sure the client is still there.

Closing the Database

One downside of keeping the connection open is that if you're using ActiveRecord, that connection will not be released until the request is complete. During the Redis subscribe phase, if you don't need to keep that connection open, you can return the current connection to the connection_pool.

1ActiveRecord::Base.connection_pool.release_connection
ruby

If you set this up to run in a before filter, and do any database communication before that, you shouldn't run into database connection limits.

Update

For an example of how this technique is used, read the post on Teaching iOS 7 at Code School. This post details the user experience that can be achieved using response streams. Guide was originally posted here.