Boring Rails: Skip the bullshit and ship fast |

Beautiful Rails confirmation dialogs (with zero JavaScript)

2025-12-15T13:00:00+00:00

This is a guest collaboration with Stephen Margheim, creator of High Leverage Rails, a video course on building high-quality Rails applications with the power and simplicity of SQLite, HTML, and CSS.

Turbo’s data-turbo-confirm attribute is convenient for quick confirmation dialogs, but the native confirm() prompt it triggers looks dated and out of place. If you want a styled confirmation dialog that matches your app’s design, the traditional approach recommends a lot of JavaScript — a Stimulus controller to open and close the dialog, event listeners for keyboard handling, and coordination between the trigger and the modal.

But, recent browser updates have changed the game. Invoker Commands landed in Chrome 131 and Safari 18.4, giving us declarative dialog control. Combined with @starting-style for animations, we can now build beautiful, animated confirmation dialogs without writing any JavaScript.

Feature	How…
Open dialog	`command="show-modal"` on a button
Close dialog	`command="close"` on cancel button
Escape key	Built-in browser behavior
Light dismiss	`closedby="any"` attribute
Enter animation	`@starting-style` CSS rule
Exit animation	`allow-discrete` on `display` transition

Let’s imagine we wanted a confirmation dialog for when a user decides to delete an item in our app. Here is how we could build such a dialog today using modern browser features with zero JavaScript:

<button type="button" commandfor="delete-item-dialog" command="show-modal">
  Delete this item
</button>

<dialog id="delete-item-dialog" closedby="any" role="alertdialog"
        aria-labelledby="dialog-title" aria-describedby="dialog-desc">
  <header>
    <hgroup>
      <h3 id="dialog-title">Delete this item?</h3>
      <p id="dialog-desc">Are you sure you want to permanently delete this item?</p>
    </hgroup>
  </header>

  <footer>
    <button type="button" commandfor="delete-item-dialog" command="close">
      Cancel
    </button>
    <%= button_to item_path(item), method: :delete do %>
      Delete item
    <% end %>
  </footer>
</dialog>

The command attribute tells the browser what action to perform, and commandfor specifies the target element by id. With command="show-modal", clicking the button calls showModal() on the target dialog. The cancel button uses command="close" to call the dialog’s close() method. Note the cancel button is type="button", not type="submit" — we don’t want it participating in any form submission.

Modal dialogs opened with showModal() automatically close on Escape. The browser handles it. Adding closedby=”any” enables “light dismiss” — clicking the backdrop closes the dialog too.

The aria-labelledby and aria-describedby attributes connect the dialog to its heading and description. Screen reader users hear both the title and explanatory text announced immediately, giving them full context before making a decision. And role="alertdialog" signals the dialog is a confirmation window communicating an important message that requires a user response.

Interactive demo

Here’s a working demo showing how different close mechanisms work. Try clicking “Delete Item”, then close it different ways: click Cancel, click Delete, press Escape, or click the backdrop. Notice how returnValue is only "confirm" when you click the Delete button.

Event Log

So much functionality with nothing but declarative HTML! I love it.

Adding Animations with @starting-style

For a polished feel, add smooth enter/exit transitions. With @starting-style and allow-discrete, we can animate dialogs purely in CSS.

The @starting-style rule defines the initial state when an element first appears. Without it, the browser renders the dialog immediately in its final state. With it, the browser starts from opacity: 0; scale: 0.95 and transitions to opacity: 1; scale: 1, for example.

For exit animations, we need transition-behavior: allow-discrete on display and overlay. Most CSS properties are continuous — opacity can be 0.5, colors can blend. But display is discrete: it’s either none or block, with no intermediate values. Historically, this meant display changes couldn’t animate.

The allow-discrete keyword tells the browser to apply transition timing even for discrete properties. For closing animations, the browser keeps the element visible, runs the exit transition, then flips to display: none only after the transition completes. The overlay property works similarly — it controls whether the dialog stays in the top layer during the transition.

dialog {
  opacity: 1;
  scale: 1;

  transition:
    opacity 0.2s ease-out,
    scale 0.2s ease-out,
    overlay 0.2s ease-out allow-discrete,
    display 0.2s ease-out allow-discrete;

  @starting-style {
    opacity: 0;
    scale: 0.95;
  }
}

dialog:not([open]) {
  opacity: 0;
  scale: 0.95;
}

dialog::backdrop {
  background-color: rgb(0 0 0 / 0.5);
  transition:
    background-color 0.2s ease-out,
    overlay 0.2s ease-out allow-discrete,
    display 0.2s ease-out allow-discrete;

  @starting-style {
    background-color: rgb(0 0 0 / 0);
  }
}

dialog:not([open])::backdrop {
  background-color: rgb(0 0 0 / 0);
}

Browser Support

Feature	Chrome	Safari	Firefox	Can I Use?
`command`	135+	26.2+	144+	link
`commandfor`	135+	26.2+	144+	link
`@starting-style`	117+	17.5+	129+	link
`closedby`	134+	Not yet	141+	link

Safari support for closedby is still pending. For production use today, add a polyfill: dialog-closedby-polyfill

If you need invoker command support for older browsers, there is also a polyfill for that: invokers-polyfill

Both polyfills are small and only run when native support is missing.

Integrating with Turbo’s Confirm System

Now, what if you want to keep using Turbo’s data-turbo-confirm attribute while getting a styled native dialog?

Turbo provides Turbo.config.forms.confirm for exactly this. Mikael Henriksson has an excellent writeup on this approach and Chris Oliver has a GoRails video as well.

First, add a dialog template to your layout, which you can of course style however you’d like using whatever CSS tooling you have in your app:

<%# app/views/layouts/application.html.erb %>
<dialog id="turbo-confirm-dialog" closedby="any"
        aria-labelledby="turbo-confirm-title" aria-describedby="turbo-confirm-message">
  <header>
    <hgroup>
      <h3 id="turbo-confirm-title">Confirm</h3>
      <p id="turbo-confirm-message"></p>
    </hgroup>
  </header>

  <footer>
    <button type="button" commandfor="turbo-confirm-dialog" command="close">
      Cancel
    </button>
    <form method="dialog">
      <button type="submit" value="confirm">
        Confirm
      </button>
    </form>
  </footer>
</dialog>

The Confirm button is type="submit" inside a <form method="dialog">. When submitted, the browser closes the dialog and sets returnValue to the button’s value attribute. This is how we detect which button was pressed — no JavaScript event coordination needed.

Then configure Turbo:

const dialog = document.getElementById("turbo-confirm-dialog")
const messageElement = document.getElementById("turbo-confirm-message")
const confirmButton = dialog?.querySelector("button[value='confirm']")

Turbo.config.forms.confirm = (message, element, submitter) => {
  // Fall back to native confirm if dialog isn't in the DOM
  if (!dialog) return Promise.resolve(confirm(message))

  messageElement.textContent = message

  // Allow custom button text via data-turbo-confirm-button
  const buttonText = submitter?.dataset.turboConfirmButton || "Confirm"
  confirmButton.textContent = buttonText

  dialog.showModal()

  return new Promise((resolve) => {
    dialog.addEventListener("close", () => {
      resolve(dialog.returnValue === "confirm")
    }, { once: true })
  })
}

The JavaScript does only three things:

set the message text,
customize the button text if provided, and
open the dialog.

Everything else — closing on button click, closing on Escape, closing on backdrop click, determining which button was pressed — is handled by the platform.

The fallback to native confirm() ensures your app still works if the dialog element is missing (e.g., on a different layout or error page).

Turbo.config.forms.confirm expects a function returning a Promise that resolves to true (proceed) or false (cancel). The function receives three arguments: the confirmation message, the element with the data-turbo-confirm attribute, and the submitter element. We listen for the close event and check returnValue. Write this handler once, add one dialog to your layout, and every data-turbo-confirm in your app uses it.

You can customize the confirm button text per-trigger using data-turbo-confirm-button:

<%= button_to item_path(item),
              method: :delete,
              data: {
                turbo_confirm: "Are you sure you want to delete this item?",
                turbo_confirm_button: "Delete item"
              } do %>
  Delete
<% end %>

This produces a more contextual confirmation dialog with “Delete item” instead of a generic “Confirm” button — better UX that makes the action clear.

Addendum: Preventing Background Scroll

One common modal requirement is preventing page scroll while the dialog is open:

body:has(dialog:modal) {
  overflow: hidden;
}

The :modal pseudo-class matches dialogs opened with showModal(). Combined with :has(), this selector targets the body only when a modal dialog is open. When the dialog opens, scrolling stops. When it closes, scrolling resumes. The browser handles the coordination.

Hotwire components that refresh themselves

2025-07-07T13:00:00+00:00

This is a guest collaboration with Jesper Christiansen, a long-time fan of most things Ruby on Rails.

Earlier this year, I made a quick tweet about a pattern that I think makes working with Hotwire apps much better.

When you need to make a bit of UI that runs some code in the background, you can use turbo_streams to ‘refresh’ the front-end when the work starts, is in-progress, and finally when it’s finished.

But the problem is that, out of the box, turbo_streams use partials. And frankly, it just kind of sucks to work with. I found myself annoying by having to match up dom_ids across files and passing in data as locals. And it’s hard to search the codebase for partials, leading to cases where you change a partial and break something elsewhere just because it wasn’t easy to find usage.

Over time, this has lead me toward a pattern that I hadn’t seen elsewhere:

Make a component for the view (and associated logic/helpers)
When you do something on the backend, use turbo_stream.replace and take advantage of the renderable option to pass the component object instead of a partial + locals
Do the same thing later when broadcasting from a job
Let the component encapsulate implementation details like the dom_ids and ActionCable broadcast channels

So let’s dive into the details!

Hotwire: Components should refresh themselves

Let’s say you have a user card component that show some basic information about the user, as well as if we have sent them an introduction email or not.

We want the card to have a button that will send the user an introduction email when clicked - with in-progress state shown in the component, until we’re done sending.

We would like to keep the user card updated in the places where it’s shown, and we do that using Hotwire’s refresh functionality.

A tangled mess of identifiers

You’ll typically perform those refreshes with something like this in a background job (or model):

Turbo::StreamsChannel.broadcast_replace_to(
  "my-unique-identifier",
  target: id,
  partial: "user/card"
  locals: { user: @user }
)

This isn’t bad at all! But.. What is this "my-unique-identifier" string we refer to? That’s of course the id of the element we need replaced.

So inside our partial we have something like

<% tag.div id: "my-unique-identifier" do %>
 ...
<% end %>

Of course in practice we’d use the dom_id method from ActionView::RecordIdentifier - so we can do dom_id(@user, :user_card).

We’ll go ahead and change the places that refresh our user_card partial to use ActionView::RecordIdentifier.dom_id(@user, :user_card) instead of the previously hardcoded my-unique-identifier-string.

Not too bad, but less than ideal! In this simple walkthrough of how to implement this functionality, we’ve now already run into the issue; that when the id of the component changes, we need to go through our code-base and update all the places that referenced this magic string. Fingers crossed that our tests are in place and we didn’t miss anything! 😅

In a bigger app, there might be many places where we need to perform updates like these.

A way of cleaning this up could be to introduce a new method in a user_helper.rb or similar, something like:

def user_card_id(user)
  dom_id(user, :user_card)
end

Then we could use said helper method in all the places we need it. It’s better! But it feels decoupled from the core logic of the user_card partial.

If in the future we need to get rid of the partial, we’ll have to remember to clean this up or it’ll be a piece of history that isn’t used anymore. Sigh.

And we haven’t even talked about turbo_stream_from [@user, :user_card_refresh] which is another thing that needs to be in sync in places where it’s needed - and it might even be conditional on some state.

View Components to the rescue

I find something like ViewComponent (..or Phlex, or whatever gets you excited) great for use-cases like these. We can build components with a greater level of complexity and have everything packaged neatly into a single Ruby class.

This means that component isn’t just responsible for what it renders, but it can also become responsible for refreshing it’s own content.

Are we mixing responsibilities and breaking the sacred Single Responsibility Principle? Probably, but I find that I much prefer working in systems that value locality of behavior these days.

Let’s take a look at our simple UI::UserCard.

(Side note: I love using UI module for components because it’s super clear this object is a view component and it lets you drop the “Component” suffix that people tend to use for every…single…component in their apps. Highly recommend!)

class UI::UserCard < ApplicationComponent
  def initialize(user:)
    @user = user
  end

  def id
    dom_id(@user, :user_card)
  end

  def broadcast_channel
    [@user, :user_card_refresh]
  end

  def broadcast_refresh!
    Turbo::StreamsChannel.broadcast_replace_to(
      broadcast_channel,
      target: id,
      renderable: self,
      layout: false
    )
  end
end

We can then use our id and broadcast_channel methods in our component’s view:

<% tag.div id: id do %>
  <%= helpers.turbo_stream_from broadcast_channel %>

  <div class="text-lg font-bold"><%= @user.name %></div>
  <div class="text-sm text-slate-500"><%= @user.email %></div>
<% end %>

We’re now referencing the id from our component, and streaming from our broadcast_channel.

This simplifies things quite a bit in places where we’re performing updates which causes us to want to re-render the UI::UserCard for a given user.

We can now make use of the broadcast_refresh! method on the component itself. The following code can be called from a background job, the model or wherever you’re making an update you want to broadcast to other users.

UI::UserCard.new(user: @user).broadcast_refresh!

The component knows how to refresh itself and all of the “magic” identifiers we relied on are kept in one central place within the component itself.

That way we can use it in our controller when we send a new introduction email to a user:

def create
  @user = Current.account.users.find(params[:id])
  @user.send_introduction_email_later!

  user_card = UI::UserCard.new(user: @user, sending_email: true)
  render turbo_stream: turbo_stream.replace(user_card.id, user_card)
end

This ends up enqueuing a background job via the send_introduction_email_later! method on the user and it responds with a turbo_stream to replace the UI::UserCard for the user that clicked the button that triggered this action.

Notice how we’re making use of the same id method on the user component as we use in the refresh example. So if we ever change this in the future, we need to update it in one place.

Do we need to keep the stream open for eternity?

Keeping the stream open for forever might not be ideal - especially since we only really want to update the sending state for a user, and nothing else.

Let’s introduce a sending_email? state to our UI::UserCard to show that we’re sending an email. We can also use this state to only stream from the channel when we’re in the middle of sending an email.

Let’s update our component’s initialize with a sending_email? method

class UI::UserCard < ApplicationComponent
  def initialize(user:, sending_email: false)
    @user = user
    @sending_email = sending_email
  end

  def sending_email?
    @sending_email
  end

  def introduction_email_sent?
    @user.introduction_email_sent_at.present?
  end

  ...
end

In our component’s view, we’ll add a conditional for this new sending_email state

<% tag.div id: id do %>
    <div class="text-lg font-bold"><%= @user.name %></div>
    <div class="text-sm text-slate-500"><%= @user.email %></div>

    <% if sending_email? %>
      <%= helpers.turbo_stream_from broadcast_channel %>
      <%= render UI::Spinner.new(size: :sm, message: "Sending introduction email") %>
    <% elsif !introduction_email_sent? %>
      <%= helpers.button_to "Send introduction email", user_emails_introduction_path(@user) %>
    <% end %>
<% end %>

Then in our background job that sends the introduction email:

class SendUserIntroductionEmailJob < ApplicationJob
  queue_as :default

  def perform(user)
    user.send_introduction_email_later!

    UI::UserCard.new(user: user, sending_email: false).broadcast_refresh!
  end
end

We now replace the UI::UserCard with a version that has sending_email set to true, which means we’ll subscribe to updates from the broadcast_channel and showing an in-progress state in the component. When that work finishes, we replace the component with the sending_email: false equivalent, which doesn’t listen to updates anymore.

Conclusion

Wrapping parts of the UI in a component containing business logic using something like ViewComponent can be a huge benefit.

It keeps the logic encapsulated, and introduces a clear way to do things like refreshing the UI when something changes.

It also makes refactoring easier in the future, since we have one place where we have the component’s id and one place where we defined the channel to push updates to (broadcast_channel).

Often times, keeping behavior together can make it much easier to work in a codebase than following a strict adherence to separation of concerns.

Event sourcing for smooth brains: building a basic event-driven system in Rails

2024-07-21T13:00:00+00:00

Event sourcing is a jargon filled mess that is unapproachable to many developers, often using five dollar words like “aggregate root” and “projections” to describe basic concepts.

While the high standards of “full event sourcing” might recommend building your entire application around the concept, it is often a good idea to start with a smaller, more focused area of your codebase.

I was familiar with the broadest strokes of event sourcing, but it always felt way overkill for me and something that involved a bunch of Java code and Kafka streams and all of the pain that comes with distributed, eventually consistent systems.

But lately I have been building with a very basic, dumbed down version of event sourcing (I call this “event sourcing for smooth brains”) and I can see how aspects of this model can be a great fit for a boring Rails monolith.

Why did I go down this path?

Your application is generating tons of events. Even if you don’t think about them as events, they are there. Imagine an Issue in a GitHub project: a new issue is created, a comment is added, a label is added, and so on.

It’s common to need to list these events in some kind of feed. And as the application becomes more complex, you’ll find that you need to do more and more “things” when an event happens.

Think back to the GitHub issue example: when a comment is added, you might need to email the person who created the issue, send a notification to a team member, update a counter, trigger an automated action, run a spam check, update the commenter’s contribution graph.

Pretty quickly you’ll be writing a bunch of code to handle all of these different things and having some standard patterns for interacting with events is going to be required.

I’ll describe the simple version we’ve been using for a while now at Arrows. We are not operating at the scale of GitHub or any other large application, but it has served us well and the patterns are simple enough that we can scale for a long time before we need to add more complexity.

It’s about the events

Instead of worrying about projections, aggregates, reactors, command query responsibility separation, and read models we’re just going to focus on the events.

We’re also going to focus on events around one specific domain: a basic version of GitHub Issues.

Create an issue_events table with this schema (adjust to your liking, but this is the basic structure I use):

create_table :issue_events do |t|
  t.references :issue, null: false, foreign_key: true

  t.references :actor, null: false, foreign_key: { to_table: :users }
  t.string :action, null: false, index: true
  t.references :record, polymorphic: true, null: true

  t.jsonb :extra, null: false, default: "{}"

  t.datetime :occurred_at, null: false, default: -> { "CURRENT_TIMESTAMP" }
  t.timestamps
end

In a Rails app, I like making the event model under the Issue namespace, especially when you are using such a common name as “event”.

class Issues < ApplicationRecord
  belongs_to :project

  has_many :events, class_name: "Issue::Event"
end

class Issue::Event < ApplicationRecord
  belongs_to :issue

  belongs_to :actor, class_name: "User"
  belongs_to :record, polymorphic: true
end

The event model

The Issue::Event model is a simple model that stores the event data.

action: the name of the event (e.g. “comment_added”, “label_added”, etc). We put some validations on this to make sure we don’t have any typos or invalid events.
actor: the user that performed the action. We also have a “system” user for events that are generated by the application itself and not by a specific person
occurred_at: the time the event occurred
record: an optional polymorphic association to the record that was acted on (e.g. a Comment or a Label)
extra: a JSONB column for storing any extra data that might be needed. Generally be a bit wary of this because it will be unstructured, but for basic things it’s fine

class Issue::Event < ApplicationRecord
  belongs_to :issue

  belongs_to :actor, class_name: "User"
  belongs_to :record, polymorphic: true

  SUPPORTED_ACTIONS = %w[
    comment_added
    comment_deleted
    comment_viewed
    label_added
    ...
  ].freeze

  validates :action, inclusion: {
    in: SUPPORTED_ACTIONS,
    message: "%{value} is not a valid action"
  }
end

Nothing fancy here, just a basic Rails model that you can query and interact with just like any other model.

Now you’ll want a nice API to create these events. One thing we quickly found in practice is that some events would need to be throttled.

For example, if you want to track that a comment was viewed, you don’t necessarily need to record every single page view. You could group up the events within a certain time period into a single “comment viewed” event.

In our app, we wanted to be able to record events around lack of activity (e.g. this issue has not been viewed in a while) using a cron job but we didn’t want to keep adding no_activity events every time we checked so we set the throttle to be greater than the polling interval.

In “proper” event sourcing, you might record each of those events, then roll them up or create an intermediate snapshot or something fancier. For us, it was simple enough to do the throttling at creation time. We lose the full, unabridged history, but we don’t need to build other mechanisms to handle this.

class Issue < ApplicationRecord
  has_many :events, -> { order(occurred_at: :desc) },
    class_name: "Issue::Event",
    dependent: :destroy

  def record_event!(
    action,
    actor: Current.user, 
    record: nil,
    extra: {},
    throttle_within: nil
  )
    if throttle_within.present?
      existing = events.find_by(
        action: action,
        record: record,
        actor: actor,
        occurred_at: throttle_within.ago..
      )
      return existing if existing&.touch(:occurred_at)
    end

    events.create!(
      action: action,
      record: record,
      actor: actor,
      extra: extra
    )
  end
end

# Recording events
@issue.record_event!(:comment_added, actor: @comment.author, record: @comment)

# Throttling events
@issue.record_event!(:comment_viewed, record: @comment, throttle_within: 15.minutes)

# Adding some extra data for extra bits of metadata
@issue.record_event!(:label_added, extra: { name: @label.name })

We add a record_event! method to the Issue model that will create the event and optionally throttle it if it is within a certain time period.

To throttle, we look up an existing event for the same action, record, and actor that occurred within the throttle time period and touch it to update the occurred_at timestamp.

Voila an activity feed!

So far, this is neat and all…but all we’ve done is create a glorified activity feed.

class Issues::FeedsController < ApplicationController
  def show
    @issue = Issue.find(params[:issue_id])
    @page, @events = pagy(@issue.events.order(occurred_at: :desc), items: 10)
  end
end

# Create a view or component to render each event in the feed
# For each item you have `action`, `actor`, `occurred_at`, and `record`
# to construct a line item. You can define icons for each type, different
# colors, etc (exercise left to the reader)
render Issues::UI::Feed.new(@events)

Now this is useful to have and will be easy to add more events to over time for sure. But it’s not really showing the power of event sourcing.

For that, we need to actually do other stuff with the events.

In my work at Arrows (and nearly every other Rails app I’ve worked on), you eventually will build up several different integrations, notification systems, and light “metric” dashboards that need to know when things happen in the app.

In the case of our Issue model, let’s say that when a comment is added, I need to send an email to the Issue creator, add it to the Issue creators GitHub notification inbox, and post it to a Slack channel.

Instead of reaching for heavier approaches like an Event Bus or a Pub/Sub library, we can use Rails after_create_commit callbacks to do this.

Gasp! A callback! Aren’t those evil? Well, no. You can certainly make a mess but callbacks are one of the powerful tools in Rails. It’s a sharp knife, which means “be careful with it”, not “ban it from the kitchen”.

class Issue::Event < ApplicationRecord
  # ...

  after_create_commit :broadcast

  private

  def broadcast
    Email::Inbox.new(self).process_later
    AppNotification::Inbox.new(self).process_later
    Slack::Inbox.new(self).process_later
    # Add whatever makes sense for your app
  end
end

The broadcast method is called after the event is created (note: it will be called the first time when an event is throttled, but not after that…this behavior may not be appropriate for all use cases).

We then send that event into a bunch of different objects that I like to call “inboxes”. Each inbox can determine: if the event should be sent, what data to send, and how to send it. By using the familiar Rails _later suffix, we hint that these should almost certainly be run as background jobs.

I won’t show the code for each inbox, but the general structure is something like this:

class Email::Inbox
  def initialize(event)
    @event = event
  end

  def process_later
    Job.perform_later(@event)
  end

  def process
    case @event.action.to_sym
    when :comment_added
      # Send an email to the issue creator
      # Send an email to any people subscribed to the issue
      # Send an email to any project maintainers with notifications enabled
      # ...
    when :label_added
      # ...
    when :comment_viewed
      # ...
    end
  end

  private

  class Job < ApplicationJob
    def perform(event)
      new(event).process
    end
  end
end

As you can imagine, some inboxes will handle a lot of different event types and some will only handle a few. But the general pattern is the same: create a class that receives the event and then processes it.

You can structure the inboxes however you want, including extracting classes to handle the events as the logic grows. This is especially nice for inboxes like a Slack integration where we can make objects like Slack::MessageBuilder that can handle converting an event object into the formatted API payloads that Slack expects.

As you add more functionality to your application, you’ll find that you have a clear and easy place to put the code to handle what to do when an event happens.

Reaping what you’ve sown

Now that you have the basics setup, features that seemed super complicated can become much more straightforward to build.

If you want to build a new integration to an external API, you have the seams to put it into place.

class Linear::Inbox
  #...

  def process
    return unless @event.issue.synced_to_linear?

    case @event.action.to_sym
    when :comment_added
      Linear::API.add_comment!(@event.issue, @event.record.body)
      #...
    end
  end

  # ...
end

If you want to build a basic workflow automation system, you have a great start.

class Issue::Workflow < ApplicationRecord
  belongs_to :issue
  has_many :conditions
  has_many :actions

  attribute :triggered_on
  validates :triggered_on, inclusion: {
    in: Issue::Event::SUPPORTED_ACTIONS
  }
end

@issue.workflows.create!(
  triggered_on: "comment_added",
  conditions: [
    { attribute: "created_at", operator: "lt", value: "2022-01-01" }
  ],
  actions: [
    { type: "reply", message: "This issue is stale, open a new one" }
  ]
)

If you want the ability to do basic “history” queries to see how often a feature is used, you’ve got a solid foundation.

Issue::Event.where(action: "comment_deleted")
  .where(issue: @account.issues)
  .count

If you need to “replay” events to backfill data, you can query the events like normal ActiveRecord models and do your own processing.

last_commented_at = @issue.events
  .where(action: "comment_added")
  .maximum(:occurred_at)
@issue.update(last_commented_at: last_commented_at)

This pattern has been powerful for us at Arrows. We’ve been able to quickly build out several “systems” with a small team. Our main domain object has around 50 different event types and we’ve found it very easy to work with over time.

Adding new features is a breeze and the code is easy to maintain. Because the event creation and processing are decoupled, it’s easy to test and we feel safe that we won’t breaking existing behaviors.

And lastly, because it is basic, simple code (instead of a full-blown event sourcing library or a bunch of extra services), it’s easy to understand and we actually use it.

Acknowledgements and further reading

The idea of event sourcing came back onto my radar after hearing about it from Daniel Coulbourne and Chris Morrellin the context of their Laravel package Verbs.

The always excellent Martin Fowler blog has a nice post on Event Sourcing that I found helpful in bridging the gap between how product engineers think and how the more academic aspects of event sourcing work.

And thanks to the Event Sourcing and CQRS books I read and yet did not understand at all back when I was slinging .NET and Java code early in my career. It didn’t click for me then, but glad to be able to take some parts of it now.

Writing better Action Mailers: Revisiting a core Rails concept

2023-01-16T13:00:00+00:00

Mailers are a feature used in literally every Rails application. But they are often an after thought where we throw out the rules of well-written applications.

Writing mailers is a “set it and forget it” part of your codebase. But recently, I’ve revisited the handful of mailers in my application and I was shocked at both how bad things were and also how many nice mailer features in Rails I wasn’t aware of.

I’ve been writing Rails applications for over 10 years and there were things I figured out just this week about mailers that I will be using as my new defaults going forward.

Psst! If you like thinking about software and writing code in the "Boring Rails" style, we are hiring Product Engineers at Arrows. Come work with me!

Better display names

Rails has a built-in helper for formatting a display name that shows in email clients.

ActionMailer::Base.email_address_with_name("help@arrows.to", "Arrows HQ")
=> "Arrows HQ <help@arrows.to>"

ActionMailer::Base.email_address_with_name(user.email, user.display_name)
=> "Matt Swanson <matt@boringrails.com>"

This might seem trivial but it handles nil values and escaping quotes for you. And it’s a really nice touch for making emails from your app feel more polished.

I’ve written about this helper before but every time I mention it, someone replies telling me this is the first they’ve heard of it, so I will keep repeating it!

Changing the view folders

One thing that always bugs me is the file structure for mailer views. By default, the view for, e.g. NotificationMailer.welcome_email will be located at app/views/notification_mailer/welcome_email.html.erb.

This structure mirrors how controllers in Rails work. But it makes it difficult for you to see all your email templates at once. Often times, when we make a change to how we display emails, I need to scan all of the templates to make sure we aren’t doing anything funky.

I came across this post by Andy Croll that shows you how to put all your mailer views in one place. If you make one small tweak to your ApplicationMailer, you can achieve a more scannable folder structure.

class ApplicationMailer < ActionMailer::Base
  prepend_view_path "app/views/mailers"
end

Now you can put your mailer view in app/views/mailers/notification_mailer/welcome_email.html.erb. It is one extra level of folder nesting, but now you have a specific app/views/mailers folder instead of mixing in mailer views with controller views.

Multiple emails per mailer

Did you know you can put multiple emails inside of a single mailer?

There is nothing in the Rails documentation that says you can’t have multiple emails from one mailer, but it also isn’t explicitly encouraged. Sometimes you just need explicit permission from a random person on the internet and I am happy to be that person!

For whatever reason, every Rails app I’ve worked in has a one-to-one relationship between Mailer classes and email methods. Not only does this make it harder to grok the emails in your system, but it presents weird friction in naming that should jump out as a code smell.

Previously, I would make mailers like:

class CommentReplyMailer < ApplicationMailer
  layout "minimal"

  def comment_reply_email(user, comment)
    # mail(to: ...)
  end
end

class UserMentionedMailer < ApplicationMailer
  layout "minimal"

  def mentioned_email(mentionee, comment)
    # mail(to: ...)
  end
end

Why? I don’t really know. I can’t defend separating them.

Instead, you can group functionality into one mailer.

class NotificationMailer < ApplicationMailer
  layout "minimal"

  def comment_reply(user, comment)
    # mail(to: ...)
  end

  def mentioned(mentionee, comment)
    # mail(to: ...)
  end
end

Now you can see all the notifications in one file. There are so many times when I forget that we have a certain email going out and it doesn’t get updated when a new feature is added to the app.

You don’t need to write the word “email”

Mailers send emails. You don’t need to append _email to your methods/views – especially if you follow the above tip to put the views into app/views/mailers.

# No one is going to be surprised that this code sends an email
NotificationMailer.comment_reply_email(@user, @comment).deliver_later

# So don't end everything with `email`!
NotificationMailer.comment_reply(@user, @comment).deliver_later

Parameterized mailers

Okay, so far all of these tips are neat but nothing too wild.

Recently, I’ve been building a feature that allows you to send transactional from your own sending domain. So instead of getting a system email from hello@arrows.to, it would come from onboarding@acme-saas.com. There is some DNS related stuff that needs to happen in the background but I want to focus on the mailer portion of this feature.

One difficulty was making sure that, if an account had setup the custom sending domain, we overwrite the From address in the mailer. The problem was that there are many mailers in the application and I didn’t want to put this conditional code in every mailer. I was worried that – down the line – I would add a new email and then forget to consider the case where the account is using a custom sending domain.

I started with this basic implementation in one of the impacted mailers.

class NotificationMailer < ApplicationMailer
  def comment_reply(user, comment)
    # ...
    mail(
      to: user.email,
      from: build_from_address(comment.account)
    )
  end

  private

  def build_from_address(account)
    if account.custom_email_sender?
      email_address_with_name(
        account.custom_email_address,
        account.custom_email_name
      )
    else
      email_address_with_name("hello@arrows.to", account.name)
    end
  end
end

My first thought was that I could add some code to my ApplicationMailer to check the Current.account. But I hit a snag because mailers should be sent in a background job (via deliver_later) and we lose the context of the current request (and thus, the Current attributes). There are ways around this – like a middleware to pass along the Current attributes to the job – but something felt off to me.

When looking for a better way to implement this, I went back to the Action Mailer documentation and I noticed something: none of the example code was passing in data to the mailer methods directly. Instead there were accessing everything via params.

So instead of:

NotificationMailer
  .comment_reply(user, comment)
  .deliver_later

You would write:

NotificationMailer
  .with(user: user, comment: comment)
  .comment_reply
  .deliver_later

I had never used this pattern before. Mailers have not changed since I first learned Rails (in the 2.x days!) but in Rails 5.1, the concept of a “parameterized” mailer was introduced.

My first impression was that I didn’t quite understand the point of this. I generally prefer having the explicit method arguments on the mailer method compared to a generic params hash.

But, this time it finally clicked!

The extra benefit of using with and parameterized mailers is that you can add before_action callbacks to your mailers to configure options like the custom sending domain outside of the context of the mailer method.

I used this concept to make a small change:

class NotificationMailer < ApplicationMailer
  before_action { @account = params[:account] }
  before_action { @from = build_from_address }

  def comment_reply(user, comment)
    # ...
    mail(to: user.email, subject: "New reply", from: @from)
  end

  def mentioned(mentionee, comment)
    # ...
    mail(to: mentionee.email, subject: "You were mentioned", from: @from)
  end

  private

  def build_from_address
    if @account.custom_email_sender?
      email_address_with_name(
        @account.custom_email_address,
        @account.custom_email_name
      )
    else
      email_address_with_name("hello@arrows.to", @account.name)
    end
  end
end

So far, this isn’t really much better but this is setting the stage for being able to pull this configuration out of each mailer.

Dynamic defaults

The next piece of the puzzle is making use of mailer default options. One thing you might not realize (because I didn’t either…) is that you can pass a lambda to default to set the value dynamically.

class NotificationMailer < ApplicationMailer
  # You can pass in a static value
  default from: "hello@arrows.to"

  # But...it's probably more useful to use dynamic values
  default from: -> { build_default_from_address }

  private

  def build_default_from_address
    # Construct the default from address here
  end
end

The nice part about using default is that individual mailers can override the from option if they need to, but if you set the default, you can omit it completely.

The key breakthrough for my implementation of the custom sending domain feature comes from combining the dynamic defaults with the parameterized mailer option for passing in data.

Connecting the dots

By using dynamic defaults and before_action callbacks, you can have access to the params hash when configuring the defaults.

class NotificationMailer < ApplicationMailer
  default from: -> { build_default_from_address }

  before_action { @account = params[:account] }

  def comment_reply(user, comment)
    # ...
    mail(to: user.email, subject: "New reply")
  end

  def mentioned(mentionee, comment)
    # ...
    mail(to: mentionee.email, subject: "You were mentioned")
  end

  private

  def build_default_from_address
    if @account.custom_email_sender?
      email_address_with_name(
        @account.custom_email_address,
        @account.custom_email_name
      )
    else
      email_address_with_name("hello@arrows.to", @account.name)
    end
  end
end

NotificationMailer.with(account: @account).comment_reply(@user, @comment).deliver_later

Now the logic for the custom sending address has been completely removed from the mailer methods. And it is now clear that this behavior is not specific to just the NotificationMailer. Since we pulled the code into callbacks and parameterized options, we can hoist it up to a new mailer base class.

This also allows me to introduce the concept of an “Account scoped email” – an email sent in the context of a specific Account, which may have additional configuration or features.

class AccountMailer < ApplicationMailer
  layout "minimal"
  default from: -> { build_default_from_address }

  before_action { @account = params.fetch(:account) }

  private

  def build_default_from_address
    if @account.custom_email_sender?
      email_address_with_name(
        @account.custom_email_address,
        @account.custom_email_name
      )
    else
      email_address_with_name("hello@arrows.to", @account.name)
    end
  end
end

class NotificationMailer < AccountMailer
  # ...
end

class DigestMailer < AccountMailer
  # ...
end

class ParticipationMailer < AccountMailer
  # ...
end

There is one subtle change that also improves the developer experience and ensures that future mailers don’t omit the with(account: @account) configuration.

class AccountMailer < ApplicationMailer
  # ...

  before_action { @account = params.fetch(:account) }
end

The before_action will fail with a NoMethodError if the params are omitted and by using fetch it will fail with KeyError: :account if the caller does not pass in the account.

Wrap it up

Mailers are the worst part of a Rails application in terms of quality. While the framework provides an extremely powerful and elegant conceptual compression around sending emails, we often write them once and never touch them in our application code.

We frequently ignore complex code and duplication in mailers because they are at the edge of the system. The difficulty in rendering HTML email views does not help and further encourages “get it working and never touch it again” behavior.

I hadn’t brushed up mailers since I first learned Rails and I was surprised by how much I could improve them with a couple of small changes.

In my specific case, I was able to abstract a cross-cutting behavior (customizing the sender address) into a base mailer class. By using dynamic defaults and parameterized mailers, I can provide a pleasant developer experience that makes it easy to do “the right thing” for future code.

By applying some extra thought on naming and folder structure, I was able to make the mailer methods read better and make it easier to see the full scope of email views in the app. And by grouping multiple emails into single mailer classes, you can keep things that are similar close together in the code.

As I move forward, I will be working to define more application specific mailer contexts: for example an AccountMailer base class for emails generated within the scope of an account and SystemMailer base class for things like login and password reset emails that have different configuration options.

Your mailers can be an exemplary part of your codebase with a little bit of work!

Sorting ActiveRecord results by enum values (in SQL)

2022-10-10T13:00:00+00:00

Rails enums are a great way to model things like a status on an ActiveRecord model. They provide a set of human-readable methods while storing the result as an integer in the database.

class JobSubmission < ApplicationRecord
  enum status: {
    draft: 0,
    submitted: 1,
    hold: 2,
    rejected: 3,
    accepted: 4,
    canceled: 5
  }
end

It is highly recommended to use a Hash to explicitly define the enum values – otherwise Rails will use the index of the enum value when storing it to the database. If you were were to change the order or remove options, you would break the reference mapping.

Out of the box, you can sort by the enum value like any other column and it will use the value.

JobSubmission.all.order(:status)
# "SELECT \"job_submissions\".* FROM \"job_submissions\" ORDER BY \"job_submissions\".\"status\" ASC

This would return JobSubmission results in ascending order, so first draft, then submitted, hold, etc.

But what if you wanted sort the results differently? You might be tempted to do something like this:

class JobSubmission < ApplicationRecord
  STATUS_SORT = {
    accepted: 0,
    hold: 1,
    submitted: 2,
    draft: 3,
    rejected: 4,
    canceled: 5
  }

  def status_sort_value
    STATUS_SORT[status]
  end
end

JobSubmission.all.sort(&:status_sort_value)

But this will do the sorting in Ruby, instead of in the database, which can have bad performance for large result sets.

Instead you can use the in_order_of to specify the sort order of the enum values. And the best part? The sorting will happen in SQL (using a CASE command under-the-hood).

Usage

To sort the JobSubmission records by the status enum value in a different order, you can use:

JobSubmission.all.in_order_of(:status, %w[accepted hold submitted draft rejected canceled])

The first argument is the column to sort by and the second argument is an ordered array of values.

The SQL generated for this query will be:

SELECT "job_submissions".*
FROM "job_submissions"
  WHERE "job_submissions"."status" IN (4, 2, 1, 0, 3, 5)
ORDER BY
  CASE "job_submissions"."status"
    WHEN 4 THEN 1
    WHEN 2 THEN 2
    WHEN 1 THEN 3
    WHEN 0 THEN 4
    WHEN 3 THEN 5
    ELSE 6
  END ASC

You can also add additional sorting to the relation. For instance, you would probably want to sort records by the status and then maybe alphabetically by the name column to break ties.

JobSubmission.all
  .in_order_of(:status, %w[accepted hold submitted draft rejected canceled])
  .order(:name)

Extra notes

You important caveat is that in_order_of also adds a WHERE/IN clause for the enum values. So only records with enum values that you specify when calling in_order_of will be returned. This is somewhat surprising and something to watch out for if you are frequently changing the enum values.

You will probably want to codify the sort order in the model as a scope to avoid having to repeat the ordering in multiple places.

class JobSubmission < ApplicationRecord
  scope :in_status_order, -> { in_order_of(:status, %w[accepted hold submitted draft rejected canceled]) }
end

JobSubmission.in_status_order.order(:name)

Additional Resources

Rails API Docs: ActiveRecord::QueryMethods#in_order_of

Thinking in Hotwire: Progressive Enhancement

2022-08-16T13:00:00+00:00

There are many tutorials about how to get started with Hotwire and how to use the individual pieces. But one thing that took me a while to grasp was how to “think in Hotwire”.

Hotwire itself is an overarching concept (HTML-over-the-wire) and you’ll need to know when to use the different pieces (Turbo Drive, Frames, Streams, Stimulus, Turbo Native, Strada).

Because Hotwire is a collection of tools, you can solve problems multiple ways. There are features you can build with Frames that you could also build with Streams. You can always drop to a “lower level” tool and make something work.

So how should you know when to reach for each tool?

I think the best approach comes from the earliest days of web development: progressive enhancement.

Scaffolds and CRUD without Hotwire

Let’s start with the absolute bare bones: the pre-Hotwire days of Rails. You would scaffold out a resource-based controller.

To add a comment to a post, you would have a comments_controller with a new action to render a form.

You would submit a new comment to comments#create where the comment model would be saved and then we send a redirect back to the post#show route.

It’s as boring as you can get, but it works and is the foundation upon which RESTful web applications were built.

Adding Turbo Drive

So what if we wanted to enhance the experience just the tiniest amount? We could use the first layer of Hotwire: Turbo Drive. If you’re on a recent version of Rails, Turbo Drive is on by default, even if you didn’t realize it.

Now when you click + Add, instead of a full-page reload of comments#new, we use Turbo Drive to fetch the page via AJAX and then Turbo Drive swaps out the <body> contents. This behavior (originally called Turbolinks) mimics the responsiveness of a single-page application, but everything is still server-rendered in Rails.

Turbo Drive does the same thing when you submit the comment form: instead of a full page reload, the <body> is swapped, even when we do a redirect. The JS and Rails portions of Turbo Drive work together to make this seamless (…assuming you follow the right conventions!).

So you could stop right here and technically you are using Hotwire. Turbo Drive is mostly invisible for Rails developers. If you don’t need any more interactivity, you don’t need to go any deeper.

Adding Turbo Frames

While Turbo Drive originated as Turbolinks, Turbo Frames are a new concept in Hotwire. I’ve described Turbo Frames before as “iFrames but they work how you would want them to”. In Hotwire, Turbo Frames are used for partial page updates.

A Frame works similar to Turbo Drive: navigation visits and form submissions are intercepted, made via AJAX, and then the response is swapped into the page. But with Frames, instead of swapping the whole <body> tag, only the contents of a matching Turbo Frame is swapped.

So if you have a link inside a Turbo Frame with an id of comment_123, Hotwire will look for a matching <turbo-frame id='comment_123'> tag in the response to swap.

The canonical use-case is inline editing. If you have a list of comments and you wrap a frame around each comment, you can add a edit button. Clicking the edit button can render a comments#edit response, but swap out the form with the frame instead of loading a separate page.

Adding Turbo Streams

Turbo Drive replaces the whole <body> tag. Turbo Frames replace a frame. Turbo Streams go even more granular.

Turbo Streams provide a set of standard operations for manipulating the HTML on the page. You can add, remove, or replace content.

In our comment example, if you want to delete a comment, you can respond to the request with a Turbo Stream to remove the comment from the page. Or if you create a comment, you can add it to the end of the list.

Turbo Streams are the Hotwire version of another classic Rails technique: server-generated JavaScript response (SJR). In the past, you could return back a snippet of JavaScript that would get executed in the page. But this caused issues with Content Security Policies and was a bit too permissive.

Turbo Streams provides a CRUD-like abstraction to handle nearly anything you should have been doing with SJR.

Adding Turbo Streams + Action Cable

Turbo Stream responses can also be broadcast out-of-band using Action Cable. One major point of confusion is that Turbo Streams do not have to be sent over Action Cable or web sockets, you can use them within a normal HTTP request/response cycle.

But let’s say we want real-time functionality so that when someone else adds a comment, it gets added to our view of the page without doing a refresh.

Adding Stimulus

Hotwire guides you to write as much of your application using server rendered Ruby code as possible. But there are cases where you still want extra sprinkles of JavaScript functionality on the client side.

Stimulus is the recommended JavaScript framework for attaching functions to your HTML markup and responding to simple browser events.

One thing to note is that Stimulus does not provide anything to support client side rendering. There are no templates or JSX. If you find yourself generating a lot of HTML in your Stimulus controllers, you should take a step back and re-assess.

Stimulus controllers are often general-purpose and under 50 lines of code.

If you want to support collapsing comments, you could add a simple Stimulus controller to show or expand a comment. (Note: in this specific case, a Hotwire enthusiast might reach for native HTML elements like <detail> but I digress…)

Adding custom components / React / client side JavaScript

There is a time and place for tools like React for high interactivity components. Whether you use a tool like React, Vue or Web Components, there are things that are best built on the client side that go beyond what Stimulus can handle.

For this, you can add a small, isolated component to the page. An example from Rails is the Trix rich text editor: it is a standard <trix-editor> web component.

In our comment example, maybe you want to add an emoji picker or some kind of fancy drag and drop file uploader. You might wrap these richer JavaScript libraries in a Stimulus controller to help manage Turbo lifecycle events.

Hotwire advocates don’t say that you should never write JavaScript components, but rather that you should start with the other tools and only reach for this level when you really need it.

Adding Turbo Native

Once you’ve built a feature in your web application, you can use the Turbo Native iOS and Android adapters to create mobile applications that wrap display your same app content inside of a web view.

This is not a “mobile optimized web app” but a real Swift or Kotlin app that renders pages from your Rails app.

There are some conveniences to create native action bars and buttons.

The main idea is to re-use as much of your Rails views as possible and then Turbo Native provides a mechanism to eject and write some screens in the platform native languages. Again, it’s the idea of progressive enhancement.

The flagship example app (the Hey email client) for instance has a fully native inbox screen, but many of the secondary or account setting screens are wrappers around Rails views.

The Turbo Native functionality should still be considered as a beta release. Most developers writing Hotwire applications are not using the native features, but it is nice to know that some folks are laying the groundwork for this style of development.

Adding Strada

The last piece of Hotwire is the [as of summer 2022] unreleased Strada library. This is an extension of the Turbo Native functionality and helps bridge the gap when you need to communicate between the Swift or Kotlin parts of your application and the HTML/Rails portions.

This library will be a convenience extension to Turbo Native, it doesn’t add anything fundamentally new to the mix.

Until the project is actually released, it’s fine to ignore.

Wrap it up

Hotwire builds on the idea of progressive enhancement. You should use the least amount of the tooling as possible to achieve your desired outcome. As you move “down the stack” of what Hotwire offers, you trade off more power for more complexity.

The nice part about this approach is that you can build versions of features quickly, test and iterate based on feedback, and then layer on more real-time and interactive functionality as needed.

The progressive enhancement approach pairs beautifully with HTML and Hotwire encourages you to explore what can be done with native browser elements and only layers on extra tooling to complement or upgrade the platform.

Hotwire as a brand is an umbrella for several different tools and hopefully this overview will help you know how far down to reach and how the individual tools come together to build a cohesive and compelling stack.

Galaxy brain CSS tricks with Hotwire and Rails

2022-07-26T13:00:00+00:00

In Hotwire applications, you need to lean more on the fundamentals of CSS and HTML. If you’re like me, you probably learned just enough CSS to get by, but never reach for it first. But that’s changed recently and I wanted to share patterns I’ve picked up recently that improve my Rails apps.

Empty States and Turbo Streams

An extremely common pattern in Rails apps is rendering a collection of elements and if the collection is empty, render an empty state.

<div id="my_list" class="flex flex-col divide-y">
  <% if @list.size > 0 %>
    <%= render partial: "list_item", collection: @list %>
  <% else %>
    <p>
      Whoops! you have no items!
    </p>
  <% end %>
</div>

This works fine when rendering a typical page, but if you use Turbo Streams to add or remove list items, you’ll find a problem.

If you had two items in the list, and you remove both via Turbo Streams, the container will be empty but you won’t have rendered the empty state. And if the list is empty and you dynamically append an item, you’ll want to remove the empty state.

You could re-render the whole list instead of inserting items, but one technique that I’ve found helpful is using the CSS only-child pseudo-selector.

I’ll show examples with Tailwind (because Tailwind is really, really good), but the same concept applies in regular CSS.

The idea is to always render the empty state and then use CSS to only show it if there are no items.

<div id="my_list" class="flex flex-col divide-y">
  <p class="only:block hidden">Whoops! you have no items!</p>
  <%= render partial: "list_item", collection: @list %>
</div>

Using Tailwind’s only modifier we set the empty state to have display block if it is the only child of the container, otherwise hide it.

Now you can stream back operations to append or remove items to the my_list container and let CSS handle hiding or showing the loading state.

Note: you may want to use last-child, first-of-type, or some other modifier depending on your specific markup. Give it a shot!

Tailwind Variants with Data Attributes

Hotwire, especially Stimulus, makes use of HTML data attributes heavily. One neat trick is using data attributes to reduce conditionals in views.

Let’s say you have a list of comments and only admins can delete the comments.

<%= tag.div id: dom_id(@comment) do %>
  <p><%= @comment.body %></p>

  <% if Current.user.admin? %>
    <%= button_to "Delete", @comment, method: :delete %>
  <% end %>
<% end %>

You could instead use a data attribute on the <body> of your page to conditionally show admin-related things.

<body <%= 'data-admin' if Current.user.admin? %>>
  ...
</body>

And then write styles based on that attribute. Tailwind makes it easy to add custom variants for this via the plugins section of the Tailwind config file:

plugins: [
  function({addVariant}) {
    addVariant('admin', 'body[data-admin] &')
  }
],

With this config change you can now use admin: with any Tailwind classes.

<%= tag.div id: dom_id(@comment) do %>
  <p><%= @comment.body %></p>

  <div class="admin:block hidden">
    <%= button_to "Delete", @comment, method: :delete %>
  </div>
<% end %>

But wait? Isn’t this super risky because someone could just fiddle with the HTML and delete a comment? Well, yes, they could – but you need to be checking authorization on the server-side anyways. It’s a trade-off but there are cases where this cleans up a ton of conditional view logic.

Shout-out to my friend Marc Kohlbrugge for sharing the idea on Twitter!

In our app we recently used this technique to change how much margin we needed on an element based on user roles. Certain roles have a fixed height header that we needed to account for. Instead of a bunch of conditionals, all we had to end up writing was viewer:top-0 editor:top-12.

Dynamic styles with erb

Don’t forget that you can use generate <style> tags dynamically!

<style>
  [data-user~="<%= Current.user.id %>"] {
    background: yellow;
  }
</style>

<div>
  <p><%= @comment.body %></p>
  <%= tag.span data: { user: @comment.author.id } do %>
    <%= @comment.author.name %>
  <% end %>
</div>

You could use a little CSS to highlight comments that you made with a yellow background by targeting a data-user attribute.

This is actually an old Rails technique from Basecamp that was popular because it works really well with fragment caching. You can cache the same chunk of HTML and then use CSS to change the styles instead of needing multiple, slightly different cache entries.

I’ve also used this concept for building custom theming features by taking advantage of CSS variables.

<style>
  :root {
    --color-brand: <%= @account.brand_color %>;
    --color-brand-contrast: <%= ColorHelper.contrast(@account.brand_color) %>;
    --color-brand-tint: <%= ColorHelper.tint(@account.brand_color) %>;
  }
</style>

You can use this to define custom Tailwind colors:

module.exports = {
  theme: {
    extend: {
      colors: {
        brand: "rgb(var(--color-brand) / <alpha-value>)",
        "brand-contrast": "rgb(var(--color-brand-contrast) / <alpha-value>)",
        "brand-tint": "var(--color-brand-tint)",
      },
    },
  },
};

And now you can use bg-brand or text-brand-contrast in your application.

Stop string interpolating class names!

You will be writing a lot more HTML markup in Hotwire apps: more views, more partials, more components. Make sure you are taking advantage of newer Rails features for generating HTML without doing a bunch of gross string interpolation.

If you’re writing markup like this:

<li class="bg-gray-50 p-2 text-gray-700 <%= 'line-through' if @task.completed? %>">
  <%= @task.name %>
</li>

Please stop! It’s hard to read and tricky to match all of the closing punctuation. There are better ways!

<%= tag.li class: ["bg-gray-50 p-2 text-gray-700", "line-through": @task.completed?] do %>
  <%= @task.name %>
<% end %>

Rails 6.1 added a class_names helper method and the tag builder will automatically use it for conditionally setting class values. It’s awesome!

It’s extra powerful when using a library like ViewComponent where you have a lot of conditional styles, or just want to group up utility classes in a more organized manner.

class MyWidget < ViewComponent::Base
  ...

  def container_classes
    [
      "flex items-center justify-center space-x-2 rounded-full",
      "disabled:pointer-events-none disabled:select-none",
      "font-medium tracking-wide",
      {"text-white bg-black hover:bg-neutral-900": variant == :primary},
      {"text-neutral-600 border hover:bg-neutral-50 hover:text-neutral-900": variant == :secondary},
      {"text-neutral-600 hover:text-neutral-900 hover:bg-neutral-50": variant == :tertiary},
      {"w-full": full_width?}
    ]
  end
end

Wrap it up

CSS is often one of the last tools Rails developers reach for when trying to solve a tricky problem. We are much more inclined to add conditionals to views or fall back to string interpolation to “make it work”. But there are a few techniques that can make working with CSS in your Rails app improve the readability and durability of your code.

Even though Hotwire’s servered rendered approach feels retro, remember that we don’t have to use CSS like it’s 1998 anymore!

Adding keyboard shortcuts and hotkeys to StimulusJS

2022-07-11T13:00:00+00:00

Keyboard shortcut support for actions is now built-in to Stimulus 3.2!

Keyboard shortcuts are a great way to level up your user experience and improve the accessibility of your web applications. Even if you aren’t ready for advanced hotkey schemes, small things like binding the Escape key to close a modal can have a big impact.

Stimulus doesn’t come with built-in support for hotkeys. As part of a recent project at Arrows, I evaluated the ecosystem and wanted to share my thoughts.

stimulus-hotkeys

This package provides a hotkeys controller and uses a JSON object to map keyboard shortcuts into Stimulus actions.

<div
  data-controller="hotkeys"
  data-hotkeys-bindings-value='{"ctrl+z, command+z": "#foo->editor#undo"}'
></div>

<div id="foo" data-controller="editor"></div>

The syntax of the bindings map is similar to the Stimulus action syntax, with the addition of a preceding selector to find the Stimulus controller to invoke the action on. Internally, stimulus-hotkeys uses the more general HotKeys.JS library so it supports all kinds of fancy combinations, modifiers, and scopes.

I liked that the shortcut mapping exists in the HTML markup; this felt very much in the spirit of Hotwire. This library adds no extra code to your Stimulus controllers, which is ideal if you are combining together other third-party controllers.

Unfortunately, adding a hash-like data structure as an HTML attribute is tricky and I don’t love the JSON syntax.

stimulus-use/useHotkeys

The stimulus-use project is a collection of reusable behaviors for Stimulus. If you are familiar with React, this project is similar to React’s hooks system, but for Stimulus controllers.

Included in this collection is useHotkeys. As with stimulus-hotkeys, the heavy lifting is done by HotKeys.JS.

Here you need to define the hotkeys and respective handlers inside of your own Stimulus controllers:

import { Controller } from "@hotwired/stimulus";
import { useHotkeys } from "stimulus-use";

export default class extends Controller {
  connect() {
    useHotkeys(this, {
      "cmd+t": [this.openPalette],
      ".": [this.editFile],
    });
  }
}

This library felt right at home in a Stimulus controller and it handles unregistering the shortcuts when a controller is disconnected automatically. It was super clear what hotkeys were configured and this felt like a great option for adding hotkeys to your own application controllers.

One downside that was specific to our needs was that I had a controller that was always on the page, but should not respond to hotkeys when hidden. While useHotkey handles the Stimulus controller lifecycle, it does make it inflexible when it comes to binding and unbinding the keyboard events.

HotKeys.JS (directly)

After exploring the two most popular Stimulus-specific solutions and realizing they both used the same underlying dependency, I decided to go straight to the source.

import { Controller } from "@hotwired/stimulus";
import hotkeys from "hotkeys-js";

export default class extends Controller {
  connect() {
    hotkeys("esc", () => this.doSomething());
  }

  disconnect() {
    hotkeys.unbind("esc");
  }
}

Adding the library directly is straightforward. You do need to be careful to clean up your event listeners by calling unbind or else you’ll end up with multiple instances of the hotkey functions getting created.

For very simple cases (one or two basic hotkeys), this approach is perfectly valid. In our application, I ended up going this route as I needed more fine-grained control of when the shortcuts were bound/unbound.

github/hotkey

The last option I evaluated was a small library from GitHub called github/hotkey. It is not Stimulus / Hotwire specific.

<button data-hotkey="Shift+?">Show help dialog</button>

<a href="/page/2" data-hotkey="j">Next</a>
<a href="/help" data-hotkey="Control+h">Help</a>
<a href="/rails/rails" data-hotkey="g c">Code</a>
<a href="/search" data-hotkey="s,/">Search</a>

I found this library via the sourcemaps of HEY.com and it has been battle-tested by GitHub in the main Rails app (from the README: “This is used on almost every page on GitHub.”).

The data-hotkey attribute has the cleanest syntax of any of the options. You would want a wrapper Stimulus controller to handle lifecycle events, but it would be only for calling install and uninstall.

The design of this library relies more heavily on browser default behaviors. Instead of calling arbitrary JavaScript functions (or Stimulus actions), github/hotkey triggers the action on the target HTML element: links would trigger a navigation visit, buttons would get submitted, input fields would get focus, etc.

If you can structure your markup to use native browser functionality, this is a great option. But there isn’t a way to hook into your existing Stimulus actions directly.

Wrap it up

It really depends on what your specific hotkey needs are:

stimulus-hotkeys is a drop-in tool that doesn’t require any extra JavaScript
stimulus-use/useHotKeys is a convenient Stimulus-native API wrapper around HotKey.js
HotKey.js is a powerful abstraction around key events that you can mix into your own code
github/hotkey leans heavily on native browser behavior and has the nicest syntax

Ultimately, you’ll need to navigate the trade-offs for what you’re trying to build. In my case, I went with directly using HotKey.js because it best fit my specific requirements.

The most underrated Rails helper: dom_id

2022-06-28T13:00:00+00:00

The dom_id helper in Rails is over a decade old, but has proven to be an invaluable concept in Hotwire.

This secret workhorse powers all kinds of HTML-related behavior in Rails. It has one key job: making it easy to associate application data with DOM elements.

dom_id takes two arguments: a record and an optional prefix.

The record can be anything that responds to to_key and model_name, but 99% of the time you are passing it an ActiveRecord model. The prefix can be anything that responds to to_s, but 99% of the time it is a symbol.

From the docs:

dom_id(Post.find(45))       # => "post_45"
dom_id(Post.new)            # => "new_post"

dom_id(Post.find(45), :edit) # => "edit_post_45"
dom_id(Post.new, :custom)    # => "custom_post"

The reason this helper is so nice is that it sets a convention. Instead of defining your own way of identifying markup and later hoping you remember the pattern, Rails provides a standard; you don’t need to switch contexts to know if you set an id to “post_23_comments” or “comments-post-23” or wait, was it “post_23-comments”?

By providing stable id values for your elements, you avoid fragile code like targeting the first element child or finding a node based on the text value.

Here are places where you should regularly reach for dom_id when building Hotwire apps.

Super clean tag builders

Leaning into HTML markup as the source of truth means adding extra attributes and conditionals when rendering templates. While you can do plain old string interpolation in your ERB templates, Rails has a set of nice tag builders that help you avoid drowning in a sea of brackets, braces, and octothorpes.

<%= tag.div id: dom_id(@post, :comments), class: "flex flex-col divide-y" do %>
  <%= render @post.comments %>
<% end %>

As you use these helpers more, make sure you check out these tips:

Set up the Tailwind Intellisense plugin for VS Code to autocomplete when using tag helpers
Take advantage of class_name helper (from the classNames React API) for conditional classes
For commmon elements, consider extracting a component with a lightweight helper or a library like ViewComponent

Deep linking anchor tags

Since Rails leans so heavily on native browser features, make sure you take advantage of anchor tags on links. You can use dom_id to scroll the browser directly to an element with the corresponding id (or generate a permalink that users can share).

<%= link_to "View comment", posts_path(@post, anchor: dom_id(@comment)) %>

You can also use this pattern for redirects. For example, after creating an item in a list, you can redirect back to the index page but scroll to the new item.

class CommentsController < ApplicationController
  def create
    @post.comments.create!(comment_params)

    redirect_to posts_path(@post, anchor: dom_id(@comment))
  end
end

A few more tips related to deep linking:

You can use the :target pseudo-class to style the element with an ID matching the URL anchor. In Tailwind, simply use the target prefix (e.g. target:bg-yellow-50 to add a subtle yellow background to an element when it matches the URL anchor)
Another handy CSS property is scroll-margin-top: the browser will scroll the targeted element all the way to the top of the window. You may want a little extra padding, but only when you scrolled to the element. Don’t add extra margin or padding to your designs or add weird wrapper divs…scroll-margin-top (Tailwind class: scroll-mt) is the answer.

With Turbo Frames

When you start adding Turbo Frames to your application, you’ll need to provide an id for the frame tag. The Rails turbo_frame_tag uses – you guessed it – dom_id under the hood. But you can also pass in your own ids as well.

turbo_frame_tag @post # => <turbo-frame id="post_123"></turbo-frame>
turbo_frame_tag dom_id(@post, :comments) # => <turbo-frame id="comments_post_123"></turbo-frame>

Since a Turbo Frame needs to be unique per page, dom_id is a convenient way to generate frame ids, especially if you have multiple frames on the page.

And since you can navigate a frame via other links, it’s a great convention to follow:

<%= turbo_frame_tag @comment, src: comment_path(@comment) %>

<!-- Elsewhere... -->
<%= link_to "Edit", edit_comment_path(@comment), data: { turbo_frame: @comment } %>

Scoping Turbo Stream responses

Making small mutations on a page with Turbo Streams is super powerful, but since your stream actions are in a separate turbo_stream.erb file, it can be tricky to match up your ids between views.

Once again, dom_id keeps things consistent.

<!-- app/views/plans/quick_edit/update.turbo_stream.erb -->
<%= turbo_stream.replace dom_id(@plan, :title), partial: "plans/title" %>
<%= turbo_stream.replace dom_id(@plan, :notes), partial: "plans/notes" %>
<%= turbo_stream.replace dom_id(@plan, :assigned), partial: "plans/assigned" %>

And by adding the correct ids to your markup, the Turbo Stream responses are super clean:

<!-- app/views/comments/destroy.turbo_stream.erb -->
<!-- Calls `dom_id(@comment)` under the hood -->
<%= turbo_stream.remove @comment %>

Wrap it up

Who would have thought that a simple helper to generate HTML id values from your application models would be such a useful concept that, more than a decade after first being introduced, it continues to prove helpful even on the newest and shiniest parts of Rails.

If you haven’t been using dom_id before, consider it the next time you write a view in your Rails app. You might be surprised at how much more pleasant it is to create HTML when you aren’t littering it with code like:

<div id='<%= "#{@post.id}-comments" %>'>
</div>

Self-destructing StimulusJS controllers

2022-06-13T13:00:00+00:00

Sometimes you need a little sprinkle of JavaScript to make a tiny UX improvement. In the olden days, full-stack developers would often drop small jQuery snippets straight into the page:

<script type="application/javascript">
  $(".flash-container").delay(5000).fadeOut();
  $(".items").last().highlight();
</script>

It got the job done, but it wasn’t the best.

In Hotwire apps you can use a “self-destructing” Stimulus controller to achieve the same result.

Self-destructing?

Self-destructing Stimulus controllers run a bit of code and then remove themselves from the DOM by calling this.element.remove().

Let’s see an example:

// app/javascript/controllers/scroll_to_controller.js
import { Controller } from "@hotwired/stimulus";

export default class extends Controller {
  static values = {
    location: String,
  };

  connect() {
    this.targetElement.scrollIntoView();
    this.element.remove();
  }

  get targetElement() {
    return document.getElementById(this.locationValue);
  }
}

This controller takes in a location value and then scrolls the page to show that element.

<template
  data-controller="scroll-to"
  data-scroll-to-location-value="task_12345"></template>

For self-destructing controllers, I like to use the <template> tag since it will not be displayed in the browser and is a good signal when reading the code that this isn’t just an empty div.

This pattern works really well with Turbo Stream responses.

Imagine you have a list of task with an inline form to create a new task. You can submit the form and then send back a <turbo-stream> to append to the list and then scroll the page to the newly created task.

<%= turbo_stream.append :tasks, @task %>

<%= turbo_stream.append :tasks do %>
  <template
    data-controller="scroll-to"
    data-scroll-to-location-value="<% dom_id(@task) %>"></template>
<% end %>

And because we wrap our small bit of JavaScript functionality in a Stimulus controller, all of the lifecycle events are taken care of. No need to listen for turbo:load events, it just works.

What else could you use this for?

Highlighter

We use this highlighter controller to add extra styles when something is “selected”.

<template
  data-controller="highlighter"
  data-highlighter-marker-value="<%= dom_id(task, :list_item) %>"
  data-highlighter-highlight-class="text-blue-600 bg-blue-100"
></template>

By using both the Stimulus values and classes APIs, this controller is super reusable: we can specify any DOM element id and whatever classes we want to use to highlight the element.

// app/javascript/controllers/highlighter_controller.js
import { Controller } from "@hotwired/stimulus";

export default class extends Controller {
  static values = {
    marker: String,
  };
  static classes = ["highlight"];

  connect() {
    this.markedElement.classList.add(...this.highlightClasses);
    this.element.remove();
  }

  get markedElement() {
    return document.getElementById(this.markerValue);
  }
}

Grab focus

We use a grab-focus controller for a form where you can quickly add tasks. Submitting the form creates the task and then dynamically adds a new <form> for the next task. This controller seamlessly moves the browser focus to the new input.

// app/javascript/controllers/grab_focus_controller.js
import { Controller } from "@hotwired/stimulus";

export default class extends Controller {
  static values = {
    selector: String,
  };

  connect() {
    this.grabFocus();
    this.element.remove();
  }

  grabFocus() {
    if (this.hasSelectorValue) {
      document.querySelector(this.selectorValue)?.focus();
    }
  }
}

Analytics “Beacons”

We borrowed this idea from HEY and use it for tracking page analytics. We add a beacon to the page that pings the backend to record a page view and then removes itself.

(If you’re fancy you could even use the Beacon Web API, but we’re justing sending an PATCH request here for simplicity!)

// app/javascript/controllers/beacon_controller.js
import { Controller } from "@hotwired/stimulus";
import { patch } from "@rails/request.js";

export default class extends Controller {
  static values = { url: String };

  connect() {
    patch(this.urlValue);
    this.element.remove();
  }
}

We wrapped this one up in a Rails view helper for a more clean API.

module AnalyticsHelper
  def tracking_beacon(url:)
    tag.template data: { controller: "beacon", beacon_url_value: url }
  end
end

<!-- Inside app/views/layouts/plan.html.erb -->
<%= tracking_beacon(url: plan_viewings_path(@plan)) %>

Wrap it up

Self-destructing Stimulus controllers are a great way to augment Hotwire applications by adding sprinkles of JavaScript behavior without having to completely eject and build the whole feature on the client-side. Keep them small and single-purpose and you’ll be able to reuse them across pages and in different contexts.

Piggybacking on the existing lifecycle of Stimulus controllers ensures that things work as expected when changing content via Turbo Streams and navigating between pages with Turbo Drive.

Tailwind style CSS transitions with StimulusJS

2022-06-01T13:00:00+00:00

If you’ve built UI elements with StimulusJS before, you’ve certainly written code to show or hide an element on the page. Whether you are popping up a modal or sliding out a panel, we’ve all written controller code like:

this.modalTarget.classList.remove("hidden");

To take your UI designs to the next level, you can use transitions so elements don’t immediately appear or disappear from the screen. You can transition opacity to gently fade elements in and use translate to slide them into place.

One issue trying to do these kind of animations with CSS transition properties, as Sebastian De Deyne lays out in this excellent primer on enter and leave transitions, is that you can’t change an element’s display property before the transition occurs. We can use basic transition styles for things like subtly changing a button color on hover, but for smooth animations when elements are shown or hidden, we need something more.

One pattern that has emerged from the Vue and Alpine communities is to use a series of data attributes to define your desired CSS classes during the lifecycle of the transition. TailwindUI also follows this pattern when specifying how to animate their components. It has proven to be powerful and easy to understand.

Each project has their own specific naming conventions, but they all define six basic lifecycle “stages”:

Entering: the classes that should be on an element during the whole time it is entering the page (sometimes called “Entering Active”, “Enter Active”, “Enter”, etc)
Enter From: the starting point that you will transition from when entering the page (sometimes called “Enter Start”)
Enter To: the ending point of the transition when entering the page (sometimes called “Enter End”)
Leaving: the classes that should be on an element during the whole time it is leaving the page (sometimes called “Leaving Active”, “Leave Active”, “Leave”, etc)
Leave From: the starting point that you will transition from when leaving the page (sometimes called “Leave Start”)
Leave To: the ending point of the transition when leaving the page (sometimes called “Leave End”)

This picture from the Vue docs really helped me visualize how it all works:

The approach of using data attributes works well with Tailwind’s transition utility classes and feels right at home with the StimulusJS philosophy of augmenting HTML markup.

Vue (Transition) and Alpine (x-transition) provide native, first-party support for these transitions, but for a Rails app using Hotwire, we’ll have to add this functionality ourselves.

Options

I explored a few different options in this space. Here are the most popular approaches I came across:

stimulus-transitions

This library provides a transition controller that you can import and register. You use this controller like a normal Stimulus controller in your application:

<div
  data-controller="transition"
  data-transition-enter-active="enter-class"
  data-transition-enter-from="enter-from-class"
  data-transition-enter-to="enter-to-class"
  data-transition-leave-active="or-use multiple classes"
  data-transition-leave-from="or-use multiple classes"
  data-transition-leave-to="or-use multiple classes"
>
  <!-- content -->
</div>

The controller will automatically detect when the element is shown or hidden and run the transitions. There are also options for listening to custom transition:end-enter and transition:end-leave events if you want to run additional code when the transitions have finished.

The transition controller needs something to trigger the display style on the element so you will need an application-level controller to kick off the process.

stimulus-use/useTransition

The stimulus-use project is a collection of reusable behaviors for Stimulus. If you are familiar with React, this project is similar to React’s hooks system, but for Stimulus controllers.

One particular mix-in available in this package is useTransition. You can call this from your own Stimulus controller and it will run the transitions on the element (either reading from data- attributes or you can specify the classes in JavaScript as options).

This particular mix-in was flagged as a “beta” release at the time of this writing.

el-transition

This library is not Stimulus specific, but implements the same Vue/Alpine transition pattern. Since it is vanilla Javascript, there are no built-in hooks for Stimulus lifecycle or controllers to register. You import enter and leave functions directly and then call them while providing the element to transition.

import { enter, leave } from "el-transition";

// in your stimulus controller somewhere
enter(this.modalTarget);
leave(this.modalTarget);

The whole library is really just one, 60 line file so you can even just drop it into your project directly if you want to vendor it.

My recommendation: el-transition

All three libraries could do what I wanted: apply Vue/Alpine style data- attribute transitions.

I had the most success with el-transition and selected it for my project.

I liked that it was super simple and not tied to the framework. It has a minimal surface-area and I didn’t have to rely on an external library to update for newer Stimulus releases if there are breaking changes.

One extra bonus was that enter and leave functions returned Promise objects, which worked much better for coordinating multiple elements that need to transition (this is pretty common in the TailwindUI components).

Let’s put this advice into practice by building a slide-over menu from TailwindUI.

Start by grabbing the HTML code template (we’re using one of the free samples for this article).

In this case, we’ll create a slide-over controller, with targets for the three parts in the Tailwind markup (backdrop, panel, and closeButton) and then one more for the whole menu (container).

Notice there are several code comments that highlight various parts of the component and how to transition them.

<!--
  Background backdrop, show/hide based on slide-over state.

  Entering: "ease-in-out duration-500"
    From: "opacity-0"
    To: "opacity-100"
  Leaving: "ease-in-out duration-500"
    From: "opacity-100"
    To: "opacity-0"
-->
<div class="fixed inset-0 transition-opacity bg-gray-500 bg-opacity-75"></div>

For each of these component parts, we’re going to bind them as Stimulus targets and also add in the data attributes to match the specification.

<div data-controller="slide-over">
  ...

  <div
    data-slide-over-target="backdrop"
    class="fixed inset-0 transition-opacity bg-gray-500 bg-opacity-75"
    data-transition-enter="ease-in-out duration-500"
    data-transition-enter-start="opacity-0"
    data-transition-enter-end="opacity-100"
    data-transition-leave="ease-in-out duration-500"
    data-transition-leave-start="opacity-100"
    data-transition-leave-end="opacity-0"
  ></div>

  ...
</div>

Repeat this for the other elements that we want to animate. We’ll also add a basic <button> to show the panel when clicked.

Here is the full markup:

<div data-controller="slide-over">
  <button class="form-input" data-action="slide-over#show">
    Show slideover
  </button>

  <!-- This example requires Tailwind CSS v2.0+ -->
  <div
    data-slide-over-target="container"
    class="relative z-10 hidden"
    aria-labelledby="slide-over-title"
    role="dialog"
    aria-modal="true"
  >
    <!--
      Background backdrop, show/hide based on slide-over state.

      Entering: "ease-in-out duration-500"
        From: "opacity-0"
        To: "opacity-100"
      Leaving: "ease-in-out duration-500"
        From: "opacity-100"
        To: "opacity-0"
    -->
    <div
      data-slide-over-target="backdrop"
      class="fixed inset-0 transition-opacity bg-gray-500 bg-opacity-75"
      data-transition-enter="ease-in-out duration-500"
      data-transition-enter-start="opacity-0"
      data-transition-enter-end="opacity-100"
      data-transition-leave="ease-in-out duration-500"
      data-transition-leave-start="opacity-100"
      data-transition-leave-end="opacity-0"
    ></div>

    <div class="fixed inset-0 overflow-hidden">
      <div class="absolute inset-0 overflow-hidden">
        <div
          class="fixed inset-y-0 right-0 flex max-w-full pl-10 pointer-events-none"
        >
          <!--
            Slide-over panel, show/hide based on slide-over state.

            Entering: "transform transition ease-in-out duration-500 sm:duration-700"
              From: "translate-x-full"
              To: "translate-x-0"
            Leaving: "transform transition ease-in-out duration-500 sm:duration-700"
              From: "translate-x-0"
              To: "translate-x-full"
          -->
          <div
            data-slide-over-target="panel"
            class="relative w-screen max-w-md pointer-events-auto"
            data-transition-enter="transform transition ease-in-out duration-500 sm:duration-700"
            data-transition-enter-start="translate-x-full"
            data-transition-enter-end="translate-x-0"
            data-transition-leave="transform transition ease-in-out duration-500 sm:duration-700"
            data-transition-leave-start="translate-x-0"
            data-transition-leave-end="translate-x-full"
          >
            <!--
              Close button, show/hide based on slide-over state.

              Entering: "ease-in-out duration-500"
                From: "opacity-0"
                To: "opacity-100"
              Leaving: "ease-in-out duration-500"
                From: "opacity-100"
                To: "opacity-0"
            -->
            <div
              data-slide-over-target="closeButton"
              class="sm:-ml-10 sm:pr-4 absolute top-0 left-0 flex pt-4 pr-2 -ml-8"
              data-transition-enter="ease-in-out duration-500"
              data-transition-enter-start="opacity-0"
              data-transition-enter-end="opacity-100"
              data-transition-leave="ease-in-out duration-500"
              data-transition-leave-start="opacity-100"
              data-transition-leave-end="opacity-0"
            >
              <button
                type="button"
                data-action="slide-over#hide"
                class="hover:text-white focus:outline-none focus:ring-2 focus:ring-white text-gray-300 rounded-md"
              >
                <span class="sr-only">Close panel</span>
                <!-- Heroicon name: outline/x -->
                <svg
                  class="w-6 h-6"
                  xmlns="http://www.w3.org/2000/svg"
                  fill="none"
                  viewBox="0 0 24 24"
                  stroke-width="2"
                  stroke="currentColor"
                  aria-hidden="true"
                >
                  <path
                    stroke-linecap="round"
                    stroke-linejoin="round"
                    d="M6 18L18 6M6 6l12 12"
                  />
                </svg>
              </button>
            </div>

            <div
              class="flex flex-col h-full py-6 overflow-y-auto bg-white shadow-xl"
            >
              <div class="sm:px-6 px-4">
                <h2
                  class="text-lg font-medium text-gray-900"
                  id="slide-over-title"
                >
                  Panel title
                </h2>
              </div>
              <div class="sm:px-6 relative flex-1 px-4 mt-6">
                <!-- Replace with your content -->
                <div class="sm:px-6 absolute inset-0 px-4">
                  <div
                    class="h-full border-2 border-gray-200 border-dashed"
                    aria-hidden="true"
                  >
                    Your content goes here! How about a lazy-loaded turbo-frame?
                  </div>
                </div>
                <!-- /End replace -->
              </div>
            </div>
          </div>
        </div>
      </div>
    </div>
  </div>
</div>

And now we need to actually implement the Stimulus controller to run the transitions.

import { Controller } from "@hotwired/stimulus";
import { enter, leave } from "el-transition";

export default class extends Controller {
  static targets = ["container", "backdrop", "panel", "closeButton"];

  show() {
    this.containerTarget.classList.remove("hidden");
    enter(this.backdropTarget);
    enter(this.closeButtonTarget);
    enter(this.panelTarget);
  }

  hide() {
    Promise.all([
      leave(this.backdropTarget),
      leave(this.closeButtonTarget),
      leave(this.panelTarget),
    ]).then(() => {
      this.containerTarget.classList.add("hidden");
    });
  }
}

When the show action is called (by clicking the button), we remove the hidden class on the whole container and then run the enter function from el-transition on each of the targets we want to animate. This will fade in the backdrop and close button and slide over the panel using the Tailwind classes we defined in the data attributes.

When we trigger the hide action (by clicking the close button), we do everything in reverse. We run the leave function and the panel slides back over and the backdrop and close button fade away. Once all the transitions are done, we hide the whole container. By using Promise.all we can wait all of the individual transitions to finish (remember they may have different durations!) before hiding the container.

No need for setTimeout or flashing of content when the transition is finished and then removed!

It’s not quite as convenient as dropping in the React or Vue snippets from TailwindUI, but it’s pretty close!

You may want to take this a step further by extracting the markup into a partial or use ViewComponent to clean up the code, but that is left as an exercise for the reader.

Wrap it up

It’s great to follow other front-end communities and bring back ideas into your own ecosystem. Vue and Alpine have established a really clear, understandable pattern for specifying CSS transitions and we can leverage that work in a StimulusJS/Hotwire project with a small library.

These transitions take a bit of time to wrap your head around, but they add a nice level of polish to your UI components with minimal effort: exactly the kind of high leverage techniques we want when building apps in the “boring Rails” style.

Dynamic user content in Rails with Liquid tags

2022-01-23T13:00:00+00:00

When building features that accept user-generated content, you may need to display dynamic content based on what the user specifies. Imagine you want to users to be able to customize a welcome message sent from your application when they invite someone to their account.

Rails programmers are deeply familiar with writing content with pieces of dynamic text: we do this all the time when writing view templates. But we don’t want to allow users to write ERB or HAML strings and execute them in our app. It’s both a huge security risk and also not super friendly for users to have to learn a complete programming language to change some text.

An alternative might be use some “magic strings” where you can swap in values for special strings like $NAME or *|EMAIL|*. These are sometimes called “merge tags”. But implementing this approach usually ends up with a soup of gsub, regular expressions, and weird edge-cases.

A great option for building this feature is to use Liquid templates. Liquid is a very striped down templating language that is most commonly used by Shopify for allowing store owners to customize their ecommerce stores.

Liquid templates solves all these issues.

There is security because you control the context used when rendering – this is a fancy way of saying that Liquid templates can only access data that you explicitly pass in. The syntax is minimal and it comes with built-in functions for common operations (default values, capitalization, date formats, etc). And the library parses the input and doesn’t rely on fragile regular expressions – instead of randomly breaking, you can catch invalid syntax and handle it appropriately.

Usage

Add the liquid gem to your project.

The basic operation is two steps:

# Create a template from user-input
template = Liquid::Template.parse("Hi {{ customer.name }}!")

# Render the template with the dynamic data
template.render({"customer": {"name": "Matt"}})
#=> "Hi Matt!"

In practice, there are a few conveniences you’ll want to incorporate into your own Rails view helper.

Liquid requires the dynamic data hash to have string keys, whereas Rails apps often use symbol keys for hashes. You can call the Rails deep_stringify_keys method on a hash to convert them.
Calling render! instead of render will raise an exception so that you can fallback to returning the raw user-input.
Liquid provides strict_variables and strict_filters options that can turn undefined variables or filters into errors. You likely want these both to be true so that users can figure out syntax errors instead of the content silently being blank.

In my project, we added this helper method:

# app/helpers/liquid_helper.rb
module LiquidHelper
  def liquid(text, context: {})
    template = Liquid::Template.parse(text)
    template.render!(context.deep_stringify_keys, {
      strict_variables: true,
      strict_filters: true
    })
  rescue Liquid::Error
    text.to_s
  end
end

And then in any view (or mailer) in your Rails app, you can call the liquid helper to display dynamic user-generated content.

@campaign = Campaign.create!(subject: "Welcome {{ customer.name }}!")

<%= liquid(@campaign.subject, context: { customer: { name: @customer.name } }) %>

Note: if you are using rich-text via ActionText, you’ll need to call html_safe after the Liquid interpolation since the output is raw HTML.

<%= liquid(@campaign.message, context: { customer: { name: @customer.name } }).html_safe %>

This helper gracefully handles error cases by returning the original input.

liquid("Hi {{ missing_value }}", context: {})
#=> "Hi \{\{ missing_value }}"

liquid("Hi {{ foo", context: {})
#=> "Hi \{\{ foo"

You may also want to add convenience methods to your models if you are using them in the Liquid rendering context (instead of building up the context hash every time).

class Customer < ApplicationRecord
  belongs_to :organization

  def to_liquid
    # Expose whatever fields you want to be able to use in liquid templates
    {
      name: name,
      email: email,
      company_name: organization.name
    }
  end
end

liquid("New sign up from {{ customer.company_name}}. Say hi to {{ customer.name }} <{{ customer.email }}>!", context: customer.to_liquid)
#=> New sign up from Arrows. Say hi to Matt <matt@arrows.to>!

Even more advanced battle-tested features

You can register your own filters if you want to provide application-specific functions for users like {{ customer | avatar_url }} or {{ task.due_date | next_business_day }} or {{ '#7ab55c' | color_to_rgb }}.

You can assign resource_limits to avoid extremely slow interpolations.

These are outside the scope of this tip and you can explore on your own.

References

Liquid docs: Shopify/liquid

Liquid for Programmers: wiki

Accessing Rails environment variables from a StimulusJS Controller

2022-01-20T13:00:00+00:00

Environment variables are a great way to configure your Rails apps. You can use the Rails.env variable to conditionally change behavior when you are in development/test or production. And you can add your own application specific variables for things like API tokens or global settings.

While Stimulus has a values API to read from HTML data attributes, sometimes you need data that you don’t want to pass in every time you create a controller.

One alternative is to use <meta> tags when rendering your view. Then, inside your Stimulus Javascript code, you can query the <meta> DOM element to retrieve the value.

Usage

First, add the <meta> tag to your application template inside the <head> section. For this example, let’s say we want to know the Rails.env because we want to change a setting when running the app in test mode (I recently had to do this to change a setting in a 3rd-party library to work with headless system tests).

You can use the Rails tag helper to avoid a bunch of string interpolation:

# Inside the <head> tag of your application layout

<%= tag :meta, name: :rails_env, content: Rails.env %>

This will render a <meta> tag like:

<meta name="rails_env" content="development">

Next, in your Stimulus controller, query for the tag and read the content.

import { Controller } from '@hotwired/stimulus'

export default class extends Controller {
  connect() {
    if (this.isTestEnvironment) {
      // Do something for testing
    } else {
      // Do something for dev/prod
    }
  }

  get isTestEnvironment() {
    return document.head.querySelector("meta[name=rails_env]").content === "test"
  }
}

You can also extract this into a helper if you want – here is a utility function from the Rails request.js library which uses this same <meta> tag pattern to access the Rails CSRF token when sending Javascript requests.

function metaContent (name) {
  const element = document.head.querySelector(`meta[name="${name}"]`)
  return element && element.content
}

Remember that this data is visible in your page source! Unlike environment variables that only exist on the server, putting data into your HTML makes it accessible to end-users. Make sure you are only exposing data that is safe to be public.

References

MDN: metadata element

Rails validations: database level check constraints

2022-01-18T13:00:00+00:00

One of the most common Rails tips is to back up your ActiveRecord model validations with database level constraints.

Because there are times when validations are skipped, it’s best to let your database be the last line of defense to maintain your data integrity. You can add a validates :name, presence: true line to your model, but if null values sneak into your database, your app will still throw an exception if you call name.downcase on nil.

The most common examples are pairing presence validations with non-null columns and unique validations with a unique database index.

But did you know you can go a step further using a database feature called “check constraints”.

Usage

Check constraints run when you attempt to store data into a column. If the data violates the constraint, an error is raised and Rails will rollback the transaction.

At Arrows, we help onboard customers by creating custom plans. Plans are based on a Template and we recently added a way to set the deadline to a fixed number of days after the Plan gets created.

We stored the deadline offset as an integer in the database. But in the context of our application, this number should never be negative. When we create a Plan from the Template, we want to write code like:

deadline = Date.current + @template.deadline_offset

But since the integer type in the database can be negative, we didn’t have a guarantee that the offset would be positive. Obviously, we don’t want the deadline to ever be before the plan is even created.

We can validate this at the model level, but we would also like the database to enforce this check.

class Template < ApplicationRecord
  validates :deadline_offset, numericality: {
        only_integer: true,
        greater_than_or_equal_to: 0
  }
end

As of Rails 6.1, you can specific check constraints straight from a migration.

class AddDeadlineOffsetCheckToTemplates < ActiveRecord::Migration[7.0]
  def change
    add_check_constraint :templates, "deadline_offset >= 0",
      name: "deadline_offset_non_negative"
  end
end

You pass in a name for the constraint and then the SQL condition to run for the check.

Now if you forgot the ActiveRecord model validation or accidentally bypass it by skipping callbacks, this check is enforced at the database level.

# Live dangerously and skip validations!
=> template.deadline_offset = -12
=> template.save(validate: false)

PG::CheckViolation: ERROR:  new row for relation "templates" violates check constraint "deadline_offset_non_negative" (ActiveRecord::StatementInvalid)

Other common examples

Here are a few more places you might want to use a check constraint:

Ensuring a minimum price as a safety mechanism: check that price > 100 to ensure no one accidentally adds a product below a minimum level
Validating fixed formats: storing a US zipcode? Make sure char_length(zipcode) = 5
Enforcing a relationship between two columns: add a constraint that start_date < end_date or sale_price <= price

Should you always add check constraints? I’ve never run into cases where I was unhappy having extra protection when it comes to data integrity. If you have mission-critical data requirements, I would highly recommend adding check constraints.

Are they strictly necessary for every little thing? Probably not. But you can weigh the headache of fixing data issues with the cost of adding constraints for each validation in your app.

References

Rails API: add_check_constraint

Thoughtbot blog: Validation, Database Constraint, or Both?

Debugging slow Heroku builds

2021-05-17T13:00:00+00:00

It’s always best to follow a systematic approach when trying to speed up slow code.

First, measure the current performance.

Next, make the change that you think will help.

Lastly, measure again to see if the change worked.

It’s no different when it comes to debugging slow test suites or deploys.

I recently noticed that my Heroku deploys were taking nearly 10 minutes to build.

Thanks to a tip from my friend Anthony, I was able to quickly measure, diagnose, and deploy a fix that shaved nearly 4 minutes off every Heroku build. Big win!

Usage

Heroku uses “buildpacks”, which are essentially shell scripts that run to install and configure your server.

You can add multiple buildpacks: it’s common for a Ruby on Rails app to have a Ruby buildpack (for running the Rails app) and a Node buildpack (for building Javascript assets).

But even if you watch the build log as your code changes deploy, there aren’t many affordances for knowing what parts of the process are slow.

Enter the heroku-buildpack-timestamps buildpack!

You can add this buildpack to your Heroku app and it will output timestamps of each step in the process.

2021-05-10 19:30:59
2021-05-10 19:30:59 -----> Creating runtime environment
2021-05-10 19:30:59
2021-05-10 19:30:59        NPM_CONFIG_LOGLEVEL=error
2021-05-10 19:30:59        YARN_PRODUCTION=true
2021-05-10 19:30:59        NODE_ENV=production
2021-05-10 19:30:59        NODE_MODULES_CACHE=true
2021-05-10 19:30:59        NODE_VERBOSE=false
2021-05-10 19:30:59
2021-05-10 19:30:59 -----> Installing binaries
2021-05-10 19:30:59        engines.node (package.json):  unspecified (use default)
2021-05-10 19:30:59        engines.npm (package.json):   unspecified (use default)
2021-05-10 19:30:59        engines.yarn (package.json):  unspecified (use default)
2021-05-10 19:30:59
2021-05-10 19:30:59        Resolving node version 14.x...
2021-05-10 19:31:00        Downloading and installing node 14.17.0...
2021-05-10 19:31:01        Using default npm version: 6.14.13
2021-05-10 19:31:01        Resolving yarn version 1.22.x...
2021-05-10 19:31:01        Downloading and installing yarn (1.22.10)
2021-05-10 19:31:02        Installed yarn 1.22.10
2021-05-10 19:31:03
2021-05-10 19:31:03 -----> Restoring cache
2021-05-10 19:31:03        - node_modules
2021-05-10 19:31:05
2021-05-10 19:31:05 -----> Building dependencies
2021-05-10 19:31:06        Installing node modules (yarn.lock)
2021-05-10 19:31:07        yarn install v1.22.10
2021-05-10 19:34:10        Done in 191.01s.
2021-05-10 19:34:10
2021-05-10 19:34:10 -----> Caching build
2021-05-10 19:34:10        - node_modules
2021-05-10 19:34:17
2021-05-10 19:34:17 -----> Pruning devDependencies
2021-05-10 19:34:17        Skipping because YARN_PRODUCTION is 'true'

I added this buildpack to my staging Heroku app and was able to pinpoint the hot spots in the process. My findings showed that running yarn install was taking 4 minutes, even when there were no Javascript changes between builds.

After identifying the troublesome build step, I decided to turn on the --verbose flag for yarn to see why things were so slow. The official nodejs buildpack is supposed to automatically cache npm packages so it was strange that every build was spending 4 minutes installing.

The --verbose flag generated a huge amount of extra detail to help me find the problem – over 23k lines of logging…

Downloading binary from https://github.com/sass/node-sass/releases/download/v4.13.1/linux-x64-83_binding.node
Cannot download "https://github.com/sass/node-sass/releases/download/v4.13.1/linux-x64-83_binding.node":

HTTP error 404 Not Found

Hint: If github.com is not accessible in your location
      try setting a proxy via HTTP_PROXY, e.g.

      export HTTP_PROXY=http://example.com:1234

or configure npm proxy via

      npm config set proxy http://example.com:8080

Building: /tmp/build_496d129f/.heroku/node/bin/node /tmp/build_496d129f/node_modules/node-gyp/bin/node-gyp.js rebuild --verbose --libsass_ext= --libsass_cflags= --libsass_ldflags= --libsass_library=

Buried deep in the logs was the culprit. We were trying to download a pre-compiled node-sass binary, but it wasn’t found (because Heroku was running Node 14 and the version we asked for was only built for Node 10). After timing out, we then built the native package manually. Aha!

(The fix was to update webpacker).

While this fix was specific to my project, the process is generalizable: measure your Heroku build to find the slow parts, then dig deeper until you can find the underlying problem.

If you find yourself waiting and waiting for your Heroku builds to finish, try adding this buildpack so you can debug why it’s slow.

Quickly explore your data with `uniq` and `tally`

2021-05-05T13:00:00+00:00

A common question you may want to answer on user-input data is: what values have been entered and how many times is each one used?

Maybe you have a list of dropdown options and you want to investigate removing a rare-used option.

Ruby has two handy methods that I reach for often: uniq and tally.

Usage

The uniq method operates on an enumerable and compresses your data down to unique values.

Outreach::Task.all.map(&:status).uniq
=> ["Confirmed w/o Outreach",
 "Awaiting Outreach",
 "Responded",
 "No Response Expected",
 "Follow-up",
 "Awaiting Reply"]

While most developers are familiar with uniq, the tally method is one of the best kept secrets in Ruby. The tally method takes an enumerable of values and returns a hash where the keys are unique values and the values are the number of times the value appeared in the list.

Outreach::Task.all.map(&:status).tally
=> {"Confirmed w/o Outreach"=>106,
 "Awaiting Outreach"=>28,
 "Responded"=>48,
 "No Response Expected"=>10,
 "Follow-up"=>4,
 "Awaiting Reply"=>8}

These two methods are great to have in your toolbox to quickly explore your data in a Rails console.

Additional Resources

Ruby API: Enumerable#uniq

Ruby API: Enumerable#tally

Improving your Rails mailers with `email_address_with_name`

2021-05-03T13:00:00+00:00

In almost all email programs, you can add a display name before your email address like so:

To: Matt Swanson <matt@example.com>

It’s a small touch, but it is a more human-readable way of addressing an email. Rails provides a helper utility to format email addresses in this style without resorting to manual string manipulation.

Usage

Use email_address_with_name to add a name in-front on an email address in a standard way

ActionMailer::Base.email_address_with_name("swan3788@gmail.com", "Matt Swanson")
=> "Matt Swanson <swan3788@gmail.com>"

This helper is available in all Rails mailers.

class UserMailer < ApplicationMailer
  default from: 'notifications@example.com'

  def welcome_email
    @user = params[:user]

    mail(
      to: email_address_with_name(@user.email, @user.display_name),
      subject: 'You have a new message'
    )
  end
end

Options

This helper handles nil gracefully as well.

ActionMailer::Base.email_address_with_name("swan3788@gmail.com", nil)
=> "swan3788@gmail.com"

And it handles escaping characters automatically:

ActionMailer::Base.email_address_with_name("mike@example.com", "Michael J. Scott")
=> "\"Michael J. Scott\" <mike@example.com>"

ActionMailer::Base.email_address_with_name("chip@example.com", 'John "Chip" Smith')
=> "\"John \\\"Chip\\\" Smith\" <chip@example.com>"

Additional Resources

Rails API: ActionMailer::Base#email_address_with_name

Building lightweight components with Rails Helpers and Stimulus

2021-04-12T13:00:00+00:00

Custom Rails helpers modules are often overlooked, but they can be a great option for building lightweight components and reducing boilerplate in your Stimulus controllers.

One nice thing about Stimulus is that you can quickly infer the functionality just from reading the markup attributes, but for components that have a couple of values and actions, you can benefit from hiding some of the implementation details.

Let’s take the example from my GitHub-style Hovercards article:

<div class="inline-block"
  data-controller="hovercard"
  data-hovercard-url-value="<%= hovercard_shoe_path(shoe) %>"
  data-action="mouseenter->hovercard#show mouseleave->hovercard#hide"
>
  <%= link_to shoe.name, shoe, class: "branded-link" %>
</div>

The hovercard_controller needs to be passed in a url value and have two actions added for showing and hiding the card on hover. The controller is wrapped around a link that can be styled and customized for each type of hovercard we want in the app.

If you have only a few places using the controller, it’s not a big deal. But if you want to re-use this controller for more and more types of hovercards, try adding your own Rails helper.

Usage

Modules in the app/helpers folder will automatically be available for you to use in your views.

# app/helpers/hovercard_helper.rb
module HovercardHelper

  # Use a helper to avoid repeating Stimulus controller attributes
  def hovercard(url, &block)
    content_tag(:div,
      "data-controller": "hovercard",
      "data-hovercard-url-value": url,
      "data-action": "mouseenter->hovercard#show mouseleave->hovercard#hide",
      &block)
  end

  # Build your own light-weight "components"
  def repo_hovercard(repo, &block)
    hovercard hovercard_repository_path(repo), &block
  end

  def user_hovercard(user, &block)
    hovercard hovercard_user_path(user), &block
  end
end

Using a helper allows us to create our own “components” in Ruby to abstract away the implementation details. And because of the power of Ruby blocks, we can create flexible components that can be customized per usage.

<!-- app/views/timeline.html.erb -->

<%= user_hovercard(@user) do %>
  <%= link_to @user.username, @user %>
<% end %>

<%= repo_hovercard(repository) do %>
  <div class="flex items-center space-x-2">
    <svg></svg> <!-- Some icon -->
    <%= link_to repository.name, repository %>
  </div>
<% end %>

For example, we could build a repo_hovercard helper that accepts a Repository model and a block to render. We have full control over what to display based on the page context but we don’t have to worry about wiring up Stimulus events correctly.

And if we want to change our Stimulus controller, it’s all in one spot, instead of spread out across many views in the app.

Additional Resources

Rails API: Helpers

Combine `redirect_to` and the `anchor` option

2021-04-01T13:00:00+00:00

Often you’ll have an application screen like this:

After editing information about an employee, you’ll redirect back to the Company Directory page.

For a bit of extra polish, you can redirect with an anchor to automatically scroll the browser to the recently updated item and maintain your position in the list.

You can combine this with the most underrated Rails helper – dom_id – for a really clean solution.

Usage

Your controller will be exactly the same, just tack on the anchor option to the path helper when redirecting.

class EmployeesController < ApplicationController
  include ActionView::RecordIdentifier # adds `dom_id`

  def update
    @company = Company.find(params[:company_id])
    @employee = @company.employees.find(params[:id])

    if @employee.update(employee_params)
      redirect_to companies_path(@company, anchor: dom_id(@employee),
        notice: "Updated #{@employee.name}!"
    else
      render :edit, status: :unprocessable_entity
    end
  end

  private

  def employee_params
    params.require(:employee).permit(:name, :department, :location)
  end
end

Just make sure you use dom_id in your view as well:

<div class="flex flex-col divide-y">
  <%= @company.employees.each do |employee| %>
    <%= content_tag :div, id: dom_id(employee) do %>
      <div class="flex flex-col space-y-2">
        <span class="font-bold"><%= employee.name %></span>
        <span class="text-gray-400">
          <%= employee.department %>
          &middot;
          <%= employee.location %>
        </span>
      </div>
    <% end %>
  <% end %>
</div>

It’s a small tweak, but it’s easy to do and makes your application just a bit more pleasant.

Additional Resources

Rails API: #url_for Options

Rails API: ActionView::RecordIdentifier#dom_id

Rails API: ActionView::TagHelper#content_tag

Lazy-loading content with Turbo Frames and skeleton loader

2021-03-30T13:00:00+00:00

Hotwire is a new suite of frontend tools from Basecamp for building “reactive Rails” apps while writing a minimal amount of JavaScript.

While the most exciting feature to some is the real-time streaming of server rendered HTML, my favorite addition is the Turbo Frame.

The Turbo Frame is a super-charged iFrame that doesn’t make you cringe when you use it. Frames represent a slice of your page and have their own navigation context.

One incredibly powerful feature is lazy-loading a Frame. One example of this pattern that you probably see everyday is the GitHub activity feed:

First you load the “outer shell” of the page and then you can make an AJAX call to fetch more content to fill in the rest of the page. It’s a great way to speed up a slow page.

But one downside is that the page content jumps around a bit. The “Loading” spinner is one small rectangle, but the result is a long feed of events.

A way to solve this problem is to use a “skeleton screen” or “skeleton loader”. This UI pattern uses a blank version of the content as a placeholder and reduces the jarring impact when the content finally loads.

These two concepts go together like peanut butter and jelly.

Usage

A basic lazy-loaded Turbo Frame looks like this:

<turbo-frame id="feed" src="/feed">
  Content will be replaced when /feed has been loaded.
</turbo-frame>

By specifying the src attribute, the Frame will automatically make an AJAX request when the page loads and replace its content with the matching <turbo-frame> in the response.

Additionally, you can set the loading property of the loading to be either “eager” (load right away) or “lazy” (load once the frame is visible on the page).

Here’s how the GitHub Activity feed might look in a Rails view:

<!-- app/views/home.html.erb -->

<div>Some other content...</div>

<%= turbo_frame_tag :feed, src: activity_feed_path, loading: :lazy do %>
  Loading...
<% end %>

You can take it to the next level by replacing the basic “Loading…” message with your own skeleton loader. Tailwind makes this really easy with the built-in animate-pulse class.

Simply add some gray rectangles as your initial frame contents:

<!-- app/views/home.html.erb -->

<div>Some other content...</div>

<%= turbo_frame_tag :feed, src: activity_feed_path, loading: :lazy do %>
  <div class="flex flex-col space-y-6">
    <% 10.times do %>
      <div class="animate-pulse flex space-x-4">
        <!-- Avatar -->
        <div class="rounded-full bg-gray-400 h-12 w-12"></div>

        <!-- Details -->
        <div class="flex-1 space-y-4 py-1">
          <div class="h-4 bg-gray-400 rounded w-3/4"></div>
          <div class="space-y-2">
            <div class="h-4 bg-gray-400 rounded"></div>
            <div class="h-4 bg-gray-400 rounded w-5/6"></div>
          </div>
        </div>
      </div>
    <% end %>
  </div>
<% end %>

One last thing: make sure your activity_feed_path action returns the content wrapped in a matching Turbo Frame so that it will automatically swap out the frame contents and replace the loading state.

class ActivityFeedController < ApplicationControler
  def show
    @events = Current.user.activity.last(20)
  end
end

Note: we don’t want the src or loading attributes set on the Frame in this response, otherwise you would create an infinite loop!

<!-- app/views/activity_feed/show.html.erb -->

<%= turbo_frame_tag :feed do %>
  <%= render partial: "feed_item", collection: @events %>
<% end %>

Once you wrap your head around the power of Turbo Frames, soon you’ll be spotting all kinds of places in your app that can benefit from lazy-loading some good old HTML.

Additional Resources

Hotwire Docs: Turbo Frames

Tailwind Docs: animate-pulse

Building a Rails CI pipeline with GitHub Actions

2021-03-28T13:00:00+00:00

GitHub Actions is an automation platform that you run directly from inside a GitHub repository.

Using GitHub Actions, you build workflows that are triggered by any kind of event. These workflows run arbitrary code as Jobs and you can piece together multiple Steps to achieve pretty much whatever you want.

Aside from automatically posting GIFs on every pull request, the most obvious use-case for this new platform is to build a testing CI/CD pipeline.

Why bother? I’m already using CircleCI / Travis / Semaphore…

Good question. All of these CI/CD platforms are, more or less, equivalent.

Dedicated CI services like CircleCI are battle-tested for the use-case of running linters, executing tests, reporting results, and kicking off deploys. GitHub Actions is more akin to LEGO – it is a generic platform for running arbitrary workflows and automation.

On the other hand, GitHub Actions come included in your GitHub projects. For public repositories, GitHub Actions are free (without limitations as of this writing). For private repositories, you get 2000 free build minutes per month per repository – and if you have an existing GitHub company plan, you’re getting upwards of 10k build minutes included at no extra cost. (Full billing/usage limits here)

All of the competing platforms have free tiers and, really, the biggest benefit to a project is to have ANY kind of CI/CD tooling set up. The differences between vendors are pretty minor.

I’m trying to keep my Rails applications as boring as possible. There is enough complexity in building products and solving customer problems that I don’t need to create more work for myself and my team with an exotic ops setup.

I also work at a custom software consultancy: I help other companies ship products and tools so it is appealing to have one less external service to juggle.

Setting up CircleCI isn’t hard, but it’s one more account to register, one more place to put a corporate credit card, one more thing that my customer might ask “hey, what is that for again?”.

Additionally, I like that we can keep our CI pipeline close to the code. One of the biggest benefits in the current wave of CI/CD tooling is the shift to Infrastructure-as-Code: you configure your build using a flat file and commit that with your project. This is miles better than fumbling around a poorly maintained Jenkins instance and changes to your pipeline can be reviewed and merged like the rest of the codebase.

GitHub Actions pushes this to the extreme. Now you have one single spot for a project’s source code, issue tracker, project management (GitHub Projects), code reviews, security alerts, and now: CI/CD testing. One account, one service, one place to look.

Migrating from CircleCI to GitHub Actions for a common Rails setup

If you’re like most people, you probably set up your CircleCI configuration on the first week of the project based on a thoughtbot blog post and haven’t touched it since. Me too.

But no worries, it is straight-forward to move from the CircleCI 2.0 configuration format to the GitHub Actions syntax.

Your configuration might be a little bit different, but conceptually a vanilla CI/CD pipeline for a Rails app:

Checks out the latest version of the code
Sets up a base image with Ruby, Node, and some browser testing stuff
Sets up a PostgreSQL database service
Installs dependencies (bundler, yarn, npm) and cache the results to speed up builds when they don’t change
Sets up the test database
Runs any linters/checkers (rubocop, eslint, brakeman, etc)
Runs the tests

Those steps run in order and if any of them fail, the build fails. You get a red :x: on your commit in GitHub, you can’t merge your PR, someone yells at you in Slack, you know the drill.

To get a build running on GitHub Actions, we simply translate those high-level actions into the matching workflow syntax.

One change I found useful to make was to take advantage of GitHub Actions allowing for parallel Jobs to run as part of a single workflow. This feature allows us to run linters and the tests as separate jobs that both must pass for the overall build to pass.

Most of the competitors do not offer much parallelization on the free/cheap tiers. While each job has to setup the basic environment, we can save a few minutes by running the final steps at the same time. (Note: this approach will use less minutes of wall clock time, but you will use more total build minutes).

Here is my current GitHub Actions workflow configuration:

# Put this in the file: .github/workflows/verify.yml

name: Verify
on: [push]

jobs:
  linters:
    name: Linters
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v2

      - name: Setup Ruby and install gems
        uses: ruby/setup-ruby@v1
        with:
          bundler-cache: true

      - name: Setup Node
        uses: actions/setup-node@v2
        with:
          node-version: 16
          cache: yarn

      - name: Install packages
        run: |
          yarn install --pure-lockfile

      - name: Run linters
        run: |
          bin/rubocop --parallel
          bin/stylelint
          bin/prettier
          bin/eslint

      - name: Run security checks
        run: |
          bin/bundler-audit --update
          bin/brakeman -q -w2

  tests:
    name: Tests
    runs-on: ubuntu-latest
    services:
      postgres:
        image: postgres:12.7
        env:
          POSTGRES_USER: myapp
          POSTGRES_DB: myapp_test
          POSTGRES_PASSWORD: ""
        ports: ["5432:5432"]

    steps:
      - name: Checkout code
        uses: actions/checkout@v2

      - name: Setup Ruby and install gems
        uses: ruby/setup-ruby@v1
        with:
          bundler-cache: true

      - name: Setup Node
        uses: actions/setup-node@v2
        with:
          node-version: 16
          cache: yarn

      - name: Install packages
        run: |
          yarn install --pure-lockfile

      - name: Setup test database
        env:
          RAILS_ENV: test
          PGHOST: localhost
          PGUSER: myapp
        run: |
          bin/rails db:setup

      - name: Run tests
        run: bin/rspec

A few notes:

Since GitHub Actions is general purpose automation platform (and not solely for CI/CD), you need to tell the action when to run: in this case, we want it to run on every every push. You can configure this further to run on only certain branches or files if you wish.
Instead of using a platform-provided Docker image with a bunch of common environment tooling setup (e.g. circleci/ruby:2.6.3-node-browsers), in GitHub Actions you are advised to compose other first-party “setup” actions that will link in binaries. In this case, we use ruby/setup-ruby and actions/setup-node to include the specific versions of ruby and node that we want to use. These steps are very quick as they are essentially linking into pre-built language binaries.
You may notice that I am using ruby/setup-ruby (a community action) instead of the first-party GitHub actions/setup-ruby. The official action has lagged behind in both features and release versions when it comes to Ruby. During the beta of GitHub Actions, we had to completely stop using Actions because you could not easily pick the specific minor-release version of Ruby and the official action was, in some cases, months behind on supporting the latest Ruby versions (include security releases). The ruby/setup-ruby action works great – you can pick any version and any flavor (jruby, truffleruby, etc) and it will pull in a pre-built binary in under 5 seconds. Additionally, the ruby/setup-ruby action supports automatically running bundle install and caching your gems without you having to manually fiddle with the cache action.
Many articles for setting up a PostgreSQL service (including the official example) include extra health-check options to make sure the database is started up before proceeding. In my experience, waiting for the health check was taking an extra 15-60 seconds. Since we have to install gems and do other setup before we try to connect to the database, I removed them to shave down the run time and I have not had any problems. :man_shrugging: At the very least, consider changing the health interval settings from the example to something like: --health-interval 10ms --health-timeout 500ms --health-retries 15. This reduced my “Initializing containers” step from ~30 seconds to ~10 seconds on EVERY BUILD.
Experiment with the alpine Postgres Docker images. These images use a stripped down Linux system that is smaller than the base Docker image. In my testing, the alpine images were about 33% faster to download and spin up. The trade-off is that you may not be running the exact same environment as your production database, but for projects that aren’t doing anything fancy with Postgres, it seems worth the trade off to me. To use these images, replace e.g. postgres:11 with postgres:11-alpine in your workflow. You can find a full list of all the official Postgres images on Dockerhub. I’ve been running with the alpine images for ~18 months now and haven’t had any weird issues.
You should check the various tools you use for options to speed up the run-time in a CI environment. For example, yarn has a --pure-lockfile option that tells it not to try to create a lockfile (since you already checked one in…) and rubocop has a --parallel flag to save a bit of time. When it comes to CI builds, saving a few seconds here and there adds up quickly.

Overall Impressions

I spent one evening setting up the workflow with the goal of getting it to feature parity with our existing CircleCI setup and I was able to achieve that.

That said, I am not a DevOps expert – I’m just trying to get a solid pipeline set up so I can focus on more important things.

GitHub Actions is a compelling option that is worthy of becoming the community default. It makes too much sense to have GitHub be the one-stop place for your repository, issue tracker, and build server. One less external service means less mental overheard and a more “boring” setup that lets us focus on shipping quickly and solving customer problems, not doing integration work.

If I was starting a new project, I would absolutely start using GitHub Actions as my default CI service.

Feb 2020 Update

This article has been updated: the original draft was written during the GitHub Actions beta period and many of the areas for improvement (namely caching) were fixed!

Mar 2021 Update

This article has been updated: switch to using built-in bundle cache options with the ruby/setup-ruby action

July 2022 Update

This article has been updated: switch to using built-in yarn cache options with the actions/setup-node step and bump various library versions

Use `to_sql` to see what query ActiveRecord will generate

2021-03-22T13:00:00+00:00

If you’re trying to write a tricky ActiveRecord query that includes joins, complex where clauses, or selecting specific values across tables, it can be hard to remember every part of the ActiveRecord DSL.

Is it joins(:orders) or joins(:order)? Should you use where(role: { name: 'Manager' }) or where(roles: { name: 'Manager' }).

It’s a good idea to test these queries in the Rails console so you can quickly iterate, but sometimes you’ll be left scratching your head because when you run the code you get a weird result like:

#<MyModel::ActiveRecord_Relation:0x23118>

And if you try to access the results, the query blows up with a cryptic error like:

ActiveRecord::StatementInvalid: PG::UndefinedTable: ERROR:  missing FROM-clause entry for table "permission"
LINE 1: ...."id" = "permissions_users"."permission_id" WHERE "permissio...

Usage

Sometimes you just want to inspect the generated SQL to debug what’s going wrong.

It’s actually really easy to do this with ActiveRecord: simply call to_sql on your query and, instead of running the SQL, it will print out the full query – even if the SQL is invalid.

User.where("email LIKE '%?'", "gmail.com").to_sql
=> "SELECT \"users\".* FROM \"users\" WHERE (email LIKE '%'gmail.com'')"

Aha! We messed up the %? syntax.

User.where("email LIKE ?", "%gmail.com").to_sql
=> "SELECT \"users\".* FROM \"users\" WHERE (email LIKE '%gmail.com')"

It’s especially helpful if you have multiple database tables involved.

User.joins(:permissions).where(permission: { key: :edit_posts }).to_sql
=> "SELECT \"users\".* FROM \"users\" INNER JOIN \"permissions_users\" ON \"permissions_users\".\"user_id\" = \"users\".\"id\" INNER JOIN \"permissions\" ON \"permissions\".\"id\" = \"permissions_users\".\"permission_id\" WHERE \"permission\".\"key\" = 'edit_posts'"

Whoops! We need to use the plural permissions in our where.

User.joins(:permissions).where(permissions: { key: :edit_posts })

This tip has saved me countless hours of debugging complex queries. I also reach for it to validate tricky queries where I want to be sure Rails is generating the intended SQL query.

Additional Resources

Rails API: ActiveRecord::Relation#to_sql

Prefer returning chainable ActiveRecord objects

2021-03-18T13:00:00+00:00

One of the best parts about ActiveRecord is the chainable query interface:

Post.includes(:comments)
  .where(published: true)
  .where(author: Current.user)
  .order(:name)

To take advantage of this strength and give you flexibility in your code, always try to return chainable objects when querying data.

Usage

It’s common to extract complex queries as your application grows.

class SpecialOffer

  def self.find_eligible_products(store, shopper)
    return [] if store.restricted?

    store.products
      .where('price >= ?', 100)
      .select{ |p| shopper.can_order?(p) }
  end
end

@products = SpecialOffer.find_eligible_products(store, shopper)
#=> [ #<Product:0x00007fb1719b7ec0>, #<Product:0x00007fb174744de8>, ... ]

While this code may work, what happens if you need to order the @products in a certain way? Or add additional logic? Or lazy-load some associations?

In this case, the return type of our SpecialOffer method are arrays. We would have to switch to using Ruby array methods like sort and select and maybe accidentally introduce an N+1 bug if we need more data.

Let’s refactor this code to make it return chainable objects.

class SpecialOffer

  def self.find_eligible_products(store, shopper)
    return Product.none if store.restricted?

    product_ids = store.products
      .where('price >= ?', 100)
      .select{ |p| shopper.can_order?(p) }
      .map(&:id)

    Product.where(id: product_ids)
  end
end

@products = SpecialOffer.find_eligible_products(store, shopper)
#=> Product::ActiveRecord_Relation

First, we make use of the none query method: this returns an empty (but still chainable!) result. You can call ActiveRecord methods like order, includes, or where on this empty relation and it will simply return no results.

Second, instead of returning the result of our complex product query directly, we collect up the right products and then return “fresh” results for just those ids. While this does incur an additional database query, we can also manipulate the results as needed.

If we want to sort the results or load an association, we can do it in the database and not be worried about any existing conditions that were run as part of the computations.

@products = SpecialOffer.find_eligible_products(store, shopper)
  .includes(:variants)
  .order(:price)

@products = SpecialOffer.find_eligible_products(store, shopper)
  .joins(:sales)
  .where("sales.count > 15")
  .order(:sku)

I’ve found this pattern to be extremely helpful for pulling out complex queries, while still maintaining flexibility to massage the data into the correct shape.

Additional Resources

Rails API: ActiveRecord::QueryMethods#none

Rails Docs: Active Record Query Interface

Rails validations: unique within a certain scope

2021-03-16T13:00:00+00:00

It’s a great idea to make your database and application validations match. If you have validates :name, presence: true in your model, you should pair it with a not null database constraint. Unique validations should be paired with a UNIQUE database index.

In real-world applications, you often have more complicated validations, but you should continue this practice whenever you can.

Something I encounter regularly is the need to have records that are unique, but within a certain scope.

Imagine you were building a typical project management tool. You might want Projects to have a unique name so they can be distinguished within your UI – but you don’t want the name to be globally unique. If I make a project called “Onboarding”, another customer should not be restricted from using that name as well.

Luckily, Rails has got us covered with a handy feature called validation scopes.

Usage

The scope option to the Rails uniqueness validation rule allows us to specify additional columns to consider when checking for uniqueness.

class Project < ApplicationRecord
  belongs_to :account

  has_many :tasks

  validates :name, presence: true, uniqueness: { scope: :account_id }
end

This rule says that “the name of this project must unique, within the scope of this account”. In other words, the combination of a name and account_id must be unique – but you can have projects with the same name in different accounts.

As we discussed earlier, you really want to back-up your application level validations with database constraints.

In this case, you’ll want to do a multiple column index. You can do this in a normal Rails migration.

class CreateProject < ActiveRecord::Migration[6.0]
  def change
    create_table :projects do |t|
      ...
    end

    add_index :projects, [:name, :account_id], unique: true
  end
end

Options

You can pass multiple columns to scope.

If you were building a dining app and wanted to enforce that a guest could only have one reservation at a restaurant per day.

class Reservation < ApplicationRecord
  belongs_to :guest
  belongs_to :restaurant

  validates :guest_id, uniqueness: {
    scope: [ :restaurant_id, :reservation_date ]
  }
end

You may wish to change the message since the defaults error message will be fairly spartan: “{field} has already been taken”

validates :guest_id, uniqueness: {
  scope: [ :restaurant_id, :reservation_date ],
  message: "Only one reservation per guest per day is permitted"
}

Note: In PostgreSQL, the default limit for index names is 63 characters so you may find yourself needing to change the index name if your model or column names are longer.

add_index :reservations, [:guest_id, :restaurant_id, :reservation_date],
  unique: true,
  name: "idx_reserveration_guest_date_uniq"

Additional Resources

Rails API: Uniqueness Validations

PostgreSQL Docs: Postgres Constraints

MySql Docs: Multi-column Indexes

Boring breadcrumbs for Rails

2021-03-12T13:00:00+00:00

Breadcrumbs are a common UI pattern in most software applications. Rails has no built-in tools specifically for breadcrumbs, and while there are a handful of existing gems, I think this is something you can easily implement in your own app with just a few lines of code.

Ultimately, you’ll want control of how you display the breadcrumbs in your app so you might as well just own all the code for this functionality.

Usage

First, add a model for a Breadcrumb. This can live in the app/models folder but does not need to be an ActiveRecord model since we don’t need it to persist to the database.

class Breadcrumb
  attr_reader :name, :path

  def initialize(name, path)
    @name = name
    @path = path
  end

  def link?
    @path.present?
  end
end

Next add a method to your ApplicationController to store and add breadcrumbs.

class ApplicationController < ActionController::Base
  ...

  helper_method :breadcrumbs

  def breadcrumbs
    @breadcrumbs ||= []
  end

  def add_breadcrumb(name, path = nil)
    breadcrumbs << Breadcrumb.new(name, path)
  end
end

Then in your layout, you can render the breadcrumbs however you’d like. In my applications, I use the breadcrumbs in both the <title> head tag and in the page header.

<head>
  <title>
    <%= breadcrumbs.map(&:name).reverse.append("My App").join(" | ") %>
  </title>
</head>

<nav>
  <ol class="breadcrumbs">
    <% breadcrumbs.each do |crumb| %>
     <li>
      <% if crumb.link? %>
        <%= link_to crumb.name, crumb.path, class: "breadcrumb-link" %>
      <% else %>
        <span class="breadcrumb-page">
          <%= crumb.name %>
        </span>
      <% end %>

      <% unless crumb == breadcrumbs.last %>
        <span class="breadcrumb-separator">/</span>
      <% end %>
     </li>
    <% end %>
  </ol>
</nav>

It’s a simple API, but this model will allow you to add breadcrumbs in each of your controllers. The breadcrumb can include an optional path if you want it to be a link.

You can setup breadcrumbs in before_actions or inside each action. You can add conditional logic just like with any regular Ruby code.

class PostsController < ApplicationController
  before_action :set_breadcrumbs

  def index
    @posts = Post.all
  end

  def show
    @post = Post.find(params[:id])

    add_breadcrumb(@post.title, @post)
  end

  def new
    @post = Post.new

    add_breadcrumb("New Post")
  end

  private

  def set_breadcrumbs
    add_breadcrumb("Admin", admin_home_path) if Current.user.admin?
    add_breadcrumb("Posts", posts_path)
  end
end

Sexy? No. Boring? Yes. Works great? Of course.

Sharing common code between Rails controllers with `Scoped` pattern

2021-03-10T13:00:00+00:00

If you follow a strict REST / nested resources approach to building your Rails app, you might get sick of repeating common controller actions.

Try the Scoped concern pattern: a place to put shared code (setting variables, authorization) and slim down your controllers.

Usage

This particular pattern comes from DHH and Basecamp – a codebase that prides itself of using lots of tiny concerns to share bits of behavior.

While the savings of repeating the same before_actions to look up a Channel would be a fine benefit on it’s own, the naming convention of Scoped is such a great, sharp name. Playlists are “scoped” to a channel so it makes total sense that the corresponding controller would be “channel scoped”.

module ChannelScoped
  extend ActiveSupport::Concern

  included do
    before_action :set_channel, :authorize_channel
  end

  private

  def set_channel
    @channel = Channel.find(params[:channel_id])
  end

  def authorize_channel
    authorize @channel # check that user has access, etc
  end
end

class Channels::SubscriptionsController < ApplicationController
  include ChannelScoped
end

class Channels::VideosController < ApplicationController
  include ChannelScoped
end

class Channels::PlaylistsController < ApplicationController
  include ChannelScoped
end

Additional Resources

DHH Gist: Models for Nested Resources

Run different ActiveRecord validations based on context

2021-03-08T13:00:00+00:00

Sometimes want to skip certain validations on your database models. Maybe you have a multi-step wizard or want admins to have more freedom in changing data.

You might be tempted to have certain forms skip validations, but there is a better way.

Rails allows you to pass in a context when saving or validating a record. You can combine context with the on: option to run only certain ActiveRecord validations.

Usage

Let’s say you want want to build a multi-step workflow for a job board. You might allow someone to create a job listing and start filling in the data, but not fully validate everything until it is time to publish the listing.

class Listing < ApplicationRecord
  belongs_to :company
  belongs_to :user

  has_rich_text :requirements

  validates :title, presence: true, length: { maximum: 50 }

  validates :salary_range, presence: true, on: :publish
  validates :application_instructions, presence: true, on: :publish

  def publish!
    self.published_at = Time.current
    save(context: :publish)
  end
end

In this case, we will always require a title (with 50 characters max) whenever we create or edit this record. But we will only validate salary_range and application_instructions when we pass :publish as the validation context.

You could implement this workflow with controller actions like:

class ListingsController < ApplicationController

  def create
    @listing = Listing.new(listing_params)

    if @listing.save
      redirect_to @listing, notice: "Listing created"
    else
      render :new, status: :unprocessable_entity
    end
  end

  def publish
    @listing = Listing.find(params[:id])

    if @listing.publish!
      redirect_to @listing, notice: "Listing published"
    else
      render :edit, status: :unprocessable_entity
    end
  end
end

You can also add validations that are different based on which user is making the change. Maybe you want to allow admins to give special, short usernames to your friends.

Here we can set one rule that requires six character usernames in the :create context (which Rails will include by default when creating a record). Then we add a rule in the :admin context that only requires three characters.

class Account < ApplicationRecord
  validates :username, length: { minimum: 6 }, on: :create
  validates :username, length: { minimum: 3 }, on: :admin
end

Account.new(username: "swanson").valid? # => true
Account.new(username: "swanson").valid?(:admin) # => true

Account.new(username: "mds").valid? # => false
Account.new(username: "mds").valid?(:admin) # => true

Account.new(username: "a").valid? # => false
Account.new(username: "a").valid?(:admin) # => false

One downside with using Rails validation contexts is that you may not be able to use database level validations. Being able to persist partially-valid records or have conditional rules is a powerful feature, but it’s not without costs.

Think carefully about moving validations from the database constraint level to your application.

Additional Resources

Rails Guides: Validations :on option

Rails API Docs: ActiveRecord::Validations#valid?

Find records missing an association with `where.missing`

2021-03-04T13:00:00+00:00

You can’t prove a negative, but what about querying a database for a negative? While the majority of the time you are writing queries to find data, there are some cases when you want the opposite: writing a query that looks for the absence of data.

When it comes to raw SQL, you can use a LEFT OUTER JOIN combined with a NULL check to find records without certain associations.

Usage

In Rails, you can apply the same concepts from SQL directly with ActiveRecord.

Let’s say you have the following model:

class Account < ApplicationRecord
  has_many :recovery_email_addresses
end

If you wanted to find Accounts that have not yet set up a backup recovery email, you could write this query and everything will work just fine:

Account.left_joins(:recovery_email_addresses).where(recovery_email_addresses: { id: nil })

# SELECT "accounts".* FROM "accounts" LEFT OUTER JOIN "recovery_email_addresses" ON "recovery_email_addresses"."account_id" = "accounts"."id" WHERE "recovery_email_addresses"."id" IS NULL

But it’s kind of verbose. Since Rails 6.1, you can use a much cleaner shorthand to write this same query.

Account.where.missing(:recovery_email_addresses)

# SELECT "accounts".* FROM "accounts" LEFT OUTER JOIN "recovery_email_addresses" ON "recovery_email_addresses"."account_id" = "accounts"."id" WHERE "recovery_email_addresses"."id" IS NULL

This will generate identical SQL and is a lot easier to read. You can use this functionality on belongs_to relationships as well.

class Contract < ApplicationRecord
  belongs_to :promiser, class_name: "User"
  belongs_to :promisee, class_name: "User"
  belongs_to :beneficiary, optional: true, class_name: "User"
end

Contract.where.missing(:promiser) # Contracts without a promiser
Contract.where.missing(:promiser, :beneficiary) # Contracts without a promiser AND beneficiary

You can also combine missing with your normal ActiveRecord chaining methods

Contact.where("amount > ?", 1200).where.missing(:promiser)
Contact.where(signed: true).where.missing(:beneficiary)

Additional Resources

Rails repository: Finding Orphan Records

Rails API docs: WhereChain#missing

Testing multiple sessions in the same test with Capybara

2021-03-02T13:00:00+00:00

Sometimes a feature in your application will involve a back-and-forth between multiple users. When it comes time to write an automated system test, you can easily simulate switching between users using Capybara’s using_session helper.

Instead of logging in and out or faking out another user making changes to the app, you can use multiple sessions within the same Capybara test.

This can be very useful for testing features like notifications, chat, or even multi-person workflows where different users have to take action to move a process forward.

Usage

There are a few options for controlling the session in Capybara.

You can set the session manually:

Capybara.session_name = "Test session #1"

But I prefer the using_session block helper, which will run any code inside the block in a separate session and then revert back to the original session when you leave the block.

Here is an example of how you could test a basic task management system where users can assign tasks to others to complete.

describe "Task Assignment", type: :system do

  it "allows users to assign tasks to other users" do
    login as: users(:kelly)

    visit "/tasks"

    click_on "Review deliverable"

    click_button "Assign to..."

    select "Sam", from: "Assignee"
    click_button "Save"

    expect(page).to have_content("Status: Pending")
    expect(page).to have_content("Assigned: Sam")

    using_session("Sam") do
      login as: users(:sam)

      visit "/tasks/me"
      expect(page).to have_content("Review deliverable")

      click_on "Review deliverable"
      click_on "Mark complete"
    end

    refresh

    expect(page).to have_content("Status: Completed")
  end
end

Additional Resources

Capybara Docs: Using Multiple Sessions

Pluck single values out of ActiveRecord models or Enumerables

2021-02-26T13:00:00+00:00

Rails has a great, expressive term called pluck that allows you to grab a subset of data from a record. You can use this on ActiveRecord models to return one (or a few) columns.

But you can also use the same method on regular old Enumerables to pull out all values that respond to a given key.

Usage

In Rails, use pluck to query a subset of columns.

Shoe.all.map(&:name)
# SELECT "shoes.*" from "shoes"
# => ["Air Force 1", "NMD_2", "Air Jordans", ... ]
# This returns an array with all shoe names, but our database query pulled down all of the columns on the `shoes` table

Shoe.pluck(:name)
# SELECT "shoes.name" from "shoes"
# => ["Air Force 1", "NMD_2", "Air Jordans", ... ]
# Same result, but we only query exactly the columns we wanted

Shoe.pluck(:name, :brand)
# SELECT "shoes"."name", "shoes"."brand" FROM "shoes"
# => [["Air Jordan 1 Mid", "Nike"], ["Air Jordan 1 Mid", "Nike"], ... ]

Shoe.distinct.pluck(:brand)
# SELECT DISTINCT "shoes"."brand" FROM "shoes"
# => ["Nike", "Adidas", ... ]

You can also use pluck with Enumerables when using ActiveSupport:

[
  { id: 1, name: "David" },
  { id: 2, name: "Rafael" },
  { id: 3, name: "Aaron" }
].pluck(:name)

# => [ "David", "Rafael", "Aaron" ]

I find the Enumerable version to be particularly handy when dealing with JSON data from external APIs.

require "httparty"
require "active_support"
require "active_support/core_ext"

response = HTTParty.get('http://api.stackexchange.com/2.2/questions?site=stackoverflow')
questions = JSON.parse(response.body)["items"]
questions.pluck("title")

# => [
#   "JavaScript to Python - Interpreting JavasScript .filter() to a Python user",
#   "Nuxt generate and firebase gives timer warning",
#   "Variable expected error when I increment the value of a map",
#   ...
# ]

While pluck is most often used with Hashes, you can use it with any object that responds to the message you pass in – including regular Ruby objects or Structs.

Next time you find yourself calling map to get back a single value, see if your code might be improved by switching to pluck.

Additional Resources

Rails API Docs: ActiveRecord#pluck

Rails API Docs: Enumerable#pluck

Never mix up greater/less than when comparing dates again

2021-02-24T13:00:00+00:00

When it comes to compare dates, for some reason my brain really struggles. I mix up < and >= all the time and end up flipping them.

Is start_date greater than end_date? Or vice-versa? I get confused because I think about dates in terms of before and after not greater_than or less_than.

Usage

Luckily, Rails is here to save the day and make sure I never make this mistake again by adding before? and after? for all Date-related comparisons.

start_date = Date.new(2019, 3, 31)
end_date = Date.new(2019, 4, 1)

start_date.before? end_date
#=> true
end_date.after? start_date
#=> true

start_date = Date.new(2020, 8, 11)
end_date = Date.new(2018, 8, 11)

start_date.before? end_date
#=> false

I find myself using these methods frequently in Rails model validations when I need to ensure two dates form a valid range.

class Promotion < ApplicationRecord
  validate :valid_eligiblity_range?

  def valid_eligiblity_range?
    return unless expiration_date? && start_date?

    if !expiration_date.after?(start_date)
      errors.add(:start_date, "must be before Expiration Date")
      errors.add(:expiration_date, "must be after Start Date")
    end
  end
end

Additional Resources

Rails API Docs: DateAndTime::Calculations#before?

Rails API Docs: DateAndTime::Calculations#after?

Super readable String operations with `delete_prefix` and `delete_suffix`

2021-02-22T13:00:00+00:00

One reason I love writing Ruby is that it’s optimized for programmer happiness. The Ruby community values code that is super readable.

Programmers coming from other ecosystems are often shocked at much Ruby looks like pseudo-code. Between the standard library and extensions like ActiveSupport, working with Ruby means you can write code in a natural way.

A great example of this are the String methods delete_prefix and delete_suffix.

Usage

You can use delete_prefix to, as the name says, delete a substring from the beginning of a string.

"BoringRails!".delete_prefix("Boring")
#=> "Rails!"

"#programming".delete_prefix("#")
#=> "programming"

"ISBN: 9780091929787".delete_prefix("ISBN: ")
#=> "9780091929787"

"mailto:test@example.com".delete_prefix("mailto:")
#=> "test@example.com"

github_url = "https://github.com/rails/rails"
repo_name = github_url.delete_prefix("https://github.com/")
#=> "rails/rails"

Compare this method to other options like gsub:

github_url = "https://github.com/rails/rails"
repo_name = github_url.gsub("https://github.com/", "")
#=> "rails/rails"

While gsub is more flexible (it can take a Regex and swap multiple occurences), the code doesn’t read as naturally as delete_prefix. And delete_prefix is more performant!

If you want to remove characters at the end of a string, you have two main options: chomp and delete_suffix.

"report.csv".chomp(".csv")
#=> "report"

"report.csv".delete_suffix(".csv")
#=> "report"

"150 cm".delete_suffix("cm").to_i
#=> 150

Stylistically, chomp has a bit more of the whimsy that you might expect from Ruby, while delete_suffix is a nice mirroring of delete_prefix and more explicitly named. Both have similar performance benchmarks and are faster than sub.

Additional Resources

Ruby API Docs: String#delete_prefix

Ruby API Docs: String#delete_suffix

Ruby Bug Tracker: Feature discussion

String method benchmarks: Fast Ruby - String

Setting CSS classes in Markdown with Jekyll / Bridgetown

2021-02-19T13:00:00+00:00

Writing blog posts in Markdown is just great. This blog is written in Markdown!

But sometimes you might be tempted to drop down to raw HTML to add some extra styling.

For example maybe you want to write this content in markdown but have it apply a “pro-tip” CSS class so that it looks like…well, this!

Usage

Popular Ruby static site generators like Jekyll and Bridgetown use Kramdown under-the-hood to render your Markdown by default.

In Kramdown, there is a feature called “Block Inline Attribute Lists”, which is an extension of standard Markdown syntax.

A simple paragraph with an ID attribute.
{: #para-one}

> A blockquote with a title
{: .pull-quote }

You use {: CSS_SELECTOR } to attach additional HTML attributes to the rendered Markdown output.

It’s great for adding small embellishments to your posts without having to drop down to raw HTML.

Additional Resources

Kramdown Doc: Inline Attribute Lists

Use Heroku Dataclips to share query and do ad-hoc data exports

2021-02-18T13:00:00+00:00

Heroku Dataclips enable you to create SQL queries for your Heroku Postgres databases and share the results with colleagues, third-party tools, and the public. Recipients of a dataclip can view the data in their browser and also download it in JSON and CSV formats.

Usage

If you are hosting your app on Heroku, you might need to run some ad-hoc queries or share a report. Instead of generating an admin page or a CSV report, you can make a Dataclip that attaches to your production database and run raw SQL queries.

The editor has autocomplete features for tables and columns and you can export the data to CSV or JSON.

Super handy for one-off SQL queries or barebones reporting and way better than having to rails c or dig up the raw connection string!

Additional Resources

Heroku Blog: A Dialog with Your Data Using the New Dataclips

Heroku Blog: Share your Heroku Postgres data with the new Dataclips

Ensure required environment variables are set when booting up Rails

2021-02-17T13:00:00+00:00

It’s common to use environment variables to configure external services or other options in a Rails app. These ENV_VARS usually are not checked into source control, but rather configured per environment.

Rails has the concept of initializers, which is code run during the boot phase of a Rails app.

You can add a custom initializer to check that required environment variables are set to avoid exceptions later on when your code expects a value to exist.

Usage

Create a new initializer in your app and add the required variables:

# config/initializers/01_ensure_environment.rb

if Rails.env.development?
  %w[
    AWS_ACCESS_KEY_ID
    AWS_SECRET_ACCESS_KEY
    S3_BUCKET
    ALGOLIA_ID
    ALGOLIA_API_KEY
    ALGOLIA_SEARCH_KEY
    ALGOLIA_INDEX
    ALGOLIA_CAMPAIGN_INDEX
    TWITTER_API_SECRET
    TWITTER_API_TOKEN
  ].each do |env_var|
    if !ENV.has_key?(env_var) || ENV[env_var].blank?
      raise <<~EOL
      Missing environment variable: #{env_var}

      Ask a teammate for the appropriate value.
      EOL
    end
  end
end

Options

Rails initializers are loaded and executed in alphabetical order. So use a name like 01_ensure_environment.rb to control the sort order and make sure this one loads first.

You may wish to check in a sample .env.sample file into git (without any values) to make it easier for new team members to get their environment into a working state.

Additional Resources

Rails Doc: Configuring Rails apps

Search and debug gems with `bundle open`

2021-02-16T13:00:00+00:00

Ever get frustrated trying to search through code on GitHub? Or wish you could put a breakpoint in a gem so you could figure out what it was doing?

Don’t mess around with cloning the gem repo or monkey patching code in your own app. Use bundle open instead.

Usage

In your shell, run the command: bundle open GEM_NAME

bundler will open the source code for the exact version of the gem you’ve got installed in your editor. You can search the code inside your editor and even add breakpoints or make code changes locally to test out things.

Options

The bundle open command launches the editor that you’ve set via the EDITOR or BUNDLER_EDITOR environment variables. You can set this to be vim, emacs, VS Code, or whatever tool you like to use.

If you’ve made changes to the local gems for debugging or experimental purposes, you can use the bundle pristine command to restore your gems to their “original” state.

Additional Resources

Bundler Docs: bundle open

Bundler Docs: bundle pristine

Automatically cast params with the Rails Attributes API

2021-02-15T13:00:00+00:00

A common practice in Rails apps is to extract logic into plain-old Ruby objects (POROs). But often you are passing data to these objects directly from controller params and the data comes in as strings.

class SalesReport
  attr_accessor :start_date, :end_date, :min_items

  def initialize(params = {})
    @start_date = params[:start_date]
    @end_date = params[:end_date]
    @min_items = params[:min_items]
  end

  def run!
    # Do some cool stuff
  end
end

report = SalesReport.new(start_date: "2020-01-01", end_date: "2020-03-01", min_items: "10")

# But the data is just stored as strings :(
report.start_date
# => "2020-01-01"
report.min_items
# => "10"

You probably want start_date to be a date and min_items to be an integer. You could add your own basic type casting to the constructor.

class SalesReport
  attr_accessor :start_date, :end_date, :min_items

  def initialize(params)
    @start_date = Date.parse(params[:start_date])
    @end_date = Date.parse(params[:end_date])
    @min_items = params[:min_items].to_i
  end

  def run!
    # Do some cool stuff
  end
end

But even better, you could take advantage of the Attributes API to handle this casting automatically.

Usage

The Rails Attributes API is used under-the-hood to type cast attributes for ActiveRecord models. When you query for a model that has a datetime column in the database and the Ruby object that gets pulled out has a DateTime field – that’s the Attributes API at work.

We can spruce up our report model by mixing in the ActiveModel::Model and ActiveModel::Attributes modules.

class SalesReport
  include ActiveModel::Model
  include ActiveModel::Attributes

  attribute :start_date, :date
  attribute :end_date, :date
  attribute :min_items, :integer

  def run!
    # Do some cool stuff
  end
end

report = SalesReport.new(start_date: "2020-01-01", end_date: "2020-03-01", min_items: "10")

# Now the attributes are native types!

report.start_date
# => Wed, 01 Jan 2020
report.min_items
# => 10

This pattern is great for reducing boilerplate code in form objects, report objects, or any other Model-ish Ruby class in your Rails apps. Let the framework do the type casting for you, instead of trying to reimplement it yourself!

Options

As of Rails 6.1, this module is technically a private API. Use at your own risk!

The Attribute API will automatically handle type casting for most primitives. All of the basics are covered.

attribute :start_date, :date
attribute :max_size, :integer
attribute :enabled, :boolean
attribute :score, :float

You can find the full list of out-of-the-box types here: activemodel/lib/active_model/type.

The coolest part is that the types are very robust in what kind of input they accept. For example, the boolean Attribute type works with any of these values for false:

FALSE_VALUES = [
  false, 0,
  "0", :"0",
  "f", :f,
  "F", :F,
  "false", :false,
  "FALSE", :FALSE,
  "off", :off,
  "OFF", :OFF,
]

You can also register your own custom types that implement cast and serialize:

ActiveRecord::Type.register(:zip_code, ZipCodeType)

class ZipCodeType < ActiveRecord::Type::Value
  def cast(value)
    ZipCode.new(value) # cast to your own ZipCode class for special handling
  end

  def serialize(value)
    value.to_s
  end
end

Additionally, you can set a default value for with the Attributes API:

attribute :start_date, :date, default: 30.days.ago
attribute :max_size, :integer, default: 15
attribute :enabled, :boolean, default: true
attribute :score, :float, default: 9.75

Additional Resources

Rails API Docs: Attributes API

Blog post: Rails’ hidden type system

Show relevant chunks of text with Rails `excerpt` helper

2021-02-12T13:00:00+00:00

The Rails helper excerpt can extract a chunk of text that matches a certain phrase, no matter where in the string it is.

Imagine you wanted to display a list of emails matching a certain search term:

Simply filter down your records and then use excerpt on the email body.

Usage

Here’s what your view might look like to build this feature.

<%= link_to email do %>
  <div class="flex justify-between items-center">
    <p class="text-sm font-medium text-gray-900">
      <%= email.sender_name %>
    </p>
    <%= local_time_ago email.received_at %>
  </div>
  <p class="text-sm text-gray-700">
    <%= email.subject %>
  </p>
  <p class="text-sm text-gray-500">
    <%= excerpt(email.body.to_plain_text, params[:q], radius: 15) %>
  </p>
<% end %>

One trick is that if you are using ActionText, call to_plain_text otherwise you might end up with unclosed HTML tags. In general, when you show an excerpt of rich text, you don’t want all of the formatting to be applied anyways.

For bonus points, combine with the Rails highlight helper!

Options

@product.description = "A one-of-a-kind vintange DHH coffee mug signed by the man himself!"

excerpt(@product.description, "coffee", radius: 10)
=> "...tange DHH coffee mug signe..."

This helper is case-insensitive and can accept a string or Regex:

@product.description = "A one-of-a-kind vintange DHH coffee mug signed by the man himself!"

excerpt(@product.description, "COFFEE", radius: 10)
=> "...tange DHH coffee mug signe..."

excerpt(@product.description, /tea|coffee/, radius: 10)
=> "...tange DHH coffee mug signe..."

Options

You can expand the “radius” – or how many characters around the match – are shown. The default is 100.

@product.description = "A one-of-a-kind vintange DHH coffee mug signed by the man himself!"

excerpt(@product.description, "coffee", radius: 5)
=> "...DHH coffee mug..."

excerpt(@product.description, "coffee", radius: 25)
=> "...e-of-a-kind vintange DHH coffee mug signed by the man hi..."

If you need to change the ... into something else, use the omission option.

@product.description = "A one-of-a-kind vintange DHH coffee mug signed by the man himself!"

excerpt(@product.description, "coffee", radius: 10)
=> "...tange DHH coffee mug signe..."

excerpt(@product.description, "coffee", radius: 10, omission: "---")
=> "---tange DHH coffee mug signe---"

excerpt(@product.description, "coffee", radius: 10, omission: "<snip>")
=> "<snip>tange DHH coffee mug signe<snip>"

Additional Resources

Rails API Docs: TextHelper#excerpt

Rails API Docs: ActionText#to_plain_text

Use Rails `link_to_unless_current` for navigation links

2021-02-11T13:00:00+00:00

We’re all familiar with the classic Rails link_to helper. But did you know there is a link_to_unless_current variant?

It works just link link_to except that it doesn’t create a link if the browsers current URL is the same as the link target.

Usage

Simply replace link_to with link_to_unless_current. If you are not already on the page, a normal <a> tag will be rendered. If you are on the page already, only the text will be rendered.

<nav class="flex flex-col space-y-1">
  <%= link_to_unless_current "Dashboard", dashboard_path %>
  <%= link_to_unless_current "Team", teams_path %>
  <%= link_to_unless_current "Projects", projects_path %>
  ...
</nav>

When you are on /dashboard, this template will output:

<nav class="flex flex-col space-y-1">
  Dashboard
  <a href="/teams">Teams</a>
  <a href="/projects">Projects</a>
  ...
</nav>

You can also pass in a block to render (instead of the link name). This can be useful for making shared views. For instance, you could make a partial with actions for a Post.

<%= link_to_unless_current "+ Comment", new_comment_path do
  link_to "Back", posts_path
end %>

Other Variants

As you might expect, there are other variants like link_to_if and link_to_unless if you want to specify your own conditional logic for rendering a link or not.

You might want to show a list of all Posts, but if you are logged in as an admin, the title should be a link to an edit page.

<% @posts.each do |post| %>
  <%= link_to_if @user.admin?, post.title, manage_post_path(post) %>
<% end %>

Additional Resources

Rails API Docs: UrlHelper#link_to_unless_current

Rails API Docs: UrlHelper#link_to_if

Rails API Docs: UrlHelper#link_to_unless

Use Rails `cycle` to avoid `i % 2 == 0` in your view loops

2021-02-09T13:00:00+00:00

Sometimes you need to keep track of how many times you’ve looped when rendering some views, for instance to alternate background colors to create a “striped” table.

<!-- @foods = ["apple", "orange", "banana"] -->

<% @foods.each do |food| %>
  <tr class="???">
    <td><%= food %></td>
  </tr>
<% end %>

You might try using :odd or :even CSS child selectors or switching to each_with_index.

<% @foods.each_with_index do |food, i| %>
  <tr class="<%= i % 2 == 0 ? 'bg-gray-200' : 'bg-gray-100' %>">
    <td><%= food %></td>
  </tr>
<% end %>

You could even refactor a bit to use i.odd? or i.even?.

<% @foods.each_with_index do |food, i| %>
  <tr class="<%= i.even? ? 'bg-gray-200' : 'bg-gray-100' %>">
    <td><%= food %></td>
  </tr>
<% end %>

Rails offers a different helper that comes in handy for these situations: cycle.

Usage

The cycle helper takes an array of arguments and loops through them each time it is called.

We could replace the above code with:

<% @foods.each_with_index do |food, i| %>
  <tr class="<%= cycle('bg-gray-200', 'bg-gray-100') %>">
    <td><%= food %></td>
  </tr>
<% end %>

The real benefits start to appear if you need more than two options.

<% @foods.each_with_index do |food, i| %>
  <tr class="<%= cycle('bg-red-100', 'bg-orange-100', 'bg-yellow-100') %>">
    <td><%= food %></td>
  </tr>
<% end %>

If you need to manually reset or share a cycle between code, you can pass a name: key as an option.

cycle("red", "white", "blue", name: "colors")
cycle("sm", "md", "lg", "xl", name: "sizes")

reset_cycle("colors")
reset_cycle("sizes")

You can also use any object that responds to to_s in the cycle.

<% @items.each do |item| %>
  <div class="rotate-<%= cycle(0, 45, 90, 135, 180) %>">
    <%= item %>
  </div>
<% end %>

Additional Resources

Rails API Docs: TextHelper#cycle

Tailwind Docs: Even/odd variants

Use the Rails helper `highlight` when showing search results

2021-02-08T13:00:00+00:00

Use the Rails highlight helper to wrap search result matches in <mark> tags.

Highlight the search term “comment” in a list of notifcations

Usage

Pass the search term to your controller via params (e.g. params[:search]) and use that to filter down your results.

# app/controllers/inbox_controller.rb
class InboxController < ApplicationController
  def index
    @notifications = Current.user.notifications.for_search(params[:search])
  end
end

# app/models/notification.rb
class Notification < ApplicationRecord
  belongs_to :recipient, class_name: "User"

  validates :message, presence: true

  def self.for_search(term)
    if term.present?
      # Implement searching however you'd like
      where("message ILIKE ?", "%#{term}%")
    else
      all
    end
  end
end

Render the notifications and use TextHelper#highlight on the message to emphasis the matching query.

<% @notifications.each do |notification| %>
  <%= link_to highlight(notification.message, params[:search]), notification %>
<% end %>

You can style the <mark> tag however you’d like.

mark {
  background-color: yellow;
}

Options

You can pass either a string, array, or a regex as the phrase to match. Matches are case-insensitive.

highlight('Boring Rails is the best', 'rails')
# => Boring <mark>Rails</mark> is the best

highlight('Boring Rails is the best', /rails|best/)
# => Boring <mark>Rails</mark> is the <mark>best</mark>

highlight('Boring Rails is the best', ['is', 'best'])
# => Boring Rails <mark>is</mark> the <mark>best</mark>

If there are no matches or you leave the phrase blank, everything still works fine.

highlight('Boring Rails is the best', 'JavaScript')
# => Boring Rails is the best

highlight('Boring Rails is the best', '') # nil works too
# => Boring Rails is the best

You can also override the HTML markup wrapped around the matches using the highlighter option. Use \1 to reference the match. The output is sanitized by default

highlight('Boring is best', 'best', highlighter: '<b>\1</b>')
# => Boring is <b>best</b>

highlight('Boring is best', 'best', highlighter: '<a href="tagged?q=\1">\1</a>')
# => Boring is <a href=\"tagged?q=best\">best</a>

If you need to run additional code, you can pass a block to render instead.

highlight('Blog: Boring Rails', 'rails') do |match|
  link_to(match, public_share_path(term: match))
end
# => Blog: Boring <a href="/public/share?term=Rails">Rails</a>

Additional Resources

Rails API Docs: TextHelper#highlight

Magic Responsive Tables with Stimulus and IntersectionObserver

2021-01-13T13:00:00+00:00

This is a guest collaboration with Pascal Laliberté, author of Modest JS Works, a short online book for those who want to write modest JavaScript, and then focus on all the other stuff that matters in building an app.

You’re working on this data table for your app. It’s mostly server-side HTML. Nothing fancy.

But as you’re adding columns, you’ve got a problem. How are you going to handle small screens?

The table has to scroll horizontally to let the user see all the columns. The table needs to become “responsive”.

In this article, we’ll look at a side-scrolling widget used in Shopify’s Polaris UI toolkit (currently built in React), and we’ll recreate the functionality using just Stimulus without having to rewrite your data table in React.

And instead of adding resize watchers and scroll watchers like the original React component uses, we’ll be using the IntersectionObserver API, a new browser feature that’s widely available.

Quick Intro to Stimulus

Stimulus a small library that helps you add sprinkles of progressive interactivity to your existing HTML.

Just as CSS adds styling to elements as they appear in the Document Object Model (DOM), Stimulus adds interactivity (event handlers, actions, etc.) to elements as they appear in the DOM too (and removes it when they’re removed from the DOM). We’ll be using it here because it pairs so well with Rails, and with server-side rendered HTML.

And just like you can tie styling by adding CSS classes to your HTML, you can tie interactivity by adding special Stimulus data- attributes to elements. Stimulus watches for those, and when there’s a match, it fires up its interactivity (matching a Stimulus “controller” here named table-scroll).

<div data-controller="table-scroll">
  <button
    class="button button-scroll-right"
    data-table-scroll-target="scrollRightButton"
    data-action="table-scroll#scrollRight"
  >
    ...
  </button>
</div>

Re-creating the Scrolling Nav from Shopify Polaris Data Tables

Shopify’s UI library introduces a clever side-scrolling navigation widget that is only shown when there are more columns than can fit in the context. There are Buttons to scroll left and right and small dots to show how many columns are in view.

While the original is in React, we’ll recreate the functionality using Stimulus. The HTML here is from Shopify’s implementation: if you strip all of the Polaris classes, you’ll have the structure to style it to fit your own application’s styles.

So let’s start by creating the overall markup structure you’ll be coding in your application and attaching the table-scroll Stimulus controller.

(Please note that some CSS styles have been omitted for brevity, I’ve tried to call out the critical classes where possible.)

<div data-controller="table-scroll">
  <div data-table-scroll-target="navBar">
    <!-- Navigation widget -->
  </div>
  <div class="flex flex-col mx-auto">
    <div class="overflow-x-auto" data-table-scroll-target="scrollArea">
      <table class="min-w-full">
        <!-- Table contents -->
      </table>
    </div>
  </div>
</div>

Next let’s set up the targets for each column by adding an attribute to the <th> tags. We can take advantage of Stimulus’ multiple target binding by setting all of the columns to a target value of column, which will allow us to automatically bind a columnTargets array in our Stimulus controller.

<!-- Table contents -->
<table class="min-w-full">
  <thead>
    <tr>
      <th data-table-scroll-target="column">Product</th>
      <th data-table-scroll-target="column">Price</th>
      <th data-table-scroll-target="column">SKU</th>
      <th data-table-scroll-target="column">Sold</th>
      <th data-table-scroll-target="column">Net Sales</th>
    </tr>
  </thead>
  <tbody>
    <!-- Table body -->
  </tbody>
</table>

Next let’s build the markup for the navigation widget. We will use a dot icon for each of the columns and a left and right arrow to scroll the table.

<!-- Navigation widget -->
<div data-table-scroll-target="navBar">
  <!-- Left button -->
  <button data-table-scroll-target="leftButton" data-action="table-scroll#scrollLeft">
    <svg></svg>
  </button>

  <!-- Column visibility dots -->
  <% 5.times do %>
    <span class="text-gray-200" data-table-scroll-target="columnVisibilityIndicator">
      <svg></svg>
    </span>
  <% end %>

  <!-- Scroll Right button -->
  <button data-table-scroll-target="rightButton" data-action="table-scroll#scrollRight">
    <svg></svg>
  </button>
</div>

And lastly, let’s pass in some class data to define CSS styles to apply when the navigation widget should be shown or hidden and how the buttons and dots should be styled. You may choose to hard-code these classes into the Stimulus controller, but you may want to make them configurable depending on the needs of your project (for example, you may want to use this controller with multiple tables but use a different color to indicate the visible columns).

<div
  data-controller="table-scroll"
  data-table-scroll-nav-shown-class="flex"
  data-table-scroll-nav-hidden-class="hidden"
  data-table-scroll-button-disabled-class="text-gray-200"
  data-table-scroll-indicator-visible-class="text-blue-600"
>
  <!-- The rest of the markup -->
</div>

Using IntersectionObserver to bring it to life

Now that we’ve annotated the markup, we can add the Stimulus controller.

We’ll need some way of watching the scrollArea position and detecting what is visible. Unlike the Polaris implementation, we’ll use the IntersectionObserver API. No need for window.resize or window.scroll, which are more costly on performance than the new native IntersectionObserver browser API.

The IntersectionObserver API watches the visibility of elements, and a callback is fired on visibility changes. In our case, we’ll be watching the visibility of column headings.

// controllers/table_scroll_controller.js
import { Controller } from "stimulus";

export default class extends Controller {
  static targets = [
    "navBar",
    "scrollArea",
    "column",
    "leftButton",
    "rightButton",
    "columnVisibilityIndicator",
  ];
  static classes = [
    "navShown",
    "navHidden",
    "buttonDisabled",
    "indicatorVisible",
  ];

  connect() {
    // start watching the scrollAreaTarget via IntersectionObserver
  }

  disconnect() {
    // stop watching the scrollAreaTarget, teardown event handlers
  }
}

Since we’re progressively enhancing the page with Stimulus, we should take care to check if the browser supports IntersectionObserver and degrade gracefully if not.

When the controller is connected, we create an IntersectionObserver and provide a callback and then register that we want to observe all of our columnTargets.

Each time the updateScrollNavigation callback is fired, (which also fires by default when intersectionObserver is initialized), we’ll update each column heading’s data-is-visible attribute, to be checked later by the other callbacks.

import { Controller } from "stimulus";

function supportsIntersectionObserver() {
  return (
    "IntersectionObserver" in window ||
    "IntersectionObserverEntry" in window ||
    "intersectionRatio" in window.IntersectionObserverEntry.prototype
  );
}

export default class extends Controller {
  static targets = [ ... ];
  static classes = [ ... ];

  connect() {
    this.startObservingColumnVisibility();
  }

  startObservingColumnVisibility() {
    if (!supportsIntersectionObserver()) {
      console.warn(`This browser doesn't support IntersectionObserver`);
      return;
    }

    this.intersectionObserver = new IntersectionObserver(
      this.updateScrollNavigation.bind(this),
      {
        root: this.scrollAreaTarget,
        threshold: 0.99, // otherwise, the right-most column sometimes won't be considered visible in some browsers, rounding errors, etc.
      }
    );

    this.columnTargets.forEach((headingEl) => {
      this.intersectionObserver.observe(headingEl);
    });
  }

  updateScrollNavigation(observerRecords) {
    observerRecords.forEach((record) => {
      record.target.dataset.isVisible = record.isIntersecting;
    });

    this.toggleScrollNavigationVisibility();
    this.updateColumnVisibilityIndicators();
    this.updateLeftRightButtonAffordance();
  }

  disconnect() {
    this.stopObservingColumnVisibility();
  }

  stopObservingColumnVisibility() {
    if (this.intersectionObserver) {
      this.intersectionObserver.disconnect();
    }
  }

There is a bit of code to set up and register things, but it’s fairly straightforward and from here, the remaining work is to sync the visibility of the columns with the navigation widget.

You can see that we use the target binding in Stimulus to toggle on and off CSS classes in the page. And since we made the CSS class configurable, you can tweak the UI by editing the HTML, not rebuilding your JavaScript bundle.

toggleScrollNavigationVisibility() {
  const allColumnsVisible =
    this.columnTargets.length > 0 &&
    this.columnTargets[0].dataset.isVisible === "true" &&
    this.columnTargets[this.columnTargets.length - 1].dataset.isVisible ===
      "true";

  if (allColumnsVisible) {
    this.navBarTarget.classList.remove(this.navShownClass);
    this.navBarTarget.classList.add(this.navHiddenClass);
  } else {
    this.navBarTarget.classList.add(this.navShownClass);
    this.navBarTarget.classList.remove(this.navHiddenClass);
  }
}

updateColumnVisibilityIndicators() {
  this.columnTargets.forEach((headingEl, index) => {
    const indicator = this.columnVisibilityIndicatorTargets[index];

    if (indicator) {
      indicator.classList.toggle(
        this.indicatorVisibleClass,
        headingEl.dataset.isVisible === "true"
      );
    }
  });
}

updateLeftRightButtonAffordance() {
  const firstColumnHeading = this.columnTargets[0];
  const lastColumnHeading = this.columnTargets[this.columnTargets.length - 1];

  this.updateButtonAffordance(
    this.leftButtonTarget,
    firstColumnHeading.dataset.isVisible === "true"
  );
  this.updateButtonAffordance(
    this.rightButtonTarget,
    lastColumnHeading.dataset.isVisible === "true"
  );
}

updateButtonAffordance(button, isDisabled) {
  if (isDisabled) {
    button.setAttribute("disabled", "");
    button.classList.add(this.buttonDisabledClass);
  } else {
    button.removeAttribute("disabled");
    button.classList.remove(this.buttonDisabledClass);
  }
}

Lastly, we need to add the actions that are triggered when clicking the navigation buttons. When the buttons are clicked, we find the next non-visible column in the scrolling direction and then scroll the table so the leading edge of the column.

scrollLeft() {
  // scroll to make visible the first non-fully-visible column to the left of the scroll area
  let columnToScrollTo = null;
  for (let i = 0; i < this.columnTargets.length; i++) {
    const column = this.columnTargets[i];
    if (columnToScrollTo !== null && column.dataset.isVisible === "true") {
      break;
    }
    if (column.dataset.isVisible === "false") {
      columnToScrollTo = column;
    }
  }

  this.scrollAreaTarget.scroll(columnToScrollTo.offsetLeft, 0);
}

scrollRight() {
  // scroll to make visible the first non-fully-visible column to the right of the scroll area
  let columnToScrollTo = null;
  for (let i = this.columnTargets.length - 1; i >= 0; i--) {
    // right to left
    const column = this.columnTargets[i];
    if (columnToScrollTo !== null && column.dataset.isVisible === "true") {
      break;
    }
    if (column.dataset.isVisible === "false") {
      columnToScrollTo = column;
    }
  }

  this.scrollAreaTarget.scroll(columnToScrollTo.offsetLeft, 0);
}

You can view the full code via this gist or play with an interactive example via this Codepen

Wrap it up

And voila! We’ve got a pretty nifty responsive scrolling table. On large screens, it looks like a normal HTML table. But as you shrink the view port, the navigation widget appears and you can see the dots helping to show what part of the table is visible.

Overall, this controller comes in at under 200 lines of code and should be able to handle various sized tables throughout your app.

With the release of Hotwire, Stimulus is an important piece for the “last mile” of interactivity in non-SPA apps. While Stimulus is often used for running small bits of JavaScript, you can build out more robust controllers that mirror fully featured UI libraries.

Before you completely change your application architecture to use a fancy client-side framework, see if you can get by with your existing HTML markup and a bit of Stimulus.

Hacktoberfest Recap: Open source Ruby/Rails work in 2020

2020-11-16T13:00:00+00:00

Hacktoberfest is a month long event that encourages developers to contribute to open source – submit four valid pull requests and you win a tshirt. Despite the criticisms and controversy this year, I’m a fan of the event and it did help motivate me to make some contributions.

I wanted to share a recap of my submissions for Hacktoberfest because I think it’s a good sampling of what options are available to folks looking to contribute to Ruby and Rails. There are lots of ways to give back outside of the main rails/rails repository!

ViewComponent

Repository: github/view_component

Pull request: Allow preview controller to be customized via config options

The view_component gem helps you organize your server-rended view partials into “components” that are backed by a Ruby class. If you’re familiar with the built-in Rails partials and gems that implement decorator or presenter patterns (like draper), you can think of view_component as a mash-up of those two approaches.

I’ve been playing around with this gem on my own projects and think it’s heading in a great direction. I found an existing issue with a “help-wanted” tag and decided to see if I could add the feature.

The view_component gem allows you to test out your components in a sandbox mode (akin to Rail’s mailer previews). There was a feature request to support customizing the controller used to render these previews in case you want to add authorization or other special application-specific things.

The pull request was merged and release in version 2.20.0

Gems like this are a great starting point for contributing. You certainly have more experience using a gem in your own application than working deep in the Rails internals so adding features or fixing bugs in a gem will be more approachable.

It was extra helpful that the maintainers tagged issues with “help-wanted” and there was low enough “traffic” that I was able to get questions answered and my code reviewed very quickly.

Bullet

Repository: flyerhzm/bullet

Pull request: Update design for footer notification

The bullet gem is one of my go-to tools when building Rails apps. This gem will flag N+1 queries as you’re developing a feature and alert you to associations you should preload. It’s a lot nicer to find and stamp out these performance problems as you go instead of months later when they appear in production.

You can configure how bullet notifies you of problems. My default preference is a browser popup – it’s annoying but it does drive me to fix the problem instead of ignoring it. But sometimes there are false positives or I’m prototyping a feature and it becomes too much. So I went to switch the configuration to use the “footer” where it would add a banner to the page with the error message.

But I really disliked the design: it took of a ton of space and was, well, ugly.

So I decided to change it. You can see the “Before” and “After” here. The new design is expandable and gets out of the way.

The pull request was merged and will be included in the next release of the gem.

LRUG

Repository: lrug/lrug.org

Pull request: Add link to StimulusJS talk slides

LRUG is a local Ruby meetup group in London. While this contribution on GitHub was tiny (merely updating their website with a link to the slides for a talk I gave), it’s still an example of how to contribute to the Ruby community.

Meetups are always looking for volunteer speakers and given that many groups are doing virtual meetups in 2020, I thought it might be fun to branch out from my local meetup (Indianapolis) and offer to speak at two groups in Europe. I gave a talk about using StimulusJS to the Berlin Ruby user group and the London group. It was cool to be an “international speaker” and I was able to take advantage of the timezone difference to simply slot the presentation into my normal workday calendar (lunchtime in the US is evening time in the EU).

Overall I had a great experience and a warm welcome from my new Ruby friends on the otherside of the world.

Ruby For Good: Circulate

Repository: rubyforgood/circulate

Pull request: Add power source field to items

I came across the Ruby For Good organization when looking for projects to contribute to and found that they manage a number of Ruby and Rails projects that help non-profits that need software development work done. There are apps for helping with conservation, diaper banks, volunteer management, and more.

I found a project that is building an inventory management system for tool-lending libraries. The idea is that you can borrow tools from a community pool or schedule time to use machinery that is usually too expensive for an individual to own themselves.

The Ruby for Good projects seem to all have a team of people to help triage and define work items and there is a Slack channel to coordinate and ask questions. I thought it was really neat to be able to contribute to an open source project where you were writing features like most developers do as part of their “normal” job. Writing frameworks and libraries is critical, but it takes a different set of skills and mindset than doing application development.

I was able to get the project up and running locally in under an hour and I banged out a feature to add a new field to Items in the system to record what kind of power source the tool uses.

The pull request was merged and deployed to the Chicago Tool Library app.

Wrap it up

Despite the flaws and issues with incentives, I find Hacktoberfest to personally be a very motivating effort. There are several ways to contribute to the Ruby or Rails open source ecosystems – even if you aren’t keen to dive into a huge framework or project.

You can start by looking at some of your favorite gems and see if there are improvements you can make – after all, your perspective as an end-user is invaluable.

Consider giving a talk at a local (or remote!) user group. No matter how far along in your programming journey you are, you can always share something about what you’ve learned that can help someone else.

And lastly, consider checking our Ruby for Good if you’re looking to practice more of the day-to-day Rails development skills. These projects seem like a great environment for junior developers as they have dedicated folks involved to help review code and write feature specifications.

Did you complete Hacktoberfest? If so, I’d love to hear about your contributions on Twitter.

Building GitHub-style Hovercards with StimulusJS and HTML-over-the-wire

2020-06-22T13:00:00+00:00

Somewhere along the way toward our current JavaScript hellscape, programmers decided that HTML was over. We’re done with it.

The emergence of tools like React shifted programmers away from writing HTML, instead writing JSX, a fancier tag-based markup language that worked nicely inside your JavaScript.

Backends were then relegated to being dumb JSON API endpoints. Or if you were fancy and chasing upvotes, you’d use GraphQL!

But HTML? Yuck!

A Brief History of HTML-over-the-wire

One of the key pillars of Rails is to “Value integrated systems”. While the industry moves towards microservices, highly decoupled front-ends and teams, and the siren song of Programming via LEGO Bricks, Rails leans into one system that does it all – termed the Majestic Monolith.

Instead of rebuilding much of what already works in Rails in a client-side JavaScript MVC framework, apps like Basecamp, GitHub, and Shopify are able to achieve snappy page loads using the concept of “HTML-over-the-wire”.

In his seminal RailsConf 2016 talk, Sam Stephenson walks through the pieces of this stack.

By using Turbolinks (or similar libraries like pjax or Inertia) and fast HTML responses (aided by caching and avoiding excessive database queries to get sub-100ms response times), you could build high performance pages, while still hanging on to the understated benefits of stateless HTTP responses and server-side logic.

As Sam points out, it was truly a “Golden Age of Web Development”.

So while much of the industry went down the JavaScript rabbit hole – creating new innovations for reactive rendering, functional state management containers, and approximately seventy different client-side routing libraries – the quiet rebellion in Rails-land was honing these techniques and plugging along building apps out of boring server-rendered HTML.

We’re seeing a renaissance of these tools in 2020 and the excitement (at least in a small corner of the Twitter!) is reaching a fever pitch as Basecamp launches HEY: a fully-featured email client with a tiny JavaScript footprint that pushed the boundaries of the HTML-over-the-wire approach.

Turbolinks / Stimulus 20XX: The Future

The stack in 2014-2016 was:

Turbolinks/pjax
Rails UJS + js.erb templates (Server-generated JavaScript Responses)
Heavy HTML fragment caching
Rails Asset Pipeline and CoffeeScript

You can even trace the origin of these techniques back even further. I was recently sent a link to a nearly 15 year old REST “microformat” called “AHAH: Asynchronous HTML and HTTP”, which is an early version of the same ideas we’re so excited about today. (You shouldn’t be surprised to see David Hansson listed as a contributor!)

Now a “state-of-the-art” 2020 version also includes:

StimulusJS (see also AlpineJS) for lightweight event management, data binding, and “sprinkles” of behavior
Partial updates with Turbolinks via a new <template> command approach (replacing js.erb and supporting CSP)
Real-time Turbolinks updates via ActionCable (see also StimulusReflex/CableReady)
First-party support for Webpack, ES6, and new CSS approaches like Tailwind and PurgeCSS

This stack is extremely powerful and the development experience allows you to really fly. You can build fast and interactive applications with a small team, all while still experiencing the joy of a 2014-era vanilla Rails codebase.

But years of a JavaScript SPA-heavy monoculture have made it hard to learn about this stack. The community is filled with practitioners, using the tools to build software and businesses. There simply has not been the same level of content produced and so many of these tools are unknown and can be unapproachable.

One of the ways I can contribute is to light the way for those want to know more by showing some real-world examples (not a TODO list or a Counter). Once you see how you can use tools like Stimulus and HTML responses to build features where you might instead reach for a tool like React, things will start to click.

Let’s Build Something Real: Hovercards

Hovercards show extra contextual information in a popup bubble when you hover over something in your app. You can see examples of this UI pattern on GitHub, Twitter, and even Wikipedia.

This feature is really easy to build with Rails using an HTML-over-the-wire approach.

Here’s the plan:

Build a controller action to render the hovercard as HTML
Write a tiny Stimulus controller to fetch the hovercard HTML when you hover

…and that’s it.

We don’t need to make API endpoints and figure out how to structure all of the data we need. We don’t need to reach for React or Vue to make this a client-side component.

The beauty of this boring Rails approach is that the feature is dead-simple and it’s equally straightforward to build. It’s easy to reason about the code and super extensible.

For this example, let’s build the event feed for a sneaker marketplace app.

When you hover over a shoe, you see a picture, the name, the price, etc. Same for the user, you can see a mini-profile for each user.

The Frontend (Stimulus + fetch)

The markup for the link looks like:

<!-- app/views/shoes/feed.html.erb -->

<div
  class="inline-block"
  data-controller="hovercard"
  data-hovercard-url-value="<%= hovercard_shoe_path(shoe) %>"
  data-action="mouseenter->hovercard#show mouseleave->hovercard#hide"
>
  <%= link_to shoe.name, shoe, class: "branded-link" %>
</div>

Note: we are using the APIs from the Stimulus 2.0 preview release!

One of the great features of Stimulus is that you can read the markup and understand what’s happening without diving into the JavaScript.

Without knowing anything else about the implementation, you could guess how it’s going to work: this link is wrapped in a hovercard controller, when you hover (via mouseenter and mouseleave events) the card is shown or hidden.

As recommended in Writing Better Stimulus Controllers, you should pass in the URL for the hover card endpoint as a data property so that we can re-use the hovercard_controller for multiple types of cards. This also keeps us from having to duplicate the application routes in JavaScript.

// app/javascript/controllers/hovercard_controller.js

import { Controller } from "stimulus";

export default class extends Controller {
  static targets = ["card"];
  static values = { url: String };

  show() {
    if (this.hasCardTarget) {
      this.cardTarget.classList.remove("hidden");
    } else {
      fetch(this.urlValue)
        .then((r) => r.text())
        .then((html) => {
          const fragment = document
            .createRange()
            .createContextualFragment(html);

          this.element.appendChild(fragment);
        });
    }
  }

  hide() {
    if (this.hasCardTarget) {
      this.cardTarget.classList.add("hidden");
    }
  }

  disconnect() {
    if (this.hasCardTarget) {
      this.cardTarget.remove();
    }
  }
}

This is all of the JavaScript we’re going to be writing for this feature: it’s only ~30 lines and we can use this for any other hovercards in the app. There isn’t really anything app specific about this controller either, you could pull it into a separate module and re-use it across projects. It’s totally generic.

The controller uses the fetch API to call the provided Rails endpoint, gets some HTML back, and then inserts it into the DOM. As a small improvement, we use the Stimulus target API for data binding to save a reference to the card so that subsequent hovers over this link can simply show/hide the markup without making another network request.

We also choose to remove the card when leaving the page (via the disconnect lifecycle method), but you could also opt to hide the card instead depending on how you want caching to work.

The Backend (Rails + Server rendered HTML)

There is nothing magic on the frontend and it’s the same story on the backend.

# config/routes.rb
Rails.application.routes.draw do
  resources :shoes do
    member do
      get :hovercard
    end
  end
end

Setup a route for /shoes/:id/hovercard

# app/controllers/shoes_controller.rb
class ShoesController < ApplicationController
  ...

  def hovercard
    @shoe = Shoe.find(params[:id])

    render layout: false
  end
end

Write a basic controller action, the only difference being that we set layout: false so that we do not use the global application layout for this endpoint.

You can even visit this path directly in your browser to quickly iterate on the content and design. The workflow gets even better when using a utility-based styling approach like Tailwind since you don’t even need to wait for your asset bundles to rebuild!

<!-- app/views/shoes/hovercard.html.erb -->

<div class="relative" data-hovercard-target="card">
  <div data-tooltip-arrow class="absolute bottom-8 left-0 z-50 bg-white shadow-lg rounded-lg p-2 min-w-max-content">
    <div class="flex space-x-3 items-center w-64">
      <%= image_tag @shoe.image_url, class: "flex-shrink-0 h-24 w-24 object-cover border border-gray-200 bg-gray-100 rounded", alt: @shoe.name %>

      <div class="flex flex-col">
        <span class="text-sm leading-5 font-medium text-indigo-600">
          <%= @shoe.brand %>
        </span>

        <span class="text-lg leading-0 font-semibold text-gray-900">
          <%= @shoe.name %>
        </span>

        <span class="flex text-sm text-gray-500">
          <%= @shoe.colorway %>
          <span class="mx-1">
            &middot;
          </span>
          <%= number_to_currency(@shoe.price.to_f / 100) %>
        </span>
      </div>
    </div>
  </div>
</div>

The hovercard is built with a server-rended ERB template, same as any other page in the Rails app. We set the data-hovercard-target as a convenience to bind to this element back in the Stimulus controller.

Finishing Touches

The data-tooltip-arrow allows us to add a little triangle to the bubble with a bit of CSS. You can add a library like Popper if you have more advanced needs, but this single CSS rule works great and doesn’t require any external dependencies.

/* app/javascript/stylesheets/application.css */

[data-tooltip-arrow]::after {
  content: " ";
  position: absolute;
  top: 100%;
  left: 1rem;
  border-width: 2rem;
  border-color: white transparent transparent transparent;
}

And voila! We’ve built hovercards!

If we want to add a hovercard to another model type in our application (like User profiles), it almost feels like cheating. We can use the same Stimulus controller. All we need to do is add User specific template.

<!-- app/views/users/hovercard.html.erb -->

<div class="relative" data-hovercard-target="card">
  <div data-tooltip-arrow class="absolute bottom-8 left-0 z-50 bg-white shadow-lg rounded-lg p-2 min-w-max-content">
    <div class="flex space-x-3 items-center p-1">
      <%= image_tag @user.gravatar_url, class: "flex-shrink-0 h-16 w-16 object-cover bg-gray-100 rounded inset shadow-inner", alt: @user.name %>

      <div class="flex-1 flex flex-col">
        <span class="font-bold text-lg"><%= @user.name %></span>
        <div class="flex space-x-1 items-center text-sm">
          <svg class="text-orange-400 fill-current h-4 w-4" viewBox="0 0 20 20">...</svg>
          <span class="text-gray-500 italic"><%= @user.bio %></span>
        </div>
        <span class="text-gray-400 text-xs mt-1">
          Kickin' it since <%= @user.created_at.year %>
        </span>
      </div>
    </div>
  </div>
</div>

Taking it to the next level

If you want to expand this feature even further, there are a few ideas you might consider:

Removing some duplication in the hovercard templates by either: extracting a Rails partial, using a gem like github/view_component, or using the Tailwind @apply directive to create components in your stylesheets
Animating the hovercard using CSS transitions to fade in and out
Adding a delay or fancy “directional aiming” (like the Amazon mega dropdown) so that you can move your mouse more easily to the hovercard
Cancel a pending AJAX request if you move away using the AbortController for the fetch API
Explore caching the hovercards (assuming the data is not specific to a user or session) in Rails with Fragment Caching

Wrap it up

This stack is a love letter to the web. Use links and forms. Render HTML. Keep your state on the server and in the database. Let the browser handle navigation. Add sprinkles of interactivity to improve the experience. For many it feels like a step backward, but in my opinion it’s going back to the way things should be.

It’s natural to be skeptical, especially in the current climate of “JS all the things”. But you really have to give these tools a try before you really get it. Once you see that the classic ways of building software can still get the job done, it’s hard to go back to debugging node_modules conflicts or rebuilding HTML forms inside of this years framework du jour.

In this year’s RailsConf remote keynote, DHH talked about the cyclical pendulum of Hegel’s dialectics that happens in software. New ideas are recycled and rediscovered every few years and now is a great time to hop along for the ride.

Writing better StimulusJS controllers

2020-06-01T13:00:00+00:00

We write a lot of JavaScript at Basecamp, but we don’t use it to create “JavaScript applications” in the contemporary sense. All our applications have server-side rendered HTML at their core, then add sprinkles of JavaScript to make them sparkle. - DHH

In early 2018, Basecamp released StimulusJS into the world. Stimulus closed the loop on the “Basecamp-style” of building Rails applications.

It’s hard to pin down a name for this stack, but the basic approach is a vanilla Rails app with server-rendered views, Turbolinks (“HTML-over-the-wire”, pjax) for snappy page loads, and finally, Stimulus to “sprinkle” interactive behavior on top of your boring old HTML pages.

Many of the tenets of Basecamp and DHH’s approach to building software weave in-and-out of this stack:

Programmer Happiness: avoiding the ever-changing quicksand of “modern” JavaScript
Majestic Monoliths: eschewing SPAs and microservices for medium-to-large Rails apps
Small teams doing big things: conceptual compression and tooling so you can build apps with 5 people, not 50
Omakase: tools that are good alone, but amazing together

And frankly, the most compelling to me: the tradition of extracting code from real-world products (and not trying to lecture birds how to fly).

I’m excited to see more refinement of this stack as Basecamp prepares to launch HEY.

In the coming months, we should see the release of Stimulus 2.0 to sharpen the APIs, a reboot of Server-generated JavaScript Responses (SJR), and a splash of web-sockets to snap everything together.

These techniques are extremely powerful, but require seeing the whole picture. Folks looking to dive into this stack (and style of development) will feel the “Rails as a Sharp Knife” metaphor more so than usual.

But I’ve been in the kitchen for a while and will help you make nice julienne cuts (and not slice off your thumb).

Server-rendered views in Rails are a known path. Turbolinks, with a few caveats, is pretty much a drop-in and go tool these days.

So today, I’ll be focusing on how to write better Stimulus controllers.

This article is explicitly not an introduction to Stimulus. The official documentation and Handbook are excellent resources that I will not be repeating here.

And if you’ve never written any Stimulus controllers, the lessons I want to share here may not sink in right away. I know because they didn’t sink in for me!

It took 18 months of living full-time in a codebase using this stack before things started clicking. Hopefully, I can help cut down that time for you. Let’s begin!

What may go wrong

The common failure paths I’ve seen when getting started with Stimulus:

Making controllers too specific (either via naming or functionality)

It’s tempting to start out writing one-to-one Stimulus controllers for each page or section where you want JavaScript. Especially if you’ve used React or Vue for your entire application view-layer. This is generally not the best way to go with Stimulus.

It will be hard to write beautifully composable controllers when you first start. That’s okay.

Trying to write React in Stimulus

Stimulus is not React. React is not Stimulus. Stimulus works best when we let the server do the rendering. There is no virtual DOM or reactive updating or passing “data down, actions up”.

Those patterns are not wrong, just different and trying to shoehorn them into a Turbolinks/Stimulus setup will not work.

Growing pains weaning off jQuery

Writing idiomatic ES6 can be a stumbling block for people coming from the old days of jQuery.

The native language has grown leaps and bounds, but you’ll still scratch your head from time to time wondering if people really think that:

new Array(...this.element.querySelectorAll(".item"));

is an improvement on $('.item'). (I’m right there with you, but I digress…)

How to write better Stimulus controllers

After taking Stimulus for a test drive and making a mess, I revisited the Handbook and suddenly I saw the examples in a whole new light.

For instance, the Handbook shows an example for lazy loading HTML:

<div data-controller="content-loader" data-content-loader-url="/messages.html">
  Loading...
</div>

Notice the use of data-content-loader-url to pass in the URL to lazily load.

The key idea here is that you aren’t making a MessageList component. You are making a generic async loading component that can render any provided URL.

Instead of the mental model of extracting page components, you go up a level and build “primitives” that you can glue together across multiple uses.

You could use this same controller to lazy load a section of a page, or each tab in a tab group, or in a server-fetched modal when hovering over a link.

You can see real-world examples of this technique on sites like GitHub.

(Note that GitHub does not use Stimulus directly, but the concept is identical)

The GitHub activity feed first loads the shell of the page and then makes an AJAX call that fetches more HTML to inject into the page.

<!-- Snippet from github.com -->
<div class="js-dashboard-deferred" data-src="/dashboard-feed" data-priority="0">
  ...
</div>

GitHub uses the same deferred loading technique for the “hover cards” across the site.

<!-- Snippet from github.com -->
<a
  data-hovercard-type="user"
  data-hovercard-url="/users/swanson/hovercard"
  href="/swanson"
>
  swanson
</a>

By making general-purpose controllers, you start the see the true power of Stimulus.

Level one is an opinionated, more modern version of jQuery on("click") functions.

Level two is a set of “behaviors” that you can use to quickly build out interactive sprinkles throughout your app.

Example: toggling classes

One of the first Stimulus controllers you’ll write is a “toggle” or “show/hide” controller. You’re yearning for the simpler times of wiring up a click event to call $(el).hide().

Your implementation will look something like this:

// toggle_controller.js
import { Controller } from "stimulus";

export default class extends Controller {
  static targets = ["content"];

  toggle() {
    this.contentTarget.classList.toggle("hidden");
  }
}

And you would use it like so:

<div data-controller="toggle">
  <button data-action="toggle#toggle">Toggle</button>
  <div data-target="toggle.content">Some special content</div>
</div>

To apply the lessons about building more configurable components that the Handbook recommends, rework the controller to not hard-code the CSS class to toggle.

This will become even more apparent in the upcoming Stimulus 2.0 release when “classes” have a dedicated API.

// toggle_controller.js
import { Controller } from "stimulus";

export default class extends Controller {
  static targets = ["content"];

  toggle() {
    this.contentTargets.forEach((t) => t.classList.toggle(data.get("class")));
  }
}

The controller now supports multiple targets and a configurable CSS class to toggle.

You’ll need to update the usage to:

<div data-controller="toggle" data-toggle-class="hidden">
  <button data-action="toggle#toggle">Toggle</button>
  <div data-target="toggle.content">Some special content</div>
</div>

This might seem unnecessary on first glance, but as you find more places to use this behavior, you may want a different class to be toggled.

Consider the case when you also needed some basic tabs to switch between content.

<div data-controller="toggle" data-toggle-class="active">
  <div
    class="tab active"
    data-action="click->toggle#toggle"
    data-target="toggle.content"
  >
    Tab One
  </div>
  <div
    class="tab"
    data-action="click->toggle#toggle"
    data-target="toggle.content"
  >
    Tab Two
  </div>
</div>

You can use the same code. New feature, but no new JavaScript! The dream!

Example: filtering a list of results

Let’s work through another common example: filtering a list of results by specific fields.

In this case, users want to filter a list of shoes by brand, price, or color.

We’ll write a controller to take the input values and append them to the current URL as query parameters.

Base URL: /app/shoes
Filtered URL: /app/shoes?brand=nike&price=100&color=6

This URL scheme makes it really easy to filter the results on the backend with Rails.

// filters_controller.js
import { Controller } from "stimulus";

export default class extends Controller {
  static targets = ["brand", "price", "color"];

  filter() {
    const url = `${window.location.pathname}?${this.params}`;

    Turbolinks.clearCache();
    Turbolinks.visit(url);
  }

  get params() {
    return [this.brand, this.price, this.color].join("&");
  }

  get brand() {
    return `brand=${this.brandTarget.value}`;
  }

  get price() {
    return `price=${this.priceTarget.value}`;
  }

  get color() {
    return `color=${this.colorTarget.value}`;
  }
}

This will work, but it’s not reusable outside of this page. If we want to apply the same type of filtering to a table of Orders or Users, we would have to make separate controllers.

Instead, change the controller to handle arbitrary inputs and it can be reused in both places – especially since the inputs tags already have the name attribute needed to construct the query params.

// filters_controller.js
import { Controller } from "stimulus";

export default class extends Controller {
  static targets = ["filter"];

  filter() {
    const url = `${window.location.pathname}?${this.params}`;

    Turbolinks.clearCache();
    Turbolinks.visit(url);
  }

  get params() {
    return this.filterTargets.map((t) => `${t.name}=${t.value}`).join("&");
  }
}

Example: lists of checkboxes

We’ve seen how to make controllers more reusable by passing in values and using generic targets. One other way is to use optional targets in your controllers.

Imagine you need to build a checkbox_list_controller to allow a user to check all (or none) of a list of checkboxes. Additionally, it needs an optional count target to display the number of selected items.

You can use the has[Name]Target attribute to check for if the target exists and then conditionally take some action.

// checkbox_list_controller.js
import { Controller } from "stimulus";

export default class extends Controller {
  static targets = ["count"];

  connect() {
    this.setCount();
  }

  checkAll() {
    this.setAllCheckboxes(true);
    this.setCount();
  }

  checkNone() {
    this.setAllCheckboxes(false);
    this.setCount();
  }

  onChecked() {
    this.setCount();
  }

  setAllCheckboxes(checked) {
    this.checkboxes.forEach((el) => {
      const checkbox = el;

      if (!checkbox.disabled) {
        checkbox.checked = checked;
      }
    });
  }

  setCount() {
    if (this.hasCountTarget) {
      const count = this.selectedCheckboxes.length;
      this.countTarget.innerHTML = `${count} selected`;
    }
  }

  get selectedCheckboxes() {
    return this.checkboxes.filter((c) => c.checked);
  }

  get checkboxes() {
    return new Array(...this.element.querySelectorAll("input[type=checkbox]"));
  }
}

Here we can use the controller to add “Check All” and “Check None” functionality to a basic form.

We can use the same code to build a checkbox filter that displays the count of the number of selections and a “Clear filter” button (“check none”).

As with the other examples, you can see the power of creating Stimulus controllers that can be used in multiple contexts.

Putting it all together: composing multiple controllers

We can combine all three controllers to build a highly interactive multi-select checkbox filter.

Here is a rundown of how it all works together:

Use the toggle_controller to show or hide the color filter options when clicking the input

Use the checkbox_list_controller to keep the count of selected colors and add a “Clear filter” option

Use the filters_controller to update the URL when filter inputs change, for both basic HTML inputs and our multi-select filter

Each individual controller is simple and easy to implement but they can be combined to create more complicated behaviors.

Here is the full markup for this example.

<div class="filter-section">
  <div class="filters" data-controller="filters">
    <div>
      <div class="filter-label">Brand</div>
      <%= select_tag :brand,
            options_from_collection_for_select(
              Shoe.brands, :to_s, :to_s, params[:brand]
            ),
            include_blank: "All Brands",
            class: "form-select",
            data: { action: "filters#filter", target: "filters.filter" } %>
    </div>
    <div>
      <div class="filter-label">Price Range</div>
      <%= select_tag :price,
            options_for_select(
              [ ["Under $100", 100], ["Under $200", 200] ], params[:price]
            ),
            include_blank: "Any Price",
            class: "form-select",
            data: { action: "filters#filter", target: "filters.filter" } %>
    </div>

    <div>
      <div class="filter-label">Colorway</div>
      <div class="relative"
        data-controller="toggle checkbox-list"
      >
        <button class="form-select text-left"
          data-action="toggle#toggle"
          data-target="checkbox-list.count"
        >
          All
        </button>

        <div class="hidden select-popup" data-target="toggle.content">
          <div class="flex flex-col">
            <div class="select-popup-header">
              <div class="select-label">Select colorways...</div>

              <button class="clear-filters"
                data-action="checkbox-list#checkNone filters#filter"
              >
                Clear filter
              </button>
            </div>

            <div class="select-popup-list space-y-2">
              <% Shoe.colors.each do |c| %>
                <%= label_tag nil, class: "leading-none flex items-center" do %>
                  <%= check_box_tag 'colors[]', c, params.fetch(:colors, []).include?(c),
                    class: "form-checkbox text-indigo-500 mr-2",
                    data: { target: "filters.filter"} %>
                  <%= c %>
                <% end %>
              <% end %>
            </div>

            <div class="select-popup-action-footer">
              <button class="p-2 w-full select-none"
                data-action="filters#filter"
              >
                Apply
              </button>
            </div>
          </div>
        </div>
      </div>
    </div>
  </div>
</div>

Wrap it up

Stimulus works best when it’s used to add sprinkles of behavior to your existing HTML. Since Rails and Turbolinks are super effective at handling server-rendered HTML, these tools are a natural fit.

Using Stimulus requires a change in mindset from both jQuery snippets and React/Vue. Think about adding behaviors, not about making full-fledged components.

You’ll avoid the common stumbling blocks with Stimulus if you can make your controllers small, concise, and re-usable.

You can compose multiple Stimulus controllers together to mix-and-match functionality and create more complex interactions.

These techniques can be difficult to wrap your head around, but you can end up building highly interactive apps without writing much app-specific JavaScript at all!

It’s an exciting time as this stack evolves, more people find success with shipping software quickly, and it becomes a more known alternative to the “all-in on JavaScript SPA” approach.

Additional Resources

Better StimulusJS: community site for emerging best practices
Awesome StimulusJs: collection of links to articles, examples, podcasts
Stimulus Community: low traffic, but features lots of snippets and thoughts from the core team
stimulus-use: collection of composable behaviors for your controllers

Feature Flags: The stupid simple way to de-stress production releases

2020-04-12T13:00:00+00:00

“So, uh, is it okay to deploy this?”
“Wait, hold on, that’s not ready to go-live!”
“Weird, they shouldn’t be able to see that yet…“
“Oh shit, the deploy failed, what do I do?”
“Update for today: I’ve been merging in changes to my feature branch”

Feature flags (or feature toggles) are a technique for incrementally rolling out functionality in an application. Work that is not completely ready to go live can be released in chunks while being hidden by a flag. These flags are usually configured per-environment so that you can, for instance, enable a feature on a staging site before turning it on in production.

Feature Toggles by Martin Fowler is the canonical article on this practice and outlines a full taxonomy of feature toggles.

Toggles are broken down into four categories:

Release toggles: allow in-progress code to be shipped to production
Experiment toggles: support A/B testing multiple code-paths
Ops toggles: panic button / circuit breakers
Permission toggles: changing product features (premium, early access, etc)

Martin Fowler: Feature Toggle Chart

Each type of toggle has different needs and properties. For most small to medium sized apps, one type stands above the others in terms of value: release toggles.

Generally, long running branches are a self-imposed disaster for teams that want to ship fast. The stock answer to “how do we avoid long running branches” is to do continuous integration – and feature flags are often the missing conceptual link needed to bridge the textbook concept of CI with the practicality of “okay, how do we, you know, actually do that?”.

While the other types of toggles are more context specific, every app can benefit from simple release toggles.

How should we use feature flags?

Factors that determine if feature flags are a joy or a rats nest of complexity:

Quantity: how many flags do you have running at a time?
Duration: how long are these flags hanging around?
Urgency: how quickly would you need to turn a flag on/off?
Depth: at what application level does the flag need to exist?
Risk: how thoroughly do you need to flag a feature?

These factors are intertwined and have cascading effects.

If you only have features flagged for a few days, you may be willing to accept more risk in partially exposing a feature.

If your deploy process is currently slow, you’ll want to pick a solution that allows changing flags at runtime.

If you have one or two flags at a time, are working on new features that can be hidden at a high level, and someone manually guessing the URL of unreleased features isn’t a problem, you can literally get away with glorified if statements.

It’s always best to start small. As Sandi Metz would say: “The future is uncertain and you will never know less than you know right now”.

Keep your options open and do the simplest thing that could possibly work and build from there (if you ever need to).

How should we implement feature flags?

There are two approaches to implement feature flags: basic conditionals and “feature manager” libraries.

Basic Conditionals

Feature flags are, at their core, an if statement.

You can wrap up a very simple class to make the developer experience better and to keep your features organized. For getting started, don’t bother with any gems or external tools. Simply add a class like this to your Rails project.

class Feature
  def self.enabled?(feature_name)
    case feature_name.to_sym
    when :meeting_transcripts
      !Rails.env.production?
    else
      true
    end
  end
end

- if Feature.enabled?(:meeting_transcripts)
  # Do your thing

Take advantage of established patterns like Rails.env or Current.user to write simple conditionals to return true or false for any given feature. Since you completely own this code, you can adapt it however you want.

This seems too primitive for the real-world, but even this kind of stupid simple “disabled in production” flag is often the only kind of logic you’ll need.

Pushing the Limits

If you do want to further push the boundaries, you can try some of these clever hacks:

Current.user.admin? || !Rails.env.production - on for everyone in test, but only admins in production

Current.account.early_access_enabled? - add fields to domain models for “opt-in” groups

Current.user.email.contains("s") - roughly split a group in half using this hack from email marketing

Current.user.id % 100 < 10 - roll out a feature to roughly 10% of users (it’s not a proper random sampling but you’re not actually going to do real statistical analysis anyways)

These advanced conditionals are clever, but clever is not inherently good. Prefer more boring per-environment flags whenever possible.

Remember that we are primarily using these flags as release toggles. If you’re looking to do actual data-driven analysis (which requires more statistical rigor than reading a blog post…) or expect the logic to be a permanent part of your application, you should look elsewhere.

Feature management tooling

The other route teams take is to manage flags at run-time with a gem like flipper or use a hosted service like Flipper Cloud.

Sadly, no product manager has ever been able to understand this…

There are some other gems (rollout, flipflop, etc), but conceptually, they all end up using Redis as a key-value store, apply a bit of logic (percentage calculations, groupings, etc), and return true/false.

In exchange for the additional machinery, you gain the ability to turn features on and off while the app is running or even have non-developers manage the roll-out.

In reality, I’ve found that I rarely needed either of these features. Heavier feature flagging tools are both overkill for a simple cases (0-3 flags) and hard to manage if you have 5+ flags (which features are on? how do they interact? why is this off on staging, but on in production?).

With flipper, the deployment process becomes decoupled from your normal development workflow. Depending on your context, this may be a positive or a negative.

The stupid-simple Feature class requires a code commit to change, which is slightly annoying but then hands-off. Using flipper requires you to pull up the admin page in the right environment and click some buttons, or remote in and remember to flip some bits in a rails console.

Where should you put feature flags?

Whenever possible, flag things at the “edges” of features. If you’re adding a new section, hide it in the navigation menu. If you’re building a new action on an existing page, hide the button.

The fewer flags you need to put in your codebase at a given time, the easier it will be to manage.

If you can get away with it, simply hide things at the view level. For 99% of users, if something doesn’t exist in the UI, it doesn’t exist in the app. Hell, it’s often hard enough to get users to notice new features that aren’t hidden!

A “loosely” flagged feature will allow you to manually key in the URL if you want to run a quick test in an environment that isn’t officially enabled.

But if you are changing some internal logic, or if there are risks with allowing users to potentially access features that in still underdevelopment, you’ll need to move your flags closer to the “core” of the system.

Adding flags deep in a controller, model, or service is fine, but should be used only when necessary. Remember the same “keep it stupid simple” approach even as you move farther from the edges.

For instance, if you want to 404 all endpoints for a new feature, you can drop in this 20-line concern to avoid sprinkling conditions everywhere.

module FeatureFlaggableController
  extend ActiveSupport::Concern

  class_methods do
    attr_reader :feature_flaggable_name

    def feature_flag(feature_name)
      @feature_flaggable_name = feature_name
    end
  end

  included do
    prepend_before_action :enforce_feature_flag!

    private

    def enforce_feature_flag!
      if self.class.feature_flaggable_name.nil?
        raise ArgumentError.new("No feature flag specified! \n\nPlease call `feature_flag(:some_flag)` in #{self.class.name}.")
      end

      unless Feature.enabled?(self.class.feature_flaggable_name)
        raise AbstractController::ActionNotFound
      end
    end
  end
end

class PostsController < ApplicationController
  include FeatureFlaggableController
  feature_flag(:posts)

  # ...
end

How do you roll-out features with this basic approach?

Rolling out features can be stressful, but by using feature flags in an incremental way, you can make your releases calmer.

I generally try to follow these steps:

Add a feature flag when starting work on a big feature (default to “on” in non-production environments)
As functionality is ready, merge frequently and deploy behind the flag
With the flag still on, deploy finished feature to production and test / check things out
Update the feature logic to always return true, deploy and observe
If everything looks good, delete the feature logic completely

These are not hard and fast rules, but the key concepts are to avoid long-running branches and high pressure deploys where a ton of code is immediately going live in production for the first time.

It may seem like it would slow you down, but for most features it’s only adding a few minutes of extra time. It only takes one production rollout screw up to burn you before you’ll start using release flags for any medium or large features.

Wrap It Up

Shipping fast means deploying frequently and hiding in-progress work behind feature flags lets you safely rollout changes in a more controlled way. No more big “hope we don’t run into problems” deploys that create high stress situations. No more long running feature branches.

Feature flags bridge the gap between the abstract concept of continuous delivery and tactical release of features. While there are many kinds of feature toggles, simple release flags provide the best benefit-to-cost ratio.

You can achieve calmer deployments by making feature roll-outs a trivial event. Start small and use a boring Feature class before investing in fancy, enterprise-grade feature management tooling.

Spring Cleaning: Tidying up your codebase

2020-03-16T13:00:00+00:00

As the weather warms up, I get energized to give my codebase a spring cleaning. I’ve worked on projects where the mess was so bad that we were afraid to touch anything and I’ve worked on projects where I aggressively tried to get to code neutral (deleting more lines of code than I added). But the right balance lies somewhere in the middle.

Over the years, I’ve found a few tasks that I think provide the biggest bang for the buck. Easy, low risk things you can do in under an hour to make your codebase a little bit more inhabitable. Do one every day for a week, or go all-out one Friday afternoon and see how much you can finish.

Tidy Up Your Dependencies

Tools like bundler and yarn generate a bunch of files and folders for your project, but most of the mess is “out of sight, out of mind” as they shuffle things under the proverbial rug of node_modules.

And we’re not even talking about unused dependencies, just old versions of libraries sitting around since you’ve upgraded or tried out new tools in a branch.

Bundler: Run bundle clean to remove unused gems from your bundler directory.

➜ bundle clean --dry-run
Would have removed rails-html-sanitizer (1.2.0)
Would have removed jekyll-sitemap (1.3.1)
Would have removed que (0.14.3)
Would have removed aws-sdk-kms (1.28.0)
Would have removed actioncable (6.0.2.1)
Would have removed rake (12.3.2)
Would have removed administrate (0.12.0)
Would have removed factory_bot (5.1.0)
Would have removed sass (3.7.4)
Would have removed html-pipeline (2.12.3)
Would have removed minitest (5.11.3)
Would have removed uniform_notifier (1.12.1)
Would have removed parallel (1.19.1)
Would have removed colorator (1.1.0)

One caveat is that if you use system-level gems and have multiple Ruby projects, running bundle clean will try to remove global gems not used in your current project. This is probably not what you want.

To avoid this, switch to using per-project bundles. You can do bundle install --path vendor/bundle to install gems to a project-specific folder and then run bundle clean to remove unused gems for the project and not mess with gems for other projects.

Tip: You can disable downloading gem documentation locally by adding gem: --no-document to ~/.gemrc

Yarn: node_modules has a bad reputation for ballooning in size. As the nested folders get larger and larger, you can even start running into OS level limitations when trying to delete this massive pile of JavaScript.

Run yarn autoclean --init to generate a template file that slims down your node_modules folder by removing cruft like test files, markdown files, and other miscellaneous junk that sneaks into the published packages.

After adding a .yarnclean file, the scrubbing process will run every time you add or install packages (and you can also run it manually).

➜ yarn autoclean --init
yarn autoclean v1.17.0
[1/1] Creating ".yarnclean"...
info Created ".yarnclean". Please review the contents of this file then run "yarn autoclean --force" to perform a clean.

➜ yarn autoclean --force
yarn autoclean v1.17.0
[1/1] Cleaning modules...
info Removed 4799 files
info Saved 17.75 MB.
✨  Done in 4.37s.

Tip: Yarn automatically prunes extraneous packages whenever you run the install command so no need to do it yourself.

If you’re really feeling ambitious, audit your dependencies to see if any can be removed. Mike Perham’s excellent Kill Your Dependencies article has a checklist to use when evaluating external libraries:

Every dependency in your application has the potential to bloat your app, to destabilize your app, to inject odd behavior via monkeypatching or buggy native code. When you are considering adding a dependency to your Rails app, it’s a good idea to do a quick sanity check, in order of preference:

Do I really need this at all? Kill it. [Uninstall the gem]

Can I implement the required minimal functionality myself? Own it. [Copy/vendor the code] If you need a gem:

Does the gem have a native extension? Look for pure ruby alternatives. [Switch gems]

Does the gem transitively pull in a lot of other gems? Look for simpler alternatives. [Switch gems]

Gems with native extensions can destabilize your system; they can be the source of mysterious bugs and crashes. Avoid gems which pull in more dependencies than their value warrants. Example of a bad gem: the fog gem which pulls in 39 gems, more dependencies than rails itself and most of which are unnecessary.

Prune your git branches

If running git branch -r fills your terminal with the ghosts of features past, you should clean things up! Keeping your branches tidy makes it easy to see which branches are open without scrolling through tens (or hundreds!) of lines.

Run git remote prune origin to delete local tracking branches that don’t exist on origin. You can use --dry-run first if you’re worried.

If you run git branch -r again, you should see a slimmer list that only shows the branches that exist in GitHub.

You can then either go to GitHub directly or use git ls-remote --heads origin to list the current remote branches. If there are any to remove, delete them by running git push origin -D BRANCH_TO_DELETE.

Tip: Turn on “Automatically delete head branches” in GitHub so your branches don’t hang around after you’ve merged a pull request

Remove unused routes and views

While the Rails generators spark joy for me, they often generate more routes and views than you actually want. Apply your inner Marie Kondo and excise these unused parts of your app.

First try searching your app for “Find me in”: this is the default generator text for some views that Rails has generated. If you have any results, congratulations! You can delete these files without impunity.

Next up: check your routes.rb file for unused routes by looking for controller actions that aren’t implemented (or are missing views).

I found the best results from this gist. Drop the script into the root of your Rails app and run it to see a list of unused routes to potentially clean up.

Checking for unused database tables and columns

A final area to check is your database, which may have accumulated unused tables or columns. One clever approach is to look for empty tables and columns that have the same value for every row. These are suspicious places to investigate.

Paste this script (adapted from this article) into your project to scan your database (you may want to run against a staging database if your development database does not have representative data):

require_relative './config/environment.rb'

connection = ActiveRecord::Base.connection
connection.tables.collect do |t|
  count = connection.select_all("SELECT count(1) as count FROM #{t}", "Count").first['count']

  puts "TABLE UNUSED #{t}" if count.to_i == 0

  columns = connection.columns(t).collect(&:name).reject {|x| x == 'id' }
  columns.each do |column|
    values = connection.select_all("SELECT DISTINCT(#{column}) AS val FROM #{t} LIMIT 2", "Distinct Check")
    if values.count == 1
      if values.first['val'].nil?
        puts "COLUMN UNUSED #{t}:#{column}"
      else
        puts "COLUMN SINGLE VALUE #{t}:#{column} -- #{values.first['val']}"
      end
    end
  end
end

➜ ruby unused_db.rb
TABLE UNUSED active_storage_blobs
TABLE UNUSED friendly_id_slugs
TABLE UNUSED active_storage_attachments
COLUMN SINGLE VALUE investor_agreements:file_extension -- pdf
COLUMN UNUSED clients:comments
COLUMN SINGLE VALUE client_tasks:archived -- false
...

In additional to some unused generated tables, the script also found dozens of columns with all null values or all single values (e.g. an archived column that is always “false”). These columns require more investigation to remove, but are a good list to start.

Check for missing validations / constraints

While ActiveRecord provides a validation layer on top of your database, you’ll still want the strong protection of database constraints (non-nullable columns, unique indices, etc) to make sure no bad data sneaks into your app.

It’s really easy for your models and underlying database to get out-of-sync. Use the database_consistency gem to run a series of checks to tell you where your application models and database schema are out of sync.

➜ bundle exec database_consistency
fail ProjectType name column is required in the database but do not have presence validator
fail ProjectType slug column is required in the database but do not have presence validator
fail Project draft_documents associated model should have proper index in the database
fail Project attachments associated model should have proper index in the database
fail Project alerts associated model should have proper index in the database
fail Company name column should be required in the database
fail PurchaseApproval date column should be required in the database
...

You may be overwhelmed by the volume of errors this tool spits out for a project, but don’t fret, every change should be straightforward and you can handle them bit by bit. Once you resolve everything, consider adding this command to your Rails CI pipeline in order to catch future errors right when the offending code is committed.

Clear out your migrations folder

The final place to check is your migrations. Over time, your db/migrate folder will build up to hundreds of migration files. If you’ve already run these migrations on production environments (and if you’re avoiding creating seed data directly in migrations), you can safely delete older migrations.

I like the approach outlined by Clutter:

Delete migrations older than 3 months
Add a new migration that raises if your database was very out-of-date (with instructions on how to load the schema)

If deleting the migrations makes you uneasy, you can also look at squasher: a tool that combines your old migrations into one mega-migration.

Deep cleaning: going even further

If you’re ready to deep clean, consider trying out these tools to help you dig further into your application code:

unused: a general-purpose tool for finding dead code and is based on ctags
debride: a Ruby tool to find potentially uncalled methods (with some specific Rails checks)
coverband: a “run in production” coverage tool that collects stats on method usage; it’s the most thorough but has a longer turn around time as you need to run it for a while against real traffic
rcov: depending on your test suite, you can get a good picture of what parts of your application may not be used
attractor: metrics visualizer for plotting churn (how often code changes) vs complexity

I haven’t had as much success with these tools and generally don’t find them to be a good ROI, but if you’re looking to keep your codebase sparkling clean, it’s worth spending some time exploring. Personally, I’m okay with a little bit of dust :)

Happy cleaning! If you’ve got any other tips that you’ve used on your projects, let me know on Twitter.

Wrangling slow reports, large file exports, and long-running tasks in Rails with Active Job

2020-01-26T13:00:00+00:00

In case you don’t like reading (Feb 2020 - Indy.rb Meetup)

In every non-trivial application you’ll have to do something that is just slow. You need to export a large data set to a CSV file. An analyst needs a complicated Excel report. Or maybe you have to connect to an external API and process a whole bunch of data.

So how do you know when it’s time to use background jobs?

If you try building these kinds of reports in a normal Rails controller action, you will see these symptoms:

Request timeouts: once response times start creeping past 10+ seconds, you run the risk of timeouts. You might have a report that is slow but just barely finishes, but you know that it scale linearly with the amount of data and will break soon. Platforms like Heroku will kill requests after 30 seconds and, in the mean time, your whole app slows down as workers/dynos are tied up with long running actions.

Memory spikes: the most straightforward implementations for file exporting generate a file in memory before serving it to the user. For small files, this is fine; but if you create 100+ MB files in memory, your RAM usage will spike up and down wildly. If even a few users are doing exports at the same time, you will hit your resource cap and see failed requests and Heroku “R14 - Memory Quota Exceeded” errors.

Bad UX: even if your server weathers the extra load of slow actions, users may not be so patient. A never-ending loading indicator in the browser tab, or worse, nothing happening for 20 seconds is not a good experience. You can throw up a “Loading…” spinner, but can we do better?

The first step is to buy some breathing room by optimizing the queries or fixing any obvious performance issues. Spend an hour looking through the Speedshop Rails performance blog to see if there is low-hanging fruit.

But once you start seeing these kind of problems, it’s time to set up infrastructure for background jobs.

Pick Your Tools

Adding the first background job is always the hardest. You’ve got frustrated users or an irate customer (that’s called an acute need in business speak!), but you don’t know your long-term needs past this first use case.

There are two initial decisions to make when setting up background jobs in Rails.

Active Job or Not?

Way back in Rails 4.2, Active Job was released as a common abstraction over background job processing. Following in the footsteps of other Rails ActiveX gems, Active Job is an API that allows you add background jobs into queue and then use tools like Sidekiq, Delayed Job, or Resque to “work” or process those jobs.

Active Job defines a core set of functionality that should serve most applications really well: you can (obviously) create jobs, but also put jobs into separate queues (priorities), schedule jobs to run in the future, and handle exceptions. These are the basic primitives that you want for background jobs.

As with all Rails abstractions, there are trade-offs: a standard interface allows for deeper framework integration but has to work with the lowest common denominator.

Using Active Job affords you a simpler development environment. In development, you can set the queue adapter to :async to run jobs with an in-process thread pool. Run bin/rails server locally and jobs will get worked without running a separate worker command or using a tool like foreman.

The downsides are that if you need more powerful features or want to utilize an adapter specific pattern, you are fighting against the current.

You can certainly set up your app to use Sidekiq directly and forgo Active Job completely. And if you need to use advanced features (special retry strategies, batching, pushing huge numbers of jobs, rate limiting, etc), there are benefits to being “closer to the metal” with the background processor (and some features may not work with Active Job at all).

Skipping Active Job can also give you a non-trival performance boost, per the Sidekiq wiki.

I recommend starting off with Active Job; you can always port later down the line when you have a clear vision of your long-term needs. If you hit limits or cannot make it work, you can justify spending more time tuning and going off the standard Rails path.

Delayed Job or Sidekiq?

The second consideration is which queueing adapter to use in production. There are only two options unless you have some extremely exotic needs: Sidekiq or Delayed Job. These are both established libraries and have the best tutorials, articles, and extensions.

Sidekiq is the current “default” option for many developers. It scales up well and the Sidekiq Pro/Enterprise option is very reassuring for the continued stability and improvement of the project. Mike Perham (and the rest of the contributors) do a really nice job on the Sidekiq documentation and wikis.

If you plan on having thousands and thousands of jobs running per day, Sidekiq is a no-brainer.

The one minor drawback to Sidekiq is that you take on a hard dependency on Redis to store the queue of jobs and metadata. If you already have Redis in your infrastructure (most likely for caching), this is a moot point. The majority of Rails developers are comfortable with having Redis in their tech stack and you can spin up hosted instances in about 5 minutes on all the major cloud platforms.

Delayed Job is the older, less shiny option. While it has fallen out of community favor in recent years, it is still a very solid gem. Delayed Job was originally extracted from Shopify’s internal codebase so it has been battle tested at high scale.

Delayed Job works with ActiveRecord and uses your PostgresQL database for storing the jobs. No additional infrastructure or Heroku add-ons needed. It’s really easy to set up and there is one less moving part.

Sharing your application’s database is also handy if you want to peek into the underlying table to debug or see how the tool works; it’s just another table.

Because Delayed Job is single-threaded (compared to Sidekiq’s multi-threaded model) it is not advised for processing 100,000s of jobs per day. That said, many applications will never reach that scale! Delayed Job can take you pretty far with no extra infrastructure.

By the time you feel the scaling pain, you may be ready for a much bigger engineering investment – or have a whole team of people to tune and manage background processing.

If you have an (actually) large web service (Shopify/GitHub/Basecamp-esque loads), there is a whole range of “state of the art” patterns and tooling to consider (see Kir Shatrov’s great writeup from the trenches at Shopify), but the rest of us will be best served with a boring choice.

Ultimately, the decision comes down to if you already have Redis as part of your infrastructure. Both libraries work well and have similar set up time.

Sidekiq will allow you to scale up more, but with a bit of infrastructure tax; Delayed Job will fall over if you are trying to crank though 5000 jobs/sec, but that should not be a primary concern for small/medium-sized applications. You can’t really go wrong with either option.

In a recent project with a tiny team and a small amount of jobs (10s-100s per day), we went with Delayed Job for the simplicity and less moving pieces. It’s fine.

The Basics of Background Jobs

At the highest level, using background jobs means taking your slow code and moving it in a Job class. Instead of running the slow code in your controller, you enqueue the Job and pass it any input data it needs.

The Active Job guide does a good job explaining the general concept but there are a few best-practices that will give you an outsized benefit if you follow them.

The top three things you can do to avoid headaches with jobs are: use Global ID, check pre-conditions, and split up large jobs.

Global ID

You may see advice for other tools that are the exact opposite of this tip, but in Rails, you can pass ActiveRecord models as arguments to your jobs and they will Just Work. Under the hood, Rails uses a concept called “Global ID” to serialize a reference to a model object. If you have a Company model with id=5, Rails will generate an app wide unique id for the model (in this case: gid://YourApp/Company/5).

Don’t: Manually query models by ID

class CloseListingJob < ApplicationJob
  def perform(user_id, listing_id)
    listing = Listing.find(listing_id)
    listing.archive!
    listing.notify_followers

    user = User.find(user_id)
    listing.notify(user)
  end
end

In other contexts, passing objects to background jobs is considered an anti-pattern as you may serialize an object that is “stale” when you go to run the job later. Rails will get a fresh copy of the object before your job runs so you don’t have to worry about it.

If you’ve dealt with weird behavior from serialized/marshaled objects (shudders), you probably have battle scars, but you don’t need to pass around IDs and manually re-fetch data anymore.

Do: Use Global ID to automatically handle common serializing

class CloseListingJob < ApplicationJob
  def perform(user, listing)
    listing.archive!
    listing.notify_followers
    listing.notify(user)
  end
end

Check Pre-Conditions

Two reasons to check pre-conditions for jobs (especially destructive actions): the world may have changed and the double-queuing problem.

Since you’ve offloaded a task to the background, confirm that nothing major changed from the time we enqueued the job and when the code actually runs.

Imagine a case where you get ready to send a late invoice email to a customer, it takes a few minutes for the job to get processed, and during that time the customer just so happens to submit their overdue payment. The job is already in the queue and without guarding against the invoice being paid, you’d send a confusing email out.

The other issue is when multiple copies of the same job are enqueued. Maybe two users triggered the same job, maybe there is a bug in your code, or maybe a job got run twice when your dyno restarted. There is a fancy computer-science word for this: idempotence. Ensure the same job could run multiple times without negative side-effects.

Don’t: Fire away destructive actions blindly

class NotifyLateInvoiceJob < ApplicationJob
  def perform(user, invoice)
    # Fire off a scary email
    InvoiceMailer.with(user: user, invoice: invoice).late_reminder_email
  end
end

There are plugins and tools that can try to enforce idempotence (see gems like sidekiq-unique-jobs), but in general, take precautions in your own application code.

Do: Ensure the conditions that cause you to enqueue a job are true when it runs

class NotifyLateInvoiceJob < ApplicationJob
  def perform(user, invoice)
    # Confirm that nothing has changed since the job was scheduled
    if invoice.late? && !invoice.notified?
      InvoiceMailer.with(user: user, invoice: invoice).late_reminder_email

      # Track state for things that should only happen once
      invoice.update(notified: true)
    end
  end
end

Split Up Large Jobs

In addition to moving work off of the “main thread”, background queues allow you to split up and parallelize work. But just putting something in a background job does not make it run faster.

Prefer many small jobs to one big job – this helps with performance (letting multiple workers crank through the task) and also helps in the event that the job is interrupted (dyno restart, unexpected errors, random bit flips due to solar flares).

Don’t: Run excessively slow loops in a single job

class ComputeCompanyMetricsJob < ApplicationJob
  def perform(company)
    company.employees.each do |employee|
      # A bunch of slow work
      employee.compute_hours
      employee.compute_productivity
      employee.compute_usage
    end
  end
end

You may hear this concept called “fanning out”: you take one job that might have 100 loop iterations and instead spin off 100 smaller jobs.

If you have 10 workers available, you can finish the job ~10x faster (ignoring overhead) and if the 62nd loop iteration fails, you can retry that one small piece of work (instead of starting the whole thing over).

Do: Enqueue smaller jobs from another job

class ComputeCompanyMetricsJob < ApplicationJob
  def perform(company)
    company.employees.each do |employee|
      # Fan-out with parallel jobs for each employee
      ComputeEmployeeMetricsJob.perform_later(employee)
    end
  end
end

class ComputeEmployeeMetricsJob < ApplicationJob
  def perform(employee)
    employee.compute_hours
    employee.compute_productivity
    employee.compute_usage
  end
end

Don’t go overboard: it’s not advisable to break every job with a loop into sub-jobs. Sometimes it’s not worth the extra complexity or you want the work all wrapped in the same transaction.

But if you find yourself creating a job where you are looping and doing more than a query or two, think about splitting up the job.

Getting Data Back to the User

Most guides on background jobs make the faulty assumption that everything is “fire-and-forget”. Sure, for sending password reset emails, it’s fine to enqueue the job and finish the request. It’s unlikely to fail, takes only a few seconds, and you don’t need the result of the job to inform the user to kindly check their inbox.

But what about generating a 20,000 row Excel file or a background process that takes 5 minutes to complete? We often need a way to report back progress and send back information once the job is completed.

The fundamental issue is that when you move work to a background job, code execution does not stop and you need a way to “connect” back to the user that caused the job to be created. It’s outside of a single request-response life cycle so you cannot rely on normal Rails controllers.

There are two approaches that are well suited for the vast majority of jobs: email later and polling + progress.

Email Later

One way to relay that a job has been completed is to send an email. The benefit is that it’s simple. You run a job and then off an ActionMailer email with any relevant status (“Your contacts have now been imported!” or “Failed to sync calendar events. Contact support”) or links back into your app (“View your annual tax documents now”).

You can see an example of this pattern in Mailchimp.

For reports that may take a long time (e.g. exporting the campaign statistics for every email you’ve ever sent), you start by clicking “Generate Report”

Mailchimp informs you that for large accounts, it may take a few minutes and to expect an email when the data is done exporting.

After some time, you get an email from Mailchimp with a link to download the report.

You are taken back to the Mailchimp app and can download a zip of your report.

Mailchimp opts to store the most recent file export in your account, in case you want to download the same report again later. If you generate another large report, it will overwrite the file.

For your own application, you can decide if that approach to saving reports makes sense. You may opt to store a complete history of all files, keep the most recent three, or something else entirely.

Here is some sample code to implement this workflow (specific details will need to adapted to your own app).

class ReportsController < ApplicationController
  def generate_campaign_export
    # Enqueue the job
    CampaignExportJob.perform_later(Current.user)

    # Display a message telling user to await an email
    render :check_inbox
  end
end

class CampaignExportJob < ApplicationJob
  def perform(user)
    # Build the big, slow report into a zip file
    zip = ZipFile.new
    user.campaigns.each do |campaign|
      zip.add_file(CsvReport.generate(campaign))
    end

    # Store the report on a file system for downloading
    file_key = AwsService.upload_s3(zip, user)

    # Record the location of the file
    user.update(most_recent_report: file_key)

    # Notify the user that the download is ready
    ReportMailer.campaign_export(user).deliver_later
  end
end

<!-- application/views/report_mailer/campaign_export.html.erb -->

Hi <%= @user.name %>,

The export of your campaign report is complete. You can download it now.

<%= link_to "Download Report", download_campaign_export_path %>

class ReportsController < ApplicationController
  ...

  def download_campaign_export
    # Create a 'safe link' to download (e.g. signed, expiring, etc)
    download_url = AwsService.download_link(User.current.most_recent_report)

    # Trigger a download in the browser
    redirect_to(download_url)
  end
end

Polling + Progress

For basic reports, emailing will suffice, but sometimes users need more frequent status updates on what’s happening in the background process.

An alternative pattern is to combine polling (periodically checking on the job via AJAX calls) with progress reporting. One benefit of this approach is that it is super extendable. You can implement the progress reporting in a variety of ways: a numeric progress bar, a single “status” message, or even a detailed log of progress.

This pattern is well suited for big user-generated reports, as well as long-running “system tasks” that the user may want to monitor.

You can see an example of this pattern in the Netlify build dashboard.

Heroku uses the same pattern for displaying progress for the minutes-long process of deploying a new version of your software.

These UIs may look complicated, but at their core, it is a repeating AJAX call to check the job status and a block of text. You can reuse this pattern for multiple different reports in your application and make the output as detailed as you need.

You can think of this pattern as building a CLI interface inside of a web page. And since the main UI element is text, it’s really easy to build and change as needed.

Here is some rough sample code to implement this workflow. This sample uses a separate database model for storing “job status” that is passed to the job and a Stimulus controller to handle client-side polling/updating.

Specific details will need to be adapted to your own app.

class MetricsController < ApplicationController
  def export_company_stats
    company = Company.find(params[:company_id])

    status = JobStatus.create!(name: "Company Stats - #{company.name}")

    CompanyStatsJob.perform_later(company, status)

    redirect_to status, notice: "Generating statistics..."
  end
end

# == Schema Information
#
# Table name: job_statuses
#
#  id            :bigint           not null, primary key
#  completed     :boolean          default(FALSE), not null
#  download_key  :string
#  error         :boolean          default(FALSE), not null
#  error_message :text
#  log           :text             default("")
#  name          :string           not null
#  created_at    :datetime         not null
#  updated_at    :datetime         not null
#
class JobStatus < ApplicationRecord
  def add_progress!(message)
    update!(
      log: log << "\n==> " << message
    )
  end

  def mark_completed!(key)
    update!(
      completed: true,
      download_key: key,
      log: log << "\n\n---\n\n" << "Report complete!"
    )
  end

  def mark_failed!(error_message)
    update!(error: true, error_message: error_message)
  end

  def log_error!(error)
    update!(
      log: log << "\n\n---\n\nERROR: " << error.to_s
    )
  end
end

class CompanyStatsJob < ApplicationJob
  def perform(company, status)
    status.add_progress!("Generating stats for #{company.name}...")
    report = CompanyStatsReport.new(company)

    # Do some complicated calculations
    status.add_progress!("Computing financial data...")
    report.compute_financial_data!

    status.add_progress!("Computing employee data...")
    report.compute_employee_data!

    status.add_progress!("Computing product data...")
    report.compute_product_data!

    # Store the report on a file system for downloading
    file_key = AwsService.upload_s3(report.as_xlsx)

    status.add_progress!("Report completed.")
    status.mark_completed!(file_key)
  rescue StandardError => e
    status.mark_failed!(e.to_s)
    status.log_error!(e)
  end
end

class JobStatusController < ApplicationController
  def show
    @status = JobStatus.find(params[:id])

    # Return a basic HTML page and a JSON version for polling
    respond_to do |format|
      format.html
      format.json { render json: @status }
    end
  end
end

<!-- application/views/job_status/show.html.erb -->

<h1>
  <%= @status.name %>
</h1>

<div
  data-controller="job-status"
  data-job-status-url="<%= job_status_path(@status, format: :json) %>"
>
  <div data-target="job-status.log">
    <%= @status.log %>
  </div>

  <%= link_to "Download", "#", class: "btn hidden", data: { target: "job-status.download" } %>
</div>

// src/controllers/job_status_controller.js
import { Controller } from "stimulus";

export default class extends Controller {
  static targets = ["log", "download"];

  connect() {
    // Start polling at 1 second interval
    this.timer = setInterval(() => {
      this.refresh();
    }, 1000);
  }

  refresh() {
    fetch(this.data.get("url"))
      .then((blob) => blob.json())
      .then((status) => {
        // Update log and auto-scroll to the bottom
        this.logTarget.innerText = status.log;
        this.logTarget.scrollTop = this.logTarget.scrollHeight;

        if (status.error) {
          this.stopRefresh();
          // Add additional error handling as needed
        } else if (status.completed) {
          this.stopRefresh();

          // Show a download button, or take some other action
          this.downloadTarget.href = `/download/${status.download_key}`;
          this.downloadTarget.classList.remove("hidden");
        }
      });
  }

  stopRefresh() {
    if (this.timer) {
      clearInterval(this.timer);
    }
  }

  disconnect() {
    // Stop the timer when we teardown the component
    this.stopRefresh();
  }
}

Aside: there are tools like sidekiq-status that you can use to track progress instead if you want more features or a different API. One thing you should not do is to try adding extra status information directly to the underlying jobs table as most processors will clear out the records once the job has been processed (leaving you unable to query the status).

Mix, Match, and Modify

These two approaches should serve as solid ways to tackle the problem, but feel free to combine or modify them to suit your own needs.

Maybe you want to add a flag that the file was downloaded and send an email later if the user closed the tab while waiting for it to finish. Maybe you want to redirect the user to a new page in the app instead of downloading a file. Maybe you just need a loading spinner or basic progress bar.

Tip: Background processing is a language-agnostic concept, so don’t be shy about learning from other development ecosystems

Think of these patterns as good building blocks and then layer on additional complexity as needed. You might want to clean up the job_statuses table after a few days or add more robust authentication or use the job status as a way to cache results. It’s up to you.

This code should serve as a conceptual guideline and should be easy to extend and grow as your needs change.

Advanced Techniques: Proceed with Caution

It is outside the scope of this article to cover more advanced techniques, but if you find yourself needing something even more powerful the next step would be to implement some kind of ActionCable or web-socket approach, maybe combined with in-app or push notifications. Most applications will never need to go this far.

If you have reports that are easy to pre-compute and store – for instance, a monthly account statement – you could switch from “on-demand” report generation to cron/scheduled generation. If the report contents are unlikely to change after a certain date, it is a good candidate for pre-computing.

Wrap It Up

You will reach a point where you cannot solve every performance problem by optimizing queries. Long-running reports or large file exports should be moved into a background job for processing outside the life cycle of a single HTTP request.

The Active Job framework in Rails provides a solid, standardized interface for enqueueing work. There are battle-tested, community-supported tools – Sidekiq and Delayed Job – that can crank through thousands of jobs a day without any hiccups.

Follow these basic (but robust!) patterns for displaying progress and sending data back to the user once it’s been processed. There is a modest up-front investment to get your first background job up and running, but it will payoff immediately as your app grows.

Managing Rails schema and data migrations without losing your mind

2019-11-05T13:00:00+00:00

As time goes by, you’ll need to change the data model for your software. Rails makes this extremely easy to do, but leaves much of the day-to-day process to you to sort out (the principle of Sharp Knifes).

Once you’re past the initial scaffolding stage of a project, you’ll have questions about how to manage your application schema, especially when working on a team.

Adding rules to your process codifies good practices; a boring checklist isn’t sexy, but it will get the job done and free you up for more valuable work.

Unless you have strong, context-specific reasons not to, follow these rules:

Use migrations only for schema changes
Use one-off scripts to seed/import data
Aggressively prune scripts and sync environments

Migrations for only schema changes

The ActiveRecord Migration DSL is one of the nicest toolkits in all of web programming: it’s easy to write migrations, test them locally, and the design of how versions are handled is very robust.

Since migration files are Ruby classes, you can put any kind of arbitrary code in them. You could use the migrations DSL, execute raw SQL commands, or create records using your application code.

But you will find plenty of advice that you should NOT reference ActiveRecord models during migrations.

Over the life of your Ruby on Rails application, your app’s models will change dramatically, but according to the Rails guides, your migrations shouldn’t:

“In general, editing existing migrations is not a good idea. You will be creating extra work for yourself and your co-workers and cause major headaches if the existing version of the migration has already been run on production machines. Instead, you should write a new migration that performs the changes you require.”

That means that if your migrations reference the ActiveRecord model objects you’ve defined in app/models, your old migrations are likely to break. That’s not good.

This advice comes from the README for good-migrations – a gem that prevents loading your models in migrations so it becomes impossible to get yourself into a mess. If you can’t load your models, you can’t accidentally reference them in migrations.

As your codebase evolves, Model.create or Model.update calls sprinkled throughout old migrations makes it hard to run the migrations against a completely new database without errors.

You can work around this by redefining models inside the scope of your migration or be being very diligent about refactoring old migrations if you make breaking changes, but the bottom-line is: it’s not worth it.

Getting your migrations into a broken state is a huge mess – everyone on your team might end up with a slightly different local database or your environments get out of sync based on what order migrations have been run. If you don’t do it correctly, your CI server won’t be able to rebuild a test database from scratch.

Fix the problem by avoiding it all together: only do schema migrations. If you see code other than the migrations DSL in a migration, don’t merge it!

One-off scripts for seeding/importing data

In a data-heavy application, it’s common to import or modify a bunch of data right after a schema change. For this situation, write one-off scripts and execute them with rails runner. I recommend putting these scripts in a db/script folder.

Write a script to create the data you need (seed, import from file, bulk update, etc)
Test it against your local database
Deploy to your other environments and run the script
Remove the script from source control once it’s been run everywhere (more on this in the next section)

Here’s an example:

class AddArticleCategories < ActiveRecord::Migration[6.0]
  def change
    create_table :article_categories do |t|
      t.string :name, unique: true, null: false
    end
  end
end

# Seed initial categories
rails = ArticleCategory.create!(name: "rails")
js = ArticleCategory.create!(name: "javascript")
git = ArticleCategory.create!(name: "git")

# Back-fill categories
Article.each do |a|
  if a.title.downcase.includes?("rails")
    a.update(category: rails)
  end
end

puts "Done!"

# Run locally
bin/rails runner db/script/seed_categories.rb

# Push script to staging and run migration
heroku run --app my-app-test bin/rails runner db/script/seed_categories.rb

# Push script to production and run migration
heroku run --app my-app bin/rails runner db/script/seed_categories.rb

Now that the script has been run on production, you can delete it from the project.

Aggressively prune and sync environments

With this approach, we treat schema migrations as durable and data scripts as disposable. Once the script has been run on production, it doesn’t need to exist anymore.

If you need to reference it again, use your git history, not a huge folder full of potentially out-dated and broken scripts. And in some cases, it is harmful to keep around old scripts that someone might accidentally re-runs a few weeks later.

But what if someone on your team was out sick and didn’t get the script run locally and now it’s already been removed? Start treating your database environments like a one-way street.

Production is the gold standard: whatever is on production is the absolute source of truth
You can pull down production to a staging/test environment
You can pull down staging to local development databases

Data can safely flow from Production -> Staging -> Development but never the other way.

If someone needs to get up-to-date, have them clone the staging database to their local database. Periodically clone the production database to the staging environment as part of your release process.

If you’re on Heroku, use parity to do this. It’s a super convenient way to add one-line “copy this environment to that environment” functionality to your project.

Optional: clean up old migrations

Depending on how complex your application is, you may find yourself drowning in migration files. There is no harm in having them in your db/migrations folder, but if you want, you can delete them.

Just make sure you switch from a db:create/db:migrate approach to a db:schema:load approach on CI/test.

As with data scripts, if you need to refer back to some old migrations you have source control for that.

When to deviate

You’ll need to reach for a different workflow if:

You need zero downtime migrations
You have sensitive data that prevents syncing environments
You have have a complicated multiple database setup

…and probably several other circumstances. As they say: “but of course there are obvious exceptions…”

If this is the case, you should deviate from these rules!

But think really hard and challenge if you actually have those requirements before adding extra complexity to your process. You can always add capabilities later as needed, but it’s much harder to take them away once you become dependant on them.

Summary

Especially when it comes to touching production data, being boring is a virtue. We don’t want fancy setups because if (let’s be honest: when…) they break, we are entering a high stress situation.

Imagine you are trying to do a deploy and the migrations fail – or the schema changes work, but there is an exception seeding some data. Now you have a problem. Do you rollback? SSH in and re-run the script? Start flailing around in the Rails console? Break out into a cold sweat?

If you’re ever spent an hour trying to help diagnosis why a teammate’s local database isn’t matching your own or why the schema.rb file seems to change every time you run migrations, you would benefit from this strategy.

If you follow these rules:

Use migrations only for schema changes
Use one-off scripts to seed/import data
Aggressively prune scripts and sync environments

You’ll be well positioned for reliable database migrations with clear guardrails around how you (and your team) should approach changing data.

Boring Rails: Skip the bullshit and ship fast |

Beautiful Rails confirmation dialogs (with zero JavaScript)

Interactive demo

Adding Animations with @starting-style

Browser Support

Integrating with Turbo’s Confirm System

Addendum: Preventing Background Scroll

Hotwire components that refresh themselves

Hotwire: Components should refresh themselves

A tangled mess of identifiers

View Components to the rescue

Do we need to keep the stream open for eternity?

Conclusion

Event sourcing for smooth brains: building a basic event-driven system in Rails

Why did I go down this path?

It’s about the events

The event model

Voila an activity feed!

Reaping what you’ve sown

Acknowledgements and further reading

Writing better Action Mailers: Revisiting a core Rails concept

Better display names

Changing the view folders

Multiple emails per mailer

You don’t need to write the word “email”

Parameterized mailers

Dynamic defaults

Connecting the dots

Wrap it up

Sorting ActiveRecord results by enum values (in SQL)

Usage

Extra notes

Additional Resources

Thinking in Hotwire: Progressive Enhancement

Scaffolds and CRUD without Hotwire

Adding Turbo Drive

Adding Turbo Frames

Adding Turbo Streams

Adding Turbo Streams + Action Cable

Adding Stimulus

Adding custom components / React / client side JavaScript

Adding Turbo Native

Adding Strada

Wrap it up

Galaxy brain CSS tricks with Hotwire and Rails

Empty States and Turbo Streams

Tailwind Variants with Data Attributes

Dynamic styles with erb

Stop string interpolating class names!

Wrap it up

Adding keyboard shortcuts and hotkeys to StimulusJS

stimulus-hotkeys

stimulus-use/useHotkeys

HotKeys.JS (directly)

github/hotkey

Wrap it up

The most underrated Rails helper: dom_id

Super clean tag builders

Deep linking anchor tags

With Turbo Frames

Scoping Turbo Stream responses

Wrap it up

Self-destructing StimulusJS controllers

Self-destructing?

What else could you use this for?

Highlighter

Grab focus

Analytics “Beacons”

Wrap it up

Tailwind style CSS transitions with StimulusJS

Options

My recommendation: el-transition

Building the Tailwind UI Slide Over Menu

Wrap it up

Dynamic user content in Rails with Liquid tags

Usage

Even more advanced battle-tested features

References

Accessing Rails environment variables from a StimulusJS Controller

Usage