Design patterns in Ruby — Pipeline pattern

The problem to solve

You want to do multiple operations on an object, passing the updated object to the next operation each time.

The use case I had when recently using this pattern is that I needed to edit a yaml file using multiple steps that would likely be changed/grow in the future. I could have done something like this:

def process(yaml) 
  updated_yaml = remove_private_config(yaml)
  updated_yaml = add_runtime_config(updated_yaml)
  updated_yaml = remove_irrelevant_config(updated_yaml)
end
def remove_private_config(yaml)
  # do stuff
end
def add_runtime_config(yaml)
   # do stuff
end
def remove_irrelevant_config(yaml)
   # do stuff
end

This code works fine but even now has a couple of problems. The main problem is it’s violating the single responsibility principle — if we want to test each step or edit (remove, add, change) steps, it’s going to be harder and harder because the process method is doing way too much. It’s not too difficult to read atm but the more steps that get added it will become a very bloated class.

The Solution

Let’s use the pipeline pattern to do this instead:

class EditYaml
  def initialize(yaml)
    @yaml = yaml
  end

  def call
    pipeline.process(@yaml)
  end

  private

  def pipeline
    Pipeline.new(pipeline_steps)
  end

  def pipeline_steps
    [
      Steps::RemovePrivateConfig,
      Steps::AddRuntimeConfig,
      Steps::RemoveIrrelevantConfig
    ]
  end
end

And our pipeline class;

class Pipeline
  def initialize(elements = {})
    @elements = elements
  end

  def process(input)
    elements.reduce(input) do |memo, element|
       # 'memo' is the object that's passed between each step
       element.process(memo)
    end
  end

  private

  attr_reader :elements
end

And one of our steps:

module Steps
  class RemovePrivateConfig
    def self.process(yaml)
      #  do stuff
    end
  end
end

When you call pipeline.process(@yaml) it will go through each step, executing the step on the object before passing that updated object to the next step.

Now we can test each of the steps in isolation, and we can add/remove steps really easily… and it’s got a very easy to read interface!