Decorating Ruby's Net::HTTP for Fun and Profit

in HTTP , Ruby

Ruby’s Net::HTTP library gets a bad wrap, somewhat appropriately, for being opaque and cumbersome to implement in any meaninful way. It works but it is gruesome to write every time you need to get data from a remote host. Obviously there are a zillion HTTP wrappers already implemented, but I wanted to keep my dependencies low so I built my own little decorator around Net::HTTP called HTTPDecorator.

It’s limited. I know that and you should too, but it does what I need and could be extended with more features as needed. Right now it supports:

  1. These HTTP verbs: GET POST PUT DELETE PATCH.
  2. Multipart uploads from the params['file'] .
  3. Send/receive JSON by default. Sidebar: I use Oj for marshalling, but figured I’d keep it simple here.
  4. Allow sending application/x-www-form-urlencoded as needed.
  5. Separate read/open timeouts configured as constants.
  6. Reusable, open connections using the start and finish methods.

Here are some examples of it in use:

 1##
 2# Create an instance with the domain we'd like to call.
 3@api = HTTPDecorator.new('endpoint.com')
 4
 5##
 6# Basic usage
 7get  = @api.get('/info')    # GET  https://endpoint.com/info
 8post = @api.post('/create') # POST https://endpoint.com/create
 9
10##
11# Using the parsed response (assuming the response is JSON).
12user = @api.get('/users/2') # { "name": "Bob" }
13user['name'] # => "Bob"
14
15##
16# Reuse one connection across requests.
17@api.start do |api|
18  api.get('/info')
19  api.put('/updated')
20end
21
22# -- OR --
23@api.start
24@api.get('/info')
25@api.put('/updated')
26@api.finish

And here’s the implementation (which I also posted as a gist):

  1# encoding: UTF-8
  2# frozen_string_literal: true
  3require 'net/http'
  4require 'json'
  5require 'uri'
  6
  7class HTTPDecorator
  8  # Timeouts
  9  OPEN_TIMEOUT = 10  # in seconds
 10  READ_TIMEOUT = 120 # in seconds
 11
 12  # Content-types
 13  CONTENT_TYPE_JSON = 'application/json'
 14  CONTENT_TYPE_FORM = 'application/x-www-form-urlencoded'
 15  CONTENT_TYPE_MULTIPART = "multipart/form-data; boundary=#{ Rack::Multipart::MULTIPART_BOUNDARY }"
 16
 17  def initialize(domain)
 18    # Build up our HTTP object
 19    @http = Net::HTTP.new(domain, 443)
 20    @http.use_ssl = true
 21    @http.verify_mode = OpenSSL::SSL::VERIFY_PEER
 22    @http.open_timeout = OPEN_TIMEOUT
 23    @http.read_timeout = READ_TIMEOUT
 24
 25    # In local development we can log requests and responses to $stdout.
 26    #   DO NOT EVER do this in production. EVER.
 27    if ENV['RACK_ENV'] == 'development'
 28      @http.set_debug_output($stdout)
 29    end
 30  end
 31
 32  # Open a connection for multiple calls.
 33  # - Accepts a block, otherwise just opens the connection.
 34  # - You'll need to close the connection if you just open it.
 35  def start
 36    if block_given?
 37      # Open the connection.
 38      @http.start unless @http.started?
 39
 40      # Yield to the calling block.
 41      yield(self)
 42
 43      # Clean up the connection.
 44      @http.finish if @http.started?
 45    else
 46      # Open the connection.
 47      @http.start unless @http.started?
 48    end
 49  end
 50
 51  # Clean up the connection if needed.
 52  def finish
 53    @http.finish if @http.started?
 54  end
 55
 56  # GET
 57  def get(path, params = {})
 58    uri       = URI.parse(path)
 59    uri.query = URI.encode_www_form(params) unless params.empty?
 60    request   = Net::HTTP::Get.new(uri.to_s)
 61
 62    parse fetch(request)
 63  end
 64
 65  # POST
 66  def post(path, params = {}, as: :json)
 67    request = Net::HTTP::Post.new(path)
 68
 69    case as
 70    when :json
 71      request.content_type = CONTENT_TYPE_JSON
 72      request.body = JSON.generate(params) unless params.empty?
 73    else
 74      request.content_type = CONTENT_TYPE_FORM
 75      request.body = URI.encode_www_form(params) unless params.empty?
 76    end
 77
 78    parse fetch(request)
 79  end
 80
 81  # DELETE
 82  def delete(path)
 83    request = Net::HTTP::Delete.new(path)
 84
 85    parse fetch(request)
 86  end
 87
 88  # PATCH
 89  def patch(path, params = {}, as: :form)
 90    request = Net::HTTP::Patch.new(path)
 91
 92    case as
 93    when :json
 94      request.content_type = CONTENT_TYPE_JSON
 95      request.body = JSON.generate(params) unless params.empty?
 96    else
 97      request.content_type = CONTENT_TYPE_FORM
 98      request.body = URI.encode_www_form(params) unless params.empty?
 99    end
100
101    parse fetch(request)
102  end
103
104  # PUT
105  def put(path, params = {}, as: :json)
106    request = Net::HTTP::Put.new(path)
107
108    case as
109    when :json
110      request.content_type = CONTENT_TYPE_JSON
111      request.body = JSON.generate(params) unless params.empty?
112    else
113      request.content_type = CONTENT_TYPE_FORM
114      request.body = URI.encode_www_form(params) unless params.empty?
115    end
116
117    parse fetch(request)
118  end
119
120  # POST multipart
121  def multipart(path, params)
122    request = Net::HTTP::Post.new(path)
123
124    request.content_type = CONTENT_TYPE_MULTIPART
125    request.body = Rack::Multipart::Generator.new(
126      'file' => Rack::Multipart::UploadedFile.new(params['file'][:tempfile].path, params['file'][:type])
127    ).dump
128
129    parse fetch(request)
130  end
131
132  private
133
134  # Perform the request.
135  def fetch(request)
136    # Shore up default headers for the request.
137    request['Accept'] = CONTENT_TYPE_JSON
138    request['Connection'] = 'keep-alive'
139    request['User-Agent'] = 'HTTPDecorator v0.1'
140
141    # Actually make the request.
142    response = @http.request(request)
143
144    # Net::HTTPResponse.value will raise an error for non-200 responses.
145    #   Simpler than trying to detect every possible exception.
146    response.value || response
147  end
148
149  def parse(response)
150    # Parse the response as JSON if possible.
151    if response.content_type == CONTENT_TYPE_JSON
152      JSON.parse(response.body)
153
154    # Otherwise just return the response body.
155    else
156      response.body
157    end
158  end
159end