Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding Fragment Cache to AMS #810

Merged
merged 1 commit into from
Apr 21, 2015
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,3 +4,5 @@
* adds method to override association [adcb99e, @kurko]
* adds `has_one` attribute for backwards compatibility [@ggordon]
* updates JSON API support to RC3 [@mateomurphy]
* adds fragment cache support [@joaomdmoura]
* adds cache support to attributes and associations [@joaomdmoura]
24 changes: 24 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -271,7 +271,10 @@ The options are the same options of ```ActiveSupport::Cache::Store```, plus
a ```key``` option that will be the prefix of the object cache
on a pattern ```"#{key}/#{object.id}-#{object.updated_at}"```.

The cache support is optimized to use the cached object in multiple request. An object cached on an ```show``` request will be reused at the ```index```. If there is a relationship with another cached serializer it will also be created and reused automatically.

**[NOTE] Every object is individually cached.**

**[NOTE] The cache is automatically expired after update an object but it's not deleted.**

```ruby
Expand All @@ -295,6 +298,27 @@ On this example every ```Post``` object will be cached with
the key ```"post/#{post.id}-#{post.updated_at}"```. You can use this key to expire it as you want,
but in this case it will be automatically expired after 3 hours.

### Fragmenting Caching

If there is some API endpoint that shouldn't be fully cached, you can still optmise it, using Fragment Cache on the attributes and relationships that you want to cache.

You can define the attribute by using ```only``` or ```except``` option on cache method.

**[NOTE] Cache serializers will be used at their relationships**

Example:

```ruby
class PostSerializer < ActiveModel::Serializer
cache key: 'post', expires_in: 3.hours, only: [:title]
attributes :title, :body

has_many :comments

url :post
end
```

## Getting Help

If you find a bug, please report an [Issue](https://github.com/rails-api/active_model_serializers/issues/new).
Expand Down
52 changes: 36 additions & 16 deletions lib/active_model/serializer.rb
Original file line number Diff line number Diff line change
Expand Up @@ -10,41 +10,54 @@ class Serializer

class << self
attr_accessor :_attributes
attr_accessor :_attributes_keys
attr_accessor :_associations
attr_accessor :_urls
attr_accessor :_cache
attr_accessor :_fragmented
attr_accessor :_cache_key
attr_accessor :_cache_only
attr_accessor :_cache_except
attr_accessor :_cache_options
end

def self.inherited(base)
base._attributes = []
base._attributes_keys = {}
base._associations = {}
base._urls = []
end

def self.attributes(*attrs)
attrs = attrs.first if attrs.first.class == Array
@_attributes.concat attrs

attrs.each do |attr|
define_method attr do
object && object.read_attribute_for_serialization(attr)
end unless method_defined?(attr)
end unless method_defined?(attr) || _fragmented.respond_to?(attr)
end
end

def self.attribute(attr, options = {})
key = options.fetch(:key, attr)
@_attributes_keys[attr] = {key: key} if key != attr
@_attributes.concat [key]
define_method key do
object.read_attribute_for_serialization(attr)
end unless method_defined?(key)
end unless method_defined?(key) || _fragmented.respond_to?(attr)
end

def self.fragmented(serializer)
@_fragmented = serializer
end

# Enables a serializer to be automatically cached
def self.cache(options = {})
@_cache = ActionController::Base.cache_store if Rails.configuration.action_controller.perform_caching
@_cache_key = options.delete(:key)
@_cache = ActionController::Base.cache_store if Rails.configuration.action_controller.perform_caching
@_cache_key = options.delete(:key)
@_cache_only = options.delete(:only)
@_cache_except = options.delete(:except)
@_cache_options = (options.empty?) ? nil : options
end

Expand Down Expand Up @@ -141,12 +154,12 @@ def self.root_name
attr_accessor :object, :root, :meta, :meta_key, :scope

def initialize(object, options = {})
@object = object
@options = options
@root = options[:root] || (self.class._root ? self.class.root_name : false)
@meta = options[:meta]
@meta_key = options[:meta_key]
@scope = options[:scope]
@object = object
@options = options
@root = options[:root] || (self.class._root ? self.class.root_name : false)
@meta = options[:meta]
@meta_key = options[:meta_key]
@scope = options[:scope]

scope_name = options[:scope_name]
if scope_name && !respond_to?(scope_name)
Expand Down Expand Up @@ -183,22 +196,29 @@ def attributes(options = {})
attributes += options[:required_fields] if options[:required_fields]

attributes.each_with_object({}) do |name, hash|
hash[name] = send(name)
unless self.class._fragmented
hash[name] = send(name)
else
hash[name] = self.class._fragmented.public_send(name)
end
end
end

def each_association(&block)
self.class._associations.dup.each do |name, association_options|
next unless object

association_value = send(name)

serializer_class = ActiveModel::Serializer.serializer_for(association_value, association_options)

serializer = serializer_class.new(
association_value,
options.merge(serializer_from_options(association_options))
) if serializer_class
if serializer_class
serializer = serializer_class.new(
association_value,
options.merge(serializer_from_options(association_options))
)
elsif !association_value.nil? && !association_value.instance_of?(Object)
association_options[:association_options][:virtual_value] = association_value
end

if block_given?
block.call(name, serializer, association_options[:association_options])
Expand Down
46 changes: 32 additions & 14 deletions lib/active_model/serializer/adapter.rb
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
require 'active_model/serializer/adapter/fragment_cache'

module ActiveModel
class Serializer
class Adapter
Expand Down Expand Up @@ -32,8 +34,38 @@ def self.adapter_class(adapter)
"ActiveModel::Serializer::Adapter::#{adapter.to_s.classify}".safe_constantize
end

def fragment_cache(*args)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kurko abstracted method. Duck typing strategy.

raise NotImplementedError, 'This is an abstract method. Should be implemented at the concrete adapter.'
end

private

def cache_check(serializer)
@cached_serializer = serializer
@klass = @cached_serializer.class
if is_cached?
@klass._cache.fetch(cache_key, @klass._cache_options) do
yield
end
elsif is_fragment_cached?
FragmentCache.new(self, @cached_serializer, @options, @root).fetch
else
yield
end
end

def is_cached?
@klass._cache && !@klass._cache_only && !@klass._cache_except
end

def is_fragment_cached?
@klass._cache_only && !@klass._cache_except || !@klass._cache_only && @klass._cache_except
end

def cache_key
(@klass._cache_key) ? "#{@klass._cache_key}/#{@cached_serializer.object.id}-#{@cached_serializer.object.updated_at}" : @cached_serializer.object.cache_key
end
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you extract all of these methods into another class? Cache perhaps? e.g

def cached_hash_root(cached_hash, non_cached_hash)
  cache.hash_root(cached_hash, non_cached_hash)
end

# ...

private

def cache
  Cache.new(self)
end

Otherwise, it's going to be hard to be maintained.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Totally agree. I'll extract it to an standalone module and include it in the Adapter. If you have another suggestions just let me know


def meta
serializer.meta if serializer.respond_to?(:meta)
end
Expand All @@ -50,20 +82,6 @@ def include_meta(json)
json[meta_key] = meta if meta && root
json
end

private

def cached_object
klass = serializer.class
if klass._cache
_cache_key = (klass._cache_key) ? "#{klass._cache_key}/#{serializer.object.id}-#{serializer.object.updated_at}" : serializer.object.cache_key
klass._cache.fetch(_cache_key, klass._cache_options) do
yield
end
else
yield
end
end
end
end
end
78 changes: 78 additions & 0 deletions lib/active_model/serializer/adapter/fragment_cache.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
module ActiveModel
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I commented everything that could make it easy to support.

class Serializer
class Adapter
class FragmentCache

attr_reader :serializer

def initialize(adapter, serializer, options, root)
@root = root
@options = options
@adapter = adapter
@serializer = serializer
end

def fetch
klass = serializer.class
# It will split the serializer into two, one that will be cached and other wont
serializers = fragment_serializer(serializer.object.class.name, klass)

# Instanciate both serializers
cached_serializer = serializers[:cached].constantize.new(serializer.object)
non_cached_serializer = serializers[:non_cached].constantize.new(serializer.object)

cached_adapter = @adapter.class.new(cached_serializer, @options)
non_cached_adapter = @adapter.class.new(non_cached_serializer, @options)

# Get serializable hash from both
cached_hash = cached_adapter.serializable_hash
non_cached_hash = non_cached_adapter.serializable_hash

# Merge both results
@adapter.fragment_cache(cached_hash, non_cached_hash)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The duck typing implementation that was defined at adapter.rb similar to serializable_hash

end

private

def cached_attributes(klass, serializers)
cached_attributes = (klass._cache_only) ? klass._cache_only : serializer.attributes.keys.delete_if {|attr| klass._cache_except.include?(attr) }
non_cached_attributes = serializer.attributes.keys.delete_if {|attr| cached_attributes.include?(attr) }

cached_attributes.each do |attribute|
options = serializer.class._attributes_keys[attribute]
options ||= {}
# Add cached attributes to cached Serializer
serializers[:cached].constantize.attribute(attribute, options)
end

non_cached_attributes.each do |attribute|
options = serializer.class._attributes_keys[attribute]
options ||= {}
# Add non-cached attributes to non-cached Serializer
serializers[:non_cached].constantize.attribute(attribute, options)
end
end

def fragment_serializer(name, klass)
cached = "#{name.capitalize}CachedSerializer"
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If different serializers share the same model, but has totally different attributes, would their caches collide, even if the key is different? I believe i see it on my machine. If you believe this could happen, i can present those tests to you, but i need time to set it up.

maybe it should go something like cached = "#{serializer.name}CachedSerializer"?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeap, I think it could happen. Not the cache_key itself, because it takes the file path into account but the Class name would colide.
I'll check it on the master codebase, create a new issue for that and update it.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@joaomdmoura thanks, that would be great!
Here is the stack trace of the error i've got, when the classes collide, in case you would like to look at it. The latter called serializer tries to look for an attribute that was held by the first serializer, but not in the latter. Am i abusing the way serializers should be used, if i'm the first one who's got that error?

Failure/Error: get '/markers/local.json?coordinates=55.75,36.00&limit=10'
     NoMethodError:
       undefined method `name' for #<MarkerDistanceSerializer:0x007f80339b6eb8>
     # /Users/antonmurygin/.rvm/gems/ruby-2.2.2/bundler/gems/active_model_serializers-e7d3323d2352/lib/active_model/serializer.rb:155:in `public_send'
     # /Users/antonmurygin/.rvm/gems/ruby-2.2.2/bundler/gems/active_model_serializers-e7d3323d2352/lib/active_model/serializer.rb:155:in `block in attributes'
     # /Users/antonmurygin/.rvm/gems/ruby-2.2.2/bundler/gems/active_model_serializers-e7d3323d2352/lib/active_model/serializer.rb:151:in `each'
     # /Users/antonmurygin/.rvm/gems/ruby-2.2.2/bundler/gems/active_model_serializers-e7d3323d2352/lib/active_model/serializer.rb:151:in `each_with_object'
     # /Users/antonmurygin/.rvm/gems/ruby-2.2.2/bundler/gems/active_model_serializers-e7d3323d2352/lib/active_model/serializer.rb:151:in `attributes'
     # /Users/antonmurygin/.rvm/gems/ruby-2.2.2/bundler/gems/active_model_serializers-e7d3323d2352/lib/active_model/serializer/adapter/json.rb:15:in `block in serializable_hash'
     # /Users/antonmurygin/.rvm/gems/ruby-2.2.2/bundler/gems/active_model_serializers-e7d3323d2352/lib/active_model/serializer/adapter.rb:51:in `block in cache_check'
     # /Users/antonmurygin/.rvm/gems/ruby-2.2.2/gems/activesupport-4.2.3/lib/active_support/cache.rb:299:in `block in fetch'
     # /Users/antonmurygin/.rvm/gems/ruby-2.2.2/gems/activesupport-4.2.3/lib/active_support/cache.rb:585:in `block in save_block_result_to_cache'
     # /Users/antonmurygin/.rvm/gems/ruby-2.2.2/gems/activesupport-4.2.3/lib/active_support/cache.rb:547:in `block in instrument'
     # /Users/antonmurygin/.rvm/gems/ruby-2.2.2/gems/activesupport-4.2.3/lib/active_support/notifications.rb:166:in `instrument'
     # /Users/antonmurygin/.rvm/gems/ruby-2.2.2/gems/activesupport-4.2.3/lib/active_support/cache.rb:547:in `instrument'
     # /Users/antonmurygin/.rvm/gems/ruby-2.2.2/gems/activesupport-4.2.3/lib/active_support/cache.rb:584:in `save_block_result_to_cache'
     # /Users/antonmurygin/.rvm/gems/ruby-2.2.2/gems/activesupport-4.2.3/lib/active_support/cache.rb:299:in `fetch'
     # /Users/antonmurygin/.rvm/gems/ruby-2.2.2/bundler/gems/active_model_serializers-e7d3323d2352/lib/active_model/serializer/adapter.rb:50:in `cache_check'
     # /Users/antonmurygin/.rvm/gems/ruby-2.2.2/bundler/gems/active_model_serializers-e7d3323d2352/lib/active_model/serializer/adapter/json.rb:14:in `serializable_hash'
     # /Users/antonmurygin/.rvm/gems/ruby-2.2.2/bundler/gems/active_model_serializers-e7d3323d2352/lib/active_model/serializer/adapter/flatten_json.rb:6:in `serializable_hash'
     # /Users/antonmurygin/.rvm/gems/ruby-2.2.2/bundler/gems/active_model_serializers-e7d3323d2352/lib/active_model/serializer/adapter/fragment_cache.rb:27:in `fetch'
     # /Users/antonmurygin/.rvm/gems/ruby-2.2.2/bundler/gems/active_model_serializers-e7d3323d2352/lib/active_model/serializer/adapter.rb:54:in `cache_check'
     # /Users/antonmurygin/.rvm/gems/ruby-2.2.2/bundler/gems/active_model_serializers-e7d3323d2352/lib/active_model/serializer/adapter/json.rb:14:in `serializable_hash'
     # /Users/antonmurygin/.rvm/gems/ruby-2.2.2/bundler/gems/active_model_serializers-e7d3323d2352/lib/active_model/serializer/adapter/flatten_json.rb:6:in `serializable_hash'
     # /Users/antonmurygin/.rvm/gems/ruby-2.2.2/bundler/gems/active_model_serializers-e7d3323d2352/lib/active_model/serializer/adapter/json.rb:10:in `block in serializable_hash'
     # /Users/antonmurygin/.rvm/gems/ruby-2.2.2/bundler/gems/active_model_serializers-e7d3323d2352/lib/active_model/serializer/array_serializer.rb:6:in `each'
     # /Users/antonmurygin/.rvm/gems/ruby-2.2.2/bundler/gems/active_model_serializers-e7d3323d2352/lib/active_model/serializer/array_serializer.rb:6:in `each'
     # /Users/antonmurygin/.rvm/gems/ruby-2.2.2/bundler/gems/active_model_serializers-e7d3323d2352/lib/active_model/serializer/adapter/json.rb:10:in `map'
     # /Users/antonmurygin/.rvm/gems/ruby-2.2.2/bundler/gems/active_model_serializers-e7d3323d2352/lib/active_model/serializer/adapter/json.rb:10:in `serializable_hash'
     # /Users/antonmurygin/.rvm/gems/ruby-2.2.2/bundler/gems/active_model_serializers-e7d3323d2352/lib/active_model/serializer/adapter/flatten_json.rb:6:in `serializable_hash'
     # /Users/antonmurygin/.rvm/gems/ruby-2.2.2/bundler/gems/active_model_serializers-e7d3323d2352/lib/active_model/serializer/adapter.rb:24:in `as_json'
     # /Users/antonmurygin/.rvm/gems/ruby-2.2.2/gems/activesupport-4.2.3/lib/active_support/json/encoding.rb:35:in `encode'
     # /Users/antonmurygin/.rvm/gems/ruby-2.2.2/gems/activesupport-4.2.3/lib/active_support/json/encoding.rb:22:in `encode'
     # /Users/antonmurygin/.rvm/gems/ruby-2.2.2/gems/activesupport-4.2.3/lib/active_support/core_ext/object/json.rb:37:in `to_json_with_active_support_encoder'
     # /Users/antonmurygin/.rvm/gems/ruby-2.2.2/gems/actionpack-4.2.3/lib/action_controller/metal/renderers.rb:116:in `block in <module:Renderers>'
     # /Users/antonmurygin/.rvm/gems/ruby-2.2.2/bundler/gems/active_model_serializers-e7d3323d2352/lib/action_controller/serialization.rb:51:in `block (2 levels) in <module:Serialization>'
     # /Users/antonmurygin/.rvm/gems/ruby-2.2.2/gems/actionpack-4.2.3/lib/action_controller/metal/renderers.rb:45:in `block in _render_to_body_with_renderer'
     # /Users/antonmurygin/.rvm/gems/ruby-2.2.2/gems/actionpack-4.2.3/lib/action_controller/metal/renderers.rb:41:in `_render_to_body_with_renderer'
     # /Users/antonmurygin/.rvm/gems/ruby-2.2.2/gems/actionpack-4.2.3/lib/action_controller/metal/renderers.rb:37:in `render_to_body'
     # /Users/antonmurygin/.rvm/gems/ruby-2.2.2/gems/actionpack-4.2.3/lib/abstract_controller/rendering.rb:25:in `render'
     # /Users/antonmurygin/.rvm/gems/ruby-2.2.2/gems/actionpack-4.2.3/lib/action_controller/metal/rendering.rb:16:in `render'
     # /Users/antonmurygin/.rvm/gems/ruby-2.2.2/gems/actionpack-4.2.3/lib/action_controller/metal/instrumentation.rb:44:in `block (2 levels) in render'
     # /Users/antonmurygin/.rvm/gems/ruby-2.2.2/gems/activesupport-4.2.3/lib/active_support/core_ext/benchmark.rb:12:in `block in ms'
     # /Users/antonmurygin/.rvm/gems/ruby-2.2.2/gems/activesupport-4.2.3/lib/active_support/core_ext/benchmark.rb:12:in `ms'
     # /Users/antonmurygin/.rvm/gems/ruby-2.2.2/gems/actionpack-4.2.3/lib/action_controller/metal/instrumentation.rb:44:in `block in render'
     # /Users/antonmurygin/.rvm/gems/ruby-2.2.2/gems/actionpack-4.2.3/lib/action_controller/metal/instrumentation.rb:87:in `cleanup_view_runtime'
     # /Users/antonmurygin/.rvm/gems/ruby-2.2.2/gems/activerecord-4.2.3/lib/active_record/railties/controller_runtime.rb:25:in `cleanup_view_runtime'
     # /Users/antonmurygin/.rvm/gems/ruby-2.2.2/gems/actionpack-4.2.3/lib/action_controller/metal/instrumentation.rb:43:in `render'
     # /Users/antonmurygin/.rvm/gems/ruby-2.2.2/gems/remotipart-1.2.1/lib/remotipart/render_overrides.rb:14:in `render_with_remotipart'
     # ./app/controllers/markers_controller.rb:20:in `block (2 levels) in local'

non_cached = "#{name.capitalize}NonCachedSerializer"

Object.const_set cached, Class.new(ActiveModel::Serializer) unless Object.const_defined?(cached)
Object.const_set non_cached, Class.new(ActiveModel::Serializer) unless Object.const_defined?(non_cached)

klass._cache_options ||= {}
klass._cache_options[:key] = klass._cache_key if klass._cache_key

cached.constantize.cache(klass._cache_options)

cached.constantize.fragmented(serializer)
non_cached.constantize.fragmented(serializer)

serializers = {cached: cached, non_cached: non_cached}
cached_attributes(klass, serializers)
serializers
end
end
end
end
end
44 changes: 30 additions & 14 deletions lib/active_model/serializer/adapter/json.rb
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
require 'active_model/serializer/adapter/json/fragment_cache'

module ActiveModel
class Serializer
class Adapter
Expand All @@ -6,31 +8,45 @@ def serializable_hash(options = {})
if serializer.respond_to?(:each)
@result = serializer.map{|s| self.class.new(s).serializable_hash }
else
@result = cached_object do
@hash = serializer.attributes(options)
serializer.each_association do |name, association, opts|
if association.respond_to?(:each)
array_serializer = association
@hash[name] = array_serializer.map { |item| item.attributes(opts) }
else
if association
@hash[name] = association.attributes(options)
else
@hash[name] = nil
@hash = {}

@core = cache_check(serializer) do
serializer.attributes(options)
end

serializer.each_association do |name, association, opts|
if association.respond_to?(:each)
array_serializer = association
@hash[name] = array_serializer.map do |item|
cache_check(item) do
item.attributes(opts)
end
end
else
if association
@hash[name] = cache_check(association) do
association.attributes(options)
end
elsif opts[:virtual_value]
@hash[name] = opts[:virtual_value]
else
@hash[name] = nil
end
end
@hash
end
@result = @core.merge @hash
end

if root = options.fetch(:root, serializer.json_key)
@result = { root => @result }
end

@result
end
end

def fragment_cache(cached_hash, non_cached_hash)
Json::FragmentCache.new().fragment_cache(cached_hash, non_cached_hash)
end
end
end
end
end
15 changes: 15 additions & 0 deletions lib/active_model/serializer/adapter/json/fragment_cache.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
module ActiveModel
class Serializer
class Adapter
class Json < Adapter
class FragmentCache

def fragment_cache(cached_hash, non_cached_hash)
non_cached_hash.merge cached_hash
end

end
end
end
end
end
Loading