Imagine you have a Rails model that does not have an integer primary key. Also imagine that you have a massive number of these models and not a whole lot of RAM. “No problem,” you’d say, “I’ll just use
find_each,” which as we all know is a convenient wrapper around
find_in_batches. However, if you’re using PostgreSQL and not sqlite, you’ll soon learn that it does not allow you to use the
>= operator on strings thus rendering
Your options here are to either try and fit them all into RAM or come up with some scheme (perhaps a jiggery-hackery one) to only load a few at a time but iterate over them all. Well hack something together no longer. Introducing Batchelor, an extension for your ActiveRecord classes that have non-integer primary keys.
It is designed to work similar to the default
find_in_batches that is a part of
ActiveRecord::Relation but allow the use of non-integer primary keys by avoiding the use of
>=. I have been told that the reason this is not in Rails is because
find_each are expected to process any new records that are created while the block is running. This will happen for incrementing integer primary keys but is not guaranteed for strings. This is just to give you a convenient and query efficient way to iterate over all records of a model with a non integer primary key.
Check it out and let me know what you think.
GitHub: https://github.com/mojotech/batchelor RubyGems: https://rubygems.org/gems/batchelor