Native extensions fallback to pure Ruby if not supported on gem install

Question

I am developing a gem, which is currently pure Ruby, but I have also been developing a faster C variant for one of the features. The feature is usable, but sometimes slow, in pu

Answer 1

Here is a thought, based on info from http://guides.rubygems.org/c-extensions/ and http://yorickpeterse.com/articles/hacking-extconf-rb/.

Looks like you can put the logic in extconf.rb. For example, query the RUBY_DESCRIPTION constant and determine if you are in a Ruby that supports native extensions:

$ irb
jruby-1.6.8 :001 > RUBY_DESCRIPTION
=> "jruby 1.6.8 (ruby-1.8.7-p357) (2012-09-18 1772b40) (Java HotSpot(TM) 64-Bit Server VM       
    1.6.0_51) [darwin-x86_64-java]"

So you could try something like wrap the code in extconf.rb in a conditional (in extconf.rb):

unless RUBY_DESCRIPTION =~ /jruby/ do

  require 'mkmf'

  # stuff    
  create_makefile('my_extension/my_extension')

end

Obviously, you will want more sophisticated logic, grabbing parameters passed on "gem install", etc.

Answer 2

This is my best result attempting to answer my own question to date. It appears to work for JRuby (tested in Travis and on my local installation under RVM), which was my main goal. However, I would be very interested in confirmations of it working in other environments, and for any input on how to make it more generic and/or robust:

---

The gem installation code expects a Makefile as output from extconf.rb, but has no opinion on what that should contain. Therefore extconf.rb can decide to create a do nothing Makefile, instead of calling create_makefile from mkmf. In practice that might look like this:

ext/foo/extconf.rb

can_compile_extensions = false
want_extensions = true

begin
  require 'mkmf'
  can_compile_extensions = true
rescue Exception
  # This will appear only in verbose mode.
  $stderr.puts "Could not require 'mkmf'. Not fatal, the extensions are optional."
end


if can_compile_extensions && want_extensions
  create_makefile( 'foo/foo' )

else
  # Create a dummy Makefile, to satisfy Gem::Installer#install
  mfile = open("Makefile", "wb")
  mfile.puts '.PHONY: install'
  mfile.puts 'install:'
  mfile.puts "\t" + '@echo "Extensions not installed, falling back to pure Ruby version."'
  mfile.close

end

As suggested in the question, this answer also requires the following logic to load the Ruby fallback code in the main library:

lib/foo.rb (excerpt)

begin
  # Extension target, might not exist on some installations
  require 'foo/foo'
rescue LoadError
  # Pure Ruby fallback, should cover all methods that are otherwise in extension
  require 'foo/foo_pure_ruby'
end

Following this route also requires some juggling of rake tasks, so that the default rake task doesn't attempt to compile on Rubies that we're testing on that don't have capability to compile extensions:

Rakefile (excerpts)

def can_compile_extensions
  return false if RUBY_DESCRIPTION =~ /jruby/
  return true
end 

if can_compile_extensions
  task :default => [:compile, :test]
else
  task :default => [:test]
end

Note the Rakefile part doesn't have to be completely generic, it just has to cover known environments we want to locally build and test the gem on (e.g. all the Travis targets).

I have noticed one annoyance. That is by default you will see Ruby Gems' message Building native extensions. This could take a while..., and no indication that the extension compilation was skipped. However, if you invoke the installer with gem install foo --verbose you do see the messages added to extconf.rb, so it's not too bad.

Answer 3

https://stackoverflow.com/posts/50886432/edit

I tried the other answers and could not get them to build on recent Rubies.

This worked for me:

1. Use mkmf#have_* methods in extconf.rb to check for everything you need. Then call #create_makefile, no matter what.
2. Use the preprocessor constants generated by #have_* to skip things in your C file.
3. Check which methods/modules are defined in Ruby.
4. If you want to support JRuby et al, you'll need a more complex release setup.

---

A simple example where the whole C extension is skipped if something is missing:

1. ext/my_gem/extconf.rb

require 'mkmf'

have_struct_member('struct foo', 'bar')

create_makefile('my_gem/my_gem')

2. ext/my_gem/my_gem.c

#ifndef HAVE_STRUCT_FOO_BAR
  // C ext cant be compiled, ignore because it's optional
  void Init_my_gem() {}
#else
  #include "ruby.h"

  void Init_my_gem() {
    VALUE mod;
    mod = rb_define_module("MyGemExt");
    // attach methods to module
  }
#endif

3. lib/my_gem.rb

class MyGem
  begin
    require 'my_gem/my_gem'
    include MyGemExt
  rescue LoadError, NameError
    warn 'Running MyGem without C extension, using slower Ruby fallback'
    include MyGem::RubyFallback
  end
end

4. If you want to release the gem for JRuby, you need to adapt the gemspec before packaging. This will allow you to build and release multiple versions of the gem. The simplest solution I can think of:

Rakefile

require 'rubygems/package_task'

namespace :java do
  java_gemspec = eval File.read('./my_gem.gemspec')
  java_gemspec.platform = 'java'
  java_gemspec.extensions = [] # override to remove C extension

  Gem::PackageTask.new(java_gemspec) do |pkg|
    pkg.need_zip = true
    pkg.need_tar = true
    pkg.package_dir = 'pkg'
  end
end

task package: 'java:gem'

Then run $ rake package && gem push pkg/my_gem-0.1.0 && gem push pkg/my_gem-0.1.0-java to release a new version.

If you just want to run on JRuby, not distribute the gem for it, this will suffice (it will not work for releasing the gem, though, as it is evaluated before packaging):

my_gem.gemspec

if RUBY_PLATFORM !~ /java/i
  s.extensions = %w[ext/my_gem/extconf.rb]
end

---

This approach has two advantages:

• create_makefile should work in every environment
• a compile task can remain prepended to other tasks (except on JRuby)