Compiling Ruby 2.1.0 with nokogiri on Windows 7

Compiling Ruby 2.1.0 on Windows 7

We are going to use the RubyInstaller scripts to compile Ruby on Windows. This installer script cares about nearly everything needed to compile Ruby on windows including downloading a C compiler and all dependencies, but it is written in Ruby. This means we need to have a running Ruby on the Windows machine we are using. In my case this means I started with installing Ruby 2.0.0 with the precompiled RubyInstaller from the official website. Make sure to check Add ruby executables to your PATH so we can run the ruby command afterwards in the Windows Command Prompt.

We need to check out the head version of the RubyInstaller from Github. We therefore first need to install Git for Windows. Make sure to select the option Run Git and included Unix tools from the Windows Command Prompt as the Unix tools can become handy in the following steps. We can then run

> cd \
> mkdir projects
> cd projects
> git clone git://github.com/oneclick/rubyinstaller.git

This clones the RubyInstaller repository. Now switch into this directory and let's see what's next by entering

> rake -T

which asks rake for all tasks of the current project. In my case the answer starts with the following advice:

You need rdoc 3.12and rdoc_chm 3.1.0 gems installed
in order to build the docs tasks.
Try `gem install rdoc -v 3.12` and later `gem install rdoc_chm -v 3.1.0`

As the installer script will need the rdoc and rdoc_chm gems later to compile the Ruby documentation we will continue installing them. rdoc is already installed in every modern Ruby so its enough to install rdoc_chm. The installation may prompt if it's ok to overwrite the existing rdoc binary—yes it is.

> gem install rdoc_chm -N

The -N option prevents RubyGems from installing the documentation for the installed gem. As I personally never use the local documentations I always install gems using -N.

A closer look at the rake tasks available presents us with the task ruby21 which will compile ruby. Let's do so.

> rake ruby21

This will download all dependencies including the C compiler into the downloads folder, install them into the sandbox folder and compiles ruby. On my machine this ends up in the following error message:

rake aborted!
undefined method `zero?' for nil:NilClass
...
Tasks: TOP => ruby21 => rbreadline => dependencies:rbreadline:download => downloads/v0.5.0.zip

It turned out that the download of the readline library is configured in the file

C:\projects\rubyinstaller\config\dependencies.rb

as follows

PureReadline = OpenStruct.new(
  :release => 'experimental',
  :version => '0.5.2-0.5.0',
  :url => 'https://github.com/luislavena/rb-readline/archive',
  :target => 'sandbox/rb-readline',
  :files => [
    'v0.5.0.zip'
  ]
)

This is the only element in that file using a secure HTTPS connection in the URL. For me it works if I change this to unsecure HTTP. Running

> rake ruby21

aigain leads to run without further errors.

Packaging the Ruby Installer

After compiling Ruby there are new rake tasks available. Running

> rake -T

again presents us the following new tasks:

rake ruby21:docs # build docs for ruby21
rake ruby21:install # install rubyinstaller-2.1.0-p0.exe
rake ruby21:package# generate packages for ruby 2.1.0-p0
rake ruby21:package:archive # generate ruby-2.1.0-p0-i386-mingw32.7z
rake ruby21:package:installer # generate rubyinstaller-2.1.0-p0.exe
rake ruby21:redocs # rebuild docs for ruby21
rake ruby21:repackage # rebuild rubyinstaller-2.1.0-p0.exe

Now its up to you. You can simply run

> rake ruby21:package:archive

which should work out of the box and generates the archive

C:\projects\rubyinstaller\pkg\ruby-2.1.0-p0-i386-mingw32.7z

which contains a fully working ruby that can be installed on your system simply by extracting this archive into the destination of your choice, e.g.

C:\Ruby210

If you want more you can run

> rake ruby21:package:install

or simply

> rake ruby21:package

which runs both tasks. This will try to generate the installer, but at first run it stops telling us

rake aborted!
You need InnoSetup installed

So lets install InnoSetup and try again. I then get the error message

To generate CHM documentation you need Microsoft's Html Help Workshop installed.
rake aborted!

So lets install Microsoft's Html Help Workshop (actually this link was part of the error message I got) and try again. The installation of the Html Help Workshop was a little bit strange on my computer as it told me, it already was installed, but afterwards the rake tasks starts without further problems, but in the end InnoSetup stops with this error message on my machine:

Error on line 5in C:\projects\rubyinstaller\resources\installer\languages\ja.is
l:Unknown language name "ja"
Compile aborted.

I tried fixing this problem but as I do not need the Japanese localization for installer at all I finally simply removed it completely by deleting this lines

;Include language files
#include"languages\ja.isl"

in the file

C:\projects\rubyinstaller\resources\installer\rubyinstaller.iss

Running the rake task again now has the desired effect:

Successful compile (32,854 sec).ResultingSetup program filename is:
C:\projects\rubyinstaller\pkg\rubyinstaller-2.1.0-p0.exe

Now you can execute this installer and install Ruby 2.1.0 on your windows machine. Again make sure to check "Add ruby executables to your PATH".

Installing DevKit

In the next sections we will install gems with native extensions which need to be compiled manually using Ruby 2.1.0. We therefore need to install the DevKit next. As this is part of the RubyInstaller project the DevKit was already used to install Ruby. The only thing left to do is to install the DevKit for the newly installed Ruby. Copy the folder

C:\projects\rubyinstaller\sandbox\devkit

to

C:\devkit

in case you want to remove the rubyinstaller folder in the end. Then run

> cd C:\devkit
> ruby dk.rb init

and check the generated file config.yml in this folder. In my case I had to remove the line

- C:/Ruby200

as I only want to install the DevKit for the newly installed Ruby 2.1.0. Then run

> ruby dk.rb install

You can check if the DevKit was successfullt installed using

ruby -rdevkit -e 'p "Hello"'

which should output the following if the DevKit was installed:

Temporarily enhancing PATH to include DevKit...
"Hello"

Installing mysql2 gem

My goal is to run a rails project using a mysql database. This made me install the mysql2 gem. Normally this is straight forward as RubyGems contains all the windows binaries needed for this gem, but using Ruby 2.1.0 we need to compile the native extensions on our own. Installing a MySql server is unrelated to compiling the mysql2 gem, but it is needed to test the gem in the end, of cause.

First download the MySql Connector/C in the Windows (x86, 32-bit), ZIP Archive version and extract it into

C:\projects\libmysql

Now run

> gem install mysql2 --platform=ruby -N ----with-mysql-dir=C:/projects/libmysql

The --platform=ruby option tells RubyGems not to use any precompiled binaries but compile the native extension of the gem right now instead. The options after the -- are handed over to the configure script and the last option sets the path to the mysql C connector. Using the --with-mysql-dir option the folder has to contain a subfolder lib containing the dll and a subfolder include containing the C header files. If this is not your setup you can use the --with-mysql-lib and --with-mysql-include options instead.

Finally the libmysql.dll has to be available in the PATH. This can be achieved easiliest by copying it into C:\Ruby210\bin.

You can now check that the newly installed mysql2 gem is working properly, e.g. by setting up a Rails project.

Install nokogiri gem

In my project I need the nokogiri XML parser so we are going to compile it next. After reading the installation guide long enough I understood that I need libxml2 and libxslt. Both are part of the The XML C parser and toolkit of Gnome which provides windows binaries. So go on and download the following files

• iconv-1.9.2.win32.zip
• libxml2-2.7.8.win32.zip
• libxslt-1.1.26.win32.zip

libiconv is a dependency of libxml2 which I ony learned from the error messages of the compiler. Extract the files into the folders

• C:\projects\libiconv
• C:\projects\libxml2
• C:\projects\libxslt

The actual DLL-files are located in the bin subfolders, but the installation process exspects them in the lib folder. We therefore copy them

cp C:/projects/libiconv/bin/*.dll C:/projects/libiconv/lib
cp C:/projects/libxml2/bin/*.dll C:/projects/libxml2/lib
cp C:/projects/libxslt/bin/*.dll C:/projects/libxslt/lib

Now we can run

> gem install nokogiri --platform=ruby -N ----with-iconv-dir=C:/projects/libiconv --with-xml2-dir=C:/projects/libxml2 --with-xslt-dir=C:/projects/libxslt

If this does not work in the first carefully read the detailed log file mkmf.log which is a little bit difficult to locate. On my system if found it here:

C:\Ruby210\lib\ruby\gems\2.1.0\gems\nokogiri-1.6.1\ext\nokogiri

This is where I learned from that libxml2 needs iconv.

Finally the DLLs must be in the PATH. Again the easiest solution is simply copy them into Rubys bin folder.

cp C:/projects/libiconv/bin/*.dll C:/Ruby210/bin
cp C:/projects/libxml2/bin/*.dll C:/Ruby210/bin
cp C:/projects/libxslt/bin/*.dll C:/Ruby210/bin

Now everything should work fine. If you were able to compile the gem but the following strange error message you failed copying all needed DLLs in the PATH.

LoadError:126:The specified module could not be found. - C:/Ruby210/lib/ruby/gems/2.1.0/extensions/x86-mingw32/2.1.0/nokogiri-1.6.1/nokogiri/nokogiri.so

Sadly to say this error message does not tell you at all which specified module could not be found. I ended up opening the nokogiri.so in the Dependency Walker to find out I failed copying the iconv.dll as you can see in the screenshot below:

Install bcrypt-ruby

As last step I needed to reinstall the bcrypt-ruby gem with compiling the native extensions manually as the precompiled binaries in the RubyGems repository are not ready for Ruby 2.1.0 yet. As the bcrypt-ruby has no dependencies this is fairly easy.

> gem uninstall bcrypt-ruby
> gem install bcrypt-ruby -N --platform=ruby

At least my project is now running using Ruby 2.1.0 on windows. If you are using other gems exspect further steps needed to get everything running. BTW: This includes the sqlite3 gem which is used as default in new Rails4 projects.