Martin Probst's weblog

Memory usage, Java vs. Rails

Wednesday, April 9, 2008, 09:52 — 3 comments Edit

This blog is running on a virtual server at Hosteurope. The server is running a plain Apache 2, my Subversion repository, this blog, and since yesterday a Tomcat instance for the XQuery pretty printer.

The interesting part in getting Tomcat to run was (apart from some mod_jk problems) that I exhausted the 128 MB of memory my virtual server has. Which surprised me a bit - the JVM instance should not consume more than 64 MB, so who was taking all that memory?

This is the current list, according to RSS, which is not precise, but still:

The interesting point is that due to Ruby not supporting kernel threads and Rails generally being single threaded (boo!), I end up with a lot more memory consumed for e.g. a simple blog running 3 mongrel instances plus ferret. RSS includes shared libraries and non-writable memory, but pmap -d reports something like 41592K non shared, private writable memory for the mongrel process.

I had to reduce the number of mongrels to one, so that I can run a Java VM on this host. Thus, my server can only process one concurrent request from one user at this time. When the Ruby process is waiting for filesystem or database operations, nothing else gets processed, even though CPU time would be available.

This is indeed pretty annoying. Some 50 MB don’t seem much memory on a regular desktop machine these days, but for (shared) hosting, it still is a large issue. In the case of my puny blog this isn’t really a problem, plus I’ve set up decent caching so most requests will be handled directly by Apache, but still…

Maybe switching to Merb and Ruby 1.9 (or JRuby) would be useful, though I have no idea if that really runs concurrently in one process, or if it’s just generally possible.

You could try to port your blog to JRuby on Rails. Given the fact, that you are already running a JVM, you might as well use it to power your website.

AFAIK it is possible to start multiple Mongrels in one VM, which might reduce the overall memory usage. Unfortunately I did not yet try this for myself, so I am not sure, if this works out.

As far as I know JRuby has to run a complete, separate Rails instance for each request, as the Rails instances themselves are not thread safe.

So the instances will probably only share the kind of code that is already shared through shared libraries on Linux.

But you’re right, at least I could get rid of the one Mongrel process and of Ferret. I already tried running the blog on JRuby, but Ferret appears to be a problem. And I’m not really looking forward to porting the whole thing to Lucene ….

Did not see this post until now, but yes, with JRuby you can run a whole Rails site, multiple instances and all, in a single process. You can use mongrel_jcluster, warbler (WAR-file deployment of Rails apps), or glassfish (in gem form…one-command production deployment). Check out the JRuby wiki…there’s walkthroughs there: