Abhinay Mehta

Hive Thrift client

Assumptions made: you know what Hive is, you know what Thrift is and you know how to install and start the Hive Thrift server.

To query the Hive server in a language that doesn’t run on the Java Virtual Machine and therefore can’t easily make use of the JBDC drivers, you will need to generate a Thrift client library in your chosen language to use the Thrift Server; as long Thrift has generators for your language of course.

I couldn’t find much documentation online on exactly how to go about this, therefore I thought I’d document the steps I took to generate the Ruby thrift client for Hive, in case someone somewhere wanted to do the same:

• Install Thrift
• Get the Hive thrift files by downloading the Hive source code using one of the methods described here: http://hadoop.apache.org/hive/version_control.html
• Then run the following commands, replacing ‘rb’ with the language of your choice

cd PATH_TO_HIVE_SOURCE_CODE
thrift —gen rb -I service/include metastore/if/hive_metastore.thrift
thrift —gen rb -I service/include -I . service/if/hive_service.thrift
thrift —gen rb service/include/thrift/fb303/if/fb303.thrift
thrift —gen rb serde/if/serde.thrift
thrift —gen rb ql/if/queryplan.thrift
thrift —gen rb service/include/thrift/if/reflection_limited.thrift

• Your thrift client will be in the folder: PATH_TO_HIVE_SOURCE_CODE/gen-rb (again replace rb in the generated folder name with your language)
• You can now copy the generated code around and start using the library to connect with your Hive server

In Ruby you would use the generated code as follows:

Or you could use our Ruby thrift client library, we’ve written a thin layer on top of the thrift code to make things a little easier. Feel free to browse and download the code from here:  http://github.com/forward/rbhive

Ultimate nginx config for Phusion Passenger

Main nginx.conf:

worker_processes  1;
events {
    worker_connections  1024;
}

http {
  include       mime.types;
  default_type  application/octet-stream;

  sendfile        on;
  keepalive_timeout  65;

  gzip  on;
  gzip_http_version 1.0;
  gzip_comp_level 2;
  gzip_buffers 16 8k;
  gzip_proxied any;
  gzip_min_length 360;
  gzip_types      text/plain text/html text/css application/x-javascript text/xml application/xml application/xml+rss text/javascript;
  proxy_set_header    Accept-Encoding  “”;

  passenger_root /usr/local/lib/ruby/gems/1.8/gems/passenger-2.2.10;
  passenger_ruby /usr/local/bin/ruby;

  include /path/to/your/app/config/nginx.conf;
  client_max_body_size 10M;
  client_body_buffer_size 128k;
}

Your application’s nginx.conf:

server {
  listen 80;
  server_name website.com;
  rewrite ^/(.*) https://www.website.com/$1 permanent;
}

server {
  listen 443;
  server_name www.website.com;

  root   /path/to/your/public/folder;
  passenger_enabled on;

  access_log  /path/to/your/log/folder/nginx_access.log;
  error_log  /path/to/your/log/folder/nginx_error.log;

  ssl         on;
  ssl_certificate      /etc/ssl/certs/your-website.crt;
  ssl_certificate_key  /etc/ssl/private/your-website.key;

  location ~* ^.+.(jpg|jpeg|gif|css|png|js|ico)$ {
    root  /path/to/your/public/folder;
    expires max;
    break;
  }
}