Gitlab 4.0 on a Gentoo server, ~amd64, hardened w/ Apache.

Posted on Dec 29, 2012 in Featured Articles, Linux and Open Source, Technology

The article about Gitlab on Ubuntu gets some views every day, so I guess it is an interesting subject -and it should be. GitLab is a great project. It could do with less RAM consumption and less frequent updates though, but I am only whining because I am using it and I am not going to uninstall it anytime soon. :p

The thing is I am in love with Gentoo for many years now and my Ubuntu server was more like an one time thing. So I set to install Gitlab on my ~amd64 hardened Gentoo and log the process for anyone interested. It is a great time too since Gitlab recently reached version 4.0 with enhanced project management.

 

Update (9 July 2013): An excellent article for installing a recent version of Gitlab (5.3 at the time of writing) is available at the official Gentoo Wiki thanks to the commenter with the inspired name 666threesixes666. :) Many thanks!

 

How to read this tutorial

This tutorial’s purpose is to be used as a generic guide. Gitlab has frequent updates, once every month for both the software and the installation instructions; unfortunately I can’t maintain such a document.

The official installation guide covers Debian and Ubuntu. Since we are on Gentoo, we have to depart here and there. Fortunately, these parts that are different between Linux distributions, doesn’t change much.

So ideally you will open my guide on one half of your screen and the official guide on the other half. You will follow the official guide and when you detect a difference, you will use my guide -but apply some logic too, you are running Gentoo after all.

I will try my best to make it clear when I copy the official guide and when I am doing Gentoo specific parts.

During most of this guide I use the sudo command, even when a command needs to run as root. I assume though that you are already root (Gentoo’s usual way). Using sudo in this case doesn’t harm in any way, I just try to keep my commands similar to Gitlab’s official instructions.

Note: Chances are if you want to run Gitlab, you already run a web server. So, I won’t go into specifics about setting up Apache, MySQL, postfix etc. I will give you only the Gitlab specific parts for these programs. Before running my commands, I expect to you to think a bit and if needed, adjust them to your setup.

 

1-2. Packages / Dependencies / Ruby (Gentoo way)

Gitlab, as most programs, has dependencies. Emerging them one by one isn’t elegant and won’t help if you ever decide to remove Gitlab from your system.
Download this ebuild instead (gitlab-deps-0.ebuild 4.0-stable), put it on your local overlay, emerge it and you are good to go. If you ever remove Gitlab, unmerge gitlab-deps and do an emerge –depclean.

Important: Gitlab depends on postfix. On my ebuild, I changed this to virtual/mta, so that it won’t ask you to change your setup. I haven’t test it though with anything else other than postfix. If you haven’t set an mta for your server (unlikely), then I encourage you to install and set up postfix before installing gitlab-deps.

If you don’t care about doing things elegantly and just want an emerge oneliner, here it is. It is strongly discouraged though. Gentoo isn’t designed to be used like this (adding dependencies as world entries).

$ emerge app-admin/sudo net-misc/wget net-misc/curl sys-devel/gcc dev-libs/libxml2 dev-libs/libxslt sys-libs/readline virtual/libc sys-devel/make sys-libs/zlib dev-libs/icu dev-db/redis net-misc/openssh dev-vcs/git dev-lang/python dev-lang/ruby:1.9 dev-ruby/rubygems dev-ruby/bundler dev-python/pip dev-libs/libyaml mail-mta/postfix dev-python/pygments

If you went with the oneliner, also emerge the database you want to use (mysql or postgres).

 

3. System Users (Gentoo way)

Create accounts for Git and Gitolite.

$ useradd -m -U -s /bin/sh -b /home -c "git repositories" git $ useradd -m -U -s /sbin/nologin -b /home -c "gitlab" gitlab

$ usermod -a -G git gitlab $ usermod -a -G gitlab git

$ sudo -H -u gitlab ssh-keygen -q -N ” -t rsa -f /home/gitlab/.ssh/id_rsa

The git account you just created is disabled, so you can’t ssh (or git/ssh) to it. Enable it by giving it a random password (no need to remember it):

$ passwd git

 

4. Gitolite (Gitlab way)

This is a direct copy from Gitlab’s official installation guide.

$ cd /home/git $ sudo -H -u git git clone -b gl-v304 https://github.com/gitlabhq/gitolite.git /home/git/gitolite

$ sudo -u git -H mkdir bin $ sudo -u git sh -c ‘echo -e “PATH=\$PATH:/home/git/bin\nexport PATH” >> /home/git/.profile’ $ sudo -u git sh -c ‘gitolite/install -ln /home/git/bin’

$ cp /home/gitlab/.ssh/id_rsa.pub /home/git/gitlab.pub $ chmod 0444 /home/git/gitlab.pub

$ sudo -u git -H sh -c “PATH=/home/git/bin:$PATH; gitolite setup -pk /home/git/gitlab.pub” $ sudo chmod 750 /home/git/.gitolite/ $ sudo chown -R git:git /home/git/.gitolite/ $ sudo chmod -R ug+rwXs,o-rwx /home/git/repositories/ $ sudo chown -R git:git /home/git/repositories/

 

4.1. Disable StrictHostKeyChecking for localhost and your domain (Gitlab way)

$ echo "Host localhost StrictHostKeyChecking no UserKnownHostsFile=/dev/null" | sudo tee -a /etc/ssh/ssh_config

$ echo “Host YOUR_DOMAIN_NAME StrictHostKeyChecking no UserKnownHostsFile=/dev/null” | sudo tee -a /etc/ssh/ssh_config

 

4.2. Custom ssh port (Gitlab/Gentoo way)

Usually you want your ssh daemon to hear to a non standard port in order to avoid automated port scans. In order for this to work with Gitlab, you have to configure the ssh client for your gitlab account:

$ sudo -u gitlab -H echo -e "Host localhost\n Port CUSTOM_SSH_PORT" >> /home/gitlab/.ssh/config

 

4.3. Test if everything works so far (Gitlab way)

$ sudo -u gitlab -H git clone git@localhost:gitolite-admin.git /tmp/gitolite-admin $ sudo rm -rf /tmp/gitolite-admin

 

5. Database

This section exists because it is present on the official guide too. No need to do anything here.

 

6. Gitlab (Gitlab way with Gentoo specific parts)

 

6.1-6.2. Clone the source and configure it (Gitlab way)

$ cd /home/gitlab/ $ sudo -H -u gitlab git clone -b stable https://github.com/gitlabhq/gitlabhq.git gitlab $ cd /home/gitlab/gitlab $ sudo -u gitlab -H git checkout 4-0-stable $ sudo -u gitlab -H cp config/gitlab.yml.example config/gitlab.yml $ sudo -u gitlab -H nano -w config/gitlab.yml $ sudo chown -R gitlab log/ $ sudo chown -R gitlab tmp/ $ sudo chmod -R u+rwX log/ $ sudo chmod -R u+rwX tmp/ $ sudo -u gitlab -H cp config/unicorn.rb.example config/unicorn.rb

 

6.3 Configure GitLab DB settings (Gentoo way)

We are going to use a MySQL database. Create a database and a user for Gitlab, then configure Gitlab.

In case you don’t remember how to create a MySQL database:

$ mysql -uroot -p mysql> CREATE DATABASE IF NOT EXISTS gitlabdb; mysql> GRANT ALL ON gitlabdb.* TO gitlab@localhost IDENTIFIED BY 'gitlabpass';

Now configure Gitlab to use MySQL. Only the production section of database.yml needs to be adjusted.

$ sudo -u gitlab cp config/database.yml.mysql config/database.yml $ sudo -u gitlab -H nano -w config/database.yml

 

6.4 Install Gems (Gentoo way)

Gentoo has a problem with charlock_holmes due to its LDFLAGS.
In order to fix it, you have to edit rbconfig.rb:

$ nano -w /usr/lib/ruby/1.9.1/x86_64-linux/rbconfig.rb

Find the line below (hint: search for “undefined”, it will be the 1st result) and remove the last ldflag:

CONFIG["LDFLAGS"] = "-L. -Wl,-O1 -Wl,--as-needed -rdynamic -Wl,-export-dynamic -Wl,--no-undefined"

Change to:

CONFIG["LDFLAGS"] = "-L. -Wl,-O1 -Wl,--as-needed -rdynamic -Wl,-export-dynamic"

Now you can install charlock_holmes and gitlab:

$ gem install charlock_holmes $ sudo -u gitlab -H bundle install --deployment --without development test postgres

If you want, revert the changes you made to rbconfig.rb.

 

6.5 Configure Git (Gitlab way)

$ sudo -u gitlab -H git config --global user.name "GitLab" $ sudo -u gitlab -H git config --global user.email "gitlab@localhost"

 

6.6 Setup GitLab Hooks (Gitlab way)

$ sudo cp ./lib/hooks/post-receive /home/git/.gitolite/hooks/common/post-receive $ sudo chown git:git /home/git/.gitolite/hooks/common/post-receive

 

6.7 Initialise Database and Activate Advanced Features (Gentoo way)

If you are on hardened Gentoo using PAX, you have to adjust your ruby by disabling mprotect for it:

$ paxctl -m /usr/bin/ruby19

Start redis and mysql if you haven’t already, and add them to the default runlevel.

$ eselect rc start mysql $ eselect rc start redis $ eselect rc add mysql default $ eselect rc add redis default

Proceed to setup Gitlab:

$ sudo -u gitlab -H bundle exec rake gitlab:app:setup RAILS_ENV=production

 

6.8 Check Application Status (Gitlab way with a note)

The normal check suggested by the official instructions, may fail:

$ sudo -u gitlab -H bundle exec rake gitlab:env:info RAILS_ENV=production

The thorough one should pass though, complaining only for Gitlab not running yet and missing the init script:

$ sudo -u gitlab -H bundle exec rake gitlab:check RAILS_ENV=production

 

6.9 Install Init Script (Gentoo way)

Fortunately a Gentoo init script already exists, so I didn’t have to write it from scratch.
Currently you can find it at this gist. In case this link stop working, google for gitlab gentoo init script and you surely get some results with updated versions of this script. ;)

Download the gist, extract the init script and move it to your init.d directory. You may need to adjust the filenames and/or urls below.

$ cd /tmp $ wget https://gist.github.com/2360407/download $ tar xf download $ mv gist2360407-aea491f595955282bdc0a259e9fccc8cae3e7274/gistfile1.sh /etc/init.d/gitlab $ rm -r gist2360407-aea491f595955282bdc0a259e9fccc8cae3e7274 $ chmod 755 /etc/init.d/gitlab

If you installed Gitlab on a different path, adjust your init script:

$ nano -w /etc/init.d/gitlab

Update: Yesterday (22/01/2013) Gitlab 4.1 came out which uses sidekiq instead of resque. I adjusted the init script and you can find it here: Gitlab 4.1 init script for Gentoo.

 

7. Apache

On my server I use Apache behind varnish. So, I will give you instructions for Apache.

Gitlab is a rails app. It runs on its own server (unicorn in our case) and Apache (or nginx) serve as a proxy.

By default, Gitlab (unicorn) listens on a socket. We want it to listen on a port. Set it below. Also you can set the worker processes to 1 in order to save some RAM. Generally, you don’t want to run Gitlab on a machine with less than 1GiB RAM, unless you only run Gitlab on it. With 1GiB though, your machine will be left with enough free RAM to work as a competent web server hosting many LAMP sites/apps.

$ sudo -u gitlab -H nano -w /home/gitlab/gitlab/config/unicorn.rb

Your Apache should have the proxy modules as well as the byrequests method. These can be enabled by the appropriate APACHE2_MODULES.

If you have to rebuild your Apache to get them, don’t forget to restart it, a reload won’t help.

Also do not forget to enable proxy and ssl (-D PROXY -D SSL -D SSL_DEFAULT_VHOST) for your Apache setup at /etc/conf.d/apache2.

Next you have to set up Apache’s virtual host for Gitlab.

$ nano -w /etc/apache2/vhosts.d/20-gitlab.conf

I hope you can adjust the file below to your setup. Notice that in my example I have set unicorn to listen on port 8080.

  ServerName gitlab.myserver.com
  ServerAlias www.gitlab.myserver.com
  DocumentRoot /home/gitlab/gitlab/public
  ErrorLog /var/log/apache2/gitlab.myserver.com_error_log
  CustomLog /var/log/apache2/gitlab.myserver.com_access_log combined

      BalancerMember http://127.0.0.1:8080

    AllowOverride All
    Options -MultiViews

  RewriteEngine on
  RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f
  RewriteRule ^/(.*)$ balancer://unicornservers%{REQUEST_URI} [P,QSA,L]

  ProxyPass /uploads !
  ProxyPass / balancer://unicornservers/
  ProxyPassReverse / balancer://unicornservers/
  ProxyPreserveHost on

      Order deny,allow
      Allow from all

  ServerName gitlab.myserver.com
  ServerAlias www.gitlab.myserver.com
  DocumentRoot /home/gitlab/gitlab/public
  ErrorLog /var/log/apache2/gitlab.myserver.com_error_log
  CustomLog /var/log/apache2/gitlab.myserver.com_access_log combined

      BalancerMember http://127.0.0.1:8080
      Header add X-Forwarded-Proto "https"

    AllowOverride All
    Options -MultiViews

  RewriteEngine on
  RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f
  RewriteRule ^/(.*)$ balancer://unicornservers%{REQUEST_URI} [P,QSA,L]

  ProxyPass /uploads !
  ProxyPass / balancer://unicornservers/
  ProxyPassReverse / balancer://unicornservers/
  ProxyPreserveHost on

      Order deny,allow
      Allow from all

  SSLEngine on
  SSLCertificateFile /etc/ssl/apache2/server.cert
  SSLCertificateKeyFile /etc/ssl/apache2/server.key

If you want to restrict Gitlab to ssl for enhanced security, add this to the plain http section of your vhost configuration and remove the proxy/rewrite lines:

RewriteEngine on
RewriteCond %{HTTPS} off
RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI}

You are almots ready to use Gitlab!

Start gitlab service and add it to the default runlevel. Also start or reload Apache:

$ eselect rc start gitlab $ eselect rc add gitlab default $ eselect rc reload apache2

Now open your browser and visit your new Gitlab installation.
Your credentials are admin@local.host / 5iveL!fe.
Congratulations!

I hope everything went ok and you can start using Gitlab right away.

Remember, Gitlab updates on 22th every month. To upgrade your installation visit Gitlab’s wiki and follow their instructions.

Update: Gitlab on a rolling distribution

When you install gems, you compile them for your system, tying them to your system’s libraries. As we know very well on Gentoo, sometimes an update to a library can brake a program that depends on it.
Same goes for Gitlab, only, because we install it for a local user instead system wide, revdep-rebuild can’t detect the breakage automatically (but it will detect it for packages installed with gem system wide at /usr/local).

Let’s try to fix this!

Condition: After an emerge -u you find Gitlab stopped working.

First make sure that there are broken files, in /usr/local and/or in /home/gitlab:

$ revdep-rebuild $ SEARCH_DIRS="/home/gitlab/" revdep-rebuild

If charlock_holmes or a system wide gem is broken, reinstall it:

$ gem install BROKEN_SYSTEM_WIDE_GEM $ sudo -u gitlab -H gem install charlock_holmes

If another user gem is broken, it is part of bundle. We have to remove bundle’s files, to force a rebuild in the next command:

$ rm -rf /home/gitlab/gitlab/vendor/bundle

Last, create a fresh bundle for Gitlab.

$ cd /home/gitlab/gitlab/ $ sudo -u gitlab -H bundle install --deployment --without development test postgres

That’s it. Now your Gitlab should once again work fine.
A final note; Gitlab may be the UI/Wiki/Emailer etc but it is not your git server. So while you try to fix Gitlab, your repositories still work fine.

Share

3 Comments

  1. gitlab 5.3 is on wiki.gentoo.org though i had problems updating the init script you’d written to launch puma rather than unicorn….

    (it doesnt matter to me, as i passenger through apache) 5.3 works well for me in testing. please update your scripts and update the wiki. i did most of the lifting. thanks for your writeup.

    satan

  2. Hi, I am sorry it took me so long to reply.

    I just upgraded myself to 5.3.

    The new init script, which is valid for both Gitlab 5.2 and 5.3 is here:
    https://gist.github.com/andmarios/5952489

    I created an account at the wiki and updated the gist link. We can include the code instead of the link if you prefer (no attribution needed :). Whatever you choose, I will try to keep at least this section up to date.
    It is a difficult article to maintain. When I wrote this post, there wasn’t gitlab-shell, there wasn’t puma, sideqik, etc and I passed through all these changes by upgrading of course instead of doing clean installs. :/

    Thanks for your work. I’ll place a link at the head of this post.

  3. no problem, i’ve been off partying for the last 9 days anyways. i’ve been building against fresh installs, and (primarily) virtual machines. aqemu stage 4, copy paste, and bam fresh system ready for me to destroy.

    i did the apache / passenger route, was trying nginx + passenger but it requires shotty non portage scripts to recompile nginx with passenger enabled. i probably shouldn’t of touched the wiki as its the ONLY route that i got to work. only reason i updated that was because of all the features dropped in.

    (& mysql so future lookers know whats working, tested, and verified working)

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>