Publishing git repositories

Different options for publishing git repositories

Publishing / Hosting Git repositories

There are a number of options for ways you can publish your git repository. We’ll outline a couple of them here, and discuss their various advantages and disadvantages.

DIY

This method assumes that you have a shell account somewhere, with git packages installed (you need 1.5 or later). You need to be able to run the git-daemon or a web server, preferably both. To publish over HTTP is slower, but is usable by people behind restrictive firewalls, the git protocol is much more efficient, but requires that you can open that port. If you can do both, its best. People should prefer the git protocol over http for a number of reasons, besides the efficiency one, if you pack the upstream repository, then the http users will have to re-download all of the objects again.

The first thing you need to do is to create a bare git repository. A bare git repository is a repository that only contains the git database, and no working copy. The reason why you want to do this is because you should never push into a repository that contains changes in the working copy, so creating a bare repository creates a repository with no working copy, thus ensuring this doesn’t happen.

On your server, where you will be publishing the git repository, do this (replace the repository name with what you want):

$ mkdir -p /srv/git/repository.git
$ cd /srv/git/repository.git
$ git --bare init 
$ touch git-daemon-export-ok
$ echo "description of project" > description
$ cp hooks/post-update.sample hooks/post-update
$ chmod +x hooks/post-update
$ git update-server-info

The final chmod ensures that the index is updated with every push. This hook gets executed whenever something is pushed to the repository, it runs the command ‘git-update-server-info’ which updates some files that makes the HTTP retrieval will work.

Then have your local repository to push all your branches to the remote repository:

$ git push --all git+ssh://remote.location/srv/git/repository.git

Then setup your local repository to push and pull from the remote:

$ git remote add origin git+ssh://remote.location/git/repository.git
$ git config branch.master.remote origin
$ git config branch.master.merge refs/heads/master
$ git fetch
$ git merge master

Another way to do this is to just do a scp -r of your local repository’s .git directory to the remote server and then clone that repository locally:

$ scp -r .git remote.location:/git/repository.git

Then go to somewhere else on your local machine and clone that remote repository:

$ cd somewhereelse
$ git clone git+ssh://remote.location/git/repository.git

Now you need to setup the protocols for people who do not have ssh access to your machine to access your repository.

First for HTTP you will need to make your /srv/git directory available through your web server’s configuration, you can do that by symlinking /srv/git to /var/www as one way to do it. Once that is setup, people can clone your repository over http by doing:

$ git pull http://remote.location/git/repository.git

To use the git protocol, you need to have the ‘git-daemon’ running. You configure git-daemon to have a git repository root directory where you are storing things, it will make public all the git repositories underneath that directory root. Those public repositories must be exported, which just means that they have the file ‘git-daemon-export-ok’ in the root of the repository.

To run the git-daemon on startup, put the following in /etc/init.d:

codetitle. /etc/init.d/git-daemon

#!/bin/sh
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
NAME=git-daemon
PIDFILE=/var/run/$NAME.pid
DESC="the git daemon"
DAEMON=/usr/bin/git-daemon
DAEMON_OPTS=" --syslog --detach --pid-file=$PIDFILE --user=git --group=nogroup --user-path=public_git --reuseaddr"

test -x $DAEMON || exit 0

[ -r /etc/default/git-daemon ] && . /etc/default/git-daemon

. /lib/lsb/init-functions

start_git() {
	start-stop-daemon --start --quiet --pidfile $PIDFILE \
		--startas $DAEMON -- $DAEMON_OPTS
}

stop_git() {
	start-stop-daemon --stop --quiet --pidfile $PIDFILE
	rm -f $PIDFILE
}

status_git() {
	start-stop-daemon --stop --test --quiet --pidfile $PIDFILE >/dev/null 2>&1
}

case "$1" in
  start)
	log_begin_msg "Starting $DESC"
	start_git
	log_end_msg 0
	;;
  stop)
	log_begin_msg "Stopping $DESC"
	stop_git
	log_end_msg 0
	;;
  status)
	log_begin_msg "Testing $DESC: "
	if status_git
	then
		log_success_msg "Running"
		exit 0
	else
		log_failure_msg "Not running"
		exit 1
	fi
	;;
  restart|force-reload)
	log_begin_msg "Restarting $DESC"
	stop_git
	sleep 1
	start_git
	log_end_msg 0
	;;
  *)
	echo "Usage: $0 {start|stop|restart|force-reload|status}" >&2
	exit 1
	;;
esac

exit 0

# chmod a+x /etc/init.d/git-daemon
# update-rc.d git-daemon defaults

Now people can retrieve via the git protocol:

$ git pull git://remote.location/git/repository.git

Once you are setup, you don’t really need to do anything on the remote server to manage your repository as you can just push and pull from your local repositories.

Gitosis

Gitosis is similar to the DIY method, but it handles some of the work for you, and is more geared towards multiple users having access to the repository. It maintains multiple repositories under one user account, using ssh keys for identification. The users do not have to have shell access to the server, but instead talk to one shared git account that keeps them from doing anything but the git commands.

Initialize the gitosis-admin repository

After you get gitosis installed, you need to initialize the gitosis-admin repository. Take your ssh public key and do the following (replacing SSH_KEY.pub with your key file):

sudo -H -u gitosis gitosis-init < SSH_KEY.pub

Now all you have to do is to clone the gitosis-admin repository to set things up:

$ git clone gitosis@localhost:gitosis-admin.git

You can now add new repositories and give people access to those repositories by simply adding their key to this repository, and changing the gitosis.conf and then pushing those changes back to the repository.

Configuring the git daemon

If you are using gitosis, then you probably will want to also run git-daemon with the base-path set to /srv/gitosis/repositories rather than /srv/git by changing the init.d example shown above as follows:

DAEMON_OPTS="--base-path=/srv/gitosis/repositories --syslog --detach --pid-file=$PIDFILE --user=git --group=nogroup --user-path=public_git --reuseaddr"

Or you can try this approach to gitosis (and gitweb and git daemon) on debian.

Configuring access

In the gitosis-admin repository, you will have a configuration file and a directory called ‘keydir’. Place in this directory the ssh public key files that are associated with the different users who you wish to give access to different repositories. These files are named after the username in the gitosis.conf file.

First of all, you need to have the person you are providing access to a repository to provide you with their SSH public key material. They should have something in their $HOME/.ssh directory with a .pub at the end of the file. If they do not, or if they do not even have a $HOME/.ssh directory, then they need to setup SSH public-key authentication. This is a bit out of the scope of this document, especially since it is different for different operating systems, but typically this is going to involve running ssh-keygen to create the files, and then ssh-add to add those to their agent. If someone has a good reference to to a HOWTO on this, I would love to add it here.

Ask your user to send your their $HOME/.ssh/id_rsa.pub, preferably as an attachment in an email that they have signed with their OpenPGP key so you have some proper identity verification. Beg them not to send you the $HOME/.ssh/id_rsa file! If they do accidentally send you the id_rsa file, then they have sent you their private key. This is bad, and they need to start over from the beginning, they need to remove those keys and regenerate them. This is analogous to them sending you their OpenPGP private key, its a no-no.

If the SSH public key you receive was pasted into IRC, or into an email, you will need to format it so it is all on one line. An improperly formed file will result in access not working and no useful information why not. This is why it is best if they can just simply attach the file in an email.

Once you have this file, you can simply place it in the gitosis-admin keydir directory as ‘username.pub’ where username is the username that you will use in a moment in the gitosis.conf file.

Once you have added it there, you can add it to the git repository:

$ git add keydir/micah.pub

Then commit it:

$ git commit -m 'added key for micah'

Then finally push that change:

$ git push

Note: you might need to pull first to get the latest repository changes.

Configuring repositories

This gitosis.conf file has two admins configured. Those users, and their associated key’s in the keydir are the people who can make changes to the gitosis repository itself.

Then there is a group with a handful of members configured and a repository that is configured to allow that group access, these are the people who can push changes to that repository:

[gitosis]

[group gitosis-admin]
writable = gitosis-admin
members = micah elijah

[group crabgrass]
members = micah abie elijah daniel

[repo crabgrass]
writable = crabgrass
daemon = yes

Setting up repositories

Lets say you have configured the above configuration for gitosis, you have committed it and then pushed it, now you can place a repository under gitosis’ control, with the above access configuration.

Pushing existing repositories to gitosis

You already have a git repository, but you want to put it under gitosis control. All you need to do is the following.

In the already existing git repository, add a remote for the gitosis repo. This is the repository name that is configured in the gitosis.conf file, with a .git added to the end. In the above configuration file, we have repo crabgrass, so we add a remote pointing at the git server, using the gitosis user, and the crabgrass.git repository:

$ git remote add labs gitosis@labs.riseup.net:crabgrass.git

The ‘labs’ in the above is just the local name for the remote that we added, this can be anything you wish.

Now you have to push to that remote the branch that you wish to have in gitosis:

$ git push labs master:refs/heads/master

This pushes to the ‘labs’ remote that we just created, the head of the master repository. You will only be able to push to this remote, if your ssh public key has been committed to the gitosis repository, and the gitosis.conf has a repository configured as detailed above.

Now to make sure things are all good, you just need to clone this new repository somewhere:

$ git clone gitosis@labs.riseup.net:crabgrass.git

Typically I will switch to this newly cloned repository as the one I am working with, and discard the previous one.

Creating new repositories

If you are creating a repository from scratch, you would do this instead.

First initialize the git repository, as you would normally:

$ mkdir myproject
$ cd myproject
$ git init

Then add a remote, as we did in the previous example:

$ git remote add labs gitosis@labs.riseup.net:myproject.git

Do some work, git add and commit files, as you would normally…. then when you are ready you can push your work to the remote as follows:

$ git push labs master:refs/heads/master

Then clone this repository somewhere to make sure it works as expected and has the files that you expect. Typically, I will discard the original repository, and use the just cloned repository from there on out.

As you can see, this process is exactly the same as the process outlined for pushing an existing repository to gitosis, with the only change being that we had to first create a new git repository.

Hosted

There are a couple hosted git repository options. The two that are most popular are github and repo.or.cz.

People really like github, mostly because it has a slick UI interface, and the networking aspects of code forking. One of the core features of git is that it is all about forking and managing changesets between various forks. Github adds a piece to that by giving you an idea of who forked your code and a visual of the changes they have made. The hype about github is that your code develops its own social network. I’m not really sure why this is so great, as I haven’t yet figured out a use for this information, people say that it encourages cross-pollination of code, but I dont see how. The network effect does have its advantages (lots of projects are hosted at github, so its a nice central location that everyone knows how to use). Its what the ruby community uses, so by default there is an amorphous, overhyped cloud hanging over it. Github is free, but only up to a certain point. You can’t have private repositories with github, unless you pay. I’ve seen people complain about github’s reliability, but they are relatively beta, so what can you say.

repo.or.cz is less slick, but also less hype involved. Its basically a public gitweb install, that automatically sets up repositories for you. They make it really easy and quick to get an account, add your ssh key and create a repository. They also provide a mirror service which will mirror to repo.or.cz any git repository that is published elsewhere. When you do this, they will then provide a pull and gitweb access for that project.
Their code is free software, and its free to use their service.