[Setting up a Subversion Server on OS X] at tagSwap.net: Open Source Lasso Code

[Setting up a Subversion Server on OS X]

Greg Willits, April 27, 2006

I won't being going into what Subversion ("svn") is, or what it does. I'll assume you've already heard about it, want to try it, and just want some shortcut guidance on getting it running on a server. Perhaps some folks can post some comments with their favorite web pages which are good for newbies that talk about how to use svn from the client side.

The method I'll be describing uses the built-in svnserve daemon. This is all I've ever used. I've never bothered to set it up with apache or under webdav. If you want to do that, then these instructions won't help you much.

You will need BBEdit or TextWrangler or some other text editor that will authenticate access to system files. You will be using Terminal, but I'll try to hold your hand as best I can.

Install Subversion

As of this writing there's no binaries for OS X from tigris itself. However, there are a few installers out there provided by some folks. If you want to build your own, you can give it a shot with help from http://hivelogic.com/articles/2006/04/19/svn_on_os_x.

I have used these two installers that have worked just fine.

http://www.metissian.com/projects/macosx/subversion/
http://www.codingmonkeys.de/mbo/articles/tag/subversion

Most people should probably use the Metissian package. It contains SSL and some other libraries needed by Eclipse. The CodingMonkeys version has neither of those, but it installs in a much cleaner manner. The Messian one uses a lot of linked libraries and installs boatloads of files. The Monkeys version has everything built-in so there are far fewer files involved. So if you know you'll never be using eclipse or need SSL access to a subversion repository, the CodingMonkeys one is a nice package. Does it really matter? If you don't kow if it matters, then it doesn't. If you have some specific reason for liking the Monkeys one, then go for it. However, which ever one you start with, you probably want to stick with that one for all future upgrades.

Oh, to use Subversion 1.3.1 as packaged by the guys above, you'll have to be running OS X 10.4. If you're using 10.3, then you'll want the 1.3.0 installers. I used the Messian 1.3.0 package on 10.3.9 and it seems to run just fine.

Regardless of which one you pick, it's a pretty straight forward install. Typical OS X installer--just click all the usual buttons 'till yer done. As expected, everything will be in /usr/local/ when all is done, but there's no /svn/ folder. The binaries just go straight in /usr/local/bin/.

While we're here, now would be a good time to open /etc/profile/ and make sure that /usr/local/bin is in the PATH list.

The default PATH looks something like this:

PATH="/bin:/sbin:/usr/bin:/usr/sbin"

You might also have a path for MySQL there too if you're using mysql. As you can see, paths are separated by colons. So assuming we add /usr/local/bin to the default string, we end up with this:

PATH="/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin"

If you have a custom install of php or python or something else to override Apple's built in one, then you may already have /usr/local/bin at the beginning of the PATH string. That's ok too.

Securing the SVN Files and Server

This next step is something you find referred to a lot in all svn documentation and books, but you won't see explicitly explained because it is assumed you already know how to do it because of course you're a unix OS server admin jockey. Right.

The easy way to use svnserve is to just start it at this point. The problem is, it will run with all the privileges of the admin user you are likely currently logged into OS X as. Generally, that is not a good idea. Apache doesn't run that way, MySQL doesn't run that way, and we don't want subversion running that way either.

So, for security purposes, we want the subversion server to run under it's own special user name which is contained to run only the server commands and has access to only its own repository files. To do this we have to create a new OS X user and a new group. That's about as much explanation as you'll find anywhere, so below is my attempt to feed you the details.

If you're using OS X Server, you'll be happy. If you're not, this is where you pay for not ponying up the $ for Server. :-)

Add User & Group in OS X Server

If you're using OS X Server, you can do this with Workgroup Manager which makes it pretty easy. First, create a new user called svnsrvr. (Be sure to keep track of that password!) Next, create a new Group called svn. Add the user svnsrvr to the group, and also add the user you typically log in as to maintain the server.

Add User & Group in OS X Client

If you're using OS X client, then creating a user and group has to be done with NetInfo or Terminal. The steps below has worked for me using Terminal. Note that we are not creating a normal user that would have a home directory and all that. This is a very basic user modeled after what MySQL creates for itself.

The first thing to determine is what user ID ("uid") to assign the new user. Generally we want something between 101 and 499. Apple reserves 0-100, and users with UIDs higher than 499 are shown in the Login dialog window. Chances are UID 401 is available, but here's how you find out. In Terminal, issue this command, and just make sure that 401 is not used.

nireport . /users name uid

Open NetInfo, authenticate yourself, and click on the users entry in the second column. Click the New button (or use command-N). This will create an entry, and in the bottom panel there will be a property called name with the value new_directory. Change the value to svnsrvr.

Now, we'll enter several property and value pairs. Use shift-command-N to start a new line, and enter each of these pairs (without the = in them):

uid = 401 
expire = 0 
home = /var/empty 
gid = 401 
shell = /bin/bash 
realname = Subversion svnserve 
change = 0

Once you have entered those pairs (order does not matter), then command-S to save the new user.

To add a password to this user, go back to Terminal and enter the following:

sudo passwd svnsrvr

Pay close attention to how you're prompted here. If you see Password: then sudo is asking for your admin password first. If you see New password: then you're being asked for the svnsrvr user's password. You'll be asked to type it in twice.

Now you can go back to NetInfo. Click on another user, then back on svnsrvr to see the new pairs added for the password.

Next we have to create a new group called svn. Again, we have to check for existing group IDs ("gid"), so we use

nireport . /groups name gid

and verify that 101 is available. If not, use the next available higher number.

Back in NetInfo, click on groups in the second column of NetInfo, and create a new entry (command-N). Like we did above, change the new_directory to svn. Then, add these pairs:

gid = 101 
passwd = * 
users = svnsrvr

Press command-S to save this group. We also want to add the user that you normally log into the machine as to use the GUI to admin the system. To do that, make sure the users property is highlighted and press option-command-i. This will add a second value in which you can enter the user name

Limiting the svnsrvr user via sudoers

So, now with a user defined we need to limit its ability to execute only svn related commands.

We do that by editing the file /etc/sudoers. In Finder, use shift-command-G, and enter /etc/ in the dialog prompt. Locate and open the file sudoers (with BBEdit or equivalent).

Despite the dire warning at the top of the file that the file "MUST be edited with the 'visudo' command as root," I have always used BBEdit, and it works just fine. In theory in a busy system with multiple admins, it is possible two people might try to edit sudoers at the same time. As I understand it, that's what visudo is for.

Somewhere near the top of the file, you'll see a line like this:

# Defaults specification

Below that line, add a line like this:

Defaults:svnsrvr  !lecture

Now, scroll to the end of the file, add a blank line, and add this and be sure there is an empty line after it.

svnsrvr  ALL=/usr/local/bin/svnserve,/usr/local/bin/svnadmin,/usr/local/bin/svn

Make sure there is a blank line after that line above.

What this does is tell OS X that the user svnsrvr is allowed to run the svnserve daemon and the svnadmin tool and that's it. It cannot perform any other system tasks. The purpose behind this is to limit what can be done as the svnsrvr user. If there's a security hole in subversion, or someone somehow gets control of your machine through subversion, then as the svnsrvr user, not much can be done.

We now have a new user, a new group, and we've limited command access for that user to only svnserve.

Create Repositories

If you have already cruised the svn docs or web pages, you probably came across discussions about organizing one or multiple repositories. For web developers working on multiple projects, I recommend creating multiple independent repositories. It keeps each one separated so it is easy to assign people to specific projects, and in the event something does go wrong with the repository, at least it is only one project, and not all of them.

I put my repositories in a common folder /svnhome/ at the root of a drive. I usually don't use the startup drive just to spread out the disk activity, and to keep my repositories from being held hostage due to any startup drive catastrophe, but you can put it wherever you want. For the rest of these instructions, I'll be assuming your /svnhome/ is at the root of a drive called drive2 just to show what the commands would be to allow for this. You will have to adjust your paths accordingly.

So, in Finder, create a folder called /svnhome/. Then Get Info on that folder. Change the permissions so that Others has No Access. Change Group to be svn with Read Only access. Change the Owner to svnsrvr with Read & Write access.

Now we're ready to make one or more repositories. We want the files created for the repositories to be restricted to the svnsrvr user we created. The commands to create the repositories will be done in Terminal, and the easiest way to confine to the svnsrvr user is to go ahead and create them as that user. So, in Terminal do this:

sudo su svnsrvr

Then enter the password of your admin user if prompted by sudo. The Terminal prompt should change to indicate that the svnsrvr user is the current user.

So, let's create two repositories for fictional projectA and projectB. These could be separate clients, or if you're a captive developer, maybe one is the company's public web site, and one is the Sales intranet.

Pop back into Terminal, and do the following.

svnadmin create --fs-type fsfs /volumes/drive2/svnhome/projectA/

svnadmin create --fs-type fsfs /volumes/drive2/svnhome/projectB/

Each of these will create a folder in /svnhome/ and several folders within their respective project folders.

From this point on, we'll focus on one project, knowing you would follow the same steps for each new project. Of course, new project repositories can be created at any time by using the above command format.

(Note: One trouble I had at this point with the OS X Client system while I was verifying my steps, is that I had to reboot before I could get this step to work. If you get a command not found or permission denied repsonse to the svnadmin create commands, try rebooting. I wouldn't think it to be necessary, but it worked for me.)

By the way, it is very easy to experiment with subversion, then nuke all the experiments and start over once you decide how you really want to structure things. I found I needed to have a couple rounds before I found the naming scheme and structure I liked. So, don't worry about creating something and being stuck with it. For now, just play. Deleting a repository is as simple as deleting the top level folder such a /projectA/. There's no more magic to it than that.

Edit Repository Configs

In Finder, locate and open the file /svnhome/projectA/conf/svnserve.conf using BBEdit or whatever.

Uncomment this line:

# anon-access = read

And change it to

anon-access = none

This makes the SVN repository require a password for all access (reading and writing).

Next, uncomment this line:

# auth-access = write

This allows anyone with an SVN password to read and write to the project.

Uncomment this line:

# realm = My First Repository

And change it to something like this:

realm = projectA

This keeps the passwords from each project separated from each other. If you want to share them, you can do some reading in the subversion docs on how to set that up.

Next, open the /svnhome/projectA/conf/passwd file.

This one is pretty much self explanatory once you see the default comments in the file. Add users as needed. It will be easier to work with svn if the user's name is the same name of the user's account on their local machine. So, if John Smith's login to his local machine is johnnyboy, then use johnnyboy as the svn user name too instead of something like jsmith. It's not necessary, it's just easier to do the user authentication at the svn prompts. I encourage strong passwords of course.

This password system is the basic one (it was the only system available prior to 1.3). We've set it so that anonymous users can neither read nor write to the repository. We've also set it up so that users defined in the passwd file have access to the whole repository. In this case we allowed all passwd users to read and write any file in the project.

For more finite control over individual users and paths, look into the authz authentication file that is new as of version 1.3.

As a side note, svn passwords are not passed in the clear. Subversion uses a nifty system where upon a login request, the server sends the client a key, the user enters the password, but the client hashes the password with the key, then sends that hash back to the server. The server then compares that hash to performing the same hash server side.

Prepare to Import a Project

It's finally time to prep the repositories. Start by creating a new folder for the project (even if you already have one). In that folder, create three folders named /trunk/, /branches/, /tags/ so that you should have something like this:

/projectA/    
    /trunk/    
    /branches/    
    /tags/

Next, put all the files you want managed by subversion inside the /trunk/ folder in whatever organization structure you prefer. So, maybe you have something like this:

/projectA/  <--- I'm going to call this one the "original"    
    /trunk/       
        /Docs/          
            Company X RFP.doc          
            My Proposal.pages          
            data_export.txt       
        /Mstr Art/          
            layout (a).psd          
            layout (b).psd          
            logo.ai          
            logo_main.gif       
        /webSource/          
            _indexRedirect.lasso          
            _pageConfig.lgc          
            /_pbInit/          
            /_pbLibs/          
                error.lasso          
                index.laso          
            /home/          
            /site/    
    /branches/    
    /tags/

Import a Project to start a Repository

The next step is to import the original /projectA/ files into the repository /projectA/. In Terminal while still operating as the svnsrvr user, do this:

svn import /path/toMasterCopyOf/projectA/ file:///volumes/drive2/svnhome/projectA/ -m "init project"

As usual with Terminal just drag the original /projectA/ folder onto the terminal window to have it fill out the /path/toMasterCopyOf/projectA/ part.

We're using svn import to pull in that master copy we made above into the repository at /svnhome/projectA/. Every time the repository is changed (i.e. via import or commit), subversion will ask for a comment. This will usually pull up a CLI editor that you (nor I) have probably ever used before and have no idea how to get out of, so I highly recommend you remember our little buddy the -m. With -m we can add a short comment very easily (which is usually some description as to the changes in the version being added to the repository). Otherwise you'll have to figure out which editor subversion pulls up for you to enter the comment, and figure out how to tell it you're done. (I just close Terminal and start over on the import command if I forget to ue -m. Totally lame I know, but that's how it is.)

If the import succeeded a whole bunch of file listing lines will go flying by, and the last line will read Committed revision 1. If you don't see that line something when wrong. Subversion doesn't seem to like Mac OS custom icon files, and I've had to delete those to get imports to work. Because they're invisible files, I used Terminal's rm command to remove the file subversion complains about (which it will do, so at least you know where the file is that it doesn't like).

OK, so now we have the original of the project in the repository. When SVN imports the project files, it doesn't actually do anything to the original files. At this stage those original files are no longer going to be used. Technically, you can throw them away, but of course you probably want to keep them for a while just in case you find you need to repeat this step before you really get under way.

Start the SVN Server

Manually starting the server isn't too difficult, but we do want to start it as the svnsrvr user. Assuming you still have a Terminal window running as the svnsrvr user, in Terminal, do this:

svnserve -d -r /svnhome/

and then go ahead and exit.

If you need to start the server from scratch, do these steps:

su svnsrvr
(enter the password for svnsrvr when asked)
svnserve -d -r /svnhome/
exit

To verify that svnserve is running under user svnsrvr, do this:

ps aux | grep svn

You should see something like this:

svnsrvr 16178  0.0 -0.0   28172  524 ?? Ss   1:47AM 0:00.01 svnserve -d -r /svnhome/

Most of those numbers will be different, but what we wanted to verify what that svnserve was listed as running, and that the first column shows the user as svnsrvr.

OK, that wraps up all the server side of things. Now, it's time to get your development system set up.

Prepare a Working Copy

I develop on OS X, and I used to put all my web projects right in the ///Library/WebServer/Documents/ folder. It just made it easier to have the files where Apache could get at them, and where they'd be all in one place to make it easy to do backups, etc. Since I started using subversion though, I started a new organization. It really has nothing to do with subversion, it was just sort of a mental change point where I adopted another idea I'd been thinking of.

Now I put all my projects in ~/webProjects/. I suppose it could go in ~/documents/webProjects/ but I tend to use ~/documents/ for personal documents and my reference documents, and put all my project work in ~/webProjects/ just to make it easier to get to. Then in /WebServer/Documents/, I create a symbolic link to the source code folder in the trunk folder of the project. So, it coul be something like this:

ln -s /Users/greg/webProjects/projectA/trunk/webSource/ /Library/WebServer/Documents/projectA

So even though I have to create symlinks now, the main advantage to this is that I only have to monitor one backup script for my home folder instead of two.

Anyway, let's just assume my organization, and you can modify my path references to your setup. To create a working copy of a new project, open Terminal on your dev system and cd into where your projects are

cd ~/webProjects/

Next, use the svn checkout command like this:

svn checkout svn://x.x.x.x/projectA/
--or--
svn checkout svn://anyDomainThatMapsToThisIP.dom/projectA/

Notice there's no info to tell svn where to put the files. That's because svn puts them in the current directory which is why we had to cd before the checkout.

You should be prompted for a password. svn will assume your current login account name (which is why I suggested using that for the svn users names), and show that to you. Enter the password you entered in the passwd file.

This will create the /projectA/ folder for you inside ~/webProjects/ and inside /projectA/ will be the trunk folder and all your project files inside that.

If you had already created Apache vhosts for your project, you'll have to update the path to include the new /trunk/ folder location.

Workflow From Here On

Swell. So now what?

Well, now is when I should probably let the many svn references out there take over. IMO, it is worth buying a book to have it handy and read about best practices in development methodology. There's some pdf versions of the straight subversion docs of it as well like here (http://svnbook.red-bean.com/), and plenty of web pages that talk about how to use svn during project development.

A key methodology principle is to make functionally incremental changes to your source code, and commit each time you have a new build of the code. By "build" I mean a non-crashing state of operability. Maybe a whole site module isn't complete, but the roughouts of each page is done, and you can navigate without having a lasso error. You don't want to commit every time you break for a snack or a phone call, but don't wait for 6 days either. Develop in chunks than can run w/o errors, and commit your project to subversion each time you hit one of those milestones. Technically, this should be once or twice a day. But everyone probably has an opinion about that, so go find some :-)

Anyway, that's it for this article. I'll leave to another day, or better yet, to other folks to provide some lectures on how to use subversion from the project develeopment side.