ClearCase Globally, Git Locally

Thanks
ClearCase deserves all the credit for encouraging me to learn and love Git. Branching and merging is as fast and painless as listing changesets between any arbitrary branch or point in time. Did I say fast? No longer bound by ClearCase’s dictations and laborious linear progression, one can work off line, rollback, and experiment in multiple branches, travel into the past, and explore limitless parallel dimensions. While inspired by some other solutions, I believe what follows is the cleanest imposition of a Git repository upon a ClearCase snapshot view.

Summary
The following setup, namely a combined snapshot cloned locally, allows Git to track a ClearCase snapshot view without external functions (such as rsync), minimizing hijacks, untracked files, and encourages somewhat standard workflows with both ClearCase UCM and Git, without putting any limits on what can be done with a Git repository. This recipe assumes that the reader is proficient with the Unix/Cygwin shell, Git, and ClearCase. In short, we will:

  • Initialize a Git repository upon an existing pristine ClearCase snapshot view
  • Clone the snapshot as a Git repository
  • Track the master branch of the clone from the snapshot
  • Perform all ClearCase rebases, updates, checkins, and deliveries only in the snapshot
  • Work in branches of the Git clone
  • Pull between the master branches of the snapshot and clone repositories

Getting started
It is easiest to start with a fresh ClearCase snapshot view (what we’ll call snapshot) of which we’ll modify the .git/info/exclude file (see below) to hide some of the ClearCase plumbing from Git. Then we add all the tracked content to a newly initialized git repository (which is one and the same as the ClearCase snapshot view), clone it, and remote track the clone’s master branch. Now the clone (by default) and snapshot are tracking each other:

 $ cd snapshot
 $ git init
 $ cat >> .git/info/exclude
.gitignore
view.dat
lost+found/
 $ git add .
 $ git commit -m init
 $ git clone -o snapshot . ../clone
 $ git remote add -t master -m master clone ../clone

(The last step, ‘git remote add’, when last tested does not seem to work. It’s not necessary, but if we could get it to work, it could be cool.)

We will want to keep the snapshot pristine (see below). It should only be used as a staging area between upstream rebases and checkins, and downstream pulls. As far as possible, all development, branches, and merging should occur in the work clone. The snapshot could just as easily be used by multiple people, say a team working on a common project. In that case, I might recommend another bare clone. But here, we’ll assume all Git repositories (including the overlapping ClearCase snapshot) are locally used by one developer.

Git ignore
There are multiple ways to hide files from Git’s tracking view including .gitignore files scattered anywhere within the directory structure. This may very well be appropriate within the clone development branches to hide build artifacts, temporary files, etc. There, in the clone, you should consider whether you want to share the .gitignore files (before committing them) or add .gitignore to the root .gitignore file, thus ignoring itself.

In the snapshot, however, I’ve chosen to use the .git/info/exclude file instead because it is applied to the entire repository and is already hidden from Git’s tree. The snapshot has very different tracking requirements. We’ll want to filter out all of the Git and ClearCase plumbing such as view.dat, .vws, and any .gitignore files that try to come upstream from the clone. However, we generally want to be aware of everything that passes through our snapshot. To that end, we want to ignore very little and consider all untracked files. A file untracked by one system indicates a new addition in the other or a deletion the other doesn’t yet know about. No files should ever be untracked by both ClearCase and Git in the snapshot.

ClearCase update and rebase
I always run “Find modified files” from the ClearCase explorer before rebasing or delivering to ClearCase. Checkin, undo hijacks, etc, as appropriate to preserve a pristine snapshot. We’ll need to add (or remove) upstream changes after updating or rebasing the snapshot. Modified files can be easily added to the Git index on commit (with the -a flag).

# cd to snapshot
# rebase or update from ClearCase upstream
 $ git status
 $ git add (/some/files)
 $ git rm (/old/files)
# repeat above until ClearCase changeset is in the index (no untracked files)
 $ git commit -c "some comment"

Downstream development
Assuming we’ve been developing in one or many branches within the clone, eventually some changes are bound to emerge interesting and stable enough to share with others. We’ll use the master branch of clone to stage our merges before delivering a pretty package upstream. First, we’ll need to be in sync with the snapshot. If snapshot is not pristine, we need to get it to that state. If there are commits in snapshot not found in the clone’s master, we should pull (or rebase).

After clone’s master is equal to snapshot or contains a strict superset of changes, we can stage our changes in the master branch of clone. How we do that in the sidestream branches is completely up to you: rebase, pull, merge, squash, octopus, rebase -i. We commit a new feature into its own branch ready to be merged and pulled upstream. Our workflow might look something like this:

 $ cd ../clone
 $ diff -r . ../snapshot
(nothing)
 $ git checkout feature
 $ git rebase master
# test, work, test
 $ git checkout master
 $ git pull feature
 $ git branch -d feature

Checkin to ClearCase
In the snapshot we’ll pull from clone’s master and deliver the changes upstream. We may have to manually add, remove, and checkout/in our changes to ClearCase. To help, we can tag before pulling and display the file names as a difference along with the status (new, deleted, modified).

 $ cd ../snapshot
 $ git tag before
 $ git pull ../clone
 $ git diff --name-status before

(The ‘git remote add’ could have been handy here)

Automation
While the difference above could help us manually deliver upstream, the output of the last line above could very well be used in a script to deliver to ClearCase. Though I imagine procedures differ from environment to environment. I have not automated the ClearCase delivery myself, but here is a rough sketch:

 $ git diff --name-status before > diff_before
 $ grep ^A diff_before | sed "s/^../cleartool mkelem /"
 $ grep ^M diff_before | sed "s/^../cleartool cc /"
 $ grep ^D diff_before | sed "s/^../cleartool rmname /"
 $ grep ^[ADM] diff_before | sed "s/^../cleartool ci /"

Pristine
Similarly, the pristine state could be checked with ‘git status’ and the ClearCase explorer. However, I’ve found the following commands helpful:

 $ find . -type f -writable | grep -v lost+found | grep -v view.dat
 $ find . -type f -name *keep
 $ find . -type d -name *unloaded
 $ git diff --name-only HEAD
 $ git ls-files --others | grep -v .git | grep -v lost+found | grep -v view.dat
 $ cleartool ls -recurse -view -short | grep -v lost+found
 $ cleartool lsco -me -recurse -short
 $ cleartool ls -recurse | grep "\[hijacked\]"

Or a script which simplifies the above. This may evolve into a full git-clearcase tool if it proves useful:

 $ pristine
Checking writable...    OK
Checking artifacts...   OK
Checking Git status...  OK
Checking CC Untrack...  OK
Checking CC Checkouts...OK
Checking CC Hijacks...  OK

$ pristine --help
usage: pristine [-[waguch]] [ dir... ]

   Flags: Check directories for...
       -w writable files (possibly hijacked)
       -a artifacts such as *.keep and *.unloaded
       -g Git status including untracked files
       -u ClearCase untracked files and directories
       -c ClearCase checked out files and directories
       -h ClearCase hijacked files and directories

   Note:
       The flags above are ordered considering
       speed and likelihood of failure (-w) to the
       slower operations (-ch).

       The ClearCase checks may be slower than other
       checks. -w is a reasonable substitute for -h
       although not technically the same (a file may
       be readonly and still hijacked)

Happy coding.