16

Moving Files from one Git Repository to Another, Preserving History

Posted May 17th, 2011 in Development and tagged , , by Greg Bayer

If you use multiple git repositories, it’s only a matter of time until you’ll want to refactor some files from one project to another.  Today at Pulse we reached the point where it was time to split up a very large repository that was starting to be used for too many different sub-projects.

After reading some suggested approaches, I spent more time than I would have liked fighting with Git to actually make it happen. In the hopes of helping someone else avoid the same trouble, here’s the solution that ended up working best. The solution is primarily based on ebneter’s excellent question on Stack Overflow.

Another solution is Linus Torvald’s “The coolest merge, EVER!“ Unfortunately, his approach seems to require more manual fiddling than I would like and results in a repository with two roots. I don’t completely understand the implications of this, so I opted for something more like a standard merge.

Goal:

  • Move directory 1 from Git repository A to Git repository B.

Constraints:

  • Git repository A contains other directories that we don’t want to move.
  • We’d like to perserve the Git commit history for the directory we are moving.


Get files ready for the move:

Make a copy of repository A so you can mess with it without worrying about mistakes too much.  It’s also a good idea to delete the link to the original repository to avoid accidentally making any remote changes (line 3).  Line 4 is the critical step here.  It goes through your history and files, removing anything that is not in directory 1.  The result is the contents of directory 1 spewed out into to the base of repository A.  You probably want to import these files into repository B within a directory, so move them into one now (lines 5/6). Commit your changes and we’re ready to merge these files into the new repository.

git clone <git repository A url>
cd <git repository A directory>
git remote rm origin
git filter-branch --subdirectory-filter <directory 1> -- --all
mkdir <directory 1>
mv * <directory 1>
git add .
git commit

Merge files into new repository:

Make a copy of repository B if you don’t have one already.  On line 3, you’ll create a remote connection to repository A as a branch in repository B.  Then simply pull from this branch (containing only the directory you want to move) into repository B.  The pull copies both files and history.  Note: You can use a merge instead of a pull, but pull worked better for me. Finally, you probably want to clean up a bit by removing the remote connection to repository A. Commit and you’re all set.

git clone <git repository B url>
cd <git repository B directory>
git remote add repo-A-branch <git repository A directory>
git pull repo-A-branch master
git remote rm repo-A-branch

Update: Removed final commit thanks to Von’s comment.

VN:F [1.9.5_1105]
Rating: 4.2/5 (13 votes cast)
Moving Files from one Git Repository to Another, Preserving History, 4.2 out of 5 based on 13 ratings
  • http://buildengineer.org/ Kate Ebneter

    Very nice writeup, and I’m happy to see that my question/answer was helpful!

    VA:F [1.9.5_1105]
    Rating: 5.0/5 (2 votes cast)
  • Von

    Just tried this on a couple quickly whipped up repos and it worked well. Only comment was there was nothing to commit after the pull, so I’m not sure you need the last two steps.

    VA:F [1.9.5_1105]
    Rating: 0.0/5 (0 votes cast)
  • http://GBayer.com Greg Bayer

    Great catch! The commit is not required with the pull approach. Will update accordingly.

    VN:F [1.9.5_1105]
    Rating: 0.0/5 (0 votes cast)
  • Anonymous

    Thank you, for explaining this!. Your post will be a big help. I have wanted to do this for some time, but kept putting it off!

    VA:F [1.9.5_1105]
    Rating: 0.0/5 (0 votes cast)
  • http://www.facebook.com/joechiu Joseph Chiu

    hi, based on what I read on http://stackoverflow.com/questions/1365541/how-to-move-files-from-one-git-repo-to-another-not-a-clone-preserving-history, I wonder if your line 5 should be “mkdir -p <directory >” and line 6 should be “git mv * <directory 1>” ?

    VA:F [1.9.5_1105]
    Rating: 0.0/5 (0 votes cast)
  • http://GBayer.com Greg Bayer

    Thanks for the feedback. In this case I believe the results should be similar either way.

    1) “mkdir -p” is only required if the new directory is more than one level deep.

    2) There shouldn’t be much difference between “mv *” and “git mv *” in this case.

    VA:F [1.9.5_1105]
    Rating: 0.0/5 (0 votes cast)
  • Mguyre

    Need to add a git fetch between steps 4 and 5 for the new repository to retrieve the tags and history

    VA:F [1.9.5_1105]
    Rating: 0.0/5 (0 votes cast)
  • Adam Monsen

    Great post, thank you. One suggestion: change the title as follows: s/file/one directory/.

    VA:F [1.9.5_1105]
    Rating: 0.0/5 (0 votes cast)
  • http://adammonsen.com Adam Monsen

    Great post, thank you. One suggestion: change the title as follows: s/Files/one directory/.

    VA:F [1.9.5_1105]
    Rating: 0.0/5 (0 votes cast)
  • http://adammonsen.com Adam Monsen

    Sorry, I meant: change “Files” to “one directory” or “a directory”.

    VA:F [1.9.5_1105]
    Rating: 0.0/5 (0 votes cast)
  • http://GBayer.com Greg Bayer

    Thanks for the suggestion. You can actually use this approach to move an arbitrary set of files by first moving them into a temporary directory. Because of this, the current title seems to be appropriate and more general.

    VA:F [1.9.5_1105]
    Rating: 0.0/5 (0 votes cast)
  • Navitf

    Hi, But when files are moved into a temporary directory, the command: “git filter-branch –subdirectory-filter” extract history that is relevant only to the temporary directory and thus real history logs are not preserved. Any idea how to overcome this?

    VA:F [1.9.5_1105]
    Rating: 0.0/5 (0 votes cast)
  • http://GBayer.com Greg Bayer

    Thats an interesting point. This wasn’t a problem in my case, so I haven’t looked into it. Maybe another reader can suggest a solution?

    VA:F [1.9.5_1105]
    Rating: 0.0/5 (0 votes cast)
  • 123456

    I get an error message when I run the get filter-branch command:
    $ git filter-branch –subdirectory-filter mt — –all
    C:Program Files (x86)Git/libexec/git-core/git-filter-branch: line 289: /libexe
    c/git-core/git: Bad file number
    Could not get the commits

    In typical git fashion, the error message is incomprehensible to me. Any idea what’s going wrong?

    VA:F [1.9.5_1105]
    Rating: 1.0/5 (1 vote cast)
  • http://GBayer.com Greg Bayer

    I think that means git can’t access one of your files. Based on a few posts I see on stackoverflow.com, this could be caused by a bad network connection or proxy configuration.

    VA:F [1.9.5_1105]
    Rating: 0.0/5 (0 votes cast)
  • 123456

    Don’t understand how this could be the case. I cloned the repo to local disk.. and to my understanding, “git remote rm origin” severs the link between my local repo and the remote one.. so I don’t see where networks/proxies would enter into it.

    VA:F [1.9.5_1105]
    Rating: 1.0/5 (1 vote cast)