Switching a local clone of a repo (& my custom repo tools) from Mercurial to git

Hardware and software discussion.
Locked
barbaz
Postaholic
Postaholic
Posts: 204
Joined: Mon Sep 15, 2014 12:55 am

Switching a local clone of a repo (& my custom repo tools) from Mercurial to git

Post by barbaz »

(continuing from viewtopic.php?p=91431#p91431 )

OK so setting up git v 2.8.1 is pretty straightforward. Cool, they fixed what had me tangled before! 8)
But working with repositories is proving tricky. (I'm using a clone of https://github.com/ryanbr/fanboy-adblock as a base repository to practice with)

1) In Mercurial, if I have local changesets at the head of a repo and new changes have been made on the server, when I hg pull there is an anonymous branch created out of my local changesets. In similar situation in git, there are no "anonymous branches" - my custom changesets are effectively gone if I switch revisions and don't grab the cset IDs :shock:
My concern there is, if I accidentally `git pull` when I should have `git fetch` (or otherwise switch revision), what is the actual risk of losing my changesets (via auto or manual `git gc` or otherwise) and what can be done about it (that is, *before* they're garbage-collected ;-) )?

2) How to search for author by exact match of username (ignoring email)?
`git log --author=bar` matches barbaz, foobarbaz, etc in entire string "username <email>", whereas `hg log -u bar` does not wildcarding.

3) What does this mean?
Warning: Running git merge with non-trivial uncommitted changes is
discouraged: while possible, it may leave you in a state that is hard
to back out of in the case of a conflict.
In Mercurial I've backed out of that state by doing `hg update -C -r <revision>` which I think is the equivalent of `git checkout --force <revision>`
Why wouldn't that work in git?
User avatar
LanikSJ
Site Owner
Site Owner
Posts: 1806
Joined: Thu Feb 15, 2007 7:44 am
Location: /dev/null

Post by LanikSJ »

1. Git will attempt to merge your changes and insert the code into the changed files. In my experience it works but still needs cleaning since it inserts text to tell you which parts were pulled from the head VS your local cached copy.

2. I've never had to do that I've been fine searching by wild card myself.

3. See #1 this can cause unintended consequences where some text is added in your merged files by Git. git checkout --force <revision> should work I've only had to do that once.
"If it ain't broke don't fix it."
barbaz
Postaholic
Postaholic
Posts: 204
Joined: Mon Sep 15, 2014 12:55 am

Post by barbaz »

Lanik wrote:1. Git will attempt to merge your changes and insert the code into the changed files. In my experience it works but still needs cleaning since it inserts text to tell you which parts were pulled from the head VS your local cached copy.
Sorry, I asked that question somewhat incoherent. If I have a repository with a HEAD (my changes) and a "master" branch (server changes), and the working directory and index both match the HEAD. How can I get back my changeset(s) on the HEAD if I (or a buggy script) unintentionally, say, git checkout some revision on the master branch (before creating a new branch or merging), and I don't have the changeset ID of the HEAD?
Lanik wrote:2. I've never had to do that I've been fine searching by wild card myself.

3. See #1 this can cause unintended consequences where some text is added in your merged files by Git. git checkout --force <revision> should work I've only had to do that once.
Thanks for these explanations, that helps a lot :-)
User avatar
LanikSJ
Site Owner
Site Owner
Posts: 1806
Joined: Thu Feb 15, 2007 7:44 am
Location: /dev/null

Post by LanikSJ »

barbaz wrote:Sorry, I asked that question somewhat incoherent. If I have a repository with a HEAD (my changes) and a "master" branch (server changes), and the working directory and index both match the HEAD. How can I get back my changeset(s) on the HEAD if I (or a buggy script) unintentionally, say, git checkout some revision on the master branch (before creating a new branch or merging), and I don't have the changeset ID of the HEAD?
Here's what happens:

Code: Select all

└❯ git pull
remote: Counting objects: 3, done.
remote: Compressing objects: 100% (2/2), done.
remote: Total 3 (delta 1), reused 0 (delta 0)
Unpacking objects: 100% (3/3), done.
From gitlab.com:lanik/mac-shell
   5c47bb3..16ce2c4  master     -> origin/master
Updating 5c47bb3..16ce2c4 
error: Your local changes to the following files would be overwritten by merge:
	README.md
Please, commit your changes or stash them before you can merge.
Aborting
Basically you have an option to overwrite your chances to "stash" them. Does that help?
"If it ain't broke don't fix it."
barbaz
Postaholic
Postaholic
Posts: 204
Joined: Mon Sep 15, 2014 12:55 am

Post by barbaz »

Good to know that git would just refuse the potentially destructive part of the operation. But I've also discovered that what people were calling commit deletion isn't actually deleting anything, just that it becomes only accessible by commit hash. :? Furthermore, on digging around in the .git directory I happened upon the hashes of the supposedly deleted commits - but idk how dependable that actually is.
Looks like the ONLY way to really delete commits is actually running

Code: Select all

git prune --expire now
which won't happen accidentally (even git gc doesn't do that even after telling it to by --prune option). So I'd have to screw up severely to lose any loose heads I want to keep.

Thanks for your help Lanik :biggrin: now it's hopefully just a matter of familiarity + adding git support in my repo tools...
barbaz
Postaholic
Postaholic
Posts: 204
Joined: Mon Sep 15, 2014 12:55 am

Post by barbaz »

Hitting some confusion trying to merging upstream changes to a local branch.
I'm trying to apply my local changes by creating my own branch, working from there and merge changes from master into my branch, but sticking with my branch the whole time. And with each merge I look at the diff to check if more changes are wanted on my side.

1) Apparently, when I last ran git fetch, master got split into 2 branches "master" and "origin/master", and the new changes went to the latter. I had (apparently incorrectly) understood from the man pages that with `git fetch` remote changes would be appended to master, and that if I ran `git pull` that it would also checkout the new HEAD of master (this is basically how Mercurial works and what I was assuming when I ported my batch-hg-pull script to git). Now I have this situation:

Code: Select all

$ git checkout master
Switched to branch 'master'
Your branch is behind 'origin/master' by 5 commits, and can be fast-forwarded. 
  (use "git pull" to update your local branch)
And git branch doesn't list origin/master :?
So what exactly does git pull do that git fetch doesn't?


2) As for actually merging, after unhelpful searches of the Internet I finally figured out to try

Code: Select all

git merge 'origin/master'
(and `git diff <mypriorcommithash>..HEAD` to see the diff resulting from the merge) and those seemed to work. But different people on the Internet all say different things, including that the merge method I used creates copies of all the merged commits from origin/master and so it's better to git rebase the local branch.
Am I making a mess of the repository with that merge method, or is it the right way to merge given that I want to see the resulting diff?
User avatar
LanikSJ
Site Owner
Site Owner
Posts: 1806
Joined: Thu Feb 15, 2007 7:44 am
Location: /dev/null

Post by LanikSJ »

barbaz wrote:And git branch doesn't list origin/master :?
I see just "master" when you run git branch.
barbaz wrote:So what exactly does git pull do that git fetch doesn't?
git fetch will tell you if your branch has any changes that you haven't merged yet. Git pull will do what git fetch does and merge any changes from origin into your branch.
barbaz wrote:Am I making a mess of the repository with that merge method, or is it the right way to merge given that I want to see the resulting diff?
You're on the right path so far. ;-)
"If it ain't broke don't fix it."
barbaz
Postaholic
Postaholic
Posts: 204
Joined: Mon Sep 15, 2014 12:55 am

Post by barbaz »

Thanks Lanik for the quick response.
Lanik wrote:git fetch will tell you if your branch has any changes that you haven't merged yet. Git pull will do what git fetch does and merge any changes from origin into your branch.
Sure, but what is "my branch" in this context?

Code: Select all

$ git branch
* local 
  master
That "master" is only changes from the server, "local" is where all my custom changes are.
User avatar
LanikSJ
Site Owner
Site Owner
Posts: 1806
Joined: Thu Feb 15, 2007 7:44 am
Location: /dev/null

Post by LanikSJ »

Your branch in the above case is called "local" probably not a good name if you ask me (but whatever works). ;-)
"If it ain't broke don't fix it."
barbaz
Postaholic
Postaholic
Posts: 204
Joined: Mon Sep 15, 2014 12:55 am

Post by barbaz »

So git pull would try to apply the new commits to whatever branch is currently checked out... so to sync master with 'origin/master', I would have to:

Code: Select all

git checkout master
git pull 
git checkout <whatever was checked out before>
Shouldn't be too hard to script.
Lanik wrote:probably not a good name if you ask me (but whatever works). ;-)
Bad because it's generally likely to be used as the name of a branch on the remote repo, or bad because git uses the name "local" for something internal relating to branches?
User avatar
LanikSJ
Site Owner
Site Owner
Posts: 1806
Joined: Thu Feb 15, 2007 7:44 am
Location: /dev/null

Post by LanikSJ »

There's nothing in Git that uses local internally that I'm aware off. Might seem confusing for folks who look at the repo to see branch called local. Standard practice is to put the feature you're working on as a name of the branch. For adding sites to a block lists you can use URL as the name of the branch. There's no real guidelines as to how or what to name your branches, you can call it "my ants foot" for all anyone cares just makes it easier for some (read me) to distinguish what branches are for. I'm too lazy to check out a branch to see what's in it, especially when there easily could be 2 to 3 dozen of them. :-D
"If it ain't broke don't fix it."
User avatar
LanikSJ
Site Owner
Site Owner
Posts: 1806
Joined: Thu Feb 15, 2007 7:44 am
Location: /dev/null

Post by LanikSJ »

Forgot to add.

When you do a git pull doesn't matter what branch you're on everything is sync'd and merged from all the other branches.

Example: say you're working on your "local" branch and someone commits something to master, then when you run git pull you will get all the updates to master without having to leave your branch. You can also git merge them if you're so inclined.

Here's a typical git pull from my job (names changed to protect the guilty):

Code: Select all

└❯ git pull 
remote: Counting objects: 208, done.
remote: Compressing objects: 100% (194/194), done.
remote: Total 208 (delta 102), reused 2 (delta 1), pack-reused 1
Receiving objects: 100% (208/208), 128.92 KiB | 0 bytes/s, done.
Resolving deltas: 100% (102/102), completed with 1 local objects.
From https://uber.secret.url/SomeOrg/some-repo
   6b109c6..ef56197  master     -> origin/master
   a7553e6..86ee4fb  development -> origin/development
 * [new branch]      somebranch1 -> origin/somebranch1
 * [new branch]      somebranch2 -> origin/somebranch2
 * [new branch]      somebranch3 -> origin/somebranch3
 * [new branch]      somebranch4 -> origin/somebranch4
 * [new branch]      somebranch5 -> origin/somebranch5
   02ca983..90f4706  some_existing_branch1 -> origin/some_existing_branch1
   5a2db81..ba1ecdd  some_existing_branch2 -> origin/some_existing_branch2
 * [new tag]         v2.2       -> v2.2
 * [new tag]         v2.1       -> v2.1
Updating 6b109c6..ef56197
Fast-forward
 directory1/somefile1.erb      | 59 +++++++++++++++++++++++++++++++++++++++++++++++++++--------
 directory1/somefile2.erb      |  2 +-
 directory2/somefile.erb        | 21 +++++++++++++--------
 directory3/somefile.erb        |  4 ++--
 4 files changed, 67 insertions(+), 19 deletions(-)
Until you check out the other branches is when the changes are merged so you're only seeing the merged changes from the master branch in my example above.
"If it ain't broke don't fix it."
barbaz
Postaholic
Postaholic
Posts: 204
Joined: Mon Sep 15, 2014 12:55 am

Post by barbaz »

fanboy added more changes, and I had backups, so this gave me the opportunity to actually try git pull (starting with branch local checked out). This is what happened:

Code: Select all

$ git pull
remote: Counting objects: 60, done.
remote: Compressing objects: 100% (60/60), done.
remote: Total 60 (delta 37), reused 0 (delta 0), pack-reused 0
Unpacking objects: 100% (60/60), done.
From https://github.com/ryanbr/fanboy-adblock 
   965927a..556837a  master     -> origin/master
There is no tracking information for the current branch.
Please specify which branch you want to merge with.
See git-pull(1) for details.

    git pull <remote> <branch>

If you wish to set tracking information for this branch you can do so with:

    git branch --set-upstream-to=origin/<branch> local

Tested to see if it worked anyway:

Code: Select all

$ git checkout master
Switched to branch 'master'
Your branch is behind 'origin/master' by 20 commits, and can be fast-forwarded.
  (use "git pull" to update your local branch)
Nope :-(
What have I got wrong? (Maybe I'd need to somehow tell git to just ignore branch local when doing git pull?)

Regarding branch names, nice tips, I'll keep that in mind for if I ever publish any git repositories or push changes somewhere. This branch I'm calling "local" is where I'm putting all my changes which (since I wisely don't have write access on the server-side repo) obviously won't be pushed and that anyway fanboy would definitely not want in his repository :mrgreen:
User avatar
LanikSJ
Site Owner
Site Owner
Posts: 1806
Joined: Thu Feb 15, 2007 7:44 am
Location: /dev/null

Post by LanikSJ »

Since your "local" branch isn't uploaded to origin or in this case github.com you'll get that error since it's not tracked. You'll need to do a git checkout origin/master and then git pull.
"If it ain't broke don't fix it."
barbaz
Postaholic
Postaholic
Posts: 204
Joined: Mon Sep 15, 2014 12:55 am

Post by barbaz »

Cool, should be set for now. Thanks again Lanik for all the help, you are awesome 8)
barbaz
Postaholic
Postaholic
Posts: 204
Joined: Mon Sep 15, 2014 12:55 am

Post by barbaz »

Now that EasyList repo has been migrated to Github, trying to convert that repo, but keep getting unexpected problems.
First tried hg-fast-export, which almost worked... except that all the commit hashes were completely different from in the Github repo, meaning I couldn't use the local repo to look up specific "upstream" csets :-(
So then I tried cloning the Github repo (then repacking which reduced size from > 600 MB down to 59 MB :-? ), start a local branch at the equivalent revision immediately before I started making changes in Mercurial, then grab from Mercurial the set of patches and feed them to `git am`... but when I went to merge in new changes from master, almost every file has conflicts, including files I hadn't modified at all. (Would that be because the merge csets from Mercurial were just applied like normal csets & not converted to merge csets in git?)

So my question is either:
1) How to run hg-fast-export such that commit hashes are the same as in the Github repo?
*or*
2) How to apply my patches such that git sees the Mercurial patches depicting merge changesets, as merging the appropriate revision to my local branch? (Which IIUC pretty much comes down to, how to automatically figure out which git revision from master to merge?)

Or, if these are bad idea or technically impossible, what is the best way to go about this?
User avatar
LanikSJ
Site Owner
Site Owner
Posts: 1806
Joined: Thu Feb 15, 2007 7:44 am
Location: /dev/null

Post by LanikSJ »

You're better off cloning the new repo from GitHub and moving your changed files to that repo instead. Like you found out if you convert your own repo there's going to a ton of conflicts which you'll have to resolve.
"If it ain't broke don't fix it."
barbaz
Postaholic
Postaholic
Posts: 204
Joined: Mon Sep 15, 2014 12:55 am

Post by barbaz »

Is it possible to migrate my change history (which in this case includes merge history) into the repo clone as well? I'd like to keep the individual commit logs for non-merge-changesets.

Thanks :-)
User avatar
LanikSJ
Site Owner
Site Owner
Posts: 1806
Joined: Thu Feb 15, 2007 7:44 am
Location: /dev/null

Post by LanikSJ »

Not that I know off without causing a lot of conflicts.
"If it ain't broke don't fix it."
Locked