Merge branch 'PR/293'

Closes #292

.github/contributing.md

@@ -0,0 +1,28 @@
+When submitting a patch make sure the commits in your pull request:
+
+* Have good commit messages
+
+  Please read Chris Beams' blog post [How to Write a Git Commit
+  Message](https://chris.beams.io/posts/git-commit/) on how to write a
+  good commit message. Although the article recommends at most 50
+  characters for the subject, up to 72 characters are frequently
+  accepted for fast-export.
+
+* Adhere to good [commit
+hygiene](http://www.ericbmerritt.com/2011/09/21/commit-hygiene-and-git.html)
+
+  When developing a pull request for hg-fast-export, base your work on
+  the current `master` branch and rebase your work if it no longer can
+  be merged into the current `master` without conflicts. Never merge
+  `master` into your development branch, rebase if your work needs
+  updates from `master`.
+
+  When a pull request is modified due to review feedback, please
+  incorporate the changes into the proper commit. A good reference on
+  how to modify history is in the [Pro Git book, Section
+  7.6](https://git-scm.com/book/en/v2/Git-Tools-Rewriting-History).
+
+Please do not submit a pull request if you are not willing to spend
+the time required to address review comments or revise the patch until
+it follows the guidelines above. A _take it or leave it_ approach to
+contributing wastes both your and the maintainer's time.

README-SUBMODULES.md

@@ -27,10 +27,10 @@ command line option.
 
 ## Example
 
-Example mercurial repo folder structure (~/mercurial):
+Example mercurial repo folder structure (~/mercurial) containing two subrepos:
     src/...
-    subrepo/subrepo1
-    subrepo/subrepo2
+    subrepos/subrepo1
+    subrepos/subrepo2
 
 ### Setup
 Create an empty new folder where all the converted git modules will be imported:
@@ -41,18 +41,18 @@ Create an empty new folder where all the converted git modules will be imported:
     mkdir submodule1
     cd submodule1
     git init
-    hg-fast-export.sh -r ~/mercurial/subrepo1
+    hg-fast-export.sh -r ~/mercurial/subrepos/subrepo1
     cd ..
     mkdir submodule2
     cd submodule2
     git init
-    hg-fast-export.sh -r ~/mercurial/subrepo2
+    hg-fast-export.sh -r ~/mercurial/subrepos/subrepo2
 
 ### Create mapping file
     cd ~/imported-gits
     cat > submodule-mappings << EOF
-    "subrepo/subrepo1"="../submodule1"
-    "subrepo/subrepo2"="../submodule2"
+    "subrepos/subrepo1"="../submodule1"
+    "subrepos/subrepo2"="../submodule2"
     EOF
 
 ### Convert main repository
@@ -60,16 +60,16 @@ Create an empty new folder where all the converted git modules will be imported:
     mkdir git-main-repo
     cd git-main-repo
     git init
-    hg-fast-export.sh -r ~/mercurial --subrepo-map=../submodule-mappings
+    hg-fast-export.sh -r ~/mercurial --subrepo-map=~/imported-gits/submodule-mappings
 
 ### Result
-The resulting repository will now contain the subrepo/subrepo1 and
-subrepo/subrepo1 submodules. The created .gitmodules file will look
-like:
+The resulting repository will now contain the submodules at the paths
+`subrepos/subrepo1` and `subrepos/subrepo2`. The created .gitmodules
+file will look like:
 
-    [submodule "subrepo/subrepo1"]
-          path = subrepo/subrepo1
+    [submodule "subrepos/subrepo1"]
+          path = subrepos/subrepo1
           url = ../submodule1
-    [submodule "subrepo/subrepo2"]
-          path = subrepo/subrepo2
+    [submodule "subrepos/subrepo2"]
+          path = subrepos/subrepo2
           url = ../submodule2

README.md

@@ -1,4 +1,4 @@
-hg-fast-export.(sh|py) - mercurial to git converter using git-fast-import
+hg-fast-export.sh - mercurial to git converter using git-fast-import
 =========================================================================
 
 Legal
@@ -133,7 +133,10 @@ is to convert line endings in text files from CRLF to git's preferred LF:
 # $2 = Mercurial's hash of the file
 # $3 = "1" if Mercurial reports the file as binary, otherwise "0"
 
-if [ "$3" == "1" ]; then cat; else dos2unix; fi
+if [ "$3" == "1" ]; then cat; else dos2unix -q; fi
+# -q option in call to dos2unix allows to avoid returning an
+# error code when handling non-ascii based text files (like UTF-16
+# encoded text files)
 -- End of crlf-filter.sh --
 ```
 
@@ -167,7 +170,7 @@ defined filter methods in the [dos2unix](./plugins/dos2unix) and
 [branch_name_in_commit](./plugins/branch_name_in_commit) plugins.
 
 ```
-commit_data = {'branch': branch, 'parents': parents, 'author': author, 'desc': desc, 'revision': revision, 'hg_hash': hg_hash, 'committer': 'committer'}
+commit_data = {'branch': branch, 'parents': parents, 'author': author, 'desc': desc, 'revision': revision, 'hg_hash': hg_hash, 'committer': 'committer', 'extra': extra}
 
 def commit_message_filter(self,commit_data):
 ```
@@ -198,11 +201,15 @@ Notes/Limitations
 
 hg-fast-export supports multiple branches but only named branches with
 exactly one head each. Otherwise commits to the tip of these heads
-within the branch will get flattened into merge commits. Chris J
-Billington's [hg-export-tool] can help you to handle branches with
-duplicate heads.
-Alternatively, you can use the [head2branch plugin](./plugins/head2branch)
-to create a new named branch from an unnamed head.
+within the branch will get flattened into merge commits. There are a
+few options to deal with this:
+1. Chris J Billington's [hg-export-tool] can help you to handle branches with
+   duplicate heads.
+2. Use the [head2branch plugin](./plugins/head2branch) to create a new named
+   branch from an unnamed head.
+3. You can ignore unnamed heads with the `-ignore-unnamed-heads` option, which
+   is appropriate in situations such as the extra heads being close commits
+   (abandoned, unmerged changes).
 
 hg-fast-export will ignore any files or directories tracked by mercurial
 called `.git`, and will print a warning if it encounters one. Git cannot
@@ -221,8 +228,8 @@ possible to use hg-fast-export on remote repositories
 Design
 ------
 
-hg-fast-export.py was designed in a way that doesn't require a 2-pass
-mechanism or any prior repository analysis: if just feeds what it
+hg-fast-export was designed in a way that doesn't require a 2-pass
+mechanism or any prior repository analysis: it just feeds what it
 finds into git-fast-import. This also implies that it heavily relies
 on strictly linear ordering of changesets from hg, i.e. its
 append-only storage model so that changesets hg-fast-export already
@@ -258,6 +265,10 @@ hygiene](http://www.ericbmerritt.com/2011/09/21/commit-hygiene-and-git.html)
   how to modify history is in the [Pro Git book, Section
   7.6](https://git-scm.com/book/en/v2/Git-Tools-Rewriting-History).
 
+Please do not submit a pull request if you are not willing to spend
+the time required to address review comments or revise the patch until
+it follows the guidelines above. A _take it or leave it_ approach to
+contributing wastes both your and the maintainer's time.
 
 Frequent Problems
 =================
@@ -301,4 +312,11 @@ Frequent Problems
   git it looks like you have deleted all files, when in fact they have
   never been checked out. Just do a checkout of the branch you want.
 
+* `Error: repository has at least one unnamed head: hg r<N>`
+
+  By design, hg-fast-export cannot deal with extra heads on a branch.
+  There are a few options depending on whether the extra heads are
+  in-use/open or normally closed. See [Notes/Limitations](#noteslimitations)
+  section for more details.
+
 [hg-export-tool]: https://github.com/chrisjbillington/hg-export-tool

hg-fast-export.py

@@ -266,7 +266,7 @@ def sanitize_name(name,what="branch", mapping={}):
   if not auto_sanitize:
     return mapping.get(name,name)
   n=mapping.get(name,name)
-  p=re.compile(b'([[ ~^:?\\\\*]|\.\.)')
+  p=re.compile(b'([\\[ ~^:?\\\\*]|\.\.)')
   n=p.sub(b'_', n)
   if n[-1:] in (b'/', b'.'): n=n[:-1]+b'_'
   n=b'/'.join([dot(s) for s in n.split(b'/')])
@@ -294,7 +294,7 @@ def export_commit(ui,repo,revision,old_marks,max,count,authors,
     brmap[name]=n
     return n
 
-  (revnode,_,user,(time,timezone),files,desc,branch,_)=get_changeset(ui,repo,revision,authors,encoding)
+  (revnode,_,user,(time,timezone),files,desc,branch,extra)=get_changeset(ui,repo,revision,authors,encoding)
   if repo[revnode].hidden():
     return count
 
@@ -308,7 +308,7 @@ def export_commit(ui,repo,revision,old_marks,max,count,authors,
     commit_data = {'branch': branch, 'parents': parents,
                    'author': author, 'desc': desc,
                    'revision': revision, 'hg_hash': hg_hash,
-                   'committer': user}
+                   'committer': user, 'extra': extra}
     for filter in plugins['commit_message_filters']:
       filter(commit_data)
     branch = commit_data['branch']
@@ -434,9 +434,15 @@ def load_mapping(name, filename, mapping_is_raw):
   def process_unicode_escape_sequences(s):
     # Replace unicode escape sequences in the otherwise UTF8-encoded bytestring s with
     # the UTF8-encoded characters they represent. We need to do an additional
-    # .decode('utf8').encode('unicode-escape') to convert any non-ascii characters into
-    # their escape sequences so that the subsequent .decode('unicode-escape') succeeds:
-    return s.decode('utf8').encode('unicode-escape').decode('unicode-escape').encode('utf8')
+    # .decode('utf8').encode('ascii', 'backslashreplace') to convert any non-ascii
+    # characters into their escape sequences so that the subsequent
+    # .decode('unicode-escape') succeeds:
+    return (
+      s.decode('utf8')
+      .encode('ascii', 'backslashreplace')
+      .decode('unicode-escape')
+      .encode('utf8')
+    )
 
   def parse_quoted_line(line):
     m=quoted_regexp.match(line)
@@ -493,7 +499,12 @@ def verify_heads(ui,repo,cache,force,ignore_unnamed_heads,branchesmap):
     sanitized_name=sanitize_name(b,"branch",branchesmap)
     sha1=get_git_sha1(sanitized_name)
     c=cache.get(sanitized_name)
-    if sha1!=c:
+    if not c and sha1:
+      stderr_buffer.write(
+        b'Error: Branch [%s] already exists and was not created by hg-fast-export, '
+        b'export would overwrite unrelated branch\n' % b)
+      if not force: return False
+    elif sha1!=c:
       stderr_buffer.write(
         b'Error: Branch [%s] modified outside hg-fast-export:'
         b'\n%s (repo) != %s (cache)\n' % (b, b'<None>' if sha1 is None else sha1, c)
@@ -547,7 +558,7 @@ def hg2git(repourl,m,marksfile,mappingfile,headsfile,tipfile,
   except AttributeError:
     tip=len(repo)
 
-  min=int(state_cache.get('tip',0))
+  min=int(state_cache.get(b'tip',0))
   max=_max
   if _max<0 or max>tip:
     max=tip
@@ -580,8 +591,8 @@ def hg2git(repourl,m,marksfile,mappingfile,headsfile,tipfile,
     for rev in range(min,max):
       c=export_note(ui,repo,rev,c,authors, encoding, rev == min and min != 0)
 
-  state_cache['tip']=max
-  state_cache['repo']=repourl
+  state_cache[b'tip']=max
+  state_cache[b'repo']=repourl
   save_cache(tipfile,state_cache)
   save_cache(mappingfile,mapping_cache)