Merge branch 'frej/fix-gh348'

Support Mercurial 7.2
In Mercurial 7.2 the iteritems() method of the branchmap has been removed. Switch to iterating over the branches and then fetching heads by the branchheads() method. In 7.2, a call to mercurial.initialization.init() is also needed to process a repo using the largefiles extension. Thanks to Michael Cho (@cho-m) for suggesting the initialization fix. Closes #348 Co-developed-by: Michael Cho <michael@michaelcho.dev>
2026-02-27 06:50:41 +01:00 · 2026-02-14 20:55:09 +01:00 · 2026-02-14 20:53:20 +01:00 · 2026-01-17 20:38:59 +01:00 · 2026-01-12 16:33:37 -05:00 · 2026-01-12 16:33:20 -05:00
39 changed files with 2195 additions and 240 deletions
--- a/.github/requirements-earliest.txt
+++ b/.github/requirements-earliest.txt
@@ -0,0 +1,4 @@
 mercurial==5.2
 # Required for git_lfs_importer plugin
 pathspec==0.11.2
--- a/.github/requirements-latest.txt
+++ b/.github/requirements-latest.txt
@@ -0,0 +1,4 @@
 mercurial
 # Required for git_lfs_importer plugin
 pathspec==0.12.1
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -0,0 +1,71 @@
 name: CI
 on:
  push:
    branches: [master]
  pull_request:
    # The branches below must be a subset of the branches above
    branches: [master]
 jobs:
  test-earliest:
    name: Run test suite on the earliest supported Python version
    runs-on: ubuntu-22.04
    steps:
      - uses: actions/checkout@v4
        name: Checkout repository
        with:
          fetch-depth: 1
          submodules: 'recursive'
      - uses: actions/setup-python@v5
        id: earliest
        with:
          python-version: '3.7.x'
          check-latest: true
          cache: 'pip'
          cache-dependency-path: '**/requirements-earliest.txt'
      - name: Install dependencies
        run: |
            python -m pip install --upgrade pip
            pip install -r .github/requirements-earliest.txt
      - name: Report selected versions
        run: |
          echo Selected '${{ steps.earliest.outputs.python-version }}'
          ./hg-fast-export.sh --debug
      - name: Run tests on earliest supported Python version
        run: make -C t
  test-latest:
    name: Run test suite on the latest supported python version
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        name: Checkout repository
        with:
          fetch-depth: 1
          submodules: 'recursive'
      - uses: actions/setup-python@v5
        id: latest
        with:
          python-version: '3.x'
          check-latest: true
          cache: 'pip'
          cache-dependency-path: '**/requirements-latest.txt'
      - name: Install dependencies
        run: |
            python -m pip install --upgrade pip
            pip install -r .github/requirements-latest.txt
      - name: Report selected version
        run: |
          echo Selected '${{ steps.latest.outputs.python-version }}'
          ./hg-fast-export.sh --debug
      - name: Run tests on 3.x
        run: make -C t
--- a/.github/workflows/codeql-analysis.yml
+++ b/.github/workflows/codeql-analysis.yml
@@ -1,71 +0,0 @@
 # For most projects, this workflow file will not need changing; you simply need
 # to commit it to your repository.
 #
 # You may wish to alter this file to override the set of languages analyzed,
 # or to provide custom queries or build logic.
 name: "CodeQL"
 on:
  push:
    branches: [master]
  pull_request:
    # The branches below must be a subset of the branches above
    branches: [master]
  schedule:
    - cron: '0 15 * * 4'
 jobs:
  analyze:
    name: Analyze
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        # Override automatic language detection by changing the below list
        # Supported options are ['csharp', 'cpp', 'go', 'java', 'javascript', 'python']
        language: ['python']
        # Learn more...
        # https://docs.github.com/en/github/finding-security-vulnerabilities-and-errors-in-your-code/configuring-code-scanning#overriding-automatic-language-detection
    steps:
    - name: Checkout repository
      uses: actions/checkout@v2
      with:
        # We must fetch at least the immediate parents so that if this is
        # a pull request then we can checkout the head.
        fetch-depth: 2
    # If this run was triggered by a pull request event, then checkout
    # the head of the pull request instead of the merge commit.
    - run: git checkout HEAD^2
      if: ${{ github.event_name == 'pull_request' }}
    # Initializes the CodeQL tools for scanning.
    - name: Initialize CodeQL
      uses: github/codeql-action/init@v1
      with:
        languages: ${{ matrix.language }}
        # If you wish to specify custom queries, you can do so here or in a config file.
        # By default, queries listed here will override any specified in a config file. 
        # Prefix the list here with "+" to use these queries and those in the config file.
        # queries: ./path/to/local/query, your-org/your-repo/queries@main
    # Autobuild attempts to build any compiled languages  (C/C++, C#, or Java).
    # If this step fails, then you should remove it and run the build manually (see below)
    - name: Autobuild
      uses: github/codeql-action/autobuild@v1
    # ℹ️ Command-line programs to run using the OS shell.
    # 📚 https://git.io/JvXDl
    # ✏️ If the Autobuild fails above, remove it and uncomment the following three lines
    #    and modify them (or add more) to build your code if your project
    #    uses a compiled language
    #- run: |
    #   make bootstrap
    #   make release
    - name: Perform CodeQL Analysis
      uses: github/codeql-action/analyze@v1
--- a/.gitmodules
+++ b/.gitmodules
@@ -0,0 +1,3 @@
 [submodule "t/sharness"]
 	path = t/sharness
 	url = https://github.com/felipec/sharness.git
--- a/README.md
+++ b/README.md
@@ -29,10 +29,9 @@ first time.
 System Requirements
 -------------------
-This project depends on Python 2.7 or 3.5+, and the Mercurial >= 4.6
+This project depends on Python (>=3.7) and the Mercurial package (>=
-package (>= 5.2, if Python 3.5+). If Python is not installed, install
+5.2).  If Python is not installed, install it before proceeding. The
-it before proceeding. The Mercurial package can be installed with `pip
+Mercurial package can be installed with `pip install mercurial`.
 install mercurial`.
 On windows the bash that comes with "Git for Windows" is known to work
 well.
@@ -43,11 +42,10 @@ Usage
 Using hg-fast-export is quite simple for a mercurial repository <repo>:
 ```
-mkdir repo-git # or whatever
+git init repo-git # or whatever
 cd repo-git
 git init
 hg-fast-export.sh -r <local-repo>
-git checkout HEAD
+git checkout
 ```
 Please note that hg-fast-export does not automatically check out the
@@ -111,8 +109,8 @@ branch/tag names. In the future -n will become the default, but in
 order to not break existing incremental conversions, the default
 remains with the old behavior.
-By default, the `default` mercurial branch is renamed to the `master` 
+By default, the `default` mercurial branch is renamed to the `master`
-branch on git. If your mercurial repo contains both `default` and 
+branch on git. If your mercurial repo contains both `default` and
 `master` branches, you'll need to override this behavior. Use
 `-M <newName>` to specify what name to give the `default` branch.
@@ -140,6 +138,51 @@ if [ "$3" == "1" ]; then cat; else dos2unix -q; fi
 -- End of crlf-filter.sh --
 ```
 Mercurial Largefiles Extension
 ------------------------------
 ### Handling Mercurial Largefiles during Migration
 When migrating from Mercurial to Git, largefiles are exported as ordinary
 files by default. To ensure a successful migration and manage repository
 size, follow the requirements below.
 #### 1. Pre-Export: Ensure File Availability
 Before starting the export, you must have all largefiles from all
 Mercurial commits available locally. Use one of these methods:
 * **For a new clone:** `hg clone --all-largefiles <repo-url>`
 * **For an existing repo:** `hg lfpull --rev "all()"`
 #### 2. Choosing Your LFS Strategy
 If you want your files to be versioned in Git LFS rather than as standard
 Git blobs, you have two primary paths:
 * **[git_lfs_importer plugin](./plugins/git_lfs_importer/README.md)
  (During Conversion)**
  Recommended for large repos. This performs Just-In-Time (JIT) conversion
  by identifying large files during the export and writing LFS pointers
  immediately, skipping the need for a second pass. This also supports
  **incremental conversion**, making it much more efficient for ongoing
  migrations.
 * **[git lfs migrate import](https://github.com/git-lfs/git-lfs/blob/main/docs/man/git-lfs-migrate.adoc)
  (After Conversion)**
  A standard two-step process: first, export the full history from Mercurial
  to Git, then run a separate full history rewrite to move files into LFS.
 ### Why use the git_lfs_importer plugin?
 For "monorepos" or very large repositories (100GiB+), the traditional
 two-step process can take days. By integrating the LFS conversion
 directly into the history export, the plugin eliminates the massive
 time overhead of a secondary history rewrite and allows for incremental
 progress.
 For detailed setup, see the
 [git_lfs_importer](./plugins/git_lfs_importer/README.md)
 plugin documentation.
 Plugins
 -----------------
@@ -170,9 +213,18 @@ defined filter methods in the [dos2unix](./plugins/dos2unix) and
 [branch_name_in_commit](./plugins/branch_name_in_commit) plugins.
 ```
-commit_data = {'branch': branch, 'parents': parents, 'author': author, 'desc': desc, 'revision': revision, 'hg_hash': hg_hash, 'committer': 'committer', 'extra': extra}
+commit_data = {
  'author': author,
  'branch': branch,
  'committer': 'committer',
  'desc': desc,
  'extra': extra,
  'hg_hash': hg_hash,
  'parents': parents,
  'revision': revision,
 }
-def commit_message_filter(self,commit_data):
+def commit_message_filter(self, commit_data):
 ```
 The `commit_message_filter` method is called for each commit, after parsing
 from hg, but before outputting to git. The dictionary `commit_data` contains the
@@ -181,9 +233,14 @@ values in the dictionary after filters have been run are used to create the git
 commit.
 ```
-file_data = {'filename':filename,'file_ctx':file_ctx,'d':d}
+file_data = {
  'data': file_contents,
  'file_ctx': file_ctx,
  'filename': filename,
  'is_largefile': largefile_status,
 }
-def file_data_filter(self,file_data):
+def file_data_filter(self, file_data):
 ```
 The `file_data_filter` method is called for each file within each commit.
 The dictionary `file_data` contains the above attributes about the file, and
@@ -191,6 +248,17 @@ can be modified by any filter. `file_ctx` is the filecontext from the
 mercurial python library.  After all filters have been run, the values
 are used to add the file to the git commit.
 The `file_data_filter` method is also called when files are deleted,
 but in this case the `data` and `file_ctx` keys map to None. This is
 so that a filter which modifies file names can apply the same name
 transformations when files are deleted.
 The `is_largefile` entry within the `file_data` dictionary will contain
 `True` if the original file was a largefile and has been converted
 to a normal file before the plugins were invoked. In this case, the `file_ctx`
 will still point to the filecontext for the original, unconverted file, while
 `filename` and `data` will contain the already converted information.
 Submodules
 ----------
 See README-SUBMODULES.md for how to convert subrepositories into git
@@ -207,7 +275,7 @@ few options to deal with this:
   duplicate heads.
 2. Use the [head2branch plugin](./plugins/head2branch) to create a new named
   branch from an unnamed head.
-3. You can ignore unnamed heads with the `-ignore-unnamed-heads` option, which
+3. You can ignore unnamed heads with the `--ignore-unnamed-heads` option, which
   is appropriate in situations such as the extra heads being close commits
   (abandoned, unmerged changes).
--- a/hg-fast-export.py
+++ b/hg-fast-export.py
@@ -1,10 +1,9 @@
-#!/usr/bin/env python2
+#!/usr/bin/env python3
 # Copyright (c) 2007, 2008 Rocco Rutte <pdmef@gmx.net> and others.
 # Copyright (c) 2025 Siemens
 # License: MIT <http://www.opensource.org/licenses/mit-license.php>
 from mercurial import node
 from mercurial.scmutil import revsymbol
 from hg2git import setup_repo,fixup_user,get_branch,get_changeset
 from hg2git import load_cache,save_cache,get_git_sha1,set_default_branch,set_origin_name
 from optparse import OptionParser
@@ -13,17 +12,7 @@ import sys
 import os
 from binascii import hexlify
 import pluginloader
-PY2 = sys.version_info.major == 2
+from hgext.largefiles import lfutil
 if PY2:
  str = unicode
 if PY2 and sys.platform == "win32":
  # On Windows, sys.stdout is initially opened in text mode, which means that
  # when a LF (\n) character is written to sys.stdout, it will be converted
  # into CRLF (\r\n).  That makes git blow up, so use this platform-specific
  # code to change the mode of sys.stdout to binary.
  import msvcrt
  msvcrt.setmode(sys.stdout.fileno(), os.O_BINARY)
 # silly regex to catch Signed-off-by lines in log message
 sob_re=re.compile(b'^Signed-[Oo]ff-[Bb]y: (.+)$')
@@ -39,26 +28,26 @@ submodule_mappings=None
 # author/branch/tag names.
 auto_sanitize = None
 stdout_buffer = sys.stdout if PY2 else sys.stdout.buffer
 stderr_buffer = sys.stderr if PY2 else sys.stderr.buffer
 def gitmode(flags):
  return b'l' in flags and b'120000' or b'x' in flags and b'100755' or b'100644'
 def wr_no_nl(msg=b''):
  assert isinstance(msg, bytes)
  if msg:
-    stdout_buffer.write(msg)
+    sys.stdout.buffer.write(msg)
 def wr(msg=b''):
-  wr_no_nl(msg)
+  wr_no_nl(msg + b'\n')
  stdout_buffer.write(b'\n')
  #map(lambda x: sys.stderr.write('\t[%s]\n' % x),msg.split('\n'))
 def wr_data(data):
  wr(b'data %d' % (len(data)))
  wr(data)
 def checkpoint(count):
  count=count+1
  if cfg_checkpoint_count>0 and count%cfg_checkpoint_count==0:
-    stderr_buffer.write(b"Checkpoint after %d commits\n" % count)
+    sys.stderr.buffer.write(b"Checkpoint after %d commits\n" % count)
    wr(b'checkpoint')
    wr()
  return count
@@ -68,39 +57,15 @@ def revnum_to_revref(rev, old_marks):
  or a mark)"""
  return old_marks.get(rev) or b':%d' % (rev+1)
-def file_mismatch(f1,f2):
+def get_filechanges(repo,revision,parents,files):
  """See if two revisions of a file are not equal."""
  return node.hex(f1)!=node.hex(f2)
 def split_dict(dleft,dright,l=[],c=[],r=[],match=file_mismatch):
  """Loop over our repository and find all changed and missing files."""
  for left in dleft.keys():
    right=dright.get(left,None)
    if right==None:
      # we have the file but our parent hasn't: add to left set
      l.append(left)
    elif match(dleft[left],right) or gitmode(dleft.flags(left))!=gitmode(dright.flags(left)):
      # we have it but checksums mismatch: add to center set
      c.append(left)
  for right in dright.keys():
    left=dleft.get(right,None)
    if left==None:
      # if parent has file but we don't: add to right set
      r.append(right)
    # change is already handled when comparing child against parent
  return l,c,r
 def get_filechanges(repo,revision,parents,mleft):
  """Given some repository and revision, find all changed/deleted files."""
-  l,c,r=[],[],[]
+  if not parents:
-  for p in parents:
+    # first revision: feed in full manifest
-    if p<0: continue
+    return files,[]
-    mright=revsymbol(repo,b"%d" %p).manifest()
+  else:
-    l,c,r=split_dict(mleft,mright,l,c,r)
+    # take the changes from the first parent
-  l.sort()
+    f=repo.status(parents[0],revision)
-  c.sort()
+    return f.modified+f.added,f.removed
  r.sort()
  return l,c,r
 def get_author(logmessage,committer,authors):
  """As git distincts between author and committer of a patch, try to
@@ -151,7 +116,7 @@ def remove_gitmodules(ctx):
 def refresh_git_submodule(name,subrepo_info):
  wr(b'M 160000 %s %s' % (subrepo_info[1],name))
-  stderr_buffer.write(
+  sys.stderr.buffer.write(
    b"Adding/updating submodule %s, revision %s\n" % (name, subrepo_info[1])
  )
  return b'[submodule "%s"]\n\tpath = %s\n\turl = %s\n' % (name, name, subrepo_info[0])
@@ -171,14 +136,14 @@ def refresh_hg_submodule(name,subrepo_info):
    revnum=mapping_cache[subrepo_hash]
    gitSha=marks_cache[int(revnum)]
    wr(b'M 160000 %s %s' % (gitSha,name))
-    stderr_buffer.write(
+    sys.stderr.buffer.write(
      b"Adding/updating submodule %s, revision %s->%s\n"
      % (name, subrepo_hash, gitSha)
    )
    return b'[submodule "%s"]\n\tpath = %s\n\turl = %s\n' % (name,name,
      submodule_mappings[name])
  else:
-    stderr_buffer.write(
+    sys.stderr.buffer.write(
      b"Warning: Could not find hg revision %s for %s in git %s\n"
      % (subrepo_hash, name, gitRepoLocation,)
    )
@@ -197,8 +162,33 @@ def refresh_gitmodules(ctx):
  if len(gitmodules):
    wr(b'M 100644 inline .gitmodules')
-    wr(b'data %d' % (len(gitmodules)+1))
+    wr_data(gitmodules)
-    wr(gitmodules)
+
 def is_largefile(filename):
  return filename[:6] == b'.hglf/'
 def largefile_orig_name(filename):
  return filename[6:]
 def largefile_data(ctx, file, filename):
  lf_file_ctx=ctx.filectx(file)
  lf_hash=lf_file_ctx.data().strip(b'\n')
  sys.stderr.write("Detected large file hash %s\n" % lf_hash.decode())
  #should detect where the large files are located
  file_with_data = lfutil.findfile(ctx.repo(), lf_hash)
  if file_with_data is None:
    # Autodownloading from the mercurial repository would be an issue as there
    # is a good chance that we may need to input some username and password.
    # This will surely break fast-export as there will be some unexpected
    # output.
    sys.stderr.write("Large file wasn't found in local cache.\n")
    sys.stderr.write("Please clone with --all-largefiles\n")
    sys.stderr.write("or pull all large files with 'hg lfpull --rev "
            "\"all()\"'\n")
    # closing in the middle of import will revert everything to the last checkpoint
    sys.exit(3)
  with open(os.path.normpath(file_with_data), 'rb') as file_with_data_handle:
    return file_with_data_handle.read()
 def export_file_contents(ctx,manifest,files,hgtags,encoding='',plugins={}):
  count=0
@@ -210,37 +200,45 @@ def export_file_contents(ctx,manifest,files,hgtags,encoding='',plugins={}):
      refresh_gitmodules(ctx)
    # Skip .hgtags files. They only get us in trouble.
    if not hgtags and file == b".hgtags":
-      stderr_buffer.write(b'Skip %s\n' % file)
+      sys.stderr.buffer.write(b'Skip %s\n' % file)
      continue
    if encoding:
      filename=file.decode(encoding).encode('utf8')
    else:
      filename=file
    if b'.git' in filename.split(b'/'): # Even on Windows, the path separator is / here.
-      stderr_buffer.write(
+      sys.stderr.buffer.write(
        b'Ignoring file %s which cannot be tracked by git\n' % filename
      )
      continue
    largefile = False
    file_ctx=ctx.filectx(file)
-    d=file_ctx.data()
+    if is_largefile(filename):
      largefile = True
      filename = largefile_orig_name(filename)
      d = largefile_data(ctx, file, filename)
    else:
      d=file_ctx.data()
    if plugins and plugins['file_data_filters']:
-      file_data = {'filename':filename,'file_ctx':file_ctx,'data':d}
+      file_data = {'filename':filename,'file_ctx':file_ctx,'data':d, 'is_largefile':largefile}
      for filter in plugins['file_data_filters']:
        filter(file_data)
      d=file_data['data']
      filename=file_data['filename']
      file_ctx=file_data['file_ctx']
-    wr(b'M %s inline %s' % (gitmode(manifest.flags(file)),
+    if d is not None:
-                           strip_leading_slash(filename)))
+      wr(b'M %s inline %s' % (gitmode(manifest.flags(file)),
-    wr(b'data %d' % len(d)) # had some trouble with size()
+                             strip_leading_slash(filename)))
-    wr(d)
+      wr(b'data %d' % len(d)) # had some trouble with size()
-    count+=1
+      wr(d)
-    if count%cfg_export_boundary==0:
+      count+=1
-      stderr_buffer.write(b'Exported %d/%d files\n' % (count,max))
+      if count%cfg_export_boundary==0:
        sys.stderr.buffer.write(b'Exported %d/%d files\n' % (count,max))
  if max>cfg_export_boundary:
-    stderr_buffer.write(b'Exported %d/%d files\n' % (count,max))
+    sys.stderr.buffer.write(b'Exported %d/%d files\n' % (count,max))
 def sanitize_name(name,what="branch", mapping={}):
  """Sanitize input roughly according to git-check-ref-format(1)"""
@@ -266,7 +264,7 @@ def sanitize_name(name,what="branch", mapping={}):
  if not auto_sanitize:
    return mapping.get(name,name)
  n=mapping.get(name,name)
-  p=re.compile(b'([\\[ ~^:?\\\\*]|\.\.)')
+  p=re.compile(b'([\\[ ~^:?\\\\*]|\\.\\.)')
  n=p.sub(b'_', n)
  if n[-1:] in (b'/', b'.'): n=n[:-1]+b'_'
  n=b'/'.join([dot(s) for s in n.split(b'/')])
@@ -274,7 +272,7 @@ def sanitize_name(name,what="branch", mapping={}):
  n=p.sub(b'_', n)
  if n!=name:
-    stderr_buffer.write(
+    sys.stderr.buffer.write(
      b'Warning: sanitized %s [%s] to [%s]\n' % (what.encode(), name, n)
    )
  return n
@@ -286,7 +284,7 @@ def strip_leading_slash(filename):
 def export_commit(ui,repo,revision,old_marks,max,count,authors,
                  branchesmap,sob,brmap,hgtags,encoding='',fn_encoding='',
-                  plugins={}):
+                  first_commit_hash="",plugins={}):
  def get_branchname(name):
    if name in brmap:
      return brmap[name]
@@ -294,15 +292,18 @@ def export_commit(ui,repo,revision,old_marks,max,count,authors,
    brmap[name]=n
    return n
-  (revnode,_,user,(time,timezone),files,desc,branch,extra)=get_changeset(ui,repo,revision,authors,encoding)
+  ctx=repo[revision]
-  if repo[revnode].hidden():
+
  if ctx.hidden():
    return count
  (_,user,(time,timezone),files,desc,branch,extra)=get_changeset(ui,repo,revision,authors,encoding)
  branch=get_branchname(branch)
  parents = [p for p in repo.changelog.parentrevs(revision) if p >= 0]
  author = get_author(desc,user,authors)
-  hg_hash=revsymbol(repo,b"%d" % revision).hex()
+  hg_hash=ctx.hex()
  if plugins and plugins['commit_message_filters']:
    commit_data = {'branch': branch, 'parents': parents,
@@ -325,61 +326,61 @@ def export_commit(ui,repo,revision,old_marks,max,count,authors,
  if sob:
    wr(b'author %s %d %s' % (author,time,timezone))
  wr(b'committer %s %d %s' % (user,time,timezone))
-  wr(b'data %d' % (len(desc)+1)) # wtf?
+  wr_data(desc + b'\n')
  wr(desc)
  wr()
  ctx=revsymbol(repo, b"%d" % revision)
  man=ctx.manifest()
  added,changed,removed,type=[],[],[],''
-  if len(parents) == 0:
+  if not parents:
    # first revision: feed in full manifest
    added=man.keys()
    added.sort()
    type='full'
    if revision == 0 and first_commit_hash:
      wr(b'from %s' % first_commit_hash.encode())
      type='simple delta'
  else:
    wr(b'from %s' % revnum_to_revref(parents[0], old_marks))
    if len(parents) == 1:
      # later non-merge revision: feed in changed manifest
      # if we have exactly one parent, just take the changes from the
      # manifest without expensively comparing checksums
      f=repo.status(parents[0],revnode)
      added,changed,removed=f.added,f.modified,f.removed
      type='simple delta'
    else: # a merge with two parents
      wr(b'merge %s' % revnum_to_revref(parents[1], old_marks))
      # later merge revision: feed in changed manifest
      # for many files comparing checksums is expensive so only do it for
      # merges where we really need it due to hg's revlog logic
      added,changed,removed=get_filechanges(repo,revision,parents,man)
      type='thorough delta'
-  stderr_buffer.write(
+  modified,removed=get_filechanges(repo,revision,parents,files)
-    b'%s: Exporting %s revision %d/%d with %d/%d/%d added/changed/removed files\n'
+
-    % (branch, type.encode(), revision + 1, max, len(added), len(changed), len(removed))
+  sys.stderr.buffer.write(
    b'%s: Exporting %s revision %d/%d with %d/%d modified/removed files\n'
    % (branch, type.encode(), revision + 1, max, len(modified), len(removed))
  )
-  for filename in removed:
+  for file in removed:
    if fn_encoding:
-      filename=filename.decode(fn_encoding).encode('utf8')
+      filename=file.decode(fn_encoding).encode('utf8')
    else:
      filename=file
    if plugins and plugins['file_data_filters']:
      file_data = {'filename':filename, 'file_ctx':None, 'data':None}
      for filter in plugins['file_data_filters']:
        filter(file_data)
      filename=file_data['filename']
    filename=strip_leading_slash(filename)
    if filename==b'.hgsub':
      remove_gitmodules(ctx)
    if is_largefile(filename):
      filename=largefile_orig_name(filename)
    wr(b'D %s' % filename)
-  export_file_contents(ctx,man,added,hgtags,fn_encoding,plugins)
+  export_file_contents(ctx,man,modified,hgtags,fn_encoding,plugins)
  export_file_contents(ctx,man,changed,hgtags,fn_encoding,plugins)
  wr()
  return checkpoint(count)
 def export_note(ui,repo,revision,count,authors,encoding,is_first):
-  (revnode,_,user,(time,timezone),_,_,_,_)=get_changeset(ui,repo,revision,authors,encoding)
+  ctx = repo[revision]
-  if repo[revnode].hidden():
+
  if ctx.hidden():
    return count
-  parents = [p for p in repo.changelog.parentrevs(revision) if p >= 0]
+  (_,user,(time,timezone),_,_,_,_)=get_changeset(ui,repo,revision,authors,encoding)
  wr(b'commit refs/notes/hg')
  wr(b'committer %s %d %s' % (user,time,timezone))
@@ -387,9 +388,8 @@ def export_note(ui,repo,revision,count,authors,encoding,is_first):
  if is_first:
    wr(b'from refs/notes/hg^0')
  wr(b'N inline :%d' % (revision+1))
-  hg_hash=revsymbol(repo,b"%d" % revision).hex()
+  hg_hash=ctx.hex()
-  wr(b'data %d' % (len(hg_hash)))
+  wr_data(hg_hash)
  wr_no_nl(hg_hash)
  wr()
  return checkpoint(count)
@@ -402,18 +402,18 @@ def export_tags(ui,repo,old_marks,mapping_cache,count,authors,tagsmap):
    if tag==b'tip': continue
    # ignore tags to nodes that are missing (ie, 'in the future')
    if hexlify(node) not in mapping_cache:
-      stderr_buffer.write(b'Tag %s refers to unseen node %s\n' % (tag, hexlify(node)))
+      sys.stderr.buffer.write(b'Tag %s refers to unseen node %s\n' % (tag, hexlify(node)))
      continue
    rev=int(mapping_cache[hexlify(node)])
    ref=revnum_to_revref(rev, old_marks)
    if ref==None:
-      stderr_buffer.write(
+      sys.stderr.buffer.write(
        b'Failed to find reference for creating tag %s at r%d\n' % (tag, rev)
      )
      continue
-    stderr_buffer.write(b'Exporting tag [%s] at [hg r%d] [git %s]\n' % (tag, rev, ref))
+    sys.stderr.buffer.write(b'Exporting tag [%s] at [hg r%d] [git %s]\n' % (tag, rev, ref))
    wr(b'reset refs/tags/%s' % tag)
    wr(b'from %s' % ref)
    wr()
@@ -447,8 +447,8 @@ def load_mapping(name, filename, mapping_is_raw):
  def parse_quoted_line(line):
    m=quoted_regexp.match(line)
    if m==None:
-      return 
+      return
-    
+
    return (process_unicode_escape_sequences(m.group(1)),
            process_unicode_escape_sequences(m.group(5)))
@@ -488,7 +488,9 @@ def branchtip(repo, heads):
 def verify_heads(ui,repo,cache,force,ignore_unnamed_heads,branchesmap):
  branches={}
-  for bn, heads in repo.branchmap().iteritems():
+
  for bn in repo.branchmap():
    heads = repo.branchmap().branchheads(bn)
    branches[bn] = branchtip(repo, heads)
  l=[(-repo.changelog.rev(n), n, t) for t, n in branches.items()]
  l.sort()
@@ -500,12 +502,12 @@ def verify_heads(ui,repo,cache,force,ignore_unnamed_heads,branchesmap):
    sha1=get_git_sha1(sanitized_name)
    c=cache.get(sanitized_name)
    if not c and sha1:
-      stderr_buffer.write(
+      sys.stderr.buffer.write(
        b'Error: Branch [%s] already exists and was not created by hg-fast-export, '
        b'export would overwrite unrelated branch\n' % b)
      if not force: return False
    elif sha1!=c:
-      stderr_buffer.write(
+      sys.stderr.buffer.write(
        b'Error: Branch [%s] modified outside hg-fast-export:'
        b'\n%s (repo) != %s (cache)\n' % (b, b'<None>' if sha1 is None else sha1, c)
      )
@@ -515,9 +517,9 @@ def verify_heads(ui,repo,cache,force,ignore_unnamed_heads,branchesmap):
  t={}
  unnamed_heads=False
  for h in repo.filtered(b'visible').heads():
-    (_,_,_,_,_,_,branch,_)=get_changeset(ui,repo,h)
+    branch=get_branch(repo[h].branch())
    if t.get(branch,False):
-      stderr_buffer.write(
+      sys.stderr.buffer.write(
        b'Error: repository has an unnamed head: hg r%d\n'
        % repo.changelog.rev(h)
      )
@@ -529,7 +531,8 @@ def verify_heads(ui,repo,cache,force,ignore_unnamed_heads,branchesmap):
 def hg2git(repourl,m,marksfile,mappingfile,headsfile,tipfile,
           authors={},branchesmap={},tagsmap={},
-           sob=False,force=False,ignore_unnamed_heads=False,hgtags=False,notes=False,encoding='',fn_encoding='',
+           sob=False,force=False,ignore_unnamed_heads=False,hgtags=False,
           notes=False,encoding='',fn_encoding='',first_commit_hash='',
           plugins={}):
  def check_cache(filename, contents):
    if len(contents) == 0:
@@ -545,7 +548,7 @@ def hg2git(repourl,m,marksfile,mappingfile,headsfile,tipfile,
  if len(state_cache) != 0:
    for (name, data) in [(marksfile, old_marks),
                         (mappingfile, mapping_cache),
-                         (headsfile, state_cache)]:
+                         (headsfile, heads_cache)]:
      check_cache(name, data)
  ui,repo=setup_repo(repourl)
@@ -564,15 +567,15 @@ def hg2git(repourl,m,marksfile,mappingfile,headsfile,tipfile,
    max=tip
  for rev in range(0,max):
-    (revnode,_,_,_,_,_,_,_)=get_changeset(ui,repo,rev,authors)
+    ctx=repo[rev]
-    if repo[revnode].hidden():
+    if ctx.hidden():
      continue
-    mapping_cache[hexlify(revnode)] = b"%d" % rev
+    mapping_cache[ctx.hex()] = b"%d" % rev
  if submodule_mappings:
    # Make sure that all mercurial submodules are registered in the submodule-mappings file
    for rev in range(0,max):
-      ctx=revsymbol(repo,b"%d" % rev)
+      ctx=repo[rev]
      if ctx.hidden():
        continue
      if ctx.substate:
@@ -585,7 +588,7 @@ def hg2git(repourl,m,marksfile,mappingfile,headsfile,tipfile,
  brmap={}
  for rev in range(min,max):
    c=export_commit(ui,repo,rev,old_marks,max,c,authors,branchesmap,
-                    sob,brmap,hgtags,encoding,fn_encoding,
+                    sob,brmap,hgtags,encoding,fn_encoding,first_commit_hash,
                    plugins)
  if notes:
    for rev in range(min,max):
@@ -659,6 +662,8 @@ if __name__=='__main__':
      help="Add a plugin with the given init string <name=init>")
  parser.add_option("--subrepo-map", type="string", dest="subrepo_map",
      help="Provide a mapping file between the subrepository name and the submodule name")
  parser.add_option("--first-commit-hash", type="string", dest="first_commit_hash",
      help="Allow importing into an existing git repository by specifying the hash of the first commit")
  (options,args)=parser.parse_args()
@@ -738,4 +743,5 @@ if __name__=='__main__':
                  ignore_unnamed_heads=options.ignore_unnamed_heads,
                  hgtags=options.hgtags,
                  notes=options.notes,encoding=encoding,fn_encoding=fn_encoding,
                  first_commit_hash=options.first_commit_hash,
                  plugins=plugins_dict))
--- a/hg-fast-export.sh
+++ b/hg-fast-export.sh
@@ -29,7 +29,7 @@ GFI_OPTS=""
 if [ -z "${PYTHON}" ]; then
    # $PYTHON is not set, so we try to find a working python with mercurial:
-    for python_cmd in python2 python python3; do
+    for python_cmd in python3 python; do
        if command -v $python_cmd > /dev/null; then
            $python_cmd -c 'from mercurial.scmutil import revsymbol' 2> /dev/null
            if [ $? -eq 0 ]; then
@@ -45,6 +45,14 @@ if [ -z "${PYTHON}" ]; then
    exit 1
 fi
 "${PYTHON}" -c 'import sys; exit(sys.version_info.major==3 and sys.version_info.minor >= 7)'
 if [ $? -eq 0 ]; then
    echo "Could not find an interpreter for a supported Python version (>= 3.7)" \
        "Please use the 'PYTHON' environment variable to specify the interpreter to use."
    exit 1
 fi
 USAGE="[--quiet] [-r <repo>] [--force] [--ignore-unnamed-heads] [-m <max>] [-s] [--hgtags] [-A <file>] [-B <file>] [-T <file>] [-M <name>] [-o <name>] [--hg-hash] [-e <encoding>]"
 LONG_USAGE="Import hg repository <repo> up to either tip or <max>
 If <repo> is omitted, use last hg repository as obtained from state file,
@@ -79,6 +87,8 @@ Options:
 	                         with <file-path> <hg-hash> <is-binary> as arguments
 	--plugin <plugin=init>  Add a plugin with the given init string (repeatable)
 	--plugin-path <plugin-path> Add an additional plugin lookup path
  --first-commit-hash <git-commit-hash>  Use the given git commit hash as the
                                          first commit's parent (for grafting)
 "
 case "$1" in
    -h|--help)
@@ -86,6 +96,14 @@ case "$1" in
      echo ""
      echo "$LONG_USAGE"
      exit 0
      ;;
    --debug)
      echo -n "Using Python: "
      "${PYTHON}" --version
      echo -n "Using Mercurial: "
      hg --version
      exit 0
 esac
 IS_BARE=$(git rev-parse --is-bare-repository) \
--- a/hg-reset.py
+++ b/hg-reset.py
@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # Copyright (c) 2007, 2008 Rocco Rutte <pdmef@gmx.net> and others.
 # License: GPLv2
--- a/hg2git.py
+++ b/hg2git.py
@@ -1,24 +1,17 @@
-#!/usr/bin/env python2
+#!/usr/bin/env python3
 # Copyright (c) 2007, 2008 Rocco Rutte <pdmef@gmx.net> and others.
 # License: MIT <http://www.opensource.org/licenses/mit-license.php>
 from mercurial import hg,util,ui,templatefilters
 from mercurial import error as hgerror
-from mercurial.scmutil import revsymbol,binnode
+from mercurial.scmutil import binnode
 import re
 import os
 import sys
 import subprocess
 PY2 = sys.version_info.major < 3
 if PY2:
  str = unicode
  fsencode = lambda s: s.encode(sys.getfilesystemencoding())
 else:
  from os import fsencode
 # default git branch name
 cfg_master=b'master'
 # default origin name
@@ -30,13 +23,21 @@ user_clean_re=re.compile(b'^["]([^"]+)["]$')
 def set_default_branch(name):
  global cfg_master
-  cfg_master = name.encode('utf8') if not isinstance(name, bytes) else name
+  cfg_master = name.encode('utf8')
 def set_origin_name(name):
  global origin_name
-  origin_name = name
+  origin_name = name.encode('utf8')
 def setup_repo(url):
  try:
    # Mercurial >= 7.2 requires explicit initialization for largefile
    # support to work.
    from mercurial import initialization
    initialization.init()
  except ImportError:
    pass
  try:
    myui=ui.ui(interactive=False)
  except TypeError:
@@ -44,7 +45,7 @@ def setup_repo(url):
    myui.setconfig(b'ui', b'interactive', b'off')
    # Avoids a warning when the repository has obsolete markers
    myui.setconfig(b'experimental', b'evolution.createmarkers', True)
-  return myui,hg.repository(myui, fsencode(url)).unfiltered()
+  return myui,hg.repository(myui, os.fsencode(url)).unfiltered()
 def fixup_user(user,authors):
  user=user.strip(b"\"")
@@ -81,22 +82,13 @@ def get_branch(name):
  return name
 def get_changeset(ui,repo,revision,authors={},encoding=''):
-  # Starting with Mercurial 4.6 lookup no longer accepts raw hashes
+  (manifest,user,(time,timezone),files,desc,extra)=repo.changelog.read(revision)
  # for lookups. Work around it by changing our behaviour depending on
  # how it fails
  try:
    node=repo.lookup(revision)
  except (TypeError, hgerror.ProgrammingError):
    node=binnode(revsymbol(repo, b"%d" % revision)) # We were given a numeric rev
  except hgerror.RepoLookupError:
    node=revision # We got a raw hash
  (manifest,user,(time,timezone),files,desc,extra)=repo.changelog.read(node)
  if encoding:
    user=user.decode(encoding).encode('utf8')
    desc=desc.decode(encoding).encode('utf8')
  tz=b"%+03d%02d" % (-timezone // 3600, ((-timezone % 3600) // 60))
-  branch=get_branch(extra.get(b'branch', b'master'))
+  branch=get_branch(extra.get(b'branch', b''))
-  return (node,manifest,fixup_user(user,authors),(time,tz),files,desc,branch,extra)
+  return (manifest,fixup_user(user,authors),(time,tz),files,desc,branch,extra)
 def mangle_key(key):
  return key
--- a/pluginloader/init.py
+++ b/pluginloader/init.py
@@ -1,19 +1,23 @@
 import os
-import imp
+import importlib.machinery
 import importlib.util
 PluginFolder = os.path.join(os.path.dirname(os.path.realpath(__file__)),"..","plugins")
 MainModule = "__init__"
 def get_plugin(name, plugin_path):
-    search_dirs = [PluginFolder]
+    search_dirs = [PluginFolder, '.']
    if plugin_path:
        search_dirs = [plugin_path] + search_dirs
    for dir in search_dirs:
        location = os.path.join(dir, name)
        if not os.path.isdir(location) or not MainModule + ".py" in os.listdir(location):
            continue
-        info = imp.find_module(MainModule, [location])
+        spec = importlib.machinery.PathFinder.find_spec(MainModule, [location])
-        return {"name": name, "info": info, "path": location}
+        return {"name": name, "spec": spec, "path": location}
    raise Exception("Could not find plugin with name " + name)
 def load_plugin(plugin):
-    return imp.load_module(MainModule, *plugin["info"])
+    spec = plugin["spec"]
    module = importlib.util.module_from_spec(spec)
    spec.loader.exec_module(module)
    return module
--- a/plugins/dos2unix/init.py
+++ b/plugins/dos2unix/init.py
@@ -6,6 +6,8 @@ class Filter():
        pass
    def file_data_filter(self,file_data):
        if file_data['file_ctx'] == None:
            return
        file_ctx = file_data['file_ctx']
        if not file_ctx.isbinary():
            file_data['data'] = file_data['data'].replace(b'\r\n', b'\n')
--- a/plugins/git_lfs_importer/README.md
+++ b/plugins/git_lfs_importer/README.md
@@ -0,0 +1,218 @@
 # git_lfs_importer Plugin
 This plugin automatically converts matching files to use Git LFS
 (Large File Storage) during the Mercurial to Git conversion process.
 ## Overview
 The git_lfs_importer plugin intercepts file data during the hg-fast-export
 process and converts files matching specified patterns into Git LFS pointers.
 This allows you to seamlessly migrate a Mercurial repository to Git while
 simultaneously adopting LFS for large files.
 Why use git_lfs_importer?
 For large repositories, traditional migration requires two sequential,
 long-running steps:
 1. Full history conversion from Mercurial to Git.
 2. Full history rewrite using git lfs import.
 This two-step process can take hours or even days for massive
 monorepos (e.g., 100GiB+).
 This plugin eliminates the second, time-consuming history rewrite. It performs
 the LFS conversion incrementally (Just-In-Time). During the initial export, the
 plugin identifies large files and immediately writes LFS pointers into the Git
 history. This results in significantly faster conversions and allows for
 efficient incremental imports of new changesets.
 ## Prerequisites
 ### Dependencies
 This plugin requires the `pathspec` package:
 ```bash
 pip install pathspec
 ```
 ### Git Repository Setup
 The destination Git repository must be pre-initialized with:
 1. A `.gitattributes` file configured for LFS tracking
 2. Git LFS properly installed and initialized
 Example `.gitattributes`:
 ```
 *.bin filter=lfs diff=lfs merge=lfs -text
 *.iso filter=lfs diff=lfs merge=lfs -text
 large_files/** filter=lfs diff=lfs merge=lfs -text
 ```
 ## Usage
 ### Step 1: Create the Destination Git Repository
 ```bash
 # Create a new git repository
 git init my-repo
 cd my-repo
 # Initialize Git LFS
 git lfs install
 # Create and commit a .gitattributes file
 cat > .gitattributes << EOF
 *.bin binary diff=lfs merge=lfs -text
 *.iso binary diff=lfs merge=lfs -text
 EOF
 git add .gitattributes
 git commit -m "Initialize Git LFS configuration"
 # Get the commit hash (needed for --first-commit-hash)
 git rev-parse HEAD
 ```
 ### Step 2: Create an LFS Specification File
 Create a file (e.g., `lfs-spec.txt`) listing the patterns of files to convert
 to LFS. This uses gitignore-style glob patterns:
 ```
 *.bin
 *.iso
 *.tar.gz
 large_files/**
 *.mp4
 ```
 ### Step 3: Run hg-fast-export with the Plugin
 ```bash
 hg-fast-export.sh \
  -r <mercurial-repo-path> \
  --plugin git_lfs_importer=lfs-spec.txt \
  --first-commit-hash <git-commit-hash> \
  --force
 ```
 Replace `<git-commit-hash>` with the hash obtained from Step 1.
 ## How It Works
 1. **Pattern Matching**: Files are matched against patterns in the
   LFS specification file using gitignore-style matching
 2. **File Processing**: For each matching file:
   - Calculates SHA256 hash of the file content
   - Stores the actual file content in `.git/lfs/objects/<hash-prefix>/<hash>`
   - Replaces the file data with an LFS pointer containing:
     - LFS version specification
     - SHA256 hash of the original content
     - Original file size
 3. **Git Fast-Import**: The LFS pointer is committed instead of the actual
   file content
 ## Important Notes
 ### First Commit Hash Requirement
 The `--first-commit-hash` option must be provided with the Git commit hash that
 contains your `.gitattributes` file. This allows the plugin to chain from the
 existing Git history rather than creating a completely new history.
 ### Deletions
 The plugin safely handles file deletions (data=None) and does not process them.
 ### Large Files and Largefiles
 If the Mercurial repository uses Mercurial's largefiles extension, those files
 are already converted to their original content before reaching this plugin,
 allowing the plugin to apply LFS conversion if they match the patterns.
 ## Example Workflow
 ```bash
 # Configuration variables
 HG_REPO=/path/to/mercurial/repo
 GIT_DIR_NAME=my-project-git
 LFS_PATTERN_FILE=../lfs-patterns.txt
 # 1. Prepare destination git repo
 mkdir "$GIT_DIR_NAME"
 cd "$GIT_DIR_NAME"
 git init
 git lfs install
 # Create .gitattributes
 cat > .gitattributes << EOF
 *.bin filter=lfs diff=lfs merge=lfs -text
 *.iso filter=lfs diff=lfs merge=lfs -text
 EOF
 git add .gitattributes
 git commit -m "Add LFS configuration"
 FIRST_HASH=$(git rev-parse HEAD)
 # 2. Create LFS patterns file
 cat > "$LFS_PATTERN_FILE" << EOF
 *.bin
 *.iso
 build/artifacts/**
 EOF
 # 3. Run conversion
 /path/to/hg-fast-export.sh \
  -r "$HG_REPO" \
  --plugin "git_lfs_importer=$LFS_PATTERN_FILE" \
  --first-commit-hash $FIRST_HASH \
  --force
 # 4. Verify
 git log --oneline
 git lfs ls-files
 ```
 ## Troubleshooting
 ### LFS Files Not Tracked
 Verify that:
 - The `.gitattributes` file exists in the destination repository
 - Patterns in `.gitattributes` match the files being converted
 - `git lfs install` was run in the repository
 ### "pathspec" Module Not Found
 Install the required dependency:
 ```bash
 pip install pathspec
 ```
 ### Conversion Fails at Import
 Ensure the `--first-commit-hash` value is:
 - A valid commit hash in the destination repository
 - From a commit that exists before the conversion starts
 - The hash of the commit containing `.gitattributes`
 ### Force Requirement
 You only need to pass the `--force` option when converting the *first*
 Mercurial commit into a non-empty Git repository. By default, `hg-fast-export`
 prevents importing Mercurial commits onto a non-empty Git repo to avoid
 creating conflicting histories. Passing `--force` overrides that safety check
 and allows the exporter to write the LFS pointer objects and integrate the
 converted data with the existing Git history.
 If you are doing an incremental conversion (i.e., running the script a second
 time to import new changesets into an already converted repository),
 the --force flag is not required.
 Omitting `--force` when attempting to import the first Mercurial commit into a
 non-empty repository will cause the importer to refuse the operation.
 ## See Also
 - [Git LFS Documentation](https://git-lfs.github.com/)
 - [gitignore Pattern Format](https://git-scm.com/docs/gitignore)
 - [hg-fast-export Documentation](../README.md)
--- a/plugins/git_lfs_importer/init.py
+++ b/plugins/git_lfs_importer/init.py
@@ -0,0 +1,49 @@
 import pathlib
 import hashlib
 import pathspec
 def build_filter(args):
    with open(args) as f:
        lfs_spec = pathspec.PathSpec.from_lines(pathspec.patterns.GitWildMatchPattern, f)
    return Filter(lfs_spec)
 class Filter:
    def __init__(self, lfs_spec):
        self.lfs_spec = lfs_spec
    def file_data_filter(self, file_data):
        """
        file_data: {
            'filename': <str>,
            'file_ctx': <mercurial.filectx or None>,
            'data': <bytes or None>,
            'is_largefile': <bool>
        }
        May be called for deletions (data=None, file_ctx=None).
        """
        filename = file_data.get('filename')
        data = file_data.get('data')
        # Skip deletions or filtered files early
        if data is None or not self.lfs_spec.match_file(filename.decode("utf-8")):
            return
        # Get the file path
        sha256hash = hashlib.sha256(data).hexdigest()
        lfs_path = pathlib.Path(f".git/lfs/objects/{sha256hash[0:2]}/{sha256hash[2:4]}")
        lfs_path.mkdir(parents=True, exist_ok=True)
        lfs_file_path = lfs_path / sha256hash
        # The binary blob is already in LFS
        if not lfs_file_path.is_file():
            (lfs_path / sha256hash).write_bytes(data)
        # Write the LFS pointer
        file_data['data'] = (
            f"version https://git-lfs.github.com/spec/v1\n"
            f"oid sha256:{sha256hash}\n"
            f"size {len(data)}\n"
        ).encode("utf-8")
--- a/plugins/shell_filter_file_contents/init.py
+++ b/plugins/shell_filter_file_contents/init.py
@@ -15,6 +15,8 @@ class Filter:
        d = file_data['data']
        file_ctx = file_data['file_ctx']
        filename = file_data['filename']
        if file_ctx == None:
            return
        filter_cmd = self.filter_contents + [filename, node.hex(file_ctx.filenode()), '1' if file_ctx.isbinary() else '0']
        try:
            filter_proc = subprocess.Popen(filter_cmd, stdin=subprocess.PIPE, stdout=subprocess.PIPE)
--- a/t/.gitignore
+++ b/t/.gitignore
@@ -0,0 +1 @@
 /test-results/
--- a/t/Makefile
+++ b/t/Makefile
@@ -0,0 +1,12 @@
 T = $(wildcard *.t)
 test: $(T)
 	@$(MAKE) --silent clean
 $(T): clean
 	./$@ $(TEST_OPTS)
 clean:
 	@rm -fr test-results
 .PHONY: test $(T) clean
--- a/t/file_data_filter-removefiles.expected
+++ b/t/file_data_filter-removefiles.expected
@@ -0,0 +1,30 @@
 blob
 mark :1
 data 7
 good_a
 reset refs/heads/master
 commit refs/heads/master
 mark :2
 author Grevious Bodily Harmsworth <gbh@example.com> 1679014800 +0000
 committer Grevious Bodily Harmsworth <gbh@example.com> 1679014800 +0000
 data 3
 r0
 M 100644 :1 good_a.txt
 commit refs/heads/master
 mark :3
 author Grevious Bodily Harmsworth <gbh@example.com> 1679018400 +0000
 committer Grevious Bodily Harmsworth <gbh@example.com> 1679018400 +0000
 data 3
 r1
 from :2
 commit refs/heads/master
 mark :4
 author Grevious Bodily Harmsworth <gbh@example.com> 1679022000 +0000
 committer Grevious Bodily Harmsworth <gbh@example.com> 1679022000 +0000
 data 3
 r2
 from :3
--- a/t/file_data_filter-removefiles.t
+++ b/t/file_data_filter-removefiles.t
@@ -0,0 +1,91 @@
 #!/bin/bash
 #
 # Copyright (c) 2023 Felipe Contreras
 # Copyright (c) 2023 Frej Drejhammar
 # Copyright (c) 2024 Stephan Hohe
 #
 # Check that files that file_data_filter sets to None are removed from repository
 #
 test_description='Remove files from file_data_filter plugin test'
 . "${SHARNESS_TEST_SRCDIR-$(dirname "$0")/sharness}"/sharness.sh || exit 1
 check() {
 	echo "$3" > expected &&
 	git -C "$1" show -q --format='%s' "$2" > actual &&
 	test_cmp expected actual
 }
 git_create() {
 	git init -q "$1" &&
 	git -C "$1" config core.ignoreCase false
 }
 git_convert() {
 	(
 	cd "$2" &&
 	hg-fast-export.sh --repo "../$1" \
 			  -s --hgtags -n \
 			  --plugin ../../plugins/removefiles_test_plugin
 	)
 }
 setup() {
 	cat > "$HOME"/.hgrc <<-EOF
 	[ui]
 	username = Grevious Bodily Harmsworth <gbh@example.com>
 	EOF
 }
 commit0() {
 	(
 	# Test inital revision with suppressed file
 	cd hgrepo &&
 	echo "good_a" > good_a.txt &&
 	echo "bad_a" > bad_a.txt &&
 	hg add good_a.txt bad_a.txt &&
 	hg commit -d "2023-03-17 01:00Z" -m "r0"
 	)
 }
 commit1() {
 	(
 	# Test modifying suppressed file
 	# Test adding suppressed file
 	cd hgrepo &&
 	echo "bad_a_modif" > bad_a.txt &&
 	echo "bad_b" > bad_b.txt &&
 	hg add bad_b.txt &&
 	hg commit -d "2023-03-17 02:00Z" -m "r1"
 	)
 }
 commit2() {
 	(
 	# Test removing suppressed file
 	cd hgrepo &&
 	hg rm bad_a.txt &&
 	hg commit -d "2023-03-17 03:00Z" -m "r2"
 	)
 }
 setup
 test_expect_success 'all in one' '
 	test_when_finished "rm -rf hgrepo gitrepo" &&
 	(
 	hg init hgrepo &&
 	commit0 &&
 	commit1 &&
 	commit2
 	) &&
 	git_create gitrepo &&
 	git_convert hgrepo gitrepo &&
 	git -C gitrepo fast-export --all > actual &&
 	test_cmp "$SHARNESS_TEST_DIRECTORY"/file_data_filter-removefiles.expected actual
 '
 test_done
--- a/t/file_data_filter.expected
+++ b/t/file_data_filter.expected
@@ -0,0 +1,29 @@
 blob
 mark :1
 data 7
 a_file
 blob
 mark :2
 data 17
 a_file_to_rename
 reset refs/heads/master
 commit refs/heads/master
 mark :3
 author Grevious Bodily Harmsworth <gbh@example.com> 1679014800 +0000
 committer Grevious Bodily Harmsworth <gbh@example.com> 1679014800 +0000
 data 3
 r0
 M 100644 :1 a.txt
 M 100644 :2 c.txt
 commit refs/heads/master
 mark :4
 author Grevious Bodily Harmsworth <gbh@example.com> 1679018400 +0000
 committer Grevious Bodily Harmsworth <gbh@example.com> 1679018400 +0000
 data 3
 r1
 from :3
 D c.txt
--- a/t/file_data_filter.t
+++ b/t/file_data_filter.t
@@ -0,0 +1,84 @@
 #!/bin/bash
 #
 # Copyright (c) 2023 Felipe Contreras
 # Copyright (c) 2023 Frej Drejhammar
 #
 # Check that the file_data_filter is called for removed files.
 #
 test_description='Smoke test'
 . "${SHARNESS_TEST_SRCDIR-$(dirname "$0")/sharness}"/sharness.sh || exit 1
 check() {
 	echo "$3" > expected &&
 	git -C "$1" show -q --format='%s' "$2" > actual &&
 	test_cmp expected actual
 }
 git_create() {
 	git init -q "$1" &&
 	git -C "$1" config core.ignoreCase false
 }
 git_convert() {
 	(
 	cd "$2" &&
 	hg-fast-export.sh --repo "../$1" \
 			  -s --hgtags -n \
 			  --plugin ../../plugins/rename_file_test_plugin \
 			  --plugin dos2unix \
 			  --plugin shell_filter_file_contents=../../plugins/id
 	)
 }
 setup() {
 	cat > "$HOME"/.hgrc <<-EOF
 	[ui]
 	username = Grevious Bodily Harmsworth <gbh@example.com>
 	EOF
 }
 commit0() {
 	(
 	cd hgrepo &&
 	echo "a_file" > a.txt &&
 	echo "a_file_to_rename" > b.txt &&
 	hg add a.txt b.txt &&
 	hg commit -d "2023-03-17 01:00Z" -m "r0"
 	)
 }
 commit1() {
 	(
 	cd hgrepo &&
 	hg remove b.txt &&
 	hg commit -d "2023-03-17 02:00Z" -m "r1"
 	)
 }
 make-branch() {
    hg branch "$1"
    FILE=$(echo "$1" | sha1sum | cut -d " " -f 1)
    echo "$1" > $FILE
    hg add $FILE
    hg commit -d "2023-03-17 $2:00Z" -m "Added file in branch $1"
 }
 setup
 test_expect_success 'all in one' '
 	test_when_finished "rm -rf hgrepo gitrepo" &&
 	(
 	hg init hgrepo &&
 	commit0 &&
 	commit1
 	) &&
 	git_create gitrepo &&
 	git_convert hgrepo gitrepo &&
 	git -C gitrepo fast-export --all > actual &&
 	test_cmp "$SHARNESS_TEST_DIRECTORY"/file_data_filter.expected actual
 '
 test_done
--- a/t/first_commit_hash_option.t
+++ b/t/first_commit_hash_option.t
@@ -0,0 +1,117 @@
 #!/bin/bash
 #
 # Copyright (c) 2025
 #
 test_description='git_lfs_importer plugin integration tests'
 . "${SHARNESS_TEST_SRCDIR-$(dirname "$0")/sharness}"/sharness.sh || exit 1
 setup() {
 	cat > "$HOME"/.hgrc <<-EOF
 	[ui]
 	username = Test User <test@example.com>
 	EOF
    # Git config for the destination repo commits
    git config --global user.email "test@example.com"
    git config --global user.name "Test User"
 }
 setup
 test_expect_success 'Mercurial history is imported over the provided commit' '
 	test_when_finished "rm -rf hgrepo gitrepo lfs-patterns.txt" &&
 	# 1. Create source Mercurial repository with binary files
 	(
 	hg init hgrepo &&
 	cd hgrepo &&
 	echo "regular text file" > readme.txt &&
 	hg add readme.txt &&
 	hg commit -m "initial commit"
 	) &&
 	# 2. Prepare destination git repo with LFS setup
 	mkdir gitrepo &&
 	(
 	cd gitrepo &&
 	git init -q &&
 	git config core.ignoreCase false &&
 	git lfs install --local &&
 	git switch --create master &&
 	cat > .gitattributes <<-EOF &&
 	* -text
 	EOF
 	git add .gitattributes &&
 	git commit -q -m "Initialize Git configuration"
 	) &&
 	FIRST_HASH=$(git -C gitrepo rev-parse HEAD) &&
 	# 3. Run hg-fast-export
 	(
 	cd gitrepo &&
 	hg-fast-export.sh \
 		-r "../hgrepo" \
 		--first-commit-hash "$FIRST_HASH" --force \
 		-M master
 	) &&
 	# 4. Verify git file is still present
 	git -C gitrepo show HEAD:.gitattributes > gitattributes_check.txt &&
 	test "$(cat gitattributes_check.txt)" = "* -text" &&
 	# 5. Verify hg file is imported
 	git -C gitrepo show HEAD:readme.txt > readme_check.txt &&
 	test "$(cat readme_check.txt)" = "regular text file"
 '
 test_expect_success 'Mercurial history has priority over git' '
 	test_when_finished "rm -rf hgrepo gitrepo lfs-patterns.txt" &&
 	# 1. Create source Mercurial repository with binary files
 	(
 	hg init hgrepo &&
 	cd hgrepo &&
 	echo "hg readme file" > readme.txt &&
 	hg add readme.txt &&
 	hg commit -m "initial commit"
 	) &&
 	# 2. Prepare destination git repo with LFS setup
 	mkdir gitrepo &&
 	(
 	cd gitrepo &&
 	git init -q &&
 	git config core.ignoreCase false &&
 	git lfs install --local &&
 	git switch --create master &&
 	cat > readme.txt <<-EOF &&
 	git readme file
 	EOF
 	git add readme.txt &&
 	git commit -q -m "Initialize Git readme file"
 	) &&
 	FIRST_HASH=$(git -C gitrepo rev-parse HEAD) &&
 	# 3. Run hg-fast-export
 	(
 	cd gitrepo &&
 	hg-fast-export.sh \
 		-r "../hgrepo" \
 		--first-commit-hash "$FIRST_HASH" --force \
 		-M master
 	) &&
 	# 5. Verify hg file is imported
 	git -C gitrepo show HEAD:readme.txt > readme_check.txt &&
 	test "$(cat readme_check.txt)" = "hg readme file"
 '
 test_done
--- a/t/git_lfs_importer_plugin.t
+++ b/t/git_lfs_importer_plugin.t
@@ -0,0 +1,189 @@
 #!/bin/bash
 #
 # Copyright (c) 2025
 #
 test_description='git_lfs_importer plugin integration tests'
 . "${SHARNESS_TEST_SRCDIR-$(dirname "$0")/sharness}"/sharness.sh || exit 1
 setup() {
 	cat > "$HOME"/.hgrc <<-EOF
 	[ui]
 	username = Test User <test@example.com>
 	EOF
    # Git config for the destination repo commits
    git config --global user.email "test@example.com"
    git config --global user.name "Test User"
 }
 setup
 test_expect_success 'git_lfs_importer converts matched binary files to LFS pointers and pointers are properly smudged when checkouting' '
 	test_when_finished "rm -rf hgrepo gitrepo lfs-patterns.txt" &&
 	# 1. Create source Mercurial repository with binary files
 	(
 	hg init hgrepo &&
 	cd hgrepo &&
 	echo "regular text file" > readme.txt &&
 	echo "binary payload" > payload.bin &&
 	hg add readme.txt payload.bin &&
 	hg commit -m "initial commit with binary"
 	) &&
 	# 2. Prepare destination git repo with LFS setup
 	mkdir gitrepo &&
 	(
 	cd gitrepo &&
 	git init -q &&
 	git config core.ignoreCase false &&
 	git lfs install --local &&
 	cat > .gitattributes <<-EOF &&
 	*.bin filter=lfs diff=lfs merge=lfs -text
 	EOF
 	git add .gitattributes &&
 	git commit -q -m "Initialize Git LFS configuration"
 	) &&
 	FIRST_HASH=$(git -C gitrepo rev-parse HEAD) &&
 	# 3. Create LFS patterns file
 	cat > lfs-patterns.txt <<-EOF &&
 	*.bin
 	EOF
 	# 4. Run hg-fast-export with git_lfs_importer plugin
 	(
 	cd gitrepo &&
 	hg-fast-export.sh \
 		-r "../hgrepo" \
 		--plugin "git_lfs_importer=../lfs-patterns.txt" \
 		--first-commit-hash "$FIRST_HASH" --force
 	) &&
 	# 5. Verify conversion: payload.bin should be an LFS pointer
 	git -C gitrepo show HEAD:payload.bin > lfs_pointer.txt &&
 	grep -q "version https://git-lfs.github.com/spec/v1" lfs_pointer.txt &&
 	grep -q "oid sha256:" lfs_pointer.txt &&
 	grep -q "size" lfs_pointer.txt &&
 	# 6. Verify non-matched file is unchanged
 	git -C gitrepo show HEAD:readme.txt > readme_check.txt &&
 	test "$(cat readme_check.txt)" = "regular text file" &&
 	# 7. Make sure the LFS pointer file is unsmeared when checked out
 	git -C gitrepo reset --hard HEAD &&
 	ls gitrepo &&
 	test "$(cat gitrepo/payload.bin)" = "binary payload"
 '
 test_expect_success 'git_lfs_importer skips files not matching patterns' '
 	test_when_finished "rm -rf hgrepo gitrepo lfs-patterns.txt" &&
 	# 1. Create source with various files
 	(
 	hg init hgrepo &&
 	cd hgrepo &&
 	echo "text" > file.txt &&
 	echo "data" > file.dat &&
 	echo "iso content" > image.iso &&
 	hg add . &&
 	hg commit -m "multiple files"
 	) &&
 	# 2. Prepare git repo with LFS
 	mkdir gitrepo &&
 	(
 	cd gitrepo &&
 	git init -q &&
 	git config core.ignoreCase false &&
 	git lfs install --local &&
 	cat > .gitattributes <<-EOF &&
 	*.iso filter=lfs diff=lfs merge=lfs -text
 	EOF
 	git add .gitattributes &&
 	git commit -q -m "Initialize Git LFS configuration"
 	) &&
 	FIRST_HASH=$(git -C gitrepo rev-parse HEAD) &&
 	# 3. Only .iso files should be converted
 	cat > lfs-patterns.txt <<-EOF &&
 	*.iso
 	EOF
 	(
 	cd gitrepo &&
 	hg-fast-export.sh \
 		-r "../hgrepo" \
 		--plugin "git_lfs_importer=../lfs-patterns.txt" \
 		--first-commit-hash "$FIRST_HASH" --force
 	) &&
 	# 4. Verify .iso is LFS pointer
 	git -C gitrepo show HEAD:image.iso | grep -q "oid sha256:" &&
 	# 5. Verify .txt and .dat are unchanged
 	test "$(git -C gitrepo show HEAD:file.txt)" = "text" &&
 	test "$(git -C gitrepo show HEAD:file.dat)" = "data"
 '
 test_expect_success 'git_lfs_importer handles directory patterns' '
 	test_when_finished "rm -rf hgrepo gitrepo lfs-patterns.txt" &&
 	# 1. Create repo with files in directory
 	(
 	hg init hgrepo &&
 	cd hgrepo &&
 	mkdir -p assets/images &&
 	echo "logo data" > assets/images/logo.bin &&
 	echo "regular" > readme.txt &&
 	hg add . &&
 	hg commit -m "files in directories"
 	) &&
 	# 2. Prepare git repo
 	mkdir gitrepo &&
 	(
 	cd gitrepo &&
 	git init -q &&
 	git config core.ignoreCase false &&
 	git lfs install --local &&
 	cat > .gitattributes <<-EOF &&
 	assets/** filter=lfs diff=lfs merge=lfs -text
 	EOF
 	git add .gitattributes &&
 	git commit -q -m "Initialize Git LFS configuration"
 	) &&
 	FIRST_HASH=$(git -C gitrepo rev-parse HEAD) &&
 	# 3. Match directory pattern
 	cat > lfs-patterns.txt <<-EOF &&
 	assets/**
 	EOF
 	(
 	cd gitrepo &&
 	hg-fast-export.sh \
 		-r "../hgrepo" \
 		--plugin "git_lfs_importer=../lfs-patterns.txt" \
 		--first-commit-hash "$FIRST_HASH" --force
 	) &&
 	# 4. Verify directory file is converted
 	git -C gitrepo show HEAD:assets/images/logo.bin | grep -q "oid sha256:" &&
 	# 5. Verify file outside directory is unchanged
 	test "$(git -C gitrepo show HEAD:readme.txt)" = "regular"
 '
 test_done
--- a/t/largefile_plugin.expected
+++ b/t/largefile_plugin.expected
@@ -0,0 +1,20 @@
 blob
 mark :1
 data 7
 a_file
 blob
 mark :2
 data 6
 large
 reset refs/heads/master
 commit refs/heads/master
 mark :3
 author Grevious Bodily Harmsworth <gbh@example.com> 1679014800 +0000
 committer Grevious Bodily Harmsworth <gbh@example.com> 1679014800 +0000
 data 3
 r0
 M 100644 :1 a.txt
 M 100644 :2 b.txt
--- a/t/largefile_plugin.t
+++ b/t/largefile_plugin.t
@@ -0,0 +1,69 @@
 #!/bin/bash
 #
 # Copyright (c) 2023 Felipe Contreras
 # Copyright (c) 2023 Frej Drejhammar
 # Copyright (c) 2025 Günther Nußmüller
 #
 # Check that plugin invocation works with largefiles.
 # This test uses the echo_file_data_test_plugin to verify that the
 # file data is passed correctly, including the largefile status.
 #
 test_description='Largefiles and plugin test'
 . "${SHARNESS_TEST_SRCDIR-$(dirname "$0")/sharness}"/sharness.sh || exit 1
 git_create() {
 	git init -q "$1" &&
 	git -C "$1" config core.ignoreCase false
 }
 git_convert() {
 	(
 	cd "$2" &&
 	hg-fast-export.sh --repo "../$1" \
 			  -s --hgtags -n \
 			  --plugin ../../plugins/echo_file_data_test_plugin
 	)
 }
 setup() {
 	cat > "$HOME"/.hgrc <<-EOF
 	[ui]
 	username = Grevious Bodily Harmsworth <gbh@example.com>
 	[extensions]
 	largefiles =
 	EOF
 }
 commit0() {
 	(
 	cd hgrepo &&
 	echo "a_file" > a.txt &&
 	echo "large" > b.txt
 	hg add a.txt &&
 	hg add --large b.txt &&
 	hg commit -d "2023-03-17 01:00Z" -m "r0"
 	)
 }
 setup
 test_expect_success 'largefile and plugin' '
 	test_when_finished "rm -rf hgrepo gitrepo" &&
 	(
 	hg init hgrepo &&
 	commit0
 	) &&
 	git_create gitrepo &&
 	git_convert hgrepo gitrepo &&
 	git -C gitrepo fast-export --all > actual &&
 	test_cmp "$SHARNESS_TEST_DIRECTORY"/largefile_plugin.expected actual &&
 	test_cmp "$SHARNESS_TEST_DIRECTORY"/largefile_plugin_file_info.expected gitrepo/largefile_info.txt
 '
 test_done
--- a/t/largefile_plugin_file_info.expected
+++ b/t/largefile_plugin_file_info.expected
@@ -0,0 +1,12 @@
 filename: b'b.txt'
 data size: 6 bytes
 ctx rev: 0
 ctx binary: False
 is largefile: True
 filename: b'a.txt'
 data size: 7 bytes
 ctx rev: 0
 ctx binary: False
 is largefile: False
--- a/t/main.t
+++ b/t/main.t
@@ -0,0 +1,144 @@
 #!/bin/bash
 #
 # Copyright (c) 2023 Felipe Contreras
 #
 test_description='Main tests'
 . "${SHARNESS_TEST_SRCDIR-$(dirname "$0")/sharness}"/sharness.sh || exit 1
 check() {
 	echo "$3" > expected &&
 	git -C "$1" show -q --format='%s' "$2" > actual &&
 	test_cmp expected actual
 }
 git_clone() {
 	(
 	git init -q "$2" &&
 	cd "$2" &&
 	git config core.ignoreCase false &&
 	hg-fast-export.sh --repo "../$1"
 	)
 }
 setup() {
 	cat > "$HOME"/.hgrc <<-EOF
 	[ui]
 	username = H G Wells <wells@example.com>
 	EOF
 }
 setup
 test_expect_success 'basic' '
 	test_when_finished "rm -rf hgrepo gitrepo" &&
 	(
 	hg init hgrepo &&
 	cd hgrepo &&
 	echo zero > content &&
 	hg add content &&
 	hg commit -m zero
 	) &&
 	git_clone hgrepo gitrepo &&
 	check gitrepo @ zero
 '
 test_expect_success 'merge' '
 	test_when_finished "rm -rf hgrepo gitrepo" &&
 	(
 	hg init hgrepo &&
 	cd hgrepo &&
 	echo a > content &&
 	echo a > file1 &&
 	hg add content file1 &&
 	hg commit -m "origin" &&
 	echo b > content &&
 	echo b > file2 &&
 	hg add file2 &&
 	hg rm file1 &&
 	hg commit -m "right" &&
 	hg update -r0 &&
 	echo c > content &&
 	hg commit -m "left" &&
 	HGMERGE=true hg merge -r1 &&
 	hg commit -m "merge"
 	) &&
 	git_clone hgrepo gitrepo &&
 	cat > expected <<-EOF &&
 	left
 	c
 	tree @:
 	content
 	file2
 	EOF
 	(
 	cd gitrepo
 	git show -q --format='%s' @^ &&
 	git show @:content &&
 	git show @:
 	) > actual &&
 	test_cmp expected actual
 '
 test_expect_success 'hg large file' '
 	test_when_finished "rm -rf hgrepo gitrepo" &&
 	(
 	hg init hgrepo &&
 	cd hgrepo &&
 	echo "[extensions]" >> .hg/hgrc
 	echo "largefiles =" >> .hg/hgrc
 	echo a > content &&
 	echo a > file1 &&
 	hg add content &&
 	hg add --large file1 &&
 	hg commit -m "origin" &&
 	echo b > content &&
 	echo b > file2 &&
 	hg add --large file2 &&
 	hg rm file1 &&
 	hg commit -m "right" &&
 	hg update -r0 &&
 	echo c > content &&
 	hg commit -m "left" &&
 	HGMERGE=true hg merge -r1 &&
 	hg commit -m "merge"
 	) &&
 	git_clone hgrepo gitrepo &&
 	cat > expected <<-EOF &&
 	left
 	c
 	tree @:
 	content
 	file2
 	EOF
 	(
 	cd gitrepo
 	git show -q --format='%s' @^ &&
 	git show @:content &&
 	git show @:
 	) > actual &&
 	test_cmp expected actual
 '
 test_done
--- a/t/plugins/echo_file_data_test_plugin/init.py
+++ b/t/plugins/echo_file_data_test_plugin/init.py
@@ -0,0 +1,18 @@
 import sys
 from mercurial import node
 def build_filter(args):
    return Filter(args)
 class Filter:
    def __init__(self, _):
        pass
    def file_data_filter(self,file_data):
        with open('largefile_info.txt', 'a') as f:
            f.write(f"filename: {file_data['filename']}\n")
            f.write(f"data size: {len(file_data['data'])} bytes\n")
            f.write(f"ctx rev: {file_data['file_ctx'].rev()}\n")
            f.write(f"ctx binary: {file_data['file_ctx'].isbinary()}\n")
            f.write(f"is largefile: {file_data.get('is_largefile', False)}\n")
            f.write("\n")
--- a/t/plugins/id
+++ b/t/plugins/id
@@ -0,0 +1,2 @@
 #!/bin/bash
 cat
--- a/t/plugins/removefiles_test_plugin/init.py
+++ b/t/plugins/removefiles_test_plugin/init.py
@@ -0,0 +1,15 @@
 import subprocess
 import shlex
 import sys
 from mercurial import node
 def build_filter(args):
    return Filter(args)
 class Filter:
    def __init__(self, args):
        self.filter_contents = shlex.split(args)
    def file_data_filter(self,file_data):
        if file_data['filename'].startswith(b'bad'):
            file_data['data'] = None
--- a/t/plugins/rename_file_test_plugin/init.py
+++ b/t/plugins/rename_file_test_plugin/init.py
@@ -0,0 +1,15 @@
 import subprocess
 import shlex
 import sys
 from mercurial import node
 def build_filter(args):
    return Filter(args)
 class Filter:
    def __init__(self, args):
        self.filter_contents = shlex.split(args)
    def file_data_filter(self,file_data):
        if file_data['filename'] == b'b.txt':
            file_data['filename'] = b'c.txt'
--- a/t/set_origin.expected
+++ b/t/set_origin.expected
@@ -0,0 +1,42 @@
 blob
 mark :1
 data 5
 zero
 reset refs/heads/prefix/master
 commit refs/heads/prefix/master
 mark :2
 author H G Wells <wells@example.com> 1679014800 +0000
 committer H G Wells <wells@example.com> 1679014800 +0000
 data 5
 zero
 M 100644 :1 content
 blob
 mark :3
 data 8
 branch1
 commit refs/heads/prefix/branch1
 mark :4
 author H G Wells <wells@example.com> 1679018400 +0000
 committer H G Wells <wells@example.com> 1679018400 +0000
 data 29
 Added file in branch branch1
 from :2
 M 100644 :3 b8486c4feca589a4237a1ee428322d7109ede12e
 blob
 mark :5
 data 8
 branch2
 commit refs/heads/prefix/branch2
 mark :6
 author H G Wells <wells@example.com> 1679022000 +0000
 committer H G Wells <wells@example.com> 1679022000 +0000
 data 29
 Added file in branch branch2
 from :4
 M 100644 :5 fe786baee0d76603092c25609f2967b9c28a2cf2
--- a/t/set_origin.t
+++ b/t/set_origin.t
@@ -0,0 +1,59 @@
 #!/bin/bash
 #
 # Copyright (c) 2023 Felipe Contreras
 # Copyright (c) 2025 Günther Nußmüller
 #
 test_description='Set origin tests'
 . "${SHARNESS_TEST_SRCDIR-$(dirname "$0")/sharness}"/sharness.sh || exit 1
 check() {
 	git -C "$1" fast-export --all > actual
 	test_cmp "$SHARNESS_TEST_DIRECTORY"/set_origin.expected actual
 }
 git_clone() {
 	(
 	git init -q "$2" &&
 	cd "$2" &&
 	git config core.ignoreCase false &&
 	hg-fast-export.sh --repo "../$1" --origin "$3"
 	)
 }
 setup() {
 	cat > "$HOME"/.hgrc <<-EOF
 	[ui]
 	username = H G Wells <wells@example.com>
 	EOF
 }
 make-branch() {
 	hg branch "$1"
 	FILE=$(echo "$1" | sha1sum | cut -d " " -f 1)
 	echo "$1" > $FILE
 	hg add $FILE
 	hg commit -d "2023-03-17 $2:00Z" -m "Added file in branch $1"
 }
 setup
 test_expect_success 'basic' '
 	test_when_finished "rm -rf hgrepo gitrepo" &&
 	(
 	hg init hgrepo &&
 	cd hgrepo &&
 	echo zero > content &&
 	hg add content &&
 	hg commit -m zero -d "2023-03-17 01:00Z" &&
 	make-branch branch1 02 &&
 	make-branch branch2 03
 	) &&
 	git_clone hgrepo gitrepo prefix &&
 	check gitrepo
 '
 test_done
--- a/t/sharness
+++ b/t/sharness
--- a/t/smoke-test.branchmap
+++ b/t/smoke-test.branchmap
@@ -0,0 +1,15 @@
 "feature"="renamed-feature"
 "a?"="valid-0"
 "a/"="valid-1"
 "a/b"="valid-2"
 "a/?"="valid-3"
 "?a"="valid-4"
 "a."="valid-5"
 "a.b"="valid-6"
 ".a"="valid-7"
 "/"="valid-8"
 "___3"="___a"
 "__2"="__b"
 "_1"="_c"
 "åäö"="abc"
 "Feature- 12V Vac \"Venom\""="venom"
--- a/t/smoke-test.expected
+++ b/t/smoke-test.expected
@@ -0,0 +1,300 @@
 blob
 mark :1
 data 5
 r0-a
 blob
 mark :2
 data 5
 r0-b
 reset refs/heads/master
 commit refs/heads/master
 mark :3
 author Grevious Bodily Harmsworth <gbh@example.com> 1679014800 +0000
 committer Grevious Bodily Harmsworth <gbh@example.com> 1679014800 +0000
 data 3
 r0
 M 100644 :1 a.txt
 M 100644 :2 b.txt
 blob
 mark :4
 data 5
 r1-c
 blob
 mark :5
 data 5
 r1-d
 commit refs/tags/2019_Spring_R2
 mark :6
 author Grevious Bodily Harmsworth <gbh@example.com> 1679018400 +0000
 committer Grevious Bodily Harmsworth <gbh@example.com> 1679018400 +0000
 data 3
 r1
 from :3
 M 100644 :4 c.txt
 M 100644 :5 d.txt
 blob
 mark :7
 data 56
 e92e41dde44f9dbbac08bbb83351a65b6728f128 2019 Spring R2
 commit refs/heads/mainline
 mark :8
 author Grevious Bodily Harmsworth <gbh@example.com> 1679019000 +0000
 committer Grevious Bodily Harmsworth <gbh@example.com> 1679019000 +0000
 data 52
 Added tag 2019 Spring R2 for changeset e92e41dde44f
 from :6
 M 100644 :7 .hgtags
 blob
 mark :9
 data 5
 r2-e
 blob
 mark :10
 data 5
 r2-f
 commit refs/heads/mainline
 mark :11
 author Grevious Bodily Harmsworth <gbh@example.com> 1679022000 +0000
 committer Grevious Bodily Harmsworth <gbh@example.com> 1679022000 +0000
 data 3
 r2
 from :8
 M 100644 :9 e.txt
 M 100644 :10 f.txt
 commit refs/heads/mainline
 mark :12
 author badly-formed-user <devnull@localhost> 1679025600 +0000
 committer badly-formed-user <devnull@localhost> 1679025600 +0000
 data 3
 r3
 from :11
 M 100644 :9 g.txt
 M 100644 :10 h.txt
 blob
 mark :13
 data 10
 feature-a
 blob
 mark :14
 data 10
 feature-b
 commit refs/heads/renamed-feature
 mark :15
 author Grevious Bodily Harmsworth <gbh@example.com> 1679029200 +0000
 committer Grevious Bodily Harmsworth <gbh@example.com> 1679029200 +0000
 data 8
 feature
 from :12
 M 100644 :13 feature-a.txt
 M 100644 :14 feature-b.txt
 blob
 mark :16
 data 3
 a?
 commit refs/heads/valid-0
 mark :17
 author Grevious Bodily Harmsworth <gbh@example.com> 1679032800 +0000
 committer Grevious Bodily Harmsworth <gbh@example.com> 1679032800 +0000
 data 24
 Added file in branch a?
 from :15
 M 100644 :16 c1086ce03e4f52aadd1c93b1d097da510138522a
 blob
 mark :18
 data 3
 a/
 commit refs/heads/valid-1
 mark :19
 author Grevious Bodily Harmsworth <gbh@example.com> 1679036400 +0000
 committer Grevious Bodily Harmsworth <gbh@example.com> 1679036400 +0000
 data 24
 Added file in branch a/
 from :17
 M 100644 :18 85ed6fbb96d655df9f194bc9107f2d86210b9263
 blob
 mark :20
 data 4
 a/b
 commit refs/heads/valid-2
 mark :21
 author Grevious Bodily Harmsworth <gbh@example.com> 1679040000 +0000
 committer Grevious Bodily Harmsworth <gbh@example.com> 1679040000 +0000
 data 25
 Added file in branch a/b
 from :19
 M 100644 :20 aae42d317509399fdda80c4d8e46774d152dbd04
 blob
 mark :22
 data 4
 a/?
 commit refs/heads/valid-3
 mark :23
 author Grevious Bodily Harmsworth <gbh@example.com> 1679043600 +0000
 committer Grevious Bodily Harmsworth <gbh@example.com> 1679043600 +0000
 data 25
 Added file in branch a/?
 from :21
 M 100644 :22 ba54a8de7fe91c5e6e0a2dd1b9b37de0976ff5a7
 blob
 mark :24
 data 3
 ?a
 commit refs/heads/valid-4
 mark :25
 author Grevious Bodily Harmsworth <gbh@example.com> 1679047200 +0000
 committer Grevious Bodily Harmsworth <gbh@example.com> 1679047200 +0000
 data 24
 Added file in branch ?a
 from :23
 M 100644 :24 d4cde16119b586025976741e87775762a2598984
 blob
 mark :26
 data 3
 a.
 commit refs/heads/valid-5
 mark :27
 author Grevious Bodily Harmsworth <gbh@example.com> 1679050800 +0000
 committer Grevious Bodily Harmsworth <gbh@example.com> 1679050800 +0000
 data 24
 Added file in branch a.
 from :25
 M 100644 :26 b4ce96ddcee0706a8c51130917f910b2b29faf77
 blob
 mark :28
 data 4
 a.b
 commit refs/heads/valid-6
 mark :29
 author Grevious Bodily Harmsworth <gbh@example.com> 1679054400 +0000
 committer Grevious Bodily Harmsworth <gbh@example.com> 1679054400 +0000
 data 25
 Added file in branch a.b
 from :27
 M 100644 :28 97051191e1a92daa11165ef10770bf964268c58b
 blob
 mark :30
 data 3
 .a
 commit refs/heads/valid-7
 mark :31
 author Grevious Bodily Harmsworth <gbh@example.com> 1679058000 +0000
 committer Grevious Bodily Harmsworth <gbh@example.com> 1679058000 +0000
 data 24
 Added file in branch .a
 from :29
 M 100644 :30 a667f8feec02fdfa6649772f844a24cf1ad5ebec
 blob
 mark :32
 data 2
 /
 commit refs/heads/valid-8
 mark :33
 author Grevious Bodily Harmsworth <gbh@example.com> 1679061600 +0000
 committer Grevious Bodily Harmsworth <gbh@example.com> 1679061600 +0000
 data 23
 Added file in branch /
 from :31
 M 100644 :32 8f27084b6294ddbe28dbcbf98f798730e8a79289
 blob
 mark :34
 data 5
 ___3
 commit refs/heads/___a
 mark :35
 author Grevious Bodily Harmsworth <gbh@example.com> 1679065200 +0000
 committer Grevious Bodily Harmsworth <gbh@example.com> 1679065200 +0000
 data 26
 Added file in branch ___3
 from :33
 M 100644 :34 9b171494eb6e5ce325934b1656e286ca0510a697
 blob
 mark :36
 data 4
 __2
 commit refs/heads/__b
 mark :37
 author Grevious Bodily Harmsworth <gbh@example.com> 1679068800 +0000
 committer Grevious Bodily Harmsworth <gbh@example.com> 1679068800 +0000
 data 25
 Added file in branch __2
 from :35
 M 100644 :36 5dca703b71d2613c6bb3262b9b1741d6165e4a2f
 blob
 mark :38
 data 3
 _1
 commit refs/heads/_c
 mark :39
 author Grevious Bodily Harmsworth <gbh@example.com> 1679072400 +0000
 committer Grevious Bodily Harmsworth <gbh@example.com> 1679072400 +0000
 data 24
 Added file in branch _1
 from :37
 M 100644 :38 2fee90e148a2afbd911b67ced9b6240151f904ec
 blob
 mark :40
 data 25
 Feature- 12V Vac "Venom"
 commit refs/heads/venom
 mark :41
 author Grevious Bodily Harmsworth <gbh@example.com> 1679076000 +0000
 committer Grevious Bodily Harmsworth <gbh@example.com> 1679076000 +0000
 data 46
 Added file in branch Feature- 12V Vac "Venom"
 from :39
 M 100644 :40 b01def8779aed4be2f4b7325a89992a9aa566fec
 blob
 mark :42
 data 7
 åäö
 commit refs/heads/abc
 mark :43
 author Grevious Bodily Harmsworth <gbh@example.com> 1679079600 +0000
 committer Grevious Bodily Harmsworth <gbh@example.com> 1679079600 +0000
 data 28
 Added file in branch åäö
 from :41
 M 100644 :42 a0d01fcbff5d86327d542687dcfd8b299d054147
--- a/t/smoke-test.t
+++ b/t/smoke-test.t
@@ -0,0 +1,163 @@
 #!/bin/bash
 #
 # Copyright (c) 2023 Felipe Contreras
 # Copyright (c) 2023 Frej Drejhammar
 #
 # Smoke test used to sanity test changes to fast-export.
 #
 test_description='Smoke test'
 . "${SHARNESS_TEST_SRCDIR-$(dirname "$0")/sharness}"/sharness.sh || exit 1
 check() {
 	echo "$3" > expected &&
 	git -C "$1" show -q --format='%s' "$2" > actual &&
 	test_cmp expected actual
 }
 git_create() {
 	git init -q "$1" &&
 	git -C "$1" config core.ignoreCase false
 }
 git_convert() {
 	(
 	cd "$2" &&
 	hg-fast-export.sh --repo "../$1" \
 			  -s --hgtags -n \
 			  -B "$SHARNESS_TEST_DIRECTORY"/smoke-test.branchmap \
 			  -T "$SHARNESS_TEST_DIRECTORY"/smoke-test.tagsmap
 	)
 }
 setup() {
 	cat > "$HOME"/.hgrc <<-EOF
 	[ui]
 	username = Grevious Bodily Harmsworth <gbh@example.com>
 	EOF
 }
 commit0() {
 	(
 	cd hgrepo &&
 	echo "r0-a" > a.txt &&
 	echo "r0-b" > b.txt &&
 	hg add a.txt b.txt &&
 	hg commit -d "2023-03-17 01:00Z" -m "r0" &&
 	hg bookmark bm0
 	)
 }
 commit1() {
 	(
 	cd hgrepo &&
 	echo "r1-c" > c.txt &&
 	echo "r1-d" > d.txt &&
 	hg branch mainline &&
 	hg add c.txt d.txt &&
 	hg commit -d "2023-03-17 02:00Z" -m "r1" &&
 	hg tag -d "2023-03-17 02:10Z" "2019 Spring R2"
 	)
 }
 commit2() {
 	(
 	cd hgrepo &&
 	echo "r2-e" > e.txt &&
 	echo "r2-f" > f.txt &&
 	hg add e.txt f.txt &&
 	hg commit -d "2023-03-17 03:00Z" -m "r2" &&
 	hg bookmark bm1
 	)
 }
 commit3() {
 	(
 	cd hgrepo &&
 	echo "r2-e" > g.txt &&
 	echo "r2-f" > h.txt &&
 	hg add g.txt h.txt &&
 	hg commit -d "2023-03-17 04:00Z" -u "badly-formed-user" -m "r3"
 	)
 }
 commit_rest() {
 	(
 	cd hgrepo &&
 	hg branch feature &&
 	echo "feature-a" > feature-a.txt &&
 	echo "feature-b" > feature-b.txt &&
 	hg add feature-a.txt feature-b.txt &&
 	hg commit -d "2023-03-17 05:00Z" -m "feature" &&
 	hg bookmark bm2 &&
 	# Now create strangely named branches
 	make-branch "a?" 06 &&
 	make-branch "a/" 07 &&
 	make-branch "a/b" 08 &&
 	make-branch "a/?" 09 &&
 	make-branch "?a" 10 &&
 	make-branch "a." 11 &&
 	make-branch "a.b" 12 &&
 	make-branch ".a" 13 &&
 	make-branch "/" 14 &&
 	make-branch "___3" 15 &&
 	make-branch "__2" 16 &&
 	make-branch "_1" 17 &&
 	make-branch "Feature- 12V Vac \"Venom\"" 18 &&
 	make-branch "åäö" 19 &&
 	hg bookmark bm-for-the-rest
 	)
 }
 make-branch() {
    hg branch "$1"
    FILE=$(echo "$1" | sha1sum | cut -d " " -f 1)
    echo "$1" > $FILE
    hg add $FILE
    hg commit -d "2023-03-17 $2:00Z" -m "Added file in branch $1"
 }
 setup
 test_expect_success 'all in one' '
 	test_when_finished "rm -rf hgrepo gitrepo" &&
 	(
 	hg init hgrepo &&
 	commit0 &&
 	commit1 &&
 	commit2 &&
 	commit3 &&
 	commit_rest
 	) &&
 	git_create gitrepo &&
 	git_convert hgrepo gitrepo &&
 	git -C gitrepo fast-export --all > actual &&
 	test_cmp "$SHARNESS_TEST_DIRECTORY"/smoke-test.expected actual
 '
 test_expect_success 'incremental' '
 	test_when_finished "rm -rf hgrepo gitrepo" &&
 	hg init hgrepo &&
 	commit0 &&
 	git_create gitrepo &&
 	git_convert hgrepo gitrepo &&
 	commit1 &&
 	git_convert hgrepo gitrepo &&
 	commit2 &&
 	commit3 &&
 	git_convert hgrepo gitrepo &&
 	commit_rest &&
 	git_convert hgrepo gitrepo &&
 	git -C gitrepo fast-export --all > actual &&
 	test_cmp "$SHARNESS_TEST_DIRECTORY"/smoke-test.expected actual
 '
 test_done
--- a/t/smoke-test.tagsmap
+++ b/t/smoke-test.tagsmap
@@ -0,0 +1 @@
 "2019 Spring R2"="2019_Spring_R2"
--- a/tests/test_git_lfs_importer_plugin.py
+++ b/tests/test_git_lfs_importer_plugin.py
@@ -0,0 +1,156 @@
 import sys
 sys.path.append("./plugins")
 import hashlib
 import pathlib
 import time
 import unittest
 import tempfile
 import os
 import pathspec
 from git_lfs_importer import Filter, build_filter
 class TestGitLfsImporterPlugin(unittest.TestCase):
    def setUp(self):
        # create an isolated temp dir and chdir into it for each test
        self._orig_cwd = os.getcwd()
        self._tmpdir = tempfile.TemporaryDirectory()
        self.tmp_path = pathlib.Path(self._tmpdir.name)
        os.chdir(self.tmp_path)
    def tearDown(self):
        # restore cwd and cleanup
        os.chdir(self._orig_cwd)
        self._tmpdir.cleanup()
    def empty_spec(self):
        return pathspec.PathSpec.from_lines(pathspec.patterns.GitWildMatchPattern, [])
    # --------------------------------------------------------
    # GIVEN-WHEN-THEN TESTS for Filter.file_data_filter
    # --------------------------------------------------------
    def test_skips_deletions(self):
        flt = Filter(self.empty_spec())
        file_data = {"filename": b"file.txt", "data": None}
        flt.file_data_filter(file_data)
        self.assertIsNone(file_data["data"])
        self.assertFalse((self.tmp_path / ".git").exists())
    def test_skips_files_that_do_not_match_spec(self):
        spec = pathspec.PathSpec.from_lines(pathspec.patterns.GitWildMatchPattern, ["*.bin"])
        flt = Filter(spec)
        original = b"not matched"
        file_data = {"filename": b"file.txt", "data": original}
        flt.file_data_filter(file_data)
        self.assertEqual(file_data["data"], original)
        self.assertFalse((self.tmp_path / ".git").exists())
    def test_converts_only_matched_files_to_lfs_pointer(self):
        spec = pathspec.PathSpec.from_lines(pathspec.patterns.GitWildMatchPattern, ["*.bin"])
        flt = Filter(spec)
        data = b"hello world"
        sha = hashlib.sha256(data).hexdigest()
        expected_pointer = (
            f"version https://git-lfs.github.com/spec/v1\n"
            f"oid sha256:{sha}\n"
            f"size {len(data)}\n"
        ).encode("utf-8")
        file_data = {"filename": b"payload.bin", "data": data}
        flt.file_data_filter(file_data)
        self.assertEqual(file_data["data"], expected_pointer)
        lfs_file = pathlib.Path(".git/lfs/objects") / sha[:2] / sha[2:4] / sha
        self.assertTrue(lfs_file.is_file())
        self.assertEqual(lfs_file.read_bytes(), data)
    def test_does_not_convert_unmatched_directory(self):
        spec = pathspec.PathSpec.from_lines(pathspec.patterns.GitWildMatchPattern, ["assets/**"])
        flt = Filter(spec)
        data = b"outside directory"
        file_data = {"filename": b"src/images/logo.png", "data": data}
        flt.file_data_filter(file_data)
        self.assertEqual(file_data["data"], data)
        self.assertFalse((self.tmp_path / ".git").exists())
    def test_converts_matched_directory(self):
        spec = pathspec.PathSpec.from_lines(pathspec.patterns.GitWildMatchPattern, ["assets/**"])
        flt = Filter(spec)
        data = b"inside directory"
        sha = hashlib.sha256(data).hexdigest()
        file_data = {"filename": b"assets/images/logo.png", "data": data}
        flt.file_data_filter(file_data)
        self.assertIn(b"version https://git-lfs.github.com/spec/v1", file_data["data"])
        lfs_file = pathlib.Path(".git/lfs/objects") / sha[:2] / sha[2:4] / sha
        self.assertTrue(lfs_file.is_file())
        self.assertEqual(lfs_file.read_bytes(), data)
    def test_does_not_overwrite_existing_blob(self):
        spec = pathspec.PathSpec.from_lines(pathspec.patterns.GitWildMatchPattern, ["*.bin"])
        flt = Filter(spec)
        data = b"abc"
        sha = hashlib.sha256(data).hexdigest()
        lfs_dir = pathlib.Path(".git/lfs/objects") / sha[:2] / sha[2:4]
        lfs_dir.mkdir(parents=True, exist_ok=True)
        lfs_file = lfs_dir / sha
        lfs_file.write_bytes(data)
        before_mtime = lfs_file.stat().st_mtime_ns
        time.sleep(0.01)  # Ensure timestamp difference
        file_data = {"filename": b"abc.bin", "data": data}
        flt.file_data_filter(file_data)
        expected_pointer_prefix = b"version https://git-lfs.github.com/spec/v1"
        self.assertTrue(file_data["data"].startswith(expected_pointer_prefix))
        after_mtime = lfs_file.stat().st_mtime_ns
        self.assertEqual(after_mtime, before_mtime)
    def test_empty_file_converted_when_matched(self):
        spec = pathspec.PathSpec.from_lines(pathspec.patterns.GitWildMatchPattern, ["*.bin"])
        flt = Filter(spec)
        data = b""
        sha = hashlib.sha256(data).hexdigest()
        file_data = {"filename": b"empty.bin", "data": data}
        flt.file_data_filter(file_data)
        self.assertIn(b"size 0", file_data["data"])
        lfs_file = pathlib.Path(".git/lfs/objects") / sha[:2] / sha[2:4] / sha
        self.assertTrue(lfs_file.is_file())
        self.assertEqual(lfs_file.read_bytes(), data)
    # --------------------------------------------------------
    # Optional: GIVEN-WHEN-THEN for build_filter
    # --------------------------------------------------------
    def test_build_filter_reads_patterns_file(self):
        patterns_file = self.tmp_path / "lfs_patterns.txt"
        patterns_file.write_text("*.bin\nassets/**\n", encoding="utf-8")
        flt = build_filter(str(patterns_file))
        data_match = b"match me"
        sha_match = hashlib.sha256(data_match).hexdigest()
        fd_match = {"filename": b"assets/payload.bin", "data": data_match}
        flt.file_data_filter(fd_match)
        self.assertIn(b"oid sha256:", fd_match["data"])
        lfs_file = pathlib.Path(".git/lfs/objects") / sha_match[:2] / sha_match[2:4] / sha_match
        self.assertTrue(lfs_file.is_file())
        data_skip = b"skip me"
        fd_skip = {"filename": b"docs/readme.md", "data": data_skip}
        flt.file_data_filter(fd_skip)
        self.assertEqual(fd_skip["data"], data_skip)
Author	SHA1	Message	Date
Frej Drejhammar	c87b66ed7d	Merge branch 'frej/fix-gh348'	2026-02-14 20:55:09 +01:00
Frej Drejhammar	76db75d963	Support Mercurial 7.2 In Mercurial 7.2 the iteritems() method of the branchmap has been removed. Switch to iterating over the branches and then fetching heads by the branchheads() method. In 7.2, a call to mercurial.initialization.init() is also needed to process a repo using the largefiles extension. Thanks to Michael Cho (@cho-m) for suggesting the initialization fix. Closes #348 Co-developed-by: Michael Cho <michael@michaelcho.dev>	2026-02-14 20:53:20 +01:00
Frej Drejhammar	5c9068a1f1	Merge branch 'frej/gh347'	2026-01-17 20:38:59 +01:00
Kévin Lévesque	42d1c89e73	Split plugin data example into multiple lines The data example was previously a single line, making it difficult to read. Replaced the one-liner with a multi-line format to improve clarity.	2026-01-12 16:33:37 -05:00
Kévin Lévesque	9d71921ed8	Allow incremental Git LFS conversion via plugin Converts large Mercurial repositories to Git/LFS significantly faster by integrating the LFS conversion into the history export process. Currently, converting large repositories requires two sequential, long-running steps: 1. Full history conversion (`hg` to `git`). 2. Full history rewrite/import (`git lfs import`). For huge monorepos (100GiB+, 1M+ files), this sequence can take hours or days. This commit introduces a new plugin that allows the repository to be converted incrementally (JIT: Just-In-Time). The plugin identifies large files during the initial `hg` to `git` conversion and immediately writes LFS pointers, eliminating the need for the second, time-consuming history rewrite step.	2026-01-12 16:33:20 -05:00
Kévin Lévesque	f6b72d248f	Allow specifying a repository root commit for conversion The current conversion process mandates an empty repository for a clean start. This presents a barrier to performance optimization strategies. This change introduces the ability to pass a repository root commit hash. This is necessary to support the immediate next commit (Incremental LFS conversion), which uses a `.gitattributes` file and LFS pointers to bypass the slow, full-history rewriting often required on large non-empty monorepos (100GiB+, 1M+ files). The immediate benefit is allowing conversion to start when a non-empty repo already contains an orphan commit, laying the groundwork for the optimized LFS conversion feature.	2025-12-22 10:11:33 -05:00
Frej Drejhammar	8e1ba281d4	Merge branch 'frej/gh341' Closes #341 Closes #342	2025-08-16 15:18:34 +02:00
Günther Nußmüller	d77765a23e	Fix UnboundLocalError with plugins and largefiles When Plugins are used in a repository that contains largefiles, the following exception is thrown as soon as the first largefile is converted: ``` Traceback (most recent call last): File "fast-export/hg-fast-export.py", line 728, in <module> sys.exit(hg2git(options.repourl,m,options.marksfile,options.mappingfile, File "fast-export/hg-fast-export.py", line 581, in hg2git c=export_commit(ui,repo,rev,old_marks,max,c,authors,branchesmap, File "fast-export/hg-fast-export.py", line 366, in export_commit export_file_contents(ctx,man,modified,hgtags,fn_encoding,plugins) File "fast-export/hg-fast-export.py", line 222, in export_file_contents file_data = {'filename':filename,'file_ctx':file_ctx,'data':d} UnboundLocalError: local variable 'file_ctx' referenced before assignment ``` This commit fixes the error by: * initializing the file_ctx before the largefile handling takes place * Providing a new `is_largefile` value for plugins so they can detect if largefile handling was applied (and therefore the file_ctx object may no longer be in sync with the git version of the file)	2025-08-11 08:30:17 +02:00
Frej Drejhammar	95459e5599	Merge branch 'gh/340' Closes #340	2025-07-17 19:45:42 +02:00
Günther Nußmüller	de5c8d9d97	Remove redundant type check in set_default_branch The passed `name` parameter is always of type `str` in Python 3 hence the type check is redundant and no longer needed.	2025-07-16 14:44:43 +02:00
Günther Nußmüller	ad96531587	Fix TypeError when using the --origin option Encode the `name` parameter to bytes (using the utf8 codec). This fixes the `TypeError` in subsequent concatenations in `get_branch`: ``` Traceback (most recent call last): # stack omitted for brevity File "C:\Dev\git-migration\fast-export\hg2git.py", line 73, in get_branch return origin_name + b'/' + name TypeError: can only concatenate str (not "bytes") to str ``` The conversion is done unconditionally since the passed parameter is currently always of type `str`.	2025-07-16 14:41:45 +02:00
Frej Drejhammar	4af9a33bd6	Merge branch 'frej/gh338'	2025-06-05 16:53:33 +02:00
Frej Drejhammar	f71385ec14	Fix "Warn if one of the marks, mapping, or heads files are empty" The commit "Warn if one of the marks, mapping, or heads files are empty" (`7224e420a7`) mixed up the state and heads caches and reported that the heads cache was empty if the state case was. Error found by Shun-ichi Goto. Closes #338	2025-06-05 16:50:56 +02:00
Frej Drejhammar	ae21cbf1a2	CI: Bump Ubuntu version used for the CI Switch to the oldest supported version.	2025-06-05 16:50:37 +02:00
Frej Drejhammar	8762fee403	Merge branch 'gh/337'	2025-03-30 13:47:22 +02:00
Frank Zingsheim	bd707b5d6e	Fix: Largefiles ignored #141 Import mercurial large files as ordinary files into git The basic idea to this fix is based on https://github.com/planestraveler/fast-export/tree/add-lfs-support-v2 from PR #65 Closes #141	2025-03-29 18:39:27 +01:00
Frej Drejhammar	0afd336d6f	Merge branch 'gh/333'	2024-07-13 19:37:00 +02:00
Thalia Archibald	dd1c8f219b	Disable core.ignoreCase in tests When core.ignoreCase is set in the global config, hg-fast-export.sh warns the user and exits. Override this for tests.	2024-07-06 02:46:07 -07:00
Thalia Archibald	f947189dcc	Consistently terminate commit messages with LF When the length logic for fast-import 'data' commands was updated in `4c10270` (Fix data handling, 2023-03-02), one branch was missed, so commit messages now do not have a final LF appended in most cases. This changed the longtime behavior, which had been consistent since the first commit of hg2git, `9832035` (Initial import, 2007-03-06), and is expected by some applications which compare against old conversions from Mercurial.	2024-07-05 05:20:35 -07:00
Frej Drejhammar	2a3806576c	Merge branch 'gh/328'	2024-04-07 15:30:23 +02:00
Frej Drejhammar	08e2297853	CI: Add tests to avoid a repeat of #328 Extend tests to cover the file content filter example plugins in order to avoid a repeat of #328.	2024-04-07 15:25:04 +02:00
Frej Drejhammar	893d6302b7	Fix errors resulting from #318 When commit `ddfc3a8300` ("Run file_data_filter on deleted files") started calling the file_data_filter plugin method, in order to make deletion of plugin-renamed files work, the example plugins were not updated. This commit updates the example plugins to not crash when the file context is None. Thanks to @hetas discovering this. Closes 328	2024-04-07 15:23:08 +02:00
Frej Drejhammar	3de7bcfc18	CI: Remove run-tests script The script should have been removed in `90c6ad5f87` ("test: use make to run the tests").	2024-03-02 20:25:29 +01:00
Frej Drejhammar	d72e96b202	Drop manual CodeQL actions Use default configuration as configured in the web interface instead of hand-configured ci-actions which gives warnings.	2024-02-23 18:11:10 +01:00
Frej Drejhammar	fb225c4700	Merge branch 'gh/321'	2024-02-23 17:07:02 +01:00
Frej Drejhammar	997e8e1a8c	Merge branch 'gh/320' Fixes warnings appearing with Python 3.12. hg-fast-export.py:231: SyntaxWarning: invalid escape sequence '\.'	2024-02-23 17:04:28 +01:00
Stephan Hohe	ddb574004f	Add tests for plugins setting file content to None	2024-02-23 13:43:28 +01:00
Stephan Hohe	e63feee1b9	Don't add file if plugin sets content to `None`	2024-02-20 17:07:23 +01:00
Stephan Hohe	7b4bb7ff1d	Fix escape in regular expression	2024-02-19 23:40:05 +01:00
Frej Drejhammar	53bbe05278	Merge branch 'frej/gh318' Closes #318	2024-02-16 17:56:17 +01:00
Frej Drejhammar	ddfc3a8300	Run file_data_filter on deleted files The `file_data_filter` method should be called when files are deleted. In this case the `data` and `file_ctx` keys map to None. This is so that a filter which modifies file names can apply the same name transformations before files are deleted.	2024-02-16 17:12:49 +01:00
Frej Drejhammar	21ab3f347b	Make plugin loader look in directories relative to cwd Make the plugin loader also look for plugins using a path relative to the current working directory.	2024-02-16 17:06:51 +01:00
Frej Drejhammar	878ba44f48	Merge branch 'frej/run-tests-with-different-python-versions'	2023-12-28 13:48:02 +01:00
Frej Drejhammar	2476d08517	Run tests with multiple Python versions Run the CI tests with both the earliest supported Python version and the latest stable release. The intent is to quickly notice when new features require adjusting the oldest supported Python version and also detect when the latest stable version breaks old code (as when 3.12 removed `imp` and we witched to `importlib` in #311).	2023-12-28 13:40:48 +01:00
Frej Drejhammar	d4298a0906	Check for a supported Python version on startup Check that hg-fast-export is running on a supported version of Python on startup. This is an attempt to avoid problems like #314 in the future.	2023-12-28 13:40:48 +01:00
Frej Drejhammar	efe934e16b	Update required version of Python to 3.7 Due to problems with handling of Unicode input in Python < 3.7, bump the required version of Python to 3.7.	2023-12-28 13:40:48 +01:00
Frej Drejhammar	59675eca22	Add command line flag to dump found versions Add `--debug` command line flag which dumps the detected versions of Mercurial and Python. This will probably help future debugging when unexpected versions are used.	2023-12-28 13:40:48 +01:00
Frej Drejhammar	3c694243c4	Merge branch 'frej/fix-314'	2023-12-28 13:39:42 +01:00
Frej Drejhammar	1bbf7028b4	Don't look for a Python 2 interpreter Don't look for a Python 2 interpreter as Python is no longer supported. If there is a Python 2 available and it had the Mercurial modules available, hg-fast-export would use it and fail to import `importlib.machinery`. This is probably the cause of #314. Closes #314.	2023-12-27 13:18:56 +01:00
Frej Drejhammar	c8fa290adf	Merge branch 'PR/312'	2023-11-18 20:39:44 +01:00
Ekin Dursun	c49dd0cf60	Remove Python 2 compatibility code Python 2 support was removed recently, so we don't need the compatibility code anymore.	2023-11-18 20:22:18 +03:00
Frej Drejhammar	4f94d61d84	Merge branch 'PR/311' Closes #311	2023-11-18 14:54:53 +01:00
Ekin Dursun	a3d0562737	Make pluginloader use importlib instead imp Python 3.12 has removed imp and it's recommended to use importlib instead. Python 2.7 doesn't have importlib, so Python 2.7 support is ceased (not a big deal since it's been more than 3 years since it was EOLed) as a part of this change.	2023-11-12 20:41:43 +03:00
Frej Drejhammar	0d0e90d328	Merge branch 'PR/305' into frej/felipec-pr-spree Closes #305	2023-03-27 20:35:36 +02:00
Frej Drejhammar	64ee34dfb0	Merge branch 'PR/303' into frej/felipec-pr-spree Closes #303 Closes #304	2023-03-27 20:34:17 +02:00
Frej Drejhammar	71834a584c	Merge branch 'PR/302' into frej/felipec-pr-spree Closes #302	2023-03-27 20:33:59 +02:00
Frej Drejhammar	4310e47760	Merge branch 'PR/301' into frej/felipec-pr-spree Closes #301	2023-03-27 20:33:36 +02:00
Felipe Contreras	278cc9966c	github: rename the main action to ci As in: Continuous Integration. Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>	2023-03-27 01:54:00 -06:00
Felipe Contreras	cf66c36a32	github: move CodeQL steps into the main action Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>	2023-03-27 01:53:49 -06:00
Felipe Contreras	269c23c5bb	github: cleanup codeql action Based on the latest walk-through: https://github.com/github/codeql-action. Gets rid of the warning: Warning: 1 issue was detected with this workflow: git checkout HEAD^2 is no longer necessary. Please remove this step as Code Scanning recommends analyzing the merge commit for best results. Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>	2023-03-27 01:49:29 -06:00
Felipe Contreras	90c6ad5f87	test: use make to run the tests Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>	2023-03-26 20:05:03 -06:00
Felipe Contreras	51db3b4236	test: update default location of sharness It's included as a module for a reason. Also, use "$0" so the tests can be run like `./t/main.t` (or any other directory). Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>	2023-03-26 20:04:38 -06:00
Felipe Contreras	fba03b95fb	github: update checkout action Gets rid of the warning: Node.js 12 actions are deprecated. Please update the following actions to use Node.js 16: actions/checkout@v2. For more information see: https://github.blog/changelog/2022-09-22-github-actions-all-actions-will-begin-running-on-node16-instead-of-node12/. Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>	2023-03-26 19:46:38 -06:00
Felipe Contreras	2cc7db7556	test: bump sharness to 1.2 It's finally released. Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>	2023-03-26 19:43:04 -06:00
Frej Drejhammar	a89033b5b1	Merge branch 'PR/299' into frej/sharness-as-submodule-and-smoke-test Closes #298 Closes #299	2023-03-26 18:40:52 +02:00
Frej Drejhammar	fd5bd48a6c	Update codeql to version 2	2023-03-26 16:48:07 +02:00
Frej Drejhammar	84a877d112	Add smoke tests to CI test suite The added test is an unpublished test, now ported to Sharness, which has been used by the maintainer to sanity check PRs.	2023-03-26 16:48:07 +02:00
Frej Drejhammar	3f57c4340a	Change CI to run tests using test runner	2023-03-24 18:46:53 +01:00
Frej Drejhammar	1e872eb235	Add primitive test runner	2023-03-24 18:11:37 +01:00
Frej Drejhammar	ecdbf0e42e	Add Sharness as a submodule	2023-03-24 17:22:23 +01:00
Felipe Contreras	9754a9f3f6	Trivial simplification Just return the values directly, no need to store them into variables. Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>	2023-03-14 22:12:50 -06:00
Felipe Contreras	d2f11bd619	Remove multiple parent logic for file changes This is already what repo.status does. Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>	2023-03-14 22:12:50 -06:00
Felipe Contreras	3582221efd	Compare changes only with the first parent It's not necessary to check both parents. Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>	2023-03-14 22:12:50 -06:00
Felipe Contreras	0ae0d20496	Remove no-op check This code is only executed when there's two parents. Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>	2023-03-14 22:12:50 -06:00
Felipe Contreras	e09a14a266	Move parents logic inside get_filechanges This way export_commit is much simpler (already quite complex), and it's easier to modify the logic. No functional changes. Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>	2023-03-14 22:12:50 -06:00
Felipe Contreras	9df2f97f6c	Rename variables in get_filechanges It's easier to understand this way. No functional changes. Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>	2023-03-14 22:12:50 -06:00
Felipe Contreras	531fa9b3a2	Simplify split_dict There's no need to keep track of the left side: if it's modified it's modified. Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>	2023-03-14 22:12:50 -06:00
Felipe Contreras	a229b39d66	Coalesce modified files Git doesn't care if they are added or changed: they are modified. Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>	2023-03-14 22:12:50 -06:00
Felipe Contreras	c666fd9c95	Trivial style cleanup Checking the array directly is more idiomatic. Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>	2023-03-14 22:12:50 -06:00
Felipe Contreras	21fa443b4a	Simplify list of files for the first commit We already have the files. Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>	2023-03-14 22:12:50 -06:00
Felipe Contreras	fd6ba361c6	github: enable tests Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>	2023-03-13 20:18:29 -06:00
Felipe Contreras	153ba2a5c1	Add main test Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>	2023-03-13 20:18:29 -06:00
Frej Drejhammar	df5278f755	Merge branch 'PR/297' Closes #297	2023-03-13 17:57:20 +01:00
Felipe Contreras	6fbe4d0ad0	Skip earlier Now that we have ctx easily available, skip early. Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>	2023-03-10 12:38:42 -06:00
Felipe Contreras	fa73d8dec9	Share the changectx more It's used everywhere, might as well pass it along. Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>	2023-03-10 12:38:30 -06:00
Felipe Contreras	e1e15b2091	Avoid revsymbol() We can just do repo[rev]. Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>	2023-03-09 19:48:44 -06:00
Felipe Contreras	534d2bdd92	Don't deal with the node in get_changeset() It's not necessary. It could be fetched with repo[rev].node(), but why bother? Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>	2023-03-09 19:48:44 -06:00
Felipe Contreras	23f41c0ff1	Use revision directly instead of revnode We don't need the revnode. Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>	2023-03-09 19:48:44 -06:00
Felipe Contreras	8b1fd408ca	Use changectx directly There's no need to call repo[revnode] when repo[rev] works perfectly fine. And since we have the context already we can just do ctx.hex() instead of hexlifying ourselves. Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>	2023-03-09 19:48:44 -06:00
Felipe Contreras	4a4d242e98	Fetch node directly No need to call get_changeset() for that. Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>	2023-03-09 19:48:44 -06:00
Felipe Contreras	432254100b	Fetch branch names directly No need to use get_changeset() for just one thing. Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>	2023-03-09 19:48:44 -06:00
Felipe Contreras	5e4bc6eb03	Remove cruft Nothing uses that variable. Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>	2023-03-09 19:48:44 -06:00
Felipe Contreras	7886016978	hg2git: set proper default branch So that cfg_master is picked up in get_branch(). Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>	2023-03-09 19:48:44 -06:00
Frej Drejhammar	18577f559d	Merge branch 'PR/296'	2023-03-04 20:21:29 +01:00
Felipe Contreras	88defe7fd1	README: cleanup initial instructions The `git init` command can create the directory, and HEAD doesn't need to be specified in `git checkout` (it's the default). Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>	2023-03-04 09:53:25 -06:00
Frej Drejhammar	4edea927fb	Merge branch 'PR/295' Closes 295	2023-03-04 16:12:26 +01:00
Felipe Contreras	bbab981130	Trivial simplification of wr No need to issue two write commands. Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>	2023-03-04 16:08:45 +01:00
Felipe Contreras	c3cbf1e04d	Add wr_data helper No functional changes. Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>	2023-03-03 19:34:29 -06:00
Felipe Contreras	4c10270302	Fix data handling The length should be exactly the same as the data, for example if the data is "hello" only 5 characters should be written on the stream. Thus it should always be `len(data)`, not `len(data)+1` as it currently is in some places. Since the first commit of hg2git.py there was a wtf comment, presumably Rocco was confused about this common discrepancy. We can shuffle the logic around by adding '\n' to the data, and removing +1 to the length. Also, the data should be written without a newline (wr_no_nl). Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>	2023-03-03 19:33:45 -06:00
Frej Drejhammar	723d8032ba	Merge branch 'PR/294'	2022-11-25 16:31:18 +01:00
df	268299a358	Fix typo in README Added dash to match the actual usage of the 'ignore-unnamed-heads' option	2022-11-19 18:15:04 +01:00