Highlights from Git 2.51

The open source Git project just released Git 2.51 with features and bug fixes from over 91 contributors, 21 of them new. We last caught up with you on the latest in Git back when 2.50 was released.

To celebrate this most recent release, here is GitHub’s look at some of the most interesting features and changes introduced since last time.

Cruft-free multi-pack indexes

Returning readers will have likely seen our coverage of cruft packs, multi-pack indexes (MIDXs), and reachability bitmaps. In case you’re new around here or otherwise need a refresher, here’s a brief overview:

Git stores repository contents as “objects” (blobs, trees, commits), either individually (“loose” objects, e.g. $GIT_DIR/objects/08/10d6a05...) or grouped into “packfiles” ($GIT_DIR/objects/pack). Each pack has an index (*.idx) that maps object hashes to offsets. With many packs, lookups slow down to O(M*log(N)), (where M is the number of packs in your repository, and N is the number of objects within a given pack).

A MIDX works like a pack index but covers the objects across multiple individual packfiles, reducing the lookup cost to O(log(N)), where N is the total number of objects in your repository. We use MIDXs at GitHub to store the contents of your repository after splitting it into multiple packs. We also use MIDXs to store a collection of reachability bitmaps for some selection of commits to quickly determine which object(s) are reachable from a given commit¹.

However, we store unreachable objects separately in what is known as a “cruft pack”. Cruft packs were meant to exclude unreachable objects from the MIDX, but we realized pretty quickly that doing so was impossible. The exact reasons are spelled out in this commit, but the gist is as follows: if a once-unreachable object (stored in a cruft pack) later becomes reachable from some bitmapped commit, but the only copy of that object is stored in a cruft pack outside of the MIDX, then that object has no bit position, making it impossible to write a reachability bitmap.

Git 2.51 introduces a change to how the non-cruft portion of your repository is packed. When generating a new pack, Git used to exclude any object which appeared in at least one pack that would not be deleted during a repack operation, including cruft packs. In 2.51, Git now will store additional copies of objects (and their ancestors) whose only other copy is within a cruft pack. Carrying this process out repeatedly guarantees that the set of non-cruft packs does not have any object which reaches some other object not stored within that set of packs. (In other words, the set of non-cruft packs is closed under reachability.)

As a result, Git 2.51 has a new repack.MIDXMustContainCruft configuration which uses the new repacking behavior described above to store cruft packs outside of the MIDX. Using this at GitHub has allowed us to write significantly smaller MIDXs, in a fraction of the time, and resulting in faster repository read performance overall. (In our primary monorepo, MIDXs shrunk by about 38%, we wrote them 35% faster, and improved read performance by around 5%.)

Give cruft-less MIDXs a try today using the new repack.MIDXMustContainCruft configuration option.

[source]

Smaller packs with path walk

In Git 2.49, we talked about Git’s new “name-hash v2” feature, which changed the way that Git selects pairs of objects to delta-compress against one another. The full details are covered in that post, but here’s a quick gist. When preparing a packfile, Git computes a hash of all objects based on their filepath. Those hashes are then used to sort the list of objects to be packed, and Git uses a sliding window to search between pairs of objects to identify good delta/base candidates.

Prior to 2.49, Git used a single hash function based on the object’s filepath, with a heavy bias towards the last 16 characters of the path. That hash function, dating back all the way to 2006, works well in many circumstances, but can fall short when, say, unrelated blobs appear in paths whose final 16 characters are similar. Git 2.49 introduced a new hash function which takes more of the directory structure into account², resulting in significantly smaller packs in some circumstances.

Git 2.51 takes the spirit of that change and goes a step further by introducing a new way to collect objects when repacking, called “path walk”. Instead of walking objects in revision order with Git emitting objects with their corresponding path names along the way, the path walk approach emits all objects from a given path at the same time. This approach avoids the name-hash heuristic altogether and can look for deltas within groups of objects that are known to be at the same path.

As a result, Git can generate packs using the path walk approach that are often significantly smaller than even those generated with the new name hash function described above. Its timings are competitive even with generating packs using the existing revision order traversal.

Try it out today by repacking with the new --path-walk command-line option.

[source]

Stash interchange format

If you’ve ever needed to switch to another branch, but wanted to save any uncommitted changes, you have likely used git stash. The stash command stores the state of your working copy and index, and then restores your local copy to match whatever was in HEAD at the time you stashed.

If you’ve ever wondered how Git actually stores a stash entry, then this section is for you. Whenever you push something onto your stash, Git creates three³ commits behind the scenes. There are two commits generated which capture the staged and unstaged changes. The staged changes represent whatever was in your index at the time of stashing, and the working directory changes represent everything you changed in your local copy but didn’t add to the index. Finally, Git creates a third commit listing the other two as its parents, capturing the entire snapshot.

Those internally generated commits are stored in the special refs/stash ref, and multiple stash entries are managed with the reflog. They can be accessed with git stash list, and so on. Since there is only one stash entry in refs/stash at a time, it’s extremely cumbersome to migrate stash entries from one machine to another.

Git 2.51 introduces a variant of the internal stash representation that allows multiple stash entries to be represented as a sequence of commits. Instead of using the first two parents to store changes from the index and working copy, this new representation adds one more parent to refer to the previous stash entry. That results in stash entries that contain four⁴ parents, and can be treated like an ordinary log of commits.

As a consequence of that, you can now export your stashes to a single reference, and then push or pull it like you would a normal branch or tag. Git 2.51 makes this easy by introducing two new sub-commands to git stash to import and export, respectively. You can now do something like:

$ git stash export --to-ref refs/stashes/my-stash
$ git push origin refs/stashes/my-stash

on one machine to push the contents of your stash to origin, and then:

$ git fetch origin '+refs/stashes/*:refs/stashes/*'
$ git stash import refs/stashes/my-stash

on another, preserving the contents of your stash between the two.

[source]

All that…

Now that we’ve covered some of the larger changes in more detail, let’s take a quicker look at a selection of some other new features and updates in this release.

• If you’ve ever scripted around the object contents of your repository, you have no doubt encountered git cat-file, Git’s dedicated tool to print the raw contents of a given object.

  git cat-file also has specialized --batch and --batch-check modes, which take a sequence of objects over stdin and print each object’s information (and contents, in the case of --batch). For example, here’s some basic information about the README.md file in Git’s own repository.

  $ echo HEAD:README.md | git.compile cat-file --batch-check
  d87bca1b8c3ebf3f32deb557ae9796ddc5b792ca blob 3662

  Here, Git is telling us the object ID, type, and size for the object we specified, just as we expect. cat-file produces the same information for tree and commit objects. But what happens if we give it the path to a submodule? Prior to Git 2.51, cat-file would just print missing. But Git 2.51 improves this output, making cat-file more useful in a new variety of scripting scenarios:

  [ pre-2.51 git ]
  $ echo HEAD:sha1collisiondetection | git cat-file --batch-check
  HEAD:sha1collisiondetection missing
  
  [ git 2.51 ]
  $ echo HEAD:sha1collisiondetection | git cat-file --batch-check 855827c583bc30645ba427885caa40c5b81764d2 submodule

  [source]

• Back in our coverage of 2.28, we talked about Git’s new changed-path Bloom feature. If you aren’t familiar with Bloom filters, or could use a refresher about how they’re used in Git, then read on.

  A Bloom filter is a probabilistic data structure that behaves like a set, with one difference. It can only tell you with 100% certainty whether an element is not in the set, but may have some false positives when indicating that an item is in the set.

  Git uses Bloom filters in its commit-graph data structure to store a probabilistic set of which paths were modified by that commit relative to its first parent. That allows history traversals like git log origin -- path/to/my/file to quickly skip over commits which are known not to modify that path (or any of its parents). However, because Git’s full pathspec syntax is far more expressive than that, Bloom filters can’t always optimize pathspec-scoped history traversals.

  Git 2.51 addresses part of that limitation by adding support for using multiple pathspec items, like git log -- path/to/a path/to/b, which previously could not make use of changed-path Bloom filters. At the time of writing, there is ongoing discussion about adding support for even more special cases.

  [source]

• The modern equivalents of git checkout, known as git switch and git restore have been considered experimental since their introduction back in Git 2.23. These commands delineate the many jobs that git checkout can perform into separate, more purpose-built commands. Six years later⁵, these commands are no longer considered experimental, making their command-line interface stable and backwards compatible across future releases.

  [source]

• Even if you’re a veteran Git user, it’s not unlikely to encounter a new Git command (among the 144!⁶)  every once in a while. One such command you might not have heard of is git whatchanged, which behaves like its modern alternative git log --raw.

  That command is now marked as deprecated with eventual plans to remove it in Git 3.0. As with other similar deprecations, you can still use this command behind the aptly-named --i-still-use-this flag⁷.

  [source]

• Speaking of Git 3.0, this release saw a few more entries added to the BreakingChanges list. First, Git’s reftable backend (which we talked about extensively in our coverage of Git 2.45) will become the new default format in repositories created with Git 3.0, when it is eventually released. Git 3.0 will also use the SHA-256 hash function as its default hash when initializing new repositories.

  Though there is no official release date yet planned for Git 3.0, you can get a feel for some of the new defaults by building Git yourself with the WITH_BREAKING_CHANGES flag.

  [source, source]

• Last but not least, a couple of updates on Git’s internal development process. Git has historically prioritized wide platform compatibility, and, as a result, has taken a conservative approach to adopting features from newer C standards. Though Git has required a C99-compatible compiler since near the end of 2021, it has adopted features from that standard gradually, since some of the compilers Git targets only have partial support for the standard.

  One example is the bool keyword, which became part of the C standard in C99. Here, the project began experimenting with the bool keyword back in late 2023. This release declares that experiment a success and now permits the use of bool throughout its codebase. This release also began documenting C99 features that the project is using experimentally along with C99 features that the project doesn’t use.

  Finally, this release saw an update to Git’s guidelines on submitting patches, which have historically required contributions to be non-anonymous, and submitted under a contributor’s legal name. Git now aligns more closely with the Linux kernel’s approach, to permit submitting patches with an identity other than the contributor’s legal name.

  [source, source, source]

…and a bag of chips

That’s just a sample of changes from the latest release. For more, check out the release notes for 2.51, or any previous version in the Git repository.

¹ For some bit position (corresponding to a single object in your repository,) a 1 means that object can be reached from that bitmap’s associated commit, and a 0 means it is not reachable from that commit. There are also four type-level bitmaps (for blobs, trees, commits, and annotated tags); the XOR of those bitmaps is the all 1s bitmap. For more details on multi-pack reachability bitmaps, check out our previous post on Scaling monorepo maintenance. ⤴️

² For the curious, each layer of the directory is hashed individually, then downshifted and XOR ed into the overall result. This results in a hash function which is more sensitive to the whole path structure, rather than just the final 16 characters. ⤴️

³ Usually. Git will sometimes generate a fourth commit if you stashed untracked (new files that haven’t yet been committed) or ignored files (that match one or more patterns in a .gitignore). ⤴️

⁴ Or five. ⤴️

⁵ Almost to the day; Git 2.23 was released on August 16, 2019, and Git 2.51 was released on August 18, 2025. ⤴️

⁶ It’s true; git --list-cmds=builtins | wc -l outputs “144” with Git 2.51. ⤴️

⁷ If you are somehow a diehard git whatchanged user, please let us know by sending a message to the Git mailing list. ⤴️

The post Highlights from Git 2.51 appeared first on The GitHub Blog.

Source: Read More