```html

Dirstate: The Definitive Guide to Mercurial's Working Directory State System

The dirstate is Mercurial's internal mechanism for tracking the status of every file in a working directory. Without it, Mercurial would have no way to distinguish between modified, added, removed, or clean files — making it one of the most critical components in the Mercurial architecture. This comprehensive review examines how dirstate works, its advantages, its drawbacks, and what you need to know to work efficiently with it.

Key Takeaways

• Dirstate is Mercurial's working directory state database, stored in .hg/dirstate and .hg/dirstate-state.
• It tracks file modes, flags, and modification status to enable fast hg status and hg diff operations.
• Performance advantage: Dirstate allows Mercurial to detect changes without scanning every file's full content, using mtimes and file sizes for quick identification.
• Known drawback: Dirstate corruption can render a repository unusable until repaired, and large repositories with thousands of files may experience slower dirstate operations.
• Bug awareness is essential — known issues like double-buffer corruption and dirstate format incompatibilities have affected users across multiple Mercurial versions.

What Is Dirstate?

Core Definition

Dirstate is a binary database maintained by Mercurial at .hg/dirstate within every repository's working directory. It records, for each file under version control, its parent revision, file mode, content flags (such as exact, added, removed, or ignored), and metadata used to detect when a file has been modified outside of Mercurial.

How It Differs From Git's Index

Unlike Git's index (staging area), which primarily tracks what will be committed next, Mercurial's dirstate serves a broader purpose: it is the single source of truth for the working directory's relationship to the parent revision. While Git re-computes file status from disk on every operation, dirstate persists state across invocations, enabling faster status checks in stable repositories.

Dirstate Data Structure

The dirstate file stores entries containing:

• The 40-character hex node ID of the parent revision
• The file path (stored as relative paths from the repository root)
• File mode bits (permissions)
• A flags byte indicating state: exact, added, removed, copied, ignored, or binary
• Modification time and file size at last scan
• A per-file state hash for detecting content changes

How Dirstate Works Under the Hood

Initialization and Format Evolution

Mercurial has evolved its dirstate format across multiple versions. The original format used a simple sorted list of file entries. Starting with Mercurial 2.6, a new double-buffered dirstate format was introduced to improve concurrency and reduce corruption risk. Mercurial 4.4+ further refined the format with compact node storage and improved performance for large repositories.

Mercurial Version	Dirstate Feature	Impact
Pre-2.6	Single-buffer dirstate format	Basic functionality; vulnerable to partial writes during crashes
2.6+	Double-buffered dirstate	Atomic writes reduce corruption risk significantly
4.0+	Compact node ID storage	Reduced dirstate file size for large repositories
4.4+	Optimized status resolution	Faster hg status with fewer full content scans
5.0+	Tree merging improvements	Better handling of renames and directory-level changes

The Double-Buffer Design

The double-buffered dirstate writes to two parallel buffers and only promotes the active buffer after a successful commit. This design means that if a process is interrupted mid-write, Mercurial can fall back to the last complete buffer rather than recovering from a partially written state — dramatically reducing the incidence of dirstate corruption after crashes or force-kills.

Dirstate-State File

Mercurial also maintains .hg/dirstate-state, a smaller companion file that tracks the global state needed for rapid reconciliation. This includes the last known parent revlog state and a cache of previously scanned files, enabling Mercurial to skip re-checking files whose mtimes and sizes have not changed since the last operation.

Advantages of Dirstate

1. Fast Status Resolution

By caching file metadata — modification time, size, and a content hash — dirstate allows hg status to resolve file states in milliseconds for the vast majority of files. Only files whose cached metadata has changed require deeper inspection. For large monorepos with tens of thousands of files, this optimization is the difference between a responsive CLI and an unusably slow one.

2. Accurate Change Detection

Dirstate uses a multi-layered approach to detecting modifications:

• Mtime + size check — First pass filters out obviously unchanged files.
• Content hash comparison — Second pass verifies actual content changes for files that appear modified.
• Exact flag tracking — For files marked exact, dirstate knows the working copy matches the parent revision precisely.

3. Supports Mercurial's Copy Tracking

Mercurial's ability to detect renames and copies across revisions depends heavily on dirstate. When a file is copied or moved, dirstate records the copied flag along with the source path, enabling hg log --follow and hg diff to trace file history across moves.

4. Enables Partial Commit Workflows

Because dirstate tracks per-file state independently, Mercurial supports committing subsets of changes within a working directory. This is fundamental to Mercurial's philosophy of fine-grained, intentional commits rather than the all-or-nothing approach some other VCS models encourage.

5. Conflict-Free Merging

Dirstate's precise tracking of which revision each file derives from enables Mercurial to perform three-way merges with minimal ambiguity. When resolving a merge, Mercurial compares the working directory against both parents using dirstate as the reference, reducing the chance of incorrect conflict resolution.

Drawbacks and Limitations

1. Corruption Vulnerability

Despite the double-buffer design, dirstate corruption remains one of the most common causes of repository dysfunction in Mercurial. Scenarios that increase corruption risk include:

• Killing a Mercurial process while dirstate is being written (e.g., Ctrl+C during a long operation)
• Running multiple Mercurial processes against the same working directory simultaneously
• Filesystem bugs or power loss during write operations
• Antivirus software intercepting or delaying file writes to .hg/

Corruption symptoms include spurious file modifications reported by hg status, inability to commit, or fatal errors on startup. Recovery typically involves hg recover or manual dirstate reconstruction.

2. Performance Degradation at Scale

While dirstate is highly optimized, repositories with extremely large numbers of tracked files (50,000+) can still experience noticeable slowdowns.