In simple terms, how do ext4 journaling file systems work?

Question

How does a journaling file system work and why does it not fragment, like NTFS?

Why hasn't Windows switched over to a journaling file system?

Could someone please explain this to me? Thanks.

Answer 1

I'll try to give an extended but simple answer, based on available sources:

Simple definition of a journaling file system from PC World:

A file system that contains its own recovery capability in the event of a failure. In a journaling file system, the information about the changes is recorded in a separate log (the journal) before the indexes to the files are updated. If a power or other system failure corrupts the indexes as they are being rewritten, the operating system can use the log to repair them when the computer is restarted.

A little more detail from Wikipedia:

Updating file systems to reflect changes to files and directories usually requires many separate write operations. This makes it possible for an interruption (like a power failure or system crash) between writes to leave data structures in an invalid intermediate state.

For example, deleting a file on a Unix file system involves three steps:

1. Removing its directory entry.
2. Releasing the inode to the pool of free inodes.
3. Returning all disk blocks to the pool of free disk blocks.

If a crash occurs after step 1 and before step 2, there will be an orphaned inode and hence a storage leak; if a crash occurs between steps 2 and 3, then the blocks previously used by the file cannot be used for new files, effectively decreasing the storage capacity of the file system. Re-arranging the steps does not help, either. (...)

(...)

To prevent this, a journaled file system allocates a special area—the journal—in which it records the changes it will make ahead of time. After a crash, recovery simply involves reading the journal from the file system and replaying changes from this journal until the file system is consistent again. (...)

In addition, ext4 uses some other techniques to reduce fragmentation, namely Extends and Delayed allocation (again from Wikipedia):

Extents

Extents replace the traditional block mapping scheme used by ext2 and ext3. An extent is a range of contiguous physical blocks, improving large-file performance and reducing fragmentation. A single extent in ext4 can map up to 128 MiB of contiguous space with a 4 KiB block size. There can be four extents stored directly in the inode. When there are more than four extents to a file, the rest of the extents are indexed in a tree.

Delayed allocation

ext4 uses a performance technique called allocate-on-flush, also known as delayed allocation. That is, ext4 delays block allocation until data is flushed to disk; in contrast, some file systems allocate blocks immediately, even when the data goes into a write cache. Delayed allocation improves performance and reduces fragmentation by effectively allocating larger amounts of data at a time.

I hope this gives the very basic of the operations of journaling filesystems, and specifically some ext4 details.

Answer 2

Quoting from PC World Ubuntu Linux, Day 16: EXT4 vs. NTFS:

EXT4 can support individual files up to 16 terabytes, and volumes up to one exabyte in size. But, one of the aspects of EXT4 which contributes to better performance, though, is that EXT4 can handle larger extents-a range of contiguous physical blocks of data. This allows it to work better with large files and reduce drive fragmentation.

Other factors include the allocate-on-flush technique used by EXT4. By delaying allocation of data blocks until the data is ready to be written to disk, EXT4 improves performance and reduces fragmentation compared to file systems that allocate blocks earlier.

Using checksums for drive journaling improves reliability and improves performance by avoiding waiting on the disk during the journaling process. When it comes to file checking, EXT4 is quicker because unallocated blocks of data are marked as such and are simply skipped during disk check operations.

Any discussion around 'why/why not' and 'defrag' needs to include Diskeeper (formerly Executive Software). Offering to license your product to Redmond, and having your company and/or product still be viable in two years, is a Herculean adventure at best ..