OS seems to apply ext4 filesystem options in arbitrary fashion

Question

I fear I may have to revert to system defaults if I can't get this sorted out.

I'm trying to set various system configurations for more robust ext4 for a single-user desktop environment. Trying to assign desired configuration settings where they will take effect properly.

I understand that some of these should be included in the file mke2fs.conf so that the filesystems are initially created with those proper settings. But I will address that later, keeping the distro default file for the following.

I understand that the EXT4 options I wanted could be set in /etc/fstab. This following entry shows what I would typically want:

UUID=00000000-0000-0000-0000-000000000000       /DB001_F2       ext4    defaults,nofail,data=journal,journal_checksum,journal_async_commit,commit=15,errors=remount-ro,journal_ioprio=2,block_validity,nodelalloc,data_err=ignore,nodiscard   0       0

where each DB001_F{p} is a partition on the root disk ( p = [2-8] ).

I repeat those options here, in the same sequence as a list, in case that makes it more easy to assimilate:

defaults
nofail
data=journal
journal_checksum
journal_async_commit
commit=15
errors=remount-ro
journal_ioprio=2
block_validity
nodelalloc
data_err=ignore
nodiscard

Mounting during boot, the below syslog shows all as reporting what I believe to be acknowledged acceptable settings:

64017 Sep  4 21:04:35 OasisMega1 kernel: [   21.622599] EXT4-fs (sda7): mounted filesystem with journalled data mode. Opts: data=journal,journal_checksum,journal_async_commit,commit=15,errors=remount-ro,journal_ioprio=2,block_validity,nodelalloc,data_err=ignore,nodiscard
64018 Sep  4 21:04:35 OasisMega1 kernel: [   21.720338] EXT4-fs (sda4): mounted filesystem with journalled data mode. Opts: data=journal,journal_checksum,journal_async_commit,commit=15,errors=remount-ro,journal_ioprio=2,block_validity,nodelalloc,data_err=ignore,nodiscard
64019 Sep  4 21:04:35 OasisMega1 kernel: [   21.785653] EXT4-fs (sda8): mounted filesystem with journalled data mode. Opts: data=journal,journal_checksum,journal_async_commit,commit=15,errors=remount-ro,journal_ioprio=2,block_validity,nodelalloc,data_err=ignore,nodiscard
64021 Sep  4 21:04:35 OasisMega1 kernel: [   22.890168] EXT4-fs (sda12): mounted filesystem with journalled data mode. Opts: data=journal,journal_checksum,journal_async_commit,commit=15,errors=remount-ro,journal_ioprio=2,block_validity,nodelalloc,data_err=ignore,nodiscard
64022 Sep  4 21:04:35 OasisMega1 kernel: [   23.214507] EXT4-fs (sda9): mounted filesystem with journalled data mode. Opts: data=journal,journal_checksum,journal_async_commit,commit=15,errors=remount-ro,journal_ioprio=2,block_validity,nodelalloc,data_err=ignore,nodiscard
64023 Sep  4 21:04:35 OasisMega1 kernel: [   23.308922] EXT4-fs (sda13): mounted filesystem with journalled data mode. Opts: data=journal,journal_checksum,journal_async_commit,commit=15,errors=remount-ro,journal_ioprio=2,block_validity,nodelalloc,data_err=ignore,nodiscard
64024 Sep  4 21:04:35 OasisMega1 kernel: [   23.513804] EXT4-fs (sda14): mounted filesystem with journalled data mode. Opts: data=journal,journal_checksum,journal_async_commit,commit=15,errors=remount-ro,journal_ioprio=2,block_validity,nodelalloc,data_err=ignore,nodiscard

But mount shows that some drives are not reporting as expected, even after reboot, and this is inconsistent as seen below:

/dev/sda7 on /DB001_F2 type ext4 (rw,relatime,nodelalloc,journal_checksum,journal_async_commit,errors=remount-ro,commit=15,data=journal)
/dev/sda8 on /DB001_F3 type ext4 (rw,relatime,nodelalloc,journal_checksum,journal_async_commit,errors=remount-ro,commit=15,data=journal)
/dev/sda9 on /DB001_F4 type ext4 (rw,relatime,nodelalloc,journal_checksum,journal_async_commit,errors=remount-ro,commit=15,data=journal)
/dev/sda12 on /DB001_F5 type ext4 (rw,relatime,nodelalloc,journal_async_commit,errors=remount-ro,commit=15,data=journal)
/dev/sda13 on /DB001_F6 type ext4 (rw,relatime,nodelalloc,journal_async_commit,errors=remount-ro,commit=15,data=journal)
/dev/sda14 on /DB001_F7 type ext4 (rw,relatime,nodelalloc,journal_async_commit,errors=remount-ro,commit=15,data=journal)
/dev/sda4 on /DB001_F8 type ext4 (rw,relatime,nodelalloc,journal_async_commit,errors=remount-ro,commit=15,data=journal)

I read somewhere about a limitation regarding the length of the option string in fstab, so I used tune2fs to pre-set some parameters at a lower level. Those applied via tune2fs are:

journal_data,block_validity,nodelalloc

which is confirmed when using tune2fs -l:

Default mount options:    journal_data user_xattr acl block_validity nodelalloc

With that in place, I modified the fstab for entries to show as

UUID=00000000-0000-0000-0000-000000000000       /DB001_F2       ext4    defaults,nofail,journal_checksum,journal_async_commit,commit=15,errors=remount-ro,journal_ioprio=2,data_err=ignore,nodiscard   0       0

I did a umount for all my DB001_F? (/dev/sda*), then I did a mount -av, which reported the following:

/                        : ignored
/DB001_F2                : successfully mounted
/DB001_F3                : successfully mounted
/DB001_F4                : successfully mounted
/DB001_F5                : successfully mounted
/DB001_F6                : successfully mounted
/DB001_F7                : successfully mounted
/DB001_F8                : successfully mounted

No errors reported for the options string for each of the drives.

I tried using journal_checksum_v3, but mount -av failed all with that setting. I used the mount command to see what was reported.

I also did a reboot and repeated that mount again for these reduced settings, and mount shows again that the drives are not reporting as expected, and this is still inconsistent as seen here:

/dev/sda7 on /DB001_F2 type ext4 (rw,relatime,journal_checksum,journal_async_commit,commit=15)
/dev/sda8 on /DB001_F3 type ext4 (rw,relatime,journal_checksum,journal_async_commit,commit=15)
/dev/sda9 on /DB001_F4 type ext4 (rw,relatime,journal_checksum,journal_async_commit,commit=15)
/dev/sda12 on /DB001_F5 type ext4 (rw,relatime,journal_async_commit,commit=15)
/dev/sda13 on /DB001_F6 type ext4 (rw,relatime,journal_async_commit,commit=15)
/dev/sda14 on /DB001_F7 type ext4 (rw,relatime,journal_async_commit,commit=15)
/dev/sda4 on /DB001_F8 type ext4 (rw,relatime,journal_async_commit,commit=15)

Since these are all ext4 type filesystems, and all on the same physical drive, I don't understand the behaviour of the journal_checksum not be uniformly actioned! I also, I find it interesting that there is a dividing line in terms of the 2 classes of behaviour, since the order listed above is the order specified in the fstab (according to /DB001_F?), which presumably is the mounting order ... so what "glitch" is causing the "downgrading" of the remaining mount actions ?

My thinking (possibly baseless) is that some properties might be better set at time of creation of the filesystems, and that this would make them more "persistent/effective" than otherwise. When I tried to again shift some of the property settings by pre-defining those in mke2fs.conf. mke2fs.ext4 fails AGAIN, I suspect, because the option string is restricted to a limited length (64 characters ?). So ... I have backed away from making any changes to the mke2fs.conf.

Ignoring the mke2fs.conf issue for now, and focusing on the fstab and tune2fs functionality, can someone please explain to me what I am doing wrong that is preventing mount from correctly reporting what is the full range of settings currently in effect?

At this point, I don't know what I can rely on to provide the actual real state of the ext4 behaviour and am considering simply reverting to distro defaults, which leaves me wanting.

Is it possible that all is well and that the system is simply not reporting correctly? I am not sure that I could comfortably accept that viewpoint. It is counter-intuitive.

Can someone please assist?

Environment

UbuntuMATE 20.04 LTS
Linux OasisMega1 5.4.0-124-generic #140-Ubuntu SMP Thu Aug 4 02:23:37 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux 
RAM = 4GB
DSK = 2TB   (internal, 8 data partitions, 3 1GB swap partitions) [ROOT]
DSK = 500GB (internal,  2 data partitions, 1 1GB swap partitions)
DSK = 4TB   (external USB, 16 data partitions) [BACKUP drive]

This is what is being reported by debugfs:

Filesystem features:
    has_journal
    ext_attr
    resize_inode
    dir_index
    filetype
    needs_recovery
    extent
    flex_bg
    sparse_super
    large_file
    huge_file
    dir_nlink
    extra_isize
    metadata_csum

Not very useful for additional insights into the problem.

debugfs shows following supported features:

debugfs 1.45.5 (07-Jan-2020)

Supported features: (...snip...) journal_checksum_v2 journal_checksum_v3

Noteworthy is that debugfs is showing either journal_checksum_v2 or journal_checksum_v3 available but not the journal_checksum which is referenced in the manual pages.

Does that mean that I should be using v2 or v3, instead of journal_checksum?

Answer 1

Given the discussion that has transpired as comments on my original post, I am prepared to conclude that the many changes to the Kernel over the 2+ years since my original install of the UbuntuMATE 20.04 LTS distro are the source of the differences in behaviour observed by the set of 8 ext4 filesystems that were created at different times, notwithstanding the fact that they reside on the same physical device.

Consequently, the only way to ensure that all filesystems of a given fstype (i.e. ext4) react identically to mounting options, tune2fs options and behave/report identically by debuge2fs or mount commands, is to ensure that they are created with the same frozen version of an OS Kernel and the various filesystem utilities that are used to create and tune those filesystems.

So, to answer my original question, there is no problem with the filesystems reporting differently because they are reporting correctly, each for their own historical context leading to their current state.

Looking forward to my pending upgrade to UbuntuMATE 22.04 LTS (why I was digging into all this to begin with), to avoid the discrepencies, because the install disk is not the latest for the Kernel or utilities, my defined process must be to:

1. upgrade to newer OS,
2. reboot,
3. apply all updates,
4. create backup image of the upgraded+updated OS now residing on the root partition,
5. re-create root partition with latest Kernel and utilities (using a duplicate fully-updated OS residing on secondary internal disk, which is the reason for existence of my 500 GB drive, namely testing, proving, confirming final desired install before rolling over into "production"),
6. recover the primary fully-updated OS from backup image to its proper ROOT partition,
7. reboot, then
8. backup all other partitions on the primary disk, recreate those partitions, then restore the data for each of those partitions.

Only in this manner can all the partitions be created as "equals" with the latest and best offered at the one snapshot in time. Otherwise, the root partition is out of step with all other partitions that are created post-updates following the distro installation.

Also, having a script similar to the one I created ensures the required actions will be applied uniformly, avoiding any possible errors that might slip in from the tedium when performing it manually many times.

For those who want to be able to manage and review these options in a consistent fashion with a script, here is the script I created for myself:

#!/bin/sh

####################################################################################
###
### $Id: tuneFS.sh,v 1.2 2022/09/07 01:43:18 root Exp $
###
### Script to set consistent (local/site) preferences for filesystem treatment at boot-time or mounting
###
####################################################################################

TIMESTAMP=`date '+%Y%m%d-%H%M%S' `
BASE=`basename "$0" ".sh" `


###
### These variables will document hard-coded 'mount' preferences for filesystems
###
BOOT_MAX_INTERVAL="-c 10"   ### max number of boots before fsck [10 boots]
TIME_MAX_INTERVAL="-i 2w"   ### max calendar time between boots before fsck [2 weeks]
ERROR_ACTION="-e remount-ro"    ### what to do if error encountered
#-m reserved-blocks-percentage

###
### This OPTIONS string should be updated manually to document
### the preferred and expected settings to be applied to ext4 filesystems
###
OPTIONS="-o journal_data,block_validity,nodelalloc"

ASSIGN=0
REPORT=0
VERB=0
SINGLE=0
while [ $# -gt 0 ]
do
    case ${1} in
        --default ) REPORT=0 ; ASSIGN=0 ; shift ;;
        --report )  REPORT=1 ; ASSIGN=0 ; shift ;;
        --force )   REPORT=0 ; ASSIGN=1 ; shift ;;
        --verbose ) VERB=1 ; shift ;;
        --single )  SINGLE=1 ; shift ;;
        * ) echo "\n\t Invalid parameter used on the command line.  Valid options:  [ --default | --report | --force | --single | --verbose ] \n Bye!\n" ; exit 1 ;;
    esac
done


workhorse()
{
    case ${PARTITION} in
        1 )
            DEVICE="/dev/sda3"
            OPTIONS=""
            ;;
        2 )
            DEVICE="/dev/sda7"
            ;;
        3 )
            DEVICE="/dev/sda8"
            ;;
        4 )
            DEVICE="/dev/sda9"
            ;;
        5 )
            DEVICE="/dev/sda12"
            ;;
        6 )
            #UUID="0d416936-e091-49a7-9133-b8137d327ce0"
            #DEVICE="UUID=${UUID}"
            DEVICE="/dev/sda13"
            ;;
        7 )
            DEVICE="/dev/sda14"
            ;;
        8 )
            DEVICE="/dev/sda4"
            ;;
    esac
    PARTITION="DB001_F${PARTITION}"
    PREF="${BASE}.previous.${PARTITION}"

    reference=`ls -t1 "${PREF}."*".dumpe2fs" 2>/dev/null | grep -v 'ERR.dumpe2fs'| tail -1 `

    if [ ! -s "${PREF}.dumpe2fs.REFERENCE" ]
    then
        mv -v ${reference} ${PREF}.dumpe2fs.REFERENCE
    fi

    reference=`ls -t1 "${PREF}."*".verify" 2>/dev/null | grep -v 'ERR.verify'| tail -1 `

    if [ ! -s "${PREF}.verify.REFERENCE" ]
    then
        mv -v ${reference} ${PREF}.verify.REFERENCE
    fi

    BACKUP="${BASE}.previous.${PARTITION}.${TIMESTAMP}"
    BACKUP="${BASE}.previous.${PARTITION}.${TIMESTAMP}"

    rm -f ${PREF}.*.tune2fs
    rm -f ${PREF}.*.dumpe2fs

    ### reporting by 'tune2fs -l' is a subset of that from 'dumpe2fs -h'

    if [ ${REPORT} -eq 1 ]
    then
        ### No need to generate report from tune2fs for this mode.

        ( dumpe2fs -h ${DEVICE} 2>&1 ) | awk '{
                if( NR == 1 ){ print $0 } ;
                if( index($0,"revision") != 0 ){ print $0 } ;
                if( index($0,"mount options") != 0 ){ print $0 } ;
                if( index($0,"features") != 0 ){ print $0 } ;
                if( index($0,"Filesystem flags") != 0 ){ print $0 } ;
                if( index($0,"directory hash") != 0 ){ print $0 } ;
            }'>${BACKUP}.dumpe2fs
        echo "\n dumpe2fs REPORT [$PARTITION]:"
        cat ${BACKUP}.dumpe2fs
    else
        ### Generate report from tune2fs for this mode but only as sanity check.

        tune2fs -l ${DEVICE} 2>&1 >${BACKUP}.tune2fs

        ( dumpe2fs -h ${DEVICE} 2>&1 ) >${BACKUP}.dumpe2fs

        if [ ${VERB} -eq 1 ] ; then
            echo "\n tune2fs REPORT:"
            cat ${BACKUP}.tune2fs

            echo "\n dumpe2fs REPORT:"
            cat ${BACKUP}.dumpe2fs
        fi

        if [ ${ASSIGN} -eq 1 ]
        then
            tune2fs ${BOOT_MAX_INTERVAL}  ${TIME_MAX_INTERVAL}  ${ERROR_ACTION}  ${OPTIONS}  ${DEVICE}

            rm -f ${PREF}.*.verify
            ( dumpe2fs -h ${DEVICE} 2>&1 ) >${BACKUP}.verify

            if [ ${VERB} -eq 1 ] ; then  
                echo "\n Changes:"
                diff ${BACKUP}.dumpe2fs ${BACKUP}.verify
            fi
        else
            if [ ${VERB} -eq 1 ] ; then  
                echo "\n Differences:"
                diff ${BACKUP}.tune2fs ${BACKUP}.dumpe2fs
            fi

            rm -f ${BACKUP}.verify
        fi
    fi
}


if [ ${SINGLE} -eq 1 ]
then
    for PARTITION in 2 3 4 5 6 7 8
    do
        echo "\n\t Actions only for DB001_F${PARTITION} ? [y|N] => \c" ; read sel
        if [ -z "${sel}" ] ; then  sel="N" ; fi
        case ${sel} in
            y* | Y* ) DOIT=1 ; break ;;
            * ) DOIT=0 ;;
        esac
    done

    if [ ${DOIT} -eq 1 ]
    then
        workhorse
    fi
else
    for PARTITION in 2 3 4 5 6 7 8
    do
        workhorse
    done
fi


exit 0
exit 0
exit 0

For those who are interested, there is a modified/expanded script in a follow-on posting.

Thank you all for your input and feedback.