How to exclude journalctl entries using a filter?

Question

Consider the following three entries from this journalctl output (json output and debug mode turned on for completeness):

SYSTEMD_LOG_LEVEL=debug journalctl -o json -u docker --since '1 hour ago'
Root directory /run/log/journal added.
Considering /run/log/journal/de1e08ac57af453bacab3cc9875b12b9.
Directory /run/log/journal/de1e08ac57af453bacab3cc9875b12b9 added.
File /run/log/journal/de1e08ac57af453bacab3cc9875b12b9/system.journal added.
File /run/log/journal/de1e08ac57af453bacab3cc9875b12b9/system@7bea274da69540c8b1676a1cd030f6ee-0000000001022a21-00054cd4f00adc68.journal added.
File /run/log/journal/de1e08ac57af453bacab3cc9875b12b9/system@7bea274da69540c8b1676a1cd030f6ee-000000000101fcf0-00054cd199b0289f.journal added.
File /run/log/journal/de1e08ac57af453bacab3cc9875b12b9/system@7bea274da69540c8b1676a1cd030f6ee-000000000101cd35-00054ccd960f91a8.journal added.
File /run/log/journal/de1e08ac57af453bacab3cc9875b12b9/system@7bea274da69540c8b1676a1cd030f6ee-0000000001019c1d-00054ccab4dac8d5.journal added.
File /run/log/journal/de1e08ac57af453bacab3cc9875b12b9/system@7bea274da69540c8b1676a1cd030f6ee-0000000001016ae3-00054cc7d76493eb.journal added.
File /run/log/journal/de1e08ac57af453bacab3cc9875b12b9/system@7bea274da69540c8b1676a1cd030f6ee-00000000010139aa-00054cc4212faa29.journal added.
File /run/log/journal/de1e08ac57af453bacab3cc9875b12b9/system@7bea274da69540c8b1676a1cd030f6ee-0000000001010d45-00054cbe6893a794.journal added.
Considering /run/log/journal/c811c8a6e38845669ba5607794d4b425.
Directory /run/log/journal/c811c8a6e38845669ba5607794d4b425 added.
File /run/log/journal/c811c8a6e38845669ba5607794d4b425/system.journal added.
Journal filter: ((OBJECT_SYSTEMD_UNIT=docker.service AND _UID=0) OR (UNIT=docker.service AND _PID=1) OR (COREDUMP_UNIT=docker.service AND _UID=0 AND MESSAGE_ID=fc2e22bc6ee647b6b90729ab34a250b1) OR _SYSTEMD_UNIT=docker.service)
{ "__CURSOR" : "s=7bea274da69540c8b1676a1cd030f6ee;i=10260ef;b=15e9d32e03844e279dc0fcce7cb3c223;m=77b2f462910;t=54cd75d2cca7e;x=c30fbcda999df142", "__REALTIME_TIMESTAMP" : "1491862748449406", "__MONOTONIC_TIMESTAMP" : "8225655499024", "_BOOT_ID" : "15e9d32e03844e279dc0fcce7cb3c223", "_UID" : "0", "_GID" : "0", "_MACHINE_ID" : "de1e08ac57af453bacab3cc9875b12b9", "_HOSTNAME" : "bnode1", "_CAP_EFFECTIVE" : "1fffffffff", "_SYSTEMD_SLICE" : "system.slice", "PRIORITY" : "6", "_TRANSPORT" : "journal", "MESSAGE" : "http: TLS handshake error from 172.17.0.4:59426: tls: first record does not look like a TLS handshake\n", "PACKAGE" : "", "SYSLOG_IDENTIFIER" : "dockerd", "_PID" : "23542", "_COMM" : "dockerd", "_EXE" : "/usr/bin/dockerd", "_CMDLINE" : "dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --storage-driver devicemapper --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=digitalocean", "_SYSTEMD_CGROUP" : "/system.slice/docker.service", "_SYSTEMD_UNIT" : "docker.service", "_SOURCE_REALTIME_TIMESTAMP" : "1491862748449026" }
Root directory /run/log/journal removed.
Directory /run/log/journal/c811c8a6e38845669ba5607794d4b425 removed.
Directory /run/log/journal/de1e08ac57af453bacab3cc9875b12b9 removed.
mmap cache statistics: 719 hit, 15 miss
{ "__CURSOR" : "s=7bea274da69540c8b1676a1cd030f6ee;i=10260f0;b=15e9d32e03844e279dc0fcce7cb3c223;m=77b2f465891;t=54cd75d2cf9ff;x=c85ca946535cd15a", "__REALTIME_TIMESTAMP" : "1491862748461567", "__MONOTONIC_TIMESTAMP" : "8225655511185", "_BOOT_ID" : "15e9d32e03844e279dc0fcce7cb3c223", "_UID" : "0", "_GID" : "0", "_MACHINE_ID" : "de1e08ac57af453bacab3cc9875b12b9", "_HOSTNAME" : "bnode1", "_CAP_EFFECTIVE" : "1fffffffff", "_SYSTEMD_SLICE" : "system.slice", "PRIORITY" : "6", "_TRANSPORT" : "journal", "PACKAGE" : "", "SYSLOG_IDENTIFIER" : "dockerd", "_PID" : "23542", "_COMM" : "dockerd", "_EXE" : "/usr/bin/dockerd", "_CMDLINE" : "dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --storage-driver devicemapper --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=digitalocean", "_SYSTEMD_CGROUP" : "/system.slice/docker.service", "_SYSTEMD_UNIT" : "docker.service", "MESSAGE" : "http: TLS handshake error from 172.17.0.4:59428: tls: client didn't provide a certificate\n", "_SOURCE_REALTIME_TIMESTAMP" : "1491862748461177" }
{ "__CURSOR" : "s=7bea274da69540c8b1676a1cd030f6ee;i=102611c;b=15e9d32e03844e279dc0fcce7cb3c223;m=77b311a8308;t=54cd75f012476;x=25ad24e998bdafaa", "__REALTIME_TIMESTAMP" : "1491862779143286", "__MONOTONIC_TIMESTAMP" : "8225686192904", "_BOOT_ID" : "15e9d32e03844e279dc0fcce7cb3c223", "_UID" : "0", "_GID" : "0", "_MACHINE_ID" : "de1e08ac57af453bacab3cc9875b12b9", "_HOSTNAME" : "bnode1", "_CAP_EFFECTIVE" : "1fffffffff", "_SYSTEMD_SLICE" : "system.slice", "PRIORITY" : "6", "_TRANSPORT" : "journal", "_PID" : "23542", "_COMM" : "dockerd", "_EXE" : "/usr/bin/dockerd", "_CMDLINE" : "dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --storage-driver devicemapper --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=digitalocean", "_SYSTEMD_CGROUP" : "/system.slice/docker.service", "_SYSTEMD_UNIT" : "docker.service", "MESSAGE" : "hello world", "CONTAINER_TAG" : "5d0ecb10c3c5", "CONTAINER_ID" : "5d0ecb10c3c5", "CONTAINER_ID_FULL" : "5d0ecb10c3c5c51ac912c174f2e5db4e9a9acecd948cfe296d0966936dae584a", "CONTAINER_NAME" : "happy_booth", "_SOURCE_REALTIME_TIMESTAMP" : "1491862779142975" }

I have three total entries. Only one of them has the custom CONTAINER_ID field.

I want to build a journalctl command that will exclude any entries that have this specific field included. I have tried the following to no avail:

SYSTEMD_LOG_LEVEL=debug journalctl -o json -u docker --since '1 hour ago' CONTAINER_ID=
Root directory /run/log/journal added.
Considering /run/log/journal/de1e08ac57af453bacab3cc9875b12b9.
Directory /run/log/journal/de1e08ac57af453bacab3cc9875b12b9 added.
File /run/log/journal/de1e08ac57af453bacab3cc9875b12b9/system.journal added.
File /run/log/journal/de1e08ac57af453bacab3cc9875b12b9/system@7bea274da69540c8b1676a1cd030f6ee-0000000001022a21-00054cd4f00adc68.journal added.
File /run/log/journal/de1e08ac57af453bacab3cc9875b12b9/system@7bea274da69540c8b1676a1cd030f6ee-000000000101fcf0-00054cd199b0289f.journal added.
File /run/log/journal/de1e08ac57af453bacab3cc9875b12b9/system@7bea274da69540c8b1676a1cd030f6ee-000000000101cd35-00054ccd960f91a8.journal added.
File /run/log/journal/de1e08ac57af453bacab3cc9875b12b9/system@7bea274da69540c8b1676a1cd030f6ee-0000000001019c1d-00054ccab4dac8d5.journal added.
File /run/log/journal/de1e08ac57af453bacab3cc9875b12b9/system@7bea274da69540c8b1676a1cd030f6ee-0000000001016ae3-00054cc7d76493eb.journal added.
File /run/log/journal/de1e08ac57af453bacab3cc9875b12b9/system@7bea274da69540c8b1676a1cd030f6ee-00000000010139aa-00054cc4212faa29.journal added.
File /run/log/journal/de1e08ac57af453bacab3cc9875b12b9/system@7bea274da69540c8b1676a1cd030f6ee-0000000001010d45-00054cbe6893a794.journal added.
Considering /run/log/journal/c811c8a6e38845669ba5607794d4b425.
Directory /run/log/journal/c811c8a6e38845669ba5607794d4b425 added.
File /run/log/journal/c811c8a6e38845669ba5607794d4b425/system.journal added.
Journal filter: (CONTAINER_ID= AND ((OBJECT_SYSTEMD_UNIT=docker.service AND _UID=0) OR (UNIT=docker.service AND _PID=1) OR (COREDUMP_UNIT=docker.service AND _UID=0 AND MESSAGE_ID=fc2e22bc6ee647b6b90729ab34a250b1) OR _SYSTEMD_UNIT=docker.service))
Directory /run/log/journal/c811c8a6e38845669ba5607794d4b425 removed.
Directory /run/log/journal/de1e08ac57af453bacab3cc9875b12b9 removed.
Root directory /run/log/journal removed.
mmap cache statistics: 16 hit, 12 miss

Setting the filter to CONTAINER_ID= will return no entries.

Is there a way to tell journalctl to only match entries that do not have a field present?

The journalctl manpage seems to list no examples that include this usecase.

grawity_u1686 · Accepted Answer · 2017-04-11 05:04:23Z

17

No, journalctl does not support exclusion / negative filters. You will have to filter the results through jq or regular grep for now.

answered Apr 11, 2017 at 5:04

grawity_u1686

465k66 gold badges977 silver badges1.1k bronze badges

10

Thanks for confirming there's no negative filters! I was able to achieve this with the following jq filter: journalctl -u docker -o json | jq -cMr 'select(has("CONTAINER_ID") | not) | .MESSAGE'
– programmerq
Commented Apr 12, 2017 at 17:12
1

Beautiful @programmerq. Level Up.
– Cymatical
Commented Sep 21, 2021 at 20:29
1

For reference, the request to add exclusion filtering to journalctl is being tracked in systemd/systemd#2720.
– Kevinoid
Commented Jan 15, 2022 at 17:32

Add a comment |

Tombart · Accepted Answer · 2021-02-28 13:11:23Z

9

You can use grep for simple inverse matching -v / --invert-match, if the CONTAINER_ID is included in the text output:

journalctl -u docker -o cat --no-pager | grep -v "5d0ecb10c3c5"

for more advanced filtering, it's better to use json output:

journalctl -u docker -b -o json | jq -C . | less -R

Where you can filter messages as @programmerq suggets:

journalctl -u docker -o json | jq -cr 'select(has("CONTAINER_ID") | not) | .MESSAGE'

answered Feb 28, 2021 at 13:11

Tombart

1,6951 gold badge16 silver badges15 bronze badges

2

Good to see side by side examples of jq and grep together.
– Cymatical
Commented Sep 21, 2021 at 20:34

Add a comment |

Stack Exchange Network

How to exclude journalctl entries using a filter?

2 Answers 2

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged
logging
redhat-enterprise-linux
.

Hot Network Questions

How to exclude journalctl entries using a filter?

2 Answers 2

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged loggingredhat-enterprise-linux.

Related

Hot Network Questions

Not the answer you're looking for? Browse other questions tagged
logging
redhat-enterprise-linux
.