Pydicom private tags showing up as 'UN'

Question

When I read a dicom file all the private tags show up with VR of 'UN', instead of whatever they should be.

import pydicom
filename = 'H:\Fuji.dcm'
ds = pydicom.dcmread(filename)
print(ds)

Gives an output of:

...
(0009, 0010) Private Creator                     LO: 'FDMS 1.0'
(0009, 1005) [Image UID]                         UN: b'\x1d\xa6h\x12\x08\x07pZ\x0f9\x0e\xc2'
(0009, 1006) [Route Image UID]                   UN: b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
(0009, 1008) [Image Display Information Version  UN: b'\x00\x00\x00\x00'
(0009, 1009) [Patient Information Version No.]   UN: b'\x00\x00\x00\x00'
(0009, 100c) [Film UID]                          UN: b'\x1d\xa6h\x12\x08\x07pZ\x0f9\x0e\xc2'
(0009, 1010) [Exposure Unit Type Code]           UN: b'F3'
(0009, 1080) [Kanji Hospital Name]               UN: None
(0009, 1090) [Distribution Code]                 UN: b'MAINPACS'
(0009, 10f0) [Blackening Process Flag]           UN: b'01'
(0009, 10f1) [Processing Information Flag]       UN: b'0001'
(0009, 10f2) Private tag data                    UN: b'01'
(0009, 10f3) Private tag data                    UN: b'01'
(0009, 10f4) Private tag data                    UN: b'01'
...

This doesn't change if I try to add them manually with:

pydicom.datadict.add_private_dict_entries('FDMS 1.0', 
                                          {0x00090005:('OW','1','Image UID'),
                                           0x00090006:('OW','1','Route Image UID'),
                                           0x00090008:('UL','1','Image Display Information Version No.'),
                                           0x00090009:('UL','1','Patient Information Version No.'),
                                           0x0009000c:('OW','1','Film UID'),
                                           0x00090010:('CS','1','Exposure Unit Type Code'),
                                           0x00090080:('LO','1','Kanji Hospital Name'),
                                           0x00090090:('ST','1','Distribution Code'),
                                           0x000900F0:('CS','1','Blackening Process Flag'),
                                           0x000900F1:('ST','1','Processing Information Flag'),
                                           0x000900F2:('CS','1','Normalization Flag'),
                                           0x000900F3:('CS','1','Tone characteristic'),
                                           0x000900F4:('CS','1','Window Value Fixed Flag'),
                                           })
print(ds)

It correctly picks up the descriptions of the missing tags, but still won't change their VR from UN:

...
(0009, 0010) Private Creator                     LO: 'FDMS 1.0'
(0009, 1005) [Image UID]                         UN: b'\x1d\xa6h\x12\x08\x07pZ\x0f9\x0e\xc2'
(0009, 1006) [Route Image UID]                   UN: b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
(0009, 1008) [Image Display Information Version  UN: b'\x00\x00\x00\x00'
(0009, 1009) [Patient Information Version No.]   UN: b'\x00\x00\x00\x00'
(0009, 100c) [Film UID]                          UN: b'\x1d\xa6h\x12\x08\x07pZ\x0f9\x0e\xc2'
(0009, 1010) [Exposure Unit Type Code]           UN: b'F3'
(0009, 1080) [Kanji Hospital Name]               UN: None
(0009, 1090) [Distribution Code]                 UN: b'MAINPACS'
(0009, 10f0) [Blackening Process Flag]           UN: b'01'
(0009, 10f1) [Processing Information Flag]       UN: b'0001'
(0009, 10f2) [Normalization Flag]                UN: b'01'
(0009, 10f3) [Tone characteristic]               UN: b'01'
(0009, 10f4) [Window Value Fixed Flag]           UN: b'01'
...

This appears to be consistent behavior across all the clinical dicom files I've tried, and it doesn't make a difference whether I add the entries before or after I read the dicom file.

The file I used for this example is here:

https://drive.google.com/file/d/1SxDXG7cQQZQFluq88zPXUInMI_-6SsKK/view?usp=sharing

Thanks in advance for any help you can provide.

For anyone from the future:

I combined the code below from MrBean Bremen with some code from pydicom.values.convert_values to get the following function to use instead of dcmread:

def dcmread_convert_private(filename):
    ds = pydicom.dcmread(filename)
    for e in ds:
        if e.tag.is_private and e.VR == 'UN':
            try:
                index = (e.tag.element >> 12) - 1
                if index >= 0:
                    private_creators = ds.private_creators(e.tag.group)
                    if len(private_creators) > index:
                        private_creator = private_creators[index]
                        e.VR = pydicom.datadict.private_dictionary_VR(e.tag, private_creator)
                        # Following code taken from pydicom.values.convert_value
                        if e.value:
                            if isinstance(pydicom.values.converters[e.VR], tuple):
                                converter, num_format = pydicom.values.converters[e.VR]
                            else:
                                converter = pydicom.values.converters[e.VR]
                                num_format = None
                            byte_string = e.value
                            is_little_endian = ds.is_little_endian
                            is_implicit_VR = ds.is_implicit_VR
                            if e.VR in pydicom.charset.text_VRs or e.VR == 'PN':
                                e.value = converter(byte_string)
                            elif e.VR != "SQ":
                                e.value = converter(byte_string,
                                                  is_little_endian,
                                                  num_format)
                            else:
                                e.value = convert_SQ(byte_string,
                                                   is_implicit_VR,
                                                   is_little_endian,
                                                   e.value)
            except KeyError:
                pass
    return ds

I've only tested this on a few dicom files, so there are probably cases that aren't handled correctly. If I ever come across any, I'll try to remember to come back here and update it after I fix it.

Answer 1

The problem is that the tags are already written as UN in the file, and that is saved as Explicit VR. I would guess that it has been written as Implicit VR originally and converted to Explicit VR later - during this conversion all unknown private tags get UN as VR.

The case that the private tags are registered, and the file re-read, is not handled in pydicom - the case of UN is only handled for non-private tags. So you are out of luck here.

I think that this is something that could be added to pydicom, though - I'm not sure if this could be a problem performance-wise, but it would make sense to file a respective issue (you could do it yourself, or I could do it if you want).

Here's a quick and dirty implementation to convert the tags in a dataset after reading it:

from pydicom import dcmread
from pydicom.datadict import private_dictionary_VR

ds = dcmread('xxx.dcm'))
for e in ds:
    if e.tag.is_private and e.VR == 'UN':
        try:
            index = (e.tag.element >> 12) - 1
            if index >= 0:
                private_creators = ds.private_creators(e.tag.group)
                if len(private_creators) > index:
                    private_creator = private_creators[index]
                    e.VR = private_dictionary_VR(e.tag, private_creator)
        except KeyError:
            pass
print(ds)

Update:
This is handled in a pydicom issue and is available since the pydicom 2.2 release.