3

Recently I moved from the Oracle JDK 8 to Open JDK 8 (version: openjdk-1.8.0.242.b08) for Cloudera. Cloudera version is 5.16.2. I have Kerberos and TLS enabled. When I am using Oracle JDK the below issue doesn't occur but when I switch to Open JDK then I am getting the below error. Please note that the proxy user is already enabled from the CM -> HDFS -> configuration.

JobHistoryServer    
Error starting JobHistoryServer
org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Error creating done directory: [hdfs://XXXX.YYY.COM:8020/user/history/done]
    at org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager.tryCreatingHistoryDirs(HistoryFileManager.java:680)
    at org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager.createHistoryDirs(HistoryFileManager.java:616)
    at org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager.serviceInit(HistoryFileManager.java:577)
    at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
    at org.apache.hadoop.mapreduce.v2.hs.JobHistory.serviceInit(JobHistory.java:95)
    at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
    at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107)
    at org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.serviceInit(JobHistoryServer.java:154)
    at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
    at org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.launchJobHistoryServer(JobHistoryServer.java:229)
    at org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.main(JobHistoryServer.java:239)
Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): User: [email protected] is not allowed to impersonate mapred/[email protected]
    at org.apache.hadoop.ipc.Client.call(Client.java:1504)
    at org.apache.hadoop.ipc.Client.call(Client.java:1441)
    at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:231)
    at com.sun.proxy.$Proxy16.getFileInfo(Unknown Source)
    at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:788)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:258)
    at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:104)
    at com.sun.proxy.$Proxy17.getFileInfo(Unknown Source)
    at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:2168)
    at org.apache.hadoop.fs.Hdfs.getFileStatus(Hdfs.java:133)
    at org.apache.hadoop.fs.FileContext$14.next(FileContext.java:1143)
    at org.apache.hadoop.fs.FileContext$14.next(FileContext.java:1139)
    at org.apache.hadoop.fs.FSLinkResolver.resolve(FSLinkResolver.java:90)
    at org.apache.hadoop.fs.FileContext.getFileStatus(FileContext.java:1139)
    at org.apache.hadoop.fs.FileContext$Util.exists(FileContext.java:1604)
    at org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager.mkdir(HistoryFileManager.java:733)
    at org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager.tryCreatingHistoryDirs(HistoryFileManager.java:663)
Improve this question
1
  • 3
    Quoting "Hadoop and Kerberos, the Madness beyond the Gate" from the top of my mind >> THE HADOOP TEAM IS SCARED OF NEW VERSIONS OF JAVA
    – Samson Scharfrichter
    Commented Feb 3, 2020 at 18:14

3 Answers 3

Reset to default
3

This is a known bug I filed with openJDK.

"KerberosTicket client name refers wrongly to sAMAccountName in AD", Major loss of Function.

It is tracked here: https://bugs.openjdk.java.net/browse/JDK-8239385

Improve this answer
3

This issue is resolved. The issue was due to the cross-realm referrals support for Kerberos (JDK-8215032). You need to set -Dsun.security.krb5.disableReferrals=true property of the service or to set this is java.security file.

Improve this answer
1
1

Might be related to my problem: https://bugs.launchpad.net/ubuntu/+source/openjdk-8/+bug/1861883

I also reported this to the OpenJDK bug report site (twice) but no reaction so far.

If your problem disappears after downgrading to OpenJDK 1.8.0 232 we've been hit by the same bug.

Improve this answer
0

Not the answer you're looking for? Browse other questions tagged or ask your own question.