Questions tagged [hadoop-yarn]
YARN (Yet Another Resource Negotiator) is a key component of second generation Apache Hadoop infrastructure. DO NOT USE THIS for the JavaScript/Node.js Yarn package manager (use [yarnpkg] instead)! Originally described by Apache as a redesigned resource manager, YARN is now characterized as a large-scale, distributed operating system for big data applications including next generation MapReduce (MR2).
hadoop-yarn
3,959
questions
0
votes
1
answer
11
views
How to understand the result of yarn queue myqueue
When I run the following command to see the status of my queue:
$ yarn queue -status my-queue
Queue Information :
Queue Name : my-queue
State : RUNNING
Capacity : 10.0%
...
- 6,194
0
votes
0
answers
9
views
Apache Ranger Yarn Plugin Installation: Class Loading and Logging Configuration Problems
Class Loading Issue with Commons Logging: I have confirmed the presence of commons-logging*.jar in the directory /usr/bigtop/3.2.0/usr/lib/hadoop-yarn/share/hadoop/hdfs/lib/. Despite this, I am ...
- 1
0
votes
0
answers
30
views
FileNotFound when creating SparkContext with YARN without HDFS
I'm just getting started with Spark / pyspark and trying to switch the resource manager from Spark Standalone to Hadoop YARN. We're using Minio for storage and so not running HDFS nodes.
The problem ...
- 3,454
0
votes
0
answers
10
views
What causes the container killed error in spark 3.2?
I read the hive table with spark 3.2, yarn log reports the error: CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM, I look at the corresponding continer log at that time, and find that the error is: ...
- 1
0
votes
0
answers
21
views
Yarn workspaces - "unknown workspace" error
I have a repo set up to use yarn workspaces. However, when I try to run a script from workspace a, I get an error: Unknown workspace "a".
I also cannot run the script directly from folder a.
...
- 23
0
votes
0
answers
16
views
How does Yarn shut down containers?
ApplicationMaster can send a request to NodeManager to close the container. At the same time, ApplicationMaster can send a heartbeat to ResourceManager to report the container to be closed.
...
- 1
0
votes
0
answers
17
views
What is the solution - Hadoop yarn is not working [closed]
I installed Hadoop to my laptop on linux but when I opened localhost:8088, yarn wasn't worked. I don't understand about this problem and I couldn't looking for anything information about this problem.
- 1
1
vote
0
answers
42
views
How to set custom tmp directory for Hadoop
We are using Hadoop version 2.10.2 and facing below error while starting server. Due to company security policy no execution permission is set on /tmp directory hence library libleveldbjni-64-1-...
- 75
0
votes
0
answers
29
views
Hadoop cannot make a MapReduce operation because is getting hang, waiting for AM container to be allocated
I used my teacher's guide to install and configure Hadoop (it's my first time using it). It says it is for Ubuntu, but it should for all linux distros since it just compiles Hadoop from source. Since ...
- 3
0
votes
0
answers
19
views
On every run jar file using hadoop it is always stuck
On every run jar file using hadoop it is always stuck here in the last line.
Here, try the Foil Jar located in the hadoop file itself, but with the same result, it gets stuck in the last line, ...
- 11
0
votes
0
answers
24
views
Redirecting distcp output to log4j from stdout
I am currently running distcp using org.apache.hadoop.util.ToolRunner.run and the output is being written to the console.
DistCpOptions opts = getDistCpOptions(srcPath, dstPath);
private ...
0
votes
0
answers
65
views
Error with `copyfiles` command When Building Docker Image: failed with exit code 127
Description:
I'm facing an issue while building a Docker image for my frontend application based on the Payload CMS ecommerce template.
You can run npx create-payload-app to try installing ecommerce ...
- 449
0
votes
0
answers
47
views
How to do an incremental migration from a multirepo to a monorepo with yarn workspaces?
I have a multirepo project. It has an "app" repository that has multiple dependencies. These other dependencies have dependencies themselves.
I would like to migrate from a multirepo to a ...
- 168
0
votes
0
answers
19
views
YARN cluster is not recognizing any active NodeManagers (Total Nodes: 0)
I am trying to run an application on using hive and hadoop.
And, my YARN cluster is not recognizing any active NodeManagers
Terminal output on: yarn node -list
anushkakundu@AnushkaHP14:/opt/hadoop/bin$...
- 1
0
votes
0
answers
13
views
Spark Application not able to run concurrently in Apache Hadoop cluster mode
when I submitted the first job to yarn cluster , the job is running ,and then submitted the second job, the second job state never run until the first job completed either success or fail. means not ...
