Questions tagged [distributed-computing]
The distributed-computing tag has no usage guidance.
50
questions
0
votes
0
answers
46
views
Singularity container with OpenMPI and InfiniBand (UCX)
I'm working as an intern currently, and I was asked to build a Singularity container for OpenMPI to make distributed programming possible on multiple machines of our HPC cluster using containers.
...
0
votes
1
answer
128
views
Using distcc across two ssh servers
For those who do not know, "distcc" is a program that provides an infrastructure for remote compilation. I compile the source code for a software called FreeCAD, a lot. It takes about 4 ...
- 101
0
votes
0
answers
91
views
Dynamically checking and allocating SLURM nodes within a python script
I have a computationally expensive simulation function I am looking to distribute accross a multi-node cluster. The code looks something like this:
input_tasks = [input_0, input_1, ..., input_n]
for i ...
- 109
0
votes
2
answers
77
views
What's currently the fastest way to get data in memory of one computer into memory of other computer?
What's currently the fastest way of making data in memory of one server available in the memory of another server?
Of course I I can use regular networking - but I could imagine there being faster ...
1
vote
0
answers
48
views
I'm following the tutorial for setting up tf_jobs as given in Kubeflow documentation and I run into error in ks apply stage
I'm following this instruction for setting up TF JOBs.
These are instructions that work just fine:
CNN_JOB_NAME=mycnnjob
VERSION=v0.4.0
This is my init code:
ks init ${CNN_JOB_NAME}
cd ${...
- 11
1
vote
0
answers
131
views
MATLAB GPU-enabled function? [closed]
Is there a way for me to find out whether the functions I'm using in a MATLAB script are GPU-enabled? There does not seem to be a comprehensive list (though see this link for a list of image ...
- 11
1
vote
1
answer
41
views
What is the correct way to turn a second computer into a simple expansion for the first? [closed]
I have two computers side-by side in the same large case. One is a gaming rig, one is currently just managing case fans and doesn't even have an OS.
I have a truly insane number of peripherals. ...
- 1,206
1
vote
0
answers
90
views
How to parallel windows pc in same network?
I have some windows pc in my local network, each of them have 8 cores cpu.
I have program that takes lot of cpu resource, so I want to parallel it on all of the pc.
Right now I running my program ...
- 11
-1
votes
1
answer
42
views
In Linux, is concept of Process similar to Container (in microservices?) [closed]
Process is executing an instance of an application.
Every process got its State, ID, address space etc, and isolated to each other.
It looks similar to Container that runs a specific service?
In some ...
0
votes
1
answer
41
views
Running a scripted program between a rapsberry pi and a laptop
What I have: A laptop(CPU: i7 6500U, RAM:8GB, OS: Windows 8.1) and a RPi Zero
What I know: Distributed computing is something that requires a lot of time and effort using special software such as ...
- 103
1
vote
1
answer
3k
views
Is it possible to use 2 Mac minis to do a distributed computing system
I need some advice on connecting and using 2 Mac minis as a distributed computing system. The situation is that, my current Mac mini went bad on discrete graphics card and is almost usable only in ...
- 133
0
votes
0
answers
375
views
Using multiple computers as a processing and data storage cluster
I have about 17 or so old computers that by themselves can't do much, but I was wondering if there was any way to combine them so that they could run multiple VMs in a distributed way. For example, ...
- 63
0
votes
0
answers
370
views
Can we use Ceph or Gluster FS to avoid IO bottleneck to start many VMs?
AIM: I want to start 50s of VM on s single server or on a set of PCs.
Scenerio: I tried to boot 15 VM on a single server with enough computing power(processor) and RAM but disk is a bottleneck in ...
- 136
0
votes
1
answer
1k
views
running a simulation software on a remote server
Some simulation software packages take so much time and processing power to finish the job. How can I run these kinds of software on a remote server? (e.g. a finite element method software such as ...
- 1,086
1
vote
1
answer
731
views
Delete logs after consumption: logstash
I have logstash watching a directory on the host. Every time it sees a log that matches the path I specified in the logstash config it starts to import the data into my elasticsearch cluster. Does ...
- 133