All Questions
Tagged with optimization bash
7
questions
1
vote
1
answer
155
views
Speed up grep usage inside bash script
I am currently working on creating a bash script that is supposed to process large log files from one of my programs. When I first started the script took around 15 seconds to complete which wasn't ...
- 11
0
votes
1
answer
246
views
Parse huge amounts of files efficiently
I have a folder that holds hunderds of thousands of files called hp-temps.txt. (There are also tons of subfolders)
The content of these files looks like this for example:
Sensor Location ...
- 111
0
votes
0
answers
188
views
Why integer division is faster than bitwise shift in shell?
I'm comparing performance of bash and dash (default sh in Xubuntu 18.04).
I expect sh to be faster than bash
I expect bitwise shift to be faster than division operator.
However, I'm getting ...
- 1,060
2
votes
1
answer
605
views
How to pipe output of command to two separate commands and store outputs
I have a really long command that runs over a huge file and I am forced to run it twice which doubles the time it takes to run.
This is what I am doing at the moment:
x=$(command | sort -u)
y=$(...
- 25
7
votes
3
answers
1k
views
Do file descriptors optimise writing to files?
Is it equivalent to have commands print to a file directly, as opposed to writing to a file descriptor?
Illustration
Writing to file directly:
for i in {1..1000}; do >>x echo "$i"; done
...
user147505
4
votes
3
answers
116
views
Script optimisation to find duplicates filename in huge CSV
I have several CSV file from 1MB to 6GB generated by inotify script with a list of events formatted as is:
timestamp;fullpath;event;size.
Those files are formatted like this :
timestamp;fullpath;...
- 9,764
2
votes
1
answer
87
views
optimisation of bash globs
Looking for a guide to optimising regexp matches in bash.
I have a script that loops over a very long list of URLs looking for patterns. Currently it looks a little like the fragment below. Is there ...
- 21