You can just do forfin * (or other shell glob), unless you need find’s fancy search/filtering features.
The shell glob isn’t just simpler, but also more robust, because it works also when the filename contains a newline; find .. | whileread -r will crap out on that. Also apparently you want while IFS= read -r because otherwise read might trim whitespace.
If you want to avoid that problem with the newline and still use find, you can use find -exec or find -print0 .. | xargs -0, or find -print0 .. | while IFS= read -r -d ''. I think -print0 is not standard POSIX though.
Still, that was a speed bump that I guess everyone dealing with mass compressed log files has to figure out on the fly because zcat can’t read uncompressed files ! argg !!!
for i in $(ls); do zcat $i 2>/dev/null || cat$i; done | grep mysearchterm
Yeah, it’s a pain. Leads to bad one liners:
for i in $(ls); do zcat $i || cat $i; doneBtw, don’t parse ls. Use
find |while read -rinstead.find -maxdepth 1 -name "term" -print |while read -r file do zcat "$file" 2>/dev/null || cat "$file" doneWon’t this cause cat to iterate through all files in the cwd once zcat encounters an issue, instead of just the specific file?
You are correct. This probably produces something more similar to what you’d want the original command to do, but with better safely:
find -- . -type f -regex '^\./[^/]*$' -exec sh -c -- 'for file in "${@}"; do zcat "${file}" || cat "${file}" || exit; done' sh '{}' '+'That assumes you want to interact with files with names like
.hidden.txt.gzthough. If you don’t, and only intend to have a directory with regular files (as opposed to directories or symbolic links or other types of file), using this is much simpler and even safer, and avoids using files in a surprising order:for i in *; do zcat -- "$i" || cat -- "$i" || exit; doneOf course, the real solution is to avoid using the Shell Command Language at all, and to carefully adapt any program to your particular problem as needed: https://sipb.mit.edu/doc/safe-shell/
Yeah, i was tired and had $file there first, then saw that you wanted to cat all in directory. Still tired, but i think this works now.
You can just do
for f in *(or other shell glob), unless you needfind’s fancy search/filtering features.The shell glob isn’t just simpler, but also more robust, because it works also when the filename contains a newline;
find .. | while read -rwill crap out on that. Also apparently you wantwhile IFS= read -rbecause otherwise read might trim whitespace.If you want to avoid that problem with the newline and still use find, you can use
find -execorfind -print0 .. | xargs -0, orfind -print0 .. | while IFS= read -r -d ''. I think-print0is not standard POSIX though.Doesn’t that depend on the shell?
I don’t think so and have never heard that, but I could be wrong.
Thanks !
But still we shouldn’t have to resort to this !
Also, can’t get the output through pipefor i in $(ls); do zcat $i || cat $i; done | grep mysearchtermthis appears to workfind . -type f -print0 | xargs -0 -I{} sh -c 'zcat "{}" 2>/dev/null || cat "{}"' | grep "mysearchterm"Still, that was a speed bump that I guess everyone dealing with mass compressed log files has to figure out on the fly because zcat can’t read uncompressed files ! argg !!!for i in $(ls); do zcat $i 2>/dev/null || cat $i; done | grep mysearchterm