awk - change field separator

[georgeocrawford]

Hi,

I'm piping the results of an

ls -l

into this awk command:

awk '{print $9"\tF\t"$5}'

However, when this is run on a filename with a space in it, I just get the first word of the filename printed.

I think I need to change the default field separator from whitespace to just tabs - is that correct? If so:

awk 'BEGIN { FS = "\t" } ; { print $9"\tF\t"$5}'

which I tried, but it doesn't work (prints nothing).

Can someone please help with my command-line syntax for awk!

______________________

George

 

[PHV]

In man awk pay attention to substr.
ls -l doesn't output tab char.

Hope This Help
PH.

 

[vgersh99]

awk -v FS="$(echo -e '\t')" '{ print $9"\tF\t"$5}'

vlad
+----------------------------+
| #include<disclaimer.h> |
+----------------------------+

 

[aigles]

ls -l | awk '
NR>1 {
file = &quot;&quot;;
for (i=9; i<=NF; i++) file = file $i &quot; &quot;;
printf file &quot;\tF\t&quot; $5;
} '

Jean Pierre.

 

[georgeocrawford]

Thanks for the replies.

However, I need a little more help please!

Here is the current line in my shell script:

find "$ROOT_FOLDER" \! -name '.*' -type f -print0 | xargs -0 ls -l  | awk '{print $9"\tF\t"$5}' | sed -e 's/'"$ESCAPED_ROOT_FOLDER"'\///g;' >/tmp/Files$table.$$

What's the best way to modify this to work with filenames with spaces?

I tried

find "$ROOT_FOLDER" \! -name '.*' -type f -print0 | xargs -0 ls -l  | awk -v FS="$(echo -e '\t')" '{ print $9"\tF\t"$5}' | sed -e 's/'"$ESCAPED_ROOT_FOLDER"'\///g;' >/tmp/Files$table.$$

which just gives lines like

             F

in the text file, and I tried

find &quot;$ROOT_FOLDER&quot; \! -name '.*' -type f -print0 | xargs -0 ls -l  | ls -l | \ awk 'NR>1 { file = &quot;&quot;;  for (i=9; i<=NF; i++) file = file $i &quot; &quot;; printf file &quot;\tF\t&quot; $5; } ' | sed -e 's/'&quot;$ESCAPED_ROOT_FOLDER&quot;'\///g;' >/tmp/Files$table.$$

which gave syntax errors.

I see what you're doing here, Jean Pierre - taking everything from $9 to the end of the line. But how do I incorporate this into my shell line?

______________________

George

 

[PHV]

Try something like this:
find &quot;$ROOT_FOLDER&quot; \! -name '.*' -type f -print0 |
while read f
do
ls -l &quot;$f&quot; | awk '{
file=&quot;&quot;;for(i=9;i<=NF;i++)file=file $i &quot; &quot;
print file &quot;\tF\t&quot; $5
}' | sed -e &quot;s!$ESCAPED_ROOT_FOLDER/!!g;&quot;
done >/tmp/Files$table.$$



Hope This Help
PH.

 

[aigles]

In my previous post, printf is to be replaced by print.
Try this (using -ls option of find)

find &quot;$ROOT_FOLDER&quot; \! -name '.*' -type f -ls | awk '{ file = &quot;&quot;; for (i=11; i<=NF; i++) file = file $i &quot; &quot;; print file &quot;\tF\t&quot; $7; } ' | sed -e 's/'&quot;$ESCAPED_ROOT_FOLDER&quot;'\///g' >/tmp/Files$table.$$


Jean Pierre.

 

[georgeocrawford]

thanks, but it won't work!

I pasted exactly from above, and get:

/path/to/script.sh: line 66: ?while: command not found
/path/to/script.sh: line 67: ?do: command not found
/path/to/script.sh: line 68: ???ls: command not found
/path/to/script.sh: line 72: ?done: command not found

what have I done wrong?

Also - did you mean

sed -e "s!$ESCAPED_ROOT_FOLDER/!!g;"

or

sed -e "s/$ESCAPED_ROOT_FOLDER///g;"

?

______________________

George

 

[aigles]

which Shell use?

Jean Pierre.

 

[georgeocrawford]

/bin/bash - OS X 10.3

______________________

George

 

[georgeocrawford]

cross-post sorry

To Jean Pierre:

with your new code I get:

find: -ls???: unknown expression primary
awk: syntax error at source line 1
 context is
        { file = >>>  &quot;&quot;;? <<< 
awk: illegal statement at source line 1

and to PHV - I get:

/path/to/script.sh: line 66: ?while: command not found
/path/to/script.sh: line 67: ?do: command not found
/path/to/script.sh: line 68: ???ls: command not found
/path/to/script.sh: line 72: ?done: command not found

______________________

George

 

[PHV]

Where are this ? before the keyword coming from ?
Don't you have ksh on your MAC ?(bash is often broken ...)

 

[aigles]

Sorry, the -ls option of find seems to be specific to AIX.
[tt]
/home/jp> find . -ls
223788 4 drwxrwxrwx 3 jp jpgroup 2048 Feb 2 17:00 .
223745 0 -rw-rw-r-- 1 jp jpgroup 0 Feb 2 15:08 ./File Name
223746 4 -rw-rw-r-- 1 jp jpgroup 112 Feb 2 15:23 ./f.awk
223747 4 -rw-rw-r-- 1 jp jpgroup 406 Feb 2 15:56 ./a.sh
223748 4 -rw-rw-r-- 1 jp jpgroup 509 Feb 2 12:20 ./GetDate.sh
223749 0 -rw-rw-r-- 1 jp jpgroup 0 Feb 2 17:00 ./f.txt
94592 4 drwxrwxr-- 2 jp jpgroup 512 Jan 26 15:29 ./PERL
94593 4 -rwxrwxr-- 1 jp jpgroup 807 Jan 26 11:52 ./PERL/first.pl
94594 4 -rw-rw-r-- 1 jp jpgroup 1245 Jan 26 15:29 ./PERL/aaa
[/tt]

field 7 = file size
field 11- = file name

Read the find man page to see if an equivalent option exists.

For your awk problem, I don't know where is the error.
The behavior of your Shell is strange.





Jean Pierre.

 

[georgeocrawford]

ok

if i run this:

find /Users/georgecrawford/Documents \! -name '.*' -type f -ls | awk '{ file = &quot;&quot;;  for (i=11; i<=NF; i++) file = file $i &quot; &quot;; print file &quot;\tF\t&quot; $7; } ' | sed -e 's/'&quot;$ESCAPED_ROOT_FOLDER&quot;'\///g' >/tmp/Files$table.$$

I get this:

awk: syntax error at source line 1
 context is
        { file = >>>  &quot;&quot;;? <<< 
awk: illegal statement at source line 1

There is no find: ls error any more

______________________

George

 

[georgeocrawford]

OK - SOLVED!

Very strange, but it's happened before. My web browser added some kind of whitespaces into the code. The two places were after

-ls

and after

file = "";

. When I deleted the gap, and added a single space, it worked perfectly.

Thanks so much for all your help and quick replies!

______________________

George

 

[georgeocrawford]

Not quite finished, I'm afraid!

For my script to work, the result I get from this line must be EXACTLY the same as the filename - i.e. no extra spaces, etc.

I changed the line to this:

find &quot;$ROOT_FOLDER&quot; \! -name '.*' -type f -ls| awk '{ file = &quot;&quot;; for (i=11; i<NF; i++) file = file $i &quot; &quot;; file = file $i; print file &quot;\tF\t&quot; $7; } ' | sed -e 's/'&quot;$ESCAPED_ROOT_FOLDER&quot;'\///g' >/tmp/Files$table.$$

... to avoid an extra space at the end of each filename.

However, if a filename DOES have a space at the end (it's not my files that I'm scanning, and they are often badly-named!), how can I ensure that it makes it into the final printout? I checked, and the

find -ls

DOES print the space, so I guess I need to alter the

for (i=11; i<NF; i++)

to include the WHOLE of the last field.

Any ideas?

______________________

George

 

[aigles]

Try this new solution :
[tt]
ROOT_FOLDER=.
ESCAPED_ROOT_FOLDER=.
find &quot;$ROOT_FOLDER&quot; \! -name '.*' -type f -ls | awk '{
pos = match($0,$9 &quot; &quot; $10 &quot; &quot ;
pos += RLENGTH ;
file = substr($0, pos, 9999);
print file &quot;\tF\t&quot; $7;
} '
sed -e 's/'&quot;$ESCAPED_ROOT_FOLDER&quot;'\///g' # >/tmp/Files$table.$$
[/tt]

Jean Pierre.

 

[aigles]

Sorry, I forgot the pipe afer the order awk command/
[tt]
ROOT_FOLDER=.
ESCAPED_ROOT_FOLDER=.
find &quot;$ROOT_FOLDER&quot; \! -name '.*' -type f -ls | awk '{
pos = match($0,$9 &quot; &quot; $10 &quot; &quot ;
pos += RLENGTH ;
file = substr($0, pos, 9999);
print file &quot;\tF\t&quot; $7;
} ' | \
sed -e 's/'&quot;$ESCAPED_ROOT_FOLDER&quot;'\///g' # >/tmp/Files$table.$$
[/tt]

Jean Pierre.

 

[georgeocrawford]

saw that - thanks.

I got it to work a slightly different way:

find "$ROOT_FOLDER" \! -name '.*' -type f -ls | awk '{
	pos = match($0,"/");
	file = substr($0, pos, 9999);
	print file "\tF\t" $7;
	}' | sed -e 's/'"$ESCAPED_ROOT_FOLDER"'\///g' >/tmp/Files$table.$$

Anything wrong with that?

______________________

George

 

[aigles]

If $ROOT_FOLDER containts a relative path, your awk program removes the first directory from file path.

For example if ROOT_FOLDER=&quot;.&quot;, the find command display file names in the form : ./file
In that case your awk program removes the leading '.' and
display /file.

If $ROOT_FOLDER containts an absolute path, your awk program is valid.


Jean Pierre.