Tek-Tips is the largest IT community on the Internet today!

Members share and learn making Tek-Tips Forums the best source of peer-reviewed technical information on the Internet!

  • Congratulations Mike Lewis on being selected by the Tek-Tips community for having the most helpful posts in the forums last week. Way to Go!

using curl with awk in command line/shell script

Status
Not open for further replies.

leopardysnow

Programmer
Mar 19, 2012
1
US
I have a text file url.txt with urls one per line. Something like:
...

If I put the following command in command line or in a shell script it works fine:
Code:
cat urls.txt | awk '{ print $0}'
This simply prints out the content of urls.txt line by line

So now I want to go 'get' the images from the urls so I thought I could do something similarly with curl such as
Code:
cat urls.txt | awk '{ curl --remote-name $0 }'
but this isn't working.
What I thought this would do is substitue each url from url.txt in $0 so that I would be doing a
Code:
curl --remote-name [URL unfurl="true"]http://www.something.com/image1.jpg[/URL]
and so on.
Any ideas how to use curl with awk and use a variable instead of hardcoded http value(since my urls are coming from a text file)? Thanks.
 
You can use the "System" function to execute shell commands within AWK:

Code:
cat urls.txt | awk '{ system("curl --remote-name " $0 }'
 
I'd use xargs:

Code:
xargs -n1 curl --remote-name <url.txt

Annihilannic
[small]tgmlify - code syntax highlighting for your tek-tips posts[/small]
 
Status
Not open for further replies.

Part and Inventory Search

Sponsor

Back
Top