is there a way to determine all files located on a web server????

[pbyford99]

i am trying to locate a particular file on a forign web server which numerous pages. Is there a way in which I can determine what files / pages are on the server in a similar fashion as when no index.html has been defined and the server contents is displayed in the form of a hiearchical tree. any help much apriciated??

 

[ifincham]

Hi,





If you have access to the server and its linux/unix you can just use 'find' or 'grep' to recusively search directories and all subdirectories to locate files or search their content.





However, if its someone else's server you are pretty much limited to using a bot of some sort (spider) to trawl the content via links. These work by retrieving a start page and then scanning for links and exploring each in turn for further content ad infinitum (i.e. the kind of software used by google etc to index websites) . See -->

http://www.botspot.com/search/s-webdev.htm

for some examples... Of course, if the site uses dynamic content generation like jsp or asp then its not so easy!



An alternative (but similar) technique is to use software to mirror the entire remote site locally and then search it locally with 'grep', etc.



Hope this helps