Asyncronous processing 4

[vcwanabe]

Hi,
I have a small app that i want to be processing, while at the same time, i want it to be looking for an ftp server. I wrote a script to connect to the server and am running the script from a system command. The problem i have is that the system command halts the apps processing until it returns(which is a little while if the ftp server is not reachable).

Is there a way in c to run a script in a loop while allowing the main app to continue processing?

 

[Salem]

Which operating system are you using?


--

 

[vcwanabe]

Linux, thanks

 

[Salem]

Well a very simplistic way would be to do this

if ( fork() == 0 ) {
    system("ftp");  // or whatever command line you have
    _exit(0);       // all done with the FTP
}
// rest of code to run in parallel with the ftp

However, you normally want to have some more control over the two processes, which usually means you have to get to grips with the execl() family of functions.

--

 

[marsd]

Well the problem with the simple way is that you don't have any way after fork and systems fork, to deal with what's
going on in the children. (I know it was a trivial example-
not criticizing at all Salem, )

I would suggest using threads on linux since this clone()
call shares ALL the parents data and is kept up to date this
way. Otherwise you have to use some IPC to synchronize and
update child and parent, even if it is as rudimentary as
signals and a pair of pipes, or a fifo. I would also
suggest looking at libcurl which is so easy to use it's
a crime and supports threaded ops.

http://curl.haxx.se/libcurl/

If you want an example let me know.

 

[jstreich]

quoting marsd in bold below:
I would suggest using threads on linux since this clone()
call shares ALL the parents data and is kept up to date this
way. Otherwise you have to use some IPC to synchronize and
update child and parent, even if it is as rudimentary as
signals and a pair of pipes, or a fifo.
I believe that things that are dynamically allocated are shared between processes, no? And there is the wait command so you can know when it is done. If all he is doing is continually looking to see if a ftp server is running then there really isn't any data each thread of execution (in this case each process) needs to know except exit status.

Going back to the fork code a moment:

[red]int* status;[/red]
if ( fork() == 0 ) {
    [red]return[/red] system("ftp");  
    // or whatever command line you have
    [red]// returns ftp's exit status to program[/red]
}
[red]else
{
  // stuff to do in parallel
  // then when need to know stuf about the ftp
  wait(status);
  if(0 != *status || NULL == status)
  {
    // ftp failed.
  }
}[/red]

Using threads is much more intresting, and allows more information to be passed (as mentioned) but can be harder to work with.
You'd start by creating a function that does that does the check you want to happen asynchronously (say constnatly using ftp or other passed command). The parameters have to be passed by way of a void pointer (if there are parameters) and the return must be a void pointer.

void* checkFTP(void*str)
{
  while(1 == 1)
  {
    system((char*)str);
    
  }
}

Then you write code to create and initalize the thread.

pthread_t check; //the thread

void main()
{
  pthread_attr_t thread_template;
  pthread_attr_init(&thread_template);
  pthread_attr_setscope(&thread_template, PTHREAD_SCOPE_SYSTEM);
  pthread_create(&check, &thread_template, (void*(*)(void*))check, (void*)"ftp"); 
  // the thing in the quotes would be your command

  // parallel stuff here
  // use shared mutable state in global scope to communicate
  // w/ other thread.  Be sure to protect it w/ mutex

  pthread_join(check,NULL);
}

 

[marsd]

Jstreich:
Dynamic allocation is to be avoided when
possible in trivial programs IMHO.

No reason to call system() after a fork.
As a matter of fact it's redundant.

The OP never said that all the the system()
called program was going to do was look
for an ftp service binding port 21 on a remote
host. Too simplistic an interpretation of
what is needed often leads to redesign.
Reaping a return code from system() hardly
ever suffices for anything.

To do this in C libcurl and threads are a
decent solution. Otherwise write it in perl
or tcl or python.

 

[cdlvj]

yes, look at the exec functions, what you can do in the ftp script is set a lock file which can be looked at by the calling program.

or by using msgctl, in an independent program which runs all the time, but is in sleep mode, the calling program sends a msg to the ftp module, it awakens, and does the ftp, and when the ftp is done you can implement signals to tell the calling program Done/error or whatever.

 

[jstreich]

Dynamic allocation is to be avoided when
possible in trivial programs IMHO.
Why? It's got some overhead, and beginners make a lot of mistakes with dynamic allocation... But, the overhead of dynamic memory allocation is trival with the faster buss speeds and such that modern computers buy.

No reason to call system() after a fork.
As a matter of fact it's redundant.
A previous post used it, and I just carried over their code, and added to it. Added code is in red. The system is in leiu of an exec command. Granted that the exec will create an instance of the shell and execute that command in it, but this allows you to run shell scripts that wouldn't be able to be exec'ed... In this case exec is the way to go.

The OP never said that all the the system()
called program was going to do was look
for an ftp service binding port 21 on a remote
host. Too simplistic an interpretation of
what is needed often leads to redesign.
And throwing giant libraries at a simple problem leads to unneeded bloat.

Reaping a return code from system() hardly
ever suffices for anything.
Err, especially when main return void in most (all?) C programs... Sorry, I'm C++/Java guy. The return code would tell you if the ftp program returned error. If the program was smaller than an FTP, it would...

That's why I offer tips on using the pthread library as well.

To do this in C libcurl and threads are a
decent solution. Otherwise write it in perl
or tcl or python.
Curl is probably over kill. All he wants to know is staus info from an FTP... The POSIX pthread libs (my later advice) is the way to go. Granted I left out the mutexs and condition varibles but if he keeps communication simple enough he should be able to get away with only one mutex.

 

[vcwanabe]

wow thanks so much to all your replies. First, i obviously am a newbie, so you need to talk to me like a child,lol. Second, I appreciate all the advice because i want to know the options that are available. As far as this particular app, the focus is on small, simplistic, solid. The curl library is probably solid but i do not wish to include libraries if do not need to. But i reserve the right to use that suggestion in the future,lol. The threads are interesting and i intend to delve into that when i have time to learn advanced things. Right now i'm going to goolge the fork and exec options and try to understand them. Thanks for all the input!!

 

[marsd]

vcwanabe:
If you are going to go with fork() and execl() you might
want to read the man page for signal, sigaction.etc and
consider what kind of interaction you will need with
the exec'd program. Once again you enter the area of
IPC here, which can be mostly avoided using threads.
The message queue idea proposed earlier is overkill
IMHO, and message queues are deprecated(Stevens).


jstreich:
We avoid dynamic allocation because it IS easy to
commit mistakes. If you have a program like:

#define LEN 255
void myabort(int);

int main(void) {
int pipes[2], len;
pid_t child;
char buf[LEN];
char cmd[] = "/bin/less";

           pipe(pipes);
           signal(SIGPIPE,myabort);

           if ( (child = fork()) < 0) {return -1;}
           if (child == 0) {
                           close(pipes[1]);
                           dup2(pipes[0],STDIN_FILENO);
                           execl(cmd,cmd,(char *) 0);
           }
           /*parent*/
           close(pipes[0]);
           if ( (fgets(buf,LEN,stdin)) != NULL) {
                 len = strlen(buf);
                 write(pipes[1],buf,len);
           }
           wait(NULL);      
           return 0;
}

Why malloc()?

..main() returns void..
The C standard(s) defines the return value from main()
as type int. Check out comp.lang.c.

..gigantic libraries...
Point taken. I'd like to point out that in the tradition
of creeping gigantism both C++ and Java have more bloat
in a comparably linked program than C does, and that
gcc and dynamic linking makes optimization and inclusion
of 'gigantic libraries' pretty painless.

..system allows you to run shell scripts that wouldn't...
Wrong. System forks and calls exec*.
Try execve() if you are having trouble with shell
scripts.

 

[jstreich]

We avoid dynamic allocation because it IS easy to
commit mistakes. If you have a program like ... Why use malloc()?
In that case there isn't a reason... but if you warnr

The C standard(s) defines the return value from main()
as type int. Check out comp.lang.c.
Ah, most C books that I've ever seen keep the legacy void return. Having written more C++ I'm used to main returning int.

I'd like to point out that in the tradition
of creeping gigantism both C++ and Java have more bloat
in a comparably linked program than C does,
I'll give you that. Java is also intrepreted and slower to load. They also are "heavier weight" and higher level languages than C. Each has there place. This particular problem would be really easy to code in Java, but program would take a longer to load and would be a lot bigger...

and that gcc and dynamic linking makes optimization and inclusion of 'gigantic libraries' pretty painless.
True, and I suppose with current technology even if you did get the whole library loaded it wouldn't make too much of an impact with as much ram as computer have and as fast as the buss speeds are. I suppose I gave curl the shaft too easily.


Wrong. System forks and calls exec*.
Try execve() if you are having trouble with shell
scripts.
system() forks a copy of the shell. From the man page:

The system() function forks to create a child process that in turn invokes one of the exec family of functions (see exec(2)) on the shell to execute string. If vfork(2) or the exec function fails, system() returns -1 and sets errno to indicate the error.

It goes on to say what shell it uses and such.

I didn't realize that exec could run shell scripts, but apperntly only if they have the first line being the interpreter:

Each of the functions in the exec family replaces the current process image with a new process image. The new image is constructed from a regular, executable file called the new process image file. This file is either an executable object file or a file of data for an interpreter. There is no return from a successful call to one of these functions because the calling process image is overlaid by the new process image.

An interpreter file begins with a line of the form

#! pathname [arg]

where pathname is the path of the interpreter, and arg is anoptional argument.

And now I see why we start scripts with the:
#! /usr/bin/bash

Learn something new everyday!

 

[cdlvj]

http://www-106.ibm.com/developerworks/library/l-pthred.html

is an article on threads, although I would use the native UNIX stuff.

Simplest way to do this is to have a main process and another process like a daemon.
look at msgctl.

The second process would simply do a msgrcv in wait mode for any messages, and be inactive. Once a message arrives from the main process, would initiate the ftp, and when done do a msgsnd with revelent info,Kill -HUP <parentid>, and go back to sleep.

The main process would define a signal function to handle the HUP interrupt. So no matter which code is executing, it stops for the interrupt, reads the message, does its thing, and return to the place where it halted.

The other way is to use the fork(), which would do the ftp in a program or shell script. Been awhile since I have done this, but you can find numerous examples and samples. The child could also use msgctl to let the parent know, and exit.

Either way, multiple ftps could be running.