0

My app periodically spawns a CasperJS scraper:

spawn(CASPER_PATH, 
        [SCRIPTS_PATH + fileName, '--ssl-protocol=any', '--user='+user, '--scrapeId='+scrapeId, '--pass='+pass], 
        { detached: true }, 
        function (err, stdout, stderr) {});

The command spawn() needs to run works out to be something like this:

/usr/local/bin/casperjs /home/custom_user/casper-scripts/script.coffee --ssl-protocol=any --user=username123 --pass=pass123 --scrapeId=some_id_123

I'm trying to use forever to autostart the node server, and ping it when I need the scrape.

Works great when I run it node server.js as a logged in user I created, or as a root user. Also runs great when I run forever start server.js, but only if I do so as a root user - otherwise, spawn() doesn't do anything.

I've tried adding forever start (with absolute and relative paths of course) to crontab both for the root and normal user. In both cases, the server runs on specified ports, logs, accepts requests, but doesn't spawn casperjs.

I've tried adding forever start server.js to /etc/rc.local even - still nothing. The server is running, logging, functioning just fine but the only way forever is able to run spawn() is if I explicitly run it while logged in as root user.

At the end of the rope here...

How do I give forever the privileges at boot to spawn a casperjs binary? Do I need to change the sudoers file somehow? Is the fact that the folder is chowned by non-root user relevant?

EDIT:

I don't know why I didn't think to check what spawn was reporting so I started forever via non-root user's crontab and added the following after var sP = spawn():

sP.stdout.on('data', function(data) { console.log('stdout', data.toString('utf8')); });
sP.stderr.on('data', function(data) { console.log('stderr', data.toString('utf8')); });
sP.stdout.on('close', function(code) { console.log('close', code); });

Output:

stdout Fatal: [Errno 2] No such file or directory; did you install phantomjs?

close false

CasperJS is a wrapper for PhantomJS. This is odd as I have both installed on the system:

casperjs --version 1.1.0-beta3 phantomjs --version 1.9.8

It seems that crontab is unaware of phantomjs` existence in /usr/local/bin?

EDIT 2:

Crontab now looks like this:

@reboot PHANTOMJS_EXECUTABLE=/usr/local/share/phantomjs-1.9.8-linux-x86_64/bin/phantomjs NODE_ENV=production /usr/bin/forever start /home/custom_user/endpoints-server-linux/server.js

The phantomjs error has disappeared but no dice with casper actually running.

1

Could you try giving the full path to forever when you start it for rc.local. Might be that the $PATH isn't set as it is you when you're logged in.

Failing that, is there some log that forever produces? Can you tee to output where you run the command, like:

/usr/local/bin/forever start <full path to>/server.js | tee /tmp/forever-start.log

  • That's how my current crontab looks like almost exactly, but good call on the logging - updated the question – dsp_099 Jan 29 '16 at 7:08
  • Please add a logging of $PATH. And the output of which phantomjs. – Anders Olsson Jan 29 '16 at 7:11
  • echo $PATH = /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games though it seems the actual binary is in /usr/local/share/phantomjs-1.9.8-linux-x86_64/bin/phantomjs, even though which phantomjs says it's in /usr/local/bin/phantomjs – dsp_099 Jan 29 '16 at 7:12
  • So, phantomjs is a symlink. There are some caveats with cron and symlinks (but I thought they only extend to the first level, i.e. files in /etc/cron.d). Still maybe trying the local.rc way could be a good idea. – Anders Olsson Jan 29 '16 at 7:21
  • rc.local, same as crontab, launches the server just fine. I altered the nodejs code so that the path to casperjs spawn is absolute as well - nothing, no output. – dsp_099 Jan 29 '16 at 7:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.