PHP keeps killing process before it's completed.

[aixxe]

I'm currently writing a php-cli script to handle video encoding jobs from a job server. Everything works apart from one thing.

Everytime I get to the part where it runs ffmpeg, PHP will start the process and it will run for a few seconds before the process abruptly ends. The script will then continue as normal.

I've tried using system, passthru, exec, proc_open, popen and shell_exec with the same result, increasing 'memory_limit' with ini_set (also in the actual php.ini) and enabling error_reporting(E_ALL) to see if anything was returned after the process ended.

Unfortunately none of this has worked and I've run out of ideas. The strange part is that this works absolutely fine outside of PHP, if I run the commands in bash they'll complete flawlessly, only in PHP does it stop before it should. (and always on the same frame)

The code for the process execution:

shell_exec("/usr/bin/ffmpeg -y -i \"{$data['filename']}\" -an -pass 1 -threads 4 -vcodec libx264 -vpre slow -s 640x360 -r 60 -b 1500k \"/var/prismriver/video_tmp.flv\"");
shell_exec("/usr/bin/ffmpeg -y -i \"{$data['filename']}\" -ar 44100 -ab 192k -pass 2 -threads 4 -vcodec libx264 -vpre slow -s 640x360 -r 60 -b 1500k \"/var/prismriver/video_tmp.flv\"");

Thanks for reading, I appreciate any ideas or help regarding this as it's really stopped me in my tracks.

[Trumple]

Okay from looking up here: [url]http://php.net/manual/en/function.shell-exec.php[/url]
It seems the shell_exec is a really temperamental function. It might be easier, if possible, to put your commands in a bash script and have the PHP script call the bash scripts, just as a work around. This might not be possible, I've never used shell_exec, but it seems like it would be

[aixxe]

Thanks for your input, I tried the following code instead.

file_put_contents("/var/prismriver/pass1.sh", "/usr/bin/ffmpeg -y -i \"{$data['filename']}\" -an -pass 1 -threads 4 -vcodec libx264 -vpre slow -s 640x360 -r 60 -b 1500k -rc_lookahead 1 \"/var/prismriver/video_tmp.flv\"");
exec("sh /var/prismriver/pass1.sh");
exit();

Unfortunately it still stops on the same frame, I'm really confused as to why this is happening. :c

Here's the entire output of my script.

Nitori 2 - Encoding Queue Bot
(C) Shadiku Izayoi, 2007 - 2012. <shadiku.com>

[22/06/2012 06:57:09] Set memory limit to 512M..
[22/06/2012 06:57:09] Successfully initialised queue listener support..
[22/06/2012 06:57:09] Connecting to encoding queue server..
[22/06/2012 06:57:09] Establishing persistent link to MySQLd ([e-mail])..
[22/06/2012 06:57:09] Now awaiting encoding jobs..
[22/06/2012 06:57:09] Changing working directory to '/var/prismriver/'..
[22/06/2012 06:57:09] Executing first-pass of ffmpeg, This may take a while..
FFmpeg version 0.6.5, Copyright (c) 2000-2010 the FFmpeg developers
  built on Jan 29 2012 17:52:15 with gcc 4.4.5 20110214 (Red Hat 4.4.5-6)
  configuration: --prefix=/usr --libdir=/usr/lib64 --shlibdir=/usr/lib64 --mandir=/usr/share/man --incdir=/usr/include --disable-avisynth
--extra-cflags='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -fPIC'
--enable-avfilter --enable-avfilter-lavf --enable-libdc1394 --enable-libdirac --enable-libfaac --enable-libfaad --enable-libfaadbin --enable-libgsm
--enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-librtmp --enable-libschroedinger --enable-libspeex --enable-libtheora
--enable-libx264 --enable-gpl --enable-nonfree --enable-postproc --enable-pthreads --enable-shared --enable-swscale --enable-vdpau --enable-version3 --enable-x11grab
  libavutil     50.15. 1 / 50.15. 1
  libavcodec    52.72. 2 / 52.72. 2
  libavformat   52.64. 2 / 52.64. 2
  libavdevice   52. 2. 0 / 52. 2. 0
  libavfilter    1.19. 0 /  1.19. 0
  libswscale     0.11. 0 /  0.11. 0
  libpostproc   51. 2. 0 / 51. 2. 0
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/root/prismriver-scripts/video.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: isommp42
  Duration: 00:00:44.66, start: 0.000000, bitrate: 2807 kb/s
    Stream #0.0(und): Video: h264, yuv420p, 1280x720, 2653 kb/s, 23.98 fps, 23.98 tbr, 48k tbn, 47.95 tbc
    Stream #0.1(und): Audio: aac, 44100 Hz, stereo, s16, 151 kb/s
[libx264 @ 0x1e35d300]using cpu capabilities: MMX2 SSE2Fast SSSE3 FastShuffle SSE4.2
[libx264 @ 0x1e35d300]profile High, level 3.1
[libx264 @ 0x1e35d300]264 - core 107 - H.264/MPEG-4 AVC codec - Copyleft 2003-2010 - http://www.videolan.org/x264.html - options: cabac=1 ref=5
deblock=1:0:0 analyse=0x3:0x113 me=umh subme=8 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0
deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=4 sliced_threads=0 nr=0 decimate=1 interlaced=0 constrained_intra=0 bframes=3 b_pyramid=2
b_adapt=2 b_bias=0 direct=3 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=1 rc=abr mbtree=1
bitrate=1500 ratetol=2.7 qcomp=0.60 qpmin=10 qpmax=51 qpstep=4 ip_ratio=1.41 aq=1:1.00
Output #0, flv, to '/var/prismriver/video_tmp.flv':
  Metadata:
    encoder         : Lavf52.64.2
    Stream #0.0(und): Video: libx264, yuv420p, 640x360, q=10-51, pass 1, 1500 kb/s, 1k tbn, 60 tbc
Stream mapping:
  Stream #0.0 -> #0.0
Press [q] to stop encoding
frame=  227 fps= 29 q=-1.0 Lsize=    2109kB time=9.37 bitrate=1844.7kbits/s
video:2104kB audio:0kB global headers:0kB muxing overhead 0.220476%
[libx264 @ 0x1e35d300]frame I:5     Avg QP:16.70  size: 26586
[libx264 @ 0x1e35d300]frame P:100   Avg QP:22.71  size: 10726
[libx264 @ 0x1e35d300]frame B:122   Avg QP:22.06  size:  7776
[libx264 @ 0x1e35d300]consecutive B-frames: 23.0%  4.5% 20.3% 52.3%
[libx264 @ 0x1e35d300]mb I  I16..4: 26.3% 35.1% 38.6%
[libx264 @ 0x1e35d300]mb P  I16..4:  8.9% 21.9%  6.9%  P16..4: 13.7% 12.4%  7.4%  0.0%  0.0%    skip:28.9%
[libx264 @ 0x1e35d300]mb B  I16..4:  1.0%  6.3%  4.3%  B16..8: 32.4% 15.1%  5.9%  direct: 8.5%  skip:26.5%  L0:36.6% L1:44.7% BI:18.8%
[libx264 @ 0x1e35d300]final ratefactor: 17.53
[libx264 @ 0x1e35d300]8x8 transform intra:55.0% inter:64.9%
[libx264 @ 0x1e35d300]direct mvs  spatial:98.4% temporal:1.6%
[libx264 @ 0x1e35d300]coded y,uvDC,uvAC intra: 69.8% 70.0% 40.1% inter: 31.1% 30.3% 4.7%
[libx264 @ 0x1e35d300]i16 v,h,dc,p: 62% 14%  5% 18%
[libx264 @ 0x1e35d300]i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 22% 19% 15%  5%  6%  6%  8%  7% 11%
[libx264 @ 0x1e35d300]i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 21% 17%  9%  6%  9%  8% 10%  8% 12%
[libx264 @ 0x1e35d300]i8c dc,h,v,p: 54% 26% 15%  5%
[libx264 @ 0x1e35d300]Weighted P-Frames: Y:16.0%
[libx264 @ 0x1e35d300]ref P L0: 56.7% 20.3% 14.5%  4.1%  2.9%  1.2%  0.3%
[libx264 @ 0x1e35d300]ref B L0: 90.2%  7.2%  2.0%  0.6%
[libx264 @ 0x1e35d300]ref B L1: 95.9%  4.1%
[libx264 @ 0x1e35d300]kb/s:1817.24

I think the issue is to do with the process being spawned by PHP (and something or some kind of limit being imposed that stops it from proceeding past frame 227) as it doesn't have any issue when the command is run from bash.

It's not memory, it can't be timeout (PHP-CLI doesn't have this, if i recall correctly), but I've got no idea why it stops on that specific frame every time.

[KmartSqrl]

Is there any particular reason you're using php?

[aixxe]

[QUOTE=KmartSqrl;36443741]Is there any particular reason you're using php?[/QUOTE]

It's just what I'm most familiar with.

[Trumple]

Okay try changing the PHP max execute time, by default I think it is 30

[editline]22nd June 2012[/editline]

[url]http://php.net/manual/en/function.set-time-limit.php[/url]

[aixxe]

[QUOTE=Trumple;36443948]Okay try changing the PHP max execute time, by default I think it is 30

[editline]22nd June 2012[/editline]

[url]http://php.net/manual/en/function.set-time-limit.php[/url][/QUOTE]

I thought of that, unfortunately:

[quote=php.net]max_execution_time integer

This sets the maximum time in seconds a script is allowed to run before it is terminated by the parser. This helps prevent poorly written scripts from tying up the server. The default setting is 30. [b]When running PHP from the command line the default setting is 0.[/b][/quote]

[Trumple]

Hmmmmmmmmm
Okay my concern is the construction of the actual command in PHP, for some reason. Is it possible just to pre-make the static bash script, then call it from PHP, just to rule out some possibilities? Just to test, obviously

[KmartSqrl]

[QUOTE=Diaklu;36443762]It's just what I'm most familiar with.[/QUOTE]

How's the file being called up? Is something hitting a webserver that's serving the page when this needs to run or is it a cron job or what?

[aixxe]

[QUOTE=KmartSqrl;36444280]How's the file being called up? Is something hitting a webserver that's serving the page when this needs to run or is it a cron job or what?[/QUOTE]

I run beanstalkd, a lightweight job server. Basically, the web server receives a file and then it's sent as a job to the job server like this.

// Queue for encoding.
$pheanstalk->useTube('ffmpeg')->put(json_encode(array('id' => $id_int, 'filename' => $config['dir']['temp_videos']."/".$_FILES['vidya']['name'])));
header("Location: {$config['host']}/upload/success/");

Then the script is waiting for jobs like this:

while ($job = $pheanstalk->watch('ffmpeg')->ignore('default')->reserve()) {

The code posted before that ran ffmpeg worked on my previous Debian 6 installation. I'm rewriting this code as I unfortunately forgot to back up the prevoius version.

[KmartSqrl]

If the full script that's waiting for jobs isn't something too complex it might be worth transferring it to another language as long as you've got time to play around with something else.

[aixxe]

[QUOTE=KmartSqrl;36444408]If the full script that's waiting for jobs isn't something too complex it might be worth transferring it to another language as long as you've got time to play around with something else.[/QUOTE]

Thanks for the advice, I'm going to try a different version of PHP and ffmpeg before resorting to rewriting the whole script.

I'll still be checking the thread for any other pointers as to what this could be caused by.

[KmartSqrl]

No problem. I'm always kind of hesitant to use PHP for things like this just because it wasn't really designed for it, and it's not really the best piece of software out there.

[aixxe]

Okay, strangely enough it appears that this problem is only happening in this file specifically.

I took the shell_exec code from the non-working file and pasted it in an empty file, ran it and it encoded the video fine.

I'll be looking through the file and debugging, this took longer than I had hoped to "resolve" but I'm glad I found it. Thanks for your input everyone.

[B]Edit:[/B] Close enough, removing the amount of stuff that was written to the console resulted in more frames being written until I finally found out what it was.

The problem was I was calling the script like this: "php < nitori.php" instead of "php nitori.php". (Mainly because an old script I tried didn't work that way so I got into a habit of running it like that.)

[swift and shift]

using php for long running processes is never a good idea