PHP, APC and shared memory

PHP is not exactly well known for its code quality (and I mean the interpreter, C code, not the canonically bad user-written PHP code). I think working with FreeBSD code spoiled me to expect more :)

One "interesting" side of it is that even now, years after FastCGI has become the way to deploy PHP in production for any non-trivial sites, the most popular opcode / shared memory cache, the APC is broken in the way it handles this situation. Instead of actually sharing memory, it allocates new "shared" memory (sysv shm or mmap) for each process, thus preventing any sharing. And the bug basically has a "wontfix" status during all these years.

Well, I'm hereby declaring I'm officially sick of it and am hereby providing a 100-line patch for the situation.

This (very simple) patch basically forces every instance to mmap the same file to the same address. This will work only if there is one APC shared memory segment (the default situation), with the only inconvenience being finding a memory region to mmap the file to (must be the same for all processes).

On my 64-bit FreeBSD dev machine it works with these settings:

The crucial settings are:

  • apc.mmap_address - the new behaviour will be activated only if this value is non-zero. It is a direct memory address to use for mmaping the shared memory region (whose size is configured as shm_size MB). This value I've given is in the middle of FreeBSD user process address space for 64-bit installations - other operating systems might (or might not) use different values.  A good value for 32-bit processes might be 1073741823. The actual address here doesn't matter as long as it's unused and page-aligned. This is not an evil hack, the VM system will work around this mmaped region.
  • apc.mmap_file_mask - this is the filename of the file to use in mmap(). If mmap_address is non-zero, this will be interpreted as an exact, absolute filename. I've so far tested it with a pre-created file (dd if=/dev/zero of=/var/tmp/apc-mmap bs=1m count=128) but it should also auto-create files.

The other settings are as usual for APC. The patch applies to APC's released version APC-3.1.3p1.

I am not sure how this will interact with APC's various locking modes. I think that both default modes (sysv semaphores and flock) are safe; I don't know if spinlocks are safe - it depends on if the spinlocks themselves are stored in the shm or somewhere too-clever.

It is also somewhat tricky in case of data corruption. If a PHP or APC bug corrupts the shared memory region, it is not enough to restart the web server (as before), but also to delete the mmaped file before the server (and any php-cgi processes) is started.

The whole thing can be easily modified to support more shm segments but it works for now and we'll see if PHP developers will accept this kind of a patch in the first place (APC conflicts with their commercial Zend Accelerator product).

#1 Re: PHP, APC and shared memory

Added on 2010-06-17T21:44 by hshh

Why not try another opcode cache module, such as xcache?

#2 Re: PHP, APC and shared memory

Added on 2010-06-19T11:53 by Ivan Voras

Two reasons basically: 1) I'm more familiar with APC and 2) I like the user-accessible key-value cache it implements

#3 Re: PHP, APC and shared memory

Added on 2010-07-16T07:53 by Rob Ryan

I'm trying this and it pretty much works but seems to crash with the apache error log giving a fcgi error of: exit(communication error), get unexpected signal 11.

I'm assuming this could be because of some kind of memory conflict? It probably worked for something like 10mins a couple of times but would eventually run into that error and restart. Think it might just require some trial and error on the memory location chosen?

Thanks anyway even if you have no idea about this, great to see someone doing something about the issue rather than just complaining. :)

#4 Re: PHP, APC and shared memory

Added on 2010-07-16T16:23 by Ivan Voras

Can you investigate what happens? E.g. do you get a coredump from php-cgi? Do you use i386 or amd64?

#5 Re: PHP, APC and shared memory

Added on 2010-10-10T04:34 by Martin

Now, a couple of moths after the posts, could you or others comment about the long term experience?  Are you still using the patch?  Is it somehow stable?

#6 Re: PHP, APC and shared memory

Added on 2010-12-06T05:51 by justyns

Is there an updated patch for 3.1.6 yet?

#7 Re: PHP, APC and shared memory

Added on 2010-12-14T12:23 by Tobias Hoffmann
Interesting. Has there been any progress on this?

#8 Re: PHP, APC and shared memory

Added on 2010-12-22T02:47 by Ivan Voras

No, the patch needs more work and I don't have the time.

#9 Re: PHP, APC and shared memory

Added on 2010-12-22T10:02 by Tim Westervoorde

Ivan, I'm working on fixing this patch for the latest APC (3.1.6) and improve the cache initialisation which is broken right now (every process that starts reset's the cache etc)

Feel free to get in touch with me to get the full patch. My name is tim and my company is, figure out my e-mail address ;)

#10 Re: PHP, APC and shared memory

Added on 2010-12-22T12:35 by Tim Westervoorde

Skip my last comment. After working for nearly two days it seems this is technically impossible with the current APC mechanism. There are just too many structures which need to be synchronised, not only the cache and I don't want to alter APC that much.

#11 Re: PHP, APC and shared memory

Added on 2011-02-21T10:31 by HHvA

Is there really no solution for this problem yet? What is APC using shared memory for if the cache is not shared between processes??

#12 Re: PHP, APC and shared memory

Added on 2011-07-08T15:49 by zeljko

I am using APC version 3.1.9 and it seems like pages taht APC has cached once, are there for as long as 2 or 3 more requests but then it dissapears from cache. Does anyone know where exactly is the cache of APC/mmap located?

My system is SLES 10, 32bit ..

Shared Memory 1 Segment(s) with 384.0 MBytes
(mmap memory, pthread mutex Locks locking)

#13 Re: PHP, APC and shared memory

Added on 2013-07-11T14:30 by prabha

I am using apc concept on ubuntu. default mmap memory . but i want IPC shared memory on ubuntu.. pls any help....

Comments !