PHP Internals personal scratch book

This is a list of things I've kept within reach to refer back or ponder while working with or around php-src, or potential projects for it.

This gets edited when I need to throw something at it. It might also not be accurate anymore.

php-src tools and explanations

Websites:

Compilation

To use CLANG instead: ./configure CC=clang ...

Add Compiler flags: ./configure CFLAGS="-Wflag" ...

Configure options for enabling ASAN/UBSAN: --enable-address-sanitizer --enable-undefined-sanitizer

Configure options for enabling Clang MSAN (PCRE JIT needs to be disabled): ./configure CC=clang --enable-memory-sanitizer --without-pcre-jit

Compiling with CFLAGS="-ggdb3" allows GDB to have access to macros and preprocessor constants.

A reasonable basic configure for minimal testing:

./configure -C CFLAGS="-DPROFITABILITY_CHECKS=0 -DZEND_RC_DEBUG=1 -DZEND_VERIFY_FUNC_INFO=1" --disable-all --enable-debug --enable-address-sanitizer --enable-undefined-sanitizer --enable-opcache --enable-zend-test

Larger MSAN build:

MSAN needs all libraries instrumented, therefore limit ourselves to stuff which relies on libc.

./configure -C CC=clang --enable-memory-sanitizer --without-pcre-jit --without-sqlite3 --without-pdo-sqlite --without-libxml --disable-dom --disable-simplexml --disable-xml --disable-xmlreader --disable-xmlwriter --without-pcre-jit --disable-opcache-jit --enable-phpdbg --with-pdo-mysql=mysqlnd --with-mysqli=mysqlnd --disable-mysqlnd-compression-support --without-pear --enable-exif --enable-sysvsem --enable-sysvshm --enable-shmop --enable-pcntl --enable-mbstring --disable-mbregex --enable-sockets --enable-bcmath --enable-calendar --enable-ftp --enable-zend-test --enable-dl-test=shared

Regenerating the PHP VM:

php Zend/zend_vm_gen.php

Testing php-src

Debugging a failed PHPT test with GDB:

failed_test.sh gdb

Testing with Valgrind:

ZEND_DONT_UNLOAD_MODULES=1 USE_ZEND_ALLOC=0 valgrind --leak-check=full --track-origins=yes make test

Testing PDO MySQL:

PDO_MYSQL_TEST_DSN="mysql:dbname=test;host=127.0.0.1;" PDO_MYSQL_TEST_DB="test" PDO_MYSQL_TEST_USER="pdo" PDO_MYSQL_TEST_PASS="password" make test TESTS="ext/pdo_mysql"

Testing the JIT:

First need to compile with CFLAGS="-DPROFITABILITY_CHECKS=0".

Then run:

make test TEST_PHP_ARGS="-q -j22 -d opcache.jit=function -d opcache.enable=1 -d opcache.enable_cli=1 -d opcache.file_update_protection=0 -d opcache.jit_buffer_size=1M" TESTS="Zend ext/opcache"

Potential way to debugging JIT Assembly:

make test TEST_PHP_ARGS="-dopcache.enable_cli=1 -dopcache.jit_buffer_size=16M -d opcache.jit_debug='1<<8'" TESTS="failing_test_file.phpt" &> debug-jit.txt

Out of tree PHP extensions

Testing with Valgrind

ZEND_DONT_UNLOAD_MODULES=1 USE_ZEND_ALLOC=0 valgrind --leak-check=full --track-origins=yes path/to/php run-tests.php -P -d extension=./modules/ext.so

Generate ARGINFO: path/to/php-src/scripts/dev/gen_stub.php ext.stub.php

Custom php-src builds for working with different versions/types (Debug/NTS/TS)

Use a MAKE INSTALL build by using the following command for php-src:

./configure --disable-all --enable-debug --prefix path/to/custom-php/

The extension is now ready to be compiled. To do so, first move into the extension directory. Then do the following steps:

phpize
./configure --with-php-config=path/to/custom-php//bin/php-config CFLAGS="-Wfatal-errors -Wall -Wpedantic -Wextra -Wno-unused-parameter -Werror"
make

Test against chosen custom debug build

/mnt/c/Dev/custom-php/bin/php run-tests.php -P -d extension=./modules/ext.so

Releasing on PECL

PECL package.xml DTD: https://pear.php.net/dtd/package-2.0.xsd

Generate test file list for package.xml:

find tests -name "*.phpt" | awk '{ print "        <file role=\"test\" name=\""$0"\" />" }'

To generate tar ball: pear package

Extension ideas:

C related knowledge:

The Lost Art of Structure Packing: http://www.catb.org/esr/structure-packing/

C99 restrict keyword

Compiler warnings to add/ideas:

Sources:

  1. -Wshadow Impossible php-src macro usage leads to variable shadowing
  2. -Wdouble-promotion Nice to have
  3. -Wformat=2 & -Wformat-truncation seems useful to be able to enable
  4. -Wundef W.I.P.
  5. -fno-common Not sure what this does
  6. -fstack-usage & -Wstack-usage= Don't think so as we use a custom allocator
  7. -Wconversion Huge amount of work
  8. -Wrestrict GCC 7 in -Wextra in GCC8
  9. -Wnull-dereference GCC 6

New GCC 8 warnings added to -Wall and -Wextra: https://gcc.gnu.org/gcc-8/changes.html

Git commands and stuff

Fetching remote GitHub PRs locally

git fetch <remote> pull/$ID/head:$BRANCHNAME

Where $ID is the PR number and $BRANCHNAME the name given locally. Followed by:

git checkout $BRANCHNAME

Fetching patch file from GitHub PR and applying it

On a GitHub PR, if one add .patch to the URL it redirects to the git email patch file.

It is then easy to apply it locally with the following command:

curl $URL | git am --signoff

Rebase onto:

git rebase [--onto <new base>] [<upstream> [<branch>]]

Refresh commit date:

git commit --amend --date=now --no-edit

Push all branches after doing a merge-up of a commit:

git push --atomic php PHP-8.0 PHP-8.1 master

Splitting up one commit into more:

https://embeddedartistry.com/blog/2018/02/19/code-cleanup-splitting-up-git-commits-in-the-middle-of-a-branch/

Deleting local branches after removing them from remote, discovered

https://medium.com/@kcmueller/delete-local-git-branches-that-were-deleted-on-remote-repository-b596b71b530c:

git branch -vv | grep ': gone]'|  grep -v "\*" | awk '{ print $1; }' | xargs -r git branch -d

Cherry-picking multiple commits:

https://stackoverflow.com/questions/1670970/how-to-cherry-pick-multiple-commits

Pretty git log graph

Someone sent me this command to generate a pretty git log graph:

git log --graph --pretty='\''%Cred%h%Creset -%C(auto)%d%Creset %s %Cgreen(%cr) %C(bold blue)<%an>%Creset'

Fixup branches when you forget to change feature branches before committing:

git checkout -b new-branch # switch to a new branch
git branch -f old-branch HEAD~3 # make old-branch point to some older commit

Git Reflog:

http://gitready.com/intermediate/2009/02/09/reflog-your-safety-net.html