PHP Internals personal scratch book

Websites

• LXR: https://heap.space

• PHP Internals Book: http://www.phpinternalsbook.com/

• Book by Zend on writing PHP extensions: https://www.zend.com/resources/writing-php-extensions

• Incomplete PHP 7.2 engine docs + articles: https://phpinternals.net/

• Check usage across open source repos: https://sourcegraph.com/

Compilation

To use CLANG instead: ./configure CC=clang ...

Add Compiler flags: ./configure CFLAGS="-Wflag" ...

Configure options for enabling ASAN/UBSAN: --enable-address-sanitizer --enable-undefined-sanitizer

Configure options for enabling Clang MSAN (PCRE JIT needs to be disabled): ./configure CC=clang --enable-memory-sanitizer --without-pcre-jit

Compiling with CFLAGS="-ggdb3" allows GDB to have access to macros and preprocessor constants.

./configure -C CFLAGS="-DPROFITABILITY_CHECKS=0 -DZEND_RC_DEBUG=1 -DZEND_VERIFY_FUNC_INFO=1 -DZEND_TRACK_ARENA_ALLOC=1 -ggdb3" \
    --enable-address-sanitizer --enable-undefined-sanitizer --disable-all --enable-debug \
    --enable-tokenizer --enable-opcache --enable-zend-test --enable-dl-test=shared

./configure -C CC=clang --enable-memory-sanitizer --without-pcre-jit --without-sqlite3 --without-pdo-sqlite \
    --without-libxml --disable-dom --disable-simplexml --disable-xml --disable-xmlreader --disable-xmlwriter \
    --without-pcre-jit --disable-opcache-jit --enable-phpdbg --enable-zend-test --enable-dl-test=shared \
    --with-pdo-mysql=mysqlnd --with-mysqli=mysqlnd --disable-mysqlnd-compression-support --without-pear \
    --enable-exif --enable-sysvsem --enable-sysvshm --enable-shmop --enable-pcntl --enable-sockets \
    --enable-mbstring --disable-mbregex  --enable-bcmath --enable-calendar --enable-ftp

./configure -C CFLAGS="-DPROFITABILITY_CHECKS=0 -DZEND_RC_DEBUG=1 -DZEND_VERIFY_FUNC_INFO=1 -DZEND_TRACK_ARENA_ALLOC=1 -ggdb3" \
  --enable-address-sanitizer --enable-undefined-sanitizer --enable-debug --enable-tokenizer --enable-opcache --enable-zend-test --enable-dl-test=shared \
  --enable-pcntl --enable-mbstring --enable-fpm --enable-posix --enable-bcmath --enable-calendar --enable-ctype --enable-exif \
  --enable-fileinfo --enable-filter --enable-ftp --enable-gd --enable-session --enable-sockets \
  --enable-sysvmsg --enable-shmop --enable-sysvsem --enable-sysvshm \
  --enable-dba --with-gdbm --with-cdb --enable-flatfile --enable-inifile --with-lmdb --with-tcadb \
  --with-zip --with-zlib --with-bz2 \
  --with-curl --with-ffi --with-gmp --with-tidy --with-enchant --with-openssl --with-sodium \
  --with-libxml --enable-dom --enable-simplexml --enable-xml --enable-xmlreader --enable-xmlwriter --with-xsl --enable-soap \
  --with-sqlite3 --with-mysqli --with-unixODBC --enable-pdo --with-pdo-firebird --with-pdo-mysql --with-pdo-pgsql --with-pdo-sqlite --with-pgsql \
  --with-iconv --enable-intl --enable-phar --with-pspell --with-readline

php Zend/zend_vm_gen.php

Testing php-src

General test command:

make test TEST_PHP_ARGS="--asan -q -j22 -d opcache.jit=function -d opcache.enable=1 -d opcache.enable_cli=1 -d opcache.file_update_protection=0 -d opcache.jit_buffer_size=1M" TESTS="Zend/ ext/

Debugging a failed PHPT test with GDB:

failed_test.sh gdb

Testing with Valgrind:

ZEND_DONT_UNLOAD_MODULES=1 USE_ZEND_ALLOC=0 valgrind --leak-check=full --track-origins=yes make test

Testing PDO MySQL:

PDO_MYSQL_TEST_DSN="mysql:dbname=test;host=127.0.0.1;" PDO_MYSQL_TEST_DB="test" PDO_MYSQL_TEST_USER="pdo" PDO_MYSQL_TEST_PASS="password" make test TESTS="ext/pdo_mysql"

make test TEST_PHP_ARGS="-q -j22 -d opcache.jit=function -d opcache.enable=1 -d opcache.enable_cli=1 -d opcache.file_update_protection=0 -d opcache.jit_buffer_size=1M" TESTS="Zend ext/opcache"

make test TEST_PHP_ARGS="-dopcache.enable_cli=1 -dopcache.jit_buffer_size=16M -d opcache.jit_debug='1<<8'" TESTS="failing_test_file.phpt" &> debug-jit.txt

Custom php-src builds for working with different versions/types (Debug/NTS/TS)

Use a MAKE INSTALL build by using the following command for php-src:

./configure --disable-all --enable-debug --prefix /path/to/custom-php/

The extension is now ready to be compiled. To do so, first move into the extension directory. Then do the following steps:

./configure --with-php-config=path/to/custom-php//bin/php-config CFLAGS="-Wfatal-errors -Wall -Wpedantic -Wextra -Wno-unused-parameter -Werror"

Test against chosen custom debug build:

/path/to/custom-php/bin/php run-tests.php -P -d extension=./modules/ext.so

Releasing on PECL

PECL's package.xml DTD location: https://pear.php.net/dtd/package-2.0.xsd

Generate test file list for package.xml:

find tests -name "*.phpt" | awk '{ print "        <file role=\"test\" name=\""$0"\" />" }'

To generate the tar ball: pear package

Extension ideas:

• Rational Number bata type: https://en.wikipedia.org/wiki/Rational_data_type

• Complex numbers (maybe better as a new zval type?)
• Matrices

C99 restrict keyword

• Restrict Texas Instrument white paper: https://software-dl.ti.com/C2000/docs/optimization_guide/phase3/restrict.html

• https://processors.wiki.ti.com/images/f/ff/Bartley=Wiki_1.1=Performance_Tuning_with_the_RESTRICT_Keyword.pdf

Compiler warnings to add/ideas

Sources:

• https://kristerw.blogspot.com/2017/09/useful-gcc-warning-options-not-enabled.html

• https://interrupt.memfault.com/blog/best-and-worst-gcc-clang-compiler-flags

• https://blogs.oracle.com/linux/making-code-more-secure-with-gcc-part-1

1. -Wshadow: Impossible php-src macro usage leads to variable shadowing

2. -Wdouble-promotion: Nice to have

3. -Wformat=2 & -Wformat-truncation: seems useful to be able to enable

4. -Wundef: W.I.P.

5. -Wconversion: Huge amount of work

6. -fstack-usage & -Wstack-usage=: To check stack usage, useful for performance optimization?

Fetching remote GitHub PRs locally

git fetch <remote> pull/$ID/head:$BRANCHNAME

git checkout $BRANCHNAME

Fetching patch file from GitHub PR and applying it

Take the GitHub PR URL, if one add .patch to the URL it now points to a redirects to the git email patch file for this pull request.

It is then easy to apply it locally with the following command:

curl -L $URL | git am --signoff

Rebase onto:

git rebase [--onto <new base>] [<upstream> [<branch>]]

Refresh commit date:

git commit --amend --date=now --no-edit

Refresh commit date and author name:

git commit --amend --date=now --reset-author --no-edit

Push all branches after doing a merge-up of a commit:

git push --atomic php PHP-8.1 PHP-8.2 master

Pretty git log graph

Someone sent me this command to generate a pretty git log graph:

git log --graph --pretty='\''%Cred%h%Creset -%C(auto)%d%Creset %s %Cgreen(%cr) %C(bold blue)<%an>%Creset'

Fixup branches when you forget to change feature branches before committing:

git checkout -b new-branch # switch to a new branch
git branch -f old-branch HEAD~3 # make old-branch point to some older commit

Deleting local branches after removing them from remote

Discovered here: https://medium.com/@kcmueller/delete-local-git-branches-that-were-deleted-on-remote-repository-b596b71b530c

git branch -vv | grep ': gone]'|  grep -v "\*" | awk '{ print $1; }' | xargs -r git branch -d