Ben Ramsey

Generating HTTPS URLs with Symfony on DigitalOcean App Platform

2026-05-31T15:10:52+00:00

I’ve been dealing with a frustrating issue in a Symfony application hosted on DigitalOcean App Platform. Since it took me quite a while to find the solution, I thought I’d share it here in case it helps someone else.

By default, every application running on App Platform includes Cloudflare CDN as a reverse proxy. This provides a number of benefits, especially if your application sets appropriate cache-control and etag headers. Cloudflare will use these to cache your application’s responses at the edge, thus reducing load on your servers. So far, so good.

Cloudflare also terminates TLS, which means the traffic is decrypted before being forwarded to DigitalOcean. This presents a problem for my application. It thinks it’s receiving traffic over HTTP rather than HTTPS. As a result, Symfony generates URLs with the http:// scheme rather than https://. This isn’t a major problem, since everything is configured to redirect to https://, but it requires an additional network request, and it’s one of those tiny details that will bother me until I figure out how to fix it.

It’s one of those tiny details that will bother me until I figure out how to fix it.

Meanwhile, Symfony’s documentation isn’t clear about how to solve this. Three pages came up in my search for a solution:

The first isn’t relevant to my problem. It’s about forcing HTTPS or HTTP for different URLs, not about generating HTTPS URLs when behind a reverse proxy. Plus, it has this scary warning at the bottom of the page:

Forcing HTTPS while using a reverse proxy or load balancer requires a proper configuration to avoid infinite redirect loops.

The second one indicates it’s for use when generating URLs in non-HTTP requests (e.g., from CLI tools, cron jobs, email sending tasks, etc.). It also includes the same scary warning:

If your server runs behind a proxy that terminates SSL, make sure to configure Symfony to work behind a proxy.

The configuration for the scheme is only used for non-HTTP requests. The schemes option together with incorrect proxy configuration will lead to a redirect loop.

That warning also links to the third page, which is relevant to my problem. However, anything involving network configuration and CIDR notation has always intimidated me. I was afraid of breaking connectivity to the application, and the only way I could find out whether it worked was to try it in production.

In the end, I needed to make two small changes to my application, neither of which are explicitly documented anywhere:

I replaced x-forwarded-for with the value of do-connecting-ip.

The note on using do-connecting-ip instead of x-forwarded-for is buried deep in the DigitalOcean documentation, and if you’ve never encountered it, it can come as a surprise. I’ve been around long enough to know this is a common thing that reverse proxies do, so I knew what to look for.

For Symfony to find the client’s correct IP address, you need to rewrite this header early in the request lifecycle. I did this by adding the following to my index.php above where the Symfony kernel is instantiated:
```
// Map DigitalOcean's custom IP header to the standard x-forwarded-for header.
if (isset($_SERVER['HTTP_DO_CONNECTING_IP'])) {
    $_SERVER['HTTP_X_FORWARDED_FOR'] = $_SERVER['HTTP_DO_CONNECTING_IP'];
}
```
I only knew to do this because Symfony shows an example with a different x-forwarded-* header.
I added the following to my config/packages/framework.yaml file:
```
framework:
  trusted_proxies: '127.0.0.1,100.64.0.0/10'
  trusted_headers: ['x-forwarded-for', 'x-forwarded-host', 'x-forwarded-proto', 'x-forwarded-port']
```
What’s important to note here is the value 100.64.0.0/10 in the list of trusted proxies. I could have used REMOTE_ADDR as suggested in the documentation, but that felt insecure. I tried using PRIVATE_SUBNETS, also suggested in the documentation, but DigitalOcean uses a subnet not found in Symfony’s list of private subnets.

What’s more, I couldn’t find this subnet listed in the DigitalOcean documentation. However, it’s defined in RFC 6598 as the reserved IPv4 prefix for Shared Address Space. I stumbled on it when I noticed the REMOTE_ADDR value on requests to my application began with 100.127, which wasn’t in Symfony’s list. So I searched to find whether it had any special meaning, and sure enough, it does.

After making these changes, $request->getClientIp() now returns the correct client IP address instead of the IP address of the ingress proxy. I no longer need to use $request->getHeaders()->get('do-connecting-ip') for that value. I am also able to use {{ url('home') }} in my Twig templates instead of the hacky and ugly https://{{ url('home', {}, true) }} I was using before.

PHP Tek: A Homecoming

2026-05-24T04:16:00+00:00

PHP Tek always feels like a homecoming. No matter how much time passes between conferences, or between the person I was the last time I attended and the one I am now, walking into that hotel and finding familiar faces feels like returning home to close friends and family. That feeling was especially pronounced in 2023, when the conference returned after a four-year hiatus imposed by the pandemic. This week in Rosemont, back at the Sheraton O’Hare, it was there again.

The conference ran May 19th through the 21st. It was three days of sessions, workshops, and the kind of hallway conversations that don’t appear on any schedule. I gave two talks: one on describing APIs with OpenAPI on Wednesday, and a beginner’s guide to OAuth and OpenID Connect on Thursday. Both went well, and I’m always glad for an audience willing to spend an hour in the weeds with me on less glamorous—but no less fascinating or important—parts of development.

A few of the talk highlights from the conference, for me, were:

Shane Rosenthal speaking about NativePHP (check out NativePHP—it’s really cool)
Sarah Peters’s “From Concert Hall to Code Review, Symphony to Software,” where she played violin interludes between sections of her talk
James Warmoth III’s “From Classroom Code to Production Code: Bridging the Gap Between School and the Software Industry”

One of the most memorable parts of PHP Tek is the game night, or after party. Game night is a tradition going back to the earliest PHP Tek conferences. You see, PHP Tek takes place in a hotel by the airport, and there’s little to do nearby. The surrounding area isn’t very walkable, and it takes about an hour to get into Chicago. This is a feature, not a bug. As a result, everyone sticks around to play games, get to know each other, and have fun.

One of my favorite game night traditions is karaoke. This tradition has been quietly accumulating across PHP conferences for twenty years. It started, as best I can recall, in a hotel bar at ZendCon in San Jose in 2006. After a pirate-themed after party, we found ourselves in the hotel bar, where they had a DJ leading karaoke. It was so memorable, they made it a part of the after party at ZendCon 2007. I remember doing karaoke after delivering a keynote at the PHP Experience in São Paulo in 2016, which was surreal (the karaoke, not the keynote). That same year we found a karaoke bar near the hotel at PHP Tek in St. Louis. Along the way, we’ve done karaoke at Lone Star PHP in Dallas, Southeast PHP in Nashville, Longhorn PHP in Austin, Cascadia PHP in Portland, among others. The conferences changed, the cities changed, and somehow karaoke kept appearing.

At PHP Tek, we’ve evolved beyond hunting for a karaoke venue. Since the conference returned in 2023, we’ve been using the sound system already set up in the ballroom during game night to run our own. It happened again on Wednesday, and I ended up leading a good stretch of it. Nobody assigns this role. It just falls to whoever’s willing to go first. (But really, it’s because I’m the most persistent at begging John to let me do it.)

Nate, Jackie, and Larry sing “What It Sounds Like” from KPop Demon Hunters at PHP Tek

Another highlight of PHP Tek is Catan Starfarers. This is the most unhinged version of Catan. I’ve never played, but watching is its own pleasure. Omni Adams GMs a game that’s not supposed to have a game master. The games go late. They’re ginormous. I don’t fully understand how anyone wins, and I suspect that’s part of the appeal. Rumor has it, there are folks who attend just to play Starfarers.

Kalen, Daniel, Scott, Nia, Omni, and John play Catan Starfarers at PHP Tek

A meaningful moment of the week, for me, was catching up with Elizabeth Barron. Elizabeth has been a fixture in the PHP community for many years, and we go way back; we organized PHP Appalachia together in 2006. We’d stayed loosely in touch over the years, but I don’t think we’d been in the same room since around 2016. She’s back in a significant way now, having joined the PHP Foundation as executive director. We talked mostly about family, life, and things unrelated to PHP or technology. I love that conferences like PHP Tek provide opportunities to catch up with people you haven’t seen in years without any particular agenda.

There were conversations with more of an agenda, of course. The hallway track at PHP Tek is where a lot of the real thinking happens, and this year, some of the recurring subjects were: context generators, supply-chain security in the PHP ecosystem, the impact of AI on how we write and think about code, and the longer question of where PHP is heading. I don’t think any of these had tidy conclusions, but they benefit from in-person discussions in ways that mailing list discussions can’t provide.

PHP Tek has outlasted hype cycles, framework wars, and periodic predictions that PHP was dying, and it keeps coming back. The community keeps showing up. That’s the thing about a homecoming: it only means something if people keep coming home.

One last thing: WurstCon returned this year, following the close of the conference. This has become a PHP community tradition. For where two or three are gathered together with hot dogs, there is a WurstCon. This time, WurstCon was at the Winking Dog.

The WurstCon conference goers outside the Winking Dog

The PHP License, Simplified

2026-05-01T00:00:00+00:00

I’ve been fascinated with copyright since high school and with open source licensing for most of my career. Yes, I’m a nerd, but I really do find it compelling to think about how humans communicate ideas—through words, songs, photographs, videos, and even code—and claim ownership over these products of the mind. Copyright is how we do that, and licensing is one of the tools we use to share what we create. So, when Matthew Sheahan posted a question to the php-general mailing list in early 2020, just as COVID-19 lockdowns were taking hold, it caught my attention.

Matthew’s team needed to use phpdbg, but their organization required OSI-approved licenses, and “the fact that 3.0 and 3.01 are substantively identical is no use to us at all.” While the PHP website said the PHP License 3.01 was OSI-approved, the OSI’s own said otherwise. Only version 3.0 had been formally approved. 3.01 was a minor version bump, nearly identical to 3.0, but that didn’t matter when company policy required it be listed on the OSI website. Since copyright and licensing is an interest of mine, I figured this was a way I could contribute to the PHP project, so I volunteered. I contacted the OSI to request legacy approval for 3.01, and the OSI Board voted to approve it in May 2020. Problem solved.

I figured this was a way I could contribute to the PHP project, so I volunteered.

But that planted a seed. Looking closely at the PHP License and the Zend Engine License, I saw two well-intentioned licenses that included increasingly awkward provisions; they were showing their age. The PHP project had been using its own custom license since PHP 3, when Rasmus Lerdorf adapted it from the Apache License, version 1.0, with modifications. The Zend Engine License came along with PHP 4, covering sources in the Zend/ directory. Each license contained conditions specific to their respective custodians—restrictions on using the names “PHP” and “Zend,” an advertising acknowledgment clause, and provisions granting only the PHP Group and Zend Technologies the authority to change the licenses.

After 25 years of cohabitation within the same source code repository, [PHP and the Zend Engine] are intertwined in ways in which the Zend Engine can no longer be separated and used as a standalone product. Together, they form the PHP programming language reference implementation.

— PHP License Update RFC

These weren’t bad licenses. Most developers, distributors, and companies accepted their intent and moved on. But gaps between intent and substance have a way of accumulating. The Zend Engine, once imagined as a separable product, had been inextricably bound to PHP for 25 years; the notion that it required its own license had become a historical artifact. Condition 4 of the PHP License—restricting derived products from calling themselves “PHP”—created real ambiguity for Linux distributions like Debian, which patch and redistribute PHP. Neither license was GPL-compatible, and the Zend Engine License had never been OSI-approved. Yet strip away the PHP Group-specific and Zend-specific conditions from both licenses, and what remains is effectively the Modified BSD License, otherwise known as BSD-3-Clause.

So, that is now the PHP License, version 4, and the Zend Engine License, version 3. Not a custom adaptation. Not a derivative. The actual BSD-3-Clause license, with its SPDX identifier, recognized by the OSI, the FSF, and every major piece of licensing tooling in use today. It’s compatible with the GPL. There are no advertising requirements, no restrictions that only a single organization can invoke, no ambiguity for distributors.

The rights granted by contributors have not changed. The rights granted to users have not changed. The new licenses are fully compatible with the old ones. This is a simplification and nothing more.

Getting here required more than writing an RFC. The PHP License gives the PHP Group the authority to change it, which meant tracking down each of the original PHP Group members and getting their written consent. Each approved the proposal. Perforce Software, the successor to Zend Technologies, needed to sign off on the Zend Engine side, as well. They provided a formal letter confirming their full authority and support for the change. I hired an attorney to review the proposal and provide advice on any legal questions that might surface during the discussion period. Speaking of which, I allowed for a six-month community discussion period preceding the vote, which passed unanimously.

Now, if you’re using software licensed under the PHP License 3.01, clause 5 of that license allows you to use the same software under the new version. If you maintain a PECL extension or any other project licensed under PHP License 3.01, clause 5 of that license allows you to adopt the new version. The same permissive spirit that covered your project before still covers it.

Once covered code has been published under a particular version of the license, you may always continue to use it under the terms of that version. You may also choose to use such covered code under the terms of any subsequent version of the license published by the PHP Group.

— The PHP License, version 3.01, clause 5

If your organization has a policy requiring use of OSI-approved licensed open source software, or if you’ve had to explain away the Zend Engine License’s non-approved status to a compliance review, those conversations are over. And if you’ve simply never thought about this because it didn’t affect you, that’s fine too. It’s worth knowing the licenses exist, they’ve been updated, and PHP’s legal foundations now match the spirit of the language and community.

There are many people to thank for seeing this through: the members of the PHP internals community who engaged with the RFC over a long discussion period, the members of the PHP Group—Thies, Stig, Shane, Andi, Rasmus, Sam, Sascha, Zeev, Jim, and Andrei—who each took the time to respond to my messages, Perforce Software—and especially Matthew Weier O’Phinney and Sara Kilian—for their support and cooperation, Zak Greant, the debian-legal and OSI license-discuss mailing lists, the FSF Licensing & Compliance Team, and Pamela Chestek of Chestek Legal. Open source licensing is unglamorous, but it matters, and I’m glad to have had a part in this.

Testing the PDO_ODBC driver on macOS

2021-05-12T23:27:23+00:00

In response to Christoph’s comment on PR #6935, I wanted to check to see exactly what PHP returns when calling PDO::getAttribute() on PDO_ODBC with PDO::ATTR_SERVER_INFO and PDO::ATTR_SERVER_VERSION. Looking at the code, I could tell that it breaks and returns a 0. I wasn’t sure how it reacted beyond that, and I wanted to test it in PHP to see what would happen.

To do this, I needed to be able to connect to a database using ODBC on macOS. Since I primarily use MySQL or PostgreSQL, I haven’t used ODBC to connect to a database in almost 20 years.

My PHP build already had PDO_ODBC installed, so I didn’t need to rebuild it.

$ php -m | grep -i odbc
odbc
PDO_ODBC

Next, I checked to see whether unixODBC was installed, and I wanted to connect to a MariaDB server, so I installed the unixodbc and mariadb-connector-odbc packages with Homebrew. (It turned out that unixodbc was already installed on my system through Homebrew.)

$ brew install unixodbc mariadb-connector-odbc

From here, I created a temporary file with the following contents (I named it mariadb-odbc.ini).

[MariaDB ODBC 3.1 Driver]
Description = MariaDB Connector/ODBC v.3.1
Driver = /usr/local/lib/mariadb/libmaodbc.dylib

The libmaodbc driver is provided by mariadb-connector-odbc, and the path to the Driver is based on where Homebrew put it, so if you’re following along and attempting to do the same, double-check the location on your system.

Then, I imported the driver configuration using odbcinst, which comes with unixODBC. Check the odbcinst documentation for more information about the above INI file format.

$ odbcinst -i -d -f mariadb-odbc.ini

This adds the configuration to /usr/local/etc/odbcinst.ini.

I then used Docker to create a MariaDB container that I could connect to using the ODBC driver.

$ docker run -p 3306:3306 --name mariadb -e MYSQL_ROOT_PASSWORD=password -d mariadb:latest

With the container up and running, I launched PsySH and started entering expressions.

> $dsn = "odbc:Driver={MariaDB ODBC 3.1 Driver};Server=127.0.0.1;Database=mysql;User=root;Password=password;Option=3;"

= "odbc:Driver={MariaDB ODBC 3.1 Driver};Server=127.0.0.1;Database=mysql;User=root;Password=password;Option=3;"

> $pdo = new PDO($dsn)

= PDO {#3907
     inTransaction: false,
     attributes: {
       CASE: NATURAL,
       ERRMODE: EXCEPTION,
       PERSISTENT: false,
       DRIVER_NAME: "odbc",
       ORACLE_NULLS: NATURAL,
       CLIENT_VERSION: "ODBC-unixODBC",
       STATEMENT_CLASS: [
         "PDOStatement",
       ],
       DEFAULT_FETCH_MODE: BOTH,
     },
   }

> $pdo->getAttribute(PDO::ATTR_SERVER_INFO)

   PDOException  SQLSTATE[IM001]: Driver does not support this function: driver does not support that attribute

> $pdo->getAttribute(PDO::ATTR_SERVER_VERSION)

   PDOException  SQLSTATE[IM001]: Driver does not support this function: driver does not support that attribute

So, there we go. PDO::getAttribute() throws a PDOException when it doesn’t support an attribute.

Thanks to Calvin Buckley, these attributes will no longer throw exceptions and will instead return actual values in PHP 8.1.

> $pdo->getAttribute(PDO::ATTR_SERVER_INFO)

= "MariaDB"

> $pdo->getAttribute(PDO::ATTR_SERVER_VERSION)

= "10.05.000009"

Using CLion with php-src

2021-05-10T17:08:02+00:00

I’ve cloned and built php-src numerous times over the years, on a variety of platforms, but I’ve never worked with it in an IDE. A long-time Vim user, I’ve only begun using PhpStorm over the last year and a half, after seeing how IntelliJ helped me easily navigate an open source Java project. So, in setting up php-src for release management tasks, I wanted to give CLion a try.

Getting php-src ready for CLion

When opening existing source code or cloning a repo with CLion, it assumes the project already supports CMake tooling for builds. For someone new to C and using a project that doesn’t use CMake, this can be confusing and frustrating. Since php-src isn’t a CMake project, CLion will display warnings at the top of every open file, saying that a “CMake project is not loaded,” giving you the option to select or generate a CMakeLists.txt file. You can try to do this for php-src, but there’s a better way.

CLion also supports JSON compilation databases. To generate one and have CLion recognize php-src as a compilation database project, you’ll need to do the following before opening php-src in CLion. If you’ve already opened it in CLion, that’s okay. Delete the .idea/ folder that CLion creates, and then do the following. (I could not figure out a way to make CLion recognize the compilation database if I had already opened the project in CLion. It seems the compilation database must exist the first time you open the project in order for CLion to recognize it.)

You’ll need the compiledb Python tool to generate the compilation database. It wraps the make command, so when you run it with make, it captures all the sources and creates a compile_commands.json file. Then, when you open the project in CLion for the first time, it will recognize your project as a compilation database project.

$ pip install compiledb

$ git clone https://github.com/php/php-src.git ~/repos/php/php-src
$ cd ~/repos/php/php-src
$ ./buildconf
$ ./configure \
    --prefix=$HOME/builds/php/8.1.0-dev \
    --disable-all \
    --disable-cgi \
    --enable-debug \
    --enable-zts

$ compiledb make

As you can see here, I use compiledb to call make. In this way, compiledb can learn about all the sources for your build.

You may also pass -jN to make, where N is the number of CPU cores your system has available. This will greatly speed up compilation.

You can run nproc to find out how many cores you have. On my system, I have 12 cores, so I run the command like this:

$ compiledb make -j12

In my example, I pass --prefix to configure, but this isn’t necessary since I’m not going to run make install. Once make has finished building PHP, you can find the newly-built PHP CLI at sapi/cli/php.

$ ./sapi/cli/php -v
PHP 8.1.0-dev (cli) (built: May  6 2021 18:27:37) (ZTS DEBUG)
Copyright (c) The PHP Group
Zend Engine v4.1.0-dev, Copyright (c) Zend Technologies

$ ./sapi/cli/php -m
[PHP Modules]
Core
date
hash
json
pcre
Reflection
SPL
standard

[Zend Modules]

Setting up CLion to build and run PHP

Now that php-src has a compilation database (which needs to be updated each time you change your build with configure or if you add new sources that get included in make), you can open it in CLion.

Launch CLion to get to the welcome screen and choose to open a project from existing sources.

Figure 1. CLion welcome screen.

Browse to your php-src folder and, rather than selecting the folder to open, select the compile_commands.json file that compiledb generated.

Figure 2. File selection dialog highlighting compile_commands.json.

When CLion prompts you, choose to open it as a project.

Figure 3. “Open Project” dialog with the “Open as Project” button highlighted.

CLion will now open the project and inform you that it successfully imported the compilation database project.

Figure 4. Compilation database project successfully imported.

With CLion open, you’ll now create custom build targets for make and make clean.

Open CLion Preferences and go to Build, Execution, Deployment -> Custom Build Targets. Create a new build target, named something like “Build PHP.” Then, next to the empty Build and Clean fields, click the more button (with three dots). This will open an External Tools window. Use the +-button here to create two external tools.

I named the first one “Build with Make.” The program is make and I added -j12 to the arguments for a faster build (see earlier). You might also consider using compiledb as the program, with make -j12 as the arguments.

Figure 5. Set up a “Build with Make” external tool.

I named the second tool “Clean with Make” and used make as the program and clean as the argument.

Figure 6. Set up a “Clean with Make” external tool.

Now, you can click OK to close these windows and the External Tools window, and select the proper tools to use with your Build PHP target.

Figure 7. Set up a “Build PHP” build target external tool.

Finally, you’ll want to create a custom run/debug configuration. To do this, go to Run -> Edit Configurations….

Use the +-button to create a new configuration and choose Custom Build Application from the menu that appears. I named my configuration “Build PHP” and I chose my “Build PHP” target (created earlier) from the Target dropdown list. For the executable, I entered the path to the PHP CLI built by this project (remember, it’s at sapi/cli/php). Last, I checked “Redirect input from” and entered the JetBrains variable $FilePrompt$ . In this way, whenever you chose to run PHP from within CLion, it will prompt you with a file dialog to execute through the PHP interpreter. This lets you select a PHP script to run and test/debug whatever feature you’re working on.

Figure 8. Set up a “Build PHP” run/debug configuration.

Now, click the hammer icon (🔨) or select Build -> Build Project to build PHP, or select Build -> Clean to run the make clean command.

Figure 9. Output after running “Build with Make.”

If you click the play icon (▶️) or choose Run -> Run ‘Build PHP’, it will prompt you to select a file and run that file as a PHP script. In the following, I’ve selected one of the .phpt tests to execute this way.

Figure 10. Using the file selection dialog to select a PHPT file to run.

Figure 11. Output after running the PHPT file.

Step debugging through php-src

One of the primary reasons I wanted to use an IDE to work with php-src is for the debugging tools. With everything set up as described earlier, I’m able to run a PHP script and break on breakpoints in the C source code.

As an example, I created a PHP script named date-debug.php and added the following code to it.

<?php
echo date('Y-m-d H:i:s', time());

Then, I opened ext/date/php_date.c and placed a breakpoint in the php_date() function. You can see the breakpoint in the following screenshot. I placed it right after the ZEND_PARSE_PARAMETERS block.

When you click the debug icon (🪲) or choose Run -> Debug ‘Build PHP’, CLion will prompt you to select a file, just like it does when when you click the play icon, as described earlier. When I choose the date-debug.php file, it begins executing it and then stops at the breakpoint in the C code, which happens to be the code called when using the PHP date() function. As you can see, I’m now able to examine the memory and variables in this function, including the zvals.

Figure 12. Inspecting variables at a breakpoint in the C source code for PHP.

I'm a release manager for PHP 8.1!

2021-05-09T18:58:14+00:00

Last year, I tossed my hat in the ring to volunteer as a release manager for PHP 8.0, but I lost the vote to Gabriel Caruso, who has shown himself to be a very capable RM for the 8.0 series (congrats, Gabriel, and great work!). This year, when Sara announced the RM selection process for PHP 8.1, I almost decided against volunteering, but I finally decided to go for it, and I’m glad I did.

Each set of release managers is usually comprised of a veteran RM and a rookie. This year, when Joe Watkins volunteered as the veteran RM, he asked to take on two rookies. As a result, Patrick Allaert and I were both selected as rookie RMs for the 8.1 series.

PHP release managers manage the release process, create a roadmap for the release, facilitate discussions and decisions about what goes into a release, package the releases, etc. It’s a commitment of about three-and-a-half years, as they shepherd their minor version from alpha to beta to RC to GA to active support to security support and, finally, end-of-life. The supported versions page at php.net provides a good overview of this cycle.

You can read more about the PHP release process here:

Additionally, Eric Van Johnson has published a three-part interview in php[architect] magazine with PHP 8.0 release managers Sara Golemon and Gabriel Caruso. They go into a lot of the specifics.

Thank you so much to everyone for entrusting me to serve as one of the release managers for PHP 8.1! I look forward to working with Joe and Patrick over the next three-and-a-half years.

Thoughts on Software Architecture

2019-04-18T13:51:43+00:00

Software architecture concerns itself with how the software is built, including high-level design and technical standards, as well as coding standards, tools, platforms, etc.^[1]

A software architect is a software developer who has become an expert in these things through extensive experience practicing them and learning from others.

A software architect (as a role) is a leader in the organization, helping the team turn a problem domain into software. They are able to grasp the concepts of the domain and communicate how to represent them in code. Additionally, they work with the team to set standards and develop practices that will ensure the quality of the software created and the long-term maintainability of the code. They balance purity with pragmatism, considering the business goals and priorities—and they articulate the risk to the business when sacrificing quality/purity for accelerated timelines.

I believe a software architect should be a mentor, using their experience to teach others how to create the best software they can. This involves in-depth code reviews and pair programming, among other teaching techniques. Good software architects are humble and patient. They don’t berate or belittle their team for not yet grasping concepts that they see clearly. Instead, they use these moments as teaching opportunities.

Many see coding as formulaic as algebra equations. Software developers are natural problem-solvers, often seeing a code solution based on a formula before fully understanding the problem. Software architecture is different. There is no formula for it. There are recurring patterns seen across problem domains, and Martin Fowler has cataloged many, but his book Patterns of Enterprise Application Architecture cannot be treated as a formula by filling in the blanks and generating the code. It doesn’t work that way.

Learning to design software from an architectural perspective is an exercise in grokking the problem, taking a Zen-like approach to perceiving its true nature. To do this, you must exercise restraint so you don’t cloud your mind with thoughts of the solution before you fully understand the problem.

“Software architect.” Wikipedia, https://en.wikipedia.org/wiki/Software_architect. ↩

Announcing PHP TestFest 2017

2017-07-13T18:00:00+00:00

For those who’ve been around the PHP community for a while, you’ll recall the successful PHP TestFest events that began after a discussion at PHP Quebec in 2008. Many user groups and mentors signed on to host and help with events, and a lot of folks became first-time contributors to the PHP project, helping improve our code coverage. It ran strong in a global sense from 2008 to 2010. After that, various groups (particularly the Brazilian groups) have continued the tradition.

A few months ago, at php[tek] in Atlanta, I mentioned to Michelangelo that I’d love to bring back PHP TestFest. Sammy had given an excellent talk on writing PHPT tests, and Gemma tweeted a link to the old PHP TestFest wiki page. From there, things snowballed.

We’re bringing back PHP TestFest!

The PHP TestFest will run for 4 months this year: September through December. This should give groups plenty of time to plan and prepare one or more local events. In early January, we will award prizes; if your organization/company is interested in offering products or services as prizes, let us know (send email to sponsors@phptestfest.org).

This time around, I’ve set up a dedicated website at PHPTestFest.org and Google Group for discussion. We also have the #phptestfest channel on Freenode IRC. All are encouraged to contribute to the website and tools repository. We need technical tutorials, as well as tutorials from veteran TestFest organizers on how to lead successful PHP TestFest events. Additionally, I’ve provided a stub for a console application in the repository, and I’d love to see that evolve into a robust tool for helping people write and contribute PHPT tests.

I’ll be updating the “Getting Started” page on the PHP TestFest website soon with this information and more, so keep posted.

Let’s get to work on improving code coverage of the PHP language!

Special thanks to Sammy for letting us use “Wooble” as a testing mascot.

This announcement was cross-posted to the php-qa, ug-admins, and phptestfest mailing lists.

Building PHP Projects on AWS CodeBuild

2016-12-02T23:00:00+00:00

I’ve had a great time at AWS re:Invent this week, attending sessions and hanging out with some of the ShootProof team. My favorite part of the week was the “mini con” on containers. I spent Thursday immersed in sessions dedicated to deploying Docker containers on AWS.

Of course, the main highlight of re:Invent is always the keynotes and the new services and features announcements they make during the keynotes. One of the new services caught my attention, and I decided to give it a try. That service is AWS CodeBuild.

CodeBuild is designed to be used as part of the AWS CodePipeline, but it may also be used by itself. Additionally, with AWS’s increased focus on their container service (ECS), it’s designed to integrate with ECS to use images you’ve stored in AWS, but it may also use images available on Docker Hub.

Out of the box, CodeBuild provides some managed images that you may use to build your projects. These include environments for Android, Java, Python, Ruby, Golang, and Node.js. PHP is missing from this list, but since you’re able to use other images, I decided to see how easy it is to get up and running on CodeBuild with a PHP project. I chose to try out my ramsey/uuid library for a simple test. Feel free to follow along with me.

Getting Started

From your Amazon console, search for “CodeBuild” from the AWS Services search box, and choose it from the drop down menu that appears.

Figure 1. Search for “CodeBuild” from the AWS console

If you’ve not yet begun playing with CodeBuild, you’ll see a page like the one in Figure 2. Click the “Get Started” button to begin.

Figure 2. Getting started with CodeBuild

Configuring Your Project

Figure 3 shows most of the options available when configuring a project, pre-filled for the ramsey/uuid library. I’ll step through each option to explain what’s going on.

First, provide a project name. Here, I’ve named the project “ramsey-uuid.”

For Source provider, choose GitHub and then click the link provided to connect to GitHub. This will take you through the OAuth dance to grant AWS with access to your GitHub account. Afterwards, the Repository drop-down will be populated with a list of your GitHub repositories. Here, I’ve selected the uuid repository.

Next, you need to tell CodeBuild how to build your project by specifying an image to use. To use an image in Docker Hub, select “Specify a Docker image” for Environment image and choose “Other” for Custom image type. For the Custom image ID, I’m using benramsey/composer-uuid:latest to specify the repository, image name, and image tag. You may place a buildspec.yml file in the root of your repository to specify build commands (similar to a .travis.yml file). Since ramsey/uuid doesn’t have a buildspec.yml file, I’m specifying the Build commands directly.

The full build command I’ve specified for ramsey/uuid is:

$ composer install --no-interaction --prefer-dist \
    && ./vendor/bin/parallel-lint src tests \
    && ./vendor/bin/phpunit --verbose --no-coverage \
    && ./vendor/bin/phpcs src tests --standard=psr2 -sp

Finally, I’m not generating any artifacts for this build, so I selected “No artifacts” for the Artifacts type, and I chose to let it automatically create a service role in my account.

Our project is now configured, so click Continue, review your choices, click Save and Build, then click Start build to run a build.

Figure 3. Configure a project (click to enlarge)

Building Your Project

As your project is building, you’re able to view the progress of each stage of the build, complete with the last 20 lines of the log output, or you may view the complete log in CloudWatch to see any errors or problems that were emitted during the build. If the build passes, you’ll see the word Succeeded in a comforting bold green.

Figure 4. Project build progress (click to enlarge)

What Next?

My team is looking into CodeBuild as an alternative to running a dedicated Jenkins instance, and as we evaluate it, there are a few things we’re considering:

Some of our tests talk to a dev database in a VPC. How do we grant VPC access to a container running in CodeBuild?
In Jenkins, we capture a handful of artifacts, including code coverage reports and generated docblock documentation. With CodeBuild, we might consider copying these artifacts into an S3 bucket configured for website hosting.
What do we do about the other information we currently capture, like JUnit and Clover XML files?

Feel free to use ramsey/uuid and my directions here to play with CodeBuild, or use your own project. In the comments, let me know what you think and whether you’ve run into any caveats with your builds. If you come up with any solutions for the things we’re considering, I’d love to hear them.

7 Tips for php[tek]

2016-05-22T00:00:00+00:00

This week, I’m attending php[tek]. This is my seventh php[tek], and the first I’ve attended not as a speaker. It’s one of my favorite conferences, and I didn’t want to miss its first year in a new city: St. Louis. As we gear up for the eleventh php[tek] conference, I thought I’d list my seven tips for getting the most out of your php[tek] experience.

Hang out in the evenings, after the conference sessions.

php[tek] is known for hosting events in the evening, from video games and board games to hackathons. Make sure you you stick around and take part. One of the best features of PHP is its community, and taking part in the events following the day’s conference sessions is a great way to build relationships in this community.

Open bar…, by Rob Allen / CC BY-NC-SA

Rock Band, by Rob Allen / CC BY-ND
After the conference events, follow folks to the bar.

Do you have to be a drinker? Nope. There’s much more to a bar than drinking. In Chicago (Rosemont), php[tek] had Shoeless Joe’s, a nearby sports bar, where attendees would congregate each evening following the conference activities. In St. Louis, I don’t know what that place will be, but I’m certain there will be one. This is where relationship-building continues. There’s so much that can come from these relationships—friendships, business opportunities, mentors, and more!

I closed down Shoeless Joe’s with @cspruck, @ElizabethN, @derickr, @coderabbi, @JimLindForPope, @dshafik, @ptahdunbar, et al. #phptek

— Ben Ramsey (@ramsey), May 23, 2014, Twitter
Take advantage of the “hallway track,” and don’t forget the regular sessions.

There is a high level of value gained by conversing with other conference attendees in the hallways between (and during) sessions at conferences. This “hallway track” has been touted as one of the most important reasons for attending a conference. The hallway allows you to build relationships and gain better understanding and insight into the technologies and tools you use on a daily basis. It’s an invaluable offering of any conference, but don’t forget about the value gained by attending the sessions. A healthy balance of both will help you have an awesome conference experience.

Untitled, by Eli White / CC BY-SA

tek11 – Red Carpet, by Blue Parabola / CC BY-NC-SA

php[tek] 2013, by Ben Ramsey / CC BY-SA
Attend the morning keynote sessions.

The morning keynote sessions set the tone for the conference, and this year’s keynotes are no different. While it’s tempting to stay up into the wee hours of the morning—building relationships at the St. Louis version of Shoeless Joe’s—and sleep in to catch up on rest, you’ll miss some of the most important messages of the conference. The keynotes keep you thinking the rest of the day, and they provide conversation fuel for the hallway track and the after-hours community building. So, plan your rest accordingly and don’t miss the keynotes.

PHP Rock Stars, by Rich Bowen / CC BY

It is forbidden…, by Sebastian Bergmann

php[tek] 2014: Root beer beer, by Ben Ramsey / CC BY-SA

The Elephpants are coming!, by Paul Reinheimer
Join the conference IRC backchannel: #phptek on Freenode.

The #phptek channel on Freenode IRC is the conference backchannel. If you’re unfamiliar with IRC or don’t have an IRC program, use Freenode’s free webchat to join the #phptek channel. The conference backchannel is an essential tool for navigating the conference. Use it to chat with other attendees during sessions to find out what they’re learning and to share what you’re learning. Use it to discuss your dinner plans, and invite others along with you. You may also use it to keep in touch when the conference is over.

Untitled, by Timothy Boronczyk

Party!, by Rob Allen / CC BY-ND

Untitled, by Timothy Boronczyk
Use the official hashtag: #phptek

The conference’s official hashtag is #phptek. Not #tek. Not #tek16. It’s #phptek. Use it when posting on Twitter. Use it for posts to Instagram. Search nearby check-ins on Swarm. Use it on your photo uploads to Flickr. Wherever you post about your php[tek] experience, use the hashtag #phptek. Using the hashtag helps connect you and your posts to the greater conversation about php[tek].

phpwomen shoot, by Derick Rethans

The phpwoman boothbabes, by Derick Rethans

Chicago style pizza!, by Rob Allen / CC BY-ND
Take lots of pictures and upload them to Flickr.

If you don’t have a Flickr account, go get one. Then, take lots of pictures at php[tek] and upload them to Flickr with the hashtag #phptek. I also recommend licensing your photos for use under a Creative Commons license (Flickr can help you with this), but this is not a requirement—it’s just helpful to the community. Here are photos on Flickr from every year of php[tek]:
There’s one more search for php[tek] 2016, but it has no photos (as of May 22). It’s up to you to add them!
- php[tek] 2016

php[tek] has been a cultural event in the PHP community over the last eleven years, inaugurating many community memes and launching many community projects. Over the years, we’ve started a Terry Chay Fuck Counter; hosted PHP internals meetings; started the PHP standards group that morphed into the PHP-FIG; held sessions on community leadership and panels on running user groups; created numerous unseemly, alliterative names for our favorite local sports bar; drank lots of root beer; held summits on mentoring, mental health, and open source; experienced Ed Finkler’s first presentation on “Open Sourcing Mental Illness;” exposed the PHP Bard; and much, much more.

The last nine years in Chicago have been amazing for php[tek]. I can’t wait to see what St. Louis will bring.

They’ve still not replaced the bat at Shoeless Joe’s. #phptek @joshholmes https://pic.twitter.com/E4TNdXYGDs

— Ben Ramsey (@ramsey), May 21, 2015, Twitter

One last root beer beer. Cheers! @paleozombie @ramsey @kevinbruce @SandyS1 @EliW @CattyCreations @omerida #phptek https://pic.twitter.com/2Vy7JD6gJt

— Beth Tucker Long (@e3BethT), May 23, 2015, Twitter

My biggest take away from #phptek is that, as a community, we are starting to think about each other as people. That’s pretty cool

— Jeff Carouth (@jcarouth), May 24, 2015, Twitter

Mastering OAuth 2.0 Published

2016-05-15T02:00:00+00:00

I’ve published my first article in php[architect] magazine since 2009! It’s only fitting that it’s an article on OAuth 2.0, since one of the last articles I published in their magazine was on OAuth 1.0. I’m proud and excited to finally publish a new article with them after such a long hiatus, and I hope my next article doesn’t take seven years to write.

OAuth 2.0 is the de facto standard for authenticating users with third-party websites. If you want access to a user’s data in Google or Facebook, for example, OAuth 2.0 is what you use. But, let’s face it: OAuth 2.0 is not easy, and to make matters worse, it seems everyone has a slightly different implementation, making interoperability a nightmare. Fortunately, the PHP League of Extraordinary Packages has released version 1 of the league/oauth2-client library. Aiming for simplicity and ease-of-use, league/oauth2-client provides a common interface for accessing many OAuth 2.0 providers.

It wasn’t easy for me to write this article. Call it writers’ block, procrastination, or what have you, I kept putting off writing the article over and over. I had more than a few false starts, and I tried a variety of different formats, convinced it was my workflow and process that was hindering me. In the end, I just had to write. I owe Oscar a few beers at php[tek] for putting him through that.

Mistakes Were Made

I began writing the article in August of last year. As a result of the long writing process, the Instagram client I registered for use in writing the article was registered before Instagram deprecated the /users/self/feed endpoint (thanks to an astute reader for pointing this out!). This means there are errors in the published version of the article, and I am truly sorry for this. As a technical author, it’s embarrassing—especially since it’s an error that could have been avoided.

Fortunately, there’s an easy fix for this. In Listing 2 of my article’s example code (app/Http/Controllers/HomeController.php), you’ll find the following lines of code:

$feedRequest = Instagram::getAuthenticatedRequest(
    'GET',
    'https://api.instagram.com/v1/users/self/feed',
    $instagramToken
);

This code uses the deprecated Instagram endpoint https://api.instagram.com/v1/users/self/feed. There is no replacement for this endpoint, but we can substitute it with another one: https://api.instagram.com/v1/users/self/media/recent. This new endpoint won’t return a user’s Instagram feed, but it will return their most recent uploads, which will suffice for the example application. The updated code should look like this:

$feedRequest = Instagram::getAuthenticatedRequest(
    'GET',
    'https://api.instagram.com/v1/users/self/media/recent',
    $instagramToken
);

Thanks For Reading

If you took the time to read the article, thank you so much. If you haven’t yet, go grab a copy of the May issue and read it.

Please take a moment to let me know what you think. OAuth can be explained in simple terms, but to grok it, it’s difficult to distill into concise code examples. That was my goal. I’m working the article into a talk, so I’d like to know how I can improve the content and examples.

On Twitter, I posted a thread of tweets where I mention a few things about OAuth and then list many of the php[architect] articles that have covered it. Feel free to read through this conversation and check out the other OAuth articles to increase your own understanding.

While OAuth is used practically everywhere these days, there’s still so much misunderstanding around how it works. #oauth (1/12)

— Ben Ramsey (@ramsey) May 13, 2016, Twitter

Finally, my friend Matt Frost recently wrote Integrating Web Services with OAuth and PHP, also published by the good folks at php[architect]. Check it out to learn more about OAuth versions 1 and 2.

Post-Open Source

2016-04-26T00:00:00+00:00

I’m a tad late to this discussion, but I think it’s still pertinent today—perhaps even more so—and Jordi Boggiano’s recent post, “Common files in PHP packages,” got me thinking about the lack of open source licenses in public repositories.

In his post, Jordi explains how he analyzed all packages at Packagist, specifically for the sake of identifying common file names developers are using for their change logs. As part of that analysis, he was also able to tell how many projects have a license file, about which he writes:

55% [of PHP packages at Packagist] have a LICENSE file, that’s.. pretty disastrous but hopefully a lot of those that don’t at least indicate in the README and composer.json

In a 2013 analysis of software licenses on Github, Aaron Williamson, then Senior Staff Counsel at the Software Freedom Law Center, found that 14.9% of repositories had a top-level license file, while 3.7% only announce the license in the project’s README^[1]. Of the top licenses, he noted that there has been a significant shift since 2000 in favor of more permissive licenses (MIT, BSD, etc.) and surmises this could be the result of “corporate influence/allergy to GPL” or a reaction against the GPL, favoring “freedom of developer over freedom of users.”

Why are so few repositories adding open source licenses?

Luis Villa posits it might be because developers are rejecting permission culture.

The open license ecosystem assumes that sharing can’t (or even shouldn’t) happen without explicit permission in the form of licenses. What if “post open source” is an implicit critique of that assumption – saying, in essence, “I reject the permission culture”?

So, when James Governor posted in September 2012 his sentiment about younger developers being about “post open source software,” it was perhaps a bit of tongue-in-cheek crotchety cane-shaking about a cultural shift in developer attitudes toward open source and the need to grant permission.

younger devs today are about POSS – Post open source software. fuck the license and governance, just commit to github.

— James Governor (@monkchips), September 17, 2012, Twitter

If we’re in a post-open source era and open source licenses represent permission granted to use one’s code, then is this era marked by a reaction against the need for that permission? After all, the “younger devs” grew up in a post-Napster world full of DRM, EULAs, IP/copyright lobbyists, and legalspeak about what we can and cannot do with the content and software we’ve purchased. Open source licenses are yet another way to proliferate that permission culture. It’s no wonder there’s a backlash against the need for licenses.

In Lawrence Lessig’s 2004 book Free Culture, Lessig warned:

Free cultures are cultures that leave a great deal open for others to build upon; unfree, or permission, cultures leave much less. Ours was a free culture. It is becoming much less so.

Are open source licenses just another manifestation of the shift to a permission culture and away from a free culture? While companies have embraced open source software and many contribute back to open source projects under a variety of permissive licenses, I can’t help but feel that open source is losing its soul. These days, I don’t hear people talking about it as a philosophy. Rather, the focus is always on licensing and business cases—the permission to use it.

What do you think? Do open source licenses propagate permission culture? Are we in a post-open source era?

The analysis used FOSSology to examine 1,692,135 Github repositories out of about 6 million. Alternate locations of licenses could not be accounted for (file headers, subdirectories, unexpected file names, etc.). ↩

Introducing ramsey/uuid

2016-04-24T18:00:00+00:00

It seems quite absurd for me to introduce ramsey/uuid, a library that saw its 1.0.0 release on July 19, 2012, and is now at version 3.4.1, having had 35 releases since its first, but what’s even more ludicrous is that I haven’t once blogged about this library. I mention it only in passing in my “Dates Are Hard” post. So, allow me to introduce you to perhaps a familiar face, an old friend, the ramsey/uuid library for PHP.

Beginnings

I’ve been asked on more than one occasion why I created ramsey/uuid. Why was it needed? Why did I open source it?

It all began with Composer. In 2012, Composer was taking off, and there was a lot of excitement around creating userland PHP packages and distributing them for others to use. I had contributed a number of times to open source projects, but I had never maintained one of my own. So, it began as an experiment. I wanted to experience what it was like to manage an open source project and accept pull requests, feedback, and bug reports from others.

Once I had resolved to create a package for this little experiment, I needed something to work on that presented a problem I felt others in the PHP community had not yet sufficiently solved. I also looked to other programming language communities to see what problems they had solved that PHP could benefit from.

At some point, I stumbled across the Java and Python UUID implementations, both of which provide rich interfaces for generating UUIDs. Aside from the PECL uuid package and a handful of small libraries generating UUIDs with mt_rand(), I couldn’t find a PHP userland implementation providing functionality similar to that of the Java and Python libraries.

I had found a problem in need of a PHP userland solution! I set to work right away, quickly releasing a 1.0.0 version. Little did I know this marked the beginning of a long road for a small package that would become popular and widely-used.

What is a UUID?

UUID is an acronym for universally unique identifier. A UUID is a 128-bit integer with some special formatting rules based on its variant and version. When presented as a string, a UUID looks something like this:

379dae82-5a2b-4c4b-8193-b8e7749a3495

A UUID aims to be practically unique such that information may be uniquely identified across distributed systems, without central coordination of identifiers. There are 16³² possible UUIDs, so it’s highly unlikely that there will be a duplicate. According to Wikipedia, for randomly generated UUIDs, “only after generating 1 billion UUIDs every second for the next 100 years, the probability of creating just one duplicate would be about 50%.”

RFC 4122 defines a specific variant of UUIDs having five versions:

Version 1 is constructed from the current timestamp and local machine MAC address
Version 2 is the DCE Security version; it is similar to version 1, but RFC 4122 does not explicitly define it, so it is left out of most implementations
Version 3 is constructed from a namespace and an MD5 hash of a name; given the same namespace and name, the UUID generated will always be the same
Version 4 is randomly-generated and is probably the most common version used
Version 5 is the same as version 3, but it uses SHA-1 hashing instead; it is the preferred version for name-based UUIDs

What’s In a Name?

What’s in a name? That which we call a rose
By any other name would smell as sweet.

William Shakespeare, Romeo and Juliet

If you used ramsey/uuid before the 3.x series, you’ll recall that this library began its life with the vendor name Rhumsaa. There were several problems with this name. It was too close to Ramsey, so many people assumed that’s what it was. As a result, in conversation, it was referred to as Ramsey UUID, and folks searching for it would use the search terms “Ramsey UUID.” It became very confusing, and those who knew it was Rhumsaa and not Ramsey didn’t know how to pronounce it.

I received complaints about not remembering how to spell it. I heard from others who didn’t realize I was the package maintainer. One developer even encountered a problem and assumed it was a result of my “GitHub username change;” they thought I had changed my GitHub username from rhumsaa to ramsey, breaking the location of the package in Packagist.

So, what does rhumsaa mean, after all? As I wrote in my “Wild Garlic” post, “[T]he word rhumsaa is a Manx word derived from the Old Norse hrams-á, meaning ‘wild garlic river.’ In English, this word is ramsey.”

I was attempting to be clever with my vendor name, and it caused a lot of confusion.

As we were deep in the middle of development on the 3.x series, I decided a vendor name change from 2.x to 3.x might be a good idea and mitigate a lot of this confusion. I asked on Twitter and opened Issue #48 to solicit feedback from the community for the name change. In the end, I made the decision to change the vendor name to Ramsey. I first updated my other Rhumsaa packages (i.e., ramsey/twig-codeblock, ramsey/vnderror, etc.) and then I changed the name of ramsey/uuid for the 3.x series.

I tried to make the transition as easy as possible. I’m sure there are better ways to handle changes like this, and in retrospect, I probably should have forked the package to allow projects to use both rhumsaa/uuid and ramsey/uuid together, similar to how the Guzzle project addressed a similar namespace and package name change. Nevertheless, I’ve only heard from a handful of those who’ve encountered problems with the upgrade or couldn’t upgrade yet due to other packages using the older 2.x series.

When UUIDs Collide

Shortly before the GA release of 3.0.0, I received a troublesome bug report. Issue #80 purported to show that version 4 UUID collisions were occurring on a regular basis even in small-scale tests, and as I mentioned earlier this should not be probable. After several more corroborating reports, we were faced with a conundrum.

Since I couldn’t reproduce the issue, and no one could produce a sufficient reproducible script, the issue sat around for a long time. Every couple of weeks or so, someone would chime in to ask the status or confirm they had seen collisions. It began to scare people, and I was worried that community confidence in the library was degrading. I was actually stressed by the whole situation; I wanted my library to be useful and dependable.

Finally, after many months attempting to identify the culprit—I was certain it wasn’t inside the library’s code, since ramsey/uuid relies on external random generators—I had a conversation with Willem-Jan Zijderveld and Anthony Ferrara in the #phpc channel on Freenode IRC. Willem-Jan pointed us to the OpenSSL random fork-safety issue, where the OpenSSL project explains:

Since the UNIX fork() system call duplicates the entire process state, a random number generator which does not take this issue into account will produce the same sequence of random numbers in both the parent and the child (or in multiple children), leading to cryptographic disaster (i. e. people being able to read your communications).

OpenSSL’s default random number generator mixes in the PID, which provides a certain degree of fork safety. However, once the PIDs wrap, new children will start to produce the same random sequence as previous children which had the same PID.

They go on to say:

OpenSSL cannot fix the fork-safety problem because it’s not in a position to do so.

OpenSSL was the culprit. More specifically, the use of openssl_random_pseudo_bytes() when using PHP in forked child processes, as is the case when using PHP with Apache or PHP-FPM. The processes were wrapping, so the children would produce the same random sequences as previous children with the same process IDs.

Discovering this launched discussions on what to do about OpenSSL for the paragonie/random_compat library. After that project decided to drop the use of OpenSSL as a fallback for generating random bytes, I decided to require paragonie/random_compat as a dependency and use random_bytes() as the default random generator for ramsey/uuid. I then released versions 2.9.0 and 3.3.0 to provide versions in both 2.x and 3.x to solve this problem.

ramsey/uuid 2.9.0 & 3.3.0 fix the UUID collision issue caused by OpenSSL. All users should upgrade. Thanks! https://t.co/N6Rwz1a5e2

— Ben Ramsey (@ramsey), March 22, 2016, Twitter

It’s interesting to note that @SwiftOnSecurity picked up on the issue and posted about it:

Fascinating investigation into developers getting tons of UUID collisions in small datasets (via @CiPHPerCoder) https://t.co/apEOzx2m3B

— SwiftOnSecurity (@SwiftOnSecurity), March 26, 2016, Twitter

3.4.1 and Beyond

ramsey/uuid has undergone many changes since its 1.0.0 release. That very first release had some severe limitations placed on it, due to the math involved. It also had some grievous bugs because of that math. I required that everyone using the library must use a 64-bit system, and I failed to factor in the unsignedness of the integers. Since all PHP integers are signed, this led to some serious problems in generating UUIDs.

The 2.x series of the library followed about seven months later, supporting both 64-bit and 32-bit systems and accounting for the unsignedness of UUID integers through the use of a BC math wrapper library, moontoast/math. We—for it really was a community effort—made many improvements and enhancements over the course of the 2.x series, but it was clear that more flexibility was desired, and this led to the changes in the 3.x series.

The 3.x series ushered in a great deal of flexibility through interfaces and dependency injection. While the standard public API was left unchanged, all the guts of the library were completely transformed to allow anyone to use their own random generator, time provider, MAC address provider, and more.

Now, as the library matures beyond the 3.4.1 version, I’m looking ahead to the 4.x series, and how it will further improve the library with more flexibility and closer adherence to RFC 4122, while providing some facilities to optimize UUIDs in databases, and more.

Here are a handful of the issues I’m considering for 4.0.0. You can read more and may submit your own from the ramsey/uuid GitHub issues page.

How To Use It

The ramsey/uuid library provides a static interface to create immutable UUID objects for RFC 4122 variant version 1, 3, 4, and 5 UUIDs. The preferred installation method is Composer:

$ composer require ramsey/uuid

After installation, simply require Composer’s autoloader (or use your own, or one provided by your framework of choice) and begin using the library right away, without any setup.

$uuid = \Ramsey\Uuid\Uuid::uuid4();
echo $uuid->toString();

The library will make some decisions about your environment and choose the best choices for generating random or time-based UUIDs, but these are configurable. For example, if you wish to use Anthony Ferrara’s RandomLib library as the random generator, you may configure the library to do so:

$factory = new \Ramsey\Uuid\UuidFactory();
$factory->setRandomGenerator(new \Ramsey\Uuid\Generator\RandomLibAdapter());

\Ramsey\Uuid\Uuid::setFactory($factory);

$uuid = \Ramsey\Uuid\Uuid::uuid4();

If you wish to provide your own random generator, you may do so by implementing Ramsey\Uuid\Generator\RandomGeneratorInterface and setting your object as the random generator to use.

Likewise, the library supports the ability to configure the time provider. If you’d like to use the PECL uuid package, for example, to generate time-based UUIDs, this is possible.

$factory = new \Ramsey\Uuid\UuidFactory();
$factory->setTimeGenerator(new \Ramsey\Uuid\Generator\PeclUuidTimeGenerator());

\Ramsey\Uuid\Uuid::setFactory($factory);

$uuid = \Ramsey\Uuid\Uuid::uuid1();

There are a variety of other ways to configure ramsey/uuid. This example configures the library to generate a version 4 COMB sequential UUID with the timestamp as the first 48 bits.

$factory = new \Ramsey\Uuid\UuidFactory();
$generator = new \Ramsey\Uuid\Generator\CombGenerator($factory->getRandomGenerator(), $factory->getNumberConverter());
$codec = new \Ramsey\Uuid\Codec\TimestampFirstCombCodec($factory->getUuidBuilder());

$factory->setRandomGenerator($generator);
$factory->setCodec($codec);

\Ramsey\Uuid\Uuid::setFactory($factory);

$uuid = \Ramsey\Uuid\Uuid::uuid4();

Thanks

I couldn’t wrap up this post without thanking a few key project contributors. Were it not for the efforts of these folks, ramsey/uuid would not be the great library it is today.

I want to first thank Marijn Huizendveld. Marijn submitted the first pull requests to ramsey/uuid and contributed the Doctrine ORM integration that I later split out into the separate ramsey/uuid-doctrine library. It was Marijn’s participation that got me excited about collaborating on an open source project and continuing the work.

I owe a debt of gratitude to Thibaud Fabre for his instrumental work in taking ramsey/uuid to version 3. He set out to re-architect the library, providing the interfaces and structure for codecs, generators, providers, and more. I’ve learned a lot about organizing software, object-oriented programming, and dependency injection from his involvement.

Most recent, Jessica Mauerhan has been a force for improving our test suite, improving overall test coverage and adding tests for internal bits that were covered but not fully tested. I’ve learned a great deal about testing from her contributions.

Last but definitely not least, there are many more without whose contributions ramsey/uuid would be a lesser library. I am grateful to you all for your hard work and help in making ramsey/uuid an awesome library.

Cheers!

Hour of Code

2015-12-21T00:00:00+00:00

It was dark when I pulled into the parking lot of the middle school where I would spend the next hour volunteering for the Hour of Code. I had no idea what to expect. I hadn’t spoken much with the site coordinator, and I thought 5pm was an odd time to host the event.

I’ve spoken numerous times at many conferences but never about technology to middle-grades students. I wanted to instill in them a love for technology, but I wasn’t sure how to relate to them, and so I ran over and over in my head different scenarios of things I would say and questions they might ask. I was nervous on a different level.

“What if I say the wrong thing and turn them off to programming?”

“What if I bore them, and they think coding is boring?”

The school day had ended some time ago, and I walked along quiet, locker-lined halls, as I made my way to the classroom, smelling those once-familiar, musty smells of decades-old, concrete block school buildings. Down a hall, somewhere, a floor cleaning machine hummed, as a custodian buffed the white, vinyl tile floors. I recalled many hours I had spent at school after hours for extracurricular activities.

I spoke briefly to Kathy, the site coordinator, before entering the classroom, and I learned that the school has a variety of after-school programs their students participate in. This one happened to be the STEM program. They were mainly focused on conducting science experiments, but they decided to do the Hour of Code during Computer Science Education Week this year. She had never done the Hour of Code and had requested a volunteer to help out. I was to be the one leading the entire program.

No problem. Winging things is my specialty. So, that’s what I did.

I spent a few minutes telling about what I do as a programmer on a daily basis, and I quickly realized that, while I’m passionate about what I do, it can sound rather banal when being explained to others, so I switched gears. I told them a brief version of my origin story, so to speak.

I began with a question: “Who has ever written code?” No one raised their hands, so I pressed on.

“I wrote my first computer program when I was in fourth grade. It was a Mad Lib generator.” The students seemed to perk up and get interested at this.

I continued with my brief introduction to tell them that I went on to build my high school’s first web site back in 1995, but I never saw programming as a career. It had always been a hobby for me. Then, I stumbled into a job building web sites and doing sever-side programming, while I was still in college. It wasn’t until a professor convinced me to keep my job and drop the education requirements of my degree (student teaching, etc.) that I began to see my hobby as a career. I wanted to impress upon the students that careers don’t have to be boring things—they can be the things we are passionate and excited about. The things we enjoy doing in our free time are things that we can do for a living.

From there, I had them open up code.org on their Chromebooks and click the “Start learning” button. They chose the “Learn an Hour of Code” exercise that appealed most to them and proceeded to complete the steps. Some students breezed through the activities, while others struggled with the concepts. I used this as a teaching point, since my primary goal was to teach the students that failure and trial-and-error are natural parts of coding. If something doesn’t work the first time, then keep trying until it does work. Sometimes it takes many attempts before you get it.

After Kathy asked me several questions about the history of programming, I decided to wrap things up and try to answer her questions in a brief review of what the students had learned. I asked them what their favorite part of the exercise was and what they felt the most difficult part was. The answers varied. Then, I introduced a tiny bit of computer science history with a discussion of ENIAC, bugs, and Grace Hopper.

By then, our hour was up, and it was time for the students and myself to go. The time had been all-too-short, but I will volunteer again, and I learned a few things that I’ll keep in mind for the next time. I encourage you to volunteer, and I’m sharing my take-aways to help you when you do:

Don’t spend a lot of time talking. Get the students right into the coding exercises.
Chances are, the teachers you are working with haven’t been through the Code.org exercises. If they contact you through the Code.org volunteer site, they probably think you’ll be prepared to lead the class. So, be prepared to lead.
Code.org provides teacher notes for each of their exercises. Familiarize yourself with the exercises and the notes.
Tell the students what you do for a living, but don’t focus too much on the specifics of your job. Our jobs are mostly boring to adolescents, unless we’re video game designers or movie animators.
Do tell what made you interested in coding and how you got started.
Tell about the first program you wrote.
Make sure you point out the importance of failure in writing code. Almost everything I have learned in coding has been a result of failure through trial-and-error. Students need to know how important this is.

Interested in volunteering for an Hour of Code near you? Sign up here to have teachers contact you.

Yak Shaving Is the Entire Job Description

2015-11-30T00:00:00+00:00

Earlier this year, I worked on a solution to help us manage changes and history when maintaining different versions of Amazon Machine Images (AMIs). I entertained a wide range of ideas from Docker to AWS CloudFormation to a collection of shell scripts.

Finally, after asking in #pynash (Nashville’s Python user group) on Freenode IRC, Jason Myers pointed me to Packer as a potential solution.

Packer is a tool for creating machine and container images for multiple platforms from a single source configuration.

Packer turned out to be the right tool for the job, but I almost scrapped it, since I ran into a few problems.

When I began my journey as a programmer, every task was fraught with problems, and I loved it. Everything was new, and every problem was an opportunity to learn and grow. It was great.

Somewhere along the way, though, problems became nuisances. As I grew older in life and my career, my tolerance for problems became lower, and my desire for things to Just Work™ became greater.

As I struggled to find a solution for the problem I had with Packer, Tate Eskew reminded me that yak shaving is a part of my job.

<teskew> i'd just patch it, then. keep it in a central place so you only compile it once.
         then when it's put into mainline, just switch out the binary
<teskew> either way, it's a pretty simple patch to test and change
<ramsey> ugh… gotta set up an environment just to build packer
<ramsey> "simple"
<teskew> you act like yak-shaving isn't the name of the game
<ramsey> :P
<teskew> hell, it's the entire job description
<teskew> :)
<ramsey> I am not an ops person
<teskew> you are right now :)

I ended up patching Packer for my needs, and I had fun doing it. I learned a valuable lesson that day: Despite how we may gripe and complain about shaving yaks, it’s part of our job as programmers and problem solvers, and often, it’s the part that brings us the most satisfaction. I’ve learned to embrace yak shaving, and doing so has changed my outlook on my job, open source contributions, and community organizing.

Yak shaving isn’t just part of our jobs, it’s the entire job description.