Dev Notes

Quick little story: I was working on a relatively big task to duplicate a bunch of data in a system I barely know. Now, the biggest problem here was that the data in question was hierarchical sql data. That makes it tricky because relational tables (rows and columns) are not well suited to make it easy to traverse such trees. You could relatively easily use a migration logger functionality and click through the UI on the backend to record the sql that you need to migrate this data. But this gets messy quick and can go easily wrong. The more data you need to duplicate the more important it is that you do the right thing for an increasingly long sequence of manual actions. Also there is cleanup necessary afterwards, since the migration logger is not optimizing its lightly abstracted sql statements well. On top of that, I could not find a good interface to program my migration against either. It seemed that my options were:

  1. use the UI and record the changes using the migration logger
  2. write what you need using raw sql yourself

I decided to go with option 2. I found a great solution that did not require a nested set approach called a recursive common table expression. I felt like a genius. I also was sitting on my high horse, “How did no one ever come up with a good API for doing common things like: copying this entity over onto this branch? Well, I guess I have to do it myself…”

Even though progress was made to create migration helper functionality, it was slow and I had lots of questions regarding the structure of the data I was supposed to duplicate.

Yesterday my boss took 2 hours of his time to look at the problem with me. He pointed out that what I had been doing for the last three days was unnecessary. Of course, what could be done in the UI in the backend could also be accomplished programmatically. It wasn’t even that hard. It had all the scaffolding for customization of the process I could have ever wanted. This will allow me to write a readable migration in a few hundred lines, with no later cleanup necessary. We solved a big chunk of my problem in no time.

I was embarrassed. Not only did I underestimate my colleagues and the maintainers before me, I also underestimated the system and its capabilities. I felt especially bad, because I thought I had done the right thing, like asking many questions and communicating with the project management why there was such a delay. I thought I was dealing with a hard problem - and in a vacuum it actually was a hard problem - but instead I just didn’t know where to look or what to ask.

It’s a new day and I think I take a bunch of lessons from this:

  1. Ask more directly: Do not ask “Is it guaranteed that every group starts with a node that has itself no parent?”, but ask “Is there an API to copy categories just like the UI has?”. See also: The XY Problem.
  2. Be okay with making such mistakes. I am new in the company. It is almost inevitable that I will run in circles and it may take some time before my knowledge of the system matches my level of general problem solving skills. Because I could have solved the hard problem - which is why I tackled it - I just didn’t need to.
  3. Accept your expertise level. As I said I am new here and at the same time I have programmed for a while so I generally know what I’m doing. This is a great recipe for falling prey to the Dunning–Kruger effect: That is overestimating one’s ability in a particular context. I may be an okay problem solver, but I do not know the new system.

There is a certain inevitability to feeling a bit of shame while learning from mistakes. It’s part of the journey in a field as complex as programming. I’m still learning to embrace these moments, understanding that they are not just hurdles but stepping stones towards greater expertise and confidence.

As I move forward, I remind myself to maintain a balance: to stay humble yet curious, to recognize the boundaries of my current knowledge while also questioning established norms. This experience has taught me that becoming a better developer isn’t just about accumulating knowledge, but also about the wisdom to question, to explore, and, most importantly, to learn from slightly embarrassing fumbles.

So, with these lessons in my toolkit, I am moving on - a little humbler and wiser, a bit more prepared, and as eager as ever to challenge and be challenged. After all, it’s through questioning and exploring that we often find the most innovative solutions and grow beyond our imagined limits.

PHPStorm's keybinding system is ridiculous

It is no news that I am pretty skeptical that PHPStorm is actually a good IDE. I especially question their handling of international keyboards. You see, on an international keyboard like the German one the keyboard shortcuts cmd + shift + 7 and cmd + / are virtually the same, because there is no way to type a / without typing shift + 7.1

To my knowledge the only modern IDE that offers a default keymap for macOs that doesn’t work out of the box with my default international keyboard is Jetbrain’s offering. The reason is that some genius thought to implement key bindings without any understanding what symbols are necessarily typed by using a modifier key. So you end up with a keyboard shortcut like cmd + / that doesn’t work in PHPStorm, but will work out of the box in VS Code.2

P.S.: The problem is old btw.: The offical issue in their bug tracker is 7(!) years old.

P.P.S.: Is this a hard problem? Maybe. The more interesting question to my mind is, though: Why is it a solved problem in all other IDEs? This is table stakes.


  1. By chance you can also type cmd + devision symbol, but that is not the same as a forward slash and is indeed - and this time correctly - its own shortcut in PHPStorm. Try to type that one on a keyboard without a numpad, though - which means this shortcut is useless on all(!) MacBooks - if you don’t have an external keyboard attached. ↩︎

  2. To be fair: The shortcut ends up being displayed as shift + cmd + 7, but at least vsc won’t act like cmd + / and cmd + shift + 7 are completely different key bindings (which they are not). ↩︎

The (Hu)go template syntax is bad.

There is no two ways about it. It’s just bad and often counterintuitive. It’s hard to read, hard to write, hard to debug, hard to fill in blanks. Easy things should be easy and hard things should be possible. I feel like everything is just hard with it - not impossible per se, but often harder then it should be.

I mean just look at this one line:

{{ $paginator := .Paginate (where .Site.Pages.ByDate.Reverse "Type" "post") (index .Site.Params "archive-paginate" | default 25) }}

This does/means the following:

  • $paginator - that’s a custom variable, it needs to be prefixed with a $
  • := - you need to use this operator to declare a variable and assign a value. But later it’s fine to just use = to reassign it.
  • . - “the dot”. It holds the context, which is basically a kind of scope. Why the scope is not implied? I don’t know. I guess if you’d have a this keyword, you’d end up writing this.whatever… and you would still need to differentiate globals and in-scope vars, so I guess that’s better?
  • .Paginate - Inside “the dot” there exists a Paginate function which has to be upper case because that is the way to make a function visible outside of its “package” in go. If you look at the list of available functions in hugo, you will know that a function was exported, but not why. Also a lot of functions were not exported, but you also do not know why. I assume that the lower case functions are all part of hugo’s templating standard lib. But it’s not explained what is going on in the docs explicitly.
  • - A space (yup). Functions and parameters are separated by spaces, so this space indicates the start of the Paginate function call (And not the parentheses in front of the where, as one might think when coming from other languages).
  • ( - The start of the parentheses denotes a nested expression.
  • where - that’s the where function which takes an array and compares a value at a given key using an operator (equals is implied and can be omitted like has been done here) to a value and only keeps the elements that pass the test.
  • .Site.Pages.ByDate.Reverse - The .Site.Pages part is an array of all pages of the blog. The ByDate.Reverse part is used (and available only) on list pages - another detail you’ll have to know - that is pages that have other pages under them in the file hierarchy. AND that the homepage is a special kind of list page. This is a snippet from the home page, so you can use it here to change the order of the retrieved array. Why you can’t retrieve pages in similar manner on non list pages is unclear to me.
  • "Type" - This is the key parameter of the where function. The key is “Type” here. Why is Type capitalized? Maybe it has to because of exports? Maybe it’s some other reason that I don’t know. In any case it is part of the keys of the element that is kept (or discarded). For the pages array an element includes things like a Permalink, a published Date and a also a Type. How do you know what an element includes? Well, it’s hard to say, because the proposed solution to use something like &#123;&#123; printf "%#v" . }} within a range function call that references the pages from the Paginator above only prints garbage. An actual solution is to use <pre>&#123;&#123; . | jsonify (dict "indent" " ") }}</pre> which gives a pretty printed JSON representation of all the available properties within a given context. But you’ll not find this solution in the docs, you have to get lucky and find it in the forum or Stack Overflow.
  • "post" - this is the match parameter of the where function. In other words the value found for the key “Type” must match this value in order to be included in the filtered array.
  • ) ( - we finish one nested expression - which is parameter one of our .Paginate call (the array to paginate) and start another nested expression - parameter two (the page size) of .Paginate
  • index .Site.Params "archive-paginate" - this returns the value at index or key n of a given array. So in this case the value at .SiteParams.archive-paginate
  • | default 25) if index does not return a value the default function can take over and return a default value. I don’t know why this has to be a function. I also find the whole part index .Site.Params "archive-paginate" | default 25 difficult to parse: Is the pipe still part of the index function call? You’ll have to know what pipes are.

Finally, we have parsed the whole thing:

instantiate customVar = 
functionCallThatReturnsAPaginator(
    functionCallThatReturnsAFilteredArray,
    (
        functionCallThatReturnsANumber ||
        functionCallThatReturnsaAdefaultValue
    )
)

And .Paginator itself is just an object that makes it easier to refer to and implement a paged navigation for an array of given elements (posts).

Apart from the unusual syntax I find (Hu)go templates hard to parse, even if I grok them somewhat. (Hu)go’s way to write function expressions, namely using spaces instead of parentheses and commas make the whole line harder to read than it needs to be. Compare:

{{ $paginator := .Paginate (where .Site.Pages.ByDate.Reverse "Type" "post") (index .Site.Params "archive-paginate" | default 25) }}

vs.

{{$paginator := .Paginate(where(.Site.Pages.ByDate.Reverse,"Type","post"),(index(.Site.Params,"archive-paginate") | default(25)))}}

Granted, this is still hard to read, because a lot is happening in this one line of code, but still: I’d argue it’s much easier to parse, because opening and closing parentheses and commas carry much more information than a simple space could. Spaces are also commonly used to align or balance things as has happened around the pipe char. Does this carry semantic meaning? Nope, not in this case!

A visualization of what different symbols of the templating syntax mean in our example. It turns out that the space char carries three differnt meanings: an aesthtic space, a start of a parameter list, a delimiter between parameters.

The space is doing an enormous amount of overtime here and I have yet to see a good justification of muddling the waters like this. The only reason I could see is that you have to balance parentheses, meaning you’re ending up with the line ending in ))). The best part is that you still need parens in any case, you just have to put them around the whole function expression! The real template version safes you two parentheses for the price of a parsing headache. I feel like that’s not worth it.

Let’s move on: It should be super easy to limit the list of pages to only include a certain category of posts, right? Would this have been what you’d come up with on the first try?

{{ $allPosts := where .Site.Pages.ByDate.Reverse "Type" "post" }}
{{ $allDailyDogos := where .Site.Pages "Params.categories" "intersect" (slice "DailyDogo") }}
{{ $onlyDogos := intersect $allPosts $allDailyDogos }}
{{ $paginator := .Paginate ($onlyDogos) (index .Site.Params "archive-paginate" | default 25) }}

So far so good. How about the inverse? You’ll find that there is no way to tell the line $allDailyDogos to simply do the inverse. There is no "not intersect" or whatever. You have to use another function called symdiff:

{{ $allPosts := where .Site.Pages.ByDate.Reverse "Type" "post" }}
{{ $allDailyDogos := where .Site.Pages "Params.categories" "intersect" (slice "DailyDogo") }}
{{ $noDogos := symdiff $allDailyDogos $allPosts }}
{{ $paginator := .Paginate ($noDogos) (index .Site.Params "archive-paginate" | default 25) }}

Symdiff is short for symmetric difference and means here that we want all elements from the $allPosts array that are not part of the $allDailyDogos array. Meaning we are keeping only those posts, which is like filtering for the inverse of the $onlyDogos array from before.

We could have mashed this all into one line to make it totally unreadable, but I think this is instructive and it would have not been anymore readable in other template languages (if they even would’ve been able to deal with this as a one liner). Still:

  • Why do I need to make a slice/array out of “DailyDogo”?
  • Why is there no NOT operator? - it would’ve been nice in two instances: 1. inside the where function to save an extra call to symdiff or 2. as a better (I’d say) alternative to using symdiff, because people think more along the lines of “all but not these kinds of things” instead of “symmetrical difference of these two sets”.
  • Why is it so hard to chain conditions to the where function to the point where you’d rather create two arrays instead of filtering the array down in one step?

Ugh. There is so much in just this one line - and the subsequent slight changes I have done - that I find weird and unergonomic. I hope I could also show that it’s not complete and utter failure to understand what’s going on, either. Sometimes things are just badly designed for the most obvious use cases in order to accommodate fancier goals. I’d rahter have a templating syntax/language that makes things easier. If that makes it more boring: Good. I’m here to improve my blog first and foremost.

Good to see that state management in SPAs is still hard since the last time (3,5 years ago) I used one of these frame works. This time it’s vue3 last time it was react.

When I try to figure how something works in my programming language I often use the service replit. It offers a simple bare bones php environment which is ready to go to test out some stuff, is portable and free to use.

One thing that is slightly annoying is that they only support PHP 7.4 out of the box, but it is very easy to upgrade the php version used to PHP 8. Let’s start with an example:

<?php
$str = "Hello, world!\n";
if (str_contains($str, 'llo')){
  echo 'YUP';
}

This code will not run as is on replit, because the function str_contains doesn’t exist before PHP 8.

Screen Shot 2023 03 09 at 14 39 02

So let’s change that. Click the three dots in the side bar and reveal hidden files:

Screen Shot 2023 03 09 at 14 41 39

Next, open the replit.nix file and change the used php version, like so:

{ pkgs }: {
  deps = [
    pkgs.php //from pkgs.php74
  ];
}

Without needing to do anything else we have instructed nix - the package and config manager underlying much of replit.com’s functionality - to use the latest php package which happens to be php 8.

If we run our little test program now, it’ll work:

Screen Shot 2023 03 09 at 14 47 10

NB: The version of nix on replit is not up to date, so trying to use php82 to get the latest and greatest PHP Version 8.2 won’t work. But php 8.0 is still better than php 7.4

It still is a challenge to find the right words in code reviews, no matter what side I’m on:

As the reviewer I want to give really good reasons, be persuasive but also signal that I know that we live in a contingent universe. If I have knowledge to give I want to explain things well, without coming off as paternalistic.

As the reviewed I want to be open minded, interested, but also being able to challenge things back without coming off as defensive.

IDE Troubles: PHPStorm and VS Code

I work as a programmer for my day job. Right now I am working on two php projects. Coming from Sublime Text but having had the need for more IDE features I came to love Visual Studio Code and made it my home. VS Code is a great editor for PHP development, especially if you use the Intelephense extension. However, the bigger the project, the more it becomes apparent, that the performance of the language server is not that great.

I lived with this for many months now. Yesterday I started working on a new issue and thought to myself: “All of my dev team members use Jetbrains PHPStorm, I should give it a try.”. I had used Jetbrains Intelij Idea for a brief moment a while back and only remembered being fairly unimpressed, but not exactly why I felt that way. Having now tried getting into their PHP IDE product, I have to say my impression has not really changed and now I also remember why.

Although the code intelligence performance is clearly better, it is the small stuff that gets me. The impetus for this post for example was that you can’t use standard shell shortcuts with the integrated terminal: Control + R (backwards history search) being one of my most often used commands.

I appreciated that they offer a keymap for people coming from Code to PHPStorm and as far as I can tell, the commands in it work. But customizing an IDE is more than just having the same default shortcuts for common tasks available in the new app. I have a bunch of my own shortcuts in code that use the pattern CMD + H <Key> (press CMD + H and then <Key>). I use CMD + H, because my last name start with H, so it was (and is) easier to remember. It just so happens that CMD + H is also a system wide shortcut for hiding an application. So when I tried to set up a frequently used shortcut to reveal the currently opened file in the sidebar navigation, CMD + H S the PHPStorm app was just hidden. In other words it did not “shadow” or override system shortcuts when asked. This feels even more arbitrary if we take those two things together: On the one hand standard shortcuts of the shell running inside the integrated terminal are shadowed per default - and this can also not be disabled -  and on the other hand wanting to shadow a system wide shortcut like CMD + H is not possible without remapping the shortcut within the OS. I feel like the IDE is not really playing along, but instead makes me jump through hoops.

These are only two compounding cuts of a thousand that make PHPStorm hard to love in my opinion. I have an especially hard time understanding the design decisions to remove long standing functionality from a shell by shadowing standard shortcuts. It’s just a bad idea. It does explain why my colleagues never use the integrated terminal, but have an extra shell window open to the side, to run terminal commands.

All in all I am very unimpressed by PHPStorm, but also worried: If the performance of Intelephense stays as bad much longer, I might have to look at other options. I know that Netbeans and Eclipse also have PHP features, I just don’t know anyone using, let alone loving these apps for PHP development…

Forward.

Taking a break from microcasting

I recently published a post about my microcasts and I‘m now writing this to tell you I’m going to microcast less. Less frequent and less scheduled. This is even weirder because I just created an intricate shortcut that automatically publishes a post with the right episode title and number without having to look up any of those details, which streamlined the entire process a lot.

But I have noticed that I‘m not as much in the mood at the moment to record my thoughts in this way. The early riser project, for example, was a great McGuffin to get me out of bed, but since I have changed my approach to waking up, it has become less important to me. I don‘t want to record these little episodes for the ramblings alone.

Being a big fan of podcasts like Back To Work and Cortex, I thought I could do a microcast about my work and how I approach it, but I have noticed that I would like to write about these things, not only talk about them extemporaenously. LeadDev is also the microcast for which the ratio of recorded/published episodes is the worst: I take this as an indicator of my wanting to express myself more carefully than I can while walking the dog and having to observe the environment.

À propos the dog: I created my first microcast, the PuppyCast, because I wanted to have a record of the challenges and joys of raising a puppy. I wanted to look (DailyDoogo, which I will not abandon, btw.) and listen back to this important time in my life. But I think that this project has run its course: I don‘t feel the need to publish this every week anymore. It‘s not like nothing changes or that there is nothing to report dog-wise, but it all comes down to the need to simplify and de-schedule my life. And I have episodes for the first six months, which seems like a good place to stop.

Seasons change. And this is how I feel about my microcasting: Right now I‘d much rather write than talk. Looking at my blog I mostly see noise that was produced hastily to satisfy a self-imposed schedule. I will let this stuff go, for now.