In order to expose a K8S cluster running on AWS EC2 instances (as an EKS cluster does) you will need an ELB in some form to load balance across the EC2 instances hosting the cluster. The ELB can either be a K8S ingress or it can be a route to the K8S ingress - that is, the actual ingress running inside the K8S cluster is exposed on ports on the EC2 instances, and the ELB balances across those nodes.

Types of AWS Load Balancer Controller

There are two types of AWS Load Balancer Controller.

1. The legacy in-tree controller

   This is baked into an EKS kubernetes cluster without any installation. You do not see any Pods or Service or Ingress Class for it as resources inside the cluster.

   It will create an NLB for any K8S Service with spec: type: LoadBalancer which will proxy to that service.

   It is legacy and deprecated.

2. The AWS Load Balancer Controller

   This is an external controller. It can be installed via helm. It will create resources (a Deployment, Pods, an Ingress Class) in the cluster.

   It will create an NLB for any K8S Service with spec: type: LoadBalancer which will proxy to that service.

   It will create an ALB for any K8S Ingress with spec: ingressClassName: alb.

   It is the recommended way to integrate with AWS load balancers.

   Prior to v2 it was called the AWS ALB Ingress Controller which is deprecated.

   Installation is documented here.

Differences between the Controller types

Importantly, many of the documented annotations are not supported by the legacy in-tree controller. This includes the service.beta.kubernetes.io/aws-load-balancer-proxy-protocol we will discuss below.

Some of the documented annotations have different default values. Notably service.beta.kubernetes.io/aws-load-balancer-scheme (whether the provisioned ELB should be internal or internet-facing) defaults to internet-facing on the AWS Load Balancer Controller but internal on the legacy in-tree controller.

The Ingress-Nginx Controller

The Ingress-Nginx Controller is probably the preferred K8S Ingress Class when running on AWS. It allows you to manage your ingress costs inside the cluster, by making your K8S Services have spec: type: ClusterIP and exposing them via an Ingress with spec: ingressClassName: nginx, whereas using a K8S Service with spec: type: LoadBalancer or an Ingress with spec: ingressClassName: alb will create further costly AWS ELBs. A single NLB can then act as the front door to the Ingress-Nginx Controller.

Installation of the Ingress-Nginx Controller creates, among other things, a K8S Service and Deployment. The K8S Service will have spec: type: LoadBalancer, which on AWS with the AWS Load Balancer Controller installed will provision an NLB to proxy traffic to the exposed port(s) on the EC2 instances, which in turn will forward that traffic to the nginx pods.

Configuring the provisioned NLB

The nature of the provisioned NLB can be controlled by adding AWS annotations to the nginx K8S Service. For our purposes important values are:

• service.beta.kubernetes.io/aws-load-balancer-scheme

  Not a URI scheme, instead whether the provisioned ELB should be internal or internet-facing.

• service.beta.kubernetes.io/aws-load-balancer-ssl-cert

  The ARN of an AWS Certificate Manager TLS certificate that will be served by the NLB to allow it to serve https requests.

• service.beta.kubernetes.io/aws-load-balancer-proxy-protocol: '*'

  If present, the generated Target Groups will be configured to send the Proxy Protocol v2 header.

Installing the Ingress-Nginx Controller

Via Manifest

The Ingress-Nginx team provide a manifest for installing the controller with TLS terminated at the NLB. It requires setting a couple of values, documented in the linked installation instructions. However, it has several quirks to be aware of:

1. It does not enable Proxy Protocol v2 on either the NLB or nginx

   See Enabling Proxy Protocol v2

2. It only creates one replica of the nginx pod

   As the pods are exposed on the EC2 instances forming the cluster’s nodes as the registered targets of the NLB’s target group(s), this means that all but one of the registered targets will be considered unhealthy.

   This can be fixed by adding spec: replicas: 2 (or however many EC2 nodes you have) to the Deployment manifest.

3. It does not specify a value for service.beta.kubernetes.io/aws-load-balancer-scheme

   So whether your NLB is internet facing will depend on the default, which differs per AWS controller type.

   This can be fixed by adding metadata: annotation: service.beta.kubernetes.io/aws-load-balancer-scheme to the Service manifest.

4. It takes control of redirecting http to https, preventing management of this at the Ingress resource level

   Full discussion below.

Enabling Proxy Protocol v2

Proxy Protocol v2 can be enabled as so:

1. On the NLB side by adding metadata: annotation: service.beta.kubernetes.io/aws-load-balancer-proxy-protocol: '*' to the Service manifest
2. On the nginx side by adding data: use-proxy-protocol: "true" to the ConfigMap manifest

http to https redirection

The nlb-with-tls-termination manifest implements http to https redirection by exposing an extra port 2443 named tohttps on the nginx Pods, adding a special listener on port 2443 to nginx via data: http-snippet: on the ConfigMap that always returns a 308 to https, and then specifying that the http port on the Service manifest has a target port of tohttps, which configures the NLB’s Target Group for port 80 to route traffic to port 2443 on the nginx pod (ultimately - there is a further port mapping on the EC2 instance to get to 2443).

Given that it does not enable Proxy Protocol v2, it makes sense for the manifest to do this, as most services that serve over TLS will want a redirect from http to https and without the Proxy Protocol nginx cannot know what the original protocol was. An NLB cannot do the redirect, so it has to happen at the nginx level.

This breaks if you enable Proxy Protocol v2, because the http-snippet on the ConfigMap will not be updated to expect the proxy protocol header despite adding data: use-proxy-protocol: "true" to the ConfigMap resource.

Redirecting via a custom port is unnecessary if you have a spec: tls: hosts array on the Ingress resource AND you have Proxy Protocol v2 enabled (so that nginx knows if the request were actually via https), as that will configure nginx to send redirects from http to https for all rules in that Ingress resource by default, and will also allow you to disable this behaviour per Ingress by adding metadata: annotations: nginx.ingress.kubernetes.io/ssl-redirect: "false" to the Ingress resource.

The Ingress redirect only works if you specify a spec: tls: hosts array on the Ingress resource. I’m not sure if this is an odd thing to do if TLS is being terminated downstream of the ingress at the NLB.

If you need precise control over which services have and do not have http to https redirects, it can be achieved with the following steps:

1. Enable Proxy Protocol v2.
2. Change the Service manifest - find the spec: ports port with name http and change its targetPort from tohttps to http.
3. Change the Deployment manifest - find the spec: template: spec: containers container with name controller. Delete the port with name tohttps.
4. Change the ConfigMap manifest - delete the data: http-snippet.
5. Make any Ingress resources you have that should redirect from http to https have a spec: tls: hosts containing all the hosts they serve, and metadata: annotations: nginx.ingress.kubernetes.io/ssl-redirect: "false" if they should serve over http.

Other options I have not yet explored in full:

1. Terminate TLS at nginx rather than the ELB. Then it makes sense to have spec: tls: hosts on the Ingress resources, and there is no need to worry about Proxy Protocol v2. Requires getting the certificate into K8S so nginx can serve it.
2. Use an ALB rather than an NLB by adding metadata: annotations: service.beta.kubernetes.io/aws-load-balancer-type: alb to the Service manifest. This should remove the need to have Proxy Protocol v2 enabled.

Via Helm

There is an Ingress-Nginx Controller Helm chart. I have not experimented with it yet but it may be possible to use it to avoid tweaking a downloaded manifest.

Proxy Protocol v2

Typically, a network layer 4 device like an NLB cannot provide any further information to the upstream services to which it proxies. The bits are simply sent “as is”. This is a problem with TLS terminated at the NLB, because a typical HTTP request does not contain the requested scheme within the body of the request. Consequently, the fact that the original scheme was https is lost to upstream services. Upstream services may need to construct URLs with that knowledge.

The Proxy Protocol v2 specifies a way for Layer 4 devices like an NLB to provide proxy information to upstream services via a header sent before the rest of the request (including before the request line). The Target Groups for AWS NLBs can be configured to send it (in the console, edit the Target Group’s “Attributes”), and nginx can be configured to expect it.

Ideally applications do not need this information - they should return links as URI references rather than URLs, either omitting the scheme (e.g. //example.com/my/thing) or the entire authority (e.g. /my/thing). However, some frameworks insist on returning URLs for e.g. the Location header, reconstructing the scheme from the X-Forwarded-Proto or Forwarded header, falling back on the scheme the process is listening for, because previous (obsolete) versions of the HTTP specification required the Location header to contain a URL.

If http to https redirection is expected to happen upstream of an NLB then proxy information will be required.

Using Clikt as a pure parser

If all you want is to turn command line arguments into a data class, without using the Clikt Command abstraction, you can do it as follows:

import com.github.ajalt.clikt.core.NoOpCliktCommand
import com.github.ajalt.clikt.parameters.options.option
import com.github.ajalt.clikt.parameters.options.required

fun main(vararg args: String) {
  val config: Config = CliParser.parseConfig(*args)
}

data class Config(val username: String)

class CliParser private constructor() : NoOpCliktCommand(
  name = "",
  help = "help text",
) {

  private val username: String by option(help = "the username").required()

  private fun toConfig() = Config(username = username)

  companion object {

    fun parseConfig(vararg args: String): Config =
      CliParser()
        .apply { parse(args.toList()) }
        .toConfig()
  }
}

Injecting the environment as a map

If you want to decouple Clikt from System.getenv you can do so as so:

class CliParser(
  env: Map<String, String> = System.getenv()
) : CliktCommand {

  init {
    context {
      envarReader = env::get
      autoEnvvarPrefix = "MY_APP"
    }
  }
}

Putting them together

import com.github.ajalt.clikt.core.NoOpCliktCommand
import com.github.ajalt.clikt.parameters.options.option
import com.github.ajalt.clikt.parameters.options.required

fun main(vararg args: String) {
    val config: Config = CliParser.parseConfig(*args)
}

data class Config(val username: String)

class CliParser private constructor(
  env: Map<String, String>
) : NoOpCliktCommand(
  name = "",
  help = "help text",
) {

  init {
    context {
      envarReader = env::get
      autoEnvvarPrefix = "MY_APP"
    }
  }

  private val username: String by option(help = "the username").required()

  private fun toConfig() = Config(username = username)

  companion object {

      fun parseConfig(
          args: List<String>,
          env: Map<String, String> = System.getenv(),
      ): Config =
          CliParser(env)
              .apply { parse(args) }
              .toConfig()

    fun parseConfig(
        vararg args: String,
        env: Map<String, String> = System.getenv(), 
    ): Config = parseConfig(
        args.toList(), 
        env,
    )
  }
}

Edit 2023-02-06 - we’ve made a couple of improvements:

• It’s now invariant in V, the type of the values, so an AbstractPropertyMap<String> is a subtype of AbstractPropertyMap<CharSequence>
• The type of the properties is now captured by the type of the argument to property, so in an AbstractPropertyMap<CharSequence> val aString by property("a string") will have its type inferred as String. (This is obviously useful if extending AbstractPropertyMap<Any>.)

Create this abstract class:

abstract class AbstractPropertyMap<out V>(
  private val properties: MutableMap<String, V> = mutableMapOf()
) : Map<String, V> by properties {
  protected fun <V2 : @UnsafeVariance V> property(initialValue: V2) =
    PropertyDelegateProvider<Any, ReadOnlyProperty<Any, V2>> { _, prop ->
      properties[prop.name] = initialValue
      ReadOnlyProperty(properties::getValue)
    }
}

You can then create subclasses as so:

class Links(
  id: String,
  otherId: String,
) : AbstractPropertyMap<URI>() {
  val self by property(URI.create("/v1/$id"))
  val other by property(URI.create("/v1/other/$otherId"))
}

val links = Links("1", "2")

links.self == URI.create("/v1/1")
links["self"] == links.self

links.other == URI.create("/v1/other/2")
links["other"] == links.other

links.keys == setOf("self", "other")
links.values.toList() == listOf(links.self, links.other)
links.entries == setOf(
  SimpleImmutableEntry("self", links.self),
  SimpleImmutableEntry("other", links.other)
)

Which begs the question - why?

I think the resulting class has a few nice properties:

1. Links is a Map<String, URI>, so you can retrieve values with keys only known at runtime and iterate over its entry set / key set / values without needing reflective access.
2. You only specify the key names once - no duplication, no room for error.
3. The compiler forces you to instantiate Links with all the expected keys.
4. The compiler enforces the type of all the properties created by delegation
5. You have type safe access to the properties on the resulting instance

In contrast, a mapOf<String, URI>() lacks 3 & 5. A simple class lacks 1 & 4.

A combination of the two could be made without the delegation magic, but would be more noisy and require duplicating key names:

class Links(
    id: String,
    otherId: String,
) : AbstractMap<String, URI>() {
    val self = URI.create("/v1/$id")
    val other = URI.create("/v1/other/$otherId")

    override val entries: Set<Map.Entry<String, URI>> = setOf(
        AbstractMap.SimpleImmutableEntry("self", self),
        AbstractMap.SimpleImmutableEntry("other", other),
    )
}

Fuller details can be found in GitHub’s documentation About security hardening with OpenID Connect and specifically for AWS at Configuring OpenID Connect in Amazon Web Services

Steps:

In AWS IAM

1) Add an Identity Provider in AWS IAM

• Provider: token.actions.githubusercontent.com
• Audience: sts.amazonaws.com

2) Create an IAM Role

3) Under Permissions add a policy granting whatever it is the role needs to do (can be an inline policy, whatever - this is a standard IAM role, specific to what you need to do, nothing to do with it being GitHub Actions that will do it).

4) Under Trust relationships add something like this:

    {
      "Version": "2012-10-17",
      "Statement": [
        {
          "Effect": "Allow",
          "Principal": {
            "Federated": "<ARN of the token.actions.githubusercontent.com Identity Provider>"
          },
          "Action": "sts:AssumeRoleWithWebIdentity",
          "Condition": {
            "StringEquals": {
              "token.actions.githubusercontent.com:aud": "sts.amazonaws.com"
            },
            "StringLike": {
              "token.actions.githubusercontent.com:sub": "repo:<GitHub org or username>/*"
            }
          }
        }
      ]
    }

The StringLike match is important - without it anyone who knows the ARN of the role could configure GitHub to authenticate as that role, but as they cannot change the sub that GitHub sends to AWS to authenticate you have complete control here of which repositories and branches / tags can assume this role. See Configuring the OIDC trust with the cloud.

In GitHub Action

1) Allow id_token write permission

By default the GitHub Action cannot write an id_token, so you need (at a minimum) this at the top of your workflow file:

   permissions:
     id-token: write
     contents: read

You may need further permissions for other actions! See Adding permissions settings for more details / examples.

2) Add the following step to your job:

   - name: configure aws credentials
     uses: aws-actions/configure-aws-credentials@v1
     with:
       role-to-assume: <ARN of the Role you created above>
       aws-region: $

And that’s it, really.

I’ve shown it as a gradle test run but it should be more widely applicable.

Nice thing about is is that by mounting the local directory the whole way through when the files are generated they are generated where you would expect to see them locally.

Basics:

1. GitHub Action steps

   jobs:
     build:
       steps:
         - name: start minikube
           uses: medyagh/setup-minikube@latest
   
         - name: K8S Tests
           run: run-k8s-tests.sh
   

2. run-k8s-tests.sh

   #!/usr/bin/env bash
   
   set -exuo pipefail
      
   main() {
     trap 'cleanup' EXIT
   
     kubectl config use-context minikube
      
     ensure_a_clean_namespace
   
     prepare_minikube
   
     kubectl apply \
       -f other-resources.yml \
       -f k8s-tests.yml
   
     wait_for_tests_to_finish
   }
   
   ensure_a_clean_namespace() {
     kubectl delete all --all -n k8s-test
   }
      
   prepare_minikube() {
     minikube mount .:/home/gradle/project &
   }
   
   wait_for_tests_to_finish() {
     # Wait for the job to start - otherwise there will be no pods
     kubectl wait --for=jsonpath='{.status.active}'=1 job/k8s-tests --timeout=30s -n k8s-test
      
     # Wait for the pods to be ready - be aware if the jobs is really fast this
     # will fail because ready will go from true to false too fast for the check
     # I cannot find a truly reliable of checking whether the pods *have* started in the past
     kubectl wait --for=condition=ready pods --selector=job-name=k8s-tests --timeout=30s -n k8s-test
   
     # Follow the logs to get real time info on what is happening
     kubectl logs job/k8s-tests --follow -n k8s-test
   
     # The pods should be back with ready=false when they exit
     kubectl wait --for=condition=ready=false pods --selector=job-name=k8s-tests --timeout=30s -n k8s-test
     # Meaning this should work
     local exit_code; exit_code=$(kubectl get pods --selector=job-name=k8s-tests --output=jsonpath='{.items[0].status.containerStatuses[0].state.terminated.exitCode}' -n k8s-test)
     return "$exit_code"
   }
      
   cleanup() {
     local running_jobs; running_jobs=$(jobs -p)
     # shellcheck disable=SC2086
     kill $running_jobs || true
     # shellcheck disable=SC2086
     wait
   }
   
   main "$@"
   

3. k8s-tests.yml

   apiVersion: batch/v1
   kind: Job
   metadata:
     name: k8s-tests
   spec:
     parallelism: 1
     completions: 1
     backoffLimit: 0
     # Set this to something appropriate!
     activeDeadlineSeconds: 600
     template:
       spec:
         containers:
           - name: k8s-tests
             image: gradle:7.4.0-jdk17
             command: ["gradle", "test"]
             volumeMounts:
               - mountPath: /home/gradle/project
                 name: gradle-project
         restartPolicy: Never
         volumes:
           - name: gradle-project
             hostPath:
               path: /home/gradle/project
   

Nice feature - run-k8s-tests.sh will run quite happily locally if you have minikube & bash installed.

The meat of the complication is the wait_for_tests_to_finish bash function.

If you are testing an image you are building, you can build and run it the tests fairly efficiently in minikube as so:

#!/bin/bash
set -euo pipefail

eval "$(minikube docker-env)"

docker buildx build . -t myimage:local

./run-k8s-tests.sh

(I’m using buildx build here, but if you don’t need buildx a docker build will do.)

I’ve experienced both - I was on a project with upwards of 15 devs, doing routine TDD on trunk with a zero bug policy - they were raised as physical pink index cards on a physical board, and always went to the top, so there were the next bit of work picked up.

My insight is that actually we were bug tracking with a bug database. The bug database was the physical cards. And we had bugs, so it wasn’t a zero bugs policy. What it was, was a very short-lived bug policy - we tolerated & tracked them for a very short period of time.

And the same was true of our trunk based development - we did branch. As soon as a pair diverged from trunk on their work station, we had a branch, just locally on that work station - and it needed to be merged by pushing to trunk (and sometimes, of course, that caused merge conflicts that had to be resolved). But crucially, those branches were very short-lived. Because they only existed locally we expected to merge them into trunk multiple times a day.

My theory is that the two practises are actually not in themselves the silver bullet, and indeed may represent local maxima. The real benefits are branches and tracked bugs that are short-lived. The pros of the TBD & zero bugs practises is that it makes it very difficult not to have those benefits. If you track your bugs as physical index cards on a physical board they can’t live for long - you’ll have to fix them or throw them away. If you only branch onto local machines no-one else can cherry-pick or merge in your changes, and you’re in danger of losing them to some hardware failure, so there’s a huge incentive to get your stuff on trunk ASAP.

However, if you can find a way to have those benefits whilst having remote branches & keeping bugs in a tracking system you may be able to get other benefits. I like a protected trunk/main branch - when doing TBD the build got broken from time to time, and it held everyone else up (the classic 5:30 cowboy check-in & run…). Set the CI up to auto-merge on successful build and you mitigate that a lot without adding much overhead - provided you keep the branch short-lived.

Everyone’s talking about Log4Shell. A lot of the talk is, of course, about the nature of code injection attacks, and the failure to separate code and data. It’s an area where I feel sympathy for the Log4J 2 devs - I think the mistake they made is a terrifyingly easy one to make, as every SQL injection or naked eval regularly demonstrates. Layers of abstraction can lead to forgetting when an input to a function is now untrusted data.

However, I think there’s another aspect of it that deserves consideration - the fact that the ability to do this was installed on so many systems.

How many systems need to do LDAP JNDI lookups at all? I’m guessing a pretty small percentage. How many need to do so from their logging system? A smaller percentage. And of those, how many need to load complex classes (with static initialisation) from a remote location via their logging system? I’m guessing an absolutely tiny fraction of the systems vulnerable to Log4shell were actually benefiting from the feature(s) that caused the vulnerability.

And yet there the code was, sitting on all those systems, waiting for the moment someone found the vulnerability.

I’ve been experimenting with JPMS and jlink, and if you create a JRE without the java.naming module then, unsurprisingly, you aren’t vulnerable to Log4Shell even if you’re running an old Log4J 2 version. But inertia and a reasonably high barrier to entry means that adoption of JPMS in general, and adoption of jlink cut down JREs in particular, has been pretty minimal, unfortunately. And some of the java namespaced modules are still disconcertingly enormous - for instance using Java Beans (as lots of things, including Log4J 2, require) means bringing in java.desktop, which brings in the swing, awt & applet packages amongst others. Perhaps rather more than you expected to call set on some data class…

However, this could have been managed at the Log4J 2 level. JndiManager is in the org.apache.logging.log4j.core.net package in the log4j-core jar, along with rather a lot of things capable of doing I/O over multiple protocols. So you can’t use Log4J 2 without having this logic installed. Had JNDI lookups been in its own little jar, brought in as a separate dependency, the bug would still have been in that code - but I’d hazard a guess that it would have been drastically mitigated because most systems would not have had it sitting there, a little otherwise pointless time bomb waiting to explode.

There’s a tension here with ease of use, and particularly keeping a low barrier to entry. It is nice when it turns out you can just use a feature without scrabbling around trying to work out which dependency you need to add to get it to work. But having so many features lying around unused can have a high price.

Keep it small. Keep it focussed. Leave it out by default. If you need a feature 99% of users do not need, it is entirely reasonable that you should need to add a new dependency for it to start working.

I want:

• a nightly build without cache, to prove that the build works from scratch and isn’t being propped up by a cache that cannot be recreated
• a regularly renewed cache, so that it doesn’t keep growing - often the cache retrieval, expansion & update is the longest part of my build.

It occurred to me that these two requirements play nicely together - if the nightly build could start with no cache, but prime the cache, then all builds during the day would benefit from a nice minimal cache.

I think the following GitHub action achieves this, by including the current date in the base cache key:

name: My Build

on:
  schedule:
    # Daily at 2AM
    # * is a special character in YAML so you have to quote this string
    - cron: '0 2 * * *'

env:
  cache-name: my-build-1

jobs:
  build:
    runs-on: ubuntu-18.04

    steps:
      - uses: actions/checkout@v2

      - name: Get current date
        id: date
        run: echo "::set-output name=date::$(date +'%Y-%m-%d')"

      - name: Cache whatever
        uses: actions/cache@v2
        with:
          path: ~/path_needing_caching
          # Always want a cache miss on the first build of the day, which should
          # be the scheduled 2AM one. Proves the build works from scratch, and
          # primes a nice clean cache to work with each day.
          key: ${{ env.cache-name }}_${{ steps.date.outputs.date }}-${{ github.ref }}-${{ github.run_number }}
          restore-keys: |
               ${{ env.cache-name }}_${{ steps.date.outputs.date }}-${{ github.ref }}-
               ${{ env.cache-name }}_${{ steps.date.outputs.date }}-

They are not natively supported yet (see KT-13108: Denotable union and intersection types). However, you can get something very like them using Kotlin’s by delegation:

// Third party code - I can't change
interface WebDriver
interface HasInputDevices
class RemoteWebDriver : WebDriver, HasInputDevices
class EventFiringWebDriver : WebDriver, HasInputDevices

// My code
interface CompositeWebDriver : WebDriver, HasInputDevices

class CompositeRemoteWebDriver(
  delegate: RemoteWebDriver = RemoteWebDriver()
) : CompositeWebDriver,
  WebDriver by delegate, 
  HasInputDevices by delegate

class CompositeEventFiringWebDriver(
  delegate: EventFiringWebDriver = EventFiringWebDriver()
) : CompositeWebDriver,
  WebDriver by delegate, 
  HasInputDevices by delegate

class WebDriverClient(driver: CompositeWebDriver) {
  // code depending on methods in WebDriver & HasInputDevices
}

// Client can now depend on either implementation
val client1 = WebDriverClient(CompositeRemoteWebDriver())
val client2 = WebDriverClient(CompositeEventFiringWebDriver())

It’s still a bit painful; obviously there’s a reasonable amount of repetition in the declaration of each implementation, and adding a new interface to CompositeWebDriver’s supertypes means also adding it as an extra supertype to every implementation with a by delegate declaration. Still, it decouples the client code from the actual third party implementation without having to implement every method in each interface.

• Language: Kotlin
• Target: JVM Module (JPMS)
• UI Framework: OpenJavaFX
• Build system: Gradle using the Kotlin DSL
• IDE: IntelliJ IDEA

In principle this didn’t seem an unreasonable stack. JavaFX is meant to be the latest and greatest JVM UI Framework. It requires JPMS usage, but then JPMS is meant to be the new “correct” way to develop for the JVM. Gradle is the oldest and most mature JVM build system that isn’t awful (no I don’t want to program in XML). It has good IntelliJ support. Kotlin is developed by IntelliJ and uses gradle as its build system; I’ve been programming with it for a year or two now and found it mature, well designed and a pleasure to use.

In practice I found it surprisingly painful to get things to a) work and b) work the way I wanted them to. So if you’re doing something similar you can now learn from my pain.

Tips:

Add plugins as runtime dependencies in buildSrc/

buildSrc/build.gradle.kts

plugins {
  `kotlin-dsl`
}

repositories {
  gradlePluginPortal()
}

dependencies {
  testRuntimeOnly("org.javamodularity:moduleplugin:1.5.0")
  testRuntimeOnly("org.openjfx:javafx-plugin:0.0.8")
}

This was the most useful thing for me in debugging plugin behaviour. Having a buildSrc/build.gradle.kts with the kotlin-dsl plugin makes IntelliJ add the gradle jars & sources to External Libraries. Adding any plugins you use as runtime dependencies in buildSrc/ means they also get added to External Libraries. This allows you to browse their source, hyperlink around it and set breakpoints. I find it invaluable in debugging plugin behaviour.

(This tip may become irrelevant if a future version of IntelliJ automatically adds these sources to External Libraries - vote on IDEA-197182 to help make that happen.)

Plugin application order matters

I am using the org.javamodularity:moduleplugin to manage the JPMS behaviour because gradle has no out of the box support. It needs to read the java main sourceSet to find & read module-info.java. If you change the sourceSet directories it will pick up on that change - but you need to apply moduleplugin after you change the srcSets, not before. Otherwise it picks up the standard src dirs and cannot find the file.

Irritatingly when it doesn’t find module-info.java it just silently does not apply itself, leaving you to try and understand the downstream errors, rather than failing good and hard and explaining the problem to you.

Use moduleplugin version 1.5.0 not 1.6.0

I encountered two issues with version 1.6.0:

• It has a breaking change (on a minor version increment…) and javafx-plugin:0.0.8 depends on a class that is no longer present in 1.6.0
• It gets the module path wrong when trying to run a standard mixed (java & kotlin) module

Things you just need to know about the gradle kotlin dsl

Configuring sub project plugins in a root project that should not apply the plugins is hard and confusing. Trial and error have shown me the following:

Import configurations:

val api by configurations
val implementation by configurations
val testImplementation by configurations

dependencies {
  api(kotlin("stdlib"))
  implementation("...")
  testImplementation("...")
}

Import tasks:

val test by tasks.existing(Test::class)
tasks {
  test {
    useJUnitPlatform()
  }
}

Configure java:

configure<JavaPluginExtension> {
  sourceCompatibility = javaVersion
  targetCompatibility = javaVersion

  configure<SourceSetContainer> {
    named("main") { java.setSrcDirs(setOf("src")) }
    named("test") { java.setSrcDirs(setOf("tests")) }
  }
}

Configure kotlin:

configure<KotlinJvmProjectExtension> {
  sourceSets {
    named("main") { kotlin.setSrcDirs(setOf("src")) }
    named("test") { kotlin.setSrcDirs(setOf("tests")) }
  }
}

Consider applying config by applied plugin

If you’ve got a multi-project build it may be convenient to configure sub projects by whether or not they have some plugin applied to them. In my root project I do this:

subprojects {
  pluginManager.withPlugin("kotlin") {
    // All config common to kotlin projects
  }
}

Then in any sub project I just need this in build.gradle.kts to apply all my common kotlin config:

plugins {
  kotlin("jvm")
}

This has the additional benefit that, because you explicitly applied the plugin, you have access to the plugin’s dsl in that sub project’s build.gradle.kts.

Share functions between multiple build.gradle.kts by putting them in buildSrc/

Any code in buildSrc/<main src dir> is available to all build.gradle.kts in the project and sub projects.

For instance if I create a file in buildSrc/<main src dir> called DependencyVersions.kt like so:

fun kotlintest(module: String) = "io.kotlintest:kotlintest-$module:3.4.2"
fun arrowkt(module: String) = "io.arrow-kt:arrow-$arrowModule:0.10.2"
fun kotlinCoroutines(module: String) = "org.jetbrains.kotlinx:kotlinx-coroutines-$module:1.3.2"

then in any build.gradle.kts I can use those functions to depend on modules as so:

dependencies {
  implementation(kotlintest("core"))
  implementation(arrowkt("core"))
  implementation(kotlinCoroutines("core"))
  implementation(kotlinCoroutines("javafx"))
}

Prevent intermediate directories becoming projects

By default, if you include deeply nested projects like this:

settings.gradle.kts

include(
  ":app",
  ":core",
  ":ui:api",
  ":ui:javafx"
)

the intermediate directories (in this case ui) will become gradle projects, despite lacking a build.gradle.kts and any other files. This can cause very confusing errors like this one:

FAILURE: Build failed with an exception.

* What went wrong:
A problem occurred configuring project ':app'.
> A problem occurred configuring project ':ui:api'.
   > Could not open cache directory add8lpbh91wftlbit7lhn37cw (/home/runner/.gradle/caches/6.0.1/gradle-kotlin-dsl/add8lpbh91wftlbit7lhn37cw).
      > org.gradle.api.internal.initialization.DefaultClassLoaderScope@47f3a892 must be locked before it can be used to compute a classpath!

You can fix this by including the specific project and setting its dir explicitly, as so:

settings.gradle.kts

include(
  ":app",
  ":core",
  ":ui-api",
  ":ui-javafx"
)
project(":ui-api").projectDir = file("ui/api")
project(":ui-javafx").projectDir = file("ui/javafx")

Example Project

These ideas can be seen implemented at https://github.com/Mahoney-example/example-gradle-kotlin-javafx