Flexible software management for a payroll engine
Part 1: GIT for storage of complex parameterisation

At Teal Partners we specialise in complex software projects. In this two-part blog series we are keen to tell you about the approach we took in one of our projects.

The developer team at Teal partners was looking for the ideal architecture on which to pin its development of an international payroll engine. That calculation engine has to contend with changing legislation that differs from one country to another. This is why parameterisation is so crucial, says Guy De Gruyter, software architect at Teal Partners. For storage they turned to GIT. He explains why in this blog.

Teal Partners is developing an application in collaboration with SD Worx to support HR departments in small to medium-sized enterprises. This software package allows SMEs to manage the entirety of the HR-related information for their employees. The new payroll engine performs background calculations of the wages, social security contributions and tax.

The payroll engine treats wage calculation and wage payment as separate processes. This offers scope for prognoses and simulations, from the employer and employee perspectives. It allows a better idea of how certain wage scenarios impact the wage costs (for the employer) and the net results (for the employee).

Flexible management of the payroll engine is crucial

Payroll calculations are regulated by law. These legal regulations are complex, subject to frequent change and specific to country, region and sector. For that reason, managing and updating the regulations is central to the development of a new payroll engine.

It is crucial to adapt quickly and correctly to changes in the law. Take the coronavirus pandemic, for example. In rapid succession new support measures were introduced to create breathing space for businesses. Payroll software must be capable of incorporating these changing regulations seamlessly.

The separation of development and parameterisation guarantees speed and flexibility

Flexible management of the parameterisation is therefore a key feature for SD Worx. For that reason, we kept the software development separate from the management and publication of the parameterisation. That separation lets us apply the new regulations sooner. In other words, this type of parametrisation allows the regulations to be defined by non-software developers. Typically, they would be business experts or functional analysts, who are more adept at transitioning to the new regulations.

The separation takes the payroll-engine software to a more abstract level. In the application we only use program code to set the payroll engine options. By this we mean things like the different entities, the calculation levels and their dependencies and the user interface supporting the process for end users.

In the parameterisation we establish three things. Firstly, the behaviour of the payroll engine, such as relevant calculation levels. Secondly, the interpretation of certain payroll input, such as hours worked or absences. And thirdly, the wage calculation itself, such as a gross and net pay comparison, employer contributions or the quarterly settlement.

Config module for parametrisation management in a microservice architecture

The payroll engine is modular and uses a microservices architecture. In that architecture we have placed parametrisation management in a separate module, the config module. This module publishes the payroll engine's parametrisation.

We opted from the outset to store the definition dataset, i.e. the relevant set of parametrisation data, in each of the operational module databases. In this way, referential integrity can be guaranteed.

While each module stores just a part of the parametrisation, the config module manages the entirety of the parametrisation data. Consistency is guaranteed by means of specific validation rules.

The illustration below clarifies the configuration module's position relative to the operational modules.

Why store our parametrisation in GIT?

Converting legislation into parametrisation looks, in some respects, like software development. It throws up similar challenges.

• Management and resolution of conflicting changes:
  
  The software solution will be deployed internationally. This means that a variety of domain specialists (per country, sector or region) will need to adjust or extend the parametrisation simultaneously. They will all use our configuration module at the same time. And this carries a risk of conflicting changes. In software development we are quite familiar with the challenge posed by conflict merging.
• Need for change tracking :
  
  Regulations are changing constantly, and any change could have a serious impact on the salary calculation and, therefore, the payroll input. This is why a transparent view of every parameterisation change is so important. A version control system like GIT allows us to track every change (automatically).
• 3. Change isolation:
  
  In regulation management it is important to isolate changes, so that each change can be published separately. Simple changes can be published sooner than the more complex issues. This corresponds to the separate feature releases we see in software development.

The above challenges gave us the idea of not storing our parametrisation in a classic SQL environment or document database, but rather a GIT repository. GIT repository technology meets the needs we have just described.

JThe JSON data format

In the GIT repo the parametrisation is stored in JSON documents. It is a file structure that is easy to read and process. There is yet another advantage to storing the parametrisation in documents rather than denormalized SQL tables. Even without a user interface these files are suitable for the content experts, due to their structure and content.

Here is an example of a JSON configuration file:

{
  "Code": "CompanyCar",
  "ConfigSource": "General",
  "ValidFrom": "1900-01-01",
  "ValidUntil": null,
  "AssetType": "LeaseCar",
  "BenefitCalculationClass": "GrossUpContract",
  "Icon": "car",
  "Labels": [{
    "Language": "Dutch",
    "Hash": null,
    "Lb1": "Leasewagen"
  }, {
    "Language": "English",
    "Hash": null,
    "Lb1": "Lease car"
  }, {
    "Language": "French",
    "Hash": null,
    "Lb1": "Vulture de leasing"
  }, {
    "Language": "German",
    "Hash": "29f5932",
    "Lb1": "Leasing-Fahrzeug"
  }]
}

Six-step branching and release strategy

Every change in parametrisation goes through the same release cycle as a feature release in the software. A change in parametrisation is isolated in a feature branch. There, it is approved by a pull request before merging with our DEV branch. Eventually, after all the tests have been run, it ends up in the MASTER branch. Once there, the change is validated in the business-acceptance environment before proceeding to the production environment.

There are six steps in the process of bringing a legislation change to production.

• A domain specialist enters a parametrisation change in a feature branch of the MASTER branch.
• That change is published through a pull request, then reviewed and added to the DEV branch.
• The change is then deployed to the development environment. Here, change testing is automated. This prevents a regression of the existing implementations. We refer to this as continuous integration.
• The change is released to the quality assurance test environment, where the legislation change can be validated by the domain specialist.
• After that, the change is merged and moves from the DEV branch to the MASTER branch for release in the ACCEPTANCE environment. Here, a final check is run.
• Once accepted by the client, the change is deployed to the production environment.

GitHub guarantees the best integration

The next challenge was to integrate GIT in the configuration module. We need to be able to query information from the config module about branches, commits and data, and to execute data changes and merges.

This integration is an important way to prevent functional profiles from having to deal directly with GIT commands like squash, rebase or merge.

We looked at a range of possibilities. In the end, we went for GitHub. It guarantees the best API support for the software integration. Besides a traditional REST API, GitHub publishes a GraphQL API.

This technology offers greater query precision and flexibility than the classic REST API. As it turned out, this was an important element in guaranteeing performance under a more intensive use of parametrisation.

Conclusion

In conclusion, we have (1) separated software configuration from software development, (2) stored the configuration as JSON files in a GIT repository, (3) used a pipeline for the configuration release matching that for a software release, and (4) built a GitHub integration that lets us develop a support process for the parametrisation.

Next week, in part 2, we will go deeper into the use of microservices architecture within this structure. Be sure to join us if you want to know more about our approach.