Michael Alabi
← All case files

Case file · June to August 2023

7+ weeks of downtime. One Java bug.

Developers and technical teams had spent over seven weeks on a failing production system without a resolution. Here is what was broken, and how it was found and fixed.

System
Patricia 6.1.1, IP management software by Patrix AB
Environment
Windows Server 2012 R2
Symptom
“Elasticsearch Service Error”
Unresolved for
Over seven weeks
Root cause
A bug in OpenJDK 1.8.0.191, the Red Hat Windows build
Outcome
Fixed, and acknowledged by Patrix support

What was installed

C:\PatriciaBrowser611

  • java-1.8.0-openjdk-1.8.0.191-1.b12.redhat.windows.x86_64 ← the bug was here
  • elasticsearch-7.17.0 ← started, then stopped
  • jboss-eap-7.3
  • jbcs-jsvc-1.1
  • LibreOffice
  • dataFiles

What Windows reported

The Elasticsearch 7.17.0 (elasticsearch-service-x64) service on Local Computer started and then stopped. Some services stop automatically if they are not in use by other services or programs.

With that service down, major services the web version of the application depends on were not running either.

The problem

Patricia is the IP management software Jackson Etti & Edu runs. In mid-2023 it went down: the Elasticsearch service would start and then stop, and the major services the web version of the application depends on stopped with it, leaving the application inaccessible. Developers and technical teams worked on it for over seven weeks without a resolution.

The diagnosis

I took it on through extensive hours of troubleshooting, in the middle of my Master's second-semester exams. The cause turned out to be a bug in the Java JDK the software runs on: OpenJDK 1.8.0.191, the Red Hat Windows build.

The fix
  1. Downloaded a different version of the Java JDK and tested it in a test environment against what we were running: both Elasticsearch and Patricia. Both services started well, and the deployments were successful.
  2. Swapped in the different Java build and reconfigured the JVM settings.
  3. Uninstalled the Elasticsearch service from the terminal.
  4. Rebooted the machine.
  5. Reinstalled the Elasticsearch service.
  6. Started a service redeploy for both Elasticsearch and Patricia, and waited until the deployment finished.
  7. Launched the application to ascertain that the Elasticsearch Service Error, which had made the application inaccessible, was corrected.
The result

My resolution worked, and the system was restored.

The follow-through

I sent the findings on to Patrix's support team. Their Support Manager replied.

“Hi Michael, thanks for this. We will add this to our knowledge base if a similar issue is reported again. We appreciate you sending this on.”
Patrix AB Support Manager, replying to my report

What it shows

Root cause analysis on a vendor's product, on a live production system, with more than seven weeks of failed attempts behind it, and a fix that was validated in a test environment before it touched production. It is the same discipline behind my Tier II support work at Micro Focus and the way I run IT.

  • Root cause analysis
  • Java JDK
  • JVM configuration
  • Elasticsearch
  • Windows Server services
  • Test environments
  • Vendor escalation

Open to Senior Technical Support and Support Engineer roles (EMEA Remote).

Start a conversation