Helios FHIR Server 0.2.6 keeps passwords out of logs and streams large SQL on FHIR results

Helios FHIR Server 0.2.6 stops writing database passwords and other secrets into its logs. It also streams large SQL on FHIR results instead of holding them in memory, speeds up search on SQLite and PostgreSQL, and stops losing audit events under load.

Slide text

hfs v0.2.6

Helios FHIR Server 0.2.6

Passwords stay out of your logs, and big SQL on FHIR results stream

Plus faster SQLite and PostgreSQL search, and audit events that survive heavy load.

hfs used to log its database connection string at startup exactly as configured. Here is a MongoDB address as it appeared, password included. In 0.2.6 the line keeps only the hosts and the database name. Help output, Debug output and errors sent to clients are redacted too, and dependency updates clear four RustSec advisories. Older logs still hold those passwords, so rotate them.

$sql-run used to build the whole JSON, CSV or FHIR response in memory before sending a byte. For a ViewDefinition subject, it now streams row by row. Over 1.8 million rows on SQLite, peak memory drops from about four and a half gigabytes to under two megabytes. PostgreSQL and MongoDB drop sharply too, and the first byte arrives in milliseconds.

Slide text

SQL on FHIR

$sql-run now streams JSON, CSV and FHIR output row by row

Peak memory, SQLite, 1.8M rows

4,542 MB

1.8 MB

PostgreSQL	4,106 MB	4.2 MB
MongoDB	4,057 MB	170 MB
First byte	3–13 s	2–90 ms

ViewDefinition subjects · json and csv · 20k Patients and 1.8M Observations · memory over idle

Before you upgrade, check these three. A $sql-run result over 100 rows sends its 200 status early, so a later error aborts the body, and the request timeout no longer ends the run. If you haven't sized the PostgreSQL pool yourself, it is now capped at half the server's max connections, shared with other pools such as database auditing. And in the terminology server, POST always assigns its own ID, while PUT creates a resource at the ID you choose.

Slide text

Check before upgrading

Streaming, the PostgreSQL pool and HTS create behave differently

$sql-run over 100 rows

Status 200 goes out after the first 100 rows. A later error aborts the body, and the request timeout stops applying.

HFS_REQUEST_TIMEOUT

Default PostgreSQL pool

Capped at half the server's max_connections, shared across pools. With database audit on a stock server: 25 each.

HFS_PG_MAX_CONNECTIONS

HTS create

POST assigns its own ID and ignores a submitted one. PUT creates or restores at the ID in the URL.

PUT /CodeSystem/{id}

Audit writes used to be fired off once per request, so under load some were lost. At 300 virtual users on a stock PostgreSQL server, about 2.8 percent went missing. Now events wait in a bounded queue with retries, and the same test lost none. By default a full queue makes responses wait, which cost about six percent of throughput in that test. Set HFS_AUDIT_OVERFLOW to drop if you'd rather keep the speed, and a metric counts each dropped event.

Slide text

Audit

Audit events stop getting lost under load

Before

5,071

of 183,600 audit events lost

After

0

audit events lost

PostgreSQL, max_connections 100 · 300 virtual users for 60 s · database audit sink

HFS_AUDIT_OVERFLOW=block | drop HFS_AUDIT_WRITERS audit_events_dropped_total

Search got much faster. On SQLite, covering indexes and a bounded first-page probe take token, quantity, composite and date searches from about 14 to about 440 requests per second at 30 virtual users. On PostgreSQL, a newest-first date index lifts search throughput by nearly half and cuts Observation date search latency by almost 90 percent. MongoDB sends a transaction Bundle's creates in chunks of up to 100.

Slide text

Performance

SQLite search handles about 30 times the requests

Before 14 req/s

After 436 req/s

Token, quantity, composite and date searches · 30 virtual users · identical results

PostgreSQL: +46% search throughput

A newest-first date index. Observation date searches: 88% lower latency.

MongoDB: Bundle creates in chunks

Up to 100 creates per insert, instead of several commands for each entry.

Among the fixes, FHIRPath equality now returns empty when two quantities carry units that can't be compared, and memberOf returns empty on empty input. A bulk submit with a deferred reindex resumes it after a restart. And on SQLite, a PUT that loses a race is retried.

Slide text

Fixed

Fixes you may have run into

FHIRPath

Quantity = on units that can't be compared, and memberOf() on empty input, return empty

Bulk submit

A deferred reindex resumes after a restart on PostgreSQL and MongoDB

SQLite

Restores of deleted resources are serialized; PUTs that lose a race are retried

S3 export

The _type pre-pass stops reading every object one after another

SQL on FHIR

unionAll keeps each branch's column type instead of guessing

Dashboard

The snapshot cache is bounded

Upgrading comes down to three steps. Rotate any passwords that sat in your connection strings. Give the first start extra time, since SQLite and PostgreSQL migrate their schemas before serving, and a large SQLite database can take minutes. If you run the terminology server, stop every writer on its database and upgrade them together. Version 0.2.6 keeps passwords out of your logs and streams big SQL on FHIR results.

Slide text

Upgrade

Rotate old credentials, then give the first start extra time

1

Rotate passwords that sat in connection strings

Logs from earlier versions still hold them.

2

Expect a longer first start

SQLite v41 to v46 and PostgreSQL v49 to v53 migrate before serving. A large SQLite database can take minutes.

3

HTS: stop every writer on a shared database

Upgrade them all before resuming imports.

$ docker run -p 8080:8080 ghcr.io/heliossoftware/hfs:latest