PM2 reloads now wait until new LINE bot workers are listening

2 hours ago

This release of the Cofacts LINE bot carries one fix, for anyone who runs it under PM2, which is how its Docker image starts the server. When PM2 reloads the bot, it now waits until the new workers are listening before it stops the old ones. That closes a short window during each reload where LINE's webhook calls were refused.

Ask about this presentation

Chapters

  1. 0:00PM2 reloads wait for ready workers
  2. 0:21The October 5 reload
  3. 0:53The most common reload failure
  4. 1:17The ready handshake
  5. 1:43The reload, fixed
  6. 2:01How to upgrade
  7. 2:24Both ends of a reload
Show transcript
  1. This release of the Cofacts LINE bot carries one fix, for anyone who runs it under PM2, which is how its Docker image starts the server. When PM2 reloads the bot, it now waits until the new workers are listening before it stops the old ones. That closes a short window during each reload where LINE's webhook calls were refused.

    Slide text
    Cofacts logo

    PM2 reloads now wait until new LINE bot workers are listening

    This closes the gap where LINE's webhook calls were refused mid-reload.
    rumors-line-bot · 2026/10/13 Bugfix · release/20261013
  2. Here's one reload, just after quarter past midnight UTC on October 5th. The new workers came online at the two-second mark, but startup takes about seven seconds. PM2 waited its default three seconds, then stopped old workers zero and one at the five-second mark, and worker two at 8.4 seconds. The new workers started listening at nine seconds. In between, cloudflared logged two refused connections, and the LINE Developers console showed two 502 errors.

    Slide text
    rumors-line-bot · release/20261013

    On October 5, PM2 stopped every old worker before a new one was listening

    Old workers 0, 1
    stopped :05
    Old worker 2
    stopped :08.4
    New workers
    online at :02, starting up
    listening
    :02
    :03
    :04
    :05
    :06
    :07
    :08
    :09
    seconds past 00:15 UTC · 2026-10-05
    connection refused ×2 in cloudflared, and two 502 errors in the LINE Developers console
  3. PM2 reloads the bot on a cron every three hours, and each reload could open that same gap. Over a month of cloudflared logs, from September 6th to October 5th, a refused connection during a reload or deploy showed up 13 times, more than any other reload-time failure. The connection resets that the October 1st release targeted showed up 5 times.

    Slide text
    rumors-line-bot · release/20261013

    Refused connections were the most common failure during reloads

    connection refused
    13 events
    The gap this release closes
    connection reset by peer
    5 events
    Targeted by the October 1 release
    cloudflared logs, Sep 6 – Oct 5 2026 · during a reload or deploy · PM2 reloads every 3 hours
  4. The fix is a handshake across two files. In ecosystem.config.js, setting wait_ready to true makes PM2 wait for a ready message, and listen_timeout gives it fifteen seconds instead of the default three. In src/index.js, the listen callback sends ready once the server accepts connections. It only sends when process.send exists, so running the app outside PM2 works as before.

    Slide text
    rumors-line-bot · release/20261013

    Each new worker now tells PM2 when it is listening

    ecosystem.config.js
    kill_timeout: 15000,
    + wait_ready: true,
    + listen_timeout: 15000,
    exec_mode: 'cluster',
    Wait for a ready message, for up to 15 s instead of PM2's default 3 s.
    src/index.js
    app.listen(process.env.PORT, () => {
    console.log('Listening port', …);
    + if (process.send) process.send('ready');
    });
    Sent once the server accepts connections. Outside PM2, the guard skips it.
  5. Here's the same reload with the fix. Startup still takes about seven seconds. PM2 now keeps the old workers running until the new ones report ready, so a worker is listening the whole way through. The new fifteen-second limit is about twice today's startup time, which leaves room for a slow start.

    Slide text
    rumors-line-bot · release/20261013

    Old workers now stop only after the new ones report ready

    Old workers
    running
    stop after ready
    New workers
    starting, about 7 s
    listening
    3 s default
    'ready'
    listen_timeout 15 s
    0
    3
    7
    15
    seconds after a reload starts · schematic
    A listening worker covers every second of the reload.
  6. Both settings ship in the repository's own ecosystem.config.js, and the Docker image copies that file in. So the upgrade is to rebuild the image with docker build, and deploy it. If you start the bot with your own PM2 config instead, add the same two lines, wait_ready true and listen_timeout fifteen thousand. The app already sends the ready message.

    Slide text
    rumors-line-bot · release/20261013

    Upgrading is a rebuild and a deploy

    Docker image
    docker build .
    The image copies ecosystem.config.js in, so the new settings come with it.
    Your own PM2 config
    wait_ready: true, listen_timeout: 15000,
    Add both lines. The app already sends 'ready'.
  7. This release changes only how a reload brings up the new workers. Together with the October 1st release, which gives old workers fifteen seconds to shut down cleanly, both ends of a reload are now covered. So with this release, PM2 reloads wait until the new LINE bot workers are listening.

    Slide text
    rumors-line-bot · release/20261013
    Cofacts logo

    PM2 reloads now wait until new LINE bot workers are listening

    October 1: old workers get 15 s to close. October 13: new workers say when they are ready.
    github.com/cofacts/rumors-line-bot/releases/tag/release/20261013

What happened in AI today, in your inbox

One short email each evening. Read it and you’re caught up.

Built in New York with ❤️