Websocket script run stays 30sec after a given duration time

[tarekbadrsh]

Hi, I'm a bit curious about K6 project and I'm trying to run K6 with different scenarios.

I was trying to run this websocket script and this one, with --duration 1s , with --duration 10s and with --duration 20s

Environment

• k6 version: v0.32.0 and latest master (with Id: 5b44097)
• OS and version: windows10

Expected Behavior

I expect the application to run for 1s, 10s or 20s

Actual Behavior

The application continues running for 30 in addition of any duration option
run with --duration 1s stay running for (31.0s)
run with --duration 10s stay running for (40.0s)
run with --duration 20s stay running for (50.0s)

As you can see below I have the latest version on master also I tried with latest tag v0.32.0 and it have the same behavior.

K6/(master)$ git log -n 1
commit 5b4409706162ddcea1644f8dbdcd69ba0e86719d (HEAD -> master, origin/master, origin/HEAD)
Author: Cem Türker <cemturker92@gmail.com>
Date:   Fri Jun 11 14:17:23 2021 +0200

    Added GetName method for metrics (#2058)

    * Added GetName method for metrics

    Closes #2053

    * fix indentation and only keep example for one metric.

    * fix comment indentation

    * remove extra new lines

    * Fix lint

K6/(master)$ go build . && ./k6.exe run --vus 5 --duration 1s script-websocket1.js

          /\      |‾‾| /‾‾/   /‾‾/
     /\  /  \     |  |/  /   /  /
    /  \/    \    |     (   /   ‾‾\
   /          \   |  |\  \ |  (‾)  |
  / __________ \  |__| \__\ \_____/ .io

  execution: local
     script: script-websocket1.js
     output: -

  scenarios: (100.00%) 1 scenario, 5 max VUs, 31s max duration (incl. graceful stop):
           * default: 5 looping VUs for 1s (gracefulStop: 30s)

WARN[0000] error getting terminal size                   error="The handle is invalid."
INFO[0001] connected                                     source=console
INFO[0001] connected                                     source=console
INFO[0001] connected                                     source=console
INFO[0001] connected                                     source=console
INFO[0001] connected                                     source=console

running (31.0s), 0/5 VUs, 0 complete and 5 interrupted iterations
default ✓ [======================================] 5 VUs  1s
WARN[0031] No script iterations finished, consider making the test duration longer

     data_received...: 1.0 kB 33 B/s
     data_sent.......: 1.0 kB 33 B/s
     vus.............: 5      min=5      max=5
     vus_max.........: 5      min=5      max=5
     ws_connecting...: avg=264.24ms min=259.35ms med=264.38ms max=269.35ms p(90)=268.73ms p(95)=269.04ms
     ws_sessions.....: 5      0.161261/s

I'm not sure if this issue might be related #2040

Steps to Reproduce the Problem

1. Get k6 version:
   • git checkout 5b44097 (latest master)
   • git checkout v0.32.0
2. Copy the first script from here into script.js locally
3. Run go build . && ./k6.exe run --vus 5 --duration 1s script.js

[na--]

@tarekbadrshalaan, this happens because you're not actually closing your websocket connection, either from the server or from k6 (see Socket.close() in the k6 ws docs) and, by default, k6 has gracefulStop: 30s (see the k6 output you pasted above). That means it will wait for your iterations to finish gracefully for up to 30s (or whatever you configure) after the stated duration. If you set it to 0, it will kill them immediately (i.e. forcefully 😅 ). See https://k6.io/docs/using-k6/scenarios/graceful-stop/ for details.

You can adjust gracefulStop only through the scenarios option, for now: https://k6.io/docs/using-k6/scenarios/#common-options

@imiric @mstoykov @codebien, while the current WebSocket API is not ideal and we plan to make a better one after we have proper event loops (#882) in VUs, I wonder if we shouldn't add a new event to the Socket.on() handler and have its callback be triggered when the normal test run duration is over? That is, something that at t=1s in the example above would have triggered and allowed @tarekbadrshalaan to gracefully close the websocket connection?

[imiric]

I'm not sure if we need a new event for this since k6 is behaving as documented, and the behavior @tarekbadrshalaan wants can be adjusted with gracefulStop.

Maybe the WS behavior if gracefulStop is reached should be to call close() on any open sockets to try to close the connection gracefully. Otherwise if they need more control over it, then a separate event might be the way to go.