Fdk test doesn't return proper exit codes if tests fail

Fdk test always returns 0 exit code. This prevents us from using the command in the build pipeline. I looked at test.js file in the sdk and saw the following code:

    mochaRunner
      .run(failureCount => {
        // Wait for the coverage to be collected before exiting
        setTimeout(() => {
          coverageUtil.shutdown();
          process.exit(failureCount > 0 ? -1 : 0);
        }, ONE_SEC_MSEC);
      });

The problem is that it never reaches process.exit line because the code exits in coverageUtil.shutdown() and always with code 0.

Looks like returning error code is intended but doesn’t happen due to a bug.

1 Like

Welcome to the developer community :tada:

@KiselN - Thanks for creating a topic on this.

We are still growing in the space where developers could use our CLI to build pipelines. If you could help me out with a couple of more questions, I could have enough knowledge handy to take this to our engineering team.

  1. At what point does the process exit? Is it when all the tests are completed running? Or when one of the tests fails?
  2. What should be the ideal way CLI should be doing here? Do you think the process should not exit? Or maybe if there’s a flag like fdk test --no-exit that doesn’t let the process exit. Could that help?
2 Likes

The process should of course exit but it should exit with code success (0) when all tests pass and error (not 0) when any of the tests fail. Right now it always exists with 0.

Then our pipeline will use the exit code to decide whether build fails or passes.

Hi @KiselN,

Thanks for elaboration. That helps. From the same view, this could be a bug that we’ve haven’t anticipated.

Let me get connect with our engineering team and get back to you.