I think I've found a condition under which this bug happens, and its reproducible.
As soon as I enable "Include Additional Variables" option in External, errors in the script or sys.exit(1) stop being counted as a failure in the workflow, and workflow/job itself doesn't show any errors and completes succesfully.
When you disable this option, everything works as usual, erros in the script or sys.exit(1) produce an error in the workflow and stop the job.
I should add that we have another workflow where External node does stop the workflow if exceptions are raised in a script, as expected.
I have a Python script where even after a raised exception the workflow shows no errors and just proceeds as usual as if nothing happened.
Meanwhile the exception is recorded in the stderr.log just fine.
Even adding sys.exit(1) doesn't stop the workflow or show an error in the workflow.
How to fix this?