/serveStaleOnError doesn't work for 502 and 504

According to How to serve stale content in case no render is available I have configured /serveStaleOnError on our dispatcher(apache 2.2 dispathcer 4.2.3/4.2.2).

For testing purpose I've created the following page:

<html>
    <head>
        <title>Throwing an Exception</title>
    </head>
    <body>
        <title>Throwing an Exception</title>
        <%
    response.sendError(503, "test failure" );
        %>
    </body>
</html>

serveStaleOnError  works fine for 503:

And doesn't work for 504,502:

What am I doing wrong? Should anything else be configured?