sendmail: tlsv1 alert decode error

[Emmanuel Dreyfus]

Hi

After upgrading OpenSSL to 1.0.1g and 1.0.1h, sendmail started
producing this error when sending messages to some sites:

Jun 17 05:47:47 merteuil sendmail[14089]: STARTTLS=client, error: connect failed=-1, reason=tlsv1 alert decode error, SSL_error=1, errno=0, retry=-1

After some investigation, it seems that the TLS padding extension, which
was introduced in OpenSSL 1.0.1g, is the culprit. There are a few workarounds:

(1) Force SSLv3, which cannot use the option. This does not require any
code change but is not very appealing.

(2) Disable the TLS padding extension in libssl, which is done by this patch:
https://ftp.espci.fr/shadow/manu/libssl-padding.patch
I tested that it builds, but not that it works

(3) Let the client disable it. In Sendmail case, this requires a patch:
https://ftp.espci.fr/shadow/manu/patch-sendmail_readcf.c
Then (provided it was built with _FFR_TLS_1, which is pkgsrc option
sendmail-ffr-tls), this can be used in sendmail.cf:
O ClientSSLOptions=-SSL_OP_TLSEXT_PADDING

I tested this latest workaround and it fixes the problem. I submitted
the change to Sendmail maintainers.

What workaround should we us in NetBSD? (1) is not reasonable. (3)
is sendmail specific, which means others MTA will not benefit it,
and we need (2). IMO we should check in both.

[John Nemeth]

On Jun 17, 1:40pm, Emmanuel Dreyfus wrote:
}
} After upgrading OpenSSL to 1.0.1g and 1.0.1h, sendmail started
} producing this error when sending messages to some sites:
}
} Jun 17 05:47:47 merteuil sendmail[14089]: STARTTLS=client, error: connect failed=-1, reason=tlsv1 alert decode error, SSL_error=1, errno=0, retry=-1
}
} After some investigation, it seems that the TLS padding extension, which
} was introduced in OpenSSL 1.0.1g, is the culprit. There are a few workarounds:
}
} (1) Force SSLv3, which cannot use the option. This does not require any
} code change but is not very appealing.
}
} (2) Disable the TLS padding extension in libssl, which is done by this patch:
} https://ftp.espci.fr/shadow/manu/libssl-padding.patch
} I tested that it builds, but not that it works
}
} (3) Let the client disable it. In Sendmail case, this requires a patch:
} https://ftp.espci.fr/shadow/manu/patch-sendmail_readcf.c
} Then (provided it was built with _FFR_TLS_1, which is pkgsrc option
} sendmail-ffr-tls), this can be used in sendmail.cf:
} O ClientSSLOptions=-SSL_OP_TLSEXT_PADDING

Although not a sendmail issue per se, it is an interoperability
issue, so it will be in the next sendmail package update.

}-- End of excerpt from Emmanuel Dreyfus

[Emmanuel Dreyfus]

Post by John Nemeth
Although not a sendmail issue per se, it is an interoperability
issue, so it will be in the next sendmail package update.

Well, without the patch, sendmail comes broken out of the box, with
e-mail that cannot reach some domains. The only workaround that does not
involve patching is to force SSLv3 or downgrade libssl. IMO it makes no
sense to ship software with such a known issue.

I updated my patch so that SSL_OP_TLSEXT_PADDING is disabled by default
and can be re-enabled with ClientSSLOptions
and I submitted the updated version upstream:
http://ftp.espci.fr/shadow/manu/patch-sendmail_readcf.c

[John Nemeth]

On Jun 18, 6:20am, Emmanuel Dreyfus wrote:
} John Nemeth <***@cue.bc.ca> wrote:
}
} > Although not a sendmail issue per se, it is an interoperability
} > issue, so it will be in the next sendmail package update.
}
} Well, without the patch, sendmail comes broken out of the box, with

Strictly speaking this isn't true as it appears that the
problem is actually with OpenSSL.

} e-mail that cannot reach some domains. The only workaround that does not
} involve patching is to force SSLv3 or downgrade libssl. IMO it makes no
} sense to ship software with such a known issue.

Given that it's likely easier to update sendmail then OpenSSL,
it makes sense to put the workaround in sendmail.

} I updated my patch so that SSL_OP_TLSEXT_PADDING is disabled by default
} and can be re-enabled with ClientSSLOptions
} and I submitted the updated version upstream:
} http://ftp.espci.fr/shadow/manu/patch-sendmail_readcf.c

Presumably SSL_OP_TLSEXT_PADDING was created for a reason.
Your new suggested patch causes a change in behaviour from the
default. Are there ANY possible downsides to this change in
behaviour?

}-- End of excerpt from Emmanuel Dreyfus

[Emmanuel Dreyfus]

Post by John Nemeth
} Well, without the patch, sendmail comes broken out of the box
Strictly speaking this isn't true as it appears that the
problem is actually with OpenSSL.

This is the OpenSSL we have in NetBSD releases, hence without the
patch, sendmail comes broken out of the box on NetBSD.

Post by John Nemeth
Presumably SSL_OP_TLSEXT_PADDING was created for a reason.
Your new suggested patch causes a change in behaviour from the
default. Are there ANY possible downsides to this change in
behaviour?

SSL_OP_TLSEXT_PADDING is a workaround for interoperability problem
if the SSL handshake grows too big (as I understand it happens if you
adversite many ciphers). Unfortunately, the workaround causes other
interoperability problems. Disabling it brings us back to the behavior
before we upgraded OpenSSL because of heartbleed.

[Emmanuel Dreyfus]

Post by Emmanuel Dreyfus
I updated my patch so that SSL_OP_TLSEXT_PADDING is disabled by default
and can be re-enabled with ClientSSLOptions
http://ftp.espci.fr/shadow/manu/patch-sendmail_readcf.c

This patch has been accepted upstream. This is another reason to
have it in pkgsrc.