Manipulate ASiC-E containers and XAdES/eIDAS signatures for Estonian e-identity services
Project description
pyasice - ASiC-E (BDOC) and XAdES Manipulation Library
The pyasice
library is designed to:
- create, read, and verify XAdES/XMLDsig/eIDAS electronic signatures,
- validate signers' certificates with OCSP,
- confirm these signatures with TimeStamping,
- create and manipulate ASiC-E or BDoc 2.1 containers, which are based on the XAdES/eIDAS stack.
Contents
- Quickstart
- Normative References
- Module Layout
- Technology Stack
- Build the XAdES XML Signature meta-file
- Secondary Services
Quickstart
ASiC-E/BDOC Container File Manipulation
Create a new container:
from pyasice import Container, XmlSignature
xmlsig = XmlSignature.create().add_document('test.txt', b'Test data', 'application/pdf')
# ... here goes the signing, confirming and timestamping part ...
container = Container()
container\
.add_file('test.txt', b'Test data', 'application/pdf')\
.add_signature(xmlsig)\
.save('test.asice')
# container is a context manager:
with Container() as container:
container.add_file('a', b'b', 'c').save('path/to')
# Open an existing container:
container = Container.open('test.asice')
# Verify container. Raises pyasice.SignatureVerificationError on failure
container.verify_signatures()
# Read files in the container
with container.open_file('test.txt') as f:
assert f.read() == b'Test data'
# Iterate over signatures
for xmlsig in container.iter_signatures():
xmlsig.get_signing_time()
Signing Flow Utilities
from pyasice import Container, finalize_signature
# get this from an external service, ID card, or elsewhere
user_certificate = b'user certificate in DER/PEM format'
container = Container()
container.add_file("test.txt", b'Test', "text/plain")
xml_sig = container.prepare_signature(user_certificate)
# Use an external service, or ID card, or a private key from elsewhere
# to sign the XML signature structure
signature_value = externally.sign(xml_sig.signed_data())
xml_sig.set_signature_value(signature_value)
# Get issuer certificate from the ID service provider, e.g. sk.ee.
# Here we use the user certificate's `issuer.common_name` field to identify the issuer cert,
# and find the cert in the `esteid-certificates` PyPI package.
issuer_cert_name = xml_sig.get_certificate_issuer_common_name()
import esteid_certificates
issuer_certificate = esteid_certificates.get_certificate(issuer_cert_name)
# Complete the XML signature with OCSP and optionally Timestamping
finalize_signature(xml_sig, ocsp_url="https://ocsp.server.url", tsa_url="https://tsa.server.url")
container.add_signature(xml_sig)
container.save("path/to/file.asice")
Normative References
The main document this library is based on: the BDOC 2.1.2 spec.
The specific standards outlined in that document:
- ETSI TS 101 903 v1.4.2 – XML Advanced Electronic Signatures (XAdES) and its Baseline Profile ETSI TS 103 171;
- ITU-T Recommendation X.509;
- RFC 3161 – PKIX Time-Stamp protocol;
- RFC 6960 – Online Certificate Status Protocol;
- ETSI TS 102 918 v1.2.1 - Associated Signature Containers (ASiC) and its Baseline Profile ETSI TS 103 174.
The difference between ASiC-E and BDOC is almost exclusively in terminology.
The BDOC 2.1.2 spec states:
The BDOC file format is based on ASiC standard which is in turn profiled by ASiC BP. BDOC packaging is a ASiC-E XAdES type ZIP container ...
So with a moderate risk of confusion, we can accept that ASiC-E and BDOC refer to the same thing.
Module Layout
- container.py -- the
Container
class, that deals with ASiC-E (BDOC v.2.1) container format - xmlsig.py -- the
XmlSignature
class, that deals with XAdES/XMLDSig XML structures - ocsp.py -- the
OCSP
class that deals with OCSP requests and responses - tsa.py -- the
TSA
class that deals with TimeStamping service requests and responses - signature_verifier.py -- the
verify
function, to verify signatures against a certificate.
Technology Stack
Dealing with the subject involves, at least:
- public key cryptography (RSA, ECDSA);
- ASN.1 encoding;
- XML processing;
- Zip archives;
- and also requests to various services (obtaining signer's certificate and the signature, validating the certificate through OCSP, time-stamping the signature).
The asn1crypto library and its higher-level complement oscrypto allow handling certificates and ASN.1 structures quite easily.
The cryptography library is by far the most powerful python library for dealing with public key cryptography algorithms.
Build the XAdES XML Signature meta-file
The structure of the XAdES XML signature file looks like this:
<asic:XAdESSignatures xmlns:asic="http://uri.etsi.org/02918/v1.2.1#" xmlns:ds="http://www.w3.org/2000/09/xmldsig#"
xmlns:xades="http://uri.etsi.org/01903/v1.3.2#">
<ds:Signature Id="S0">
<ds:SignedInfo Id="S0-SignedInfo">...</ds:SignedInfo>
<ds:SignatureValue Id="S0-SIG">...</ds:SignatureValue>
<ds:KeyInfo Id="S0-KeyInfo">...</ds:KeyInfo>
<ds:Object Id="S0-object-xades">
<xades:QualifyingProperties Id="S0-QualifyingProperties" Target="#S0">
<xades:SignedProperties Id="S0-SignedProperties">
<xades:SignedSignatureProperties Id="S0-SignedSignatureProperties">
<xades:SigningTime>2019-06-07T14:03:50Z</xades:SigningTime>
<xades:SigningCertificate>...</xades:SigningCertificate>
<xades:SignaturePolicyIdentifer>...</xades:SignaturePolicyIdentifer>
</xades:SignedSignatureProperties>
</xades:SignedProperties>
</xades:QualifyingProperties>
</ds:Object>
</ds:Signature>
</asic:XAdESSignatures>
We'll go over each section below.
SignedInfo
The SignedInfo
node is the source of the data being signed. The XML content of the node, canonicalized
using the CanonicalizationMethod
as per the respective child node, is hashed using an algorithm defined in
the SignatureMethod
child node, and this hash is fed to a signing service (ID card, SmartID etc.)
<ds:SignedInfo Id="S0-SignedInfo">
<ds:CanonicalizationMethod Algorithm="http://www.w3.org/2006/12/xml-c14n11"></ds:CanonicalizationMethod>
<ds:SignatureMethod Algorithm="http://www.w3.org/2001/04/xmldsig-more#ecdsa-sha256"></ds:SignatureMethod>
<ds:Reference Id="S0-ref-0" URI="test.pdf">
<ds:DigestMethod Algorithm="http://www.w3.org/2001/04/xmlenc#sha256"></ds:DigestMethod>
<ds:DigestValue>...</ds:DigestValue>
</ds:Reference>
<ds:Reference Id="S0-ref-sp" Type="http://uri.etsi.org/01903#SignedProperties" URI="#S0-SignedProperties">
<ds:Transforms>
<ds:Transform Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#"/>
</ds:Transforms>
<ds:DigestMethod Algorithm="http://www.w3.org/2001/04/xmlenc#sha256"></ds:DigestMethod>
<ds:DigestValue>...</ds:DigestValue>
</ds:Reference>
</ds:SignedInfo>
The Reference
fields are different in purpose and formation.
The first Reference
field is about the signed document and as such, has an URI
attribute of the document's file name.
Its child DigestValue
element is the SHA256 hash of the document, it is, incidentally, the very hash that is sent to the SmartID API for signing.
The second Reference
is built on the basis of some fields defined later in the SignedProperties section.
Its child DigestValue
is calculated as a SHA256 hash of the canonicalized XML output of the SignedProperties
tag, after that one is formed:
The URI
attribute of this Reference
tag is the #
-prefixed Id
attribute of the SignedProperties
tag.
import base64
import hashlib
from lxml import etree
buf = etree.tostring(el, method='c14n', exclusive=True or False) # NOTE below
digest_value = base64.b64encode(hashlib.sha256(buf).digest())
(Assuming the el
here to be the XML <SignedProperties>
element)
Canonicalization
The exclusive
kwarg controls whether the namespace declarations of ancestor tags should be included in the resulting canonical representation, or excluded.
Whether to use exclusive=True
depends on the canonicalization tag's Algorithm
attribute:
http://www.w3.org/2001/10/xml-exc-c14n#
, usesexclusive=True
,- the two others, the required
http://www.w3.org/TR/2001/REC-xml-c14n-20010315
, orhttp://www.w3.org/2006/12/xml-c14n11
, are not exclusive.
The aforementioned <ds:CanonicalizationMethod>
tag controls the c14n of the SignedInfo
node before feeding its digest to the signature service.
The c14n of SignedProperties
prior to getting its digest is determined by the ds:Transform
tag within this ds:Reference
node.
If it's not present, then the default, ie. not exclusive, c14n is used.
KeyInfo
This section contains the base64-encoded user certificate value, e.g. the SmartID API response's cert.value
,
or the certificate obtained from an ID card:
<ds:KeyInfo Id="S0-KeyInfo">
<ds:X509Data>
<ds:X509Certificate>MIIGJDCCBAygAwIBAgIQBNsLtTIpnmNbbE4+laSLaTANBgkqhkiG9w0BAQsFADBr...</ds:X509Certificate>
</ds:X509Data>
</ds:KeyInfo>
More details about the certificate in the SigningCertificate subsection.
SignedProperties
The XML section of SignedProperties
consists of, at least,
the SigningTime
, SigningCertificate
and SignaturePolicyIdentifer
elements.
:question: The signatures returned by e.g. Dokobit,
do not contain the SignaturePolicyIdentifer
node.
SigningTime
A timestamp in ISO 8601 format.
SignaturePolicyIdentifier
This appears to be a static^1 XML chunk referencing the BDOC 2.1 Specifications document:
<xades:SignaturePolicyIdentifier>
<xades:SignaturePolicyId>
<xades:SigPolicyId>
<xades:Identifier Qualifier="OIDAsURN">urn:oid:1.3.6.1.4.1.10015.1000.3.2.1</xades:Identifier>
</xades:SigPolicyId>
<xades:SigPolicyHash>
<ds:DigestMethod Algorithm="http://www.w3.org/2001/04/xmlenc#sha256">
</ds:DigestMethod>
<ds:DigestValue>3Tl1oILSvOAWomdI9VeWV6IA/32eSXRUri9kPEz1IVs=</ds:DigestValue>
</xades:SigPolicyHash>
<xades:SigPolicyQualifiers>
<xades:SigPolicyQualifier>
<xades:SPURI>https://www.sk.ee/repository/bdoc-spec21.pdf</xades:SPURI>
</xades:SigPolicyQualifier>
</xades:SigPolicyQualifiers>
</xades:SignaturePolicyId>
</xades:SignaturePolicyIdentifier>
[1] The DigestValue is the hash value of the document referenced by SPURI
, encoded in base64.
Refer to BDOC 2.1:2014 Specification for more information.
SigningCertificate
The user certificate is a base64-encoded DER certificate which can be loaded as follows:
import base64
from cryptography import x509
from cryptography.hazmat.backends import default_backend
cert_asn1 = base64.b64decode(cert_value)
cert = x509.load_der_x509_certificate(base64.b64decode(cert_asn1), default_backend())
or with pyopenssl
:
import base64
from OpenSSL.crypto import load_certificate, FILETYPE_ASN1
cert_asn1 = base64.b64decode(cert_value)
openssl_cert = load_certificate(FILETYPE_ASN1, base64.b64decode(cert_asn1))
These objects expose a slightly different but similar API.
What we need is the issuer name and certificate serial number:
assert openssl_cert.get_serial_number() == cert.sertial_number == '6454262457486410408874311107672836969'
assert cert.issuer.rfc4514_string() == 'C=EE,O=AS Sertifitseerimiskeskus,2.5.4.97=NTREE-10747013,CN=TEST of ESTEID-SK 2015'
assert openssl_cert.issuer.get_components() == [(b'C', b'EE'), (b'O', b'AS Sertifitseerimiskeskus'), (b'organizationIdentifier', b'NTREE-10747013'), (b'CN', b'ESTEID-SK 2015')]
Also we need a SHA256 digest value of the certificate:
cert_digest = base64.b64encode(hashlib.sha256(cert_asn1).digest())
With these values we can build the certificate information entry of the SignedProperties:
<xades:SigningCertificate>
<xades:Cert>
<xades:CertDigest>
<ds:DigestMethod Algorithm="http://www.w3.org/2001/04/xmlenc#sha256"></ds:DigestMethod>
<ds:DigestValue>hdsLTm4aaFKaGMwF6fvH5vWmiMBBnTCH3kba+TjY+pE=</ds:DigestValue>
</xades:CertDigest>
<xades:IssuerSerial>
<ds:X509IssuerName>C=EE,O=AS Sertifitseerimiskeskus,2.5.4.97=NTREE-10747013,CN=TEST of EID-SK 2016</ds:X509IssuerName>
<ds:X509SerialNumber>98652662091042892833248946646759285960</ds:X509SerialNumber>
</xades:IssuerSerial>
</xades:Cert>
</xades:SigningCertificate>
:question: Does X509IssuerName
content need to be a cert.issuer.rfc4514_string()
or can it be anything else?
So, in the end, we get a <xades:SignedProperties>
element which we then canonicalize and calculate a sha256 hash of this string,
to place it in the appropriate <ds:Reference>
element.
SignatureValue
<ds:SignatureValue Id="SIG-{SIGNATURE_ID}"><!-- Base64-encoded SIGNATURE_VALUE, gotten externally --></ds:SignatureValue>
A base64-encoded value of the signature calculated over the signed data.
The signed data is the ds:SignedInfo
section, as described above.
When using SmartID/MobileID, this is taken from the signature.value
field of the response.
KeyInfo
Contains the base64-encoded certificate, as gotten from the SmartID response.
<ds:KeyInfo Id="S0-KeyInfo">
<ds:X509Data>
<ds:X509Certificate>...</ds:X509Certificate>
</ds:X509Data>
</ds:KeyInfo>
Secondary Services
OCSP
OCSP (Online Certificate Status Protocol)
is designed to check that the signing certificate is valid at the point of signing. It is a binary protocol, and uses ASN.1 encoding in both request and response payload.
To deal with it, we're using the asn1crypto
library.
The OCSP request should be made immediately after signing, and the base64-encoded response is embedded in the XAdES signature as a xades:UnsignedSignatureProperties
descendant node,
namely xades:EncapsulatedOCSPValue
.
Estonian eID
URLs for OCSP services:
- Demo:
http://demo.sk.ee/ocsp
- Production:
http://ocsp.sk.ee/
More detail on the sk.ee OCSP page
Timestamping service
The TimeStamp protocol is also a binary protocol, for getting a Long-Term Validity Timestamp for a signature.
Also handled with the help of the asn1crypto
library.
The TSA request should be made immediately after OCSP validity confirmation, and the base64-encoded response is embedded in the XAdES signature as a xades:UnsignedSignatureProperties
descendant node,
namely xades:EncapsulatedTimeStamp
.
Estonian eID
URLs for timestamping services:
- Demo:
http://demo.sk.ee/tsa/
- Production:
http://tsa.sk.ee
More detail on the sk.ee TSA page
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.