IT: XML Signature

XML Signature (also called XMLDSig, XML-DSig, XML-Sig) defines an XML syntax for digital signatures and is defined in the W3C recommendation XML Signature Syntax and Processing. Functionally, it has much in common with PKCS#7 but is more extensible and geared towards signing XML documents. It is used by various Web technologies such as SOAP, SAML, and others.

XML signatures can be used to sign data–a resource–of any type, typically XML documents, but anything that is accessible via a URL can be signed. An XML signature used to sign a resource outside its containing XML document is called a detached signature; if it is used to sign some part of its containing document, it is called an enveloped signature; if it contains the signed data within itself it is called an enveloping signature.

?means the preceding item appears 0 or 1 times.
+means the preceding item appears 1 or more times.
*means the preceding item appears 0 or more times.

Example:
<Signature xmlns="http://www.w3.org/2000/09/xmldsig#">
<SignedInfo>
<CanonicalizationMethod Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#"/>
<SignatureMethod Algorithm="http://www.w3.org/2000/09/xmldsig#rsa-sha1"/>
<Reference URI="">
<Transforms>
<Transform Algorithm="http://www.w3.org/2000/09/xmldsig#enveloped-signature"/>
</Transforms>
<DigestMethod Algorithm="http://www.w3.org/2000/09/xmldsig#sha1"/>
<DigestValue>...</DigestValue>
</Reference>
</SignedInfo>
<SignatureValue>...</SignatureValue>
<KeyInfo>
<X509Data>
<X509SubjectName>...</X509SubjectName>
<X509Certificate>...................</X509Certificate>
</X509Data>
</KeyInfo>
</Signature>

<Seal xmlns="http://resource">
<ReceivedTime xmlns="http://resource">2017-12-12+02:00</ReceivedTime>
<SignaturesOK xmlns="http://resource">true</SignaturesOK>
</Seal>

Soap signature:

<env:Envelope
xmlns:env="http://schemas.xmlsoap.org/soap/envelope/">
<env:Header>
<SOAP-SEC:Signature
xmlns:SOAP-SEC="http://schemas.xmlsoap.org/soap/security/2000-12"env:actor="http://foo.example/bar"env:mustUnderstand="1">
<ds:Signature
xmlns:ds="http://www.w3.org/2000/09/xmldsig#">
<ds:SignedInfo>
<ds:CanonicalizationMethodAlgorithm="http://www.w3.org/TR/2000/CR-xml-c14n-20001026"/>
<ds:SignatureMethodAlgorithm="http://www.w3.org/2000/09/xmldsig#dsa-sha1"/>
<ds:Reference URI="#Body">
<ds:Transforms>
<ds:TransformAlgorithm="http://www.w3.org/TR/2000/CR-xml-c14n-20001026"/>
</ds:Transforms>
<ds:DigestMethodAlgorithm="http://www.w3.org/2000/09/xmldsig#sha1"/>
<ds:DigestValue>j6lwx3rvEPO0vKtMup4NbeVu8nk=</ds:DigestValue>
</ds:Reference>
</ds:SignedInfo>
<ds:SignatureValue>rMzv774yMoCLi8kHB23q6rva2hs5OreamqFwcFDg4AaXl0qhoYQGBg==</ds:SignatureValue>
</ds:Signature>
</SOAP-SEC:Signature>
</env:Header>
<env:BodySOAP-SEC:id="Body"
xmlns:SOAP-SEC="http://schemas.xmlsoap.org/soap/security/2000-12">
<m:GetLastTradePrice
xmlns:m="http://example.com/trade">
<m:symbol>EXAMPLE</m:symbol>
</m:GetLastTradePrice>
</env:Body>
</env:Envelope>

KeyInfo example:

<X509IssuerName>CN=TAMURA Kent, OU=TRL,O=IBM, L=Yamato-shi, ST=Kanagawa,C=JP</X509IssuerName>

</X509IssuerSerial>

</X509Data>

<X509SubjectName>Subject of Certificate B</X509SubjectName>

</X509Data>

<X509Certificate>MIICXTCCA..</X509Certificate>

<X509Certificate>MIICPzCCA...</X509Certificate>

<X509Certificate>MIICSTCCA...</X509Certificate>

</X509Data>

</KeyInfo>

Reference elements identify the data that is digested. Each Reference element identifies the data via a URI. In this example, the value of the URI is the empty String (""), which indicates the root of the document.

X509SubjectName element, which contains an X.509 subject distinguished name that should comply with [RFC 2253]

X509Certificate element, which contains a base-64 encoded X509 V.3 certificate

Info about private key:
$ openssl x509 -in Visma_MinaMeddelanden_SignCertTest.pem -text -noout

From Wikipedia:

The SignedInfo element contains or references the signed data and specifies what algorithms are used.

The SignatureMethod and CanonicalizationMethod elements are used by the SignatureValue element and are included in SignedInfo to protect them from tampering.
One or more Reference elements specify the resource being signed by URI reference; and any transforms to be applied to the resource prior to signing. A transformation can be a XPath-expression that selects a defined subset of the document tree.[1]

DigestMethod specifies the hash algorithm before applying the hash.
DigestValue contains the Base64 encoded result of applying the hash algorithm to the transformed resource(s) defined in the Reference element attributes.

The SignatureValue element contains the Base64 encoded signature result - the signature generated with the parameters specified in the SignatureMethod element - of the SignedInfo element after applying the algorithm specified by the CanonicalizationMethod.
KeyInfo element optionally allows the signer to provide recipients with the key that validates the signature, usually in the form of one or more X.509 digital certificates. The relying party must identify the key from context if KeyInfo is not present.
The Object element (optional) contains the signed data if this is an enveloping signature.

Validation and security considerations
When validating an XML Signature, a procedure called Core Validation is followed.

Reference Validation: Each Reference's digest is verified by retrieving the corresponding resource and applying any transforms and then the specified digest method to it. The result is compared to the recorded DigestValue; if they do not match, validation fails.
Signature Validation: The SignedInfo element is serialized using the canonicalization method specified in CanonicalizationMethod, the key data is retrieved using KeyInfo or by other means, and the signature is verified using the method specified in SignatureMethod. Signature Validation: The SignedInfo element is serialized using the canonicalization method specified in CanonicalizationMethod, the key data is retrieved using KeyInfo or by other means, and the signature is verified using the method specified in SignatureMethod.

Fields explained

The <DigestMethod> element identifies the algorithm used to calculate the digest.
[s09-10] DigestMethod is the algorithm applied to the data after Transforms is applied (if specified) to yield the DigestValue. The signing of the DigestValue is what binds a resources content to the signer's key.

DigestMethod is a required element that identifies the digest algorithm to be applied to the signed object. This element uses the general structure here for algorithms specified in Algorithm Identifiers and Implementation Requirements (section 6.1).

If the result of the URI dereference and application of Transforms is an XPath node-set (or sufficiently functional replacement implemented by the application) then it must be converted as described in the Reference Processing Model (section 4.3.3.2). If the result of URI dereference and application of transforms is an octet stream, then no conversion occurs (comments might be present if the Canonical XML with Comments was specified in the Transforms). The digest algorithm is applied to the data octets of the resulting octet stream.

The <DigestValue>:

This value is the resulting hash derived from encrypting the contents of the message (be it the assertion or the entire response, as per your configuration).

DigestValue is an element that contains the encoded value of the digest. The digest is always encoded using base64 [MIME].

One way to make DigestValue:
$ openssl dgst -binary -sha1 some_file.txt | openssl enc -base64

The <CanonicalizationMethod> element indicates the algorithm was used to canonize the <SignedInfo> element.
CanonicalizationMethod is a required element that specifies the canonicalization algorithm applied to the SignedInfo element prior to performing signature calculations. This element uses the general structure for algorithms described in Algorithm Identifiers and Implementation Requirements (section 6.1). Implementations MUST support the REQUIRED canonicalization algorithms.

The <SignatureMethod> element identifies the algorithm used to produce the signature value.
The ds:SignedInfo/ds:SignatureMethod element specifies what type of signature—e.g., Kerberos or RSA—is used to create the signature. Taken together, these two elements tell us how to create the digest, and how to protect it from modification.

<SignatureValue>
Another method to protect the digest is to use public-key cryptography, such as RSA. In public-key cryptography, there are two keys, a private key, known only to the holder, and a public key, accessible to anyone who wants to communicate with the key holder. In public-key cryptography, anything encrypted with the private key can be decrypted with the public key, and vice versa.

<Transforms>
One or more Reference elements specify the resource being signed by URI reference; and any transforms to be applied to the resource prior to signing. A transformation can be a XPath-expression that selects a defined subset of the document tree.[1]

SignatureValue value = Encrypt hash DigestValue with RSA private key and base64result

X509Certificate value = is certificate, that holds RSA public key

Process

1. Determine which resources are to be signed.
This will take the form of identifying the resources through a Uniform Resource Identifier (URI).
<Reference URI=""> part.

2. Calculate the digest of each resource. DigestValue

In XML signatures, each referenced resource is specified through a <Reference> element and its digest (calculated on the identified resource and not the <Reference> element itself) is placed in a <DigestValue> child element like

3. Collect the Reference elements

Collect the <Reference> elements (with their associated digests) within a <SignedInfo> element like

The <CanonicalizationMethod> element indicates the algorithm was used to canonize the <SignedInfo> element. Different data streams with the same XML information set may have different textual representations, e.g. differing as to whitespace. To help prevent inaccurate verification results, XML information sets must first be canonized before extracting their bit representation for signature processing. The <SignatureMethod> element identifies the algorithm used to produce the signature value.

4. Signing

Calculate the digest of the <SignedInfo> element, sign that digest and put the signature value in a <SignatureValue> element.

5. Add key information

If keying information is to be included, place it in a <KeyInfo> element. Here the keying information contains the X.509 certificate for the sender, which would include the public key needed for signature verification.

6. Enclose in a Signature element

Place the <SignedInfo>, <SignatureValue>, and <KeyInfo> elements into a <Signature> element. The <Signature> element comprises the XML signature.

Verifying an XML Signature
A brief description of how to verify an XML signature:

1. Verify the signature of the <SignedInfo> element. To do so, recalculate the digest of the <SignedInfo> element (using the digest algorithm specified in the <SignatureMethod> element) and use the public verification key to verify that the value of the <SignatureValue> element is correct for the digest of the <SignedInfo> element.

2. If this step passes, recalculate the digests of the references contained within the <SignedInfo> element and compare them to the digest values expressed in each <Reference> element's corresponding <DigestValue> element.

XML Signature: How to calculate the digest value?

I came across this question when attempting to find out the exact same thing. I later worked out how to do it, so figured I'd post the answer here.

The things that need to happen are:

canonicalization

create digest value, typically SHA1 (but could be SHA256 amongst others)

base64 encode it

The canonicalization part was fairly simple, as the Java libraries did that for me. What I struggled with was the next bit, the creating of the digest, because I made a fatal error in that the SHA1 digest I generated was the SHA1 in HEX form. SHA1 is 160 bits, so 20 bytes, but if you output these 160 bits in HEX, you get 40 characters. If you then base64 encode that, you get totally the wrong value compared to what should be in the DigestValue.

Instead, you should generate the SHA1 digest and base64 encode the 20 byte output. Don't try to output the 20 bytes to STDOUT as it's highly unlikely to be readable (which is why people often output the HEX equivalent, since it is readable). Instead, just base64 encode the 20 bytes and that's your DigestValue.

Dictionary:

Canonicalization:
In computer science, canonicalization (sometimes standardization or normalization) is a process for converting data that has more than one possible representation into a "standard", "normal", or canonical form.
A Canonical XML document:
By definition an XML document that is in XML Canonical form, defined by The Canonical XML specification. Briefly, canonicalization removes whitespace within tags, uses particular character encodings, sorts namespace references and eliminates redundant ones, removes XML and DOCTYPE declarations, and transforms relative URIs into absolute URIs.
Message digest:
Are functions that convert sequences of bits, possibly quite long, called messages, into fixed-length binary "fingerprints" or message digests of the original sequences.
Digital Signatures:
You can use public key authentication to produce "digital signatures." These signatures have a very desirable characteristic—namely, it is computationally infeasible for anyone without the private key to produce a signature that will verify for a given message. Modern digital signatures consist of (1) a message and a message digest of that message asymmetrically transformed under a private key of the signer.
A Message Authentication Code (MAC)
MACs are syntactically identical to signatures but imply a shared secret key.
MAC and signature algorithms are syntactically identical, but a signature implies public key cryptography.

Acronyms:
ASP Active Server Pages
API Application Programming Interface
BOM Byte Order Mark
CR Carriage Return
CSS Cascading Style Sheets
COM Component Object Model
UTC Coordinated Universal Time
DOM Document Object Model
DTD Document Type Definition
XML Extensible Markup Language
XSL Extensible Stylesheet Language
HTML HyperText Markup Language
HTTP HyperText Transfer Protocol
IDL Interface Definition Language
ISO International Standards Organization
IETF Internet Engineering Task Force
JDK Java Development Kit
JSP Java Server Pages
LF Line Feed
MIME Multipurpose Internet Mail Extensions
NCName Non-colonized Name
OMG Object Management Group
QName Qualified Name
RPC Remote Procedure Call
RFC Request For Comments
SAX Simple API for XML
SOAP Simple Object Access Protocol
SQL Strutured Query Language
UTF Unicode Transformation Format
UML Unified Modelling Language
URI Uniform Resource Identifier
URL Uniform Resource Locator
URN Uniform Resource Name
UCS Universal Character Set
VB Visual Basic
WD Working Draft
W3C World Wide Web Consortium
XInclude XML Inclusions
Infoset XML Information Set
XLink XML Linking Language
XPointer XML Pointer Language
XSLT XSL Transformations

Python library for signing;

xmlsec - documentation.
pyXMLSec - Documnetation

Resources:

https://www.xml.com/pub/a/2001/08/08/xmldsig.html

https://www.w3.org/TR/xmldsig-core/

https://www.youtube.com/watch?v=UYQPkWDaHHM - video apie XML enveloped Signature

https://docs.oracle.com/cd/E27515_01/common/tutorials/content_sign_message.html

https://www.w3.org/TR/xmldsig-core/
https://msdn.microsoft.com/en-us/library/ms996502.aspx - Digital Signature Cryptography
https://stackoverflow.com/questions/24961073/is-that-what-exactly-does-the-enveloped-signature-transform - About enveloped signature transform
https://www.di-mgt.com.au/xmldsig2.html - Comprehensive tutorial
https://www.decalage.info/en/python/xmldsig - python libraries
https://www.di-mgt.com.au/xmldsig.html - detailed signing process