NAV Navbar
  • Verifying Signed Event Notifications
  • Verifying Signed Event Notifications

    This guide walks you through the process of verifying that event notifications you receive at a webhook URL are sent from the Form3 platform. It covers the lookup of public keys using the Form3 API as well as the verification process using the obtained public key.

    If signature validation fails, or you receive a webhook without a signature from us you should reject the message and contact Form3 immediately.

    To learn how to subscribe to and receive event notifications in general, see our tutorial about the topic. If you're looking for guidance on how to sign your own requests to the Form3 API, see this tutorial.

    Form3 only signs event notifications sent to webhook URLs. If you receive event notifications on an Amazon SQS queue, IAM account control already ensures the origin of the notifications.

    Overview

    Form3 uses cryptographic signatures to ensure that event notification messages can be identified as coming from Form3. We sign the HTTP messages of the notification by adding a signature to the request header. This string is generated using the private key component of an RSA key pair.

    You can obtain the public component of the RSA key pair via the Form3 API and use it to verify the signature string. A successful validation of signature string and the header of the HTTP message verify that the notification was created by Form3.

    The Form3 API uses request signing as defined by the Network Working Group in the Signing HTTP Requests draft using the RSA-SHA256 algorithm. The private and public key are generated following the PKCS #8 standard in PEM base64 format.

    Retrieving and Storing Public Keys

    Each event notification message contains a key ID in its header. This ID identifies which public key you need to use to verify the legitimacy of the message. Form3 provides access to all valid public keys via its API.

    To look up a public key, use the Fetch call of the Signing Keys endpoint: GET v1/platform/security/signing_keys/{signing_key_id}

    Fetched public keys should be stored on your system together with their key IDs to limit repeating lookup calls to the Form3 API and speed up the verification process. We recommend that when you receive an event notification, you first check your system for the public key using the key ID and only if you can't find the key on your system perform a lookup via the Form3 API

    What to Do When Encountering an Invalid Notification

    A notification is considered invalid in the following cases:

    If any of these cases occurs, contact Form3 immediately.

    Step by Step Guide

    In this example, we will receive a signed event notification and verify it using the public key available from the Form3 API. In addition to the steps, code examples are provided to simplify implementation.

    The example will take you through the following steps:

    Prerequisites

    Before getting started, make sure you have the following things ready to follow along:

    Receive an Event Notification

    To validate an event notification, we first need to receive one. In this example, we do this in Form3's staging environment by subscribing to updated events of Payment Submission resources and then sending an outbound payment to trigger these events.

    First, create a Subscription resource. Use the following key parameters:

    For more information on how to subscribe to events, see our Subscription tutorial.

    Then, submit a payment by creating a Payment resource and a Payment Submission resource. See our tutorial on sending payments for more information.

    As soon as the payment is processed by Form3, you will receive event notifications at your endpoint.

    Get the Notification

    The headers section of the event notifications arriving at your callback endpoint will look similar to this:

    Notification header

    x-form3-signature: Signature keyId="6e6431da-0b00-480c-8ff5-388d29a6d42c",algorithm="rsa-sha256",headers="(request-target) host date content-type digest content-length", signature="eQHE3mFN7BGuqSbkwLuUuOi4VSVDn0lSAlxQC5M606APxk5rXoDsPT0xRzVJObtnT4rWcmCk3GviRuvHmTvd3q5ttSBz6VlMo8VrxdsOcXXgQ45zu63v4bgS4MtjACMJipS9ypcpz4VF8lkvj7dqwDyYzt/p1hrLJ81w8mh0CQHWam3T2mecigH63UUZdxFG/C4dFHz+WiN1qkz2FwU6QyN4AMvWZ598c15PH8Q8Xr5WnrTTNDM9pV1a5B7WCKW0cHdL+iPgM9Qp3LFKh3CYwRZ5P+vHDe/sao2GnzWLYU66aVeiDpv3rdFHAWB1/DHjyJXMjKK2x2iU2MDMEkIsq9IIQCXagp8/y2MLvNHcHAMLxSsN029spUTLpYbL3ravj8eTO3kgaq4s3fJd0EvyZ91E1t0KPMIDdGwN/8GqnPrHg3FL2sKhLkd3T9H1Qz4DLUsdvlVJmzdRzFX9FY2KNCQ7TZVS7nlQUTU2qgDB+1nl8QMSimV/Scj1gd+SqrvdS1c4VX6RQVVEmAbHN8mvHATLtqLtD4SGWhtfnJBExo+5EbqQflpD0M6Jrn0k/vJdjprqFIAgMbwUoSaHiknHwOkM5huU+BaCGySGjNsPBy7jGV/6prjGwHrNv9c3sINhTCPKMiumfCfdRstJ1vAAW2ABqEI95+gCS/3Lqr4mUK4="
    digest: TJ64Q13Shxp68FaCxT27itpEuCscxlfC7+G5E1kLuhc=
    date: Thu, 25 Jun 2020 12:39:13 UTC
    content-type: application/json
    content-length: 1471
    host: webhook.site
    


    The x-form3-signature header contains several important values. The signature value contains a base-64 representation of Form3's signature for this message. In preparation for validating it later, we store it in a file called signature.txt.

    The keyId identifies which public key can be used to verify the signature . In this example, the key ID is 6e6431da-0b00-480c-8ff5-388d29a6d42c.

    The headers value lists the message headers needed to build the signature string, see the Build the Signature String section below.

    We also store the body of the notification in a file called body.txt. Note that this is the raw body content without linebreaks or formatting spaces. It should look similar to this:

    body.txt

    {"id":"d91613dd-0d3c-467a-9f69-e6230a5d7820","organisation_id":"743d5b63-8e6f-432e-a8fa-c5d8d2ee5fcb","event_type":"updated","record_type":"payment_submissions","data_record_type":"PaymentSubmission","data":{"data":{"type":"payment_submissions","id":"918f4a74-0de0-47ba-8a63-94da111b7bd2","version":1,"organisation_id":"743d5b63-8e6f-432e-a8fa-c5d8d2ee5fcb","created_on":"2020-06-25T12:39:13.522Z","modified_on":"2020-06-25T12:39:13.621Z","attributes":{"status":"validation_pending","submission_datetime":"2020-06-25T12:39:13.511Z","transaction_start_datetime":"2020-06-25T12:39:13.477Z"},"relationships":{"payment":{"data":[{"type":"payments","id":"18d9b52d-5f7b-4328-88a7-1c0c8c88b42e","version":0,"organisation_id":"743d5b63-8e6f-432e-a8fa-c5d8d2ee5fcb","created_on":"2020-06-25T12:39:08.134Z","modified_on":"2020-06-25T12:39:08.134Z","attributes":{"amount":"14.00","beneficiary_party":{"account_name":"Mrs Receiving Test","account_number":"71268996","account_number_code":"BBAN","account_type":1,"account_with":{"bank_id":"400302","bank_id_code":"GBDSC"},"country":"GB"},"currency":"GBP","debtor_party":{"account_name":"Mr Sending Test","account_number":"62727606","account_number_code":"BBAN","account_with":{"bank_id":"040059","bank_id_code":"GBDSC"},"country":"GB"},"numeric_reference":"0001","scheme_transaction_id":"85475449881836850","payment_scheme":"FPS","processing_date":"2020-06-25","reference":"1112223330","scheme_payment_type":"ImmediatePayment"}}]}}}}}
    

    Fetch the Public Key

    You can now fetch the public key from the Form3 API using the key ID. To do this, make a Fetch call to the Signing Keys endpoint: GET v1/platform/security/signing_keys/{key_id}.

    In the call above, replace key_id with the value of keyId from the notification header, in our example that's 6e6431da-0b00-480c-8ff5-388d29a6d42c.

    The Form3 API will respond with a Signing Keys resource like this:

    Signing Key resource

    {
        "data": {
            "type": "signing_keys",
            "id": "6e6431da-0b00-480c-8ff5-388d29a6d42c",
            "version": 0,
            "organisation_id": "170028f8-b78b-4920-bbd5-825d504737a5",
            "created_on": "2020-06-16T08:23:47.662Z",
            "modified_on": "2020-06-16T08:23:47.662Z",
            "attributes": {
                "public_key": "-----BEGIN RSA PUBLIC KEY-----\nMIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEAvmhDVbzZVhKOUYrX/LAM\n/qMcDcWOq4fQpXQBdk8c670p6GkgynF/L7wN9CW21Ub2+YKOb8XDqWJrGxTpy1w9\n3nXD+xaWFhh2PkBqN+WASlog9rm9542SoOSrDiKcaUPnA+hBc+9ngtQkTbnhZOX1\nCReb1HLQXVP9OK0BE5NL2HvMPYWUcufUM0q4alrDlJTsEFE1QgdYQ5sSYFAvcQ26\nSDeoRe0CzX3dAbZVgGoupBhyH0EtylmygDiECjn4twz4/9YCHO8c3cDCMtYKjNoG\nslWT4wsqfqlx20Cxfm6oKY5H2vcBceZ7QcQhnEaRG/Ccj5/wLqlJcr8+DctVGDOD\nBdLB26k2vDrpGBJf7UdT4W+ZpDJo1oXHXQqUqSjU09Vk185nQvV0e1OSysL9hpeI\nmsoWDmwRvIhMkQ1AORuFfPpjuW/WRODj3khbQqukWWdaGDCbd3gKk4GuXEfGcINY\nubGl+wf39dPNeUBmvtRfDrm7FkZJcykQNcT+Jsyg9+yczoImniN67Ot4+yBoRyTk\nr8wUSMAhGRSSjU8UU1eF2PUpBSG1iPgJhyiHQj8ogv2asePURnqWFk4HxKomFVFW\nJPu4Xota6M9chqXw6gDXaWStCWZ8QhR5x5RxKVxfEkatkqq/0IYvgFdiDyF5yQ4f\n2eBXBvd+YJaelEl1CyplSukCAwEAAQ==\n-----END RSA PUBLIC KEY-----\n"
            }
        }
    }
    


    The public_key attribute of the resource contains the public RSA key you can use to verify the notification signature. We store the key in a file called key.pem.

    When doing so, note that you may need to replace the \n in the public_key attribute with actual newlines.

    If you're using the openssl command line tool to verify the key like we do in this example, you may need to remove the RSA from the beginning and end lines of the key since some versions of openssl stumble over this.

    The key.pem file then looks like this:

    key.pem

    -----BEGIN PUBLIC KEY-----
    MIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEA38w1qw2xf3vKzUkiCDka
    mfrSvC7H2wTsmhqxjxIZ5ZFqh0WYuCLuc8QdZqT8EMKi8AtHqGLiqPJaPG4UmkbY
    atHdXxgs33hkHEU20W5nTVkopmgtahkCX1Z8Ggl5IJl4S9HV21ROgr79Dil7xcMG
    Wrwmri1oEROSLsoydKa7vwt+9Og+u0uvc5JojUK5gj4aJ0J80z2z3jbmjJZg6yC7
    A0O6tOBWYLIuoUUB6ycwx0CswRBpJacAc1ME0Wz3zRdXNww2QBPncmqjCcxWXZZ9
    xS5pGv+TMqasuC5NXZ8Jx2RiULf1mb92JbBdAOucEUDFpHUZm1iaiVH1RkMjCGSm
    tJYDo2yYWGAVs52qlvDhnJrKl49MGU3sjKRFgtQBe1HcYsoPov2AubMn/pHZaD0y
    ntv6vxjh2ZF7IXkv6+ZtNHSiMar9o1wbWwIYNt0+AfkIl+QkkV2NvbOENd1/Lms+
    pW4VoHs4RQ4FwpghQxDtoOzuBxJfHuBYnqcChm1TZp2pdI2k3UfPVFa510uBWsgX
    aL+Q8EYYTWzkePweZsM4wq3Nuqg8GGHgNfMGWvtDNmHeZvWTgOFF9+kcMEHmEMAk
    LreZpKxhjD20wpoO83/But02bqb+b9lLkJgXSraK/palEe/G2e7hUOJDyJs3Pl64
    E38CxG28lQIjLJAx1DKFdVkCAwEAAQ==
    -----END PUBLIC KEY-----
    


    Build the Signature String

    In the next step, we will validate the signature string from the signature. To be able to do that, we first have to create the signature string that was used to generate the signature.

    The signature string is a subset of the notification header. You can identify the headers that form the signature string from the headers section of the x-form3-signature header. In the example notification header above, these are:

    While host, date and content-type are easily extracted from the notification header itself, the other headers need a bit more work to construct:

    1. (request-target) is the full path of the callback URI. In our example that is post /bb01ea78-88c2-4634-bfcf-807c26191a83.

    2. We recommend calculating the content-length to verify its correctness. The content length is simply the number of bytes in the message body. Note that this refers to the raw body of the message and does not include any linebreaks or whitespace for formatting. In this example, the content length is 1471.

    3. We also recommend calculating the digest header to verify the message body. You can use openssl to do that:

    echo -n "$(<body.txt)" | openssl dgst -sha256 -binary | openssl enc -base64 -A

    This command assumes that the body of the notification is stored in a file called body.txt. From the command output, create the digest header by adding SHA-256= in front of it: digest: SHA-256=TJ64Q13Shxp68FaCxT27itpEuCscxlfC7+G5E1kLuhc=

    With the steps above complete, you can create the complete signature string and store it in a file. In this example, we'll call that file signature_string.txt. The content will look similar to this:

    signature_string.txt

    (request-target): post /bb01ea78-88c2-4634-bfcf-807c26191a83
    host: webhook.site
    date: Thu, 25 Jun 2020 12:39:13 UTC
    content-type: application/json
    digest: SHA-256=TJ64Q13Shxp68FaCxT27itpEuCscxlfC7+G5E1kLuhc=
    content-length: 1471
    

     Verify the Notification

    Now use the public key to verify the signature string from notification. Using the openssl command line tool, this requires two steps:

    1. Decode the base-64 encoded signature into binary format: openssl enc -base64 -d -A -in signature.txt -out signature.txt.sha256
      This stores the binary signature in a file called signature.txt.sha256.

    2. Then, verify the signature: echo -n "$(<signature_string.txt)" | openssl dgst -binary -sha256 -verify key.pem -signature signature.txt.sha256

    If everything goes well, openssl will response with Verified OK.

    You can also use the code from the examples below to implement the verification process.

    Code Examples

    Code Example Golang

    This code example written in Golang and will only cover verifying a request's signature, it will not show you how to load the public key from storage if you have a copy already or Fetch a new keyt from the Form3 API

    package verification
    
    import (
        "bytes"
        "crypto"
        "crypto/rsa"
        "crypto/sha256"
        "encoding/base64"
        "fmt"
        "io/ioutil"
        "net/http"
        "regexp"
        "strings"
    )
    
    func VerifyRequestSignature(req *http.Request) error {
        authHeader := req.Header.Get("X-Form3-Signature")
    
        signatureRegex, err := regexp.Compile("signature=\\\"([^\\\"]*)")
        if err != nil {
            return err
        }
    
        keyRegex, err := regexp.Compile("keyId=\\\"([^\\\"]*)")
        if err != nil {
            return err
        }
    
        hasher := sha256.New()
    
        msgToHash:= fmt.Sprintf(`(request-target): %s %s
    host: %s
    date: %s`,
            strings.ToLower(req.Method),
            req.RequestURI,
            req.Host,
            req.Header.Get("Date"),
        )
    
        if (req.Method == http.MethodPost ||
            req.Method == http.MethodPut ||
            req.Method == http.MethodPatch) &&
            req.Body != nil {
          ReqMimeType := req.Header.Get("content-type")
    
          reqBody, err := ioutil.ReadAll(req.Body)
          if err != nil {
            return err
          }
    
          if _, err := hasher.Write(reqBody); err != nil {
            return err
          }
    
          digest := base64.StdEncoding.EncodeToString(hasher.Sum(nil))
    
          msgToHash += fmt.Sprintf(`
    content-type: %s
    digest: SHA-256=%s
    content-length: %d`,
          ReqMimeType,
          digest,
          req.ContentLength,
          )
        }
    
        hasher.Reset()
        if _, err := hasher.Write([]byte(msgToHash)); err != nil {
            return err
        }
        hashedMsg := hasher.Sum(nil)
    
        // Get the public key associated with the KeyID in the Authorization header
        pubKey, err := getPublicKeyById(strings.Split(keyRegex.FindString(authHeader), "=\"")[1])
        if err != nil {
            return err
        }
    
        signature := strings.Split(signatureRegex.FindString(authHeader), "=\"")[1]
        decodedSig , err := base64.StdEncoding.DecodeString(signature)
        if err != nil {
            return err
        }
    
        if err := rsa.VerifyPKCS1v15(
            pubKey,
            crypto.SHA256,
            hashedMsg[:],
            decodedSig,
        ); err != nil {
            return err
        }
    }
    
    func getPublicKeyById(keyID string) (*rsa.PublicKey, error){
        // You need to fetch the key from storage and return a pointer to the rsa.PublicKey
        // If you do not have a copy of the specified key then request a copy of the key from Form3's API
    
    }
    

    Code Example Python

    This code example written in Python (version 3.8+) will only cover verifying a request's signature, it will not show you how to call the Form3 API to request a public key if you don't have it or how to load the public key from storage if you have a copy already.

    import re
    
    from Crypto import Signature
    from Crypto.PublicKey import RSA
    from Crypto.Signature import PKCS1_v1_5
    from Crypto.Hash import SHA256
    from base64 import b64decode, b64encode
    from typing import str, bool
    
    def fetch_key(key_id: str) -> str:
        # You need to fetch the key from storage and return the string representation of the public key
        # If you do not have a copy of the specified key then request a copy of the key from the Form3 API
        return ""
    
    def validate_request(req) -> bool:
        data = '''(request-target): {} {}
    host: {}
    date: {}'''.format(req.method.lower(), req.headers.environ["REQUEST_URI"], req.host, req.headers["Date"])
    
        if req.method == "POST":
            # Write the digest of the body
            hasher = SHA256.new()
            hasher.update(req.data)
            hashed = b64encode(hasher.digest()).decode("utf-8")
    
            data = data + '''
    content-type: {}
    digest: SHA-256={}
    content-length: {}'''.format(req.content_type, hashed, req.content_length)
    
        # Use the data we created above to verify the request
        auth_header = req.headers.get("X-Form3-Signature")
    
        key_matcher = re.compile('keyId=\\\"([^\\\"]*)')
        sig_matcher = re.compile('signature=\\\"([^\\\"]*)')
        request_signature = sig_matcher.findall(auth_header)[0]
    
        public_key = fetch_key(key_matcher.findall(auth_header)[0])
        rsa_key = RSA.importKey(public_key)
    
        sha_digest = SHA256.new()
        sha_digest.update(data.encode("utf-8"))
        verifier = Signature.PKCS1_v1_5.new(rsa_key)
    
        # returns true if valid, false otherwise
        return verifier.verify(sha_digest, b64decode(request_signature))
    

    Further Reading

    This tutorial explained how to validate that cryptographically signed event notifications are sent by Form3. For more information on request signing and subscriptions, see the following resources.