NAV Navbar
  • Verifying Signed Event Notifications
  • Verifying Signed Event Notifications

    This guide walks you through the process of verifying that event notifications you receive are sent from the Form3 platform. It covers the lookup of public keys using the Form3 API as well as the verification process using the obtained public key.

    If signature validation fails, or you receive a notification without a signature from us you should reject the message and contact Form3 immediately.

    To learn how to subscribe to and receive event notifications in general, see our tutorial about the topic. If you're looking for guidance on how to sign your own requests to the Form3 API, see this tutorial.

    Form3 signs event notifications sent to webhook URLs and Amazon SQS queues.

    Overview

    Form3 uses cryptographic signatures to ensure that event notification messages can be identified as coming from Form3.

    We sign notifications by adding a signature to the message:

    This string is generated using the private key component of an RSA key pair.

    You can obtain the public component of the RSA key pair via the Form3 API and use it to verify the signature string. A successful validation of signature string verifies that the notification was created by Form3.

    The Form3 API uses request signing as defined by the Network Working Group in the Signing HTTP Requests draft using the RSA-SHA256 algorithm. The private and public key are generated following the PKCS #8 standard in PEM base64 format.

    Retrieving and Storing Public Keys

    Each event notification message contains a key ID in its header. This ID identifies which public key you need to use to verify the legitimacy of the message. Form3 provides access to all valid public keys via its API.

    To look up a public key, use the Fetch call of the Signing Keys endpoint: GET v1/platform/security/signing_keys/{signing_key_id}

    Fetched public keys should be stored on your system together with their key IDs to limit repeating lookup calls to the Form3 API and speed up the verification process. We recommend that when you receive an event notification, you first check your system for the public key using the key ID and only if you can't find the key on your system perform a lookup via the Form3 API

    What to Do When Encountering an Invalid Notification

    A notification is considered invalid in the following cases:

    If any of these cases occurs, contact Form3 immediately.

    Step by Step Guide

    In this example, we will receive a signed event notification and verify it using the public key available from the Form3 API. In addition to the steps, code examples are provided to simplify implementation.

    The example will take you through the following steps:

    Prerequisites

    Before getting started, make sure you have the following things ready to follow along:

    Receive an Event Notification

    To validate an event notification, we first need to receive one. In this example, we do this in Form3's staging environment by subscribing to updated events of Payment Submission resources and then sending an outbound payment to trigger these events.

    First, create a Subscription resource. Use the following key parameters:

    For more information on how to subscribe to events, see our Subscription tutorial.

    Then, submit a payment by creating a Payment resource and a Payment Submission resource. See our tutorial on sending payments for more information.

    As soon as the payment is processed by Form3, you will receive event notifications at your endpoint.

    Get the Notification

    The event notifications arriving at your HTTP callback endpoint or your SQS queue will look similar to this:

    Notification headers or message attributes

    x-form3-signature: Signature keyId="6e6431da-0b00-480c-8ff5-388d29a6d42c",algorithm="rsa-sha256",headers="(request-target) host date content-type digest content-length", signature="eQHE3mFN7BGuqSbkwLuUuOi4VSVDn0lSAlxQC5M606APxk5rXoDsPT0xRzVJObtnT4rWcmCk3GviRuvHmTvd3q5ttSBz6VlMo8VrxdsOcXXgQ45zu63v4bgS4MtjACMJipS9ypcpz4VF8lkvj7dqwDyYzt/p1hrLJ81w8mh0CQHWam3T2mecigH63UUZdxFG/C4dFHz+WiN1qkz2FwU6QyN4AMvWZ598c15PH8Q8Xr5WnrTTNDM9pV1a5B7WCKW0cHdL+iPgM9Qp3LFKh3CYwRZ5P+vHDe/sao2GnzWLYU66aVeiDpv3rdFHAWB1/DHjyJXMjKK2x2iU2MDMEkIsq9IIQCXagp8/y2MLvNHcHAMLxSsN029spUTLpYbL3ravj8eTO3kgaq4s3fJd0EvyZ91E1t0KPMIDdGwN/8GqnPrHg3FL2sKhLkd3T9H1Qz4DLUsdvlVJmzdRzFX9FY2KNCQ7TZVS7nlQUTU2qgDB+1nl8QMSimV/Scj1gd+SqrvdS1c4VX6RQVVEmAbHN8mvHATLtqLtD4SGWhtfnJBExo+5EbqQflpD0M6Jrn0k/vJdjprqFIAgMbwUoSaHiknHwOkM5huU+BaCGySGjNsPBy7jGV/6prjGwHrNv9c3sINhTCPKMiumfCfdRstJ1vAAW2ABqEI95+gCS/3Lqr4mUK4="
    digest: TJ64Q13Shxp68FaCxT27itpEuCscxlfC7+G5E1kLuhc=
    date: Thu, 25 Jun 2020 12:39:13 UTC
    content-type: application/json
    content-length: 1471
    host: webhook.site
    

    In the case of SQS, ensure that you configure your client to fetch those message attributes as part of your receive request.


    The x-form3-signature header contains several important values. The signature value contains a base-64 representation of Form3's signature for this message. In preparation for validating it later, we store it in a file called signature.txt.

    The keyId identifies which public key can be used to verify the signature . In this example, the key ID is 6e6431da-0b00-480c-8ff5-388d29a6d42c.

    The headers value lists the message headers needed to build the signature string, see the Build the Signature String section below.

    We also store the body of the notification in a file called body.txt. Note that this is the raw body content without linebreaks or formatting spaces. It should look similar to this:

    body.txt

    {"id":"d91613dd-0d3c-467a-9f69-e6230a5d7820","organisation_id":"743d5b63-8e6f-432e-a8fa-c5d8d2ee5fcb","event_type":"updated","record_type":"payment_submissions","data_record_type":"PaymentSubmission","data":{"data":{"type":"payment_submissions","id":"918f4a74-0de0-47ba-8a63-94da111b7bd2","version":1,"organisation_id":"743d5b63-8e6f-432e-a8fa-c5d8d2ee5fcb","created_on":"2020-06-25T12:39:13.522Z","modified_on":"2020-06-25T12:39:13.621Z","attributes":{"status":"validation_pending","submission_datetime":"2020-06-25T12:39:13.511Z","transaction_start_datetime":"2020-06-25T12:39:13.477Z"},"relationships":{"payment":{"data":[{"type":"payments","id":"18d9b52d-5f7b-4328-88a7-1c0c8c88b42e","version":0,"organisation_id":"743d5b63-8e6f-432e-a8fa-c5d8d2ee5fcb","created_on":"2020-06-25T12:39:08.134Z","modified_on":"2020-06-25T12:39:08.134Z","attributes":{"amount":"14.00","beneficiary_party":{"account_name":"Mrs Receiving Test","account_number":"71268996","account_number_code":"BBAN","account_type":1,"account_with":{"bank_id":"400302","bank_id_code":"GBDSC"},"country":"GB"},"currency":"GBP","debtor_party":{"account_name":"Mr Sending Test","account_number":"62727606","account_number_code":"BBAN","account_with":{"bank_id":"040059","bank_id_code":"GBDSC"},"country":"GB"},"numeric_reference":"0001","scheme_transaction_id":"85475449881836850","payment_scheme":"FPS","processing_date":"2020-06-25","reference":"1112223330","scheme_payment_type":"ImmediatePayment"}}]}}}}}
    

    Fetch the Public Key

    You can now fetch the public key from the Form3 API using the key ID. To do this, make a Fetch call to the Signing Keys endpoint: GET v1/platform/security/signing_keys/{key_id}.

    In the call above, replace key_id with the value of keyId from the notification header, in our example that's 6e6431da-0b00-480c-8ff5-388d29a6d42c.

    The Form3 API will respond with a Signing Keys resource like this:

    Signing Key resource

    {
        "data": {
            "type": "signing_keys",
            "id": "6e6431da-0b00-480c-8ff5-388d29a6d42c",
            "version": 0,
            "organisation_id": "170028f8-b78b-4920-bbd5-825d504737a5",
            "created_on": "2020-06-16T08:23:47.662Z",
            "modified_on": "2020-06-16T08:23:47.662Z",
            "attributes": {
                "public_key": "-----BEGIN RSA PUBLIC KEY-----\nMIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEAvmhDVbzZVhKOUYrX/LAM\n/qMcDcWOq4fQpXQBdk8c670p6GkgynF/L7wN9CW21Ub2+YKOb8XDqWJrGxTpy1w9\n3nXD+xaWFhh2PkBqN+WASlog9rm9542SoOSrDiKcaUPnA+hBc+9ngtQkTbnhZOX1\nCReb1HLQXVP9OK0BE5NL2HvMPYWUcufUM0q4alrDlJTsEFE1QgdYQ5sSYFAvcQ26\nSDeoRe0CzX3dAbZVgGoupBhyH0EtylmygDiECjn4twz4/9YCHO8c3cDCMtYKjNoG\nslWT4wsqfqlx20Cxfm6oKY5H2vcBceZ7QcQhnEaRG/Ccj5/wLqlJcr8+DctVGDOD\nBdLB26k2vDrpGBJf7UdT4W+ZpDJo1oXHXQqUqSjU09Vk185nQvV0e1OSysL9hpeI\nmsoWDmwRvIhMkQ1AORuFfPpjuW/WRODj3khbQqukWWdaGDCbd3gKk4GuXEfGcINY\nubGl+wf39dPNeUBmvtRfDrm7FkZJcykQNcT+Jsyg9+yczoImniN67Ot4+yBoRyTk\nr8wUSMAhGRSSjU8UU1eF2PUpBSG1iPgJhyiHQj8ogv2asePURnqWFk4HxKomFVFW\nJPu4Xota6M9chqXw6gDXaWStCWZ8QhR5x5RxKVxfEkatkqq/0IYvgFdiDyF5yQ4f\n2eBXBvd+YJaelEl1CyplSukCAwEAAQ==\n-----END RSA PUBLIC KEY-----\n"
            }
        }
    }
    


    The public_key attribute of the resource contains the public RSA key you can use to verify the notification signature.

    Store the key in a file called key.pem. When doing so, ensure you make the following changes to the key:

    The key.pem file then looks like this:

    key.pem

    -----BEGIN PUBLIC KEY-----
    MIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEAvmhDVbzZVhKOUYrX/LAM
    /qMcDcWOq4fQpXQBdk8c670p6GkgynF/L7wN9CW21Ub2+YKOb8XDqWJrGxTpy1w9
    3nXD+xaWFhh2PkBqN+WASlog9rm9542SoOSrDiKcaUPnA+hBc+9ngtQkTbnhZOX1
    CReb1HLQXVP9OK0BE5NL2HvMPYWUcufUM0q4alrDlJTsEFE1QgdYQ5sSYFAvcQ26
    SDeoRe0CzX3dAbZVgGoupBhyH0EtylmygDiECjn4twz4/9YCHO8c3cDCMtYKjNoG
    slWT4wsqfqlx20Cxfm6oKY5H2vcBceZ7QcQhnEaRG/Ccj5/wLqlJcr8+DctVGDOD
    BdLB26k2vDrpGBJf7UdT4W+ZpDJo1oXHXQqUqSjU09Vk185nQvV0e1OSysL9hpeI
    msoWDmwRvIhMkQ1AORuFfPpjuW/WRODj3khbQqukWWdaGDCbd3gKk4GuXEfGcINY
    ubGl+wf39dPNeUBmvtRfDrm7FkZJcykQNcT+Jsyg9+yczoImniN67Ot4+yBoRyTk
    r8wUSMAhGRSSjU8UU1eF2PUpBSG1iPgJhyiHQj8ogv2asePURnqWFk4HxKomFVFW
    JPu4Xota6M9chqXw6gDXaWStCWZ8QhR5x5RxKVxfEkatkqq/0IYvgFdiDyF5yQ4f
    2eBXBvd+YJaelEl1CyplSukCAwEAAQ==
    -----END PUBLIC KEY-----
    


    Build the Signature String

    In the next step, we will validate the signature string from the signature. To be able to do that, we first have to create the signature string that was used to generate the signature.

    The signature string is a subset of the notification header. You can identify the headers that form the signature string from the headers section of the x-form3-signature header. In the example notification header above, these are:

    While host, date and content-type are easily extracted from the notification header itself, the other headers need a bit more work to construct:

    1. (request-target) is the full path of the callback URI. In our example that is post /bb01ea78-88c2-4634-bfcf-807c26191a83. Note the following requirements:
      • The HTTP verb must be lowercase, so post, not POST or Post.
      • The callback URI must remain unchanged, including all capital letters present in the original URI.

    2. We recommend calculating the content-length to verify its correctness. The content length is simply the number of bytes in the message body. Note that this refers to the raw body of the message and does not include any linebreaks or whitespace for formatting. In this example, the content length is 1471.

    3. We also recommend calculating the digest header to verify the message body. You can use openssl to do that:

    echo -n "$(<body.txt)" | openssl dgst -sha256 -binary | openssl enc -base64 -A

    This command assumes that the body of the notification is stored in a file called body.txt. From the command output, create the digest header by adding SHA-256= in front of it: digest: SHA-256=TJ64Q13Shxp68FaCxT27itpEuCscxlfC7+G5E1kLuhc=

    With the steps above complete, you can create the complete signature string and store it in a file. In this example, we'll call that file signature_string.txt. The content will look similar to this:

    signature_string.txt

    (request-target): post /bb01ea78-88c2-4634-bfcf-807c26191a83
    host: webhook.site
    date: Thu, 25 Jun 2020 12:39:13 UTC
    content-type: application/json
    digest: SHA-256=TJ64Q13Shxp68FaCxT27itpEuCscxlfC7+G5E1kLuhc=
    content-length: 1471
    

     Verify the Notification

    Now use the public key to verify the signature string from notification. Using the openssl command line tool, this requires two steps:

    1. Decode the base-64 encoded signature into binary format: openssl enc -base64 -d -A -in signature.txt -out signature.txt.sha256
      This stores the binary signature in a file called signature.txt.sha256.

    2. Then, verify the signature: echo -n "$(<signature_string.txt)" | openssl dgst -binary -sha256 -verify key.pem -signature signature.txt.sha256

    If everything goes well, openssl will response with Verified OK.

    You can also use the code from the examples below to implement the verification process.

    Code Examples

    Code Example Golang

    This code example written in Golang and will only cover verifying a request's signature, it will not show you how to load the public key from storage if you have a copy already or Fetch a new keyt from the Form3 API

    package verification
    
    import (
        "crypto"
        "crypto/rsa"
        "crypto/sha256"
        "crypto/x509"
        "encoding/base64"
        "encoding/pem"
        "errors"
        "fmt"
        "io/ioutil"
        "net/http"
        "net/url"
        "regexp"
        "strings"
    
        "github.com/aws/aws-sdk-go/aws"
        "github.com/aws/aws-sdk-go/aws/session"
        "github.com/aws/aws-sdk-go/service/sqs"
    )
    
    // VerifySQSNotification reads notifications from SQS and
    // verifies their signature
    func VerifySQSNotification(queueURL url.URL) error {
    
        sess := session.Must(session.NewSessionWithOptions(session.Options{
            SharedConfigState: session.SharedConfigEnable,
        }))
    
        svc := sqs.New(sess)
    
        queueURLString := queueURL.String()
    
        timeout := int64(20)
    
        msgResult, err := svc.ReceiveMessage(&sqs.ReceiveMessageInput{
            MessageAttributeNames: []*string{
                aws.String(sqs.QueueAttributeNameAll), // required
            },
            QueueUrl:            &queueURLString,
            MaxNumberOfMessages: aws.Int64(1),
            VisibilityTimeout:   &timeout,
        })
        if err != nil {
            return fmt.Errorf("Error reading from SQS: %w", err)
        }
    
        message := *msgResult.Messages[0]
    
        body := []byte(*message.Body)
        attributes := message.MessageAttributes
    
        // Reject the notification if the Form3 signature is
        // not present
        signatureAttr, ok := attributes["x-form3-signature"]
        if !ok {
            return errors.New("Form3 signature is not present in notification")
        }
    
        form3Signature := *signatureAttr.StringValue
        method := "post"
        path := queueURL.RequestURI()
        host := queueURL.Host
        date := *attributes["date"].StringValue
        contentType := *attributes["content-type"].StringValue
    
        return verifySignature(form3Signature, method, path, host, date, contentType, body)
    }
    
    // VerifyHTTPNotification reads notifications from a
    // HTTP request and verifies their signature
    func VerifyHTTPNotification(req *http.Request) error {
    
        form3Signature := req.Header.Get("X-Form3-Signature")
    
        if form3Signature == "" {
            return errors.New("Form3 signature is not present in notification")
        }
    
        method := req.Method
        path := req.RequestURI
        host := req.Host
        date := req.Header.Get("Date")
        contentType := req.Header.Get("content-type")
    
        // we assume this is a POST request with an actual
        // body
        body, err := ioutil.ReadAll(req.Body)
        if err != nil {
            return err
        }
    
        return verifySignature(form3Signature, method, path, host, date, contentType, body)
    }
    
    func verifySignature(form3Signature, method, path, host, date, contentType string, content []byte) error {
    
        signatureRegex, err := regexp.Compile("signature=\\\"([^\\\"]*)")
        if err != nil {
            return err
        }
    
        keyRegex, err := regexp.Compile("keyId=\\\"([^\\\"]*)")
        if err != nil {
            return err
        }
    
        hasher := sha256.New()
    
        if _, err := hasher.Write(content); err != nil {
            return err
        }
    
        digest := base64.StdEncoding.EncodeToString(hasher.Sum(nil))
    
        msgToHash := fmt.Sprintf(`(request-target): %s %s
    host: %s
    date: %s
    content-type: %s
    digest: SHA-256=%s
    content-length: %d`,
            strings.ToLower(method),
            path,
            host,
            date,
            contentType,
            digest,
            len(content),
        )
    
        hasher.Reset()
        if _, err := hasher.Write([]byte(msgToHash)); err != nil {
            return err
        }
        hashedMsg := hasher.Sum(nil)
    
        // Get the public key associated with the KeyID in the Form3Signature header
        pubKey, err := getPublicKeyByID(strings.Split(keyRegex.FindString(form3Signature), "=\"")[1])
        if err != nil {
            return err
        }
    
        signature := strings.Split(signatureRegex.FindString(form3Signature), "=\"")[1]
        decodedSig, err := base64.StdEncoding.DecodeString(signature)
        if err != nil {
            return err
        }
    
        return rsa.VerifyPKCS1v15(
            pubKey,
            crypto.SHA256,
            hashedMsg[:],
            decodedSig,
        )
    
    }
    
    func getPublicKeyByID(keyID string) (*rsa.PublicKey, error) {
        // You need to fetch the key from storage and return a pointer to the rsa.PublicKey
        // If you do not have a copy of the specified key then request a copy of the key from Form3's API
    
        //pem := "-----BEGIN RSA PUBLIC KEY-----\n ... "
        return parseRsaPublicKeyFromPemStr(pem)
    
    }
    
    func parseRsaPublicKeyFromPemStr(pubPEM string) (*rsa.PublicKey, error) {
        block, _ := pem.Decode([]byte(pubPEM))
        if block == nil {
            return nil, errors.New("failed to parse PEM block containing the key")
        }
    
        pub, err := x509.ParsePKIXPublicKey(block.Bytes)
        if err != nil {
            return nil, err
        }
    
        switch pub := pub.(type) {
        case *rsa.PublicKey:
            return pub, nil
        }
        return nil, errors.New("key type is not RSA")
    }
    

    Code Example Python

    This code example written in Python (version 3.8+) will only cover verifying a request's signature, it will not show you how to call the Form3 API to request a public key if you don't have it or how to load the public key from storage if you have a copy already.

    import re
    import boto3
    
    from Crypto import Signature
    from Crypto.PublicKey import RSA
    from Crypto.Signature import PKCS1_v1_5
    from Crypto.Hash import SHA256
    from base64 import b64decode, b64encode
    from urllib.parse import urlparse
    
    
    def fetch_key(key_id: str) -> str:
        # You need to fetch the key from storage and return the string
        # representation of the public key If you do not have a copy of the
        # specified key then request a copy of the key from the Form3 API
        return '''-----BEGIN RSA PUBLIC KEY-----\n ...'''
    
    
    def verify_http_notification(req) -> bool:
        form3_signature = req.headers.get("X-Form3-Signature")
        if form3_signature is None:
            raise Exception('Form3 signature is not present in notification')
    
        method = req.method
        path = req.headers.environ["REQUEST_URI"]
        host = req.host
        date = req.headers["Date"]
        content_type = req.content_type
        content = req.data
        return verify_signature(
            form3_signature, method, path, host, date, content_type, content
        )
    
    
    def verify_sqs_notification(queue_url: str) -> bool:
        parsed_queue_url = urlparse(queue_url)
    
        try:
            client = boto3.client('sqs')
            response = client.receive_message(
                QueueUrl=queue_url,
                MessageAttributeNames=['All'],
                MaxNumberOfMessages=1,
                WaitTimeSeconds=20
            )
            message = response['Messages'][0]
            attrs = message['MessageAttributes']
    
            if attrs.get('x-form3-signature') is None:
                raise Exception('Form3 signature is not present in notification')
    
            form3_signature = attrs['x-form3-signature']['StringValue']
    
            method = 'post'
            path = parsed_queue_url.path
            host = parsed_queue_url.hostname
            date = attrs['date']['StringValue']
            content_type = attrs['content-type']['StringValue']
            content = message['Body']
    
            return verify_signature(
                form3_signature, method, path, host, date, content_type, content
            )
        except Exception as error:
            raise error
    
    
    def verify_signature(
        form3_signature: str,
        method: str,
        path: str,
        host: str,
        date: str,
        content_type: str,
        content: bytearray
    ) -> bool:
        # Write the digest of the content
        hasher = SHA256.new()
        hasher.update(content.encode("utf-8"))
        digest = b64encode(hasher.digest()).decode("utf-8")
    
        data = '''
    (request-target): {} {}
    host: {}
    date: {}
    content-type: {}
    digest: SHA-256={}
    content-length: {}
    '''.format(
          method.lower(), path, host, date, content_type, digest, len(content)
        ).strip()
    
        key_matcher = re.compile('keyId=\\\"([^\\\"]*)')
        sig_matcher = re.compile('signature=\\\"([^\\\"]*)')
        request_signature = sig_matcher.findall(form3_signature)[0]
    
        public_key = fetch_key(key_matcher.findall(form3_signature)[0])
        rsa_key = RSA.importKey(public_key)
    
        sha_digest = SHA256.new()
        sha_digest.update(data.encode("utf-8"))
        verifier = Signature.PKCS1_v1_5.new(rsa_key)
    
        # returns true if valid, false otherwise
        return verifier.verify(sha_digest, b64decode(request_signature))
    

    Further Reading

    This tutorial explained how to validate that cryptographically signed event notifications are sent by Form3. For more information on request signing and subscriptions, see the following resources.