collective.volto.formsupport 3.1.5

Add support for customizable forms in Volto

collective.volto.formsupport

Add some helper routes and functionalities for Volto sites with form blocks provided by volto-form-block Volto plugin.

volto-form-block version

Works with volto-form-block >= v3.8.0

plone.restapi endpoints

@submit-form

Endpoint that the frontend should call as a submit action.

You can call it with a POST on the context where the block form is stored like this:

> curl -i -X POST http://localhost:8080/Plone/my-form/@submit-form -H 'Accept: application/json' -H 'Content-Type: application/json' --data-raw '{"block_id": "123456789", "data": [{"field_id": "foo", "value":"foo", "label": "Foo"},{"field_id": "from", "value": "[e-mail]"}, {"field_id":"name", "value": "John Doe", "label": "Name"}]}'

where:

• my-form is the context where we have a form block

• block_id is the id of the block

• data contains the submitted form data

Calling this endpoint, it will do some actions (based on block settings) and returns a 200 response with the submitted data.

@form-data

This is an expansion component.

There is a rule that returns a form-data item into “components” slot if the user can edit the context (Modify portal content permission) and there is a block that can store data.

Calling with “expand=true”, this endpoint returns the stored data:

> curl -i -X GET http://localhost:8080/Plone/my-form/@form-data -H 'Accept: application/json' -H 'Content-Type: application/json' --user admin:admin

Specifying a block_id parameter returns only the records associated with a specific block on the page.

> curl -i -X GET http://localhost:8080/Plone/my-form/@form-data?block_id=123456789 -H ‘Accept: application/json’ -H ‘Content-Type: application/json’ –user admin:admin

And replies with something similar:

{
    "@id": "http://localhost:8080/Plone/my-form/@form-data?block_id=123456789",
    "items": [
        {
        "block_id": "123456789",
        "date": "2021-03-10T12:25:24",
        "from": "[e-mail]",
        "id": 912078826,
        "name": "John Doe"
        },
        ...
    ],
    "items_total": 42,
    "expired_total": 2
}

@form-data-export

Returns a csv file with all data (only for users that have Modify portal content permission):

> curl -i -X GET http://localhost:8080/Plone/my-form/@form-data-export -H 'Accept: application/json' -H 'Content-Type: application/json' --user admin:admin

If form fields changed between some submissions, you will see also columns related to old fields.

@form-data-clear

Reset the store (only for users that have Modify portal content permission):

> curl -i -X DELETE http://localhost:8080/Plone/my-form/@form-data-clear --data-raw '{block_id: bbb}' -H 'Accept: application/json' -H 'Content-Type: application/json' --user admin:admin

Optional parameters could be passed in the payload:

• block_id to delete only data related to a specific block on the page, otherwise data from all form blocks on the page will be deleted

• expired a boolean that, if true, removes only records older than the value of days specified in the block configuration (the above block_id parameter is required)

Form actions

Using volto-form-block you can set if the form submit should send data to an email address or store it into an internal catalog (or both).

Send

If block is set to send data, an email with form data will be sent to the recipient set in block settings or (if not set) to the site address.

If there is an attachments field in the POST data, these files will be attached to the email sent.

XML attachments

An XML copy of the data can be optionally attached to the sent email by configuring the volto block’s attachXml option.

The sent XML follows the same format as the feature in [collective.easyform](https://github.com/collective/collective.easyform). An example is shown below:

`xml <?xml version='1.0' encoding='utf-8'?><form><field name="Custom field label">My value</field></form> `

The field names in the XML will utilise the Data ID Mapping feature if it is used. Read more about this feature in the following Store section of the documentation.

Acknowledgement email

It is possible to also send an email to the user who filled in the form.

Set the ‘Send to’ value to include acknowledgement to enable this behaviour. The additional block field acknowledgementMessage can then be used to write the message being sent to the user and the acknowledgementFields block field used to choose the field that will contain the email address the acknowledgement will be sent to.

Store

If block is set to store data, we store it into the content that has that block (with a souper.plone catalog).

The store is an adapter registered for IFormDataStore interface, so you can override it easily.

Only fields that are also in block settings are stored. Missing ones will be skipped.

Each Record stores also two service attributes:

• fields_labels: a mapping of field ids to field labels. This is useful when we export csv files, so we can labels for the columns.

• fields_order: sorted list of field ids. This can be used in csv export to keep the order of fields.

We store these attributes because the form can change over time and we want to have a snapshot of the fields in the Record.

Data ID Mapping

The exported CSV file may need to be used by further processes which require specific values for the columns of the CSV. In such a case, the Data ID Mapping feature can be used to change the column name to custom text for each field.

Block serializer

There is a custom block serializer for type form.

This serializer removes all fields that start with “**default_**" if the user can’t edit the current context.

This is useful because we don’t want to expose some internals configurations (for example the recipient email address) to external users that should only fill the form.

If the block has a field captcha, an additional property captcha_props is serialized by the serialize method provided by the ICaptchaSupport named adapter, the result contains useful metadata for the client, as the captcha public_key, ie:

{
    "subblocks": [
        ...
    ],
    "captcha": "recaptcha",
    "captcha_props": {
        "provider": "recaptcha",
        "public_key": "aaaaaaaaaaaaa"
    }
}

Captcha support

Captcha support requires a specific name adapter that implements ICaptchaSupport. This product contains implementations for:

• HCaptcha (plone.formwidget.hcaptcha)

• Google ReCaptcha (plone.formwidget.recaptcha)

• Custom questions and answers (collective.z3cform.norobots)

• Honeypot (collective.honeypot)

Each implementation must be included, installed and configured separately.

To include one implementation, you need to install the egg with the needed extras_require:

• collective.volto.formsupport[recaptcha]

• collective.volto.formsupport[hcaptcha]

• collective.volto.formsupport[norobots]

• collective.volto.formsupport[honeypot]

During the form post, the token captcha will be verified with the defined captcha method.

For captcha support volto-form-block version >= 2.4.0 is required.

Honeypot configuration

If honeypot dependency is available in the buildout, the honeypot validation is enabled and selectable in forms.

Default field name is protected_1 and you can change it with an environment variable. See collective.honeypot for details.

Attachments upload limits

Forms can have one or more attachment field to allow users to upload some files.

These files will be sent via mail, so it could be a good idea setting a limit to them. For example if you use Gmail as mail server, you can’t send messages with attachments > 25MB.

There is an environment variable that you can use to set that limit (in MB):

[instance]
environment-vars =
    FORM_ATTACHMENTS_LIMIT 25

By default this is not set.

The upload limit is also passed to the frontend in the form data with the attachments_limit key.

Content-transfer-encoding

It is possible to set the content-transfer-encoding for the email body, settings the environment variable MAIL_CONTENT_TRANSFER_ENCODING:

[instance]
environment-vars =
    MAIL_CONTENT_TRANSFER_ENCODING base64

This is useful for some SMTP servers that have problems with quoted-printable encoding.

By default the content-transfer-encoding is quoted-printable as overridden in https://github.com/zopefoundation/Products.MailHost/blob/master/src/Products/MailHost/MailHost.py#L65

Email subject templating

You can also interpolate the form values to the email subject using the field id, in this way: ${123321123}

Header forwarding

It is possible to configure some headers from the form POST request to be included in the email’s headers by configuring the httpHeaders field in your volto block.

[volto-formblock](https://github.com/collective/volto-form-block) allows the following headers to be forwarded:

• HTTP_X_FORWARDED_FOR

• HTTP_X_FORWARDED_PORT

• REMOTE_ADDR

• PATH_INFO

• HTTP_USER_AGENT

• HTTP_REFERER

Data retention

There is a script that implements data cleansing (i.e. for GDPR purpose):

bin/instance -OPlone run bin/formsupport_data_cleansing  --help
Usage: interpreter [OPTIONS]

bin/instance -OPlone run bin/formsupport_data_cleansing [--dryrun|--no-dryrun]

Options:
--dryrun        --dryrun (default) simulate, --no-dryrun actually save the
                changes

--help          Show this message and exit.

The form block as an integer field remove_data_after_days, the retention days can be defined on a single block, If the value is lower or equal to 0 there is no data cleaning for the specific form.

Examples

This add-on can be seen in action at the following sites:

• https://www.comune.modena.it/form/contatti

Translations

This product has been translated into

• Italian

Installation

Install collective.volto.formsupport by adding it to your buildout:

[buildout]

...

eggs =
    collective.volto.formsupport

and then running bin/buildout

Contribute

• Issue Tracker: https://github.com/collective/collective.volto.formsupport/issues

• Source Code: https://github.com/collective/collective.volto.formsupport

License

The project is licensed under the GPLv2.

Authors

This product was developed by RedTurtle Technology team.

Contributors

• RedTurtle Technology, [e-mail]

• Mauro Amico, [e-mail]

Changelog

3.1.5 (2024-10-24)

• Fix otp verification logic: do not break if otp is not in POST call [cekk]

3.1.4 (2024-09-27)

• Add missing collective.volto.otp include for pip environment setup [folix-01]

• Switchable email bcc fields OTP verification. [folix-01]

• Added ISO formatted strings being allowed as date inputs [JeffersonBledsoe]

3.1.3 (2024-09-11)

• Send messages to emails signed as ‘use_as_bcc’ independently from ‘send’ flag. [folix-01]

• Update Italian translations. [cekk]

3.1.2 (2024-08-08)

• Fix upgradestep from version. [folix-01]

3.1.1 (2024-08-08)

• Fix mail header and footer links. [folix-01]

• Plone5 compatibility in the upgradesteps [folix-01]

• Install otp package by upgradestep [folix-01]

• Maintain the fields order in email message. [folix-01]

• Remove otp logics (moved to collective.volto.otp) [folix-01]

3.1.0 (2024-07-04)

• Configurable header and footer for email message. [folix-01]

• Subject templating

• Handle the edge cases where the blocks attribute is not set. [mamico]

• Configuring with plone/meta [mamico]

3.0.2 (2024-05-05)

• Cleanup all possible HTML in user formitted data and convert it to plain text. [cekk]

3.0.1 (2024-04-18)

• Fix README. [cekk]

3.0.0 (2024-04-18)

• Add functionality to check the user inserted email by an OTP. [folix-01]

• Breaking change: clear data method changed from GET to DELETE [mamico]

• Fix: with multiple blocks on the same page, all data is deleted. Now, if you pass a parameter block_id, only the records related to the block are deleted. [mamico]

• Feat: clear data restapi accept a parameter for remove the expired records [mamico]

• data cleaning script [mamico]

• Allow attaching an XML version of the form data to the sent email #22 [JeffersonBledsoe]

• Allow the IDs of fields to be customised for CSV download and XML attaachments #22 [JeffersonBledsoe]

• Add Spanish translation. [macagua]

• Add German translation. [mbarde]

• Allow forwarding request headers in the sent emails #27 [JeffersonBledsoe]

• Added support for sending emails as a table #31 [JeffersonBledsoe]

• Add validation for email fields. [cekk]

• Better compose email message with plain and html text. [cekk]

• Prevent XSS applying safe_html transform to all string values passed on form. [cekk]

• Discard fields submitted that are not defined in form schema. [cekk]

• On form submit, reply with a 200 with submitted data (eventually cleaned) for confirm message. [cekk]

• Fix label in send_mail_template. [cekk]

• Prevent XSS also in send_message field. [cekk]

2.7.0 (2023-04-03)

• Override content-transfer-encoding using MAIL_CONTENT_TRANSFER_ENCODING env [mamico]

• The form block can now be stored in a Volto block container (columns, accordion, tabs, etc) [tiberiuichim]

2.6.2 (2022-11-07)

• Fix collective.honeypot version. [cekk]

2.6.1 (2022-11-07)

• Fix dependencies. [cekk]

2.6.0 (2022-11-07)

• Add collective.honeypot support. [cekk]

2.5.0 (2022-10-04)

• Add limit attachments validation. Can be configured with environment variable. [cekk]

• Export also compiling date in csv. [cekk]

2.4.0 (2022-09-08)

• Add collective.z3cform.norobots support [erral]

2.3.0 (2022-05-26)

• Breaking change: changed the way to store data keys. Now we use field_id as key for Records. [cekk]

• Fix quoting in csv export. [cekk]

• Generate csv columns with proper field labels, and keep the form order. [cekk]

• Captcha support #13. [mamico]

2.2.0 (2022-04-07)

• Notify an event on submit. [mamico]

2.1.0 (2022-03-25)

• Support for user_as_bcc field in volto-form-block: send a separate mail for each email field with that flag. [cekk]

2.0.3 (2021-10-25)

• Fix permission checks in serializer. [nzambello]

2.0.2 (2021-10-25)

• Fix permission checks. [cekk]

2.0.1 (2021-10-18)

• [fix] do not send attachments multiple times. [cekk]

2.0.0 (2021-08-19)

• Convert field types: checkbox => multiple_choice and radio => single_choice to follow new schema in volto-form-block (https://github.com/collective/volto-form-block/pull/7). [cekk]

1.0.5 (2021-05-12)

• Added Brazilian Portuguese (pt_BR) translations. [ericof]

1.0.4 (2021-04-15)

• Changed the classifiers inside setup.py. This should fix some installation issues. [arsenico13]

1.0.3 (2021-04-08)

• NEW: The @submit-form entry-point now takes into account if a field is “marked” with “use_as_reply_to” and use that field for “from” and “reply to”. [arsenico13]

1.0.2 (2021-03-24)

• Fix form_data for anon. [cekk]

1.0.1 (2021-03-24)

• Fix README for pypi. [cekk]

1.0.0 (2021-03-24)

• Initial release. [cekk]