Archetypes storage for storing fields raw values on the file system.
Project description
#################
FileSystemStorage
#################
By Ingeniweb_.
--------------------------
.. contents:: **Table of contents**
--------------------------
About FileSystemStorage
#######################
FileSystemStorage (FSS) is an Archetypes storage for storing fields
raw values on the file system. This storage is used to avoid
unnecessary growth of the ZODB's FileStorage (Data.fs) when using a
lot of large files.
Please note that FSS is a Plone component for **content types
developers**. Do not expect anything more in your Plone site if you
don't use Plone components that may use FSS, such as AttachmentField_.
Copyright and license
#####################
Copyright (c) 2005 - 2009 Ingeniweb_ SAS
This software is subject to the provisions of the GNU General Public License,
Version 2.0 (GPL). A copy of the GPL should accompany this distribution.
THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
FOR A PARTICULAR PURPOSE
See the `LICENSE` file that comes with this product.
Requirements
############
* Plone 3.0+ bundle
* or Plone 4.0+ bundle
Upgrading
#########
From FSS 2.5.x and older
========================
The way we configure the storage and backup paths as well as the
storage strategies **HAVE CHANGED IN DEPTH**. There's no automatic
migration from this version. Please create a configuration file that
clones your old configlet preferences as indicated in `FSS main
configuration`_.
From iw.fss 2.7.1
=================
From this version and later version, iw.fss registers upgrade steps in
GenericSetup. When upgrading to a newer version of iw.fss, just have a
look at the "Upgrade" tab of the portal_setup tool in your Plone site.
Warning, due to the removal of the `portal_fss` tool upgrading from 2.7.1 does
not retain the RDF options (activated and specific script). If you used RDF with
2.7.1, you need to redo the same RDF setting in the FileSystemStorage config
panel.
Installation
############
With buildout
=============
This example speaks of itself::
[buildout]
...
eggs =
...
iw.fss
...
[instance]
...
zcml =
...
iw.fss
iw.fss-meta
...
You may also use the `iw.recipe.fss`_ recipe egg to configure fully an
instance.
From SVN reposo
===============
* Inflate iw.fss package into your zope instance lib/python directory.
I'm sure it's already done.
* Add `$INSTANCE_HOME/etc/package-includes/iw.fss-meta.zcml` file with
this line::
<include package="iw.fss" file="meta.zcml"/>
* Add `$INSTANCE_HOME/etc/package-includes/iw.fss-configure.zcml` file
with this line::
<include package="iw.fss" />
Configuring your Plone sites
============================
* **Read carefully** `Storage strategies`_ and `FSS main
configuration`_ below and configure your instance in
`plone-filesystemstorage.conf` accordingly. You may use the
`iw.recipe.fss`_ recipe in your `buildout.cfg` to do this.
* Start your Zope instance
* In ZMI, go into your plone site and install the product using
portal_quickinstaller
* Optionally, tweak some - self explained - preferences in the
FileSystemStorage configuration panel of your Plone site.
Storage strategies
##################
A storage strategy defines how field values will be stored in your
files system.
Strategies overview
===================
FSS comes with 4 storage strategies. Each strategy requires two directories:
* The storage directory stores the values of relevant fields in files
according to the selected strategy.
* The backup directory that keeps the values of the fields of deleted
contents for "Undo" purpose.
* Only one storage strategy can be selected on a given Plone site,
but different Plone site in the same Zope instance may have distinct
storage strategies.
* You need the best performances: choose the `Flat storage strategy`_
or `Directory storage strategy`_.
* You need a storage that looks like your Plone site (in example to
publish the storage directory in a read only samba or NFS share):
choose the `Site storage strategy 1`_ or the `Site storage strategy 2`_.
* If you care about CMFEdition support (means that one of the content
types using FSS is versioned with CMFEditions), you should *not* use
the `Site storage strategy 1`_ or the `Site storage strategy 2`_.
See `FSS main configuration`_ to configure the best suited strategy on
your Zope instance and Plone sites.
Flat storage strategy
=====================
All field values are stored in a flat structure. This strategy is the
default one.
* Filename of these stored values: `<uid of content>_<field name>`
* Filename of rdf file: `<uid of content>_<field name>.rdf`
* Filename of backup values: `<uid of content>_<field name>.bak`
Rdf files are not backed up. They are automatically generated.
Example of storage::
fssDirectory
|
|- f42ad00adb7d4580f929d734bf1ed3ab_image
|
|- f42ad00adb7d4580f929d734bf1ed3ab_image.rdf
|
|- f42ad00adb7d4580f929d734bf1ed3ab_file
|
|- f42ad00adb7d4580f929d734bf1ed3ab_file.rdf
fssBackupDirectory
|
|- 9efeb7638fb35f5f9b9955da3e2dbfec_file.bak
Directory storage strategy
==========================
All field values are stored in a directory structure. Sub directories
are defined on two level. First level of directory uses the 2
characters of content uid. Second level of directory uses the 4
characters of content uid. Backup files are stored in a flat
structure.
* Filename of these stored values: `<uid of content>_<field name>`
* Filename of rdf file: `<uid of content>_<field name>.rdf`
* Filename of backup values: `<uid of content>_<field name>.bak`
RDF files are not backed up. They are automatically generated
Example of storage::
fssDirectory
|
|- f42
|
|- f42ad
|
|- f42ad00adb7d4580f929d734bf1ed3ab_image
|
|- f42ad00adb7d4580f929d734bf1ed3ab_image.rdf
|
|- f42ad00adb7d4580f929d734bf1ed3ab_file
|
|- f42ad00adb7d4580f929d734bf1ed3ab_file.rdf
fssBackupDirectory
|
|- 9efeb7638fb35f5f9b9955da3e2dbfec_file.bak
Depending in one hand on your file system performances facing a huge
amount of files in the same directory, and in the other hand on the
number of contents relying on FSS, you might choose this strategy or
the `Flat storage strategy`_.
Site storage strategy 1
=======================
All field values are stored in a directory structure mirroring
structure of PloneSite. Backup files are stored in a flat structure.
* Filename of these stored values: Filename of field value or field
name if not defined
* Filename of rdf file: `<field name>.rdf`
* Filename of backup values: `<uid of content>_<field name>.bak`
Rdf files are not backed up. They are automatically generated
Example of storage::
fssDirectory
|
|- members
|
|- john
|
|- dummy-document
|
|- image
| |
| |- moutain.jpg
| |
| |- image.rdf
|
|- file
|
|- diary.odt
|
|- file.rdf
fssBackupDirectory
|
|- 9efeb7638fb35f5f9b9955da3e2dbfec_file.bak
Site storage strategy 2
=======================
All field values are stored in a directory structure mirroring
structure of PloneSite. Backup files are stored in a flat structure.
* Filename of these stored values: Filename of field value or field
name if not defined
* Filename of rdf file: `<field filename>.rdf`
* Filename of backup values: `<uid of content>_<field name>.bak`
Rdf files are not backed up. They are automatically generated
Example of storage::
fssDirectory
|
|- members
|
|- john
|
|- dummy-document
|
|- fss.cfg
|
|- moutain.jpg
|
|- mountain.jpg.rdf
|
|- diary.odt
|
|- diary.odt.rdf
fssBackupDirectory
|
|- 9efeb7638fb35f5f9b9955da3e2dbfec_file.bak
FSS main configuration
######################
FSS is mainly configured the ZConfig way. At startup, the
configuration file will be searched (in that order) in:
* $INSTANCE_HOME/etc/plone-filesystemstorage.conf
* /path/to/iw/fss/etc/plone-filesystemstorage.conf
* /path/to/iw/fss/etc/plone-filesystemstorage.conf.in
A sample working configuration in provided in this last file. It
assumes you have $INSTANCE_HOME/var/fss_storage and
$INSTANCE_HOME/var/fss_backup directories, both being read/write
enabled to the user that runs the Zope process, unless Zope will raise
an error at startup.
All configuration doc you need is in `plone-filesystemstorage.conf.in`
comments.
Inconsistent configuration features raise explicit error messages at
Zope startup.
Configuration panel
###################
FSS and RDF
===========
As mentioned above in `Storage strategies`_, FSS can optionally store
RDF files in the back-end storage. These RDF files conform the
`DCMES-XML standard`_ on XML expression of DublinCore elements.
If you select the `Flat storage strategy`_, the RDF files may be used
to build a files tree as close as possible of the Plone tree structure
from the storage back-end, with the `build_fs_tree.py` utility::
$ cd /path/to/iw/fss/bin
$ python build_fs_tree.py --help
FSS maintenance
===============
The "Maintenance" tab of FSS configuration panel shows some statistics
about the amount of files managed by FSS.
In addition 2 additional buttons are provided:
* **Update FSS files** : Cleans up the back-end storage
directories. Files not referenced from a content object are removed.
* **Update RDF files** : Builds all RDF files, in the event you chose
to select the generetion of RDF files after having inserted relevant
contents.
Zope backups
############
Of course, you need to synch your Data.fs backups with the storage
paths backups, unless you may restore corrupted/incomplete sites.
Caveats and pittfalls
#####################
Note that whatever's the selected strategies, **DO NEVER CHANGE ANY
FILE IN THE STORAGE PATHS UNLESS YOU KNOW WHAT YOU DO**, otherwise
you'll loose your contents, your job, your money, your friends, your
wife/husband and children.
**DO NEVER CHANGE THE STRATEGY FOR A PLONE SITE ONCE THIS SITE HAS
FIELDS STORED IN ITS FSS STORAGE PATH** unless you ran the appropriate
strategy migrator - see `Migrating between strategies`_ otherwise
you're loose your contents, your job, ...
Thought it is possible to share the same storage path and backup path
within various Plone sites, since you don't mind of your data in
development sites, you should really configure FSS such each Plone
site has its own private storage path and backup path. In cas you
ignore this warning, **DO NEVER CLICK ANY BUTTON IN THE MAINTENANCE
TAB OF THE FSS CONFIGURATION PANEL** otherwise you'll loose your
contents, your job, ...
Zexp exports **don't embed the associated FSS storage**. So don't move
your Plone site within Zope instances using zexp exports unless you
move the storage and backup directories along with the zexp file.
After changing your configuration (see `FSS main configuration`_),
always restart your Zope instance in foreground since configuration
errors are not reported when Zope is started as daemon or Windows
service.
Developer's corner
##################
Using FSS in your content types
===============================
We assume that creating AT based content types is a familiar activity.
When creating the schema of a content type, you usually don't mention
the storage of your fields since AT provides a default ZODB
storage. You can choose another storage, either another one that ships
in the Archetypes bundle, or a third party storage like FSS.
Here is a small example that doesn't need much comments::
# Usual Zope/CMF/Plone/Archetypes imports
...
from iw.fss.FileSystemStorage import FileSystemStorage
...
my_schema = Schema((
FileField('file',
...
storage=FileSystemStorage(),
widget=FileWidget(...)
),
...
))
...
You may have a look at `examples/FSSItem.py` to see a demo content
type using FSS. If you want to play with this content type::
$ $INSTANCE_HOME/bin/zopectl stop
$ export FSS_INSTALL_EXAMPLE_TYPES=1
$ $INSTANCE_HOME/bin/zopectl start
Monkey patching a third party content type
==========================================
The Python way
++++++++++++++
This works with any Zope version listed in the Requirements_. If you
run a Zope 2.9 or later instance, you should prefer `The ZCML way`_.
This simple example shows how to plug the ATFile standard type from
any custom product. Let's say we're in the `__init__.py` of your
custom product::
...
from iw.fss import zcml as fss_utils
from Products.ATContentTypes import atct
...
fss_utils.patchATType(atct.ATFile, ['file'])
...
The ZCML way
++++++++++++
This works only with Zope 2.9 and up, and is the recommanded way to
wire your AT based content types with FSS since this way hides the API
details that may be deprecated in the future.
You just need to add the "fss" namespace support and some directives
in your "configure.zcml" like this to have the same effect as in the
example of `The Python way`_ above::
<configure
...
xmlns:zcml="http://namespaces.zope.org/zcml"
xmlns:fss="http://namespaces.ingeniweb.com/filesystemstorage"
...>
...
<fss:typeWithFSS
zcml:condition="installed iw.fss"
class="Products.ATContentTypes.atct.ATFile"
fields="file" />
...
</configure>
The (required) attributes of this directive are:
* **class**: the dotted name of the AT based content type class.
* **fields**: one or more (space separated) field names to wire with FSS.
Storing Plone standard content types with FSS
=============================================
Include this ZCML element either in your instance `package-includes`
directory or in any personal `configure.zcml` in your instance
packages::
<include package="iw.fss" file="atct.zcml" />
This ZCML file wires FSS with the relevant content types that ship with Plone bundle:
* File
* Image
* News Item
Customize RDF info set
======================
You can add data to the RDF info set via a FSS dedicated - and
optional - hook.
Open the FSS configuration panel and give the name of your personal
RDF data script, let's call it `fss_custom_rdf` for this example. This
script will add or change the RDF default data set.
RDF script interface
++++++++++++++++++++
The script may be in any Plone layer like `custom` or any other in the
skins path.
Such script is expected to have 4 parameters:
* `name`: the name of the field
* `instance`: the content object being processed
* `properties`: FSS field properties mapping, depending on the type of
field associated to FSS storage.
* `default_rdf`: a mapping structure of the RDF data, including the
namespaces, and the elements.
Such script is expected to return an updated `default_rdf` mapping
structure.
Please read the source of classes `FSSInfo`, `FSSFileInfo` and
`FSSImageInfo` from module `FileSystemStorage.py`, `RDFWriter` from
module `rdf.py` for detailed information on formats of expected data.
Simple example
++++++++++++++
We need to add the date of RDF file generation in an element like
this::
<fss:rdf_date>2007-05-13 13:21:00</fss:rdf_date>
Here's the script::
## Script (Python) "fss_custom_rdf"
##bind container=container
##bind context=context
##bind namespace=
##bind script=script
##bind subpath=traverse_subpath
##parameters=name, instance, properties, default_rdf
##title=
##
"""We add the date of RDF file generation"""
rdf_args = default_rdf.copy()
# Add the date of RDF generation
new_props = []
new_props.extend(rdf_args['field_props'])
new_props.append({'id': 'fss:rdf_date', 'value': DateTime().ISO()})
rdf_args['field_props'] = new_props
return rdf_args
More complex example
++++++++++++++++++++
In this more complex (and stupid - don't do this ;) example, we remove
the elements from `fss` namespace and simulate the standard behaviour
for other elements (`dc` and `rdf` elements).
Here's the code::
## Script (Python) "fss_custom_rdf"
##bind container=container
##bind context=context
##bind namespace=
##bind script=script
##bind subpath=traverse_subpath
##parameters=name, instance, properties, default_rdf
##title=
##
from Products.CMFCore.utils import getToolByName
rdf_args = {}
# Set RDF charset
ptool = getToolByName(instance, 'portal_properties')
rdf_args['charset'] = ptool.site_properties.default_charset
# Set RDF namespaces
rdf_args['namespaces'] = (
{'id': 'xmlns:rdf', 'value': 'http://www.w3.org/1999/02/22-rdf-syntax-ns#'},
{'id': 'xmlns:dc', 'value': 'http://purl.org/dc/elements/1.1/'},
{'id': 'xmlns:fss', 'value': 'http://namespace.ingeniweb.com/fss'},)
# Set field url
utool = getToolByName(instance, 'portal_url')
portal_path = utool.getPortalObject().getPhysicalPath()
portal_path_len = len(portal_path)
rel_path = '/'.join(instance.getPhysicalPath()[portal_path_len:])
fss_path = '%s/%s' % (rel_path, name)
rdf_args['field_url'] = fss_path
# Set RDF properties
props = (
{'id': 'dc:title', 'value': instance.title_or_id()},
{'id': 'dc:description', 'value': instance.Description()},
{'id': 'dc:language', 'value': instance.Language()},
{'id': 'dc:creator', 'value': instance.Creator()},
{'id': 'dc:date', 'value': instance.modified()},
{'id': 'dc:format', 'value': properties.get('mimetype', 'text/plain')},
)
rdf_args['field_props'] = props
# Returns the RDF structure
return rdf_args
Recommanded practices on custom RDF
+++++++++++++++++++++++++++++++++++
Do not change the existing elements under `fss` namespace unless the
`bin/build_fs_tree.py` utility won't work. In other terms, do not use
the script of the `More complex example`_.
Adding or changing elements to the `dc` and `rdf` namespaces may screw up
external utilities based on canonical `DCMES-XML standard`_.
Consider adding your own namespace for your custom extra elements.
Migrating from AttributeStorage or AnnotationStorage
####################################################
We assume that, at this point, you got :
* A production Plone site filled with contents which storage is supported by the
standard AT storages (AttributeStorage or AnnotationStorage).
* A development site with the **same** AT based content types (ATCT plus
others), `iw.fss` and a ZCML setup that stores fields of some contents with
`iw.fss`. See how to do it `The ZCML way`_ or use the `atct.zcml` file that's
included in this package.
Backup your production site
===========================
Yes, at this early stage, we didn't test migration with lots and lots of content
types. Only the basic OTB content types have been tested. But this should work
with any other. Whatever, you should always backup a site before proceeding to a
deep migration like this one.
Add the `iw.fss` settings
=========================
Install `iw.fss` and the ZCML settings from your develoment site on your
production site. Note that at this step, it's safer to deny access to the users
(use Apache settings or whatever).
Start your Zope instance in forground mode (recommanded if you want to follow
the migration logging).
Do the migration
================
Open the FileSystemStorage configlet. Check that the main control panel reports
the expected settings (storage paths and strategies, supported content types and
fields).
Click the "migration" tab of this control panel, check the options, and proceed
to migrations.
Pack the ZODB
=============
After a successful migration, you may pack immediately the ZODB to appreciate
the effect of using `iw.fss`.
FAQ
###
Migrating between strategies
============================
Use the `bin/strategymigrator.py` shell utility that ships with FSS. Get
more info on this utility using the `--help` option.
Note that only flat -> directory and directory -> flat are available
today. Sponsorship is welcome - as indicated in `Support and
feedback`_ - if you need other strategy migrations.
Where are the files for this item?
==================================
Sometimes we need to know what files store the fields of a content item. For
this, login as site manager and browse to
http://my.site.tld/where/is/my/content/@@fss_where_are_my_files
Testing
#######
Please read `./tests/README.txt`.
Other documentation
###################
See `./doc/*` and `./locales/README.txt`
Downloads
#########
You may find newer stable versions of FSS and pointers to related
informations (tracker, doc, ...) from
http://pypi.python.org/pypi/iw.fss
Subversion repository
#####################
Stay in tune with the freshest (maybe unstable) versions or participate to
the FileSystemStorage evolutions:
https://svn.plone.org/svn/collective/iw.fss
Support and feedback
####################
Please read all the documentation that comes with this product before
asking for support, unless you might get a RTFM reply ;)
Localisation issues - other than french - should be reported to the
relevant translators (see Credits_ below).
Report bugs using the tracker (the `Tracker` link from
http://plone.org/iw.fss). Please provide in your
bug report:
* Your configuration (Operating system+Zope+Plone+Products/versions).
* The storage strategy in use.
* The full traceback if available.
* One or more scenario that triggers the bug.
Note that we do not support bug reports on Subversion trunk or
branches checkouts.
`Mail to Ingeniweb support <mailto:support@ingeniweb.com>`_ in English or
French to ask for specific support.
`Donations are welcome for new features requests
<http://sourceforge.net/project/project_donations.php?group_id=74634>`_
Credits
#######
* Main developer: `Cyrille Lebeaupin <mailto:cyrille.lebeaupin@ingeniweb.com>`_
* ZConfig and ZCML support: `Gilles Lenfant <mailto:gilles.lenfant@ingeniweb.com>`_
* HTTP "Range" support: `Youenn Boussard <mailto:youenn.boussard@ingeniweb.com>`_
* Paris 2008 sprint additional contributors:
- `Mehdi Benammar <mailto:mehdi.benammar@kitry.lu>`_
- `Romain Griffiths <mailto:wid@epita.fr>`_
See `locales/README.txt` about localization credits.
--------------------------
.. sectnum::
.. _Ingeniweb: http://www.ingeniweb.com/
.. _AttachmentField: http://plone.org/products/attachmentfield
.. _DCMES-XML standard: http://dublincore.org/documents/dcmes-xml/
.. _iw.recipe.fss: http://pypi.python.org/pypi/iw.recipe.fss
.. $Id: README.txt 226275 2010-11-09 09:57:03Z glenfant $
CHANGES
#######
2.8.0rc5
========
* Fixed a critical issue where an image would be attempted restored at
every request. [malthe]
* Fixed an issue where the AT tool for scaling images would pass a
unicode file path which would result in a bug because the code was
only looking for a regular string (else deciding that it's a stream
object). [malthe]
* Fixed an issue with HTTP range leading to a missing file size. With
some clients (seen in Firefox 3.6), the ``end`` value is trivial and
must be explicitly set to the file size before returning the
iterator. [malthe]
2.8.0rc4
========
Plone 4 support
* "from Globals import InitializeClass" <- "from App.class_init ..." closes
deprecation warning.
* AT moved from Zope2 style to Zope3 style interfaces since the old Zope 2
"Interfaces" package has gone away with Zope 2.12. We have a workaround that
supports both styles.
[glenfant]
* (Deprecated) tabindex removed from all templates.
[glenfant]
* Added a view for content items that allows managers to see where in the file
system the fields are stored.
[glenfant]
2.8.0rc3
========
* Change the behaviour of upload on fss. No more memory consuption for that
now. We use always use fileiterator when inspecting big file. To achieve to
that, VirtualDate override _read_data of OFS.Image.File to construct an clean
FSSPdata.
[yboussard]
* Monkey patching with collective.monkeypatcher. All monkey patches should be
made that way.
[glenfant]
* Using z3c.autoniclude when available.
[glenfant]
* Change behaviour of VirtualData class. Now VirtualData work as OFS.image ie to
acces data you MUST pass by a chained list (FSSPdata) that contain only a
block of data. This block of data is getting by a filestream iterator. So no
more memmory consuption (max 65536 , a block of data). This is an important
change because when Products.Archetype.Field.FileField get metatada of a File
old version of fss get all file data for this (ie myField.getFilename() -> you
have your all data in memory !!). Also when you invoke
Field.download(instance, no_output = True) you get now a filestream iterator
(There is a patch to this method because of dowload must set content-length
header for the publisher)
[yboussard]
* fix content_type retreving for migration process.
[yboussard]
2.8.0rc2
========
* Fixed crappy error logging on migration
[glenfant]
2.8.0rc1
========
* Worked around (and cancel migration) when AT tries (and fails) stupidly to
guess the charset of a binary data.
[glenfant]
* Spanish translation is up to date
[dmunicio]
2.8.0b2
=======
* Fixed bad mime type setting when migrating.
[glenfant]
* Fixed loss of text when migrating News items with image
[glenfant]
* Fixed logging on transaction when migrating and show the total migrated
items in the log too.
[glenfant]
2.8.0b1
=======
* Upgraded translations (.pot and french translation)
[glenfant]
* Fixed issues reported by test users.
[glenfant]
* Faster and safer method to guess the mime type when migrating.
[glenfant]
2.8.0a3
=======
* Fixed a NameError in migrations that could not be raised by unit tests.
[glenfant]
2.8.0a2
=======
* Fixed an issue on optional (non "required") fields: getting an empty
field must raise an AttributeError and not return an empty string.
[glenfant]
* ATCT settings disabled
[glenfant]
2.8.0a1
=======
* Adding migration utility (XStorage -> FSS)
[glenfant]
2.7.6
=====
* Safety belt against GS irrelevant upgrades.
See https://dev.plone.org/plone/ticket/8507
[glenfant]
* fix i18n initialization under win32
[bmathieu]
2.7.5
=====
* removed fss_get in VirtualBinary urls: this old script has been removed with
skins. This caused problems with content such as plone news item calling
image.tag().
[b_mathieu]
* run tests without shell variable ZOPE_TESTCASE
[b_mathieu]
2.7.4
=====
* Fix a bug on versioning : previous version of FSS was unable to deal with
content having archetypes fields with no storages
[clebeaupin]
2.7.3
=====
* GenericSetup upgrade step (any version) -> 1.0
[glenfant]
* Red warnings when using the default common config in config panels
[glenfant]
* Config panel enhancements (config details view, documentation)
[glenfant]
* Code cleanup (useless stuffs)
[glenfant]
* Fixed test cases
[cbosse] [glenfant]
* CMFEditions compatibility
[glenfant]
* Annotations replacement for the attributes (FSSInfo), and some
cosmetics.
[mehdi]
* CMF skins layer removed in favour of views.
[wid]
* Add a test verifying that the storage path contains one item after
adding a file, and zero items after deleting it.
[maikroeder]
* "site1" and "site2" strategies are forfidden as global strategies
[glenfant]
* 100% GenericSetup installation
[clebeaupin]
2.7.2
=====
* Fix the bobo traverse mess: we should not honour Plone 2.5(-)
compatibility (remove patches.py) and fix the traverse deprecation
warning.
[glenfant]
* pyflakes everything.
[glenfant]
* Removed Plone 2.5 (and previous) compatibility.
[glenfant]
* Protect with appropriate permission ZMI views/methods.
[glenfant]
2.7.1 - 2008/03/17
==================
* Test case for FSS registration.
[glenfant]
* Hint in README.txt to set FSS storage only if iw.fss is installed.
[glenfant]
* The ZMI "Overview" tab shows the various types changed to use FSS.
[glenfant]
* I18N domain is now "iw.fss" being consistent with new styles
components.
[glenfant]
* Translations moved to zope 3 style in place of PTS style. As a
consequence, the config panel title is not translated any more
("plone" domain).
[glenfant]
* Public function to patch a type and registering patched types
for migration purposes
[glenfant]
* Renamed flat2directory.py into strategymigrator.py since future
other strategy migrations will go there.
[glenfant]
* Added bin/flat2directory.py that migrates "flat" strategy storage to
"directory" strategy storage.
[glenfant]
* Added discriminator in ZCML support that warns on duplicate config
of the same content type.
[glenfant]
2.7.0 - 2008/02/26
==================
* Convert FileSystemStorage Zope 2 product to iw.fss python package
[tarek]
* Added ZCML namespace/directive to wire fields of content types with
FSS.
[glenfant]
2.6.2 - 2007/12/07
==================
* Fixed range header processing: if one range - specific file stream
iterator if multiple range - OFS.image traitment (1680760
sourceforge tracker) Add some unit test to test range header
2.6.1 - 2007/10/29
==================
* ZConfig bug that lowercases some config strings: we test path in
lowercase too (ED)
* Remove the OSError when moving file into backup
* Plone 3.0 compatibility [glenfant]
* Do not override Content-Disposition HTTP header if already set in
VirtualBinary.index_html (zegor)
* filestream_iterator is not "range proof". Add a new
range_filestream_iterator to negociate Range request
2.6.0 RC1 - 2007/06/15
======================
* Changed in depth the way we configure FSS with the ZConfig
way. Changing the storage and backup path, and the storage strategy
TTW is disabled [glenfant]
* As a consequence, changing the storage strategy doesn't require to
hack config.py any more [glenfant]
* As a consequence, each Plone site in a Zope instance can have its
own storage strategy [glenfant]
* Configuration view and README.txt view in ZMI [glenfant]
* Fixed documentation issues [glenfant]
* SiteStorageStrategy and SiteStorageStrategy2 didn't work under Win32
[glenfant]
* Make FSS compatible with Plone 2.0 : just rewrite some testcases
2.5.6 - 2007/06/15
==================
* When renaming a folder containing FSS content, FSSInfo attribute was
stolen one each content of the folder. Provides a solution to avoid
this problem (CL)
2.5.5 - 2007/06/13
==================
* Make FSS works when calling _setUID method on an AT object (CL)
* Add a strategy mark in the storage in a '.strategy' file such we
could forbid strategy changes of storages at Zope startup. (i.e.,
using a 'flat' storage strategy in a storage filled using a
'directory' strategy)
2.5.4 - 2007/03/13
==================
* Fix a bug on Plone 2.5 when moving a folder containing items using
FSS storage. Those items were losing their FSS content. (CL)
2.5.3 - 2007/02/12
==================
* Don't monkey patch BaseObject __bobo_traverse__ under Plone 2.5, but
register ITraversable adapter for this. This change is required as of
Archetypes 1.4.1 (see AT HISTORY.txt, search '__bobo_traverse__')
2.5.2 - 2006/10/27
==================
* Fix bug on maintenance tab: update files and backup, and update RDF (ED)
* Fix some renaming bugs (ED)
* Extend index_html method of VirtualFile and VirtualImage to use a
filestream iterator. FSS is now compliant with CacheFu
* Fix bug in FSSTool:updateFSS - replace restoreBackup method by
restoreValueFile
* Add new strategy: SiteStorageStrategy2
2.5.1 - 2006/07/28
==================
* Check existence of backup and storage paths in configlet
* Fix bug on maintenance tab
2.5 - 2006/07/19
================
* Update tests to use all strategies
* Add storage strategies : FlatStorageStrategy,
DirectoryStorageStrategy, \ SiteStorageStrategy
* Patch BaseObject to make it possible to access images like this
mysite/myobj/image or mysite/myobj/image_mini
* Added a docstring on VirtualData class and subclasses. This should
fix http://plone.org/products/atphoto/issues/28 : VirtualImage can
be published directly by ATImage
2.5RC1
======
* Add a hook to customize RDF files
* Add script to build filesystem tree from FSS files
* Add RDF implementation
2.1RC2
======
* Remove use of ComputedAttribute to avoid strange errors
* If file is empty don't create an empty file on filesystem
2.1RC1
======
* Add Content-Disposition header in fss_get python script
* Make FSS compliant with AT 1.3.4 and Plone 2.1
2.0RC3
======
* Fix bug on delete action (In previous version, it was impossible
to delete non required field value)
2.0RC2
======
* Add translations
* Fix bug in configlet when no files are stored on filesystem.
2.0RC1
======
* First public release
-------------
.. $Id: CHANGES 226275 2010-11-09 09:57:03Z glenfant $
FileSystemStorage
#################
By Ingeniweb_.
--------------------------
.. contents:: **Table of contents**
--------------------------
About FileSystemStorage
#######################
FileSystemStorage (FSS) is an Archetypes storage for storing fields
raw values on the file system. This storage is used to avoid
unnecessary growth of the ZODB's FileStorage (Data.fs) when using a
lot of large files.
Please note that FSS is a Plone component for **content types
developers**. Do not expect anything more in your Plone site if you
don't use Plone components that may use FSS, such as AttachmentField_.
Copyright and license
#####################
Copyright (c) 2005 - 2009 Ingeniweb_ SAS
This software is subject to the provisions of the GNU General Public License,
Version 2.0 (GPL). A copy of the GPL should accompany this distribution.
THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
FOR A PARTICULAR PURPOSE
See the `LICENSE` file that comes with this product.
Requirements
############
* Plone 3.0+ bundle
* or Plone 4.0+ bundle
Upgrading
#########
From FSS 2.5.x and older
========================
The way we configure the storage and backup paths as well as the
storage strategies **HAVE CHANGED IN DEPTH**. There's no automatic
migration from this version. Please create a configuration file that
clones your old configlet preferences as indicated in `FSS main
configuration`_.
From iw.fss 2.7.1
=================
From this version and later version, iw.fss registers upgrade steps in
GenericSetup. When upgrading to a newer version of iw.fss, just have a
look at the "Upgrade" tab of the portal_setup tool in your Plone site.
Warning, due to the removal of the `portal_fss` tool upgrading from 2.7.1 does
not retain the RDF options (activated and specific script). If you used RDF with
2.7.1, you need to redo the same RDF setting in the FileSystemStorage config
panel.
Installation
############
With buildout
=============
This example speaks of itself::
[buildout]
...
eggs =
...
iw.fss
...
[instance]
...
zcml =
...
iw.fss
iw.fss-meta
...
You may also use the `iw.recipe.fss`_ recipe egg to configure fully an
instance.
From SVN reposo
===============
* Inflate iw.fss package into your zope instance lib/python directory.
I'm sure it's already done.
* Add `$INSTANCE_HOME/etc/package-includes/iw.fss-meta.zcml` file with
this line::
<include package="iw.fss" file="meta.zcml"/>
* Add `$INSTANCE_HOME/etc/package-includes/iw.fss-configure.zcml` file
with this line::
<include package="iw.fss" />
Configuring your Plone sites
============================
* **Read carefully** `Storage strategies`_ and `FSS main
configuration`_ below and configure your instance in
`plone-filesystemstorage.conf` accordingly. You may use the
`iw.recipe.fss`_ recipe in your `buildout.cfg` to do this.
* Start your Zope instance
* In ZMI, go into your plone site and install the product using
portal_quickinstaller
* Optionally, tweak some - self explained - preferences in the
FileSystemStorage configuration panel of your Plone site.
Storage strategies
##################
A storage strategy defines how field values will be stored in your
files system.
Strategies overview
===================
FSS comes with 4 storage strategies. Each strategy requires two directories:
* The storage directory stores the values of relevant fields in files
according to the selected strategy.
* The backup directory that keeps the values of the fields of deleted
contents for "Undo" purpose.
* Only one storage strategy can be selected on a given Plone site,
but different Plone site in the same Zope instance may have distinct
storage strategies.
* You need the best performances: choose the `Flat storage strategy`_
or `Directory storage strategy`_.
* You need a storage that looks like your Plone site (in example to
publish the storage directory in a read only samba or NFS share):
choose the `Site storage strategy 1`_ or the `Site storage strategy 2`_.
* If you care about CMFEdition support (means that one of the content
types using FSS is versioned with CMFEditions), you should *not* use
the `Site storage strategy 1`_ or the `Site storage strategy 2`_.
See `FSS main configuration`_ to configure the best suited strategy on
your Zope instance and Plone sites.
Flat storage strategy
=====================
All field values are stored in a flat structure. This strategy is the
default one.
* Filename of these stored values: `<uid of content>_<field name>`
* Filename of rdf file: `<uid of content>_<field name>.rdf`
* Filename of backup values: `<uid of content>_<field name>.bak`
Rdf files are not backed up. They are automatically generated.
Example of storage::
fssDirectory
|
|- f42ad00adb7d4580f929d734bf1ed3ab_image
|
|- f42ad00adb7d4580f929d734bf1ed3ab_image.rdf
|
|- f42ad00adb7d4580f929d734bf1ed3ab_file
|
|- f42ad00adb7d4580f929d734bf1ed3ab_file.rdf
fssBackupDirectory
|
|- 9efeb7638fb35f5f9b9955da3e2dbfec_file.bak
Directory storage strategy
==========================
All field values are stored in a directory structure. Sub directories
are defined on two level. First level of directory uses the 2
characters of content uid. Second level of directory uses the 4
characters of content uid. Backup files are stored in a flat
structure.
* Filename of these stored values: `<uid of content>_<field name>`
* Filename of rdf file: `<uid of content>_<field name>.rdf`
* Filename of backup values: `<uid of content>_<field name>.bak`
RDF files are not backed up. They are automatically generated
Example of storage::
fssDirectory
|
|- f42
|
|- f42ad
|
|- f42ad00adb7d4580f929d734bf1ed3ab_image
|
|- f42ad00adb7d4580f929d734bf1ed3ab_image.rdf
|
|- f42ad00adb7d4580f929d734bf1ed3ab_file
|
|- f42ad00adb7d4580f929d734bf1ed3ab_file.rdf
fssBackupDirectory
|
|- 9efeb7638fb35f5f9b9955da3e2dbfec_file.bak
Depending in one hand on your file system performances facing a huge
amount of files in the same directory, and in the other hand on the
number of contents relying on FSS, you might choose this strategy or
the `Flat storage strategy`_.
Site storage strategy 1
=======================
All field values are stored in a directory structure mirroring
structure of PloneSite. Backup files are stored in a flat structure.
* Filename of these stored values: Filename of field value or field
name if not defined
* Filename of rdf file: `<field name>.rdf`
* Filename of backup values: `<uid of content>_<field name>.bak`
Rdf files are not backed up. They are automatically generated
Example of storage::
fssDirectory
|
|- members
|
|- john
|
|- dummy-document
|
|- image
| |
| |- moutain.jpg
| |
| |- image.rdf
|
|- file
|
|- diary.odt
|
|- file.rdf
fssBackupDirectory
|
|- 9efeb7638fb35f5f9b9955da3e2dbfec_file.bak
Site storage strategy 2
=======================
All field values are stored in a directory structure mirroring
structure of PloneSite. Backup files are stored in a flat structure.
* Filename of these stored values: Filename of field value or field
name if not defined
* Filename of rdf file: `<field filename>.rdf`
* Filename of backup values: `<uid of content>_<field name>.bak`
Rdf files are not backed up. They are automatically generated
Example of storage::
fssDirectory
|
|- members
|
|- john
|
|- dummy-document
|
|- fss.cfg
|
|- moutain.jpg
|
|- mountain.jpg.rdf
|
|- diary.odt
|
|- diary.odt.rdf
fssBackupDirectory
|
|- 9efeb7638fb35f5f9b9955da3e2dbfec_file.bak
FSS main configuration
######################
FSS is mainly configured the ZConfig way. At startup, the
configuration file will be searched (in that order) in:
* $INSTANCE_HOME/etc/plone-filesystemstorage.conf
* /path/to/iw/fss/etc/plone-filesystemstorage.conf
* /path/to/iw/fss/etc/plone-filesystemstorage.conf.in
A sample working configuration in provided in this last file. It
assumes you have $INSTANCE_HOME/var/fss_storage and
$INSTANCE_HOME/var/fss_backup directories, both being read/write
enabled to the user that runs the Zope process, unless Zope will raise
an error at startup.
All configuration doc you need is in `plone-filesystemstorage.conf.in`
comments.
Inconsistent configuration features raise explicit error messages at
Zope startup.
Configuration panel
###################
FSS and RDF
===========
As mentioned above in `Storage strategies`_, FSS can optionally store
RDF files in the back-end storage. These RDF files conform the
`DCMES-XML standard`_ on XML expression of DublinCore elements.
If you select the `Flat storage strategy`_, the RDF files may be used
to build a files tree as close as possible of the Plone tree structure
from the storage back-end, with the `build_fs_tree.py` utility::
$ cd /path/to/iw/fss/bin
$ python build_fs_tree.py --help
FSS maintenance
===============
The "Maintenance" tab of FSS configuration panel shows some statistics
about the amount of files managed by FSS.
In addition 2 additional buttons are provided:
* **Update FSS files** : Cleans up the back-end storage
directories. Files not referenced from a content object are removed.
* **Update RDF files** : Builds all RDF files, in the event you chose
to select the generetion of RDF files after having inserted relevant
contents.
Zope backups
############
Of course, you need to synch your Data.fs backups with the storage
paths backups, unless you may restore corrupted/incomplete sites.
Caveats and pittfalls
#####################
Note that whatever's the selected strategies, **DO NEVER CHANGE ANY
FILE IN THE STORAGE PATHS UNLESS YOU KNOW WHAT YOU DO**, otherwise
you'll loose your contents, your job, your money, your friends, your
wife/husband and children.
**DO NEVER CHANGE THE STRATEGY FOR A PLONE SITE ONCE THIS SITE HAS
FIELDS STORED IN ITS FSS STORAGE PATH** unless you ran the appropriate
strategy migrator - see `Migrating between strategies`_ otherwise
you're loose your contents, your job, ...
Thought it is possible to share the same storage path and backup path
within various Plone sites, since you don't mind of your data in
development sites, you should really configure FSS such each Plone
site has its own private storage path and backup path. In cas you
ignore this warning, **DO NEVER CLICK ANY BUTTON IN THE MAINTENANCE
TAB OF THE FSS CONFIGURATION PANEL** otherwise you'll loose your
contents, your job, ...
Zexp exports **don't embed the associated FSS storage**. So don't move
your Plone site within Zope instances using zexp exports unless you
move the storage and backup directories along with the zexp file.
After changing your configuration (see `FSS main configuration`_),
always restart your Zope instance in foreground since configuration
errors are not reported when Zope is started as daemon or Windows
service.
Developer's corner
##################
Using FSS in your content types
===============================
We assume that creating AT based content types is a familiar activity.
When creating the schema of a content type, you usually don't mention
the storage of your fields since AT provides a default ZODB
storage. You can choose another storage, either another one that ships
in the Archetypes bundle, or a third party storage like FSS.
Here is a small example that doesn't need much comments::
# Usual Zope/CMF/Plone/Archetypes imports
...
from iw.fss.FileSystemStorage import FileSystemStorage
...
my_schema = Schema((
FileField('file',
...
storage=FileSystemStorage(),
widget=FileWidget(...)
),
...
))
...
You may have a look at `examples/FSSItem.py` to see a demo content
type using FSS. If you want to play with this content type::
$ $INSTANCE_HOME/bin/zopectl stop
$ export FSS_INSTALL_EXAMPLE_TYPES=1
$ $INSTANCE_HOME/bin/zopectl start
Monkey patching a third party content type
==========================================
The Python way
++++++++++++++
This works with any Zope version listed in the Requirements_. If you
run a Zope 2.9 or later instance, you should prefer `The ZCML way`_.
This simple example shows how to plug the ATFile standard type from
any custom product. Let's say we're in the `__init__.py` of your
custom product::
...
from iw.fss import zcml as fss_utils
from Products.ATContentTypes import atct
...
fss_utils.patchATType(atct.ATFile, ['file'])
...
The ZCML way
++++++++++++
This works only with Zope 2.9 and up, and is the recommanded way to
wire your AT based content types with FSS since this way hides the API
details that may be deprecated in the future.
You just need to add the "fss" namespace support and some directives
in your "configure.zcml" like this to have the same effect as in the
example of `The Python way`_ above::
<configure
...
xmlns:zcml="http://namespaces.zope.org/zcml"
xmlns:fss="http://namespaces.ingeniweb.com/filesystemstorage"
...>
...
<fss:typeWithFSS
zcml:condition="installed iw.fss"
class="Products.ATContentTypes.atct.ATFile"
fields="file" />
...
</configure>
The (required) attributes of this directive are:
* **class**: the dotted name of the AT based content type class.
* **fields**: one or more (space separated) field names to wire with FSS.
Storing Plone standard content types with FSS
=============================================
Include this ZCML element either in your instance `package-includes`
directory or in any personal `configure.zcml` in your instance
packages::
<include package="iw.fss" file="atct.zcml" />
This ZCML file wires FSS with the relevant content types that ship with Plone bundle:
* File
* Image
* News Item
Customize RDF info set
======================
You can add data to the RDF info set via a FSS dedicated - and
optional - hook.
Open the FSS configuration panel and give the name of your personal
RDF data script, let's call it `fss_custom_rdf` for this example. This
script will add or change the RDF default data set.
RDF script interface
++++++++++++++++++++
The script may be in any Plone layer like `custom` or any other in the
skins path.
Such script is expected to have 4 parameters:
* `name`: the name of the field
* `instance`: the content object being processed
* `properties`: FSS field properties mapping, depending on the type of
field associated to FSS storage.
* `default_rdf`: a mapping structure of the RDF data, including the
namespaces, and the elements.
Such script is expected to return an updated `default_rdf` mapping
structure.
Please read the source of classes `FSSInfo`, `FSSFileInfo` and
`FSSImageInfo` from module `FileSystemStorage.py`, `RDFWriter` from
module `rdf.py` for detailed information on formats of expected data.
Simple example
++++++++++++++
We need to add the date of RDF file generation in an element like
this::
<fss:rdf_date>2007-05-13 13:21:00</fss:rdf_date>
Here's the script::
## Script (Python) "fss_custom_rdf"
##bind container=container
##bind context=context
##bind namespace=
##bind script=script
##bind subpath=traverse_subpath
##parameters=name, instance, properties, default_rdf
##title=
##
"""We add the date of RDF file generation"""
rdf_args = default_rdf.copy()
# Add the date of RDF generation
new_props = []
new_props.extend(rdf_args['field_props'])
new_props.append({'id': 'fss:rdf_date', 'value': DateTime().ISO()})
rdf_args['field_props'] = new_props
return rdf_args
More complex example
++++++++++++++++++++
In this more complex (and stupid - don't do this ;) example, we remove
the elements from `fss` namespace and simulate the standard behaviour
for other elements (`dc` and `rdf` elements).
Here's the code::
## Script (Python) "fss_custom_rdf"
##bind container=container
##bind context=context
##bind namespace=
##bind script=script
##bind subpath=traverse_subpath
##parameters=name, instance, properties, default_rdf
##title=
##
from Products.CMFCore.utils import getToolByName
rdf_args = {}
# Set RDF charset
ptool = getToolByName(instance, 'portal_properties')
rdf_args['charset'] = ptool.site_properties.default_charset
# Set RDF namespaces
rdf_args['namespaces'] = (
{'id': 'xmlns:rdf', 'value': 'http://www.w3.org/1999/02/22-rdf-syntax-ns#'},
{'id': 'xmlns:dc', 'value': 'http://purl.org/dc/elements/1.1/'},
{'id': 'xmlns:fss', 'value': 'http://namespace.ingeniweb.com/fss'},)
# Set field url
utool = getToolByName(instance, 'portal_url')
portal_path = utool.getPortalObject().getPhysicalPath()
portal_path_len = len(portal_path)
rel_path = '/'.join(instance.getPhysicalPath()[portal_path_len:])
fss_path = '%s/%s' % (rel_path, name)
rdf_args['field_url'] = fss_path
# Set RDF properties
props = (
{'id': 'dc:title', 'value': instance.title_or_id()},
{'id': 'dc:description', 'value': instance.Description()},
{'id': 'dc:language', 'value': instance.Language()},
{'id': 'dc:creator', 'value': instance.Creator()},
{'id': 'dc:date', 'value': instance.modified()},
{'id': 'dc:format', 'value': properties.get('mimetype', 'text/plain')},
)
rdf_args['field_props'] = props
# Returns the RDF structure
return rdf_args
Recommanded practices on custom RDF
+++++++++++++++++++++++++++++++++++
Do not change the existing elements under `fss` namespace unless the
`bin/build_fs_tree.py` utility won't work. In other terms, do not use
the script of the `More complex example`_.
Adding or changing elements to the `dc` and `rdf` namespaces may screw up
external utilities based on canonical `DCMES-XML standard`_.
Consider adding your own namespace for your custom extra elements.
Migrating from AttributeStorage or AnnotationStorage
####################################################
We assume that, at this point, you got :
* A production Plone site filled with contents which storage is supported by the
standard AT storages (AttributeStorage or AnnotationStorage).
* A development site with the **same** AT based content types (ATCT plus
others), `iw.fss` and a ZCML setup that stores fields of some contents with
`iw.fss`. See how to do it `The ZCML way`_ or use the `atct.zcml` file that's
included in this package.
Backup your production site
===========================
Yes, at this early stage, we didn't test migration with lots and lots of content
types. Only the basic OTB content types have been tested. But this should work
with any other. Whatever, you should always backup a site before proceeding to a
deep migration like this one.
Add the `iw.fss` settings
=========================
Install `iw.fss` and the ZCML settings from your develoment site on your
production site. Note that at this step, it's safer to deny access to the users
(use Apache settings or whatever).
Start your Zope instance in forground mode (recommanded if you want to follow
the migration logging).
Do the migration
================
Open the FileSystemStorage configlet. Check that the main control panel reports
the expected settings (storage paths and strategies, supported content types and
fields).
Click the "migration" tab of this control panel, check the options, and proceed
to migrations.
Pack the ZODB
=============
After a successful migration, you may pack immediately the ZODB to appreciate
the effect of using `iw.fss`.
FAQ
###
Migrating between strategies
============================
Use the `bin/strategymigrator.py` shell utility that ships with FSS. Get
more info on this utility using the `--help` option.
Note that only flat -> directory and directory -> flat are available
today. Sponsorship is welcome - as indicated in `Support and
feedback`_ - if you need other strategy migrations.
Where are the files for this item?
==================================
Sometimes we need to know what files store the fields of a content item. For
this, login as site manager and browse to
http://my.site.tld/where/is/my/content/@@fss_where_are_my_files
Testing
#######
Please read `./tests/README.txt`.
Other documentation
###################
See `./doc/*` and `./locales/README.txt`
Downloads
#########
You may find newer stable versions of FSS and pointers to related
informations (tracker, doc, ...) from
http://pypi.python.org/pypi/iw.fss
Subversion repository
#####################
Stay in tune with the freshest (maybe unstable) versions or participate to
the FileSystemStorage evolutions:
https://svn.plone.org/svn/collective/iw.fss
Support and feedback
####################
Please read all the documentation that comes with this product before
asking for support, unless you might get a RTFM reply ;)
Localisation issues - other than french - should be reported to the
relevant translators (see Credits_ below).
Report bugs using the tracker (the `Tracker` link from
http://plone.org/iw.fss). Please provide in your
bug report:
* Your configuration (Operating system+Zope+Plone+Products/versions).
* The storage strategy in use.
* The full traceback if available.
* One or more scenario that triggers the bug.
Note that we do not support bug reports on Subversion trunk or
branches checkouts.
`Mail to Ingeniweb support <mailto:support@ingeniweb.com>`_ in English or
French to ask for specific support.
`Donations are welcome for new features requests
<http://sourceforge.net/project/project_donations.php?group_id=74634>`_
Credits
#######
* Main developer: `Cyrille Lebeaupin <mailto:cyrille.lebeaupin@ingeniweb.com>`_
* ZConfig and ZCML support: `Gilles Lenfant <mailto:gilles.lenfant@ingeniweb.com>`_
* HTTP "Range" support: `Youenn Boussard <mailto:youenn.boussard@ingeniweb.com>`_
* Paris 2008 sprint additional contributors:
- `Mehdi Benammar <mailto:mehdi.benammar@kitry.lu>`_
- `Romain Griffiths <mailto:wid@epita.fr>`_
See `locales/README.txt` about localization credits.
--------------------------
.. sectnum::
.. _Ingeniweb: http://www.ingeniweb.com/
.. _AttachmentField: http://plone.org/products/attachmentfield
.. _DCMES-XML standard: http://dublincore.org/documents/dcmes-xml/
.. _iw.recipe.fss: http://pypi.python.org/pypi/iw.recipe.fss
.. $Id: README.txt 226275 2010-11-09 09:57:03Z glenfant $
CHANGES
#######
2.8.0rc5
========
* Fixed a critical issue where an image would be attempted restored at
every request. [malthe]
* Fixed an issue where the AT tool for scaling images would pass a
unicode file path which would result in a bug because the code was
only looking for a regular string (else deciding that it's a stream
object). [malthe]
* Fixed an issue with HTTP range leading to a missing file size. With
some clients (seen in Firefox 3.6), the ``end`` value is trivial and
must be explicitly set to the file size before returning the
iterator. [malthe]
2.8.0rc4
========
Plone 4 support
* "from Globals import InitializeClass" <- "from App.class_init ..." closes
deprecation warning.
* AT moved from Zope2 style to Zope3 style interfaces since the old Zope 2
"Interfaces" package has gone away with Zope 2.12. We have a workaround that
supports both styles.
[glenfant]
* (Deprecated) tabindex removed from all templates.
[glenfant]
* Added a view for content items that allows managers to see where in the file
system the fields are stored.
[glenfant]
2.8.0rc3
========
* Change the behaviour of upload on fss. No more memory consuption for that
now. We use always use fileiterator when inspecting big file. To achieve to
that, VirtualDate override _read_data of OFS.Image.File to construct an clean
FSSPdata.
[yboussard]
* Monkey patching with collective.monkeypatcher. All monkey patches should be
made that way.
[glenfant]
* Using z3c.autoniclude when available.
[glenfant]
* Change behaviour of VirtualData class. Now VirtualData work as OFS.image ie to
acces data you MUST pass by a chained list (FSSPdata) that contain only a
block of data. This block of data is getting by a filestream iterator. So no
more memmory consuption (max 65536 , a block of data). This is an important
change because when Products.Archetype.Field.FileField get metatada of a File
old version of fss get all file data for this (ie myField.getFilename() -> you
have your all data in memory !!). Also when you invoke
Field.download(instance, no_output = True) you get now a filestream iterator
(There is a patch to this method because of dowload must set content-length
header for the publisher)
[yboussard]
* fix content_type retreving for migration process.
[yboussard]
2.8.0rc2
========
* Fixed crappy error logging on migration
[glenfant]
2.8.0rc1
========
* Worked around (and cancel migration) when AT tries (and fails) stupidly to
guess the charset of a binary data.
[glenfant]
* Spanish translation is up to date
[dmunicio]
2.8.0b2
=======
* Fixed bad mime type setting when migrating.
[glenfant]
* Fixed loss of text when migrating News items with image
[glenfant]
* Fixed logging on transaction when migrating and show the total migrated
items in the log too.
[glenfant]
2.8.0b1
=======
* Upgraded translations (.pot and french translation)
[glenfant]
* Fixed issues reported by test users.
[glenfant]
* Faster and safer method to guess the mime type when migrating.
[glenfant]
2.8.0a3
=======
* Fixed a NameError in migrations that could not be raised by unit tests.
[glenfant]
2.8.0a2
=======
* Fixed an issue on optional (non "required") fields: getting an empty
field must raise an AttributeError and not return an empty string.
[glenfant]
* ATCT settings disabled
[glenfant]
2.8.0a1
=======
* Adding migration utility (XStorage -> FSS)
[glenfant]
2.7.6
=====
* Safety belt against GS irrelevant upgrades.
See https://dev.plone.org/plone/ticket/8507
[glenfant]
* fix i18n initialization under win32
[bmathieu]
2.7.5
=====
* removed fss_get in VirtualBinary urls: this old script has been removed with
skins. This caused problems with content such as plone news item calling
image.tag().
[b_mathieu]
* run tests without shell variable ZOPE_TESTCASE
[b_mathieu]
2.7.4
=====
* Fix a bug on versioning : previous version of FSS was unable to deal with
content having archetypes fields with no storages
[clebeaupin]
2.7.3
=====
* GenericSetup upgrade step (any version) -> 1.0
[glenfant]
* Red warnings when using the default common config in config panels
[glenfant]
* Config panel enhancements (config details view, documentation)
[glenfant]
* Code cleanup (useless stuffs)
[glenfant]
* Fixed test cases
[cbosse] [glenfant]
* CMFEditions compatibility
[glenfant]
* Annotations replacement for the attributes (FSSInfo), and some
cosmetics.
[mehdi]
* CMF skins layer removed in favour of views.
[wid]
* Add a test verifying that the storage path contains one item after
adding a file, and zero items after deleting it.
[maikroeder]
* "site1" and "site2" strategies are forfidden as global strategies
[glenfant]
* 100% GenericSetup installation
[clebeaupin]
2.7.2
=====
* Fix the bobo traverse mess: we should not honour Plone 2.5(-)
compatibility (remove patches.py) and fix the traverse deprecation
warning.
[glenfant]
* pyflakes everything.
[glenfant]
* Removed Plone 2.5 (and previous) compatibility.
[glenfant]
* Protect with appropriate permission ZMI views/methods.
[glenfant]
2.7.1 - 2008/03/17
==================
* Test case for FSS registration.
[glenfant]
* Hint in README.txt to set FSS storage only if iw.fss is installed.
[glenfant]
* The ZMI "Overview" tab shows the various types changed to use FSS.
[glenfant]
* I18N domain is now "iw.fss" being consistent with new styles
components.
[glenfant]
* Translations moved to zope 3 style in place of PTS style. As a
consequence, the config panel title is not translated any more
("plone" domain).
[glenfant]
* Public function to patch a type and registering patched types
for migration purposes
[glenfant]
* Renamed flat2directory.py into strategymigrator.py since future
other strategy migrations will go there.
[glenfant]
* Added bin/flat2directory.py that migrates "flat" strategy storage to
"directory" strategy storage.
[glenfant]
* Added discriminator in ZCML support that warns on duplicate config
of the same content type.
[glenfant]
2.7.0 - 2008/02/26
==================
* Convert FileSystemStorage Zope 2 product to iw.fss python package
[tarek]
* Added ZCML namespace/directive to wire fields of content types with
FSS.
[glenfant]
2.6.2 - 2007/12/07
==================
* Fixed range header processing: if one range - specific file stream
iterator if multiple range - OFS.image traitment (1680760
sourceforge tracker) Add some unit test to test range header
2.6.1 - 2007/10/29
==================
* ZConfig bug that lowercases some config strings: we test path in
lowercase too (ED)
* Remove the OSError when moving file into backup
* Plone 3.0 compatibility [glenfant]
* Do not override Content-Disposition HTTP header if already set in
VirtualBinary.index_html (zegor)
* filestream_iterator is not "range proof". Add a new
range_filestream_iterator to negociate Range request
2.6.0 RC1 - 2007/06/15
======================
* Changed in depth the way we configure FSS with the ZConfig
way. Changing the storage and backup path, and the storage strategy
TTW is disabled [glenfant]
* As a consequence, changing the storage strategy doesn't require to
hack config.py any more [glenfant]
* As a consequence, each Plone site in a Zope instance can have its
own storage strategy [glenfant]
* Configuration view and README.txt view in ZMI [glenfant]
* Fixed documentation issues [glenfant]
* SiteStorageStrategy and SiteStorageStrategy2 didn't work under Win32
[glenfant]
* Make FSS compatible with Plone 2.0 : just rewrite some testcases
2.5.6 - 2007/06/15
==================
* When renaming a folder containing FSS content, FSSInfo attribute was
stolen one each content of the folder. Provides a solution to avoid
this problem (CL)
2.5.5 - 2007/06/13
==================
* Make FSS works when calling _setUID method on an AT object (CL)
* Add a strategy mark in the storage in a '.strategy' file such we
could forbid strategy changes of storages at Zope startup. (i.e.,
using a 'flat' storage strategy in a storage filled using a
'directory' strategy)
2.5.4 - 2007/03/13
==================
* Fix a bug on Plone 2.5 when moving a folder containing items using
FSS storage. Those items were losing their FSS content. (CL)
2.5.3 - 2007/02/12
==================
* Don't monkey patch BaseObject __bobo_traverse__ under Plone 2.5, but
register ITraversable adapter for this. This change is required as of
Archetypes 1.4.1 (see AT HISTORY.txt, search '__bobo_traverse__')
2.5.2 - 2006/10/27
==================
* Fix bug on maintenance tab: update files and backup, and update RDF (ED)
* Fix some renaming bugs (ED)
* Extend index_html method of VirtualFile and VirtualImage to use a
filestream iterator. FSS is now compliant with CacheFu
* Fix bug in FSSTool:updateFSS - replace restoreBackup method by
restoreValueFile
* Add new strategy: SiteStorageStrategy2
2.5.1 - 2006/07/28
==================
* Check existence of backup and storage paths in configlet
* Fix bug on maintenance tab
2.5 - 2006/07/19
================
* Update tests to use all strategies
* Add storage strategies : FlatStorageStrategy,
DirectoryStorageStrategy, \ SiteStorageStrategy
* Patch BaseObject to make it possible to access images like this
mysite/myobj/image or mysite/myobj/image_mini
* Added a docstring on VirtualData class and subclasses. This should
fix http://plone.org/products/atphoto/issues/28 : VirtualImage can
be published directly by ATImage
2.5RC1
======
* Add a hook to customize RDF files
* Add script to build filesystem tree from FSS files
* Add RDF implementation
2.1RC2
======
* Remove use of ComputedAttribute to avoid strange errors
* If file is empty don't create an empty file on filesystem
2.1RC1
======
* Add Content-Disposition header in fss_get python script
* Make FSS compliant with AT 1.3.4 and Plone 2.1
2.0RC3
======
* Fix bug on delete action (In previous version, it was impossible
to delete non required field value)
2.0RC2
======
* Add translations
* Fix bug in configlet when no files are stored on filesystem.
2.0RC1
======
* First public release
-------------
.. $Id: CHANGES 226275 2010-11-09 09:57:03Z glenfant $
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
iw.fss-2.8.0rc5.tar.gz
(506.2 kB
view hashes)