django-phpBB3 0.1.6

django database models of phpBB3

Django-phpBB3

Django-phpBB3 provides the database models of a existing phpBB3 installation. This is usefull for all people who like to access the phpbb forum database from a django application. It’s not a phpBB clone or a forum!

Django-phpBB3 has also a script for migrate a phpBB3 forum into a DjangoBB forum. See below.

Notes:

• There are a few mismatches between the origin phpBB3 Database schema and the models.py form Django-phpBB3. Because of this: Creating the phpBB3 tables via syncdb will not work with phpBB3.

  • phpBB3 used often 0 for a not set value, but we need a real null value in db (to get None back)

• phpBB3 Models with composite keys are unsupported, but stored in django_phpBB3/unsupported_models.py

Warning: Write access to the original phpBB database is not tested, yet. So it’s not guaranteed that changes trough the django admin will work with phpBB3! So, in the current state, you should only access ‘read-only’ to the origin phpBB3 data ;)

Patches are welcome! Please send pull requests.

example projects

We add two example projects:

`django_phpBB3_project`_	minimal project without extra depencies
`phpBB2DjangoBB_project`_	DjangoBB example project for use the phpBB2DjangoBB migration command

setup

~$ virtualenv DjangoBB_env
~$ cd DjangoBB_env/
~/DjangoBB_env$ source bin/activate
(DjangoBB_env)~/DjangoBB_env$ pip install django-phpBB3 # Will also install Django

If you would like to test/use the DjangoBB migration install this:

(DjangoBB_env)~/DjangoBB_env$ pip install -e git+git://github.com/slav0nic/DjangoBB.git#egg=DjangoBB
(DjangoBB_env)~/DjangoBB_env$ pip install Whoosh

init database

If you will test the two example projects, you need to initialises the database, e.g.:

~$ cd DjangoBB_env/
~/DjangoBB_env$ source bin/activate
(DjangoBB_env)~/DjangoBB_env$ cd src/django-phpbb3/phpBB2DjangoBB_project
(DjangoBB_env).../phpBB2DjangoBB_project$ ./manage.py syncdb --all
(DjangoBB_env).../phpBB2DjangoBB_project$ ./manage.py migrate --fake

test django-phpBB3

We added a test project for easy start in: /django_phpBB3_project/

Create a local_settings.py in /django_phpBB3_project/ At least you must setup DATABASES, e.g:

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.mysql', # Add 'postgresql_psycopg2', 'mysql', 'sqlite3' or 'oracle'.
        'NAME': 'phpbb3', # Or path to database file if using sqlite3.
        'USER': 'phpbb3', # Not used with sqlite3.
        'PASSWORD': 'foo bar', # Not used with sqlite3.
        'HOST': '', # Set to empty string for localhost. Not used with sqlite3.
        'PORT': '', # Set to empty string for default. Not used with sqlite3.
    }
}

optional settings, that you can change in your local_settings.py are:

# The prefix of the phpBB3 installation - default: u"phpbb3_"
PHPBB_TABLE_PREFIX = u"phpbb3_"

You must initialises the database (see above).

You can startup a test server with, e.g:

./manage.py runserver --traceback --insecure

migrate a phpBB3 installation to DjangoBB

The migration from phpBB3 to DjangoBB is implement as a migration command here: /django_phpBB3/management/commands/phpbb2djangobb.py

To run the migration you can use the minimal phpBB2DjangoBB example project example. This project includes the DjangoBB forum app and django-phpBB3 app. Before start, you must install the dependencies!

You must create a local_settings.py into /phpBB2DjangoBB_project/. At least you need these settings:

# filesystem path to the /files/ sub directory of the phpBB installation:
PHPBB_ATTACHMENT_PATH = "/path/to/phpBB/files/"

# database table prefix (default is: "phpbb3_")
#PHPBB_TABLE_PREFIX = "phpbb_"

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.mysql', # Add 'postgresql_psycopg2', 'mysql', 'sqlite3' or 'oracle'.
        'NAME': 'phpbb3', # Or path to database file if using sqlite3.
        'USER': 'phpbb3', # Not used with sqlite3.
        'PASSWORD': 'foo bar', # Not used with sqlite3.
        'HOST': '', # Set to empty string for localhost. Not used with sqlite3.
        'PORT': '', # Set to empty string for default. Not used with sqlite3.
    }
}

(You must also point the attachment directory from DjangoBB to a existing filesystem path.)

Before you start: You should check if database encoding settings are ok. e.g. in default ubuntu/debian setup a mysql server will use latin-1 ! So before you start, you should correct this. e.g.:

ALTER DATABASE test DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci

Make a SQL Backup before doing anything!!!

Run the migration with ./manage.py phpbb2djangobb, e.g:

~$ cd DjangoBB_env/
~/DjangoBB_env$ source bin/activate
(DjangoBB_env)~/DjangoBB_env$ cd src/django-phpbb3/phpBB2DjangoBB_project
(DjangoBB_env).../phpBB2DjangoBB_project$ ./manage.py phpbb2djangobb

after migration

You must update the haystack search index with e.g.:

(DjangoBB_env).../phpBB2DjangoBB_project$ ./manage.py update_index --verbosity=2

more information: http://django-haystack.readthedocs.org/en/latest/management_commands.html

Depend on the used database you must reset sequences.

Sequences are indexes used by some database engines to track the next available number for automatically incremented fields.

more info: https://docs.djangoproject.com/en/dev/ref/django-admin/#sqlsequencereset-appname-appname

options

cleanup_users

Add --cleanup_users=x to setup which users are migrate, where x is a number:

0	all users
1	users with email addresses (skip bots)
2	users with email and has a ‘lastvisit’ date (default)
3	email + lastvisit and has created at least one post

migration limitations

Things that would not be transfered, yet:

• Forum moderators

• Category groups

• sort of Category / Forum would not be converted to position number

Handling missing features in DjangoBB:

• We transfer phpBB Users who are in the user group ADMINISTRATORS or GLOBAL_MODERATORS to all forum moderators.

• global posts and announcement converted to normal sticky posts

• sub forums would be flatten

• ghost moved posts are ignored (only the moved post would be created)

• ForumWatch are ignored

• All user must do a passwort reset. (We can’t reuse the phpBB password hash)

• Guest posts would be mapped to a new “Anonymous” user (has unusable password and no email address). Note: guest posts are planed in DjangoBB, see: http://support.djangobb.org/topic/98/

TODO:

• How to handle non-ascii characters in nick names?

• Bugfix for user post count migration - https://github.com/jedie/django-phpBB3/issues/6

• migrate user avatar - https://github.com/jedie/django-phpBB3/issues/7

• add a separate command for migrate phpBB private messages to django-messages (very low priority, patches are welcome)

migration - open questions

• Are the phpBB times in phpbb_posts in UTC oder in local time from user?

phpBB3 links

Here some links for phpBB documentations:

• http://wiki.phpbb.com/Tables

• http://www.phpbbdoctor.com/doc_tables.php

The SQL schema files from phpBB can be found here:

• https://github.com/phpbb/phpbb3/tree/develop/phpBB/install/schemas

history

• v0.1.6

  • Add more information in README

  • Fix unicode errors

• v0.1.5 - 31.07.2012

  • migrate user language, too.

• v0.1.4 - 30.07.2012

  • many bugfixes in models.

• v0.1.3 - 27.07.2012

  • catch error if a post can’t be migrated and continue migration

  • remove django_authopenid and django_messages from phpbb2djangobb project settings

  • nicer output while migrating with eta time

• v0.1.2 - 27.07.2012

  • remove date from version string

  • add this histroy list ;)

• v0.1.0 - 24.07.2012

  • migrate post subscriptions, too.

• 23.07.2012

  • migrate phpBB3 file attachments, too.

  • bugfixes in bbcode cleanup

  • handle html entities in cleanup

• 20.07.2012

  • bbcode cleanup contributed by nitely

• 19.07.2012

  • start with phpBB3 migration command

• 16.07.2012

  • project starts by Jens Diemer

Project links

Homepage	https://github.com/jedie/django-phpBB3
IRC	#pylucid on freenode.net
Forum Thread	http://support.djangobb.org/topic/300/
PyPi	http://pypi.python.org/pypi/django-phpBB3/