jarn.xmpp.core 0.34

Core package for jarn.xmpp

Introduction

jarn.xmpp.core is a Plone add-on providing the following functionality based on XMPP services:

• Integration of plone user accounts with XMPP accounts and authentication.

• A microblogging environment similar to twitter/yammer based on XMPP PubSub.

• Messaging.

It is part of a suite of packages aiming to provide XMPP services to Plone. The other two packages are

• jarn.xmpp.twisted, provides XMPP-specific protocol implementation for twisted.

• jarn.xmpp.collaboration provides an XMPP protocol to do real-time collaborative editing as well as a Plone-targeted implementation.

Installation

Before setting up the package you need to have a working XMPP server and access to the administration account on the server. The package has only been tested with ejabberd version 2.1.5 and above which is recommended. In any case the following XMPP extensions need to be supported by the server you are going to use:

• XEP-0071 XHTML-IM.

• XEP-0054 vCard-temp.

• XEP-0144 Roster Item Exchange.

• XEP-0060 Publish-Subscribe.

• XEP-0248 PubSub Collection Nodes.

• XEP-0133 Service Administration.

• XEP-0124 Bidirectional-streams Over Synchronous HTTP (BOSH)

• XEP-0206 XMPP over BOSH

• XEP-0049 Private XML Storage

Buildout

A sample buildout you can use as a starting point can be found at jarn.xmpp.buildout.

Setting up ejabberd (>=2.1.5)

Automatic configuration

• Use the recipe provided in jarn.xmpp.buildout (in which case you will need to have erlang installed) or download the ejabberd installer.

• A minimal configuration for ejabberd can be generated for convenience by the ejabberd.cfg part of jarn.xmpp.buildout. You will need to copy the templates directory and modify the recipe configuration accordingly:

  [ejabberd.cfg]
  recipe = collective.recipe.template
  input = templates/ejabberd.cfg.in
  output = ${buildout:directory}/etc/ejabberd.cfg
  xmppdomain = localhost
  admin_userid = admin
  collaboration_allowed_subnet = 0,0,0,0
  collaboration_port = 5347
  component_password = secret

where xmppdomain is the domain (or virtual host) running on your XMPP server and admin_userid is the id the the administrator account that Plone is going to use to interact with the server. The rest of the options are used by jarn.xmpp.collaboration for the collaborative editing component connecting to the XMPP server. Here, collaboration_allowed_subnet specifies from which IPs the XMPP server is going to accept connections and should match the IPs your Plone instances are going to be using. Leaving it to 0,0,0,0 will allow all IPs, 127,0,0,1 will allow only localhost. Finally collaboration_port is the port to which the collaboration component is going to connect to and component_password is the shared password between the component and the XMPP server.

Manual configuration

If you already run an XMPP server here are some hints on how to set it up:

• We assume that your xmpp domain is localhost. There should exist an administrator account admin@localhost. In addition if you intend to run some of the tests in any of the jarn.xmpp.* packages you will need to be running an additional XMPP node on localhost if you use some other domain for production. You can safely remove any references to localhost if you are not interested in doing that.

• Make sure you have enabled the http_bind module, as this is what the javascript clients will use to connect. You should have something like this in your ejabberd.cfg:

  {5280, ejabberd_http, [
       http_bind,
       web_admin
       ]}

• Because ejabberd’s implementation of XEP-0060 is not standard use of the ejabberd’s dag module is necessary. So, make sure your pubsub module is configured appropriately:

  {mod_pubsub,   [
      {access_createnode, pubsub_createnode},
      {ignore_pep_from_offline, true},
      {last_item_cache, false},
      {nodetree, "dag"},
      {plugins, ["dag", "flat", "hometree", "pep"]},
      {max_items_node, 1000}
      ]},

• In order to test and run custom XMPP components (for instance the collaborative editing component provided by jarn.xmpp.collaboration) you will need to allow them to connect. This means you should have something similar to this configuration:

  {5347, ejabberd_service, [
            {access, all},
            {shaper_rule, fast},
            {ip, {127, 0, 0, 1}},
            {hosts,
             ["collaboration.localhost"],
             [{password, "secret"}]
            }
           ]},

The rest of the standard options should be fine.

Administrator account

If you have not done so during installation you might need to create manually the administrator account. In the ejabberd folder execute:

./bin/ejabberdctl register admin localhost your_password

Test that you can access your ejabberd by logging to the admin interface (typically http://localhost:5280/admin). You should also be able to access the http-bind interface at http://localhost:5280/http-bind.

Setting up your front-end proxy

On the client-side every authenticated user will be connected to your jabber server through an emulated bidirectional stream through HTTP. To allow for this you need a proxy in front of Plone that will be redirecting the XMPP stream to your XMPP server. It is possible to do without one using the inferior solution of Flash plugins but this is not going to be supported.

So assuming you run nginx as a proxy at port 8080 for the domain localhost, Zope listens on 8081, there exists a Plone site with id Plone and your ejabberd has the http_bind configured for port 5280, your nginx configuration will look like this:

http {
    server {
        listen       8080;
        server_name  localhost;
        location ~ ^/http-bind/ {
            proxy_pass http://localhost:5280;
        }

        location / {
            proxy_pass http://localhost:8081/VirtualHostBase/http/localhost:8080/Plone/VirtualHostRoot/;
        }
    }
  }

Setting up your Plone instances

Your instances will need to maintain a connection to the administrator account of your XMPP server. This is accomplished through Twisted and you will need to run a Twisted reactor on each of them. To do so include this in your instance section of your buildout:

zcml-additional =
  <configure xmlns="http://namespaces.zope.org/zope">
    <include package="jarn.xmpp.twisted" file="reactor.zcml" />
  </configure>

Again, it will help you to have a look at the sample buildout provided in jarn.xmpp.buildout.

Setting up a new Plone site

• Start ejabberd (if you used the recipe to build ejabberd, bin/ejabberd will do the job).

• Start the Nginx frontend. bin/frontend start

• Start your zope instance.

• Access Zope directly at http://localhost:8081/manage and create a new Plone site with jarn.xmpp.core (or jarn.xmpp.collaboration if you want that package installed as well).

• Go to the Plone control panel, into the registry settings. Edit the jarn.xmpp.* settings to reflect your installation, passwords etc.

• Restart your Plone instance.

• Upon the first request the administrator will log to his account. You should see things happening in the logs and if there are any errors something might be wrong with your installation.

• Setup the the users and pubsub nodes. You do this by calling @@setup-xmpp like http://localhost:8080/@@setup-xmpp. The form will not report any errors as everything will happen asynchronously but you will get the results/failures on the console.

If you are going to use this on an existing site, you only need to perform the last step after making sure that your XMPP admin is connected.

Making sure things work ;)

This is a complex infrastructure so it can be hard to know what goes wrong sometimes. Do not despair, here are a few things to try:

• Make sure your ejabberd is running. Connect to it normal client as the admin user.

• Verify that http-binding is setup properly on ejabberd. Visiting http://localhost:5280/http-bind should tell you it’s working.

• Verify that XMPP requests will get properly through your proxy. Visiting http://localhost:8080/http-bind/ should give you the same result as above.

• When you start your Zope instance in foreground you can verify the Twisted reactor is running fine:

  2011-09-01 14:37:38 INFO jarn.xmpp.twisted Starting Twisted reactor...
  2011-09-01 14:37:38 INFO jarn.xmpp.twisted Twisted reactor started
  2011-09-01 14:37:38 INFO Zope Ready to handle requests

• After the first request to the site, you should also see in the logs:

  2011-09-01 14:45:48 INFO jarn.xmpp.core XMPP admin client has authenticated succesfully.

• After having run @@setup-xmpp, logging-in to the Plone site with a user should also authenticate him with the XMPP server. This is indicated in the logs by:

  2011-09-01 14:45:50 INFO jarn.xmpp.core Pre-binded ggozad@localhost/auto-QravOoyEeE

Experimenting

Usage

• Add a few users.

• Login as one of them, and in a different browser as some other. Use the frontend to access the site, if you used the settings above this should be http://localhost:8080.

• All actions are performed through the viewlet on the top right: Online users will display the users currently logged in. Clicking it will give you the list of users. You can message them directly or look at their personal feed.

• Try posting an entry to your feed. Links will be transformed automatically. As soon as you submit other logged-in users will receive a notification in real-time. Using a recent browser that supports geolocation will allow you also share your location at the time of the post.

• Try commenting on a feed post.

• By clicking on the “Following” user action you can select which users you want to follow, or follow them all.

• You can see all posts by clicking on Site feed on the viewlet.

Security

jarn.xmpp.twisted includes an implementation of an authenticating client over BOSH according to XEP-0206. This practically means that the javascript client never needs to know the password of the XMPP user. Instead, the user is authenticated directly between the XMPP server and the Plone instance. A pair of secret tokens are exchanged, valid for a short time (~2 minutes). It is this pair that is given to the javascript client and not the password.

When a user is created (either through the Plone interface or by running @@setup-xmpp for existing users), a random password is generated and stored internally in a persistent utility.

If you do not need to access the XMPP accounts outside of the Plone instance you can additionally hide the entire XMPP service behind a firewall and only allow connections to it from the Plone instances. This in combination with HTTPS should be enough for the paranoid among us.

Testing

Some of the included tests are functional tests that require a XMPP server running on localhost as well as an administrator account setup up on this server with JID admin@localhost and password admin. If you wish to run those you have to specify a level 2 on your testrunner, i.e.

./bin/test -a 2 -s jarn.xmpp.core

Credits

• The UI was designed and implemented by Denys Mishunov.

• Most of this work was done using the 10% time available to Jarn AS employees for the development of open-source projects.

Changelog

0.34 - 2011-12-9

• Fix Google maps initialization. [ggozad]

0.33 - 2011-12-07

• IE9 compatibility. [ggozad]

• No-cache for BOSH authentication. [ggozad]

0.32 - 2011-11-30

• UI/UX improvements. [mishunov]

• Fixed a title-less portlet bug. [ggozad]

• Fixed a bug that would allow the storage to load only 10 items from each node. [ggozad]

• Fixed json views to set properly json headers. [ggozad]

0.31 - 2011-11-28

• Version fix. [ggozad]

0.3 - 2011-11-28

• Support one-level threading on pubsub nodes. [ggozad]

• Allow users to subscribe to individual nodes. [ggozad]

• Load PubSub items asyncronously. [ggozad]

• Implement XEP-0049 private storage on XMPP. [ggozad]

• Implement vCard 2.0 support. Allows all user meta-data/avatars to be stored on the XMPP server, avoiding all calls to Plone. [ggozad]

• Fix rare condition where a Safari browser would really misbehave and sometimes crash. Reverted patch to Strophe to allow WebKit browsers to make a synchronous instead of async call to disconnect on page unload. [ggozad]

• Styling! [mishunov]

• Update German translations [potzenheimer]

• Update french translation [toutpt]

0.23 - 2011-11-02

• Add German translations [potzenheimer]

• Added Italian translation [giacomos]

• Added French translation [toutpt]

0.22 - 2011-11-02

• i18n support. Norwegian and Greek for the time being. [ggozad]

• Provide a portlet that shows a PubSub feed. [ggozad]

• Fixed magic link error when the url points to a binary. https://github.com/ggozad/jarn.xmpp.core/issues/13 [ggozad]

0.21 - 2011-10-27

• Do not track user’s geolocation unless it’s necessary. [ggozad]

• Proxy the downloading of favicons (Closes #11). [ggozad]

• Link portrait to personal stream in feed views. (Closes #10). [ggozad]

• “Pretty dates” for the pubsub feed. (Closes #8). [ggozad]

0.2 - 2011-10-20

• Add geolocation capabilities. Posting to feeds can now include geolocation data. [ggozad]

• Use sessionStorage for user-info data, so that they are only fetched once per session. [ggozad]

0.2b1 - 2011-10-09

• Design and implement awesome UI for messaging, presence, pubsub feeds. [mishunov, ggozad]

• Replace python views/utilities for publishing to nodes by js. [ggozad]

• Provide a pubsub item view and load them asynchronously in the pubsub feed view. [ggozad]

• Start using sessionStorage in browsers where this is supported. [ggozad]

• Remove multi-user chat. It will go in again properly when the use-cases have been defined. [ggozad]

0.1b4 - 2011-09-30

• Revert to using Strophe 1.0.1 as a workaround against broken xml parser that insists on escaping everything. [ggozad]

0.1b3 - 2011-09-18

• Patch strophe to make synchronous requests in order to be able to really disconnect on onbeforeunload. This is to stop WebKit-based browsers from keeping users connected until they timeout. Also, no longer use minified versions of strophe libraries, since we patch them. [ggozad]

• Documentation updates. [tkimnguyen]

0.1b2 - 2011-09-02

• Fix startup subscriber to not do anything if the product has not been installed. This allows you to install on an existing site. [ggozad]

• Update to Strophe 1.0.2 [ggozad]

0.1b1 - 2011-09-01

• Simplified configuration and fixed confusing typos in the registry [ggozad]

• The admin client now handles failures to connect as well as getting disconnected. When this happens, the admin client utility will get unregistered and will retry on the next request. [ggozad]

• A viewlet is now available through which all actions can be performed. [ggozad]

• Added recipe in buildout to build ejabberd. Requires a working erlang installation. [ggozad]

• JS fixes [ggozad]

0.1a3 - 2011-06-06

• Included recipe to generate a working ejabberd.cfg using collective.recipe.template. [ggozad]

• Added some basic tests for pubsub. [ggozad]

• Run tests requiring the EJABBERD layer only when level 2 is specified on the testrunner. [ggozad]

0.1a2 - 2011-05-11

• Updated documentation on how to add a recent wokkel. [ggozad]

0.1a1 - 2011-05-09

• Initial release [ggozad]