Final Report For My GSoC Project

I hope this report is not too late.

In the last 3 months, I implemented XEP-0136 for Prosody. This experience is really awesome: great mentor(Matthew Wild), active communities(Prosody, Gajim) and the wonderful organization(XSF), they've helped me a lot on this project. Many thanks to them! And, thanks to Google, who makes all of these possible, :)

What have been done

XEP-0136 is almost completely implemented:

• Archiving Preferences
• Automatic Archiving
• Manual Archiving
• Archive Management
• Replication

Not perfect, but workable.

The 3rd part of XEP-0136, Off The Record, is too complicated, and not practical I think. So MattJ and I decided not to implement this feature, and implement another "XEP" instead -- XEP-xxxx: Message Archive Management, which is a much simpler version of XEP-0136.

What to be done

I'll continue maintaining mod_archive and mod_archive_muc. There are still some little issues, and I'll fix them in the future.


All the code is available at prosody-modules. Any suggestions are welcome. :) Please report issues here, or contact me at shinysky1986 (A T) gmail dot com.

Week #11: mod_archive_muc

This week I implemented a new module: mod_archive_muc, a simple version of mod_archive, which is the implementation of XEP-0136.

In mod_archive_muc, we simply archive every message regardless of the collection. And retrieve them later with any combination of three filtering attributes - 'with', 'start' and 'end'. This is much simpler than mod_archive.

The Next Step:
* Go on finalizing the code.
* Do testings.
* Prepare for the final evaluation.

Week #10: Efficiency Improvement

Last week, I mainly focused on efficiency improvement, testing and bug fixing. Up to now, most of them are finished.

Improvements and Bug-fixing:
* Keep collections stored in REVERSE chronological order.
* Make sure result set are in chronological order(as specified in XEP-0136).
* Use binary search to improve collection inserting speed.
* Use util.jid.compare to do JID matching.
* Fix a bug about time stamp.
* Some other minor fixes.

The Next Step:
* Go on finalizing the code.
* Do some testings.

Week #9: Preferences Interpretation

This week I mainly finished these features:
* Interpreting preferences when performing automatic archiving, to determine whether archive the given message.
* Mapping an message to some collection based on timestamp, to determine which collection the give message should be archived into.

The next step:
* Keeping archived collections in chronological order.
* Making sure result sets returned to clients are sorted in chronological order.

Week #8: Replication

This week I implemented 'Replication'.
So the most part of this project has been finished. Yeah~

But, there are still a lot of TODO-es remain.
So in the next several weeks, I'll fix these problems.

Next step:
* Parsing preferences to determing whether archive the incoming messages or not.
* Mapping an message to some collection based on timestamp(?)

Week #7: Archive Management - Part II

This week I finished the implementation of "Archive Management", including:

1. Retrieving a list of collections
2. Retrieving a collection
3. Removing a collection

Actually, I've finished it on July 7. After that, I was always thinking about how to implement "Replication". This part(Replication) seems like implementing a "version control system"(Simpler, of course). I still can't figure out an efficient way to do this, so I decide to make it "workable" first.

Here is my thoughts:

1. Add a "access" attribute to chat element, representing the time this collection is last accessed(created/modified/removed).
2. modified_handler check this "access" attribute to determine if include this collection in the result set or not.
3. If a collection has no any child, that means it has been deleted.

The next step:

1. Mid-term Evaluation, :)
2. Replication

Week #6: Archive Management - Part I

This week and the next week(before the mid-term evaluation), I planned to implement these features:
* Retrieving a List of Collections
* Retrieving a Collection
* Removing a Collection

Actually, these features involve implementing XEP-0059: Result Set Management. So this part would take more time.

After this, I'll implement Replication.

The things I don't like about XEP-0136

Update 7/12, 2010
* "version" attribute of a collection. It seems isn't used anywhere.
-------------------------

Update 7/7, 2010
* note Element of a collection. What is this element for? I only see how to set, but don't see how to retrieve. Is it useful for users?
-------------------------

MattJ want me to give a list of the things I don't like about XEP-0136, now here is the list:

* 'secs' attribute of from and to Elements. It's inefficient and hard to figure out this attribute when archiving a new message. Using a timestamp(like the 'utc' attribute) would be easier.
* It's NOT easy to find out duplicated messages.
* Linking Collections. I think it's not very practical. :/
* Collections. Without the 'thread' element, it's hard to determine which collection a new message belong to.
* Each collection of messages and notes is uniquely identified by the combination of the 'start' and 'with' attributes, but we're using 'thread' element to map messages and conversations to collections.

That's all for now.

It's said that XEP-0136 will have some improvements. Happy to see that, :)

Week #5: Manual Archiving - part I

In this week, these features have been implemented(All about Manual Archiving):
* Uploading Messages to a Collection
* Offline Messages
* Groupchat Messages
* Changing the Subject of a Collection
* Linking Collections
* Associating Attributes with a Collection

Next step:
Make it more robust.

Week #4: Automatic Archiving - part II

This week, I mainly did these things:
* Reading through XEP-0136: Message Archiving again, to determine how to store the messages.
* Writing code to do automatic archiving.

But there are still some issues remain:
* If a thread ID is not included in the message stanza, mod_archive should use some methods for mapping messages and conversations to collections.
* Dealling with the time attributes.

Next step:
* Solve the problems above
* Manual Archiving

Week #3: Automatic Archiving - part I

In week #3, I was always doing something on "Automatic Archiving".

At first, I tried to add message hooks for mod_archive, then I found it didn't work.
Finally I figured out that it's because the priority of my handler was not set correctly.

After that, I encountered another problem, the messages were passed to my handler, but just stoped there and were not passed to the other handlers. waqas told me it's because it returned true at the end of my handler.

Finally I got my handler to run.

I think I've underestimated the workload of "Automatic Archiving" that to be done. I plan to spend more time, maybe one or two weeks, on this part, because it's the fundamental of other features about archiving.

Week #2: Preference Handling

In the last week, these things have been done:
* Setting Default Modes
* Setting Modes for a Contact
* Setting Modes for a Chat Session
* Setting Archiving Method Preferences
* Client Enables/Disables Automatic Archiving

So, the preference handling is mostly done, far ahead of the schedule.

The next step:
* Automatic Archiving. Grab messages and store them.

Week #1: Determining Server Support & Determining Preferences

A week has passed since the coding phase began.

In the last week, these things have been done:
* Committing code to project prosody-modules for the first time.
* Adding a new wiki page for my module(mod_archive).
* Communicating with the Gajim team. I've gotten a working copy of the message archiving branch of Gajim, so it'll be more convenient to test and debug my module. Thanks to johnny and Asterix.
* Discussing with MattJ about some implementation details, and getting Prosody 0.7 to run. Thanks to MattJ and waqas.
* Working on the module. I've implemented these two features: Determining Server Support & Determining Preferences as planned.


The plan for the next week:
* Setting Default Modes
* Setting Modes for a Contact
Maybe more, hopefully. :-)

Hello, world!

I will post my progress here.