Psi Team Blog

Remko Tronçon: Migrating from Openfire to Prosody

Remko Tronçon — Fri, 03 Jul 2009 07:07:19 +0000

Because Openfire has been hogging too much of my limited el-tramo.be server resources lately, and because I don’t need a beast of an XMPP server for only 2 users, I decided to replace it by the lightweight Prosody. The migration went flawless, with the help of two tools: Sleek Migrate, and a Prosody XEP-0227 Importer.

First of all, I used Sleek Migrate to retrieve the roster (and other) data from the server, and store it in the standard XEP-0227 format. I extended the tool a bit such that it supports Openfire’s User Import/Export format, a format generated by an Openfire plugin that is distributed with the server software by default. Using this format as input for Sleek Migrate avoids the need to create a user file manually. The changes I made to Sleek Migrate are currently available from my Git repository, awaiting to be pushed to the main repository.

I then wrote a short script that populates the Prosody data dir with the server data from the XEP-0227 XML file. Currently, the script only generates roster and account data, but adding vCard and Private XML Storage (used amongst others to store MUC bookmarks) should not be very hard. Until Prosody creates a native XEP-0227 importer, you can get the script from my Git repository.

Justin Karneges: Psi 0.13-rc2 released

justin — Tue, 30 Jun 2009 07:12:34 +0000

Psi 0.13-rc2 is here! Notably, voice calling is now a standard feature of the Windows version. Please download and test.

Final on July 13th if no showstoppers are found.

New in 0.13
- Voice calls (Jingle RTP).
- Basic XMPP URI handler.
- Ability to permanantly trust certificates at connect time.
- Mini command system (Ctrl+7 in chat window).
- Various bugfixes.

If you are building from source (e.g. on Linux) and want voice calls, you will need to obtain the PsiMedia plugin separately. On Linux, the plugin file is called libgstprovider.so, and must be put in Psi’s $LIBDIR/psi/plugins directory. You’ll know it worked if “About GStreamer” appears in the Help menu.

Justin Karneges: Psi 0.13-rc1 released

justin — Sat, 23 May 2009 20:19:48 +0000

Psi 0.13-rc1 is here! Please download and test.

Final on June 6th if no showstoppers are found.

New in 0.13
- Voice calls for Mac OS X and Linux (Jingle RTP).
- Basic XMPP URI handler.
- Ability to permanantly trust certificates at connect time.
- Mini command system (Ctrl+7 in chat window).
- Various bugfixes.

Remko Tronçon: “Beautiful Testing” XMPP Chapter

Remko Tronçon — Sun, 03 May 2009 11:57:21 +0000

Adam Goucher and Tim Riley (Director of QA at Mozilla) announced a few months ago that they are putting together a Beautiful Testing book for O’Reilly. I took the opportunity to write a chapter about testing in the context of XMPP (more specifically, about testing protocol implementations in Swift), and just submitted the final draft for technical review. The book is expected to be released this August.

Although there are many types of testing being done in the XMPP world, the chapter focuses on the beauty of testing the functionality of XMPP protocol implementations. After a brief introduction on XMPP, it starts out with a description of unit testing simple IQ request/response protocols, and then gradually moves on to higher-level testing of more complex, multi-stage protocols such as session initialization. As you might expect from a developer like me, the chapter is quite heavy on the (C++) code, but I’m told it compensates for the rest of the book

As with all other books in the O’Reilly “Beautiful” series (which started with Beautiful Code, but has since been followed up by Beautiful Architecture, Beautiful Teams, Beautiful Security, and Beautiful Data), all proceeds of the book go to charity, in this case “Nothing But Nets” (which provides mosquito netting to malaria infested areas of Africa). This means that I can plug this book as much as I want, and still have the feeling I’m actually doing a noble, unselfish thing. (contrary to when I casually mention that you can buy our book XMPP: The Definitive Guide at very sharp prices these days). Some time after the book’s release this summer, I will even make a free version of the chapter available here, so check back soon!

Kevin Smith: Psi under new^h^h^h^h old management

Kev — Thu, 05 Mar 2009 22:16:54 +0000

A copy of an email I just sent to the Psi development list:

Hi all,
After a few days, I’ve given up doing a long and explanatory post.
After four and a half great years of Psi, I know that I don’t have the
energy for running the project that I had back in 2004 when Justin
handed over to me. Fortunately, Justin is recharged, active again, and
available to continue his calling as project leader, so I’m handing
project leadership back to him. I won’t be going anywhere, so please
feel free to contact me about Psi things, and maybe I’ll even have a
chance to write some code again, I’ll certainly still be around the
XMPP world doing some new and interesting things

Thanks to everyone for their support over the last 4 1/2 years in
allowing me such a rewarding role.

I hand you now to the new project leader of Psi … Justin Karneges.
Good luck to Justin

Remko Tronçon: XMPP 101 @ FOSDEM

Remko Tronçon — Wed, 04 Mar 2009 19:00:45 +0000

The slides of the “XMPP 101” talk that Peter and I gave at FOSDEM are available below. This presentation gives a fast-paced introduction to XMPP, and is mostly based on “XMPP: The Definitive Guide”. If all goes well, we will be giving a more extended version of this talk as a tutorial at OSCON.

PDF

Remko Tronçon: Swift Messaging

Remko Tronçon — Tue, 03 Mar 2009 18:31:25 +0000

I’m excited to announce a new player in the Jabber/XMPP game: Swift. Shortly after finishing the XMPP book, I started working on Swift, a pragmatic, cross-platform, user-friendly IM client. Together with Kevin Smith, we are building this project from the ground up, driving its development using agile methodologies. Underneath the IM client, we are working on an extensible and robust XMPP library, written in C++.

Until we launch the project and its website, you can subscribe to the Swift blog and identi.ca group to stay up to date with the latest news and developments around the project. Thanks to Dave Cridland for lending us his graphical capabilities and drawing us a pretty logo.

Remko Tronçon: We have an animal

Remko Tronçon — Sun, 08 Feb 2009 12:25:02 +0000

O’Reilly just sent us the cover for our upcoming XMPP Book, and it seems we got the world’s smallest ungulate: the lesser mouse-deer. I haven’t seen one in real life before, am not sure I ever want to, but still: great! Have a look below to see what the cover of the book will look like when it hits the stores in 2 months.

Remko Tronçon: Final revision of the XMPP book submitted

Remko Tronçon — Mon, 02 Feb 2009 23:54:21 +0000

After a few weeks of heavy labour and long nights, Peter, Kevin, and I just submitted the final revision of “XMPP: The Definitive Guide” to the folks at O’Reilly. All the feedback from our (thorough) reviewers has been processed, we added quite a few extra bits and clarifications (58 pages to be exact), polished the whole thing up, and went through the resulting manuscript with a fine toothed comb. We hope the people who will read this book will be as satisfied with the end result as we are. If all goes according to plan, the book should roll out of the presses in about 2 months. In the mean time, you can expect an update to the on-line rough cut version of the book in the next couple of days.

Kevin Smith: Why I /do/ document and unit test

Kev — Sat, 17 Jan 2009 18:17:19 +0000

I’ve just read Justin’s Why I don’t document or unit test post that explains the way he works, so here’s a counterpoint from the way I work. I’ve inherited not documenting and not unit testing from Justin for the Psi project, where it’s fairly hard to change, and at my day job I unit test and document quite a lot, so I’ve worked with both approaches - this post mimicks Justin’s, as I would have written it - it will only make sense immediately after reading his article.

Why I document and unit test

I write documentation and create unit tests, because they’re great things. Software projects are, almost universally, written to address a perceived problem, and fail if they can’t address this need, so they need to be tested. Similarly, very little software is write-only, and if someone needs to read it, it needs to be understandable; it needs documentation. Not all projects satisfy these criteria, but those that do are successful and fun to work with. Maybe that’s because you have some degree of certainty that your code does what you want it to, maybe it’s because you can stand straight and say “I did what I thought was best” and not feel guilty, maybe it’s because you can show your code to people and not have to make excuses, or explain what your method ‘doSomethingWithSideEffects()’ does, maybe it’s because your unit tests caught a critical bug that you just know would have taken you days to find in 6 months without them, maybe it’s just a matter of pride in your craft as a developer. Tests are particularly gratifying - when they pass you can move on, and when they fail you save yourself a lot of heartache (and headache) in the future.

Or maybe it’s because documentation and tests add such huge value to everything else that you’re doing. I’ve worked (and continue to) in both professional and open-source environments where code is unit tested and sufficiently documented - I suspect that it takes a brave person to notice that an undertested, underdocumented project is a well of unproductivity and to invest the manpower to turn it around, usually in the face of considerable pressure for more, more, more, now, now, now. On the other hand, I can’t imagine any type of person who’d deliberately go the other way. I think the biggest problem for a lot of people is legacy - it’s hard to update your practices when you’re knee deep in code that’s certifiably untestable, but I never hear those that do as much as they are able to do complaining about the results.

Deferring documentation and testing is a misguided attempt to get the most bang for your development buck this morning by sacrificing your productivity this afternoon. By the time this afternoon comes, you find yourself wondering why you don’t have the time anymore to be productive. It’s easy to be lazy about these things if you don’t believe in them, but if you don’t believe your code is worth testing are you saying that you don’t believe the code is worth writing? If it’s worth writing something, surely it’s worth writing the right thing. Do you find yourself saying that you don’t want to add features now because you’re going to rewrite it later? How bad must the code quality have been for you not to trust yourself to add a new feature without breaking something? Yes, there is an overhead writing code that can be unit tested - that overhead is the overhead of writing code that works. If you find yourself resisting testing, is it because you know that your code is broken, but you can’t bear to find out how badly? Would unit testing your code show that you wasted a lot of time that could have been saved if you’d written the tests before? If you do fall into this trap, then you’re going to have a hard time getting out of it, because you’ll be wondering how much time you’ve wasted. Ostrichs aren’t the only creature that (mythically) can bury its head in the sand.

I document my code, because I like my design

Do you find yourself writing a big paragraph of Doxygen, or Javadoc comment for your method? Why? Shouldn’t your method name tell you what it does? Shouldn’t your method parameters’ names, plus their type (if such is your language) tell you how to call it? If not, why not? If your method is called “addTwoNumbers(int num1, int num2)”, you shouldn’t be documenting to say that, by the way, it also resets some internal mutable state; if you need to put that comment in, the smart money says that your code’s broken. Why bother spending the time with sensible naming and self-documenting code when you don’t even know that you need this method? Back up a minute - why are you writing this method if you don’t know that you need it yet? Yes, requirements for a project change (and we embrace those changes, as Agile developers), and if code goes out of (project) scope, we’re glad that our methods were small, single-responsibility methods so we don’t spend time splitting out the functionality we don’t need from that we do. We’re also glad that we wrote no more code for that requirement than we were sure that we needed.

It happens in software that you have to refactor stuff. That’s just the way it goes. Requirements are never right first time, but at least we know that, and we keep checking them with the users of the system, and the users of our methods, as appropriate. When you know that your requirements are going to change, you have no business writing code that’s going to be hard to refactor, or code that doesn’t need to be there. You had your requirements from your user, and turned those into the minimal API requirements for your code, wrote your unit tests to assert that the minimal requirements are met, and wrote the minimal code to satisfy the tests, and therefore the requirements. You really don’t want to be committing to such a heavy design up front that it’s going to be a hassle to update it.
As the saying could go: code that needs documentation is worse than wrong documentation, is worse than no documentation.

In general, if the code is such that other developers are able to tell you what your code does from the class, method and variable names, your energies are being well spent.

I test, because I care about my implementation

And I always care about my implementation - you do too, or you wouldn’t be so far down this gargantuan post. There’re only three reasons that immediately come to my mind why you might not care about your implementation:

You hate the task that your code is written for. This is going to be a problem if you’re captured by an evil genius, ideally with a cat, and are working on an advanced intelligence, or orbital laser platform, to be used to enslave all mankind, but for most of us this doesn’t apply.
You simply don’t care about the result. You’re demotivated, and if it doesn’t work then it’s someone else’s problem to solve, the testers will pick up the problems and you’ll be on holiday when the bug reports come in. This isn’t you.
You do care about your implementations, but you don’t know how to do a job your proud of, so you tell yourself that you don’t care about them as a self-preservation mechanism. This may be you, it’s been most of us. Thankfully there are smart people who’ve written books giving advice on how to be a better developer. In fact, titles like Agile Software Development, Clean Code, Extreme Programming Explained, and The Pragmatic Programmer make this post redundant - read them and save yourself reading the rest of my post. Books may not make you smarter, but they’re not going to make you a worse developer.

It’s possible to get tied up with design, and delegate implementation to being a second class entity. Think about this for a moment. What’s design, and what’s implementation? When class A calls a few methods on class B, what is that code - design or implementation? It may be a design for class B, but it’s the implementation of class A. Don’t get too hung up on which is which - generally the design is driven by the needs of the implementation. You know what methods class B needs once you’ve written class A’s method that makes the calls. Divorcing design and implementation is a great way to make sure your code isn’t going to do quite what it was meant to. I’ve done this, you may have done this. Live and learn. This is just another case of ‘code what you need’. If you only code what you need, you only test what you need (equally: if you only test what you need then you only code what you need).

Some people will start testing and find that they can’t test it all. If this is you, and you’re finding that your methods have side-effects that you can’t test, then you’re one of the lucky ones: congratulations! Your unit testing has found problems with your code even before you’ve written your first line of a test. There is a definite desire for mutable objects (objects whose state can be changed through method calls) - certainly, Java code would look very different if ArrayLists were immutable, but this doesn’t mean that your objects should be mutable. In the rare cases where your object /does/ need to be mutable, make it explicit: don’t have a method that does one thing superficially, but also changes state. Oh, and don’t make classes such that your objects can get into an inconsistent state, either - a common example of this from my past is constructing an object, but requiring another ’set’ method call before the object is usable. I kick myself when I wonder why I didn’t pass the value in the constructor. If you can’t unit test your code, you probably need to reconsider your approach. The obvious counter to this is a dependency on external state (particularly databases, or networks) - unit testers get around this with mock objects that look like the external state object, but are deterministically controllable from the tests, and you can easily fall in love with them very quickly.

In general, code that works will continue working if you don’t touch it. Psi’s a great case in point - if it didn’t consist of a majority of working code, then nobody would use it. It hasn’t traditionally had any unit tests, and that’s taken its toll on development - development cycles are long because no-one dares to refactor the code to support new features, and once the features are in, release candidate cycles are typically even longer because it’s difficult (and demoralising) to hunt down the bugs that have been introduced, especially when the bugs often cause crashes in unrelated code, due to mutable objects. Until we build up adequate tests, the cycle will continue, as people spend their energies on new features, avoiding fixing bugs that’re difficult to reproduce, or are in code that is perceived to be risky to modify. That doesn’t mean that Psi’s buggy, or that it will become buggy - we don’t leave the Release Candidate phase until we have reasonable confidence that we’ve got all those bugs out, and we do a pretty good job of it (I hope). It’d just be nice to not need to introduce them, or spend so much time in RC getting rid of them.

There are no exceptions (almost)

It’s easy in the early stages of a project to neglect testing, or to write code whose behaviour isn’t obvious, but this is the greatest of false economies - when your early code doesn’t work, your later code isn’t going to work. It’s also easy to slip into the mindset that unless you can get complete coverage now you shouldn’t test at all. If, for some reason, you can’t get as much coverage as you want, every behaviour that you test is another bug that can’t happen.

About Documentation

Above, I claim that all code should be self-documenting. I stand by that - it should be obvious what code does from its method name and signature. That doesn’t mean that I believe code should never have a Doxygen comment, or a Javadoc - some people (that I respect) hold this opinion, and I may too, some day. At the moment, though, I can still see a value in a greater explanation in a comment, especially if there’s a business requirement that makes something non-obvious - you know the sort of thing “oh, yeah, all numbers must always be rounded up, 2.1 is really 3″. Why you have to do something, where it’s not obvious, is a great thing to document. If you have to document what you’re doing then you probably need to consider why it’s not obvious from your code.

Conclusion

Deferring documentation or unit tests is not always about laziness. It’s about efficiency. Note my usage of the word “defer” - someone else will need to understand your code eventually, and you’re going to need to do the testing, either manually or automatically, to make it work someday. You’re sacrificing your efficiency later for a tiny gain now. It’s clear to me that it’s necessary to prioritise the tasks with highest impact first, with your tests to ensure that they work, and to only put effort into the next features once all the problems with the first code are solved, and the probability of later needing to expend considerable effort fixing them, or being unable to modify them for fear of breaking them, is low.

Happy coding!

Here ends my version - thanks to Justin for providing the original as a talking piece, which gave me a chance to get some thoughts down on electrons that previously I hadn’t taken the time to express.

Justin Karneges: Psi in 2009

justin — Thu, 08 Jan 2009 20:19:02 +0000

So Psi moves along sluggishly as usual. 0.13 was not released by Christmas. There were reasonable explanations for that. Despite this, I think 2009 will be a good year for Psi, and possibly its best year yet.

I suppose I can’t put words in anyone else’s mouth, but I have high hopes we’ll see Kev and Remko return to regular development after their XMPP book, and that Misha will finally be contributing Yandex stuff back. As for myself, these past few months stand as a record for the longest time Barracuda let me focus on Psi issues (which, being all Jingle stuff targetting Psi 0.14 or later, is easy to miss).

So, post-book and post-0.13, I predict a sudden surge of visible development, and more attention to submitted patches.

Despite the perception that Psi lags “behind” other client projects, keep in mind we have a number of significant features that have already been started (with major progress) and just need to be finished:

Jingle audio and video
Whiteboarding
WebKit chat windows
Revamped message history interface
Revamped roster
Plugins
Bonjour chat
Automatic updates
Certificate/SmartCard authentication
SSO
XMPP URIs

Feel free to add more to this list. I often forget what’s on the back burner.

Do note that the above is not a list of priority features, just a list of features that are partially complete for one reason or another. The priorities going forward are a topic of a different message. I just wanted to show that when you count all of this unfinished stuff, Psi is farther along than it appears to be. The project has been around for so many years, and yet there is still so much to do.

Here’s to Psi in 2009! I look forward to a great year with all of you.

Kevin Smith: XMPP: The Definitive Guide - Rough Cuts edition

Kev — Fri, 19 Dec 2008 07:51:12 +0000

It’s been an exciting time recently for the “XMPP: The Definitive Guide” book that Peter, Remko and I have been working on. Just over a week ago, Remko announced the availability of the book in the Rough Cuts scheme, where you can buy a PDF of the unfinished manuscript to read now, and receive the full version when it’s ready. Then, earlier this week, Peter posted about our submission of the full manuscript to O’Reilly, ready for technical review. Now it’s my turn with the good news: O’Reilly have updated the Rough Cut of the book with our submission, meaning that there is a complete, current book on the XMPP technologies available.

There will still be changes to the manuscript as a result of reviewers’ comments, but we think this is now a text we’re happy to have available. O’Reilly have feedback mechanisms for Rough Cuts, so if you read the book and anything is unclear (or, let us hope not, wrong) please help us make this book something that can guide current and future developers in their work with XMPP.

XMPP: The Definitive Guide

Remko Tronçon: Rough cuts of XMPP book now available

Remko Tronçon — Wed, 10 Dec 2008 16:25:10 +0000

While Kevin, Peter, and I are working very hard to finish the first draft of our upcoming book ‘XMPP: The Definitive Guide’, O’Reilly has recently released early versions of most of the chapters of the book as Rough Cuts. People interested in learning about XMPP today can now get a preliminary version of the book, and get updates as the book progresses.

Hal Rottenberg: Psi Keeps Rollin’

halr9000 — Thu, 04 Dec 2008 17:00:00 +0000

We’ve averaged 1,500 downloads a day of Psi for years. It truly amazes me. We’re almost at 2.4 MILLION downloads, can you believe it?

I whipped up this chart from Sourceforge data:

Hal Rottenberg: PowerShell Script: Decode Psi Passwords

halr9000 — Mon, 01 Dec 2008 17:38:55 +0000

This came out of a question in the Psi chat room this morning. A user wanted to retrieve a password which was saved in the Psi config file. We don’t use any strong crypto here on the basis that if someone has access to your user profile, you have bigger problems than your IM accounts. There was a Perl script around which did the trick. I couldn’t wrap my head around that, so pulled in a buddy (Joel Bennett) who did the porting to PowerShell. Here it is: