[JDEV] Jabber DevZone News - Jabber Progress Report

[Jabber DevZone]

Jabber Progress Report

The following was posted by stpeter at jabber.org via the Jabber DevZone web site (http://dev.jabber.org/):

The recent explosion of Jabber has put many pressures on
some of the original team. We're so used to being
everywhere at once, but now we're finding it hard to do 
that
because Jabber is just so massive. We all know we have a
common goal and a common interest in building this
technology, but we don't necessarily know exactly where we
fit in anymore. To help figure out where we're striving to
be, we need to step back and take a look at Jabber as a whole
once again, and specifically identify the rough spots in the
project. Then we'll be able to set goals and define our new
roles.


Rough Spots
We've often started to roll out ideas and tools, but
sometimes those efforts stall. Examples include Bugzilla
and QA practices. Both of these would be extremely
beneficial to the project, but they never got fully
implemented or utilized for different reasons. Manpower has
often been a factor with these projects, because no one had
the time to step back and focus on something like this.
We'll explore this more in a bit.



Jabber has been innovative from day one up until the 1.4
server release. That's not to say we haven't done new
things since 1.4 was released, we just haven't been pushing
the edge of Jabber as hard as we used to. One of the
strongest reasons for this has seemed to be lack of goals.
We have no clearly outlined plan of where we want to be, or
how we want to get there. Or do we? I believe that each of
us has a personal goal list that we work towards, but we
often get thrown off track by having to help the project as
a whole, or work towards someone else's goal. These all
slowly meshed into a somewhat common goal for Jabber, but
in some ways it became too vague to work hard at it. It's
probably time to refine those goals more solidly and
definitively for Jabber as a whole.



Server development has always been a hard issue. The
issue is two-fold, joining and participating. Joining the
server development project was easy because it was just a
necessity for the project, but we haven't been successful
at getting new contributors to server developmenti (not
that we've been actively recruiting, either). Participation
once joining is a key issue, because it has struck a number
of the core contributors personally many times. Most of the
server has been coded by jer, and very much reflects his
style. Sometimes people have felt like helping, yet turn
away because jer is intermingling more pieces, or because
things are in his head and not in a shareable state. This
environment often left people feeling like they were not
contributing. Another grey area was often finding a large
check-in of a whole new concept or idea that had been
largely undiscussed and again implemented by one person.
This isn't necessarily bad as we started up and needed to
get the ideas together fast, but it was by far not the
fastest process, because it was a very select few that
hacked on the server. The answer to many of these woes
again involves manpower, structure, and opening up server
development more through Jabelin.



I'm a strong believer when it comes to organizational
structures for a group, and Jabber has long lacked a
structure. Does someone lead a server dev team, QA,
protocol dev? All of those? None of those? Basically no,
not in its current form. The structure has been so lax that
outside people just look for one of the common names to ask
a question, and they often get bounced around between
people until they magically land with the right person.
This discourages both the Jabber team and the outside
individuals.



Now that we've identified the rough spots, we can begin
to explore the growth, identify our goals and find our
roles.



Understanding the Growth

Our new growth has brought about two large expansions to
the Jabber world: The Jabber Foundation and Jabelin. Both
bring their own ability to help strengthen Jabber, but not
without some work and planning.



The Foundation

The Foundation will bring a solid structure to the
Jabber protocol and the Jabber community as a whole. The
more formalized process of protocol additions is greatly
needed, and will immediately allow for greater
participation by third parties. The mostly open process
of starting the Foundation has been very beneficial to
the Jabber community as well, and will help generate a
solid group of individuals as members.



For specifics, visit http://foundation.jabber.org.



Jabelin

Jabelin will be the server development team. They're
the ones that will take the specs from the Foundation and
make a reference implementation for the world. They will
also give server and component developers a place to work
with the server, communicate, and learn about the actual
internals. This way the server development community is
partially isolated from the protocol development
community, allowing for less confusion to people new to
Jabber and looking for a place to join. Finally, Jabelin
will maintain a repository of installable components and
modules so they are organized and easily obtainable.



Initially Jabelin will work on maintenance of the 1.4
server release. This includes patching some major bugs
that have been identified, integrating patches from
people, and hunting down some small leaks. During this
time a branch will probably be split from the server to
begin work on some more larger projects that have been
suggested. Some of these projects are Keith "tsbandit"
Minkler's xmlnode rewrite, hashed/centralized jid's, and
new xdb functionality. There are also some experimental
branches that are being tossed around, such as completely
removing pth for an event-driven model, and remolding the
Jabber Session Manager (JSM). To be able to work on all
these projects with many differnet people, there will
have to be some organizational structure to Jabelin.



Jabelin will have people to guide specific areas such
as bug tracking and code architecture, as well as people
to help schedule and coordinate the efforts. The
structure will not be overly rigid, but will allow for a
very coordinated and organized effort. This will make it
easy for people to find who can immediately get them
started with where they want to help, and keep current
members focussed on the current goals.



Jabber Goals

After discussing the rough areas of Jabber and where we
want to go in the future, some of the core team members
were able to identify what we think are some worthy goals
for Jabber (it's not a complete list by any means).




Six-Month Goals



Document and standardize the "trusted services".
These are the internal protocols used by the server
and components such as xdb, log, and route.





Document and standardize the new conferencing
protocol. This is currently under development, being
led by David "mass" Waite. Once there is a spec, it
needs to be put through the Foundation and officially
endorsed and implemented.





Get 15 people involved in Jabelin development. We
don't have to reach this number and stop, but this is
the minimum amount we wish to find and get active.
When we say involved we also mean people that are
helping code, and active in development discussions.
More and more people have been expressing an interest
and sending in patches, so we should be able to
recruit some good contributors.





Create a project index for clients. There is an
index page http://www.jabber.org/?oid=70 for
projects right now, but it's poorly formatted and
probably needs more features.





Establish client development/interface
guidelines.





Create an index of server components, including a
tool that will enable people to quickly and easily
install new pieces.





Establish component development/interface
guidelines.





Prepackage the server. The server needs to have
predefined packages for many different interfaces and
setups. One example would be one that has all the
daemontools scripts installed in the correct places
by the installer. This has potential integration
points with the component index.





Establish a build test team -- a group that
specifically works on getting out specialized
releases (e.g., *BSD, Solaris, Debian, Red Hat), and
tests builds of CVS betas.





Blue-Sky Goals

Currently there is only one major blue sky goal. One
of the long outstanding goals of the project has been to
have many server implementations. This is one of the
reasons for the open protocol. These server development
projects need to be seeded for other languages such as
Java and Perl (perhaps also Python and C++).



Specific Growth Areas

This is just a simple list of some specific problems or
other grey areas that have been discussed recently.





Transports - Although they've been worked on so
hard, they've only gone so far. Because of the
scattered attention of the people who work on the
transports, and the fact that each one is developed by
one person, this has been very hard.





Server Uptime - Server downtime for jabber.org has
been a major problem recently. Yes they are denial of
service attacks, but jabber.org should be handling
these fine. The first step would be to setup a
firewall/route box to defend and control the jabber.org
domain. Next, there need to be separate boxes for
production, development, and email/web, since this
would drastically cut downtime of the primary services.
The boxes actually serving the apps should be
configured in a similar manner, so it would be
relatively easy to move apps between boxes if
necessary. In addition, for an app to go live on the
production server it needs to have passesd some sort of
QA and load testing by the team. Only after that can it
be moved from the QA box to the Production box.





The Jabber Network - The reality of a Jabber Network
isn't here yet. Realistically we really have only two
public servers: jabber.com and jabber.org. We need to
push the idea out more. One of the best ways to do this
is by having predefined packages that are easy to
install and setup. This would make it simple for ISPs
and other providers to begin offerring Jabber
services.





Distributed JUD - Many of the components (such as
JUD) need to better support the true distributed nature
of Jabber.





Documentation - It's growing but it still has
problems. Mostly all the distractions of other projects
or more pressing needs cause docs to be left behind.
Along these lines there is a lot that can be exposed
through docs such as client guidelines (next
topic).





Clients - Client development is easy at first, but
then can become cumbersome and slow down. Client
guidelines -- e.g., a list of features suggested for
basic, intermediate and advanced clients -- would
probably help client authors have goals they could
achieve.





File Transfer - Many current clients do not fully
expose file transfer capabilities because they are very
roughly defined within the Jabber framework. These need
to be more solidly defined and then have
implementations pushed.





Onward and Upward

This is an exciting and growing period for Jabber. The
original core team is ready and willing to accept where
we're going and try to get more people involved. Currently
many of us have picked some roles to fill, and others are
thinking about how they can contribute going forward. For
example, Jeremie "jer" Miller is continuing to be the
visionary and overall guide to Jabber (mainly helping to do
so from the Foundation board), Dave "dizzyd" Smith and
myself will be leading the Jabelin group, and Peter
"stpeter" Saint-Andre is going to continue as our patron
saint and evangelist working on docs and such. Yet we need
to continue to grow and expand beyond the core team. One
initiative that will help us expand is the formation of
Jabber Interest Groups (JIGs) within the Jabber Foundation.
These groups will enable people with similar interests to
work together and most likely define proposals for
consideration by the Foundation. We're still working out
many of the details and we want everyone to have a place
in Jabber, so if you have any ideas feel free to join the discussion
on the JDEV list or dev.jabber.org!



http://jabber.org/?oid=1406