Java To SharePoint Interface – Follow-up

Sorry its been a while since I blogged on this topic (or any topic for that matter).

As you saw in the previous post on this subject, it is possible to interface to SharePoint’s web service interface from a Java client. What I didn’t get into in that post was the extreme grief that you have to go through to parse the XML that comes back from SharePoint. In fact, SharePoint does some stuff with it’s web service implementation that Axis2 didn’t like at all.

The other key issue with this direct interface approach is that SharePoint occasionally puts CAML (SharePoints special XML language) into it’s SOAP responses. Well, Axis2 really doesn’t like that. This became so painful that we decided to look at alternatives.

Some of the alternatives were:

1. Be masochistic and continue with the pain (not preferred option as too time consuming and risky).

2. Look at alternate web service technologies (the option of using JDK 5 based implementations became an option as the client upgraded to J5). A basic prototype of a Java to SharePoint web service interface was tried, but we found this to be almost as painful as the Axis2 implementation.

3. Implement dedicated standards (WS) based custom web service interface in .NET on the SharePoint server, which talks directly to the SharePoint APIs. These custom web services provide dedicated APIs to do only what the client Java application requires (retrieving content from SharePoint).


I’m guessing that by now you have already guessed which option we chose? Correct, #3.

By using WCF to create standards based dedicated web services we could provide an interface that Java clients could talk to easily, we could limit these services to just the capabilities we needed and could handle any changes in the SharePoint APIs without affecting the Java client.

Problem solved and lessons learned.


Unfortunately, we never did get to proceed with this project, but next time I have to interface from SharePoint from a Java client, I’ll know how to do it the most effective way. Tags:

DSL Project (2)

So to get started, create a DSL Designer project in Visual Studio (I’m using VS2008 but it should be almost the same in VS2005 although there have been some improvements). This creates a solution with 3 sub-projects (and why Microsoft keeps insisting on placing project folders under solution folders, I’ll never understand, as it makes it more confusing to re-use projects in other solutions). In the case of DSL designer, I’d highly suggest you leave it in the structure the VS guidance creates as there are heaps of dependencies on the structure and relative locations of things in that structure that is very easily broken and very difficult to correct afterwards (this is speaking from experience, I reverted to guidance structure). In any event it’s pretty much stand-alone and creates a VS add-in so its unlikely you would re-use any of these projects in other solutions.


The 3 projects the guidance creates for you are:

1. DSL Designer – The actual DSL project.

2. DSL Package – Setup

3. DSL Debugging  – VS hive for testing your DSL during development


I’ll follow up on this with details on creating a basic class diagram DSL in a future post.


Domain Specific Language Project

I’ve been very interested in what has been happening in the world of Domain Specific Languages and Software Factories for quite some time now. I’ve been using a common persistent object pattern (engine, collection, class variant) since 2001, albeit moving from C++ ATL to C#. I’ve been using a code generator to generate these objects since early .NET days.

The only problem with this scenario is optimum efficiency. Working this way, one models the domain object for the solution in a UML tool, then re-enters all that information into the custom code generator application which then produces C#, SQL, unit test and GUI code that is manually incorporated into Visual Studio. Much more efficient than hand-coding but still not optimal.

With the advent of Domain Specific Language Tools in Visual Studio 2005 SDK, Microsoft has provided a very good environment to design and implement DSLs and Software Factories. The modeling of the objects is done in Visual Studio (VS) and code is generated directly into the VS environment…a simple build and it’s ready to go.

I started looking at DSL tools in VS 2005. It’s pretty much the same in VS 2008. I won’t go into all the details, but this is one of the best modeling and template transformation tools I’ve come across (with albeit limited experience in this area).

Basically Microsoft provides an excellent DSL creation tool and very powerful solution for template transformations via a flexible and extensible mechanism:

  1. Engine – Controls text template transformation process
  2. Host – Provides the interface between the Engine and the user environment (can create custom hosts).
  3. Directive Processors – Classes which handle directives in templates and allow template processing to be extended to do anything in code (can create custom directive processors).

The general process to follow to create a software factory using DSL tools is:

– Define your DSL (model type and key elements)

– Create the text template(s) needed to generate the code

– Install the DSL package to Visual Studio and start generating code and tests

It sounds straight forward, and when doing a persistent object factory it should be straight forward as Microsoft kindly provides a number of DSL starter templates, one of which, the Class Diagrams template pretty much creates the whole class modeling DSL for you (in fact I’m having to remove DSL elements that I won’t need). In other cases I can imagine this will be difficult for typical business system developers/architects as it works by providing an abstraction that defines an abstraction, so its interesting to see that you use the DSL tools to model your domain modeling tool! One needs to take a step back to get your head around this, then it starts to make a lot of sense.

Next post we’ll get into actually implementing it.




WLW Will Run in Server 2008

This post provides a link to the .msi, which will install on Windows Server 2003. Obviously not a technical issue. Wish somebody at MS would explain why they are doing this as it makes no sense to me.


This post was done in WLW RTM version on Windows Server 2003.

Last post using Windows Live Writer?

Well sadly, Microsoft has decided to disadvantage developers with it’s consumer products…Windows Live Writer no longer runs on Windows Server operating systems.

Why is this an issue I hear you ask….well..I’m a big fan of developing against target servers on your primary development environment (to avoid frequently poor remote debugging experience). This means Windows Server 2008 is my primary operating system on my development laptop.

I can’t really see using the web based interface (the resultant formatting is atrocious, see my mate Rob’s blog, where wrap around causes loss of spaces between words, etc…..not horrific, but not clean the way I like it.


So can anybody suggest a blog writing tool that will work on Windows Server 2003?



Java to SharePoint Integration – Part I

SharePoint (MOSS) or Windows SharePoint Services (WSS) seem to be everywhere now..but what is the story around integrating these products with Java applications?

Well…from my investigations, that depends on a number of things:

  • What version of Java is being used ?
  • What version of MOSS/WSS is being used ?
  • What level of integration is required ?

For the purposes of this integration exercise we will assume Java 1.4 (J2EE 4) and SharePoint 2007/WSS 3.0 are being used (typical scenario when integrating existing Java applications to new SharePoint/WSS services).

While it’s difficult to say where this will ultimately end up, one of the objectives of this exercise is to produce a matrix of the appropriate integration approach to use across the various Java and MOSS/WSS versions.

Key Requirements

The key requirements (made up for this exercise) that must be kept in mind are :

  1. Support for IIS authentication schemes (NTLM V1 for this exercise)
  2. Support for SharePoint/WSS remote access protocols
  3. Support for highest possible level of integration
  4. LAN based integration (no Internet specific transport required)

SharePoint/WSS Service Access Options

Fortunately MOSS/WSS products come with a number of service based access options, depending on the version. With MOSS 2007/WSS 3.0 the following are the main options. Please keep in mind that previous versions exposed less capabilities on these interfaces, so check the specific capabilities you need before choosing please:

  • WS-* SOAP web service interface
  • XML-RPC interface on object API

Java Options

Depending on what version of Java is being used, the integration options vary.

Java 1.4 (J2EE 4):

In version 1.4 of the Java platform, very limited support for WS-* based services exists in the base distribution. To support for WS-* SOAP web services interfaces to MOSS/WSS, additional web service framework may be required. While there are a number of possible java web service frameworks that could be used, I have selected Axis2 as it supports NTLM V1 as per the above requirements. Support for IIS authentication schemes is a key consideration! XML-RPC support is in the base distribution.

Java 1.5 (J2EE 5):

In version 1.5 of the Java platform, full support for WS-* based web services and XML-RPC interfaces is supported out of the box (including NTLM V1 authentication). I may look at an actual implementation using this later in the series.

Java 6:

Not considered in this series although full WS-* and XML-RPC support are in the base distribution and support NTLM V1 authentication.


Java 1.4 To SharePoint 2007 WS

Given the above background information, lets proceed with some coding. I have setup a SharePoint 2007 instance on my local machine to test against, however, the following code should work equally well against a WSS 3.0 installation.

While I have chosen to work with the NetBeans IDE (V5.5.1 – V6 out now) for this series, everything should work in other Java IDEs (and maybe we’ll try some later?). While I am not using a Java Application server in Part I, we will be looking at the Sun application server for J2EE 4 and 5 (and may look at the GlassFish distribution for J2EE 6 if I get that far in this series).

As I am using Axis2 V1.3, you can find an architectural overview of Axis2 here: Axis2 Architecture Overview. We do not need the full Axis2 server side stuff, so follow the instructions here for a simple binary distribution installation Axis2 Installation. The full user guide for Axis2 can be found here Axis2 User Guide. You need to download and install the binary distribution and setup the AXIS_HOME environment variable.

So to get this started, I created a new project in NetBeans. I’ve used a Java Class Library so that I can create a number of classes that encapsulate the web service interface to SharePoint and present a simple method based interface for the Java application (the methods defined will align with the functional capabilities of the interface). As SharePoint exposes a number of services (for functional service areas,, e.g Lists service provides a functions for managing SharePoint lists), we will align each service with a client class. Obviously, you may decide to do this differently, but from experience this pattern provides a simple to use client, hiding all web service complexity from the main Java application.

The next step is to use the Axis2 tools to generate the client proxy class. In order to do this, you need to first grab the WSDL produced by the service you wish to interface to. Simply type the URL of the service followed by ?wsdl into the address bar in your web browser (e.g. and save the resulting XML/WSDL to a file on your local machine. I suggest you create a sub-folder for your project such as resources/META-INF/. For clarity I have saved the WSDL as Lists.asmx.wsdl.

Axis2 provides a number of options for generating client proxy code, depending on which SOAP data binding framework you wish to use:

  1. AXIOM client – Generic client proxy classes (not dedicated databound classes), does not data bind SOAP to java classes. Uses generic AXIOM classes for accessing web services. Must work with raw XML.
  2. Axis Data Binding. Default data binding framework but only really works with Axis2 web services. Defaults to AXIOM classes when used against SharePoint. Won’t work with all SOAP features.
  3. XMLBeans. Fully functional schema compiler, works with most WS compatible web services. More classes and more complex object model than ADB.
  4. JiBX. Full schema support, flexible but more complicated to setup.

We’ll try each of them and see where we end up.

To generate the client proxy classes using ADB, use the following command (run in the directory where the wsdl file is):

%AXIS2_HOME%\bin\WSDL2Java -Uri Lists.asmx.wsdl -p java2sharepoint.axisadb -d adb -s

This uses the WSDL2Java code generator to create the client proxy classes ( in the package java2sharepoint.axisadb, using Axis Data Binding and outputting source files.

Next create the SharePoint interface wrapper class in java2sharepoint package and be sure the ListsStub code is being compiled into the project.

At this point I have decided to shortcut around the wrapper class and have created a Java console application that calls the web service directly. The code ( for this test application is as follows:


package java2sharepoint;

import java2sharepoint.axisadb.ListsStub.GetListCollection;
import java2sharepoint.axisadb.ListsStub.GetListCollectionResponse;
import java.util.ArrayList;
import java.util.List;
import org.apache.axis2.context.MessageContext;
import org.apache.axis2.databinding.ADBException;
import org.apache.axis2.transport.http.HTTPConstants;
import org.apache.axis2.transport.http.HttpTransportProperties;
import org.apache.axis2.util.JavaUtils;
import org.apache.commons.httpclient.HttpClient;
import org.apache.commons.httpclient.HttpConnectionManager;
import org.apache.commons.httpclient.MultiThreadedHttpConnectionManager;
import org.apache.commons.httpclient.auth.AuthPolicy;

public class Main {

    /** Creates a new instance of Main */
    public Main() {

    public static void main(String[] args) {
        try {
            // Setup Authentication parameters
            String username = "Derek.Miller";
            String password = "mypassword";
            String host = "";   // NOTE: Important to ensure fully qualified domain name is used 
            String domain = "syddev01";
            String realm = "syddev01";
            int port = 80;

            // Configure authenticator
            HttpTransportProperties.Authenticator auth = new HttpTransportProperties.Authenticator();

           // Ensure NTLM authentication used
            List authPrefs = new ArrayList(1);

           // Get stub
            ListsStub lists = new ListsStub("");

                    // Set authenticator
                    lists._getServiceClient().getOptions().setProperty(org.apache.axis2.transport.http.HTTPConstants.AUTHENTICATE, auth);
                    // Get request and execute
                    GetListCollection req = new GetListCollection();
                    GetListCollectionResponse res = lists.GetListCollection(req);
                    DisplayResult(req, res);
        catch (Exception ex) {

    protected static void DisplayResult(GetListCollection req, GetListCollectionResponse res) throws ADBException, XMLStreamException {
            System.out.println("Result OK : " + res.localGetListCollectionResultTracker + "\n");
            System.out.println("Result – Local Name:" + res.getGetListCollectionResult().getExtraElement().getLocalName() + "\n");
            // TODO: Parse out XML and look at results



Since that seems to work, the next steps will be to pull out the results and have a look at what comes back. NOTE that although we used ADB, no real databinding is happening and it’s defaulting back to AXIOM classes. We’ll get into this in a future post. Tags:


Hi and welcome. This blog is going to be my personal channel to the world where I identify useful things I find, discuss software architecture topics and occasionally rant about some of the stupid stuff that happens out there. This will be more professional than personal, but you may expect occasional personal issues to come into play.

My name is Derek Miller. I have been a professional software engineer and architect for around 24 years. Wow have things changed in that time frame. I currently work in a Solution Architect role at CGI (a world leading software consultancy) in the Microsoft practice. We do custom solution development, as well as, full system integration projects (sometimes hard to tell them apart lately).

Some of the areas I plan to cover in this blog are:

  1. Custom solution development in .NET (Web, Windows, Mobile – WCF, WF – SOA, Smart Client)
  2. Integration of Java solutions with Microsoft platforms (Java2MS).
  3. Methodology (especially focusing on "Lean" techniques), architecture, patterns.
  4. Software factories
  5. Microsoft platforms and technologies – SharePoint, PerformancePoint, Silverlight, CardSpace, others.


So now that the first "welcome" post is done, please stay tuned to see what we run into this week.


DISCLAIMER: The opinions expressed in this blog are mine and should not be construed to be coming from my employer.